Final Report: Development of Object-Based Simulation Tools for Distributed Modular Ecological ModelingEPA Grant Number: R827960
Title: Development of Object-Based Simulation Tools for Distributed Modular Ecological Modeling
Investigators: Bolte, John P. , Budd, Timothy
Institution: Oregon State University
EPA Project Officer: Carleton, James N
Project Period: October 1, 1999 through September 30, 2002 (Extended to March 31, 2004)
Project Amount: $247,394
RFA: Computing Technology for Ecosystem Modeling (1999) RFA Text | Recipients Lists
Research Category: Environmental Statistics
The primary objective of this research project was to provide a language-neutral, object-oriented simulation framework for ecological modeling in four focused areas: (1) underlying object-based technology for implementing and coordinating collections of simulation objects (modules) in an integrated simulation environment; (2) interobject communication technology supporting both single machine- and network-based communication between components of a modular simulation; (3) spatial and nonspatial data input, collection, analysis, and visualization; and (4) development of visual programming tools for rapid definition and assembly of model modules into complete, fully functional ecological models and visual interpretation of model results.
The goal of this project was to develop a modular simulation system for coupling existing simulations and development of new simulation modules. This involved four primary steps: (1) development of model concepts and design requirements, (2) prototyping, (3) simulation framework implementation, and (4) design specification for a visual modeling tool (VMT). Project goals were met in each of these areas. This project involved collaboration between computer science and engineering researchers funded through the project as well as external collaboration with an international group of ecological and agroecological modelers, through workshop participation, direct contact, and an electronic list-serv established by the project for promoting communication about requirements and design specifications for modular model development. The resulting requirements specification led to a design specification that is guiding the implementation of a language-neutral software system addressing the requirements. The design specification covers a range of areas ranging from core simulation services, data management, intermodule communication, security, and model construction services. This specification was implemented in a simulation package (Modern Communication [ModCom]) and has been used to successfully develop and deploy simulation models that consist of extendable and reuseable modules.
Development of Model Concepts and Design Requirements
The ModCom project began in the spring of 2000. At that time, we initiated regular meetings to discuss model concepts and plan the development of ModCom. One of the first concepts developed was a general view of how its users will view ModCom. This view is shown in Figure 1. ModCom is composed (conceptually) of three levels of components: foundation level, tool level, and model level.
Figure 1. Conceptual Levels in the ModCom Framework
The foundation level contains simulation components that are common to all simulations. This includes elements like simulation clocks, facilities for solving differential equations, interobject communication, and synchronizing execution of various model components. The foundation level is also where users find the general definitions of how models interact. The Tool level contains simulation components that are used by most simulations. This includes elements such as statistical tools, graphing, and database management. Components in the tool level represent features that exist in addition to the basic components of the foundation level. The model level represents domain specific components. This includes components that contain implementation specific calculations. When existing simulation models are integrated into ModCom, they will reside at the Model level.
The basis for the design specification of ModCom was an existing object-based simulation toolkit (behavioral assessment grid [BAG]). We undertook a refactoring process to assess the strengths and weakness of the original design, resulting in a significantly reengineered architecture. The purpose of the refactoring was to familiarize all participants with the existing framework design and to recognize which features of the existing framework should be part of the core ModCom implementation and which should be moved to supporting components. We were able to recognize what features need to be added or enhanced to support ModCom. The result of these discussions was the development the requirements specifications described below.
Several prototypes where constructed in collaboration with an international group of ecological modelers. The purpose of these prototypes was to test the efficacy of our initial design strategies. We found out early in the process that one of our design methodologies was problematic. Originally, we had intended to construct wrapper classes (in component object model [COM], Microsoft’s language-neutral object protocol) that would contain components from the existing framework. This strategy would save time and still permit reimplementation of native COM/C++ if performance limitations required it. However, COM places certain limitations on how information can be exchanged between components. These limits combined with the way the existing components exchanged information meant that wrappers could not be used.
The prototypes that followed examined different styles of defining how the components exchanged information. From these, we concluded that the best approach would be to reimplement in COM the core components of the existing BAG framework. Doing this allows us to move directly to a fast native COM implementation. The prototypes also showed that native COM implementation do not produce significant performance reduction when compared to non-COM implementations.
During the late fall and winter, we implemented (in C++ using COM interfaces) the core components of the framework. These classes and their relationships are shown in Figure 2. The framework design is based largely on the existing BAG framework and enhancements developed in this project. Although the limitations imposed by COM did require changes in how the components exchange information, the limitations did no require any significant changes to the overall object-oriented design of the framework. Several new features where added to the framework. They include more advanced event processing, listener interfaces for synchronous interobject communication, and the Integrator interfaces.
Figure 2. Core MODCOM Interfaces
The following general requirements were determined.
- Language-Neutral Interfaces for all Services Utilizing Standard Protocols (COM). Our goal is that simulation services should be available to all developers regardless of the language or development environment that they are using.
- Keep it as Simple as Necessary, but no More so. The central elements/interfaces for simulation should only contain what is common to all simulations and is necessary for performing simulations. Added feature, extensions, or advances simulation tools should not be part of the core.
- Models Should be Understandable and Self Evidently Correct. The expressions/equations for defining a simulation should, when expressed in the simulation system, be uncluttered by technical details and easily understood. The manner of expressing a model should be similar to how a model is expressed in a conceptual (paper and pencil or analytical) context.
- Models Should be Easy to Modify. When you want to add a new feature; it should be obvious how to do it. The simulation framework (and the simulation system itself) should be designed such that the modules that developers create will be easy to modify later. Our goal here is to minimize and manage software accretion.
Core Simulation Services:
- Registration and Coordination of Simulation Objects. The simulation environment should be able to accept new simulation objects at runtime and manage their (simultaneous) execution.
- Clock Services. Clock Services are for synchronizing time flow interaction between simulation objects, supporting both continuous, discrete-event, and mixed simulations. The same mechanisms present in the existing BAG simulation libraries should be present.
- Data storage for Multirun (e.g. Monte Carlo) Model Analysis.
- Network-Distributed Simulation Object Coordination.Simulation users should be able to use simulation components that reside on other computers without needing to copy/download the files that implement the components. This feature should be as transparent as possible to simulation users and designers.
Core Simulation Object/Module Services:
- 2-D Time Series Data Collection/Input/Output. Simulations should be able to store and retrieve time series data at runtime. This requirement applies to any 2-D data set, regardless of how it is stored (i.e., any mechanism that can retrieve data should be able to do so at runtime).
- 3-D Time/X/Y Data Collection Input/Output. For example, to support spatially distributed models), particularly with links to ArcInfo/OpenGIS datasets. We should be able to support spatially distributed models.
- Capabilities Exposure. Capabilities exposure is used for the ability to expose data/functionality in a standardized manner and to allow more intelligent multimodule simulation assembly and efficient object-to-object communication. This requirement also facilitates the use of a visual model building tool in that users can see at runtime what capabilities a module has without having to delve in the module code.
- Intellectual Property Control. Intellectual property control should provide the ability to control the use of specific simulation objects. This will allow some form of licensing for modules containing commercially developed code.
- Simulation Object Self-Documentation Facilities. Each module should contain some information about what it does, what data it needs, and what information it provides. The goal is that information (in a human readable format) should always be available with a module. This feature also facilitates use of a visual tool. Also, it simplifies management of modules because there are fewer files to manage.
- Provide Standard, Prebuilt Simulation Objects for Common Tasks/data Structures (e.g., lists, queues, timers, etc.)
- Standardized Data Organization. Standardized data organization is a database schema for storing/retrieving data used or produced by a simulation. This includes multirun simulations. Also, the schema should include a representation of simulation objects to facilitate object persistence.
- Transparent Access to Data. A uniform (single application programming interface) should be available for storing/retrieving data from the simulation database.
- Support for 2-D and 3-D Datasets. The data base schema/database implementation must be able to store time-series and spatial data in addition to simulation objects.
- Integration Methods for Ordinary Differential Equations. It is assumed that many models will be expressed in terms of differential equations. ModCom provides methods (e.g., Euler, Runge-Kutta) that allow any simulation object to solve differential equations. These methods are essentially the same integration services that are present in the existing BAG framework.
- Integration Methods for Partial Differential Equations. Same as above except that the existing BAG framework does not allow for partial differential equation (e.g., difference grids), so a new implementation will be necessary.
- Parameter Estimation Methods. Large models may have hundreds of parameters. Automated numerical estimation of parameter values is an essential feature for calibration. Interfaces are provided to allow the inclusion of any number of parameter optimization algorithms; a Marquardt based method and a genetic algorithm based methods were developed.
Interobject Communication Services:
Asynchronous Message Posting (Blackboard). Mechanisms should be available that allow simulation objects to exchange information. Furthermore, the method of exchange should not limit or impose any structure on how the information is formatted. To this end, two styles of interobject communication were implemented: asynchronous (blackboard) and synchronous (listeners). Both methods will allow simulation objects to communicate directly (via their interfaces).
Synchronous Message Broadcasting (Listeners): See above.
- Random Number Generators for Common Distributions. These should include all the distributions normally associated with stochastic simulations. A uniform means of using the distributions should also be provided.
- Analysis of Monte Carlo Simulations. The module should be able to compute statistics associated with mote Carlo simulations.
- Sensitivity Analysis. The module should be able to compute basic statistics associated with sensitivity analysis. Automated methods should also be present that allow simulation designers to run sensitivity analyses without significant modification to existing simulations.
Analysis and Visualization Services:
- Visualization of 2-D Datasets. A module should be available for visual (on screen) display of 2D graphs (e.g., variables x time). The model should be able to graph input data and simulation results.
- Visualization of 3-D Datasets. A module should be available that can display (on screen) spatially distributed data (GIS data sets), either input or generated.
- Statistical Analysis of Simulation Results. Facilities should exist to compute common statistics for analysis and hypothesis testing.
Model Construction Services:
- Model Construction Environment. The purpose is to allow assembly/connection of existing compiled simulation objects/modules. One of the primary goals of this project is to provide tools that will allow users to form new simulations by connecting existing modules without writing any code. The model construction environment will provide a visual (graphical user interface driven) environment for establishing connections between modules, running a simulation with the connected modules, and using visualization modules to view simulation results.
- Object-Oriented Simulation Language. The purpose is to create compiled/interpreted simulation objects. The goal is to provide a machine independent specification of connected modules (i.e., ones specified with the model construction environment). In addition, the language should be able to specify new simulation objects that can be used in the model construction environment.
Visual Modeling Tool
The ultimate goal of the Visual Modeling Tool (VMT) is to enable modelers to concentrate on the science involved in their simulations and to free them from the details of programming their simulations by hand. Very few of the users that are the target community for the ModCom framework are C++ programmers. In fact, they are much more likely to be Visual Basic or Delphi programmers. It is also hoped that eventually the user base of the ModCom framework will grow to include nonprogrammers. That said, it would always be possible for those modelers who wish to write their own simulations to extend the ModCom framework. It is a goal of the developers of the ModCom framework to enable the creators of new models to work in the programming language of their choice.
The ModCom framework itself has been designed and implemented using Microsoft’s COM. This implementation choice dictates that the VMT is COM-aware. There are several possible ways in which the VMT could satisfy this constraint. The VMT we developed was written in C++ and takes advantage of COM client services to host ModCom-based simulation components. Other efforts to develop VMTs based on the ModCom specification are underway. These VMTs provide simulation support in the following areas: defining the requirements, analyzing specification and implementation issues, and building a prototype simulation.
Specification and Implementation Issues
Components and tools supporting all aspects of the specification described above were implemented in this project. The ModCom framework is designed to allow its users to employ a component-based development strategy. Although the notion of a component defies exact characterization, the ModCom uses as an operational definition objects that are instantiated from the classes SimEnv and SimObj. Given this definition, for the VMT to be able to satisfy its first requirement (assemble collections of predefined components), the VMT must be able to create instances of and be able to manipulate SimEnv and SimObj objects and objects created from classes that inherit from SimEnv and SimObj. The VMT makes use of the fact that interfaces to the SimEnv and SimObj classes are exposed via COM interfaces.
The VMT must be able to satisfy two different usage scenarios. In the first case, the VMT will only be required to work with those objects that are created from classes that are delivered as a part of the ModCom framework. In the second case, the VMT must also be able to work with classes that inherit from SimEnv and SimObj and that may not have been available at compile time. This enables users to define their own custom, inheritance-based SimEnv and SimObj classes. The users can then integrate their newly created classes with the predefined ModCom classes via the VMT.
Our investigations have shown that Microsoft’s C# programming language combined with the .Net framework is capable of satisfying both of these usage scenarios. The first scenario can be satisfied via a mechanism known as early binding. In this case, the predefined ModCom classes, in the form of COM classes, are available at compile time. The second scenario makes use of a technique known as late binding. The use of late binding allows objects and their public methods to be discovered at run-time. This methodology is obviously not as efficient as early binding. That said, the amount of flexibility that late binding allows for extending the ModCom framework is believed to outweigh any performance risk. Nonetheless, any use of late binding will need to be carefully thought out. Both of these methods are documented in a wide variety of currently available literature.
One of the aims of this project to enable model developers to create new models in their preferred programming language. Our work leads us to believe that the Net framework will be able to fulfill this goal. In addition to support for Microsoft’s own stable of programming language environment—Visual Basic, C#, and C++—various commercial entities have announced that they will be providing support for languages such as Cobol, Delphi, Haskell, Mercury, Oberon, Scheme, and Smalltalk. This is not an exhaustive list.
Project Research has Addressed the Following Issues
This project dealt with requirements and design specification issues, as well as implementation of core simulation services and modeling tools derived from those specifications. As a result of these efforts, a robust specification and toolset was created that can assist modelers in rapidly developing reusable simulation components and rapid assembly of complete simulation models with those components. Additionally, a variety of simulation support services were implemented, ranging from data management, visualization, Monte Carlo simulation, and parameter optimizations, to assist the modeler in focusing on those domain-specific aspects of model construction. We also established communications with several other groups involved in model development and successfully incorporated them into the design process for this project. A large issue has been the need for a language-neutral approach that allows object-oriented constructs to be used in model development. We pursued this through by taking advantage of Microsoft’s COM architecture and related datatypes. By careful use of COM-like interfaces definitions, without reliance on COM’s internals, we have successfully managed a language-neutral approach that is not strictly tied to COM; for example, the basic libraries have been successfully compiled and executed on a non-COM platform (Linux). All products from this project, including source code, are available for download on the Web (see http://biosys.bre.orst.edu/modcom Exit for more information).
Journal Articles on this Report : 1 Displayed | Download in RIS Format
|Other project views:||All 4 publications||1 publications in selected types||All 1 journal articles|
||Hillyer C, Bolte J, van Evert F, Lamaker A. The ModCom modular simulation system. European Journal of Agronomy 2003;18(3-4):333-343.||