Hydrological Decision Support Framework ( HDSF ) – Initial design #

A Hydrological Decision Support Framework (HDSF) is being developed within a Water Research Commission (WRC) -funded project. The HDSF is intended as an integrated modelling tool for use by water resource managers and catchment management agencies (CMAs) in assessing and managing water resources. The HDSF will facilitate linking and running models within a common graphical user interface containing appropriate geographic information system and data analysis tools linked to a common database designed to store spatial and temporal data. Selected hydrological modelling framework applications and modelling framework development tools were reviewed. A framework for linking models was also reviewed. As a consequence of these reviews, it was apparent that no single currently available system could provide all the requirements for the HDSF. One of the modelling framework applications reviewed which met many of the requirements was selected and will be further developed. The initial design of the HDSF is presented in this document.


Introduction
As demand for water resources increases in South Africa it is necessary for water to be managed and utilised with greater efficiency.The management paradigm has changed from a focus on surface water supply quantity to a more holistic approach which also considers water quality, the environment as a water user, surface water-groundwater interactions, and social and economic impacts.In recent years there has been an international trend towards this new management paradigm known as Integrated Water Resources Management (IWRM) (Havnø et al., 2002;Schulze, 2002).In South Africa this new paradigm has been captured in Section 3 of the National Water Act (NWA) of South Africa (Act 36 of 1998) as follows: '3.(1) As the public trustee of the nation's water resources the National Government, acting through the Minister, must ensure that water is protected, used, developed, conserved, managed and controlled in a sustainable and equitable manner, for the benefit of all persons and in accordance with its constitutional mandate.
(2) Without limiting subsection (1), the Minister is ultimately responsible to ensure that water is allocated equitably and used beneficially in the public interest, while promoting environmental values.
(3) The National Government, acting through the Minister, has the power to regulate the use, flow and control of all water in the Republic.'(NWA, 1998).
IWRM requires a greater understanding of complex hydrological systems.Since the 1980s advances in computer science combined with improved understanding of hydrological processes has resulted in the development of computer simulation models that capture and aid understanding of hydrological systems.However, many of the models were developed for a particular domain within the hydrological system, such as surface water, groundwater or ecology, by experts in that domain.IWRM requires that the models for each domain be integrated and that scientific experts collaborate in the holistic assessment and management of water resources.
The need to integrate water resource models has led to another trend, which is towards the development of modelling frameworks.Traditionally models have each been run within their own modelling systems, each consisting of similar tools to prepare model input data, write model input files and analyse model output.Modelling frameworks seek to reduce duplication of effort and ease model use and development by providing common data preparation and post-processing tools, including a Geographic Information System (GIS) and database management, for use with several models.Some modelling frameworks have progressed further than this by breaking models down into sets of modules which may be linked together to create models tailored to meet specific IWRM scenarios (Kralisch et al., 2004;Rahman et al., 2003;Argent and Rizzoli, 2004).However, there are many legacy models in existence, and as it is not financially practical to restructure all these models into a modern programming language, some other means of integration has to be developed.Such legacy models have been linked in series by running one model for a simulation period, converting the model output files to the input format of the second model, and then running the second model.This method of linking models is suitable if there are no feedback relationships between components of the two hydrological systems simulated by the models.Recent developments in computer science provide solutions to this problem and allow integration of legacy models on a time-step-by-time-step basis, which is referred to as linking models in parallel.Integrated models allow modelling at different but appropriate spatial and temporal scales.For example, modelling surface water runoff at a daily scale but baseflow at a monthly scale, or modelling areas under irrigation at field scale but the upstream catchment areas at a broader catchment scale.However, the ability to link models does not make integrating models a trivial matter and it may be incorrect to link two models of different spatial or temporal scales if the concepts on which the models are based are incompatible.Hence the collaboration of experts in the different water resource modelling domains is crucial.
From the literature it is apparent that there is a growing realisation of the need for hydrological modelling frameworks and that there are a number of different frameworks under development internationally (Argent and Rizzoli, 2004;HarmonIT, 2002;Marston et al., 2002).The South African Water Research Commission (WRC) is funding a project (WRC, 2004), in which the primary objective is to adopt, further develop and implement a modelling framework, termed the Hydrological Decision Support Framework (HDSF), within which integrated hydrological and related simulation modelling can take place.The HDSF is intended as an integrated modelling tool for use by water resource managers and catchment management agencies (CMAs) in assessing and managing water resources.The following definition of a modelling framework was proposed for use in this project: A modelling framework is an open modelling environment in which existing models (or modules), databases and user interfaces can be linked or organised in a consistent manner.The requirements for the HDSF were first identified by the project team, then refined and prioritised by delegates at a workshop.An initial review of modelling frameworks led to six being selected to be reviewed in more detail and two of these frameworks were selected for use in the HDSF.The objective of this paper is to present the list of framework requirements, a brief overview of each of the six frameworks reviewed, the reasons for selecting two of the frameworks, and an initial design for the proposed framework.

Framework requirements
The process of designing the HDSF started with the identification of the requirements for the HDSF.These requirements are summarised in Table 1.The identification of requirements was an important part of the design process as the requirements form the criteria on which the reviewed frameworks were evaluated for use in the HDSF.

Review of hydrological modelling frameworks
After an initial review of modelling frameworks, six were selected to be reviewed in more detail.These six frameworks were selected as they had either been developed in South Africa, used in South Africa or were based on new computer programming technology.On completion of the reviews it was found that the six modelling frameworks reviewed could be divided into three categories: The modelling framework applications are complete functional applications and have the potential to be customised to suit the requirements of the HDSF.If none of the modelling framework applications were suitable for the HDSF project then the modelling framework development tools could be used to build a whole new modelling framework application for the HDSF.In addition to development tools for graphical user interfaces (GUIs), OMS and TIME both include protocols and modelling infrastructure supporting the development of hydrological modelling modules which may be used to built customised hydrological models.The OpenMI model linking framework could be used to link models in whichever modelling framework option was selected.A brief overview of each of the six frameworks reviewed is presented in this section.Allow exchange of data with commercial GIS systems using common formats.Provide tools as extensions to commercial GIS systems.Model input User friendly graphical user interface (GUI) allowing model input data to be entered and reviewed with facility to do range and logic checking of data entered.
Ideally include an extensible generic GUI but allow for model specific GUIs.Easy configuration of catchments and networks.

Data analysis
Comprehensive and easy to use set of tools for processing and display of model input and output data.Tools should include the graphing and comparison of time series data and statistical analyses.

Reviewed framework: BASINS
The Better Assessment Science Integrating point and Nonpoint Sources (BASINS) system is described as a multipurpose environmental analysis system designed for use by regional, state and local agencies in the United States of America (USA) to perform catchment and water quality-based studies (USEPA, 2001).It is designed to support the estimation of total maximum daily loads (TMDLs) using a catchment-based approach including both point and non-point sources for a variety of pollutants, at a variety of scales (USEPA, 2001).The objectives of BASINS are to facilitate examination of environmental information, support analysis of environmental systems and provide a framework for examining management alternatives.BASINS was developed by the United States Environmental Protection Agency's (USEPA's) Office of Water and was first released in 1996.
BASINS is a system integrating GIS, data analysis and modelling.The BASINS system includes a set of custom databases compiled from a wide range of federal sources in the USA including national databases at the USEPA and United States Geological Survey (USGS).Data are extracted from these databases and stored in Water Data Management (WDM) and DBF files which may be accessed by BASINS models and tools.BASINS uses ArcView to provide a customised user interface.The user interface contains all the standard ArcView menu, button, and toolbar items which access query, spatial analysis, and map generation tools.The tools and menus specific to BASINS are accessed through the BASINS Extension Manager and include assessment tools, data management utilities, catchment characterisation reports, water quality models and catchment hydrology models.BASINS uses the GenScn post-processing tool to facilitate the display and interpretation of observed water quality and other time series data, and the analysis of model output data.Models to be run from BASINS would need to be able to read from and write to the relevant BASINS files in WDM and DBF format.Currently the HSPF, SWAT, QUAL2E and PLOAD models are included in BASINS.

Reviewed framework: SPATSIM
The Spatial and Time Series Information Modelling (SPATSIM) software is described as an integrated hydrology and water resource information management and modelling system (IWR, 2005).SPATSIM integrates spatial and time series information in a flexible framework that includes a variety of data storage, retrieval, analysis and display options suitable for the application of a range of different models (Hughes, 2002).SPATSIM was developed by the Institute for Water Research (IWR), Rhodes University, Grahamstown, and was first released in 1999.
The design of SPATSIM is based on the approach adopted by the Centre for Ecology and Hydrology (CEH) at Wallingford, United Kingdom (Hughes, 2002).SPATSIM contains an extensible data model enabling storage of spatial and temporal data.Spatial data are stored in Environmental Software Research Institute (ESRI) shape files and a set of related attributes stored in a Paradox-based relational database.The link between geographic data and attribute data is achieved using four data dictionary tables stored in the database.The GIS functionality of SPATSIM includes vector-based GIS tools for visualisation and management of spatial input and output data.SPATSIM includes tools for the analysis and display of time series data.SPATSIM has been designed in an extensible manner such that new models may be easily added without changing SPATSIM.For a model to be run from SPATSIM, the model or the model user interface needs to be modified to read from and write to a SPATSIM database.The data model used in SPATSIM facilitates linking of models in series.

Reviewed framework: WR IMS
The Water Resources Information Management System (WR IMS) is a framework for the management of water resource modelling studies and their data requirements.WR IMS also facilitates the storage and retrieval of metadata related to modelling studies.WR IMS was developed by the Water Resource Planning Systems -Systems Analysis sub-directorate at the Department of Water Affairs and Forestry (DWAF), South Africa.WR IMS was originally developed as a database management system for hydrological and system data to run the Water Resources Yield Model (WRYM) for a project in the Vaal River Basin in 1999 (DWAF, 2004).It has been developed further into a more general purpose information management system.
WR IMS uses a relational database to store data and metadata related to studies, models, sub-areas and scenarios.WR IMS currently uses a Microsoft Access database but other relational database management systems (RDBMSs) could be used.Each model included in WR IMS would require data tables specific to the model to be included in the WR IMS database or to be accessed from a different database.WR IMS includes simple GIS functionality to display spatial data stored in ESRI shape files.Each model included in WR IMS requires a model manager which may be accessed by means of a COM interface.Each model manager includes a graphical user interface specific to its model, and should be able to display and set up input data for the model, run the model, save model output, and display and analyse output from the model.

Reviewed framework: OMS
The Object Modelling System (OMS) is described as a Javabased modelling framework consisting of a library of science, control and database components, which facilitates the assembly of selected modelling components into a modelling package suited to the problem, data constraints and scale of application (OMS, 2005;Ahuja et al., 2004).OMS is a modelling framework that facilitates model development, evaluation and deployment (David et al., 2004).The concept behind OMS is to create all system and model tools as independent reusable components that may be coupled using standardised software interfaces to create an application-specific modelling package (Kralisch et al., 2004).Development of OMS began in 1996 in the Institute for Geography at Friedrich Schiller University, Jena, Germany (Ahuja et al., 2004).Since October 2000 development of OMS has continued as an inter-agency project supported by the United States Department of Agriculture -Agricultural Research Service (USDA-ARS), United States Geological Survey (USGS) and United States Department of Agriculture -Natural Resources Conservation Service (USDA-NRCS) (Ahuja et al., 2004).
The two main types of components in OMS are model components and system components.Model components are the building blocks from which models are created within OMS.System components are those used to assemble user selected model components to create an application specific model, populate the model with suitable data and then execute the model.OMS is supported by graphical user interface (GUI) components and utility components which include a data dictionary, data retrieval and storage, GIS, graphical visualisation and statistical analysis (OMS, 2005;Ahuja et al., 2004).OMS uses custom metadata tags to support component documentation, testing, proper component integration into a model, automatic user interface generation and model execution (David et al., 2004).Tools are available in OMS to assist in migrating legacy models either by direct implementation in OMS or by means of wrappers.It appears that individual models built from OMS model components would be able to be linked.

Reviewed framework: TIME and the Catchment Modelling Toolkit
The Invisible Modelling Environment (TIME) is described as a modelling and programming system for developing, applying and deploying environmental models (Murray et al., 2004).TIME has been developed on the Microsoft.Net platform and is a collection of .Net classes, libraries and visualisation components for the development of models and model applications.The Catchment Modelling Toolkit is a system of environmental modelling software which integrates a new generation of catchment models and modelling support tools (Marston et al., 2002).The aim of the Catchment Modelling Toolkit is to provide land and water managers, researchers and educators with an integrated collection of software tools and components to simulate catchment response to management and climate variability, at a range of scales and using a variety of approaches (Marston et al., 2002).To achieve this aim a modelling framework was required which allowed models to be developed and integrated quickly and consistently (Rahman et al., 2003).TIME is an environmental modelling framework developed to meet these requirements and is the foundation on which the models, model applications and other modelling tools included in the Catchment Modelling Toolkit are built (Catchment Modelling Toolkit, 2005).TIME and the Catchment Modelling Toolkit were developed by the Cooperative Research Centre for Catchment Hydrology (CRCCH) in Australia (Rahman et al., 2003).
The architecture of TIME is divided conceptually into five layers: Kernel, Data, Models, Tools, and Visualisation and User Interface.Each layer consists of a family of classes, with the classes in the upper layers using services provided by classes in lower layers.Developers create models in the Model layer using classes in the Kernel and Data layers.The Tools and Visualisation and User Interface layers contain classes that interact with models and provide most of the framework functionality, such as user interface generation and model linking.The use of TIME custom metadata tags allows models to remain independent of these tools, resulting in better model stability (Rahman et al., 2002).TIME makes use of .Net's introspection capabilities for dynamic discovery of system properties at runtime (Rahman et al., 2003).These system properties include class structure, class fields and methods, and custom metadata tags which allow TIME to automate several tasks which facilitate model integration and automatic user interface generation (Rahman et al., 2003).TIME seems to support the approach of restructuring models into a set of linked modules and does not appear to make provision for wrapping or linking to legacy models.

Reviewed framework: OpenMI
The Open Modelling Interface (OpenMI) is described as a generic interface allowing models simulating different waterrelated processes to be linked on a temporal and spatial basis, allowing simulation of process interactions (Gijsbers, 2003;HarmonIT, 2004a).The objective of OpenMI is to simplify the linking of models running in parallel and which operate at different spatial and temporal scales by means of direct transfer of data values between models without writing to or reading from intermediate text files.OpenMI focuses on resolving or improving several complicated model-linking issues, including differences in spatial and temporal scales, feedback loops, differences in spatial and temporal concepts (distributed vs. lumped, steady state vs. dynamic), different units and naming of variables, and distributed computing (Blind and Gregersen, 2004).OpenMI simplifies the linking of models from a computer science point of view, allowing modellers to concentrate on the complexities of linking models from a hydrological point of view.OpenMI has been developed under the European Commission-funded HarmonIT project towards meeting the goals of the European Union's Water Framework Directive.The project partners include three commercial partners (DHI -Water & Environment, WL | Delft Hydraulics, HR Wallingford Group) and several other partners from research institutes and universities in Europe.The fact that competing software vendors have joined forces in creating OpenMI, is a key advantage to achieving the objective of setting a standard (Blind and Gregersen, 2004).
The architecture for OpenMI is shown in Fig. 1.The most important part of OpenMI is the Standard (org.OpenMI.Standard), which consists of a set of interfaces.The Standard is not a piece of software, so may be implemented in any object-oriented programming language and related computing platform.Any model implementing the relevant interfaces contained in the OpenMI Standard is described as being OpenMI compliant and may be linked to any other OpenMI compliant model.While OpenMI focuses on data exchange between models at runtime, it may also be used to link models to databases and user interfaces (Blind and Gregersen, 2004).The HarmonIT project has concentrated on implementing the Standard in the C# programming language on the Microsoft .Net computing platform but will also provide a Java implementation (HarmonIT, 2004b).The other namespaces in the OpenMI architecture form the Open Modelling Environment, and provide a set of classes whose purpose is to simplify the migration process and to facilitate the linking and running of the OpenMI compliant models (HarmonIT, 2004a).A default implementation of each interface in the Standard has been created to form a group of classes known as the Backbone (org.OpenMI.Backbone).The org.OpenMI.DevelopmentSupport namespace contains a generic set of low-level classes that can be used in the development of an OpenMI modelling environment.The org.OpenMI.Utilities namespace contains a set of classes that have been created to reduce the amount of re-engineering required to migrate existing model engines and software systems to become OpenMI compliant.A primary design objective for OpenMI was that the cost, skill and time required to migrate an existing model to the standard should not be a deterrent to its use (HarmonIT, 2004a).The HarmonIT project recognises that there are many legacy models written in programming languages such as FORTRAN.To facilitate the linking of legacy models, a group of wrapper classes have been created which allow linking to these legacy models, without rewriting these models in an object-oriented programming language to meet the OpenMI Standard, and with a minimum of changes to the legacy models themselves.These wrapper classes take care of all the bookkeeping associated with handling links, events, exceptions, buffering and basic spatial and temporal interpolation.For a legacy model engine not written in a .Net language or Java to be suitable for wrapping it must be compiled to a dynamic link library (DLL) and must be able to separately initialise, perform single time-steps, finalise and to be disposed of (HarmonIT, 2004b).The org.OpenMI.Configuration namespace contains a set of classes created to help developers to administer, configure and deploy coupled OpenMI compliant modelling systems.The org.OpenMI.Tools namespace contains user interface components and other classes to facilitate user interaction with OpenMI compliant models at configuration and run-time.

Framework selection
The first step in selecting a modelling framework for use in the HDSF project was deciding on whether to use one of the existing modelling framework applications or to build a new framework using one of the modelling framework development tools.One of the considerations in the decision was the use of a whole model approach to incorporating models in a framework adopted by the existing frameworks, vs. the modular modelling approach adopted by the framework application development tools.It was decided that the whole model approach was the safest in terms of preserving the scientific integrity of legacy models, requires the least changes to legacy models, and would gain wider acceptance in South Africa amongst users of well established hydrological modelling tools.However, the adoption of an existing modelling framework application using the whole model approach would not preclude the implementation of modular modelling.One of the main advantages of using an existing modelling framework application is that it would allow a working version of the HDSF to be developed in a shorter time with fewer resources.The advantages of using one of the modelling framework development tools are that a framework could be developed to meet the specific requirements of the HDSF, and that the framework developed would be based on newer programming languages and technology.Though the development of a new framework specific to the requirements of the HDSF project, based on modular modelling and using the latest programming technology was attractive, it was decided at a framework selection workshop that the best approach, given the constraints of the HDSF project, would be to use one of the existing modelling framework applications.
Many factors were taken into account at the framework selection workshop when considering which existing modelling framework application to use in the HDSF project.One of the primary considerations was that the selected framework should meet most of the requirements listed in Table 1.Other important considerations were related to the accessibility to the code, suitability of the framework for further development, and availability of software support to the project team and future users of the framework.In particular it was acknowledged that the selection of a framework developed in South Africa would be beneficial to the project and encourage collaboration between hydrological modelling groups in South Africa.Given the above considerations, SPATSIM was selected for use in the HDSF project.SPATSIM satisfies many of the requirements for the HDSF and was developed in South Africa.The developer of SPATSIM is willing to collaborate and supports further development.A particular strength of SPATSIM is its generic extensible data model that includes storage of spatial and time series data.SPATSIM also includes generic GIS and data analysis tools.SPATSIM is currently being used in South Africa by water resource consultants, DWAF and in other WRC projects such as the Water Resources of South Africa, 2005 Study (WR2005).SPATSIM already includes several models including Reserve determination models and the widely used Pitman monthly model.The adoption of SPATSIM will facilitate both the further development of SPATSIM software and collaboration between hydrological modelling groups in South Africa.
To satisfy the requirement for the HDSF of being able to link models in parallel, OpenMI was adopted.OpenMI seems to be conceptually well designed and appears to be the best attempt so far at creating a standard for linking models, which is a result of the relatively large and diverse group of partners involved with the HarmonIT project.Some of these partners will be using OpenMI in commercial hydrological modelling software products.OpenMI has been well documented during the Har-monIT project.OpenMI is intended to be used for linking whole models, and not for the plug-and-play building of models from a collection of model components, which makes it suitable for the requirements of the HDSF project.Implementing OpenMI will give users access to a variety of other OpenMI compliant modelling tools.The purpose of OpenMI is to provide a standard means of linking models, particularly legacy models, and will also allow linking of models to databases and user interfaces.Any attempt to link legacy models requires changes to the model code, although OpenMI tries to minimise these changes as far as possible, some changes will still be necessary.

Initial design of the HDSF
The SPATSIM modelling framework will form the core of the HDSF and will be further developed within the HDSF project.The components of the proposed HDSF and main data flows are shown in Fig. 2.
The HDSF will include the SPATSIM user interface, the SPATSIM data model and the existing set of SPATSIM compliant models.The SPATSIM user interface and data model will be further developed to meet the requirements of the HDSF.It is intended that the HDSF project will implement OpenMI by providing tools to enable the use of the OpenMI to link OpenMI compliant models.In addition it is proposed that an OpenMI wrapper be created for SPATSIM databases as a means of linking OpenMI compliant models to SPATSIM databases.A set of external GIS tools will be created as extensions to ArcGIS software to provide more complex GIS functionality than provided by the internal SPATSIM GIS tools and to enable exchange of spatial data between ArcGIS and SPATSIM.Different options for linking models to a SPAT-SIM database to facilitate the use of legacy models will be investigated.

471
include a GUI to setup model inputs, the GUI application could itself be run from SPATSIM.An OpenMI wrapper for SPAT-SIM databases would enable OpenMI links with OpenMI compliant models.Models could be made OpenMI compliant by implementing the OpenMI Standard directly or by means of an OpenMI wrapper.As proof of the OpenMI concept, it is planned that the ACRU agro-hydrological model be added to SPATSIM by linking it to a SPATSIM database directly and also via OpenMI.
Many legacy models do not have GUIs which enable easier setting up and validation of model inputs by users.The feasibility of developing some form of generic GUI for hydrological models will be investigated.All hydrological models require similar types of model input variables describing the attributes of the hydrological spatial features they are simulating.It would be necessary to identify a set of hydrological spatial and nonspatial feature types, a set of attribute data types and a set of model input validation rule types.Based on these feature, attribute and validation rule types it may be possible to design a standard document structure for expressing these feature, attribute and validation rules possibly using Extensible Markup Language (XML) (http://www.w3c.org/XML ; http://www xml.org ).The generic GUI would be based on this standard structure and standard controls could be used to display each model input variable.

Discussion and conclusion
The trend towards IWRM has created a requirement for the integration of models representing different components of the hydrological system.The purpose of a hydrological modelling framework is to provide a modelling environment facilitating the integration of models.Six modelling frameworks were reviewed in detail to determine their suitability for use in the HDSF.The modelling frameworks reviewed were categorised into three distinct types: modelling framework applications, modelling framework development tools and a model-linking framework.The modelling frameworks reviewed were evaluated against a list of requirements identified for the HDSF.Due to time and resource constraints for the project it was decided that one of the modelling framework applications should be selected.The SPATSIM modelling framework was selected to form the core of the HDSF and will be further developed to meet the requirements of the HDSF.SPATSIM was selected as it was locally developed, is suitable for further development and meets many of the requirements for the HDSF.Further development could include internal and external GIS tools, data analysis tools and enhancements to the SPATSIM data model.In addition, it is intended that the OpenMI model linking framework be implemented in the HDSF to enable models to be linked in parallel and as a means of linking models to SPATSIM databases.Both SPATSIM and OpenMI make provision for the use of legacy models.By providing common data management, GIS, data analysis and model linking tools the HDSF will allow model developers to focus on model development instead of modelling infrastructure.The HDSF has the potential to be a useful tool for water resource managers and to promote collaboration between water resource modelling groups in South Africa.