Quality and effectiveness of strategic environmental assessment ( SEA ) as a tool for water management within the South African context

The water management sector has been one of the first sectors in South Africa to explore the application of strategic environmental assessment (SEA) during the mid 1990s. This paper presents the results of a performance evaluation on the quality and effectiveness of a key landmark SEA case study, namely the Mhlathuze Catchment SEA. The case study provides an ideal opportunity to gain valuable insights into the interpretation as well as the strengths and weaknesses of SEA as applied within the water management sector. The research results showed that the SEA achieved an ‘average’ overall input quality performance and a mixed output effectiveness performance. Reviewed against ‘direct output’ indicators it achieved a ’poor’ effectiveness performance results. However, the SEA also achieved significant successes in terms of ‘indirect outputs’, such as a more holistic approach to water management, facilitated more effective public participation and contributed to broader strategic planning in the department. The paper concludes by making recommendations to improve the quality and effectiveness of SEA within the water sector.


Introduction
These are exciting times for anyone involved with environmental assessment in general and SEA in particular.International events such as the European Union (EU) SEA directive in 2001 and the World Summit on Sustainable Development (WSSD) in 2002 have provided impetus and facilitated wide adoption, both within developed as well as developing country contexts (Dalal-Clayton and Sadler, 2005;Jones et al., 2005;Schmidt et al., 2005).The lack of empirical research on the performance of SEA remains the single biggest hurdle in our quest to improve SEA practice (Dalal-Clayton and Sadler, 2005).The reason why progress in performance evaluation has been so slow could be attributed to the very difficult conceptual and methodological challenges it presents (Cashmore et al., 2004;Partidario and Fischer, 2004;Partidario and Arts, 2005), but also because it requires some level of agreement on the identity and application of SEA (Sadler, 2004), which has not been forthcoming from an international perspective.It could be argued that a crosscutting relation exists between performance evaluation and the debates on the identity and application of SEA.Now that more contextspecific understandings of SEA have emerged, as reflected in national guidance and legislation of different countries, the possibility for performance evaluation becomes more viable.
Developing countries in particular have shown increasing interest in SEA as a tool that could deal with the limitations of project level EIA and integrate the concept of sustainability with strategic level decision making.South Africa has been highlighted as a leading developing country in terms of SEA and one of the few with a home grown approach (DEAT, 2000;Rossouw et al., 2000;Therivel and Partidario, 2000;Dalal-Clayton and Sadler, 2005).However, after years of extensive voluntary SEA practice within a range of different sectors (Retief, 2005) there is still no clarity on the identity of SEA, how it should be applied or, most importantly, what it is achieving within these sectors.One of the first sectors within the country to embrace SEA as a decision-aiding tool was the water sector during the mid to late 1990s.A landmark event in the introduction of SEA was the Mhlathuze Catchment SEA pilot study in 1999.However, application of SEA within this sector seems to have waned since 2000 (Retief, 2005).
This paper presents a retrospective performance evaluation of the 'quality' and eventual 'effectiveness' of the Mhlathuze case study, which provides an ideal opportunity to gain valuable insights and new perspectives on how SEA performed as a decision-aiding tool within the water management sector.Firstly the research design and methodology are explained followed by a brief description of the SEA and water management interface.Thereafter the Mhlathuze case study is introduced followed by the research results and analysis.The paper ends by highlighting main conclusions and proposing a way forward.
considered essential that a case study review protocol be developed which contains the instrument, as well as procedures and general rules, to be followed in data gathering and analysis (Yin, 2003).Such a protocol typically includes a conceptual framework, data collection procedures and questions for evaluation (Eisenhardt, 2002;Yin, 2003).
The case study review protocol applied to this research was designed by Retief (2005) and was based on a literature review of international, as well as South African SEA literature combined with a piloting exercise.The literature was reviewed to inform the development of a conceptual framework and the identification of existing performance criteria (and where criteria did not exist, methods to develop criteria) for SEA 'quality' and 'effectiveness', while the piloting process tested its internal and external validity.In line with international literature the protocol makes a distinction between 'input quality' at the level of application and 'output effectiveness' at the level of implementation (Sadler, 1996;Lawrence, 1997;Thissen, 2000a).Furthermore, the protocol distinguishes between three SEA 'input quality' components, namely: 'process', 'methodology' and 'documentation'.In terms of 'output effectiveness', two components were identified namely 'direct' and 'indirect outputs'.Direct outputs are understood to relate to the objectives of SEA, which include aspects such as changes to decisions, improvement in the environmental quality and changes to the contents of plans or programmes.Indirect outputs are more difficult to measure and include outputs such as changes in attitudes towards the environment, improved awareness, changes in institutional arrangements and departmental traditions, etc.
To provide some form of measurement, indicators were developed.Use of the term 'indicator' was preferred to 'criteria' because indicators suggest that they are indicative, whereas criteria imply precision not always achievable due to the complex nature of SEA (Todd, 2001).The methodology for the design of KPAs and KPIs was based on the agreed notion that overarching context-specific SEA principles and objectives should form the basis for the application of SEA (Marsden, 1998;Rossouw et al., 2000;Thissen, 2000b;Verheem and Tonk, 2000).Examples of such SEA principles have been designed for SEA internationally (IAIA, 2002), and also for South Africa specifically (DEAT, 2000).Ultimately, 14 key performance areas (KPAs) and 48 key performance indicators (KPIs) were identified for 'quality' review of SEA.It is based on the understanding that SEA involves a context-specific, sustainability-led, participative, pro-active and efficient process, which requires different methodologies during screening, situation analysis, scoping, environmental assessment and monitoring and review phases, the results of which need to be documented and communicated to decision makers and interested and affected parties (IAPs).In terms of effectiveness, 4 KPAs and 9 KPIs were developed.Table 1 provides a summary of the KPAs and KPIs for quality and effectiveness and their relation to SEA principles and objectives.To deal with the qualitative and subjective nature of the conformance measurement only three scales were used, namely, 'conformance', 'partial conformance' and 'non-conformance'.The basic assumption being that conformance to more indicators implies better quality and/or effectiveness performance.However, the literature clearly warns: '… just adding up separate variables, as in a quantitative survey approach will destroy the local web of causality, and result only in a smoothed down set of generalizations that may not apply to any specific case, let alone others ' (Miles and Huberman, 1994).
Thus, the analysis could not rely on merely aggregating the results for each performance area and then expect to conclude with an overall quality and effectiveness 'score'.The assumption that conformance to more indicators implied better performance remains qualitative and subjective in nature and could thus not be considered absolute, especially since the relative weight of the different indicators has not yet been established.For a more detailed description of the protocol see Retief (2005).

SEA interface with water management
Water management is particularly important in the South African context since the country is classified as potentially a country with chronic water scarcity (WRI, 1998).The new political dispensation after 1994 introduced a number of new paradigms to water management reflected in the National Water Act (NWA) (36 of 1998).This Act has been described as: '… the most far-reaching environmental law reform made by the new government and illustrates an unsurpassed example in South African law of how social equity, economic needs and natural resource management can be accommodated in a composite Act of Parliament.' (Glazewski, 2000).
The purpose of the NWA is to ensure that the nation's water resources are protected, used, developed, conserved, managed and controlled in ways which take into account, amongst other factors, meeting the basic human needs of present and future generations, promoting equitable access to water and promoting the efficient, sustainable and beneficial use of water in the public interest.To achieve its purpose the concept of 'private water' was abolished, with all water transferred to the state as public trustee, represented by the national Department of Water Affairs and Forestry (DWAF).The provisions of the NWA allow for the development of a National Water Resource Strategy (NWRS) that describes the division of the country into 19 Water Management Areas (WMAs), each to be managed by a Catchment Management Agency (CMA).Key responsibilities of the CMAs would be to formulate Catchment Management Strategies (CMSs) and issue water use authorisations.Furthermore, the Act is underpinned by ecological considerations as reflected by the introduction of the pioneering concept of a 'Reserve', which determines that catchment resources should first and foremost be allocated to meet primary or basic human needs and the 'environmental Reserve' before other uses are considered.The NWA also allows for the declaration of some land-based activities, which may reduce the availability of water as streamflow reduction activities (SFRAs).SFRAs are defined as water users, and require authorisation as such.
It is from the water use authorisation perspective that SEA was first introduced into the water management debate.Discussions on the possible introduction of SEA in the water sector started in 1994, and culminated in the 'SEA for forestry initiative' in 1997.After extensive consultation, and in line with the more holistic approach propagated in the new water policy, the scope of SEA was expanded beyond forestry to include other water uses, and in 1998 the 'SEA for SFRAs initiative' was launched.
'The main aim of the SEA initiative was to provide an information base and decision-making framework to ensure that relevant sections of the NWA are implemented with regards to SFRAs.The SEA would not, itself, make Public participation forms an integral part of SEA because SEA puts people and their needs at the forefront of its concern.

•
To inform and involve IAPs throughout the SEA process.

•
To incorporate public inputs and concerns into decision making processes.

•
To facilitate information sharing.
Participative KPI 3.1: Was a formal public participation process followed, which informed and involved the IAPs throughout the SEA process?KPI 3.2: Were the IAPs satisfied with the public participation process?KPI 3.3: Did the SEA explicitly address public inputs and concerns?KPI 3.4: Were all key state departments and other governing bodies consulted during the SEA?
SEA provides a means of influencing decision making throughout its life cycle, from conceptualisation to implementation in an incremental and iterative way while facilitating the concepts of pre-caution and continuous improvement.

•
To ensure that the SEA is implemented early enough to influence decision making.

•
To facilitate continual improvement.
Pro-active KPI 4.1: Did the SEA ensure availability of the assessment results early enough to influence the decision making process?KPI 4.2: Was commitment confirmed to ensure that the results of the SEA be considered in future decision-making?
The benefits of implementing SEA exceed the costs.SEA adds value to existing decision making by focusing on key strategic environmental issues.

•
To provide sufficient, reliable and usable information.
• To optimise the use of time and resources.

•
To focus decision making on the key environmental issues.2000).

Setting the scene -Mhlathuze Catchment SEA
As mentioned in the previous section SEA was identified by DWAF as a decision-aiding tool that could inform water use authorisation, but could also help realise the objectives of the NWA.The Sub-directorate: Stream Flow Reduction Activities (SFRA) in the DWAF Directorate: Water Utilisation was tasked to develop SEAs for use within DWAF and, based on the outcome of a national level consultation and screening exercise, it was recommended that an

•
To justify the need for the SEA.

•
To clearly define the project objectives of the SEA.
Screening KPI 6.1: Was a formal screening method or criteria applied?KPI 6.2: Was the need for the SEA clearly defined?KPI 6.3: Was the purpose and/or objectives of the SEA clearly defined that could serve as reference for effectiveness review?
SEA determines the opportunities and constraints that the environment places on development.

•
To provide sufficient information on environmental attributes to identify opportunities and constraints.
Situation analysis KPI 7.1: Was a resource inventory prepared which describes the social, economic and biophysical aspects in the area at the appropriate scale and level of detail?KPI 7.2: Was the state of the environment (including economic, social and bio-physical) determined against set objectives, criteria or indicators?KPI 7.3: Were environmental opportunities and constraints identified by means of a justified methodology?
SEA identifies the most significant key strategic environmental issues.

•
To ensure that key strategic environmental issues are identified.
Scoping KPI 8.1: Was a formal scoping method applied?KPI 8.2: Did scoping assist in defining the scope and extent of the SEA? KPI 8.3: Did the scoping method(s) focus the SEA on key significant strategic issues?KPI 8.4: Were public inputs considered during scoping?
SEA determines the implications of strategic decisions on the environment.

•
To ensure that environmental implications of strategic decisions are considered.
Environmental assessment KPI 9.1: Was an assessment conducted against a sustainability framework (it may include sustainability parameters / objectives / criteria and indicators)?KPI 9.2: Were different scenarios and/or alternatives considered to identify the best option?KPI 9.3: Were the assessment techniques appropriate in terms of the context, available resources as well as data quality and availability?KPI 9.4: Were cumulative effects considered?
SEA aims for continuous improvement, which relies on monitoring and review mechanisms.

•
To ensure that the SEA is reviewed and the implementation of proposals monitored.
Monitoring and review KPI 10.1: Did the SEA propose a plan for monitoring?KPI 10.2: Has the SEA been independently reviewed?KPI 10.3: Has environmental monitoring been conducted?

KPAs Documentation KPIs
Information should be documented in a sound and justifiable manner.

•
To provide sound and justifiable information, which allows for verification of results

•
To contribute to existing environmental data and information Description of context KPI 11.1: Were the purpose and objectives of the SEA described in the documentation?KPI 11.2: Was the decision making contexts and linkages with other decision making processes described?KPI 11.3: Was a description provided of the SEA process followed?KPI 11.4: Were those involved in consultation and participation indicated?
Description of the state of the environment KPI 12.1: Was a description provided of the current state of the environment (either as a separate volume or integrated with the description of the baseline environment)?KPI 12.2: Was the state of the environment described against clear thresholds and/or limits of acceptable change in a way that highlights relative significance?Description of assessment methodology and results KPI 13.1: Were the different methods applied in the SEA described (relating to for instance screening, scoping and environmental assessment)?KPI 13.2: Was a description of key significant strategic environmental issues given?KPI 13.3: Were different scenarios and/or alternatives described?KPI 13.4: Were the recommendations and/or terms of approval described?KPI 13.5: Was a summary provided of difficulties encountered and subsequent uncertainties in results?
SEA should be documented in a manner that ensures effective communication of results in order to optimise the possibility of it influencing decision making.

•
To communicate the results of the SEA to decision makers.

Dynamics of the Mhlathuze catchment
The Mhlathuze secondary catchment is situated in the northern coastal region of KZN and covers an area of over 4 000 km 2 .By South African standards it represents a high rainfall catchment, with an annual average of 800 to 1 400 mm (a space denotes thousands) along the coastal belt.It includes nine quaternary catchments and as shown in Fig. 1 the coastal area is characterised by several coastal lakes.The catchment can be divided into three fairly distinct zones (DWAF, 2000c).Firstly, an upland region above the Goedertrouw Dam that comprises largely undeveloped communal land and extensive tracts of forestry around Melmoth.Secondly, a central belt, which also includes extensive communal areas combined with very intensive irrigated agriculture producing sugar and citrus products below the Goedertrouw Dam.Finally, a coastal belt with high rainfall, which includes forestry, dry-land sugar-cane farming and heavy industry.All industrial development takes place in the Richards Bay/Empangeni complex linked to the deep-sea port facilities at Richards Bay.In this regard the SEA highlighted the following key aspects in 2000: • Over 90% of the population (approximately 400 000 people) of the catchment live in communal rural areas where they practise subsistence agriculture.These areas also cover more than 50% of the catchment and have the greatest development need.Yet these people consume just over 1% of the total available system yield.If water is recognised as the cornerstone for development then significant water allocations have to be redirected to these areas.The lack of equity in water distribution within the catchment is identified as one of the key challenges facing strategic catchment management.Significantly the areas of highest biodiversity and conservation value (such as the Natal mist-belt grasslands) are also located here.Elsewhere in the catchment these areas have been transformed into commercial forestry and sugar-cane agriculture.• Commercial agriculture in the catchment involves forestry, dry land and irrigated sugar-cane as well as citrus farming.
Of these, irrigated sugar-cane received by far the largest water allocation, 65% of the total system yield.Forestry and dry-land sugar cane used substantially less at 5% and 1% respectively.This shows that tackling the water allocation to irrigated sugar-cane could potentially make the largest contribution to water savings for the catchment.• The industrial activities within the catchment are of national importance and there are expectations that the sector may double in size between 2000 and 2020.Industry utilises 28% of total water yield.Future expansion of industry will lead to competition for new and already allocated water in the catchment on the basis that it can provide a far more effective use of water for the greater good than other water users.The main difference with agriculture also lies with the high degree of assurance of supply demanded by industry.

Direct outputs KPAs Direct outputs KPIs
SEA influences the contents of plans and programmes.

•
To influence the contents of plans and programmes.
Policies, plans and programmes KPI 15.1: Were any plans or programmes amended based on the proposals of the SEA? KPI 15.2: Did the SEA facilitate the incorporation of sustainability objectives into relevant plans or programmes?
SEA facilitates the achievement of sustainability objectives

•
To achieve the SEA project objectives.

•
To achieve the SEA sustainability/ environmental objectives.
SEA objectives KPI 16.1: Were the SEA project objectives achieved (as described in the TOR)?KPI 16.2: Were the sustainability / environmental objectives achieved (as might be described in relation to the vision)?
SEA influences decision making SEA improves environmental quality

•
To improve environmental quality Environmental quality / sustainability KPI 18.1: Were changes to the environment observed since the completion of the SEA process, which could be attributed to the influence of the SEA? KPI 18.2: Did the SEA accurately identify the key significant strategic environmental issues?

Poor:
Failure to conform to large majority of indicators.

Average to poor:
Failure to conform, and/or partial conformance, to the majority of indicators.

Average:
Partial conformance to the majority of indicators, or an even spread in performance.


Average to good: Partial, and/or conformance, to the majority of indicators.

Good:
Conformance to large majority of indicators.

TAblE 1 contininued
• The total system yield for the catchment is estimated at 270 x 10 6 m 3 /a.Yet, water allocation exceeds the system yield by 32.4 x 10 6 m 3 /a.Whilst the water resources in the catchment were already clearly over-allocated, not all allocated water was necessarily utilised.In order to address the demand for water the system yield will need to be enlarged through importation of water or the existing yield will need to be allocated more effectively to ensure the greatest benefit to the largest number of people.
The diversity and complexity of the Mhlathuze catchment are reflected by a combination of a highly developed agricultural sector with strong dependence on irrigation, a growing population, pressing needs for rural development in the communal areas as well as growing water demands for an expanding industrial sector.The latter features could be considered broadly representative of many catchments within the country.

Input quality components: Review results and analysis
Table 2 provides the overall input quality and output effectiveness review results.The input quality review results show that the SEA inputs failed to conform to 14 of the 48 KPIs as identified by Retief (2005), partly conformed to 20 of these KPIs and conformed to 14.The results reflect a relatively even spread, which suggest an 'average' overall input quality.The following sections discuss the input quality review results in terms of the process, methodology and documentation components.

SEA Process
'One of the most important innovations in SEA lies in the emphasis on the process by which it is conducted rather than concentrating solely on its products.'(DWAF, 2000c).
The research results show that the SEA failed to conform to 4, partly conformed to 7 and conformed to 5 of the 16 indicators relating to process quality.This section describes the review results in relation to the different KPAs and special reference to KPIs 1.1, 1.3, 2.4, 3.1, 4.1, 4.2 and 5.3 (see Table 2).The first, and most obvious, feature of the SEA process was that it was not integrated or linked to any specific strategic decision-making process, but rather aimed to function as a standalone information tool for various levels of decision-making, from project level water licensing to strategic level CMSs (KPI 1.1).This feature was highlighted by international external reviewers as somewhat unusual and different from their understandings of SEA as primarily an assessment tool linked to a specific decision making level (Lee, 2000a).In theory, it could thus be classified as what is referred to in the literature as a 'proactive SEA', with the slight difference that it was expected to eventually integrate with decision making processes, most notably the formulation of CMSs.The classification of 'pro-active' SEAs is unique to South Africa and was coined by Retief (2005) for those examples where SEA was not linked to a particular decision-making level (policy, plan or programme).The SEA report even goes as far as to suggest that ultimately SEA would be integrated to such an extent that CMAs would not be required to 'do SEA' separately since the CMSs would fully integrate   2000c).This notion seems rather idealistic and conceptually vague.However, from a more practical perspective the understanding was that SEA should not be developed to replace or duplicate other statutory processes but rather to strengthen them and facilitate better decision making.
In retrospect, the timing of the SEA process was not ideal since it was initiated as a pilot study during a period before certain crucial statutory decision making structures, such as the CMA were established (in terms of the NWA), let alone the preparation of policies, plans and programmes by these struc-tures (Warren, 2004).Even four years after completion, at the time of the research, CMAs were still not functioning (Perkins, 2004).Moreover, the SEA was also initiated before the finalisation of the new municipal demarcation process in 2000 and subsequently the administrative boundaries and democratic systems in terms of local and regional authorities were in a transitional phase.The SEA team envisaged that these new structures would benefit greatly from the SEA already being completed, because it would pro-actively set the context and so assist in providing a 'soft landing' in terms of decision-making responsibilities (DWAF, 2001b).So the timing of the SEA resulted in various uncertainties in relation to the formulation of actor and process configurations (KPI 1.3) and ironically it can be concluded that the timing of the SEA led to it being perhaps too pro-active and ahead of its time (Warren, 2004) (KPI 4.1).
To deal with public participation a formal public participation process was conducted as part of the social analysis component and was considered a corner stone of the SEA (DWAF, 2000c;Van Jaarsveld, 2000) (KPI 3.1).The challenges for public participation related to the diverse profile of interested and effected parties (IAPs), ranging from big industry to rural subsistence farming communities.It was decided to focus the participation process on the communal rural areas because they were the largest group and presented the biggest need.However, a combined stakeholder workshop was also held, attended by representatives of agriculture and industry.The good attendance suggested that the participation process was relatively well received, although the non-alignment of catchment boundaries with institutional boundaries did leave pockets of the catchment unrepresented (Van Jaarsveld, 2000).
The apparent lack of focus of the SEA process (KPI 5.3) prompted an international external reviewer, to highlight some misconceptions, '…, you should decide whether it is to be an SEA or an integrated appraisal covering economic, social and environmental impacts' (Lee, 2000a).In reaction, the SEA team indicated that they were in line with the South African SEA understanding, which interprets the term 'environment' in its broadest sense (including the social, economic and biophysical components) and so by definition incorporating the concept of sustainability into the SEA (DWAF, 2001b) (KPI 2.4).This aspect might have been a misconception on the part of the international reviewer but it had a much more pertinent implication for the traditional approach to water management within DWAF.The consideration of social aspects relating to water management is a relatively new concept for DWAF and some were convinced that this paradigm shift had not taken root (Steyl, 2004;Versfeld, 2004;Warren, 2004).
This aspect was also reflected by the lack of top-level buy-in from the department.The initial intention was to establish a SEA directorate within DWAF to act as a 'clearing house' for strategic decision making.This never materialised, and subsequently commitment to implement the outcomes of the SEA waned (DWAF, 2000b) (KPI 4.2).The SEA project manager highlighted the lack of top level buy-in as one of the primary shortcomings of the SEA (Warren, 2004).The regional office of the Department also confirmed that they viewed the SEA as merely an information tool (Perkins, 2004).

SEA methodology
The first external review session conducted between December 1999 and February 2000 already highlighted certain difficulties in terms of the analytical/technical elements of the SEA (DWAF, 2000a;Lee, 2000b).Many of these aspects also emerged as part of this research where the SEA failed to conform and partly conformed to 7 each and conformed to only 3 of the 17 indicators.This section discusses the review results in relation to the methodology KPAs and with special reference to KPIs 6. 2, 6.3, 7.1, 8.2, 9.1, 10.1 and 10.2 (see Table 2).
SEA was originally considered as a tool to inform licensing decisions relating to forestry.However, during the three years' run-up to the Mhlathuze SEA it was decided to expand this original remit to allow a much broader scope.This meant that the SEA had to consider all activities (not just forestry) within the catchment but also all levels of decision making, from project to policy level.It is safe to say that this expansion of the SEA remit caused an 'identity crisis' and contributed to it losing its original clarity of purpose.So, instead of a screening process that assisted in clarifying exactly why the SEA was needed (KPI 6.2) and exactly what it intended to achieve (KPI 6.3), it managed to over-elaborate its potential and erode its clarity of purpose.The envisaged users, and even members of the SEA team, described the SEA as 'very vague' (Perkins, 2004) and 'amorphous' (Versfeld, 2004).External reviewers highlighted this point by referring to the SEA report: 'I could not find a single statement of purpose that I could use to evaluate whether the team had met its brief.' (Weaver, 2001) and '…, the apparent difficulties in focussing the work on SEA, as generally understood, has reduced its effectiveness and may have resulted in some misdirected effort.'(Lee, 2000a).
It became evident that the situation analysis was the main focus of the SEA and provided extensive data at the catchment scale (KPI 7.1).It included specialist studies on water quantity (Mallory, 2000), conservation and biodiversity (KZNCS, 2000), social characteristics (Van Jaarsveld, 2000) as well as macro- (Dallimore et al., 2000) and hydro economics (Creemers and Pott, 2000), some of which were incorporated into a GIS-based decision support system (DSS).The methods used for each specialist study varied, but they all focussed on determining and presenting the current state without providing much sense of anticipated changes in the system or baseline conditions.Furthermore, future trends and scenarios relating to, for instance, estimated future water demand, population growth, etc. were not fully anticipated (KPI 9.2).International external reviewers also highlighted this aspect (Wilson, 2001).DWAF admitted that they did not always know when to stop in terms of data gathering (Warren, 2004).The tendency from the SEA team to demand 'all the information' was an indication that the scoping component of the SEA was not properly conducted and failed to focus the extent and level of detail of the situation analysis (KPI 8.2).The latter was also raised by two of the external reviewers (Lee, 2000a;Weaver, 2001).The justification put forward by the SEA team for the extensive data gathering was that the GIS decision support system (DSS) required it (DWAF, 2001b).
Although the SEA did not actually assess any strategic level decisions, it did present sustainability criteria (which can also be considered as objectives) and indicators against which future decisions could be assessed (DWAF, 2000c) (KPI 9.1).For example the criteria included consideration of possible effects on the biophysical environment, social and cultural conditions as well as the economy.The implications of proposed actions or decisions then had to be considered as 'supporting', 'neutral to' or 'actively working against' the sustainability criteria (or objectives).These criteria and indicators were considered a good basis from which to refine the application of the SEA (Weaver, 2001), although importantly they did not include thresholds or limits of acceptable change.The external review also highlighted that the criteria were not weighted in terms of relevant importance, which would have been useful for decision makers when applying them (Wilson, 2001).However, no examples were found where these criteria had actually been used to inform decision making in the catchment.
Finally, although the SEA did not propose a specific monitoring programme it did raise a number of issues in relation to monitoring (KPI 10.1).Firstly, it stressed the need for more rainfall and runoff gauging stations in the catchment, data from which underpins the hydrological understanding of the catchment (DWAF, 2000c).Secondly, it pointed out that high-quality information inputs would be needed to support stakeholder involvement and capacity building initiatives.Thirdly, the fact that the SEA dealt largely with water quantity, and excluded water quality and groundwater issues, was primarily due to data constraints (DWAF, 2000c).However, in order to support the learning experience, a formal review process was implemented, which made use of local and international SEA specialists as reviewers who submitted reports during (DWAF, 2000a;Lee, 2000b;Weaver, 2000;Wilson, 2000) and on completion of the SEA (DWAF, 2000a;Lee, 2000a;DWAF, 2001b;Weaver, 2001;Wilson, 2001).Reviewers were tasked to review the methodology and contents of the studies and present ways to remedy deficiencies (KPI 10.2).

SEA documentation
The SEA documentation consisted of an SEA report with specialist studies on the social, biophysical and economic components attached as appendices.It also included a 13 page executive summary.The documentation failed to conform to 3, partly conformed and conformed to 6 each of the 15 indicators.This section reflects on the performance of the documentation in relation to the KPAs and with special reference to KPIs 11.1,11.3,11.4,12.1,13.1,13.5,14.1 and 14.4 (see Table 2).
The report was generally well structured, clearly explained and logically arranged (KPI 14.1).It gave a vivid description of the state of the environment as the outcome of the situation analysis (KPI 12.1) but also of difficulties encountered and uncertainties in results (KPI 13.5).Moreover, it clearly indicated who was involved with the participation process (KPI 11.4) and the issues raised (KPI 14.4).However, the report did not clearly describe the purpose and objectives of the SEA (KPI 11.1), nor was a description provided of the SEA process followed (KPI 11.3).Furthermore, the report lacked a description of the overall SEA methodology (KPI 13.1) linked to a particular SEA process description.It was interesting that the opinions of the external SEA reviewers differed markedly and were contradictory in relation to the SEA report.For example Weaver (2001) stated that, '… I found it difficult to clearly align the various chapters (2-8) with the overall purpose.The report is as a result, not well integrated and leaves the reader with the impression that integration occurred through stapling' and 'The information presented suggests great potential for duplication, overlap and conflict.' On the other hand Wilson (2001) felt that, 'This is a very full, informative and well structured report.I found especially stimulating its breadth of vision, and the way it pulled the critical issues together at the end.As the issues are so inter-linked, I found the cross-referencing within the text helpful.'Finally, the SEA documentation seemed to have reflected some of the limitations prevalent in the process and methodology, with a good description of the state of the environment and communication of results but lacking in clarifying the purpose and context as well as the SEA methodology and process.

Output effectiveness components: Review results and analysis
The effectiveness of the SEA was reviewed approximately four years after completion.The output effectiveness performance results show conformance to none, partial conformance to 3 and failure to conform to 4 of the 9 indicators (the status of 2 indicators could not be established).This suggests an 'average to poor' output effectiveness.
Based on the outcome of the review it is safe to say that the broader SEA initiative within DWAF has lost some of its original momentum and impetus.The vision of establishing SEA as a directorate within the department has not materialised (DWAF, 2000a).Moreover, the original members of the SEA team have been, in the words of the SEA team project manager, 'decimated' (Warren, 2004).He was the only member of the original team still employed by DWAF at the time of the review.The personnel losses have crippled the human resources required to drive the SEA concept forward and also had a negative influence on overall morale.Additionally, four years later the statutory structures required to implement the SEA, such as the CMAs, were still not established (Perkins, 2004).In light of limited personnel to champion SEA within the department and no formal structures to apply it, the poor review results on effectiveness are not surprising.DWAF confirmed that in future it would be unlikely for them to conduct another SEA for an entire catchment, but would rather focus on selective areas and or issues (Warren, 2004).Although the review showed that the SEA did not seem to perform well in terms of direct outputs it did make significant indirect contributions as described in the following sections.

Direct outputs
This section reflects on the general poor performance of the SEA in terms of direct outputs with specific reference to KPIs 15.1, 16.1, 17.1 and 17.2 (see Table 2).The results show that the SEA only partly informed the contents of strategic plans (KPI 15.1) and partly achieved its project objectives (KPI 16.1).Moreover, it was confirmed that the SEA did not, at the time of the review, directly influence decision making related to any specific water licensing application (KPI 17.1) and also that it was not adopted as a formal decision making tool within DWAF (KPI 17.2).Conformance to the indicator on environmental quality could not be determined during the review due to a lack of monitoring arrangements (KPI 18.1).
The two main areas that the SEA aimed to influence were the CMS prepared by the CMA as well as decisions with regard to water licensing.However, four years on, the CMA has not yet been established so no CMS had been formulated and the process for granting water licences also had not started.Thus, at the time of the review, the SEA had not yet informed CMS directly nor licensing decisions within the catchment.KZN DWAF, responsible for administrating and receiving water licences (issuing of water licences is vested at national level) did indicate during interviews that they were not yet sure exactly how they were going to use the SEA but did intend to use it, once they had completed their 'water use verification process' and started their 'compulsory licensing process' (Perkins, 2004).However, they also expressed their concern that the information contained in the SEA was probably already dated, which could negate its eventual usefulness.In terms of strategic planning, DWAF commissioned so-called internal strategic perspectives (ISPs) as an interim measure and a forerunner to the CMSs.The main dif-ference between ISPs and CMSs is that ISPs are internal working documents and that CMSs require public input.According to the consultant appointed by DWAF to assist in the formulation of ISPs, the SEA informed the ISP for Usutu-Mhlathuze WMA to some extent by providing baseline information (Versfeld, 2004), which is probably the most significant achievement in terms of direct outputs.

Indirect outputs
Interviewees indicated that 'It is sometimes difficult to gauge the exact influence of the SEA' (Versfeld, 2004) and that 'SEA is unlikely to get formal recognition from within the department for improvements in decision making' (Warren, 2004).This highlights the difficulties associated with identifying and isolating indirect outputs even for those directly involved with the SEA.In order to identify some of the indirect effects of the SEA it is also necessary to understand and appreciate the traditional role and function of DWAF, which has been to manage water resources largely in isolation from social and biophysical considerations.The department had what could be termed a 'hard approach' to water management focusing primarily on water quality and quantity.The SEA project manager stated that: 'The biggest mistake in this department is that we have always focused on integrated water resource management (IWRM) without really knowing what it means.IWRM is not just about water because it has a basis in all the water resource elements.I believe we should focus much closer on integrated catchment management (ICM)' (Warren, 2004).
It was apparent that there were deep-rooted traditions in DWAF that made them stick to their water management mandate, coupled with internal and external political forces which support specific water allocations.However, the review did identify some success in influencing these traditional ways of thinking about water management, towards more holistic approaches considering social aspects and incorporating public inputs.An example of this would be the concept of 'water for rural development', borne out of the inequities in water distribution highlighted by the SEA.This concept has now become central to strategic thinking in the department (Versfeld, 2004;Warren, 2004).Another indirect effect of the SEA seemed to have been to facilitate internal communication within DWAF.It was also highlighted that those involved with the SEA did carry their experience into other areas of strategic planning within DWAF.In the end it was expressed that the SEA team would be satisfied if the SEA influenced the way the department thinks and acts, even if SEA as a tool was not adopted (Versfeld, 2004;Warren, 2004).
Some of the products of the SEA had been taken forward since the completion of the SEA.The KZN Biobase study had been expanded by KZN Wildlife in the form of a conservation plan (C-plan) for the province.DWAF had also taken forward the public participation component by developing participation guidelines.Although the DSS had not been as successful as the SEA team would have hoped, it was confirmed that the state of the environment report compiled for the uMhlathuze local municipality did incorporate information directly from the DSS (CSIR, 2002).More generally it was argued by the SEA team that the SEA indirectly influenced the NWRS (Versfeld, 2004;Warren, 2004), although it was difficult to determine the exact extent of influence.
One of the most significant indirect outputs of the SEA linked to the public participation process was the realisation that rural communities were under represented and not satisfied with existing structures such as the catchment management forum (CMF).In reaction, the SEA process facilitated the attendance of rural representatives at the Mhlathuze CMF during April 2000, and although this was not the first time they attended, it was the first time that the issue of rural representivity was an official item on the agenda (DWAF, 2000c).Although public participation almost always leaves room for improvement, the formal DWAF review report concluded that, 'There is, particularly, a higher level of social understanding in forming decisions.Stakeholders have also become far more involved in the decision-making process.' (DWAF, 2000b).
Finally it is worth noting that the Mhlathuze SEA pilot study did provide the basis for the (still draft) SEA for water use guidance (DWAF, 2001a).All these elements suggest that the SEA did have a broad indirect internal and external effect.

Conclusion
This case provides an example of SEA applied to deal with contentious water management issues, which includes rights and access to water.The diversity and complexity within the Mhlathuze catchment encapsulates a microcosm of the overall South African context in relation to water management with high biodiversity, high social need (especially in rural areas) combined with extensive commercial agriculture and an expanding industrial sector.Ultimately the research results provide insight into the interpretation of SEA within the water sector, an indication of its strengths and weaknesses as well as brief glimpses into the realm of causality.
Because the evolution of SEA within the water sector developed largely in isolation from the broader national SEA debate, it is not surprising that its interpretation was also fundamentally different from that reflected in the national and international literature (DEAT, 2000).Within the water sector SEA is seen as an information tool whereas the literature describes SEA as a decision support tool.The main difference between the two interpretations is that as an information tool SEA serves as an independent database (or DSS) to be accessed whenever information is required at a range of different decision making levels (policy, plan, programme and even project level), while SEA as a decision support tool seeks to integrate with a particular strategic decision-making process with a purpose to inform and influence that process and its outcomes.
Critically considered it could be questioned whether the interpretation of SEA in the water sector could be viewed as SEA at all because, as the name suggests, SEA is essentially a strategic 'assessment' tool.SEA application should thus resemble some form of strategic level 'assessment', which was lacking based on the research results for the uMhlathuze case study.Ultimately it can be concluded that the lack of clarity on the identity of SEA, its very wide remit, uncertainty as to which decision-making processes it aimed to inform and general failure to convince decision makers of the added value of SEA, saw the SEA initiative within DWAF fade.However, DWAF's recent involvement in a major SEA project in the Eastern Cape (with a particular focus on forestry), as well as the appointment of new staff with SEA responsibility suggest that SEA is still being pursued by the department.There is furthermore talk of SEA for various problem areas such as the Western Cape Sandveld groundwater abstraction area.
To take the debate forward it is proposed that the identity as well as the remit for SEA within the water sector be reconsidered against lessons learned.Such debate could be informed through expanded case study research combined with comparative research with other sectors.Moreover, to deal with causality, real time action research could also be explored.Finally, the research results suggest that if SEA tries to be everything to everybody it will remain vague and amorphous and fail to achieve its objectives within the water management sector.

Figure 1 :Figure 1
Figure 1: Water features of the Mhlathuze catchment (Source: DWAF, 2000c, p16) Available on website http://www.wrc.org.zaISSN 0378-4738 = Water SA Vol.33 No. April 2007 ISSN 1816-7950 = Water SA (on-line) 161 Were the contents clearly explained, justified and logically arranged in sections or chapters?KPI 14.2: Were the specialist reports well referenced and integrated in a way that promotes a self-contained document?KPI 14.3: Was a non-technical summary provided of the main results and conclusions?KPI 14.4: Were the inputs received from IAPs incorporated in the report?

TAblE 1 contininued
AvailableSEA pilot study be conducted on a secondary catchment scale.The pilot study would test the application of SEA and assist in developing guidance for wider application.Ultimately the Mhlathuze secondary catchment in the KwaZulu-Natal (KZN) Province was identified as a suitable candidate for such a pilot study.It was against this background that the Mhlathuze Catchment SEA was initiated by DWAF, with the project management assigned to the sub-directorate SFRA.

•
To influence decision making.Decision making KPI 17.1: Were decisions changed or amended based on the outcomes and proposals of the SEA? KPI 17.2: Was the SEA implemented as a decision-support guideline for future development proposals?KPI 17.3: Did the SEA inform/guide subsequent project level decision making (such as EIA or water licensing)?