Introduction How to Evaluate Complex Landscape Initiatives for Learning & Transformation?

The Rhodes 老虎机游戏_pt老虎机-平台*官网 Chair in Environment and Sustainability Education has explored since 2016 (and earlier) methods, processes and frameworks for evaluation to more strongly support learning and transformation towards social justice and environmental sustainability. In 2019 this work gained new momentum with the formation of the Community of Practice on Social Learning and Sustainable Development under the SARChI Chair in Global Change and Social Learning. The NRF CoP created an opportunity to deepen, synthesize and more widely share the work done thus far.

 

Central to this work has been the development of a ‘hybrid’ framework for monitoring, evaluation, reporting and learning in systemic social-ecological programmes focussed inter alia on restoration, resilience and natural resource management, under conditions of complexity. The hybrid model accommodates indicator-based reporting that can be aggregated by donors and government, and seeks ways in which this data can be used more effectively for communications and learning. At the same time, it overcomes the limitations of indicator- and target-based monitoring and reporting through dialogic theory-of-change mapping, collective case studies and action experiments as part of adaptive learning cycles, and realist evaluation designs to answer system-level questions. Key ingredients in the hybrid model are dialogic reflections, up-down and all-round communications, trust, and investing strongly in the starting conditions in the system, so as not to rely excessively and exclusively on compliance and control measures.

The evaluation research and development work of the Chair in Environment and Sustainability Education is intended for a dialogue with evaluators, policy makers, evaluation commissioners, programme designers, managers and funders of multi-dimensional, multi-party programmes responding to complex issues such as climate change, food and water security, unemployment, loss of biodiversity and livelihoods in Africa’s culturally rich and naturally beautiful landscapes.

 

The hybrid model has been developed over several years in a number of programmes and partnerships (see figure). These include:

• The Resilience in the Limpopo-Olifants (RESILIM-O) programme of the Association of Water and Rural Development (AWARD); SANParks and USAID;
• the Rhodes 老虎机游戏_pt老虎机-平台*官网 Tsitsa Programme with the Department of Forestry, Fisheries and Environment (DFFE, formerly DEA);
• the Department of Higher Education and Training (DHET) and 21 Sector Education and Training Authorities (SETAs)
• SANParks’ Adaptive Planning, Implementation and Evaluation Project in the context of Park Management Plans.
• Ongoing research of the SARChI Chair of Global Change and Social Learning and the students of the Environmental Learning Research Centre in the Education Faculty at Rhodes.

 

 

Current projects include a knowledge synthesis across 12 relevant sub-fields of evaluation.

 

 

Recent Publications

• Rosenberg, E. Kotschy, K. Burt, J., Mudau, V. and Pollard, S. 2019. Complexity-sensitive M&E in a coupled social-ecological system in southern Africa: A hybrid methodology developed in AWARD’s RESILIM-O program. Submitted for publication.
• Rosenberg, E. & Kotschy, K. 2020. Monitoring and evaluation in a changing world: A Southern African perspective on the skills needed for a new approach. African Evaluation Journal Vol 8, No 1 DOI: https://doi.org/10.4102/aej.v8i1.472
• Ward, M.R. & Rosenberg, E. 2020. Revealing mechanisms in a transdisciplinary Community Reforestation Research Programme. African Evaluation Journal | Vol 8, No 1 DOI: https://doi.org/10.4102/aej.v8i1.467
• Rosenberg, E. 2020. Synthesis and elaboration of critical realist methodology for green skills research. Chapter 13 in Rosenberg, Ramsarup and Lotz-Sisitka (Eds) 2020. Green Skills Research in South Africa: Models, cases and methods. Routledge, London.
• Roux, D., Nel, J., Novellie, P., Freitag, S., Rosenberg, E. 2021. Evaluating and reflecting on coproduction of protected area management plans. Conservation Science and Practice, in press.

 

Recent Student Studies

• Mushwana, M.V. 2020. A study on reporting and learning in three natural resource management programmes in South Africa. M.Ed. Thesis, Education Faculty, Rhodes 老虎机游戏_pt老虎机-平台*官网.
• Human, J.S. 2020. Developing social indicators for the evaluation of natural resource management programmes using a capability approach in the Eastern Cape, South Africa. M.Ed. Thesis, Education Faculty, Rhodes 老虎机游戏_pt老虎机-平台*官网.
• Mtati, N. 2020. Towards realising the benefits of citizen participation in environmental monitoring: a case study in an Eastern Cape natural resource management programme. M.Ed. Thesis, Education Faculty, Rhodes 老虎机游戏_pt老虎机-平台*官网

 

Importance of Evaluation - Policy and Wider Contexts

In many South African government and programme contexts there is a general recognition of the importance of M&E, as evident in, for example:

1. The existence of a Department of Planning, Monitoring and Evaluation (DPME) in the Office of the Presidency. This Department has produced a National Evaluation Framework and National Evaluation System, along with a number of M&E guidelines (www.dpme.gov)
2. In the Post-School Education and Training (PSET) sector, the Department of Higher Education and Training (DHET) has a high-level M&E framework, and the National Skills Authority has been designated to evaluate the performance and impact of the 21 Sector Education and Training Authorities (SETAs), all of which are also meant to have their own M&E strategies and systems
   
   
3. South Africa’s National Biodiversity Framework 2019 - 2024 (gazetted in March 2021) builds on the 2018 NBF which named “strengthening evaluation for adaptive management” as one of the priority actions for the field (p.4); the updated NBF further outlines a number of key national strategies, including
1. the Biodiversity Human Capital Development Strategy
2. the Man and the Biosphere Strategy
3. the Strategy for Ecosystem-based Adaptation
4. the Biodiversity Sector Climate Response Strategy,
5. the Strategy for Mainstreaming Gender in the Environmental Sector, all of which include an explicit evaluation component.
   
   
4. Funders of landscape initiatives (like the Global Environmental Fund) have additional M&E systems.

 

While governments, funders and implementers invest considerable resources and time in M&E and reporting, the returns are often limited. Implementers, too, are mindful of the value of M&E and are often looking for ways to evaluation that overcome the strictures of the given frameworks, while also yielding greater insights to inform practice.

 

Can we make M&E more useful? Reframe it as a (governance) process of / supporting learning and transformation, rather than merely reporting on targets met?  

 

 

Evaluation as a cross-cutting, transdisciplinary activity

Evaluation is a cross-cutting, transdisciplinary activity that is applied in many different fields of study and endeavour. While evaluation is well established as a field in its own right and served by a number of evaluation-focused journals, it is also discussed separately in the literature of the many different fields in which it is used, often without awareness of relevant discussions, debates and solutions being proposed elsewhere.

The purpose of this work is to synthesise how evaluation has been conceptualised and used in a selection of different fields; specifically, how different fields have addressed the challenges of evaluating complex systems. The aim is to build awareness of the breadth of the evaluation enterprise, identify areas of strength and weakness, build linkages across fields, and point the reader towards key literature and publication channels. Thirteen rapid reviews were compiled on the use of evaluation in different loosely-defined fields of study. The fields of study were chosen to provide a range of different complex contexts within which evaluation has been applied. Each short review (1-3 pages) outlines key methods and theories used in that field of study, current debates, trends and challenges, and potential publication channels. This work will be of value to evaluators, researchers, practitioners and policy makers interested in using evaluation to foster learning and transformation in a variety of complex social-ecological systems.

 

Synthesis:

Kotschy, K., Simelane, H Rosenberg, E. A Review of Evaluation Trends and Practices Across 13 Different Environment-Related Fields

 

Short Reviews:

 

 

The Environmental Learning Research Centre (ELRC), Rhodes 老虎机游戏_pt老虎机-平台*官网 functions as the Makana and Rural Eastern Cape Regional Centre of Expertise for Sustainable Development (RCE).The ELRC forms part of a worldwide network of such centres connected through United Nations 老虎机游戏_pt老虎机-平台*官网. As an RCE, we aim to promote environmental sustainable development in our region through partnering with local and regional stakeholders, building platforms for sharing information and experience, and encouraging education around sustainable development in institutions and professional environments.

 

Project Executant
Professor Eureta Rosenberg 
Chair of Environment and Sustainability Education
Rhodes 老虎机游戏_pt老虎机-平台*官网, ELRC

Research Associate
Dr Karen Kotschy

 

 

Last Modified: Tue, 28 May 2024 13:53:34 SAST