OXFAM EFFECTIVENESS REVIEWS USING PIALA IN MYANMAR, PAKISTAN AND BANGLADESH
Assessing the performance and impact of water governance, sustainable built industry, and climate resilience projects with Oxfam
Clients
Oxfam GB
Period
2015 - 2019
Services
Evaluation
Innovation Capacity
Locations
Myanmar, Pakistan, Bangladesh
Areas
Governance and Social Accountability,
Climate Resilience,
Migration and New settlements
Oxfam GB adopted our Participatory Impact assessment and Learning Approach (PIALA) for outcome and impact evaluations in Myanmar, Pakistan and Bangladesh as part of its global Effectiveness Review (ER) series. Adinda Van Hemelrijck, founder of Collaborative Impact, spearheaded this work.
PIALA-based evaluation of ‘Building Equitable and Resilient Livelihoods in the Dry Zone’ (DZ) in Myanmar (2015-2016)
With the aim of fostering sustainable livelihood opportunities and bolstering climate resilience in Myanmar's Dry Zone, Oxfam's DZ project spearheaded the development of community-led Membership Organizations (MOs) that could mobilize broad citizen participation in local development, advocate with township departments and parliament for the necessary water infrastructure, foster business partnerships with traders and suppliers, and cultivate civil society networks across localities.
The Dry Zone is an area that spans from lower Sagaing region to the western and central parts of Mandalay region and most of Magway region, where approximately one-quarter of the country’s population is living. Situated in the shadow of the Rakhine mountain range, the land is very dry and heavily hit by climate change. The project was coordinated by Oxfam and implemented between 2011 and 2014 by Network Activities Group (NAG) in Minbu and Oxfam in Thazi.
The PIALA-based evaluation sought to:
Assess to what extent the 64 MOs developed by the project were still functioning effectively;
Assess if/how they were positively impacting communities’ climate resilience 18 months after project exit, and
Learn about what influenced MO sustainability.
Oxfam wanted to ‘walk the talk’ of its Right To Be Heard Framework and uphold the principle of voice and participation in evaluation through the use of a participatory evaluation approach. The evaluation was expected to build local capacity in using this approach, for which it trained and engaged community researchers and local Oxfam staff.
PIALA-based evaluation of ‘Improving Urban WASH Governance and Accountability’ (IUWGA) in Pakistan (2017-2018)
Oxfam’s IUWGA project developed and piloted a model for participatory WASH governance between 2015 and 2017 that aimed at building a social contract between local authorities and citizens in New Urban Settlements, based on the concept of collaborative rights and accountability relations. The model was piloted four fast-expanding suburban areas: two in Dadu (Sindh province) and two in Muzafargarh (Punjab province). The model consisted of:
Effective Citizen Groups (ECGs) at the local Mohallah and Union Council levels that organise communities around their citizen rights and responsibilities;
A community-driven planning, prioritisation and resource mobilisation process that enables local authorities to be more responsive to people’s needs; and
A Participatory Monitoring & Evaluation Group (PMEG) at the District level that promotes social accountability for WASH investments while reinforcing existing transparency and accountability laws and regulations.
The PIALA-based evaluation was expected to: (a) assess the effectiveness and likely sustainability of this governance model one year after project closure; and (b) generate useful inputs to developing an analytical framework for Oxfam’s new global thematic area of ‘equitable and sustainable access to water’.
Field inquiries investigated:
How the different stakeholders valued the WASH facilities and services, how satisfied they were with the delivery and the way it was governed, and their level of engagement and sense of ownership.
Emergent changes in relationships within and between the communities and the local government authorities (e.g. the underlying perceptions/attitudes, levels of trust, and power dynamics) as a proxy of the likely sustainability of the governance model.
The PIALA-based design combined a learning-focused quasi-experimental design with a ‘realist’ contribution tracing approach, comparing the WASH governance processes and outcomes in the two project locations with those in a non-project location in each province, to draw lessons about the model’s added value.
PIALA-based evaluation of ‘Promoting Sustainable Building in Bangladesh’ (PSBiB) (2018-2019)
The primary goal of Oxfam's PSBiB project was to mitigate the adverse effects of the burgeoning building industry on the environment, livelihoods, and communities in Bangladesh, thus reinforcing national climate resilience.
Spanning from 2016 to 2019 and funded by EU SWITCH Asia, the initiative was executed in collaboration with the national Housing and Building Research Institute (HBRI), the Jagorani Chakra Foundation (JCF), and the Bangladesh Environmental Lawyers Association (BELA), seeking to facilitate a transition from unsustainable Traditional Bricks (TB) to eco-friendly Alternative Building Blocks (AB) by employing a multi-faceted approach:
Engaging in advocacy and coalition building to foster an enabling policy environment at the national level.
Raising awareness among users and consumers to stimulate the demand for ABs both locally and nationally.
Strengthening the capacity of producers to transition to AB production while effectively managing associated risks at the local level.
The evaluation was expected to assess the project’s effectiveness and relevance, with a focus on policy changes and contributions to climate resilience. Furthermore, its PIALA-based design was expected to contribute to the development of an analytical framework for Oxfam’s global thematic area of ‘climate resilience’, for which the evaluative Theory of Change was framed around the development of three interrelated capacities, namely:
The transformative capacity of relevant national institutions to enable the transition;
The adaptive capacity of the brick industry to adopt new technologies and business models; and
The absorptive capacity of local producers and communities and their environment to cope with stresses and shocks through inclusive supply chain linking and risk management.
