Participatory Impact Assessment & Learning Approach (PIALA)
“In Impact Evaluation, it may be mixed designs (rather than mixed methods) that are most useful (…) to answer the various questions posed by commissioners and other stakeholders.”
Elliot Stern (2015: 13). Impact Evaluation. A Guide for Commissioners and Managers. Bond.
PIALA (Participatory Impact Assessment & Learning Approach) is an approach for rigorous impact evaluation that facilitates evidence-based and collaborative learning with partners and stakeholders around how multiple interventions and influencing factors interact and combine to generate transformative change. It offers an adaptive model for mixing different designs and methodologies to meet multiple learning needs and optimise the value of the impact evaluation for the different stakeholders along the value chain. It can incorporate a wide range of methods – from Conventional Statistics to SenseMaker®, Constituent Voice and Participatory Statistics.
PIALA is an approach that attempts to tackle the twin challenge of impact evaluation in complex contexts:
(i) rigorously assessing systemic change and impact at scale; and (ii) meaningfully engaging stakeholders at all levels. It offers a model for creatively combining old and new traditions and methodologies to rigorously assess, explain and debate system change and impact with stakeholders at a relatively large scale. If applied systematically, it allows for comparitive reviews across contexts and countries. The model consists of:
☐ 5 methodological elements adaptable to context: Systemic Theory of Change, robust multi-stage sampling of/in systems, Participatory Mixed-Methods, Participatory Sense-Making, and Configurational Analysis.
☐ 2 design principles to guide the design: evaluate systemically and enable meaningful participation.
☐ 10 design sliders to make thoughtful design decisions related to: scale, scope, engagement levels, counterfactuals, methods, sampling and analysis.
☐ 3 quality standards to achieve optimal value:
rigour, inclusiveness and feasibility.
This approach is most useful for
Assessing impact in complex contexts where the “causal density” is high, credible control groups are difficult (if not impossible) to find, and a baseline for comparison is not available (or outdated).
Assessing and communicating the contributions of investments (individual as well as combined strategies; project-based as well as institutional) to systemic or transformative impact.
A PIALA process comprises three important phases
PHASE 1 - Focusing and framing.
Builds shared language and understanding of context and causal assumptions regarding system change (using a Systemic Theory of Change) as the basis for determining the scope, scale and focus of the impact inquired.
PHASE 2 - Collecting and linking data.
Designs and employs a mixed (Qual and Quant) or combined (mostly Qual) set of methods and processes suitable for investigating the causal pathways alongside the Theory of Change from a 360° perspective and engaging the downstream stakeholders in a meaningful way.
PHASE 3 - Analysing and debating contributions.
Collates and compares all the data alongside the Theory of Change, and presents it back to the stakeholders for collective sense-making, so that by the time the analysis is done and the final report submitted, everyone already knows and starts using the findings.
Publications & Resources
Mertens. (2009). Transformative Research and Evaluation. Guilford Press.
Pawson. (2013). The science of evaluation: a realist manifesto. SAGE.