FORWARD-LOOKING EVALUATION OF THE POTENTIAL IMPACT OF INTERNET RESEARCH

Assessing the influence of ISOC Foundation’s Research Programme on Internet inclusion, decolonisation, trustworthiness and greening from an ‘Alternative Futures’ perspective

Client
Internet Society (ISOC) Foundation

Period
2023-2024

Services
Evaluation

Location
Global

Areas
Research to Policy and Practice,
Climate Resilience

From December 2023 to May 2024, Collaborative Impact worked with the Internet Society (ISOC) Foundation to evaluate its Research Programme and generate evidence-based insights into its effectiveness, value-addition, and influence on shaping the desired Internet of the future. The findings and recommendations from the evaluation were anticipated to inform and guide the Foundation’s priorities and design for the next strategic period and programme phase. 

Launched in 2020, the Research Program is one of the ISOC Foundation’s first and biggest programmes. Between 2020 and 2023, it has funded research projects for a total of USD 9.6 million in grants in four thematic areas:

  • Greening - Making the Internet environmentally sustainable and resilient.   

  • Decolonisation - Making access to the Internet inclusive of all (in particular non-Western) peoples, cultures, and languages.  

  • Trustworthiness - Making access to and use of the Internet reliable and safe for everyone.

  • Economy - Making the Internet economy trustworthy, inclusive and fair for everyone.

The evaluation objectives, formulated and agreed with the programme management and MEL team through the co-design process, were to: 

  • Assess the relative effectiveness and value-addition of the thematic grantmaking and learning approach within the broader digitalisation research landscape.

  • Generate evidence and learning on the reach and influence of the research and collaboration efforts at individual grant and programme levels on the broader Internet research community and on (private and public) internet policy and decision makers.

  • Generate evidence-based insights and recommendations for focusing and improving the grantmaking and learning approach towards achieving the ISOC Foundation’s vision —an open, democratic, connected, inclusive, secure, and trustworthy Internet that is for everyone and a force for good for people and the planet.

Collaborative Impact’s Participatory Impact Assessment and Learning Approach (PIALA) was used as a framework for co-designing and conducting the evaluation together with programme management and MEL staff. Scoping, design, synchronising and sensemaking workshop sessions combined with ongoing informal exchanges on Slack and email allowed the evaluation to flexibly respond to emerging insights and needs and combine a (backward-looking) evaluative perspective with a (forward-looking) strategic foresight lens.

The evaluation methodology combined:

  • Ecosystem network mapping based on a desk review, web-based search, and automated (with python scripts) content analysis of grantee documents

  • Rapid outcome-harvesting surveys with ecosystem actors (incl.: grantees, technologists, researchers, activists, and policy makers) to understand the extent to which the Research Programme was emerging as a ‘mover and shaker’ in the sector, and to collect insights on the influence of the research in the four thematic areas.

  • Trends analysis and cross-thematic Three Horizons Analysis dialogues and interviews with ecosystem actors, using an ‘Alternative Futures’ perspective.

As dominant current trends and emerging technologies rapidly change the social, economic, environmental, and geopolitical landscape around us, the future of the Internet stands at a crossroads. Internet research has a critical role to play in helping achieve a more positive future. The Three Horizon Analysis dialogues and interviews introduced this notion and presented participants with two opposite scenarios —a positive vision of the future, and a nightmare vision— to prompt future-oriented thinking as the basis for identifying research priorities and strategies to enhance the potential influence and impact of research towards a desirable positive future. In the Sensemaking Workshop, these were further discussed, narrowed, and turned into recommendations.