ICTworks

As countries around the world have closed learning institutions in response to the COVID-19 pandemic, teaching and learning have pivoted from in-person instruction to distance learning.

Simultaneously, there has been a surge in efforts to promote access to distance learning programming. Distance learning is commonly used to reach learners who need flexible learning opportunities, as well as to reach groups excluded from formal education institutions. It can serve as the main form of instruction or can complement or supplement in-person learning.

As countries and education agencies take up distance learning for COVID-19 digital response, it is important to design and implement evidence-based strategies for monitoring and evaluation to measure whether distance learning efforts are serving the intended communities and achieving intended objectives.

A Roadmap for Measuring Distance Learning

The purpose of A Roadmap for Measuring Distance Learning, a review of evidence and emerging best practices from USAID, is to support education practitioners, host country government representatives, donors, implementers, non-governmental organizations (NGOs), civil society organizations, and other stakeholders in applying best practices to monitor and evaluate distance learning initiatives designed for diverse learners and implemented both within and outside of learning institutions.

This review covers the four key distance learning modalities: radio/audio, television/video, mobile phone, and online learning. Printed texts, which are often developed to accompany these first four modalities, can also be a fifth modality in contexts where technology is not used.

Most of the data sources were drawn from work in the primary education sub-sector. However, much of the guidance can be applied to secondary and tertiary-level distance learning. This review is also applicable to data collection in both crisis and non-crisis contexts.

A Roadmap for Measuring Distance Learning presents guidance for users to go through four steps of planning and designing how distance learning delivered through any of these modalities can be monitored and evaluated.

• Step 1: Determine the Objectives of Monitoring and Evaluating Distance Learning
• Step 2: Determine What Will Be Measured (Reach, Engagement, and Outcomes)
• Step 3: Determine How Data Will Be Collected (In-Person or Remotely)
• Step 4: Determine the Methods and Approaches for Measurement

Based on emerging global evidence, this review guides users through the process of measuring the reach, engagement, and outcomes of distance learning initiatives.

In addition to providing step-by-step guidance, this review provides three overarching recommendations for developing and implementing evidence-based monitoring, evaluation, and learning (MEL) plans for distance learning initiatives. The recommendations are to:

1. Integrate in-person and remote approaches, use multi-modal interfaces, and employ mixed methods to measure distance learning. Integrating in-person and remote data collection (e.g., in-person testing and remote interviews), using multi-modal interfaces (e.g., phone calls and SMS surveys), and mixed-methods approaches (e.g., interviews, surveys, and photographs) helps promote greater participation and leads to more accurate results.
2. Encourage innovative solutions to measure reach, engagement, and outcomes during a quick pivot to distance learning, while also developing high-quality MEL strategies for the longer term. The guidance in this review helps teams think about short- term MEL needs while working toward longer-term strategies for assessing the effectiveness of distance learning.
3. Design equitable monitoring and evaluation approaches and conduct systematic equity analyses of distance learning initiatives. Evaluative approaches to distance learning must attempt to measure and analyze whether marginalized individuals and groups are being systematically included or excluded through distance learning programming as well as during in- person and remote data collection.

This review complements other practical How To Guides produced by USAID to support high-quality distance learning program development and delivery. A complete listing of these documents may be found on USAID’s COVID-19 resources page on Education Links. These include:

• Delivering Distance Learning in Emergencies: A Review of Evidence and Best Practice
• Return to Learning During Crises: Decision-making and Planning Tools Toolkit
• USAID’s Guidance Note on Measuring Reach for Distance Learning

As with all of USAID’s resources, this review is not specific to any one context, event, or distance learning initiative. It is meant to be useful to users both in responding to short-term learning needs during the COVID-19 pandemic and long-term distance learning strategies.