ICTworks

There is enormous interest and investment in the potential of educational technology (ICT4Edu) to improve the quality of teaching and learning in low and lower-middle income countries. The primary aim of the DfID-funded Educational Technology Topic Guide is to contribute to what we know about the relationship between edtech and educational outcomes.

Subscribe Now for More ICT4Edu Insights!

Taking evidence from over 80 studies, the guide addresses the overarching question: What is the evidence that the use of ICT4Edu, by teachers or students, impacts teaching and learning practices, or learning outcomes? It also offers recommendations to support advisors to strengthen the design, implementation and evaluation of programmes that use edtech.

Educational technology was defined as the use of digital or electronic technologies and materials to support teaching and learning. Recognising that technology alone does not enhance learning, evaluations must also consider how programmes are designed and implemented, how teachers are supported, how communities are developed and how outcomes are measured.

And while the Millennium Development Goals prompted improvements in access to education, quality remains a challenge. This issue is also reflected in edtech programmes.

Research reports of programmes that move beyond access to technology (both in programme design and evaluation) are emerging, but as yet relatively few programme evaluations focus on adequately capturing improvements in the teaching and learning process or measuring improvements in learning outcomes or digital literacy. The findings below are drawn from those that do.

Educational Technology for Students

Among the studies reviewed, the strongest evidence of changes in learning outcomes and classroom practice came from the use of mobile devices (such as eReaders) and CAL programmes to support improvement in mathematics:

• eReaders and tablets to support early literacy.
  Several programmes presented evidence of improved learning outcomes (in terms of increased reading fluency in the mother tongue or English) that combined provision of eReaders and eBooks for students with TD programmes on phonics-based literacy instruction (Worldreader, 2012, 2013; Murz, 2011; USAID, 2013; PRIMR, 2013).
• Remedial CAL programmes in mathematics.
  Although CAL programmes in maths as a replacement for regular teaching were found to have limited impact (Banerjee et al., 2007, p. 1,240) or lower learning outcomes (Linden, 2008, p. 26), there is some evidence of improved learning outcomes from remedial CAL programmes as supplements for under-privileged students (Banerjee et al., 2007, p. 1,238) or under-performing students (Lai et al., 2011).

In addition, several studies presented evidence of students working more independently and collaboratively using online or offline digital resources to support project work. This was usually in the context of a teacher development programme, with clear curricular and pedagogic focus (for example: Light, 2009; Were et al., 2009; Leach et al., 2005).

Educational Technology for Teachers

The following uses of edtech by teachers were associated with positive changes in learning outcomes and classroom practice:

• Interactive radio instruction (IRI).
  Several studies reported positive impacts on learning outcomes from IRI, particularly with early primary students. A World Bank review showed average effect sizes of +0.5 (World Bank, 2005, p. 16), while a subsequent review found effect sizes ranging from -0.16 to +2.19 (Ho & Thrukal, 2009). The variability in effectiveness was attributed to factors including quality of programme implementation, monitoring, and local human resources. The greatest effect sizes were seen at Grade 1, suggesting IRI is particularly effective for early primary years.Improvements in classroom practice from IRI were evidenced by two studies in which IRI was used in the context of teacher professional development.Sous le Fromager in Guinea supplemented IRI with radio programmes for school staff and face-to-face professional development to instill respectful behaviour of teachers towards students. Qualitative classroom observations suggest teachers hit students less often and allowed more time for students to develop understanding (Burns, 2006, p. 9). Similarly, an IRI programme in Mali supplemented IRI with radio-based, in-service training. Systematic classroom observations showed year- on-year improvements in the percentage of observed lessons demonstrating select classroom practices (e.g. brainstorming, group work, total physical response) (Ho & Thrukal, 2009, p. 10).
• Mobiles for classroom audio and teacher development videos.
  Several studies arising from one programme (English in Action [EIA], Bangladesh) reported positive impacts from mobile use on English language teaching (ELT) practices and student learning outcomes. EIA is primarily a teacher development (TD) programme. The approach has some similarities to IRI, in that mobile phones provide access to audio resources for classroom use, particularly for primary teachers. Mobiles are also used to provide access to TD materials, including videos of classroom practice, which underpin the programme. Materials are not broadcast, or downloaded, but provided as a library of digital resources, on a small memory card.Several large-scale systematic observations of classroom practice (EIA, 2011b, 2012b, 2014b) showed significant increases in students’ talk time (including talk in pairs and groups), and students’ and teachers’ use of English (the target language), compared with baseline studies (EIA, 2009). Associated improvements in student learning outcomes were also evidenced, most recently with 35% more primary students achieving Grade 1 or above, and 20% more primary achieving Grade 2 or above, on recognized international frameworks of English language competence (Graded Examinations in Spoken English (GESE), Trinity College London, 2013, which map onto the Common European Framework of Reference, Trinity College London, 2007, EIA, 2014b).
• Mobiles for classroom video.
  The BridgeIT programme (India and Tanzania) provided evidence of improved learning outcomes from teachers’ use of smartphones to play video lessons for their classes via flat-screen TVs or data-projectors. Teachers also had activity guides to support or extend the video lessons.In Tanzania, students showed average gains of 10–20% over control groups for maths and science.However, while some groups of students excelled, others showed modest gains if any (Enge, 2011). In India, there were average gains of 10% over control groups for science, but no gains for English (Wennerstan & Qureshy, 2012). BridgeIT also carried out systematic classroom observations pre- and post-intervention in India. These showed a 31% increase in the proportion of lessons identified as ‘high quality’, with a corresponding 24% drop in the proportion of (traditional) ‘direct instruction’ lessons (Wennerstan & Qureshy, 2012, p. 32).

In the context of enormous global challenges to improve the quality of education, particularly in low to lower- middle income countries, governments, donors, schools and communities often seek to explore or exploit the potential of edtech. The studies reviewed for this guide provided some compelling examples of evidence that this potential can be realized, to produce educationally significant impacts on practice and outcomes.

In particular, there is some evidence that mobile technologies (radios, mobile phones, and tablets) – used for curriculum-specific purposes in a context of appropriate support – can be particularly effective. There is also tentative evidence that such approaches may contribute to addressing issues of equity, in relation to gender and rurality.

Overall, effective educational technology programmes are characterised by a clear and specific curriculum focus, the use of relevant curriculum materials, a focus on teacher development and pedagogy, and evaluation mechanisms that go beyond outputs.

4 Recommendations

1. Edtech programmes should focus on enabling educational change, not delivering technology. In doing so, programmes should provide adequate support for teachers and aim to capture changes in teaching practice and learning outcomes in evaluation.
2. Advisors should support proposals that further develop successful practices or that address gaps in evidence and understanding.
3. Advisors should discourage proposals that have an emphasis on technology over education, weak programmatic support or poor evaluation.
4. In design and evaluation, value-for-money metrics and cost-effectiveness analyses should be carried out.