The aim of the SMS Story research project was to determine if daily mobile phone text message stories and lesson plans would improve children’s reading in Papua New Guinea (PNG) elementary schools. The research was a controlled trial in which half of the teachers received text messages for twenty weeks and half did not.
The stories and lesson plans were designed to introduce children to reading English and followed an underlying phonics and key word based methodology. Teachers in the trial received a cartoon poster explaining how to use the daily text messages and received a total of 100 text message stories and 100 related text message lessons for two academic terms. They did not receive any in-service training.
Baseline Start Point
Research was conducted in rural elementary schools in two provinces, Madang and Simbu, and has involved a baseline reading assessment, mid-point lesson and classroom observations and an end-point reading assessment. At baseline, there was no statistically significant difference between the active and control groups, with respect to school characteristics and children’s reading assessment results.
The baseline results showed that many children had limited or no reading (for example, half of the children could not read any high frequency English words). At the time of enrollment, all participating schools had very few reading books, if any, available in the classroom.
Experiment Results
On average, across both sets of schools, children’s reading did improve over the two terms with children at SMS Story schools improving significantly more. Random visits to active schools during the intervention period showed that most teachers were actively engaging with the content sent to them as text messages. This demonstrated that the SMS technology (using FrontlineSMS delivered over the Digicel mobile network) was effective in reaching teachers.
There was a large change in the reported use of teaching strategies promoted by SMS Story lesson plans and poster (for example, 42 teachers in active schools (n=51) against 12 teachers in control schools (n=51) reported “reading stories to the children every day”).
At the end-point reading assessment, there was a statistically significant difference between the results of the control and active groups, with the active group performing better than the control group across four of the five reading skills tested. This improvement is seen in both grade 1 and grade 2 and with girls and boys.
Children who did not receive the SMS Story were approximately twice as likely to be unable to read a single word of three sub – tests (decodable words, sight words and oral reading). In other words the intervention almost halved the number of children who could not read anything compared with the control schools.
Therefore, the text messages to teachers improved students’ reading ability in decoding, fluency, reading familiar high frequency words and reading phonetically correct nonsense words. The research did not find a statistically significant improvement in reading comprehension and generally children showed low reading comprehension skills in both grades and little progression between grade 1 and 2.
Other Results
The trial also found a strong negative impact on students’ reading caused by the absence of the classroom teacher to attend provincial trainer-directed training. Unsurprisingly the students of these teachers performed poorly on the final reading assessment. Importantly SMS Story does not require a teacher to be absent from a class for training.
Recommendations
As a control led trial, this intervention has a rigorous research base. The results demonstrate that appropriate use of mobile phone technology can have a positive impact upon educational outcomes in resource-constrained settings.
In PNG, it is recommended that the methodology of sending daily text messages to teachers be pursued further. In other countries, it is recommended that trials be undertaken as controlled trials so that statistically significant data can be generated.
SMS Story was funded by the Australian Government, through a research grant from the Economic and Public Sector Program. The project was designed and managed by Voluntary Services Overseas, in partnership with the Department of Education.
Thanks for sharing this report. This research seems solid. My impression is that SMS here is only a mean of transmitting information. So the key aspect is the content of the messages. That’s the real added value and that’s what’s evaluated after all.
Interesting example of a program where both the content and the delivery might be important, therefore it would have been helpful to compare this to delivering the lesson content in another way – eg providing by mail at the start of term or phone calll, or some other reasonable alternative.