ICTworks

With the landmark UN Sustainable Development Summit over, the discussion is now shifting to the question of how to operationalize the 17 ambitious goals in the Agenda 2030. One of the key challenges will be how to measure progress, and to create clarity around who should be responsible to whom and for what. Projects like The World We Want There and others have been driving a growing consensus that sustainable development should be genuinely accountable to the proposed beneficiaries. In other words, people should be participants in the processes and decisions that determine their future. But how can donor organizations put this in practice?

A great way to better include beneficiaries of project interventions in monitoring and evaluation of projects in sustainable development is by using digital tools. At its core, monitoring and evaluation means working with data. The work of innovators like the Global Partnership for Development Data and the DataShift project have highlighted the usability of digital tools to crowdsource data collection in development contexts at scale, and to close the feedback loop to beneficiaries by relaying information about development interventions back to them.

The rapid spread of mobile phones throughout developing countries in the last years has opened up new channels of communication between communities, governments, and donor agencies. No other technology has been in the hands of so many people in so many countries in such a short amount of time, and with almost 7 billion mobile cellular connections today more people have access to mobile phones than to clean toilets.

Here are 7 tips that can help aid practitioners to achieve better results in mobile monitoring and evaluation (M&E) projects:

1. Involve Everybody Avoid siloing mobile M&E as a function into a specific team. Instead, mainstream the mobile data and engagement component throughout the project cycle and all teams, including proposal teams, implementation teams, M&E teams, and tech teams. At the very minimum, everybody involved in the project needs to be able to articulate the theory of change, and how the mobile component allows measuring that theory of change.

2. Simple Technology Smartphones can capture rich data like photos or GIS coordinates. However, smartphones are also still relatively pricey, and to work properly they require expensive data plans. That’s why the vast majority of people in developing countries continues to use simple feature phones. The best choice for mobile M&E in these contexts are text messages (SMS) or Integrated Voice Response (IVR) that work reliably on any kind of mobile phones. All mobile phone users will be familiar with at least one of these protocols, and they do not require additional actions from users like downloading an app.

3. Free to Use and Incentives Most target communities are poor and cannot afford extra charges on their mobile phone. Services or surveys that cost users airtime or SMS charges will not be widely used. Mobile M&E projects need to include reverse billing agreements with MNOs to ensure that any charges incurred are paid for by the development agency. In addition, the use of incentives to facilitate higher participation and questionnaire completion rates should be considered. The incentives can be rewards like airtime or micro-transfers into mobile money accounts.

4. Match Literacy and Languages Mobile M&E projects need to adapt to the literacy rates in target populations. Voice solutions (IVR) are useful in cases where a majority of the target population possesses only sufficient numeracy skills to operate a phone dial, while SMS and USSD are more suited for target populations with functional literacy. In areas in which various languages are spoken, mobile M&E flows need to be translated into all pertinent languages, and include an option for the user to choose the most suitable language at the very beginning.

5. KISS: Keep It Short and Simple Experience shows participation rates in mobile surveys drop dramatically after five questions. In order to keep attrition rates low mobile M&E projects need to reduce the complexity of traditional M&E questionnaires. One possible approach is to look for proxies: using simpler variables that allow inferring the value of a more complex variable. Good examples for this approach include the Progress out of Poverty Index (PPI) to measure poverty levels in groups and individuals, and work of Kieren Moffat to boil down the complex “social license to operate” indicator that is important in extractive resource governance. Another possible approach is to break down larger questionnaires into shorter sets of questions that are sent out to target populations in multiple data collection cycles, and to collate data for analysis once the whole questionnaire has been answered by a sufficient number of people.

6. Check Early For Bad Data Bad data is unavoidable, but can often be remedied by making small changes in questions, outreach channels, or analysis tools. Budget extra time to check raw data in the first iterations of data collection to check for duplicates, obviously wrong entries, and other potential issues. Consider data validity spot checks, where a team checks if the reported data matches observations in the field.

7. Opt-in and Opt-out Most countries have privacy laws that regulate how individuals can be contacted by commercial or non-governmental organizations. Non-compliance with these regulations can lead to fines even if individuals were contacted for non-commercial purposes. On a minimum level, mobile M&E projects need to document user consent to participation (opt-in), and ensure that users are given the opportunity to stop receiving messages at any time (opt-out).

Written by Cornelius Graubner, the co-founder at Ulula, a New York-based startup that provides digital data and engagement solutions for sustainable development. He has worked with donor organizations, governments, communities, and private businesses on social and economic development issues in 15+ countries in Asia and Europe. He tweets at @cgraubner about sustainability, digital data, ESG risks and extractive resource governance, and is happy to connect on LinkedIn.