Way back in 2001, Satellife pioneered the use of PDA’s for health-related data collection in Uganda and Kenya. In the 15 years since, Satellife became the TechLab at FHI 360 and mobile data collection has become routine and easy with Open Data Kit, countless derivatives, and a whole plethora of companies dedicated to delivering seamless mobile data collection as a service.
Yet, there are still people advocating for paper-based surveys and projects that actually send out enumerators with questionnaires and clipboards. Why?
Here are five reasons why you are wasting everyone’s time if you are still collecting data on paper.
1. Accuracy
When humans fill out paper forms, they are bound to introduce simple errors, such as misspellings and numeric transpositions, that a digital form can look for and correct automatically. Electronic forms can also require specific formats, verify answers, and even add in geographic locations well beyond humans’ native abilities.
2. Speed
Paper-based forms require the paper form to be physically moved to a data input station, then input into data systems manually. This will add hours or even days to the data collection process. In addition, all data validation and aggregation has to wait on the paper form submission, preventing anything near real-time analysis of the data.
3. Feedback
Participants rarely see any feedback on their data collection efforts with paper forms, as the data collection and analysis process is too long to get any meaningful analysis back to participants in time for it to be relevant. However, mobile data collection can instantly showcase participant’s impact and their status relevant to others in the process. This includes those giving data, those collecting it, and key actors all along the decision matrix.
4. Inevitability
Whatever is collected using paper-based systems will be digitized eventually, so it might as well start digital, and save the countless hours and efforts of an analog effort. Quality digital data collection systems will usually be faster and cheaper than paper – even when factoring in the mobile tools needed to collect the data. And this has been true since 2006 – a decade ago!
5. Planning
Finally, analog data collection is often done with minimal planning, creating multiple issues during and after data collection. Yet digital data collection can force a level of planning that doesn’t happen with paper collection, increasing the overall data collection effectiveness of an organization for that survey, and all future ones.
Still Undecided?
Check out the Mobile-based Technology for Monitoring & Evaluation reference guide on the benefits of real-time data sharing and data analysis. The Guide has three informative sections:
- Quick Start: Is mobile-based monitoring and evaluation for you? How to get started?
- Mobile Technology: How is mobile technology better than paper survey? What are your choices?
- Implementing Mobile Technology in M&E: How to roll out a mobile system? What are best practices, dos and don’ts?
The five reasons were collected electronically on Facebook – an awesome survey tool few use effectively.
Thanks Wayan for the article. It is really interesting.
For quantitative data collection, yes, I agree using mobile is the best and easiest way to use. However, data collection is not only restricted to collecting quantitative data but there is also qualitative data which involves Focus Group Discussion/KIIs..etc . Will mobiles be the good option for collecting such qualitative data or you suggest different and a better way?
Looking forward to hearing from you
Best, Omar
hi wayan, great read!
i certainly agree that digital forms are better than paper in general. They are cheaper, faster, and more accurate than their antiquated paper based competition. That said, let’s acknowledge that some organizations have legitimate reasons for neglecting to adopt digital data capture.
For one, digital forms are always cheaper in the long run, but the start up costs are generally higher and often prohibitive for a one-off study or a tiny capital-constrained NGO. You need to spend staff time to identify a system, build your forms with that system, and develop training materials, then purchase devices, train, and roll out. Meanwhile, anyone with Microsoft office can create and print a paper-based form in a matter of minutes, and training on paper is often more intuitive than with digital gizmos.
In the long-run with digital capture, you must also continually replace broken devices, update the software (whos still using ODK v1.0?) and retrain new employees–all details new entrants into digital data capture often forget at onset. This entire process takes some expertise, so many opt to hire consultants (e.g. me) to help budget, build and deploy the tool, which is an extra cost they wouldnt have on paper forms.
Finally, *sometimes*, you find digital just doesnt make sense for the context. If you want to refer a patient to a health facility outside your program, for example, how do you monitor that the referral is completed? You can ask the facility to enter patient data electronically, but this is voluntary and you way to ensure compliance. It might make more sense just to call, or give the patient a paper slip.
All this isnt to say “DigitalBAD! / PaperGOOD!” Electronic really is inevitable, just like elimating extreme poverty is inevitable. Lets just recognize there are no absolutes in this field!
Brian,
The great thing about digital forms is that they make you think about the survey process in a way that printing out a few forms from MS Office does not, usually resulting in a survey design that is waaaaay better than the paper version. However, if you need quick, Google Forms are just as fast as MS office, and you get all the digital data benefits.
As to keeping survey tools updated, don’t orgs invest in keeping *all* their digital tools current, or outsource to consultants if their digital needs are not reoccurring enough? How should survey design be any different than printer management?
Finally, of course you can find/create exceptions where paper is better. I miss Polaroids as a practical gift to the data giver – at least they get a photo of themselves during the data extraction process. That doesn’t make them, or paper, practical in 2017.
So yes, overall, paper really is bad and digital is good for surveys.
From Twitter, Patty Mechael says, “I feel like I have been making this argument for years & find it strange that people still ask the Q.”
That’s why I wrote this post. I was asked just recently why we should abandon a long-held paper-based survey process for a digital version. Thanks to all those on Facebook who helped me build this post.
Wayan: great to see you taking on the “Paper Lobby”, we like to call it! As you point out, organizations have costs associated with every management activity, yet digital M&E is often treated as an extra cost outside of the operations or overhead budget. Too many development organizations have been conditioned to think of it as “innovation”, rather than a cost of doing business in the modern world.
The sad truth of the development industry is that most organizations are not using data to make decisions. They go through the motions of collecting data in forms, but too often they end up in a filing cabinet or a mess of excel sheets. Compared to the cost of actually analyzing and taking action on this data, the paper system will seem more cost-effective. But compared to the cost of making poor program decisions because of a lack of data, going digital looks like a bargain.
Ugh. Enough with anyone thinking mobile data collection is “innovation” – its now a commoditized activity. You can’t swing a tablet without hitting multiple organizations that focus on mobile data collection as a core offering, from FHI 360 to your organization. That means it should be incorporated into budgets just like any other program infrastructure.
Your second point is the larger elephant in the room with mobile data collection. The cost of bad decisions far, far outweighs data collection costs – paper or digital!
Hi Wayan,
Thank you for this thought-provoking article and your work advancing the field. As someone who works for a company that digitizes data on paper (Captricity), I suppose I am part of what one of your commenters referred to as the “paper lobby.” Even so, I fully agree with the underlying assumptions of your article: digital data collection has many advantages over data collected on paper and digital data from start to finish is ideal. However, I think the reasons you crowd-sourced on facebook are not universal. Context is everything.
Captricity’s customers, including small NGOs as well as large US Corporations, would also agree with your assumptions, but all have good reasons why they retain paper data collection.
Sometimes it’s advantageous to pass out a survey instrument to many individuals in a clinic, classroom, or event. You wouldn’t do this with an expensive device. Paper works offline, never runs out of batteries, and workers know how to respond when a pen malfunctions. Some researchers also report to us paper gives them higher response rates, more thoughtful responses, and more intimacy and trust between subjects and researchers discussing sensitive issues.
The biggest challenge with paper is the slow speed, poor accuracy, and high expense of data entry. Our technology addresses all of these challenges. It’s obvious that a fully digital workstream is preferable, but often there are good reasons why this isn’t the best solution in every context.
Best,
Jacob
Oh, wow, I like the idea of a person filling out their own response, in longhand, being more intimate than typing it into a screen or telling it to someone else to type into a screen. I had not thought of that angle.