⇓ More from ICTworks

Please Stop Publishing to PDF Graveyards

By Wayan Vota on May 25, 2017

Academic Publications PDF Graveyards

I have a philosophical question for you to ponder:

If a research paper is published, and its hypothesis is proven, but no one acts on it, did the paper even exist?

I ask this question as I’ve seen one too many academic research reports, written by and for scientists, that proves a theory, and can be taken as new scientific fact, yet because its written in obtuse jargon, published to obscure journals, and both the scientist and their organization only count publications, and not impact, that amazing research is lost to history.

Another tombstone in PDF graveyards.

I’m not alone in my question. The World Bank found that more than 31% of policy reports are never downloaded. Almost as bad, almost 87 percent of policy reports were never cited, which questions their worth, even after they were downloaded. And that’s the Bank, which spent $93 million on those unread reports.

For you and your organization, does publication really equal impact? I think not. And I am willing to be crazy and say that a poor farmer, pregnant mother, or caring parent couldn’t care less about yet another peer-reviewed journal article, or its citation.

The Only Thing That Matters is Impact

For our constituents, published reports that are not acted on, are a waste of everyone’s time and money. We shouldn’t think “publish or perish” but “impact or die” for impact is the only thing that matters.

Now impact can be multifaceted, and the occasional report does break through the noise and become central to system-wide change, but it never does that alone, regardless of its brilliance.

Change takes time and effort to spread the key messages in a report, to build champions at all levels of the target ecosystem, and crucially, the dedication and resources to support those champions and their change over the long term. Only then can publication come close to impact.

Don’t Fail the Poor with Your ICT4D Research

Sadly, too many research institutes fail to connect their research to actual impact. They consider impact the work of others, even thinking of it as less important than their scientific discovery. It is not. It is equal and should be considered as such.

Of course, there is already academic research proving that ICT4D academics are failing the poor. We already know that:

  • ICT4D researchers do not engage closely with the users of their research findings.
  • Few researchers engage in advancing policy positions or working with others who might do so.
  • The incentives of the institutions within which researchers work militate against the activities that are deemed necessary for research to have practice and/or policy influence.

Now its our turn to turn research into impact. To pay more attention to how research can be used outside academia, and match emphasis on academic publication with an equal emphasis on constituent impact.

Filed Under: Featured, Thought Leadership
More About: , , , ,

Written by
Wayan Vota co-founded ICTworks. He also co-founded Technology Salon, MERL Tech, ICTforAg, ICT4Djobs, ICT4Drinks, JadedAid, Kurante, OLPC News and a few other things. Opinions expressed here are his own and do not reflect the position of his employer, any of its entities, or any ICTWorks sponsor.
Stay Current with ICTworksGet Regular Updates via Email

11 Comments to “Please Stop Publishing to PDF Graveyards”

  1. Ehud Gelb says:

    These issues have been discussed in detail since 2005 in the free public domain e-Book
    ICT Adoption in Agriculture: Perspectives of Technological Innovation. The paper’s follow experience as it produced “ICT Intermediaries” in a wide variety from agents of change, extension services, private against, public entities, PPPs and many more.with scientists and their results and insights fitting into these and other categories.

    A graveyard does not guarantee oblivion!!

  2. Nicolas Durand says:

    This way-too-often-cited report of 31% of WB pdf never downloaded is sadly based on wrong methodology… They used Omniture (Adobe Analytics) data, which does not track direct downloads, i.e. downloads originating e.g. on Google search. Most systems have this issue (at least google analytics too). To make matters worse, the WB has several document repositories (6 or 7, I think), so if a report is not downloaded in one repo., it doesn’t mean that it’s *never* downloaded. Finally, most reports are also emailed to people of interest, so not downloaded does not equal not read…

    This is not to say that some reports are little read and some organizations over-report, but just to point out a limitation of that methodology and that the 31% is probably overestimated.

    But your question is definitely valid: whose job is it to transform research into action and policy?

    • Wayan Vota says:

      I am sure your comment may bring some comfort to the report writers. I’ve certainly seen the emailing aspect at play with ICTworks posts, which get forwarded well beyond what the website reader stats might show.

  3. Elena Vinogradova says:

    Wayan, great post! The question is why. Private corporations look at the relevant data and policy reports because those can help them improve their profits. Until we are clear on what we are considering “profits” in international development, we will continue to have impact evaluations and policy reports that are not very helpful For example, one large-scale evaluation recently found (and I am quoting here) that “teacher training programs can positively affect learning outcomes in high income countries when they are well-implemented”. This is not very useful for program design or implementation, is it! Clarity and agreement around key outcomes of interest and sufficient granularity with such factors as “works for whom” and “under what conditions” are necessary to make reports relevant and actionable.

  4. Megan Hill says:

    Great posts, including the need for care when estimating download statistics. I’d love to learn more about how other programs are engaging their target audience(s) to go beyond the .pdf. What has worked best for you (especially in global, decentralized contexts)?

  5. Tim Denny says:

    I like the challenge presented in the blog. Whether the methods of coming up with this percentage or that may be correct is of little concern to me. The bottomline is clear – that is that we need to use empirical research more effectively.

    Take for instance national policy in a developing country. I have seen numerous international missions to assist in the process of writing a sector analysis all told maybe 50 people involved over a period of 1 year or more. Papers are written and eventually policy is drafted and approved. It then sits collecting dust. Yet was a requirement to ensure other funds flow cause publishing such documents is a standard.

    In similar light to what Wayan has positioned these countless policies that use great amount of resources to create then are little if at all understood and surely not engaged in full practice, going unnoticed by most and rewritten when they get old it a new adminstration come in contribute to the confusion.

    We need better solutions. What I hope to see come out of this blog is a cheat sheet on C4D to help build capacity on utilization of such documents. As Wayan infers we need to ensure impact. How to do that is the next step.

  6. Jane says:

    Thanks for the continued work here. I cannot express how essential these articles are for my work, but allow me to express my heartfelt gratitude for this particular article.

    It so happens that I had just tabled this case to our editorial committee last week. There wasn’t much enthusiasm to go beyond publishing and look at the impact our research has on the targeted audience-policy makers and colleagues on our client institutions.

    I must admit that this is where it gets complex because no one wants to following up beneficiaries to see whether they actually read and applied the recommendations or whether they even thought about the results at all!!

  7. Tobias Denskus says:

    I also wrote about “pdf graveyards” & paywalls in academic research: Don’t post direct links to your new journal article!
    At the very, very least make your pdfs accessible, summarize them & share them beyond the single document!

  8. Ted Johnson says:

    In a similar vein, we have been thinking of the graveyard of training materials for the beneficiaries of development projects. What happens to local language training videos, posters, how-to documents at the end of a project?

    A training on (for example) hand washing in Hausa ought to be evergreen. What could be done to make sure a Hausa speaker with a feature phone could find that somewhere and reuse it?

    We have some ideas brewing, but nothing yet on our development roadmap.

    • Wayan Vota says:

      Oui! That’s another rabbit hole of waste – all the project deliverables that are re-useable yet no one ever shares them, much less re-uses them unless the donor forces the share and re-use. An extension of the “not invested here” syndrome for sure.