Back in 2014, I was excited to help lead the adoption efforts for the Principles for Digital Development, nine living guidelines that are designed to help integrate best practices into technology-enabled programs.
Fast forward four years and I am excited for the Principles yet again. This time it is to announce what I believe is a first in our industry: using the Digital Principles to evaluate a major ICT4D project.
Evaluating iHRIS Using the Digital Principles
IntraHealth International released the first version of iHRIS health workforce information software in 2007, and since then at least 27 countries have adapted the open-source software and are using it to gather and analyze data on their human resources for health.
I assumed leadership of the Digital Health team at IntraHealth two years ago, and from the start, I wanted to honestly evaluate iHRIS using the Digital Principles, to:
- Find out our strengths and weaknesses
- Inform how we improve the software
- Share the results publicly, per our open development process
Late last year, we contracted with NPOKI, a consulting group, to interview focus groups of stakeholders who provided feedback and determined the extent to which the suite of iHRIS products adheres to the Principles.
Join us on April 16 to learn more about the evaluation process
Digital Principles Evaluation Process
NPOKI used an Information Audit to evaluate iHRIS against the Principles for Digital Development, The information audit is an interview process that included:
- Capturing information about the stakeholder’s background and iHRIS experience.
- Asking questions developed from the PDD framework.
- Scoring the answers using a Likert scale of 1 to 5.
In addition to the quantitative responses, the team also collected comments and insights into the reason for a respondent’s answer. This qualitative data was important to the team’s understanding of the thought process and the considerations of the respondents.
iHRIS-PDD Evaluation Results
For each of the nine principles, iHRIS was assessed on a scale of 1 to 5. Those scores ranged from a low of 3.51 for the principle of using “open standards, open data, open source, and open innovation” to a high of 4.28 for “be data-driven.” The average score was 3.88, or a solid B+.
Open Principle: The 3.51 score on the Open Principle was surprising since iHRIS is a good example of an Open Source and Open Standards application. I believe that the general confusion around what “open” translates to for standards, data, source, and innovation in general, and in iHRIS, kept this score low.
Be Data Driven Principle: While in theory, each of the Principles has equal weight, the Be Data Driven Principle has perhaps the most importance for digital health. iHRIS’ 4.28 score on that Principle is both notable and indicative of our focus as the shepherds of the iHRIS software development process.
The full findings and recommendations are a refreshing validation of iHRIS develop processes from the start of the program in 2007 through to today. This report affirms that we were living the ideals behind the Principles at IntraHealth even before we endorsed them.
Overall, iHRIS is perceived as providing great value to the health community. Its users extolled its flexibility and features and acknowledged its loyal community and the niche it fulfills. It is clearly a success story for IntraHealth and a model for other NGOs considering moving into the open source application development space. Now we are actively seeking to improve our score with the support of the iHRIS community.
I hope that other organizations will use a similar process to evaluate their own projects and ICT4D initiatives using the Digital Principles.
Join us on April 16 to learn how you can use Digital Principles to evaluate ICT4D efforts
In early 2017, CRS evaluated one of our major ICT4D projects against the Principles and submitted a case study of our findings – https://forum.digitalprinciples.org/t/case-study-from-catholic-relief-services-crs-posted-implementing-the-principles-for-digital-development-the-remind-project-reducing-maternal-and-newborn-deaths/133
We have not repeated this for other projects since.
In early days of the Principles for Digital Development, I recall a requirement of any organization endorsing the Principles was to submit two case studies each year about how well (or not) their work was adhering to the Principles. This requirement is now described as a recommendation (with a case study template)
at https://digitalprinciples.org/endorse/
I am curious if others are using this template and submitting case studies? If so, where are they being catalogued?