ICTworks

Digital health interventions are often very dynamic, evolving through several stages of maturity during which the monitoring and evaluation needs of the intervention also change rapidly. These digital health intervention stages include:

• Exploring the identified needs of constituents
• Developing the technical functionality and feasibility
• Deploying the technology solution with stakeholders
• Assessing user satisfaction with the deployed technology
• Evaluating the intervention’s effectiveness and attributable impact
• Calculating the key stakeholders’ “value for money”.

The new World Health Organization Practical Guide to Monitoring and Evaluating Digital Health Interventions can help you improve the quality and value of your monitoring and evaluation efforts when deploying digital health solutions.

The Guide can help you navigate through the development of you entire M&E process. However, in this post, we are going to focus on how to determine the incremental cost–effectiveness of the digital health intervention as compared to existing services.

Join Your Digital Health Peers at 2019 Global Digital Health Forum

Economic and Financial Evaluations

Financial evaluations deal with the questions of whether the organization and digital health users can afford the digital health system, and how it will be financed. Economic evaluations aim to determine a probable value for money from an investment.

Your economic evaluation identifies what you should do; your financial evaluation shows if you can pay for it, and how you will pay for it.

Economic Evaluations

Economic evaluations allow you to determine whether your project is good value for money, by comparing two or more alternatives in terms of their inputs and their effects on a common outcome or impact.

Economic evaluations are generally undertaken using one of five main methodologies:

• Cost–effectiveness analysis (CEA)
• Cost–utility analysis (CUA)
• Cost–benefit analysis (CBA)
• Cost–consequence analysis (CCA)
• Cost-minimization analysis (CMA).

Table 4.8 outlines the characteristics of these five different methodologies. In this Guide, we focus on the three most common forms of economic evaluations, CEA, CUA and CBA, which are listed first in the table.

1. Cost–effectiveness analysis (CEA)

This estimates the difference between the costs of your resources and the effectiveness of their use. Cost–effectiveness analysis is the most commonly used form of evaluation in health economics. Effectiveness is most frequently estimated by a single measure, such as the number of lives saved, or episodes of illness averted.

2. Cost–utility analysis (CUA)

This considers people’s quality of life and the length of life they will gain as a result of an intervention. Cost–utility analysis aims to overcome the one-dimensional limitations of consequence in a CEA by using utility-based outcome units to compare different interventions.

Quality-adjusted life years (QALYs) and disability-adjusted life years (DALYs) are two types of summary measures that can be used to estimate cardinal values assigned to health states. They are both based on a scale of 0 to 1, but have inverted interpretations: whereas a score of “1” represents “perfect health” for QALYs, it represents “death” for DALYs.

Use of these utility measures allows you to capture more comprehensively the full state of health and well-being. Moreover, these measures allow you to compare your digital health intervention not just to the comparator of your intervention (e.g. the status quo/usual care), but to other uses of your resources across the health sector, such as a comparison between your digital health SMS programme for pregnancy care and a tuberculosis control programme, or an HIV programme.

You may find that for your initial economic evaluations, you won’t have access to the health data needed to calculate QALYs or DALYs. You’ll need medical advice if you want to use QALYs and DALYs in your digital health evaluation.

3. Cost–benefit analysis (CBA)

This measures estimated costs and benefits over time in monetary values for all types of stakeholders involved in your intervention. Cost–benefit analysis values come from the prices and costs of components that have market values, and from estimated monetary values for intangible components that do not have market values, using willingness-to-pay (WTP) techniques.

Because of this assignment of monetary value to consequences, cost–benefit analysis is arguably the most controversial form of economic evaluation. Where benefits are measured in terms of lives saved, assigning a monetary value to these lives can elicit strong reactions.

Many digital health programmes don’t directly save lives – they aim to improve health care – and therefore life valuations are seldom relevant for these. Many analysts consider CBAs to be “the methodological foundation for turning theory into a pragmatic evaluation tool”.

The eHealth Impact methodology is a version of CBA applied to digital health evaluations. Cost–benefit analysis is relevant when programme managers and the broader group of decision-makers want to compare intervention results to other projects that are competing for your organization’s scarce resources.

Financial Evaluations

Financial evaluation has two goals. The first is to determine the affordability of your preferred digital health intervention by comparing net costs with cash flows over time, which is known as budget impact analysis. The second goal is to develop sustainable financing plans to ensure that your project has ongoing resources available to pay for it.

Viable financing and affordability are essential for delivering and sustaining the value for money identified by your economic evaluation, so the financial evaluation should follow on from your economic evaluation. The United Kingdom’s Green Book methodology provides guidance on both economic and financial evaluations.

Seriously, if you read this far, you should join us at 2019 Global Digital Health Forum