ICTworks

We all live in a world where we need to have good quality data, quickly and (ideally) inexpensively, for better decision making. And to share that with partners to build collaboration and transparency and make data open as a reusable public good. Of course, we should also protect the privacy of this data, especially when it is sensitive (like with personal health information or on sensitive topics). That’s why all of us in digital development have heard at least one of these sentences before:

• “Wouldn’t it be great if we could have real-time data on this topic?”
• “Data security? Oh, you don’t want to talk to program or M&E people, you need to talk to the IT department.”
• “So, I asked my guy at the ministry if he could share a copy of that dataset with me and he emailed to it me today…”

Therefore, the real question we need to ask ourselves is: How can we support the responsible management of data to protect privacy as well as promote reuse when appropriate?

USAID articulated the broad ideas in the Considerations for Responsible Data Usage and we’ve been focused on translating these principles into easy-to-use tools on how to balance the need for data with protecting that data from misuse and harmful impacts on our beneficiaries.

5 Steps to Measure the Benefits and Risks of Data

One example is Data.FI’s new internal Benefits and Risks Assessment Tool, a framework that I co-authored with Linda Raftree under the USAID mSTAR Responsible Data activity. We took the Responsible Data framework and created an Excel-based template, inspired by responsible data work at CARE by Linda Raftree and Kelly Church, to help document and report on the benefits from a data activity vs the risks.

Responsible data isn’t achieved through following hard rules, rather there are practical tradeoffs that you’ll need to consider and balance in each program. This synopsis of the Benefits and Risks Assessment Tool walks you through how to make decisions on tradeoffs in five steps.

Step 1. Identify the anticipated benefits.

At the outset, list how the data is going to be used to benefit different stakeholders, organized by area of benefit and by stakeholder. These benefits are likely part of the theory of change.

• What are the anticipated benefits of this data? Common benefits are improved service delivery, linking case data together, etc. The more precise, the better. Often, we think of benefits as “services will be improved” but we don’t then plan for (or fund) that feedback cycle.
• Who does this benefit actually serve? What impact will it have and how likely? For example, if you are collecting reproductive health behaviors from clients as part of a survey, the direct benefit to the clients may be minimal vs the impact and likelihood of benefits to the researchers, program designers, and policy makers. But if you are collecting this data as part of direct service delivery and case management, the direct benefit to the client is going to be more impactful and likely.
• How will you measure these benefits? Do you need to add M&E indicators to measure the benefit you are anticipating?

Step 2.  Assess the potential risks.

What are the risks to the data and risks posed by the existence of the data to help quantify the need to collect the data in the first place?

• What are the anticipated risks for this data? For each data element, the same analysis needs to be performed for risk. What are the common risks and the vectors by group?
• Who is actually at risk and how? It is very important to check the risks by group because not all groups have the same risks. Estimate the severity of each of the three categories of risk and the likelihood that the risk will occur. Some risks may have a high likelihood but a low severity score (e.g., a lost device); others may have low likelihood but a devastating severity (e.g., loss of life). In the example above, the impact of a data breach on a client providing data about reproductive health behaviors is likely be much more severe than on the researcher, especially if the client―such as a member of a key population―is already marginalized.
• How will you measure the risks? What will you do if you discover harm is occurring? Keep in mind that this step may need to be conducted frequently as programs or program context changes.

Step 3.  Compare your risks vs your benefits by group.

Once you have your list of benefits and risks, you can then compare across types and groups. It may be come very clear whether the benefits outweigh the risks or if any risks are disproportionately borne by any particular group.

• Which groups have the most to benefit relative to risks? A program may have many different groups, including funders, implementing partners (IPs), civil society organizations, and clients. Are there groups who have very high benefits and very low risks? Or are benefits and risks relatively balanced within groups? Those who benefit do not need to face risks, but you need to assess whether those who face risk are also the group that is unlikely to benefit. As much as possible we want to shift the risks to the most powerful (funders, IPs, etc.) and away from vulnerable groups like clients.
• Which groups have the greatest risks? Measure the risks in the absolute sense, and relative to their position in that community. Are risks disproportionately affecting funders and IPs? For example, funders who support innovative programs accept a higher than usual tolerance of financial risk. But these programs should not negatively impact clients. If the example is flipped― where clients face a high amount of risk in relation to benefits―the program needs to be adjusted by increasing the benefits or mitigating risks, described in Step 4.

You may want to share your analysis with representatives of the different stakeholders to confirm or improve your benefit /risk assessment.

Step 4. Identify strategies for maximizing benefits.

With the groups that have the highest risks, seek to find ways that you can avoid, mitigate, or reduce the negative risks while maintaining benefits. For example:

• Identify strategies to increase direct benefit, such as the use of feedback sessions or additional information to those who share data; or provide remuneration or other types of rewards for data subjects.
• Minimize or avoid data collection of specific types or categories of personal or sensitive data. In the case of PEPFAR data, that includes but is not limited to all personally identifiable health data, biometrics/DNA, sexual and reproductive behaviors, personal financial data, or summary data on hot spots or other sensitive groups.
• Create a detailed data protection plan if you do collect high-risk data. It will likely include IT security specialists. Please note that the more security needed, the higher the cost.

Step 5. Assess if the data collection is worth the risk

• Are the benefits and risks fairly distributed? Or do the most vulnerable people bear the risks while the most powerful reap the benefits?
• Can the benefits be improved to those who are risking the most?
• Is it feasible to reduce the risks and threats to an acceptable degree?
• How are we measuring these impacts?
• What are the costs of increasing the benefits and mitigating the risks? Will they render the initiative infeasible in terms of cost of envisioned outcomes?

A benefits and risks assessment should be conducted for any initiative that involves personal or sensitive data. Moreover, these steps should occur during the design phase of any tool, service, platform, or program.

The assessment should be consulted and revised any time new technology features are implemented, when substantive changes are made to the types of data being collected, when data privacy laws change, or if context significantly changes (e.g., elections, conflict, humanitarian emergencies etc.).

By Siobhan Green, MA and Kelly Church, MA, with the USAID/PEPFAR-funded Data.FI project.