ICTworks

At M&E Tech in DC and NYC, we heard from development professionals who were struggling with the question on how to build an organization-wide technology-enabled system for collecting, storing and analyzing M&E data. Some of the questions that development organizations are asking when thinking about organization-wide management information systems (MIS) are:

• How do we collect and manage M&E data on complex programmes across hundreds of countries?Many multi-national organizations have multiple projects within one country, and/or projects that stretch internationally. In these instances, each project or country needs the ability to customize their M&E work to their particular setting. This becomes even more challenging when multiple partner organizations are involved in delivering one project or programme.
• How do we balance flexibility, while at the same time keep standards and common measures of activities, outcome and impact in place? Many organizations have experience building their own systems (after perhaps researching systems and not finding the right fit) but they are still finding that the system is limited, doesn’t work perfectly or do everything it needs to do.
• What can we expect from our software solutions? And what other systems do we need to put in place to support or enable software to fulfill its promise of greater efficiency and coordination? Or do we have it the wrong way around? Quality communication around the process of providing an enterprise M & E solution will help to control expectations, and inform stakeholders of the bigger issues involved in achieving success. Change Management is the step that is often overlooked.
• How will you support an enterprise M& E system, including rollout, training, helpdesk, and upgrades? Once your application is ready, rolling it out and training staff on its use and features is not a one-time effort. Do you have the resources (staff, funds, time) to provide the needed support services? Some organizations have reported a successful initial project rollout only to see it die from lack of support. Staff who don’t feel comfortable using an application or who don’t see improvements in functionality will go back to the tools they’ve used in the past.

We heard experiences from organizations that have launched organization-wide technology systems for M&E (and other) data. We heard that the several tips for how to start the conversation within your organization and what to do/not do:

• Start with a discussion about the processes for collecting data. Establish standards and test them across the organization
• Standardize your theory of change and results framework (as much as possible); this is important so that data are comparable across the system
• Focus on data for decision-making – if data are not useful for decision making, consider why you are collecting, storing and analyzing that data
• Start small – test your system in Excel or Google Forms first before moving to a more complicated system. Starting small will allow you to identify where the most time is being spend, and therefore where technology can be most impactful
• Manage expectations – technology cannot solve all of your M&E challenges!

If you do find yourself assessing tools for organization-wide M&E data (after you have piloted standardized your indicators and tested some spreadsheets) there are three main options:

Build

If your organization has a software development team and the ongoing resources to invest in a system, you can consider building your own from scratch, using technology that is new to your organization. If you do so, you should consider these questions:

• Is Software Development in your mission statement?
• Consider sustainability – will you develop in-house capacity? Or work with consultants?
• What resources do you have?
• Think about how resources will change – both staff (staff-turnover) and software (will the software still be supported in 3-5-1- years)

Buy

Buy or license a system from an existing commercial vendor that has the skills, experience, and track record of implementing quality M&E systems for organizations of similar size and complexity to yours. Examples include TaroWorks, Fluxx, and DevResults. In making your decision, be sure to include these considerations:

• The product should meet 80% to 90% of your requirements “out-of-the-box” – no additional work required
• The product should hold your information in a generic file format (not a proprietary file type!)
• What is the longevity of the “buy” product? What happens if the company is sold or goes out of business?
• Verify that the software is real (not just a demo)

Hybrid

Customize an existing, open-source solution for managing your data, or build upon an existing platform with which your in-house staff currently uses and supports. Examples include Sharepoint and Salesforce. This option will also require some in-house (or consultant) time but maybe be more sustainable than building from scratch or buying/licensing from a commercial vendor. Benefits include:

• Controlling your customizations, including making them open source
• Creating collaboration across organizational teams and specialities
• Building upon and enhancing existing in-house expertise
• Allowing flexibility between the use of in-house resources and consultants

And it is not only M&E data that development organizations, researchers and practitioners are struggling to manage – there are many other data and software solutions development organizations may be using or considering. How will your M&E solution connect with other appropriate data silos like these?

Internal Silos

• Human Resources systems
• Intranet
• Email and internal communications (chat, calendars, etc.)
• Customer Relationship Management (CRM)
• Financial Management
• Project Management
• Knowledge Management

Internal/External Silos

• Grants management
• Financial accounting
• M&E data management
• Extranets or external web sites

As you can see, the choice between building, buying, or doing both when it comes to technology systems to support M&E activities is complex and should not be made in a rush, on a whim, or as a favour to a friend. Organizations need to take a close and nuanced look at their needs, resources, and expectations, and make the decision that is best for them.

Good luck! And if you have lessons you learned from your own build, buy, or other M&E technology journey, please add them in the comments below.

Written by Jamie Lundine, Spatial Collective Ltd, and William Lester, NPOKI based on their joint session at M&E Deep Dive in NYC