A recent post from Eva Schiffer on “agile international development” prompted me to share more widely some reflections I have been brooding on for quite a while on the very same topic.
I am Mitchell Toomey and I have been thinking about what can UNDP (and others in the human development business) learn from web 2.0 startups? That iterative, adaptable “agile” processes can provide exponential improvement in the success and relevance of development work, while dramatically reducing risk of project failure.
First a bit of background on software development processes:
In the “old days” software development was a very high-risk, somewhat mysterious proposition. Computers (and programmers) were rare and expensive, errors were hard to find and correct, and most of the application logic had to be fully designed months or even years before the app would finally be available for use in the real world. This lengthy, serialized process was pretty inefficient, and made software development a major, risky undertaking where the outcomes were “generalized,” only indirectly linked to measurable results.
The standard method for development software came to be referred to as the “waterfall” method, where things needed to happen in a fixed, serialized, multi-step process, with nearly everything fully documented and designed before the first line of code was written.
This method actually worked great when developing, for example, the software that was used to guide a spacecraft to the moon. Such high-risk projects are well suited to this highly rigorous, though rigid life cycle. (Once the rocket launched, it was pretty hard to “iterate” the software, so it really needed to work!)
The big problem with the waterfall model was that in most cases (where life-or-death rocket science was not involved) business requirements changed more rapidly than the model would tolerate, requiring lengthy “change order” process where the full set of project plans, specifications and schedules would need to be re-examined and reformulated to account for what was actually a natural characteristic of life, change.
Over time, a new method of developing software emerged, called “agile development.” The basic idea behind agile development is to document very fundamental goals (“the search engine product should allow a user to enter a keyword and get a list of highly relevant results, sorted by the most relevant first”), but then, instead of trying to design every necessary function and line of code in advance, the development cycle was split into multiple short “sprints” or iterations, where first a basic version is developed, then over time, and in reaction to “conditions on the ground,” the model can be tweaked and altered to fit the needs that emerge as each iteration is tested.
It turns out that this iterative, sprint-driven model worked great; agile techniques worked, especially when coupled with instant online software distribution that allowed for quick revisions, instant bug fixes, and most importantly continuous learning and improvement based on discoveries made along the way. This “endless experimentation” model resulted in the explosion of Internet-based software solutions like Google, Facebook, Baidu, along with thousands of others.
The various styles of agile software development have become pretty well standardized – going “agile” does not mean going without a plan. The concept of highly iterative, resilient design and build processes simply compartmentalizes a project into realistic chunks that fit together into a basic architecture.
How does this relate to the work of human development?
The traditional model of international development aid could be said to operate in a waterfall model – long term plans dictate specific requirements and success criteria, and these plans take a really long time to design, negotiate and agree upon. We often see that by the time a multi-year plan is finally signed, conditions on the ground have changed, and the agreed framework and reporting cycle is not responsive to the emerging development needs.
In a similar way, the long-term projects that are the foundation of traditional country development programmes are also designed and “fixed” in this long-term, highly documented way. In more cases than not, the original plan for how to design and execute a project needs to be revised and altered to fit with new conditions (regime changes, natural disasters, economic crises, famine, disease outbreak etc.).
The work effort that must be dedicated to administering and managing these highly scripted long term projects is significant, since the success of the entire enterprise is measured against the original design plans.
In a similar way, the procurement processes that a project manager is required to follow assume that long-term, detailed plans are not only possible, but preferred, so that very specific deliverables are identified long in advance and detailed, tightly bound contracts with partners can be drawn up and bid upon.
The exponential speed of value-generation seen in the software industry can and should be replicated in the business of international human development.
The basics might already be there
Project methodologies such as PRINCE2 are somewhat agile processes. In PRINCE2, a project is split into multiple phases, and between each phase the project managers and stakeholders evaluate the outcome of the prior phase, and consider if the plans for the upcoming phase might need to be modified to adapt to the conditions on the ground. It is actually a great comfort to see that one can lean on existing tools and processes to “go agile.”
The big problems remain: large development institutional processes are still primarily waterfall, when the work calls for an agile approach.
In my experience as a manager operating within such a traditional context, I often run into roadblocks that are basically caused by a systemic bias towards “waterfall” thinking. Take for example the process of procuring specialized professional services (such as training consultants, or software developers). In contrast to the highly volatile conditions into which a project is being deployed, the rules governing the procurement of services to support and achieve the project are still governed by lengthy documentation processes that were originally designed to “manage risk” before the advent of the more iterative lower-risk processes were made possible and transparent via information technology.
Who will become the first international institution to actually mandate an agile development model in its programming? What would donor agreements look like? What about programming agreements? Procurement rules? Project management tools? Success criteria? Evaluation methods?
The parallels between software programming and “human development” programming are encouragingly clear. Both help people by providing access to tools and capabilities that they did not have before, and both can be successfully managed, with a clear focus on results, if the most appropriate methods are employed.
It’s time for development 2.0.
Mitchell Toomey first published this as Agile development: What human development can learn from software development and its republished here with his permission
Perhaps human development projects follow the waterfall approach so strongly because of their sheer nature. I work in the private sector, so I think “budgets” sometimes can be taken for granted. I am hardly implying that all companies are totally copasetic with having an open budget, but it is true that clients tend to be more flexible in relation to changing project scopes. Sometimes we estimate that projects will cost a bit less than what they actually come to (and vise versa). However, being that these projects are done in the private sector, there is much more flexibility than I would imagine exists in public, governmental, or non-profit ICT work. Are not many of these projects funded by certain grants or specific donations? I find that something useful in agile software development is the flexibility to adjust not only project requirements, but timelines and, as I am referring to here, budgets to meet changing criteria (obviously project budget cannot be immune to project changes either). Being that agile projects are living and breathing, does that make allocation of major grants or donations (from governments, charities, what have you) difficult to apply? I would imagine so. This difficulty would explain the need for such structure in procurement rules, programming contracts, timelines, expected results, etc. On another note, I think many mobile money platforms – many of which ARE funded in part, at least, by private companies – follow the agile “test and develop” approach much more, whether organically or intentionally.