Agile development processes
Agile development describes a set of principles for building software that allow collaborative, cross-functional teams to accommodate changing user needs and requirements.
One of the great strengths of agile development is that delivery is incremental, which spreads the risk across the entire development process rather than loading all of it at the very end. When things go wrong, they’re smaller problems that can be fixed more quickly and more easily.
Agile development is based on the reality that you will discover new requirements during the development process.
In any complex system, needs change, evolve, or are discovered over time. Instead of ignoring these changing needs, or going through a painful contract amendment and renegotiation process, we embrace the concept that we will discover new requirements as we go.
To help this process, we pair identifying requirements at a higher level (an “epic” in agile terminology) with an ongoing user research process. These epics describe the user need, not the solution. Through the user research process, these epics are turned into a set of increasingly-specific user stories. This is done continuously, so that user stories are ready for the development team no more than a month ahead of when the development team implements them. (For example, a user story might be: “As a caseworker, I want to see all my current cases and next steps so I know what I need to do next, and when I need to do it.” We wouldn’t write: “The system shall display the next step next to each case in a grid on the main screen.”)
There are three leading causes why IT projects fail: a lack of reliable input from users, incomplete requirements and specifications, and changes in the requirements and specifications. The Standish Group, an IT research firm, publishes an annual report on IT projects, in which they point out that these causes of failure are about the impossibility of “knowing” requirements up front.
Agile development requires a shift in how you understand the costs of software development.
The traditional “waterfall” method of software development assumes that software can be built once for a large, one-time investment. In practice, this is untrue, because requirements can rarely be fully known and change constantly. Users’ needs tend to arise through research and testing, and development teams must discover appropriate solutions by building iteratively and interacting with users regularly.
Ultimately, the state’s investment should be measured in working software, not phase documents or milestones. Only working systems are of value to real constituents. Based on that premise, teams are only able to measure the success of a waterfall project after most costs have been incurred, because that is the first time working software is delivered. This is a huge risk. Agile projects allow the state to measure success at more regular intervals, and reduce the overall risk to the state.
Starting with small projects can help build agile project management skills.
We recommend starting with a small, less important project in order to develop your team’s understanding of agile project management, then expand. It is certainly much more difficult (and higher risk), and requires more coaching and training, to dive right into a large, high profile, critical project first.
Incremental development can allow end users to start seeing benefits even before the legacy system can be fully replaced.
Old systems must be kept up until the new system is fully operational, no matter what approach is used. In a monolithic approach, the end user must wait many months or years to use the new system and see any benefit — and that’s assuming there are no problems with the rollout. In an incremental approach, the end user could start seeing benefits in weeks. Depending on the architecture of the legacy system, parts of it may even be able to be decommissioned and turned off before the full new system is in place to save money.
Agile principles recommend documentation that supports long-term maintenance.
One common misconception about agile development, which calls for just enough documentation, is that there will be no documentation. That is not true: agile prescribes documentation that is of value to the long-term team.
Like any process, agile development has risks — but they are avoidable.
In some cases, teams incorrectly using agile as justification for poor or no architecture, design, or documentation. Incomplete, incorrect, or uninformed adoption of agile techniques can lead to an ineffective process we call “agilefall.” We’ve written a blog post about how to avoid this pitfall.