'Business does not pause and await the completion of the IT transformation - costs and business pressures will escalate if it does. It is critical that a development group demonstrate its value to the business as soon and as concisely as possible.' - Rallas Buttriss
It's a simple question but the answer is complex: 'Did you give me what I need to improve my business?' In all projects, regardless of Industry vertical, scope, cost and timeline, there is a desire to understand if the project is a success. When the project is performed in an agile manner, this does not change. As an organisation transforms from a traditional waterfall approach to meeting business requirements for more agile processes, many teams are challenged to demonstrate if the change from traditional approaches was worth it.
Businesses fund projects to deliver value - not software. Yet many projects define success by using measurements that have little if any relationship to business value.
To adequately address conclusions on the success of an agile transformation, it is necessary to firstly consider what metrics your clients rely on to understand progress and success. Consider what metrics may now be relevant for them to collect and report on, to best reflect an Agile project's success. Beware of comparing apples and oranges.
Business remains focused on quality metrics such as defects per phase or release; schedule metrics, such as actual effort/days versus estimates and delivery milestone achievements, and cost metrics such as budgeted versus actual costs, internal (sunk) costs, and also production support financials. Basic application life cycle (ALM) management tools do a decent job of unobtrusively collecting this data, however none of these metrics demonstrates how and why an agile process can affect project success and business value.
Delivering a piece of capability - assuming it is an IT software release - that:
- meets the budget;
- was delivered on time;
- had no critical defects; and
- was released according to compliance requirements - and yet;
- does not address the desired business outcome;
cannot be considered a success. Yet, by most traditional measures, even addressing the stated objective requirements, this would be reported as a successful project.
*Delivering defect-free code doesn't mean delivering valuable code. Value is in the eyes of the customer: if you're not meeting their needs you're wasting time and money. *
In many cases, project reporting focuses on indicative 'in progress' measures of the activity that is being performed to create a deliverable. These are important as they provide 'lighthouse' markers of the progress and direction of the activities. However, they are not in and of themselves a true reflection of the project's success.
By moving away from relying solely upon the project's indicative 'in progress' measures, toward incorporating the businesses achieved 'value provided' measures, agile projects can better represent their success rate.
THE FOUR MEASURES OF SUCCESS
A measure of stakeholder satisfaction is one of the most important metrics that a team (Agile or not) should introduce. Stakeholder satisfaction includes responses from Showcases and User Retrospectives; Walkthroughs and capability presentations; OCM provisioning and endearing Change Champion group buy in. Considering all of this will allow a project team to communicate to the business in the terms that the business understands and more succinctly provide adequate metrics that do not require interpretation to derive value.
Secondly, a measure of delivered capability against desired requirements (user journey elements; value chain facilities and functions; use case points; user story points; epics and themes).
Thirdly, Defect trends; (defect trend rate and severity classifications; defect fix introduction and speed; change velocity; backlog and feature priority list), and finally, a measurement of actual cost (hours expended, $ expended, FTE required).
WEIGHING IT UP
Start by turning the discussion with IT and business staff to one of priorities. Logically, the highest priority items will deliver the most value in the customer's eyes. For an individual project as well as for a portfolio of projects, the team can use this prioritization and begin to think about (and ultimately determine) value.
A major benefit to using agile processes (or any iterative and incremental approach) is that you can quickly deliver a number of releases to the user community. Not only can you prioritize the features you release based on the value the user will receive, but you will also have quicker access to post-release information to best derive business value.
For example, as soon as a software product can help drive revenue, you can begin to incorporate the project's costs and determine the return on investment for the project. For companies with short funding windows, this is the best way to quantify results and justify investment.
The previous four measures of success are usually 'just enough' information for both IT and business management to monitor development projects in the context of business value. If required, specific managers who are interested in greater detail can use these categories to drill down to more granular data points to satisfy their queries. None of these measures negate the need to have the finer levels of detail. It uses this detail as quantitative measures of the qualitative summary.
BEST-OF-BREED TOOLS
Agile reporting tools: The Atlassian suite (Jira, Confluence, Gliffy, and more) has steadily been incorporated as a 'go to' set of reporting capabilities. There are others such as Rally, VersionOne and XPlanner. All share the same philosophy that reporting derived business value (qualitative) as measured by defined indicators (quantitative) is the best measure of success.
*Scrum tools, continue to pare down unnecessary information and emphasize value-driven metrics.
