Creating a Risk-Adjusted Backlog
January 16, 2021
This article explains what a risk-adjusted backlog is, why they are useful, how to create one and how teams work with them.
What is a Risk-Adjusted Backlog?
A risk-adjusted backlog is a backlog that contains activities relating to managing risk in addition to the usual features associated with delivering value.
Agile projects typically prioritize the backlog based on business value or perceived needs. The Product Owner or business representative prioritizes the backlog elevating the highest value features to the top, so they get delivered first.
Taking an Economic View of Decision Making
Prioritizing based on business value is an example of the lean concept of 'Taking an Economic View of Decision Making.' In deciding which feature to develop first, those with the highest economic value are selected. Taking an economic view of decision making has a couple of advantages.
First, the business gets the highest value features early. If, for some reason, the project was stopped or interrupted part-way through execution, there may still be a valuable partial product to use. It also gives the option to curtail a project when the value of remaining backlog items does not justify the average iteration burn rate. Instead of continuing work, all the financially viable elements have been delivered, and work can stop or be directed elsewhere.
The other advantage is that it lets decision making be more objective and less of a subjective, political contest. It is normal to see situations where department A argues with department B over which features are more important based on first-dibs, you-owe-me, or I-out-rank you type debates. Putting a monetary value on features can help reduce the arguments. Manipulation and gaming the numbers can still occur, but it brings the conversation to value, which should ultimately be in the overall organization's interest.
Non-profits and projects for social-benefit can still use economic decision-making. Often they are more aware of the value of limited funding or the multiplication impact of meaningful investment and change. While profit is not their goal, usually an economic impact figure can be estimated for the outcomes they aim to achieve.
The Economics of Risk Management
A risk is an uncertain event or condition that, if it occurs, has a negative or positive effect on the project. Negative risks are called threats, and positive risks are called opportunities. If a threat occurs, it becomes an issue that can cost the project time, money or both.
For example, if the reporting engine we were asked to use does not support the data volume needed, we may need to stop using it, select a replacement, test it and implement that solution instead. That could take two weeks and cost $100,000 when factoring in the new software license, the team's burn rate, and the cost of delay to the organization. If we believe there is a 50% chance that we will need a new reporting engine, then the expected monetary value (EMV) of this risk is 50% x $100,000 = $50,000.
Multiplying guessed losses or benefits by guessed probabilities is an inexact science. However, it is one that the insurance and investment industries have been using for centuries. Assessing the chances of loss or gain occur throughout all forms of business and relies on taking an economic view of decision making.
Risks Actions in the Backlog
For many years, I was content to put threat avoidance and reduction actions in the backlog alongside features. It seemed like an effective way to maximize the overall value delivered. I wrote about these ideas when I started blogging in 2006 as Risk Profile Graphs. By this time, I'd been using them for 4-5 years and knew they were well received by sponsors and executives.
The image below shows how threats have the potential to reduce project value.
When threats occur, they create negative value impacts. Like bank deposits and bank fees, to get a true perspective of the value, we need to maximize positive deposits and minimize negative fees.
Later in 2006, I wrote about risk adjusted backlogs and Agile Risk Management, explaining how to insert threat avoidance and threat reduction activities in the backlog. Then, in 2012 I presented a collection of Collaborative Games for Risk Management at the Agile 2012 Conference in Dallas and PMI Global Congress in Vancouver. After these conferences, people started experimenting with these techniques and sharing their experiences.
I learned I was missing out on potential benefits by focusing too much on negative risks (threats). When I started asking teams about what opportunities we could exploit, I realized the real benefits of collaborative risk management.
To maximize the business value, we should be delivering the highest business value features while also avoiding or reducing threats and trying to ensure any opportunities occur through exploiting or enhancing them. This way, we get the feature business value + opportunity value.
Organizational Value = Feature values + Opportunities realized – threat impacts
An example of opportunities realized includes new trials. If the average profit for new customers is $20,000 per annum, then we can determine if inviting qualified leads on a factory tour with product demos and giveaways that costs us $500 per head is worth it at a 5% conversion rate. Here $20,000 x 0.05 = $1,000, so yes, it appears worth offering factory tours and giveaways for qualified leads.
These days, when I look at the overall effectiveness of our risk management strategies at the end of a project, it is not usual to see 1.5 – 2 times the value in achieved opportunities compared to avoided threats. I do not think the teams have been weak at threat avoidance. Instead, I believe many projects leave a lot of development team sourced opportunities unactioned.
Creating a Risk-Adjusted Backlog
First, we need to find the risks that we intend to manage. The activities' Doomsday Clock' and 'Karma Day' described in Collaborative Games for Risk Management provide tools for facilitating teams through a risk identification workshop. These activities focus on the 12 common risk categories for IT projects identified by the IEEE and act as a good starting point for finding risks.
From identifying the risks, we need to sort, rank and analyze them. If we were following the PMI risk management process, this would involve the qualitative and quantitative risk management steps. Using a flip chart and engaging the team in estimating the probability and impact of the risks is an excellent way to tap into team members' insights and create an information radiator of opportunities and threats.
We do not worry too much about numeric analysis of probabilities at this stage but instead use relative-ranking to determine the vertical positions. The benefits come from the conversations amongst team members about analyzing the threats & opportunities and gaining consensus about where they belong relative to each other on the chart.
The Benefits of Visualization
The Kanban concept of making policies explicit is centered on visualization. This helps us talk about elements and share information to improve them. These charts use the same idea. We need to elevate and promote the process of risk management, so not only is it okay for people to be talking about it, but they are expected to be talking about it and practicing it in their daily work.
Visualizing and agreeing on a spatial reference for the risks and opportunities also engages the right hemisphere of the brain and makes us less likely to forget them. (Assigning things we want to remember to a location is a memory aid that many memory-improvement techniques use. Probably relating back to our hunter-gatherer days when our survival relied upon remembering where to find food and water. We have better recall of things assigned to a physical location.)
Having a visual reminder of the threats and opportunities is useful for us as the project progresses. It reminds everyone of the risks at play, so we are more likely to tailor our behavior and everyday decisions toward them appropriately.
Updating the Backlog
Finding and categorizing risks is a start, but not sufficient. The real value comes from converting them into actionable stories for the prioritized backlog. Calculating the expected monetary value of the risk allows us to prioritize any resulting actions in the product backlog by taking an economic view of decision making.
Not all threats can be avoided or reduced; some may have to be transferred or accepted. Likewise, not all opportunities can be exploited or shared. Some might have to be escalated to other stakeholders to get the benefits, and a final category may have to be ignored if we cannot influence them.
However, for those risks we can influence, we should create risk response actions and discuss how best to prioritize them with the Product Owner.
The idea is to calculate the expected monetary value for each risk and then decide, based on this value, where in the backlog the action should be placed. Usually, a Product Owner controls the prioritization of the backlog, so once the team has calculated EMV figures for the risk actions, someone needs to work with the PO to get them in the backlog.
Product Owners typically do not know the return on investment figures for each item in the backlog, so it is not as simple as ranking everything by value. Still, they usually have a good feel for when to avoid threats or exploit opportunities, and I have found them to be receptive when we talk through the process.
Once in the backlog, these risk response stories are treated like feature development stories and worked on by the team when pulled for the next sprint.
Team Activities and Tools
Risk management, like backlog prioritization, is not a once-and-done process. We need to monitor and address risks throughout the project. Fortunately, the cadence of iterations and events such as retrospectives and sprint planning provides all the occasions we need to keep the process going.
Backlog prioritization takes into consideration the current backlog, any new change requests and any new risk responses. During demos, the team presents the threat reduction and opportunity enhancement work they did along with regular features developed. During retrospectives, we review the effectiveness of the risk management process as we would any other process.
We can create some useful charts for tracking trends and the effectiveness of threat reduction work.
A risk burndown graph summarizes the threat profile for the entire project and can also be used to highlight any new and escalating threats. They are also useful for showing the progress made in early iterations where more work was done on architectural risk reduction rather than purely delivering business functionality. See this post on Creating Risk Burndown Graphs
Risk Retrospectives are periodic reviews of the threat and opportunity logs along with the risk management processes being used. Just as we review the evolving product and team processes throughout the project, we should evaluate the team's effectiveness of the risk management plan and approaches being used.
The types of questions we could/should be asking when we regularly review our risk management approach include:
- Are we reducing our threats and promoting our opportunities?
- How is our remaining threat EMV burning down?
- How much value have we been able to achieve from opportunity exploitation?
- Do we have any new or escalating risks?
- What are the root causes of our threats, and can we eliminate any of them?
- Are there lessons from opportunities we can share with other teams?
- Which risk avoidance or elimination strategies are working and which are not?
- For risks that we chose to transfer or escalate, how are the third parties managing them? What can we learn from them, or would we be better bringing them back internally?
- How are our team risk management capabilities developing?
- Where do we still need mentoring and support?
Rinse and Repeat
Finally, reviewing is not enough; we need to update how we work to improve. Update our risk lists and EMV scores, and reprioritize the backlog with new features and new risk responses; always be rebalancing the priorities. Update the risk information radiator graphs (like our risk burndown graphs), and make sure people are not only looking at the impacts of new work in terms of estimates--but potential risks, too.
Risk management, like estimation, should not be just a project management activity. We can significantly raise a project team's ability to manage risk and avoid project failures through socialization, collaboration and practice. If nothing else, these team activities make risk management basics more accessible to a larger pool of project stakeholders. In doing so, provide more eyes to find risks before they impact the project—which is the heart of effective risk management.
[If anyone would like more information, please get in touch. I have been using these approaches on projects for many years and have taught several workshops on collaborative games for team risk management. I am happy to share tips and resources.]