It’s Not a Target; It’s a Forecast

Product managers and product owners often struggle with estimating agile product delivery. They sometimes devote too much time in effort to calculations and observations that could be better spent on other tasks. Story Points offer one simple technique that will help product owners achieve actionable, timely, and accurate delivery. 

While this technique uses a set of values to determine a confidence interval range around the median of the Binomial distribution, the intent is to keep it simple and not use math or waste time with extensive observations. Read on to learn how to simplify a complex process into something that anyone can memorize without requiring math! It only requires arranging a sequence of numbers and remembering a few simple rules to have a 90% confidence interval range.

One of the most common questions when working towards an agile transformation is “How long will it take when practicing the scrum framework?”  Predictions are uncertain regardless of which method an individual chooses to use. The reason? Estimating is more than using a crystal ball or a gut feeling. Scrum teams don’t have all the information they need when asked to predict “how long will it take”; we know that the future could be different than the prediction. Click to Page 8 of this white paper from the Beyond Budgeting Institution for more information. 

A forecast is an expectation of what we think will happen. 

Forecasting is about looking ahead and making decisions based on understanding likely future outcomes. It is essential to look ahead to determine how much work can be done in any given period. It is unproductive to rely on a single predicted number and no probability of occurrence.

It is impossible to ideally assign a total estimate to user stories on the product backlog. Some estimates will turn out bigger than estimated, and some will be smaller. However, once you start the work, it is possible that the estimation can turn out to be more complex or more straightforward than you thought. It is not that important that the actuals are bigger or smaller than estimated, but the team must apply what they learned to future estimates of user stories in the backlog.

The scoreboard reflects the score commonly measured in an abstract unit of points in sports. The points are a quantitative measure of the players’ relative performance in a sporting discipline. In scrum, story points are an abstract measure used for relative estimation/sizing of user stories or backlog items of a single unified backlog.

Relative estimation is the process of finding an estimate or an approximation. The estimate is a usable value for sizing user stories even if the input data is incomplete, uncertain, and unstable. The estimated value is based on the best information available. A relative estimation is not based on a unit of time. It is an estimate that considers three factors: a combination of complexity, effort, and risk of uncertainty that involves implementing one product backlog item and comparing it to the implementation of another product backlog item.

The story points identify the size of the “user story.” Story points commonly represent a sequence of numbers, such as the binary (1, 2, 4, 8,16, 32 ...) or the Fibonacci sequence (0,1, 2, 3, 5, 8 ...). Incidentally, “user story" is not a scrum term; it is an Extreme Programming (XP) term.

Related Article: Agile Forecasting with Focus Factor

A team’s sprint velocity can be compared to the number of points scored by a sports team in a sequence of games. 

The team uses story points to measure velocity. Knowing the team's velocity helps in various ways. It is a historical measure of how much work the team accomplishes in the sprint. Product owners use the team’s velocity to forecast how much work the team can do in a sprint using burn-up charts and release planning. Knowing the velocity helps the team and satisfies the product owner’s expectation of work delivery. The team's velocity will vary from sprint to sprint. Therefore, it is best not to rely heavily on a single or desired value.

The most common point metric in team sports is the “goal.” The team makes a goal every time it scores. The respective teams add these goals. The match score represents the total score accumulated by each team. In scrum, the sprint goal is the unifying theme of the sprint (e.g., we’ll focus on “logging in” for the sprint). Sprint goals are essential; they tie the work of the sprint back to the product roadmap. Sprint goals help communicate to the stakeholders what the teams are working on.

Forecasting capacity for delivery is an agile project management technique. The technique uses forecasting to determine how much work a team can complete during a planned number of upcoming sprints. The team is better off considering velocity as a range instead of a specific value. Rather than saying, “Our history shows an average velocity of 21,”   It is better to say, “Based on our history, we are 90% confident that the velocity for the remaining sprints on this project will be somewhere between 17 and 23.”  This provides forecast information that is actionable, timely, and accurate enough to meet the product owner’s expectations.

This simple approach is based on a formula that calculates a confidence interval around the median using the Binomial distribution. As mentioned earlier, over the years, I have simplified the forecasting process to a simple technique that anyone can memorize. The process does not require math. It involves arranging a sequence of numbers and remembering a few simple rules to have a 90% confidence interval range. Mike Cohn, Brian Mudden, and Ken Rubin have also written articles using the Binomial distribution 90% confidence interval.

The product owner can use the 90% confidence interval velocity to make informed scope/schedule tradeoff decisions. The product owner can forecast how much functionality can be delivered by various dates. "The purpose of forecasting is not to predict the future; it is a projection of the future based on a set of assumptions.” (Mortidge, Steve and Player, Steve).

To determine a range around the velocity, you need data for at least eight sprints. Sort the historical velocity from lowest to highest. To do this, use the table below that shows two data points in the sorted set of velocities to determine the 90% confidence interval range. Use the range of numbers to forecast how much work can be completed in that time.

To find the 90% confidence interval range, follow the steps below:

1.  Collect the velocity of the past eight sprints.

2.  Sort velocity values from lowest to highest.

3.  Use the two data points in the sorted set of velocities in determining the 90% confidence interval.

a)  The first data point is in position 2. It is the lowest value of the 90% confidence interval.

b) The second data point is in position 7. It is the highest value of the 90% confidence interval.

4.  The value in position 4 is the median value of the 90% confidence interval.

5.  The 90% confidence interval range data points provide the forecasted velocity of how much functionality can be delivered by a given date.

The table below is an example of the observed velocity of the last eight sprints listed chronologically.

Sort the values lowest to the highest.

After sorting, we can see that our second lowest value is 34. This value is the lowest range value of the 90% confidence interval. The second highest value is 41. This value is the highest range value of the 90% confidence interval. Therefore, we are 90% confident that this team's velocity will fall between the range of 34 and 41 story points.

To estimate how much the team is likely to complete in five sprints, multiply the number of sprints by the lowest value (34) and the highest value (41) in the range. In these five sprints, the team can deliver 170 (5 x 34) story points and 205 (5 x 41) story points.

Using the results of the forecasting capacity for delivery, the first step is to use a backlog that has been previously prioritized and relatively sized — plan full capacity using the highest range of 205 story points. A key element of the technique is agreeing to a flexible scope. Flexible scoping is planning the critical items first but limiting the scope of the lowest range value of 170 story points. It is an opportunity to apply Pareto's Principle that suggests you get 80% of the value of any project from 20% of the requirements.

Plan the remaining range of 35 (205 - 170) story points of capacity with not critical items but might delight the users. The product owner and management should help the team remove obstacles that prevent the team from reaching its highest value in velocity. Each sprint should have a mixture of critical and non-critical items.

Apply this forecasting technique as often as there is a need to make decisions or if there is a need to refresh the values. It goes back to the scrum pillars of transparency, inspection, and adaptation. The range interval provides transparency for the stakeholders of the release plan and the vision. Inspecting the results to measure the value delivery helps determine how often to measure and adapt to maximize return on investment. The variability of the values may also determine the frequency of refreshing. Over time, measure the performance of the technique to provide acceptable forecasting. The values can provide opportunities for improvement. The team may want to take corrective action…

●  …if the range is very wide, maybe greater than +/- 20 % of the median.

●  …if the median's trend is negative.

●  …if the median's spikes in a positive direction.

Over the years of using this forecasting technique, I have seen teams with a wide range of over 50% between the lowest to the highest value improve to a range of less than 10% as they became aware of their situation. The technique above can be scaled and applied to multiple teams using a single unified backlog. After determining the median, low value, and high value of each team's backlog, sum all the median numbers of each team. Repeat the same process to determine the sum of the low range and the high range numbers. The sum represents the 90% confidence interval range velocity for the multiple teams. The teams can initially use the worksheet above. After using it a few times, it is easy to remember and apply automatically.

In making decisions, forecasting capacity is simple, timely, and accurate enough to understand likely outcomes. Relative estimation is used to size “user stories.”  Three factors: complexity, effort, and risk/or uncertainty are commonly used for the relative estimation of “user stories.”

Related Article: Story Points Explained 

Often in software development, we tend to assign success or failure to individuals rather than the environments in which they work. 

Any member of the team or the team itself can apply this technique when asked the following question, “how long will it take.” There is often an inclination to blame someone for poor forecasting. The forecasting solution is to allow the team doing the work to relative estimate the product backlog. As mentioned earlier, the future might be different than the predictions. There is no perfect forecast!

About the Author

Joe Alvarado, CSM, CSP, CAL1, CAL, and PMI-PMP/ ACP, is a Senior Business Consultant for Tata Consultancy Services. Joe’s 20 plus years of experience includes providing solutions for various Fortune 100 companies and the U.S. Navy. Joe has guided organizations in the next-gen digital era through a Lean-Agile transformation at scale by training, coaching, and program management. Joe focuses on leading product development projects in adaptive planning approaches. Leaders and teams value Joe’s coaching and team-building approach. Joe is a catalyst for cultural change, a strategic thinker, influencer, value-conscious, business optimizer, and has a passion for practicing scrum in business and everyday life.