This Blog post may be somewhat controversial, but I really think it provides a viewpoint that I don’t normally see represented. I urge you to read the entire post before jumping to conclusions. Here goes:
I’ve read many a good post on estimating Agile projects, but I must admit the discussion about estimating via User Story Points versus User Story Hours seems to suffer from partisan politics. If someone believes in User Story Points, they paint the only alternative as a bleak Orwellian future of accounting for time in 5 minutes increments and shock therapy for the failure of missing estimates. They usually also assume that if you don’t buy in fully to User Story Points, you also don’t believe in Agile in general, Planning Poker, and are an avid fan of Waterfall. Nothing could be further from the truth.
What I would like to have is a bi-partisan discussion of the pros and cons of different estimating approaches.
Before I start I would like to state that I believe that User Stories, Planning Poker and Story Point Estimating should be done on each and every project. What I am discussing and proposing here is not about not doing Planning Poker and Story Point Estimating, but rather discussing whether the estimating should stop there or continue with other additional methods.
There is a continuum of approaches for estimating. This is illustrated in the diagram below:
When discussing the pros and cons of Story Point Estimating, we are usually limiting our discussions to just the two extremes. I would like to propose a hybrid approach as well.
|Agile||The Agile approach is the one that is most commonly proposed in the Agile literature. This approach proposes estimating by User Story Points and these estimates are typically generated in a Planning Poker Session(s). The Planning is then done by planning the Iterations based on the number of User Stories and their associated points.Success is then usually judged by being able to achieve a stable User Story Point Velocity from which to execute the remainder of the project in iterations. This Velocity is used in an iterative and ongoing basis to refine the plan for subsequent iterations based on past results.|
|Traditional||The Traditional approach is defined by creating very detailed task level estimates in hours. These estimates may be done by the team, but usually are done without the team or with a subset of the team. These estimates are usually created without the input of developers. The Planning is then done by creating a Work Breakdown Structure. The Work Breakdown Structure is then applied over a timeline or schedule to create the fully detailed Project Plan.Success is usually judged by the accuracy of the task estimates which will then translate into meeting the estimated schedule generated from the Work Breakdown Structure.|
|Hybrid||The Hybrid approach proposes estimating by User Story Points and these estimates are typically generated in a Planning Poker Session(s). The Planning is then done by planning the Iterations based on the number of User Stories and their associated hours. The User Story Hours are created by converting the User Story Points into User Story Hours.Success is then usually judged by being able to achieve a stable User Story Point and Hour Velocity from which to execute the remainder of the project in iterations. The User Story Point Velocity is used in an iterative and ongoing basis to refine the plan for subsequent iterations based on past results. User Story Hours are provided to the owners of User Stories so they know what the original estimates of the User Stories were.|
What are the pro and cons of each approach?
Well we all know the pitfalls of the Traditional approach, so let’s not even discuss that approach. 🙂 But why is the Hybrid approach compelling when compared to the Agile Approach? In fact, the Agile and Hybrid approach are very similar except for the fact that the planning is done in User Story Hours and the velocity is measured in User Story Points. In addition, User Story Hours are then provided to the owner of each User Story to provide a frame of reference as to the initial estimate. Let’s look at the pro and cons of Agile estimating concepts and how User Story Hours can help mitigate the cons.
|Concept||Pros||Cons||How do User Story Hours help?|
|User Story Points||Minimizes the human element of having duration or hour perceptions influence estimates.||May force the ‘slotting’ stories into non-ideal slots. (i.e. It is a 30, but all I have are 20 and 40 slots)||Provides a second review once you have converted to User Story Hours to ensure the estimates are consistent and make sense.|
|Relative estimating||Establish standard grouping and increase accuracy by relative estimating. Also leverages human ability to excel at relative estimating.||Creates single point of failure. Base line estimate which all other estimates are relative to can create a ripple effect of estimate inaccuracy. This may take multiple iterations to correct.||Remove single point of failure with reality check. Second review will ensure estimated hours make sense.|
|Planning poker||Increases team buy in, shares estimating expertise and promotes team growth, helps to refine solution understanding, and increase estimating accuracy.||None.||Not Applicable. This concept is used in both the Agile and Hybrid approaches.|
|Velocity and Burn down||Validate estimates and plan at a macro level. Not at an individual, user story or task level||None.||Not Applicable. This concept is used in both the Agile and Hybrid approaches.|
Perhaps the most important benefits of User Story Hours are that they allow for the derivation of a project budget and schedule. One of my biggest challenges with the User Story Points approach is the concept that it will take two or three iterations to determine your velocity and then you can inform your client when you can complete the project and how much it will cost. Very few projects I have been a part of can get a blank cheque for two or three iterations and then confirm if the project is feasible and has business value. User Story Hours still allows you to have that check at the end of each iteration AND also create an estimate at the start to ensure the budget and schedule satisfy the project’s business case. Although the estimate still is not ideal, it is better than not having an estimate at all.
And please stop saying that we shouldn’t estimate at all. Just stop, you are hurting innocent people. 🙂
I believe it is disingenuous to propose something as a principle that you know applies to a very small minority of projects. We need more ideas to bridge this reality gap of executing project in an Agile way and requiring project estimates up front to validate a business case.
I’ve read many posting on the perils of converting User Story Points to User Stories Hours and how a common conversion factor may not be feasible due to the difference confidence factors for each User Story size. I believe this is a red herring. The postings discuss how the conversion factor will be more of less accurate for different stories of different sizes. I couldn’t agree more. And it doesn’t matter.
All we are concerned with is if the conversion factor holds on average for the entire project. I’m not expecting it or requiring it to apply to individual estimates. I’m not holding individuals accountable for User Story Hours for each User Story, as I know these items will follow a Normal Distribution. (In fact the studies I have seen have prototypical Standard Distributions. An excellent post by Mike Treadway illustrating this can be found here: http://the-tread-way.blogspot.com/2011/04/how-many-hours-are-in-story-point.html)
Let’s look at the data Mike Treadway compiled:
|User Story Points||Average Estimated Hours||Average Actual Hours|
In this example,converting to and using Estimated Story Point Hours results in an 11.3% difference between estimates and actuals. This is very much in line with the 10-15% project contingency commonly quoted. So it appears that this method is not introducing additional inaccuracy. Please note I am not saying you don’t continue to manage in an agile way monitoring velocity and adapting as you execute. But this would allow for a checkpoint before you start your project that your schedule and budget are feasible. Without this I fear we may give Agile a bad reputation by starting projects that should not have even been started because they did not satisfy the business case.
I also realize that the User Story Hour Velocity will change from iteration to iteration. This factor does not take away from the value of converting to User Story Hours. Trying to use this argument is another partisan point where the argument implies we are going to use the conversion factor to manage in a traditional manner. We are not. We are using the User Story Hours to only plan and manage in an Agile manner. The important factor for both User Story Points and User Story Hours is that we manage by Iterations and at a higher level of abstraction so the individual deviations in estimates are expected and do not matter.
So given the fact that we need to convert to User Story Hours to validate the budget and schedule before we start, why wouldn’t we use them in addition to User Story Points as we manage iterations? I can’t think of any reason but I’d like to hear opposing viewpoints.
Two Final Points
1) The conversion to User Story Hours is done by the team and changes are only made by the team. There is no Project Management influence to modify the hours. This is absolutely critical.
2) I’ve heard the comment that providing estimates in hours will affect the developers doing the User Stories and they may alter their behaviour and deliverables. I believe User Story Hours is merely another piece of information that developers should have when undertaking a story. I think having the User Story Hours will let the developer provide feedback immediately if the estimate is incorrect and allow for immediate adaptation. Otherwise we may only find out that our velocity is an issue a week into an iteration. Why wouldn’t we want to know if there are concerns immediately? Developers I have polled have stated they would like to know the initial hour estimate to be able to possibly raise a red flag early. They stated User Story Points do not allow this to be done as easily.
I’d like to hear why people are averse to using User Story Hours and communicating User Story Hours to developers. I’m sure there are some factors I have not discussed.
I think the benefits realized by developers knowing the initial estimates outweigh the dangers of the developers working to those estimates. A true Agile environment and team can allow User Story Hours to be used for good rather than evil.
Please let me know if you find this interesting and whether you would like to see a subsequent post on how you can convert from User Story Points to User Story Hours.