A case for Contingency #NoEstimates

It seems like everyone has a different view of the role that contingency should play on projects. While some recommend removing it so as to not inflate the cost of features, others feel it is a critical element in the estimation of the project. Some people feel it is best to hide it away in the features themselves, while others like it explicitly separated for clear communication.

One thing is clear to me after many projects that have gone well and poorly. Contingency is required. Contingency is required to portray a realistic image of how the project will go. While many people try to remove Contingency to not inflate the cost of a project, it actually reflects reality.

Contingency is the cost factor of likely issues that the project will hit based upon the experience of the team and company executing the project. To remove contingency would be ignoring the past.

Even the noble attempt to remove contingency and work solely with feature velocity so that the team and the client can jointly work on the prioritization of features is misplaced. Removing that contingency will change the decisions made by the business early on. Without the contingency , the client may feel that they will get more done and will add more scope. Then once issues start being encountered, it becomes very apparent that the client won’t get everything they need because they added items early on based on the perceived velocity and lack of need of contingency.

For example, Let us say we have 100 features that we need to accomplish and based on the first month we determine we appear to have a velocity of 20 features a month. The project is required to go live in 6 months. This appears to be a solid 5 month project. Early on in the project we encounter 12 features that the client would also like to have. So we add them since we have an extra month. What we find a couple of months in is our velocity is really 16 not 20. (as we encounter some more difficult features) Now we determine we will only be able to complete 96 out of the 112 features we have. Although we are working in priority sequence, there probably are some features that the client can’t go live without.

We have a pickle. All because we didn’t use contingency to reserve budget and schedule based on realities.

To me No Estimates sometimes feels like doing Physics problems without friction. While the equations hold and produce logical results, they don’t reflect reality. And once you add friction to the equations, the change in the results can be quite dramatic. If No Estimates work for you, then you are probably more in the product rather than project space and I’m envious. 🙂

But for the most part most of us need to deal with friction.

While No Estimates is intriguing, the removal of the analysis and incorporation of contingency is dangerous and can be deadly. While past velocity is an important factor, future planning and learning from the past is equally important. To manage mathematically based on past history is overlooking one very key factor – prior projects and experience. As we know most of the issues happen later on the projects. Contingency isn’t linear. Planning with only velocity requires contingency to be linear. In reality, we adapt to early velocity and make scope decisions based on that. They when issues hit that require contingency, it is no longer there as we have made decisions based upon what we think we can do based on past velocity.

Summary

Contingency isn’t evil or bad. It doesn’t inflate the cost of a project. If anything, it is the one thing that takes a paper plan and makes it real. Leave it out at your own peril.

The Selfish #Agilist

As I was finishing my last post on what I termed Agile Sketches, there was a second point that occurred to me that I thought was relevant and probably deserved a separate post.

I see a lot of recent comments and posts that propose we provide no estimates, no deadlines and do no planning. While some of these discussions are valuable and the questions they ask are important. I wonder if Agilists are becoming selfish.

It seems like we are proposing to move away from any practices in which we have to provide predictability and where we could be wrong. In their place are principles and practices that limit any predictability so that we will not likely be wrong.

Estimates are difficult? No problem lets not estimate at all…

Agreeing to a schedule and deadlines can be difficult? No problem we won’t have deadlines at all…

The Problem

The proponents of no estimates and no deadlines seem to be focusing on the requirements of developers and the development process at the expense of the clients. Have we become focused on what processes are best for us or the client? I agree that actions over planning will typically return the most value, but eventually there is a tipping point where the total elimination of these practices can have a detrimental effect. We always have to remember that value is always defined by the client. Do we have clients asking us for no estimates and no deadlines?

The Psychology

It may surprise you that I agree with the psychology of no estimates and no deadlines. Yes, it is difficult to have estimates and deadlines. Yes  both estimates and deadlines can have adverse effects on individual and team dynamics. But we as professionals need to not just eliminate things that are difficult, but to learn how to manage them so we return the most value for our clients and teams. Most importantly, we must always support the practices that deliver the most value to the clients.

Being Brave

To avoid becoming Selfish Agilists we need to rediscover our bravery for being wrong. I sometimes hear that people refuse to give estimates because they would be lying to the clients. At this point we need to remember the immortal words of George Constanza.

“It is not lying if you believe it to be true”

Yes, when we give estimates we will be wrong, but through the act of estimating we can hone our estimation skills and be more and more accurate. We are not lying, we are giving our expert opinion and we will be wrong. And that is ok.

Our team mates will also give estimates and deadlines and they will be wrong… and that is ok as well…

But the act of being brave and putting our opinions forward are what we do as professionals. It isn’t easy and managing the expectations once these estimates are known is also not easy at all.

I see a long continuum of estimating solutions.

At one end you have the detailed estimates with a work breakdown structure and bad management like micro-managing team members. Due to the excessive planning and management you deliver less value to the client.

At the other end you have no estimates and limited predictability. This also delivers less value to the client who requires some predictability.

But in the middle, you can have environment where people give their honest opinions on what it will take to get to end of job. Clients get the predictability that maximizes value. The consequence is that more discussions are required when new things are uncovered and learned and expectations need to be managed. It is never anyone’s fault, everyone is doing the best they can and we are constantly learning. This environment takes the right type of leaders, teammates, and clients. And managing the relationship is not easy. Like all good relationships it takes a lot of hard work.

But are we about doing whatever is easiest or what is in the client’s best interest?

Top Three Rules of #Agile Software Estimation

Yep, Estimating is hard. It isn’t easy and it isn’t without peril. Unfortunately it is a fact on 95% of the projects we work on. So given that, I’m not sure that telling people not to estimate is productive. There are a lot of misinformation about estimating though. Not all estimates are evil and not all are a waste of time. Here are the three estimating rules that I follow that have served me well.

1. Estimate Iteratively

I can’t figure out for the life of me why people who discuss Agile projects assume that if you estimate you must be BEUF. (Big Estimate Up Front) I always estimate iteratively with the estimate getting more refined as I go along. Usually the stages are:

Proposal Estimate – At the very start of the process when you don’t know much

Plan Estimate – Creating an executable plan and estimate before you start after you know more about scope

Execution Estimate – Re-planning with the final team just before execution

Iteration Estimate – Re-planning before each Iteration

With an iterative approach to estimating you can provide a line of sight to the clients as to what they can expect and you can also communicate how the whole project will evolve. Even the estimates.

2. Estimate with the team

This is a no-brainer. Always, always, always estimate with the team. If your team composition changes, I would re-estimate with the new team. Friends don’t let friends estimate alone. It is as much a learning experience as anything else.

3. Estimate the Minimum Viable Product

This is probably the most important rule. Let me be clear. You have no business starting or being involved on a Software Development project if you can’t estimate that you can at least deliver the Minimum Viable Product. (MVP) Yes, I know it is hard and difficult and fraught with errors, but an estimation based on years of experience if better than no opinion at all.  It can save wasting money on a project that had no hope of success. And it doesn’t need to be a very detailed estimate at the start, just a high-level estimate that confirms the team believes it can be done. (see rule #1)

Some Estimating Falsehoods

There are some common Estimating Falsehoods out there. I have taken these from the Blog post on estimating by Matt Rogish that takes the point that estimating is harmful.

Here we go:

“In really terrible places to work, someone other than the developer will actually do the estimating. This estimate will then be given to the developer as an implicit (or at especially evil places, explicit) time not to exceed. This is such a depressing situation I cannot dare think of it further.”Correct. This is really a problem with bad leadership rather than estimating. Don’t throw the baby out with the bath water.

“Most estimation is drawn from past experience. What if the developer doesn’t have this experience? How can they estimate? This is where things get tricky. And by tricky, I mean “completely made up.” Sure, for things that have been done before by someone the developer can draw some parallels and make a guesstimate. But what about the stuff that has never been done before (presumably, the secret-sauce that makes you money)? Usually you just take a Wild-Ass Guess (WAG-estimation) and hope you’re not wrong. Given the rarity of being punished for under-promising and over-delivering this WAG tends to be a massive over-estimation.”This is a common Falsehood. As mentioned in Rule 2, no one estimates alone. Senior developers help the less experienced and share their wisdom. This point also again highlights just plain old bad management.

“Completely spec out the entire system ahead of time, spend a lot of time researching and estimating the problem, determine project dependencies, and you can determine the “finish date” before writing a line of code! It’s seductive and taps into the part of our brain that craves order and dependability.”See Rule 1 – Estimate Iteratively. I can’t imagine any Agile professional would fall into this line of thinking.

“If the value of software we’re writing is so low, is it worth being written? If the project has such tight time constraints that a schedule slip will doom the project then you’re in a world of hurt. (The solution is agile: work on the most important things first, have an always-working project, and cut scope to hit a date.)

This exposes a major weakness in most software companies: the inability to determine project value. Given a project of unknown value, the conservative business will then attempt to minimize cost. Unfortunately as we’ll see, cost minimization ends up becoming very expensive in the long run.”This is a strange point to me. We as software developers can’t estimate our projects, but YOU the business need to estimate the value. Well we can’t have it both ways. I agree Agile is the solution, IF we estimate we can complete the Minimum Viable Product within the current budget. See Rule #3.

I’m not entirely sure what this means, but I’ve never seen the folks who are writing the specs, requirements, etc. ever being asked to estimate how long each spec will take to write. Or have the CEO give a date when the company will hit some arbitrary metric, although see the perversity in public company earnings estimates and reporting.”I’ve seen both on all the projects and companies I’ve worked for. CEO bonuses are commonly tied to unrealistic estimates on an arbitrary timeframe.

“Ultimately companies require estimation because they don’t trust anyone. They don’t trust developers to deliver results. They don’t trust mid-management to motivate their reports. The underlying reality is that the folks “in charge” are worried that everyone is going to rip off the company and only by holding people to arbitrary deadlines are they assured stuff will get done.

Committing to a date allows them to sleep better at night and shifts the blame from them to the folks on the front lines. “Wow. If anyone works at a place like this, leave. But don’t blame the estimates. If you don’t produce estimates, this company will just find another way to pull a fast one on you.

“Estimation is ultimately a futile effort. Software, more or less, is like writing poetry. Or solving mathematical proofs. It takes as long as it takes, and it’s done when it’s done. Perfect, or imperfect, estimation won’t change how long it takes to complete the work. If that sounds horrible to you, then go do something else.”I could not disagree more. It is correct that estimation won’t change the amount of work it will take to complete something, but that isn’t the point. The point is about giving some imperfect information to the client to help him/her make business decisions. You remember the client right? The one who is going to pay the bills and we are trying to deliver value to right? I would agree that truly unique software solutions are like writing Poetry. I was on a R&D project recently and the estimating was challenging to say the least. But 95% of Software Development projects are following semi-established patterns. The projects are not routine, but nor are they like creating something fundamentally new all the time.

“Estimation actually slows down development because estimation isn’t free. The developer is incentivised to take as long as possible estimating in an effort to not get beaten when they miss the estimate. Why waste that time when the metric the developer is attempting to hit ultimately generates negative value?”See Rule 1 on estimating iteratively and this is just an example of bad management once again.

Summary

Estimation is difficult, never correct, and is fraught with danger. But if you are like me, the vast majority of my clients require them. So the question is not why can’t we stop doing estimates?,  but rather how can we do them better?

In almost all cases, the problems with estimating can be traced to bad leadership on management. That is where I believe our focus should be.

Estimation, Story Points, Hours, and the Theory of Constraints

There was another interesting discussion on estimating and the use of Story Points versus Hours on Twitter between myself, Steve Rogalsky, D’Arcy Lussier, Mike Iwasiow, David Alpert, and a few others.

Steve quoted the book Beyond the Goal by Eliyahu M. Goldratt and blogged his opinion on using hours versus Story Points.

Suffice it to say, the opinions were widely varied and informative. I’ve made no secret of my opinion that although I use Story Points I also use hours for projects I am part of. But I this discussion enlightened me on some new ideas and concepts I was not aware of before. Before we start that discussion, I’d like to review what I believe the benefits are of Relative Estimating/Use of Story Points/Macro Estimating/Macro Tracking first. Lets just term the process Agile Estimating for brevity. 🙂

The Value of Agile Estimating

  1. Relative Estimating allows the estimating process to better leverage the human ability to estimate relatively instead of using absolute, discrete numbers.
  2. Use of Story Points allows for the estimates to be generated without internal biases as to what is a 1 day task, 2 day task, or a week task. This helps to make the Relative Estimates even more accurate.
  3. Macro estimating refers to the estimates being created by the entire group in a session that also collaboratively defines the requirements. Actually this type of group estimating and shared estimates has been around for almost 40 years and was initially termed Wideband-Delphi estimating. Mike Cohn popularized it in the Agile circles and made the process more collaborative, but it has remained essentially the same process.
  4. Macro Tracking refers to the progress of the project being tracked primarily through the project team’s iteration velocity. The tracking at a task level that is typically all the rage in a Work Breakdown Structure is not done. All tracking and subsequent planning is done through the use of the project’s iteration velocity.

I don’t believe anyone would argue against the four types of value listed above? I think all four of these types of value in the Agile Estimating practices return better estimates and better projects.

The vocal proponents of Story Points would also state that Story Points are required for both Macro Estimating and Macro Tracking. I would propose that this is incorrect, I can perform Macro Estimating and Macro Tracking using hours just as well as Story Points. If you don’t do Macro Estimating and Macro Tracking that isn’t the fault of the lack of Story Points, that is just bad project leadership.

Theory of Constraints

The book Beyond the Goal by Eliyahu M. Goldratt had proposed that estimating hours was local optimization. (I must admit I don’t fully understand the concept and must continue my reading on the Theory of Constraints)

a few of the quotes Steve had on his blog from Eliyahu M. Goldratt I believe illustrates that we are just talking about bad project leadership:

“The way to ensure that the project will finish on time is to ensure that every task will complete on time.”

This simply is an incorrect statement and illustrates a lack of macro thinking. Of course I can have a project complete on time without every task finishes on time. A project will be a continuum of tasks that finish early, on-time, and late. If you estimate and execute well, more time will be realized by tasks finishing early, than realized by those tasks finishing late.

“Suppose you are this person and you are asked how much time will this task take? You are extremely reluctant to give any number. And when you give that number the Project Manager will try to squeeze it. Why are you reluctant? Because you know that the number you give is an estimate but that it will be converted to a commitment – because the project needs to finish on time.”

This again illustrates bad project leadership where the Developer is the protagonist and is proposing estimates and the antagonist Project Manager is forcing the squeezing of the estimates and also converting them to task-based, or micro-level, commitments. Let’s be honest here, the problem is not that the estimates are made in hours. The problem is that the project leadership is wanting and the estimates are being tracked at far too low a level.

And finally…

“Due to the result that our local optimization turns any estimate into a commitment, we have developed the habit of giving estimates which includes in it the safety. If murphy (i.e. Murphy’s Law) doesn’t hit, we waste the time rather than giving it to the next link in the chain or project. If we do give it to the next link then we are declared as exaggerating and next time they won’t give us the extra time. So look what a horrible situation this is. And this… is what kills the project. This is totally idiotic”.”

I’m not sure calling something idiotic advances the discussion in any way, shape, or form. But maybe that is just me. 🙂 Seriously, I’m not sure beating the estimate on one task results in that excess time being wasted and not being applied to the next link in the chain or project? If you manage at the Macro level and one task comes in under, you realize that you have gained some buffer to offset future overages. If the sponsors of the projects view this assignment of extra time to the next link(task) as exaggerating, then the project leadership again has been found wanting. It is the project leadership’s responsibility to explain the reality of estimates. This again has nothing to do with estimates being in hours. Just good old-fashioned bad project leadership.

So do we enforce the abstraction of Story Points to address bad project leadership or just address the bad project leadership?

That said, I’d never recommend not using Story Points and Planning Poker sessions. Relative Estimating and Story Points are invaluable.

But to state that you can only do Macro Estimating and Macro Tracking on a project if you use Story Points is incorrect. I’ve converted Story Points to hours on many projects and been very successful.

And Finally….

The vast majority of us we need to find a way to work with hours as the number of clients and projects are not comfortable not having an estimate and schedule. Even if we all agree that estimating and estimating hours in particular causes problems, proposing to not estimate in hours is just not realistic. And you can get all the benefits of Agile Estimating using hours as well.

P.S. When we didn’t translate Story Points to hours, the developers I worked with did the translation themselves to ensure their progress was on track. When I’ve asked developers if they would like to also see the conversion to hours, they have all asked for it.

 

 

 

 

Top 10 practices for estimating Agile projects successfully

I’ve had multiple posts in the past that provided my passionate opinion on why I believe we should estimate Agile projects. (Both from the perspective of the client and the team) But rather than get into that discussion once again, I thought it would be more valuable if I shared the 10 practices that we have used over the past 6 years that have allowed our teams to successfully estimate Agile projects.

Context

I work for a company that bids competitively on projects against other companies. In almost all situations we have to create estimates to provide to the client or else we would not be successful in winning the business. So estimating really is a fact of life for the type of business we are in. Although it might be nice to proceed on a project without an upfront estimate, that really isn’t a luxury we are provided. I believe we are in a similar boat to the majority of people out there.

The Practices

Over the past 20 years I have honed my estimating skills with the experiences on projects. Over the past 6 years, this estimating has been predominantly for Agile projects. Although we have not been perfect in my estimating, I am quite proud of our track record and feel that we have been correct far more than we have been incorrect. Where we have been incorrect, we have augmented the estimating process so we won’t be incorrect in that way again.

The practices are not in order of priority as I had a real problem trying to do that as I think they are all important. So without further ado:

1) Remember to estimate Project Management and Technical Management

We frequently ran across missing these factors on early estimating attempts. This is usually done by everyone as they work on estimating as the work is easy to forget. It is a critical part of the estimate to ensure that these estimates are included so that we don’t scrimp on these tasks and cause larger issues for the project.

2) Don’t plan on your Technical or Application Architects being any more than 20% on construction tasks

This one continues to plague projects I estimate. No matter what the work is, the Architects will always want to work on their fair share of construction tasks. It is a noble desire and will return great benefit if they can work more on construction tasks. But I wouldn’t bet on it. In fact, I would even consider reducing this to zero depending on team size and complexity of the solution. If the Architects can work on more construction tasks, just consider that a bonus.

3) Remember to estimate for meetings and other factors in your schedule estimate

OK, now you have the estimates so lets just plan the hours out on the calendar right? Not quite. Remember to reserve some time every day for meetings, stand ups, and other non-construction activities. In addition, remember to plan for at least some time for vacations, holidays, and sick time. Use your statistics from your company to generate a rough factor for the project. It will just reserve a percentage of your schedule at the start, but then you will have the wiggle room when people start dropping like flies during flu season.

4) Hold a risk workshop and include the weighted estimates in the overall estimate

Mention Risk Workshops and most people yawn before you have completed the sentence. But they have great value. They are a great team building exercise at the start of the project as well. But ultimately, the output from the Risk Workshop should be used to generate an estimate and apply that estimate to the overall project estimate. By calculating the impact and probability of Risks occurring you will generate an estimate of the Risks that will likely occur. My experience is that if you don’t include these estimate, you will frequently address these risks occurring with Change Requests. The project can be much smoother if the estimate are included up front and you don’t surprise the clients. Yes, we won’t know all the Risks at the start, but it is better than assuming there are no risks at all. As Dr. Phil says “How is not estimating any Risks at all working for you?”

5) Estimate at multiple levels and triangulate

Probably the most important practice. We never just create one estimate. We typically estimate bottom-up (object level), deliverable (mid level), and top-down (schedule level). We use metrics for the bottom up and mid level estimates. (how long does a simple CRUD screen take, moderate report, etc..) Then we compare all three estimates and rationalize the differences. It is amazing how this rationalization uncovers estimating oversights.

We also then use Planning Poker sessions as we execute the project to confirm requirements and plan our iterations.

6) Proactively get information required to provide an estimate

Many times there are factors that may cause the estimate to change greatly. (Data Conversions, Legacy interfaces, unknown technology) Ask these questions early on and factor them into your estimates.

7) Say No when you don’t have information to provide an estimate. But be proactive and say what you need to provide an estimate.

If you don’t have the required information to properly estimate, then don’t. 🙂 But don’t just say you can’t estimate. Provide the details of what information you need to be able to provide an estimate. I know on past projects we have guessed many times on estimates where in hindsight, the client would have gladly provided all the information we required.

8) Track estimates and actuals and use the metrics for future estimating. (both within the project and for future projects)

Yep. We need to track estimates. Sorry. These metrics will then be used to determine on average how long certain objects will take and the percentage that certain areas of the project will require. (Like Project Management!)

We also hold retrospectives and review iteration estimates and actuals so that we can revise our estimates as soon as possible within the project and taken them into consideration for future iterations.

9) Factor educated contingency into your estimates.

We have 17 contingency factors (and counting) that are weighted and applied. (at the team’s discretion) If nothing else, it is a least a checkpoint for the team to think about whether certain situations exist that may affect estimating. (new team? data conversion? new technology? complex interfaces?) Don’t just apply a standard 10% blindly to estimates. It has to be based on reality.

Like Risk, these contingency factors should then we applied to the estimate. I would also recommend that you don’t show what part of the estimate is Risk versus Contingency versus base estimate. Clients will then say that the contingencies won’t happen so just subtract that estimate. <sigh> The estimate is holistic.

10) Execute in a manner that respects that estimates are not actuals. They will be incorrect.

And after all the estimates are done, don’t execute in a manner that makes Captain Bligh look like a level fellow. Don’t micro-manage, realize that these were only estimates – some will be higher and some will be lower. Very rarely will any be totally correct. Let the clients trade requirements and stories as priorities change! You could not possibly know all the scope up front. But also don’t just agree to add new scope. A lot of issues about estimating do come down to good old-fashioned scope control. The estimate should be a guide and a placeholder for a discussion, not a roadmap.

Summary

My experience is that is we do all of these practices, our estimates will still be wrong. But much less wrong. More importantly, these practices help us to discover better ways of estimating. Discovering better ways doesn’t only apply to development practices, but all project practices. Ultimately these estimates also help the team members to feel better about their work as well. No one like to miss estimates and managing projects in a manner that respects that the estimates are not actuals will alleviate many of the negative connotations with estimates.

<SoapboxMode>

The correct thing is to fix how the estimates are managed, not to stop doing estimates.

</SoapboxMode>

P.S. I forgot one! Here is a bonus practice 11) Estimate that you will be wrong – remember to estimate for refactoring, defect fixing, and some rework. We are all good, but not that good. 🙂

 

#agile2011 Monday night thought – Does using only #Velocity limit the opportunities for #acceleration?

I just finished reading an excellent Blog entry on transitioning from traditional estimating to Story Point estimating by Ilan Goldstein. You can find the blog post at the following link:

transitioning-from-timebased-to-relative-estimation

The article walks through the estimation process used during the transition there are three excellent points.

1. I loved the idea about being above-board and listing the conversion table between Story Points and the range of hours typically associated with those Story Points. Let’s face it, everyone is doing the mental math and the only thing worse than people doing the calculation, is everyone using a different version of the table in their own head to do the calculation. So let’s all just agree what that conversion table is. Excellent.

2. Taking previous features developed and incorporating the effort they took into the estimation process is wonderful. I have not heard mention often enough about tracking and comparing estimates versus actuals on User Stories so that our estimating get better each Iteration. This process of creating reference User Stories in points by converting the hours they actually took helps to leverage the previous project work to make the estimating of future work as accurate as possible. Brilliant. The additional step to then throw away the hours per feature and use on the Story Points going forward is ideal.

3. The point to then track both the time completed and the time remaining on active User Stories ensures that we are continuing to learn and get better with our estimating as the project proceeds.

I think these three points just add the value of planning poker and makes the entire estimation process the best it can possibly be.

Why do I like these ideas and processes so much and what does that have to do with the title of this Blog?

It has always been a personal belief of mine that setting completion expectations for a project team with only a collective measure (velocity) for the entire iteration is not optimal. Using relative estimating to produce accurate estimates and using velocity to plan for the average progress of a project is fine, but even Mike Cohn stated he does not recommend using Story Points for Sprint Planning.

why-i-dont-use-story-points-for-sprint-planning

Mike Cohn mentions that velocity is not a useful short-term predictor and I agree. I would also add that not only is it not a useful short-term predictor, but also can possibly not set the proper short-term expectations for User Story completion times. When used, Story Points by their nature allow for a separation from day-to-day complexities that can cause inaccurate estimates. When used, Story Points by their nature can also allow for a separation from day-to-day expectations. For example:

  • How long should a 1,3,5, or 8 point story take?
  • Do we need to wait until the end of an iteration to react to a story that is taking longer than thought?
  • Would knowing that a story should take 2 days on average help to alert the developer of potential issues?
  • Would knowing that a story is going to take longer than expected allow for the splitting of a story in the middle of an iteration and the ability to deliver more ‘Done’ work in the iteration?

If I am for the most part only reviewing velocity at the end of the iteration and not at some level on each User Story during the iteration, am I limiting my opportunities for acceleration?  (Acceleration being the rate I can change my velocity.)

Thoughts?

Project Estimates versus Targets

I was going to have another Blog post that talked about Agile estimating, but I think a Blog about estimates versus targets should be done first. In short, I think the confusion and mingling of the terms have led to many ‘perceived’ project failures. As usual, let’s start at the start. 🙂

Definitions

Estimate – An opinion and educated guess as to the amount of work, people, and ultimately duration. An estimate can only considered an estimate if practitioners have created it. Ideally, an estimate is only an estimate if the project team has created it. But it the consulting world, this is sometimes impossible. The most I can hope for is that skilled and competent technical resources have created the estimate, even if they are not going to be on the project team.

Target – A opinion and uneducated guess as to the amount of work, people, and ultimately duration. A target is different from an estimate in that it is usually created by non-Practitioners. The target is usually created to put a stake in the ground to reflect an Enterprise objective or goal, Regulatory requirements or the criteria required so that a business case holds.

The Problem

The problem that I have seen over and over again in the Information Technology industry is the inter-mingling of targets and estimates. Many times we ourselves are the problem. In an effort to be non-confrontational we accept targets as now the project estimate. Subsequently, the corporation wonders why our Software Development projects never meet estimates. In my experience, Software Development teams meet estimates pretty well, where we struggle is meeting targets. Is this any surprise? I’m sure if I created a target for the annual budgeting process for a large corporation I also would be way off because I can’t appreciate all the processes and factors that need to be considered. (Hey, an analogy I can use!) 🙂

I have usually found that even if a project team acquiesces to a target provided/imposed, the project itself usually comes in at about the estimate the project team created. Typically, the project team can estimate accurately unless the project is in a totally new technology or domain.

But what If I am not given a choice?

This is a situation that probably occurs more often that not. I would estimate that most of the times project teams adopt a target as an estimate, they do so because of direction from Senior Management. So what can we do?

I would recommend the following approach:

  1. Education – Start diligent and professional educational communications with Senior Management. Ensure they become aware of the differences between targets and estimates.
  2. Language – Use the language and terminology in all discussions and documentation. This is part of the education plan to communicate the difference between the two terms.
  3. Track both – Even if the project is executing according to an imposed target, track the project actuals against both the target and estimate. Evaluate after the project is complete and build up data from multiple projects. Don’t use this data to support an unprofessional ‘I told you so’ communication. This is very bad form. But this data will be interesting to present in a professional way. Executives don’t like surprises and being called out on the carpet when their initiatives always exceed the plan. Presenting these figures in a professional and respectful manner may convince executives to accept estimates rather than targets. (when possible)
  4. Use estimates internally – Even if targets have been imposed on your team, I would recommend to still create estimates and then use these estimates to track progress with your team. Tracking with targets is unfair to the team. It will create an atmosphere of mistrust and unhappiness. The team will feel that they are constantly failing. The team wants to feel they are doing a good job. Tracking by estimates gives the best opportunity to do that and to measure progress.  Ideally create these estimates with Planning Poker and manage in an Agile way, but that is a topic for another Blog.

Decision Point

These are strategies to move the company and Senior Management in the right direction to have successful projects. It is not a short-term strategy will take multiple projects. If Senior Management chooses to not listen after you have been able to gather the information and present your case, every individual has to decide if the culture of the company is right for them.

But I believe that unless we go through the process to inform Senior Management, we are also part of the problem. The worst thing we can do is just to accept the targets as estimates.

User Story Points and User Story Hours

This Blog post may be somewhat controversial, but I really think it provides a viewpoint that I don’t normally see represented. I urge you to read the entire post before jumping to conclusions. Here goes:

The Politics

I’ve read many a good post on estimating Agile projects, but I must admit the discussion about estimating via User Story Points versus User Story Hours seems to suffer from partisan politics. If someone believes in User Story Points, they paint the only alternative as a bleak Orwellian future of accounting for time in 5 minutes increments and shock therapy for the failure of missing estimates. They usually also assume that if you don’t buy in fully to User Story Points, you also don’t believe in Agile in general, Planning Poker, and are an avid fan of Waterfall. Nothing could be further from the truth.

What I would like to have is a bi-partisan discussion of the pros and cons of different estimating approaches.

Disclaimer

Before I start I would like to state that I believe that User Stories, Planning Poker and Story Point Estimating should be done on each and every project. What I am discussing and proposing here is not about not doing Planning Poker and Story Point Estimating, but rather discussing whether the estimating should stop there or continue with other additional methods.

The Continuum

There is a continuum of approaches for estimating. This is illustrated in the diagram below:

When discussing the pros and cons of Story Point Estimating, we are usually limiting our discussions to just the two extremes. I would like to propose a hybrid approach as well.

Agile The Agile approach is the one that is most commonly proposed in the Agile literature. This approach proposes estimating by User Story Points and these estimates are typically generated in a Planning Poker Session(s). The Planning is then done by planning the Iterations based on the number of User Stories and their associated points.Success is then usually judged by being able to achieve a stable User Story Point Velocity from which to execute the remainder of the project in iterations. This Velocity is used in an iterative and ongoing basis to refine the plan for subsequent iterations based on past results.
Traditional The Traditional approach is defined by creating very detailed task level estimates in hours. These estimates may be done by the team, but usually are done without the team or with a subset of the team. These estimates are usually created without the input of developers. The Planning is then done by creating a Work Breakdown Structure. The Work Breakdown Structure is then applied over a timeline or schedule to create the fully detailed Project Plan.Success is usually judged by the accuracy of the task estimates which will then translate into meeting the estimated schedule generated from the Work Breakdown Structure.
Hybrid The Hybrid approach proposes estimating by User Story Points and these estimates are typically generated in a Planning Poker Session(s). The Planning is then done by planning the Iterations based on the number of User Stories and their associated hours. The User Story Hours are created by converting the User Story Points into User Story Hours.Success is then usually judged by being able to achieve a stable User Story Point and Hour Velocity from which to execute the remainder of the project in iterations. The User Story Point Velocity is used in an iterative and ongoing basis to refine the plan for subsequent iterations based on past results. User Story Hours are provided to the owners of User Stories so they know what the original estimates of the User Stories were.

What are the pro and cons of each approach?

Well we all know the pitfalls of the Traditional approach, so let’s not even discuss that approach. 🙂 But why is the Hybrid approach compelling when compared to the Agile Approach? In fact, the Agile and Hybrid approach are very similar except for the fact that the planning is done in User Story Hours and the velocity is measured in User Story Points. In addition, User Story Hours are then provided to the owner of each User Story to provide a frame of reference as to the initial estimate. Let’s look at the pro and cons of Agile estimating concepts and how User Story Hours can help mitigate the cons.

Concept Pros Cons How do User Story Hours help?
User Story Points Minimizes the human element of having duration or hour perceptions influence estimates. May force the ‘slotting’ stories into non-ideal slots. (i.e. It is a 30, but all I have are 20 and 40 slots) Provides a second review once you have converted to User Story Hours to ensure the estimates are consistent and make sense.
Relative estimating Establish standard grouping and increase accuracy by relative estimating. Also leverages human ability to excel at relative estimating. Creates single point of failure. Base line estimate which all other estimates are relative to can create a ripple effect of estimate inaccuracy. This may take multiple iterations to correct. Remove single point of failure with reality check. Second review will ensure estimated hours make sense.
Planning poker Increases team buy in, shares estimating expertise and promotes team growth, helps to refine solution understanding, and increase estimating accuracy. None. Not Applicable. This concept is used in both the Agile and Hybrid approaches.
Velocity and Burn down Validate estimates and plan at a macro level. Not at an individual, user story or task level None. Not Applicable. This concept is used in both the Agile and Hybrid approaches.


Perhaps the most important benefits of User Story Hours are that they allow for the derivation of a project budget and schedule. One of my biggest challenges with the User Story Points approach is the concept that it will take two or three iterations to determine your velocity and then you can inform your client when you can complete the project and how much it will cost. Very few projects I have been a part of can get a blank cheque for two or three iterations and then confirm if the project is feasible and has business value. User Story Hours still allows you to have that check at the end of each iteration AND also create an estimate at the start to ensure the budget and schedule satisfy the project’s business case. Although the estimate still is not ideal, it is better than not having an estimate at all.

And please stop saying that we shouldn’t estimate at all. Just stop, you are hurting innocent people. 🙂

I believe it is disingenuous to propose something as a principle that you know applies to a very small minority of projects. We need more ideas to bridge this reality gap of executing project in an Agile way and requiring project estimates up front to validate a business case.

The Data

I’ve read many posting on the perils of converting User Story Points to User Stories Hours and how a common conversion factor may not be feasible due to the difference confidence factors for each User Story size. I believe this is a red herring. The postings discuss how the conversion factor will be more of less accurate for different stories of different sizes. I couldn’t agree more. And it doesn’t matter.

All we are concerned with is if the conversion factor holds on average for the entire project. I’m not expecting it or requiring it to apply to individual estimates. I’m not holding individuals accountable for User Story Hours for each User Story, as I know these items will follow a Normal Distribution. (In fact the studies I have seen have prototypical Standard Distributions. An excellent post by Mike Treadway illustrating this can be found here: http://the-tread-way.blogspot.com/2011/04/how-many-hours-are-in-story-point.html)

Let’s look at the data Mike Treadway compiled:

User Story Points Average Estimated Hours Average Actual Hours
1 8 6
2 5.5 4.5
3 5.67 5.33
5 5.4 5.8
8 4.75 4.375
Overall Average 5.864 5.201

In this example,converting to and using Estimated Story Point Hours results in an 11.3% difference between estimates and actuals. This is very much in line with the 10-15% project contingency commonly quoted. So it appears that this method is not introducing additional inaccuracy. Please note I am not saying you don’t continue to manage in an agile way monitoring velocity and adapting as you execute. But this would allow for a checkpoint before you start your project that your schedule and budget are feasible. Without this I fear we may give Agile a bad reputation by starting projects that should not have even been started because they did not satisfy the business case.

I also realize that the User Story Hour Velocity will change from iteration to iteration. This factor does not take away from the value of converting to User Story Hours. Trying to use this argument is another partisan point where the argument implies we are going to use the conversion factor to manage in a traditional manner. We are not. We are using the User Story Hours to only plan and manage in an Agile manner. The important factor for both User Story Points and User Story Hours is that we manage by Iterations and at a higher level of abstraction so the individual deviations in estimates are expected and do not matter.

So given the fact that we need to convert to User Story Hours to validate the budget and schedule before we start, why wouldn’t we use them in addition to User Story Points as we manage iterations? I can’t think of any reason but I’d like to hear opposing viewpoints.

Two Final Points

1)      The conversion to User Story Hours is done by the team and changes are only made by the team. There is no Project Management influence to modify the hours. This is absolutely critical.

2)      I’ve heard the comment that providing estimates in hours will affect the developers doing the User Stories and they may alter their behaviour and deliverables. I believe User Story Hours is merely another piece of information that developers should have when undertaking a story. I think having the User Story Hours will let the developer provide feedback immediately if the estimate is incorrect and allow for immediate adaptation. Otherwise we may only find out that our velocity is an issue a week into an iteration. Why wouldn’t we want to know if there are concerns immediately? Developers I have polled have stated they would like to know the initial hour estimate to be able to possibly raise a red flag early. They stated User Story Points do not allow this to be done as easily.

I’d like to hear why people are averse to using User Story Hours and communicating User Story Hours to developers. I’m sure there are some factors I have not discussed.

I think the benefits realized by developers knowing the initial estimates outweigh the dangers of the developers working to those estimates. A true Agile environment and team can allow User Story Hours to be used for good rather than evil.

Please let me know if you find this interesting and whether you would like to see a subsequent post on how you can convert from User Story Points to User Story Hours.

Agile Project Characteristics – Two Controversial Factors

In my last post I listed the Agile Dozen Characteristics and Practices that I felt were critical on all projects. In that list though, there were two characteristics that I did not expect myself and that may raise the eyebrows of Agile purists when mentioned. Before there is discussion on all the factors and I present the experience reports, I thought it would be of value to discuss the following two characteristics:

  • Developer to Designer Ratio
  • Team meeting estimates

1) Developer to Designer Ratio

Where to start on this one. The first immediately apparent item is that there is a difference between developers and designers. Yes, I would say that there is room for both of these roles and that I believe that perhaps all of the design and development activities can’t be pulled by any member of the team. I know this principle is tantamount to heresy in some of the Agile circles, but I believe that context is everything and there are no absolutes. Based upon the problem domain, team composition and solution complexity, there may be a requirement for a role that owns the solution vision at a very high level. Now this does not imply the creation of detailed requirements and architects and designers defining exactly what developers need to code, but merely implies that there is a role that provides the overall vision of the solution at a high level and then works with the entire team as they flesh out user stories to ensure that the solution remains cohesive and consistent. This designer can then also provide leadership in the architecting the automated test suite as well.

This has worked well on our projects. I’d really like to hear back from others on this concept…

2) Team meeting estimates

This was the other characteristic that I thought would be somewhat controversial. Now I firmly believe in Planning Poker and collaborative estimating techniques, but at the end of the day, no matter how the estimates have been created, a successful project must deliver to the estimates that they have made. 😐 Whether this is a Traditional Project where estimates are defined at the task level through to an Agile project where User Stories and tasks are defined in points and through planning poker, eventually the execution must somewhat match the plan. (now matter how defined or loose the plan is) If this continues to happen through many iterations, the clients will get concerned and possible business plans that depend on the iteration outcomes may be compromised. To demand 100% meeting of the estimates is not feasible, but if the project consistently deliver less than 50% of the plan then some action needs to be taken. (Which could be to just be less optimistic and committing to less)

What do people think of these two more traditional characteristics? I’d love to hear feedback!