Why I believe in Estimates #Agile #Science #NoEstimates

It became clear to me the other day that my belief in estimates was not due solely to my previous project experience. It was not due to any brain-washing or “boiling of frogs” that had occurred to me as I was on more traditional projects. My belief in estimates is deeply rooted in another belief I have. The belief of Science and the Scientific Method.

The Scientific Method

Wikipedia briefly describes the Scientific Method as the following:

“The scientific method is a body of techniques for investigating phenomena, acquiring new knowledge, or correcting and integrating previous knowledge.To be termed scientific, a method of inquiry must be based on empirical and measurable evidence subject to specific principles of reasoning. The Oxford English Dictionary defines the scientific method as: “a method or procedure that has characterized natural science since the 17th century, consisting in systematic observation, measurement, and experiment, and the formulation, testing, and modification of hypotheses.”

The chief characteristic which distinguishes the scientific method from other methods of acquiring knowledge is that scientists seek to let reality speak for itself,[discuss] supporting a theory when a theory’s predictions are confirmed and challenging a theory when its predictions prove false. Although procedures vary from one field of inquiry to another, identifiable features distinguish scientific inquiry from other methods of obtaining knowledge. Scientific researchers propose hypotheses as explanations of phenomena, and design experimental studies to test these hypotheses via predictions which can be derived from them. These steps must be repeatable, to guard against mistake or confusion in any particular experimenter. Theories that encompass wider domains of inquiry may bind many independently derived hypotheses together in a coherent, supportive structure. Theories, in turn, may help form new hypotheses or place groups of hypotheses into context.”

The Scientific Method is composed of 5 distinct steps:

Note: I am using a little bit of poetic license on the application of the Hypothesis step to a Software Development project. Please forgive me. 🙂

As Khan would say, “Let us Begin”

Formulation of a Question – The questions usually describes an explanation of an observation or an open-ended question for which an answer is desired? For example, “Why is the sky blue?” or “How long will it take to redevelop a financial re-balancing engine?”

Hypothesis – The hypothesis is a conjecture based upon the knowledge of formulating the question and past experience and expertise of the people involved. The Hypothesis can be very specific in the case of Scientific experiments (“The Sky is Blue because of the density of the Atmosphere”) or broad constraints and assumptions in the case of Software Development. (“The most efficient way to develop a financial re-balancing engine will be to have a team co-located with a dedicated client, using Agile methods, adding extra hours for learning new technology, and counting on only 6 hours a day”)

Prediction – The prediction involves determining the logical consequences of the Hypothesis. In the case of Software Development, if all the situations/assumptions hold in the Hypothesis, what would be the end result? How long would it take to redevelop a financial re-balancing engine? Usually the prediction is based upon past experiments and the skill and expertise of the team making the prediction.

Testing – This is an investigation of whether the real world behaves as predicted.

Analysis – This step involves reviewing the results and determining how to best explain the data. Why did we take two months longer to deliver the new financial re-balancing engine? How can we leverage this knowledge into the formulation of the next questions formulation? If the delay was caused by Regulatory Requirements, how can we ensure that information is taken into consideration in the future and become part of future Hypothesis?

Agile Estimation

Now before anyone paints me with the “You are asking for a detailed estimates and you will be micromanaging your team to those estimates in a Death-March” brush – I’ve always said those symptoms are due to bad management and not caused by estimates. What I am proposing is the minimal amount of estimates.

So why do I think estimates are important?

I believe there are a few problems caused by not doing any estimates:

1) We don’t know if a project has an acceptable Return on Investment before we have potentially wasted money. The only question will be how much money we waste. Many projects may only make sense from a ROI point of view  based upon the estimate. The Minimum Viable Product may be more that the Return on Investment allows for. In that case, we shouldn’t even start the project.

2) Estimates make you think through possible factors and can help refine the solution. When you need to create an estimate you do apply more rigor and this helps to harden the solution and minimize future rework.

But most importantly….

3) I have faith in the Scientific Method. It has driven and developed the society we live in. I find it ironic that in the Computer Science field we are now embracing not following the Scientific Method.

The Goal of the Scientific Method has always been to better understand reality and to make theories and predictions that become better and better at describing and predicting reality. The results of that analysis are then reviewed by their peers and used to make the future theories and predictions better. This feedback loop is crucial to science. Interestingly enough, by Agile proponents recommending No Estimates they have eliminated a feedback loop across projects that provide feedback on how we plan and estimate. Other Agile teams can/will encounter the same challenges that another team faced without a feedback mechanism that highlights divergence from an estimate or schedule. Without an estimate or schedule, the team may not even think of it as a divergence…

The questions may be one of scope.

1) If your primary concern is your current project then creating estimates may provide less value. After all, you are going to hit those issues anyway.

2) But if your primary concern is a corporation, enterprise, and industry – then estimates have much more value. Multiple projects that shouldn’t have been started can bankrupt a company, projects with accurate estimates can help to make sure the company works on the right projects. And for consulting companies, making the same mistakes on projects estimates can send clients scurrying to your competitors.


The Grand Unified Theory is a great example. It still isn’t complete and scientists are working to refine and improve the theories by reviewing past works and designing new theories. These past theories weren’t lies told  by Einstein, Hawking, and Penrose. (That is the language the No Estimation supporters frequently use) No, they have been a theory on reality. Just like estimates are a theory on the reality of a project. And yes they are incorrect. Can anyone think of a scientific theory that was proposed that was right the first time?

Maybe we should get Estimates a new Public Relations campaign and call them project theories.

But we do need them. F=M x A and a cure for Polio tells me so.

Author: Terry Bunio

Terry Bunio is passionate about his work as the Manager of the Project Management Office at the University of Manitoba. Terry oversees the governance on Information Technology projects to make sure the most important projects are being worked on in a consistent and effective way. Terry also provides leadership on the customized Project Methodology that is followed. The Project Methodology is a equal mix of Prince2, Agile, Traditional, and Business Value. Terry strives to bring Brutal Visibility, Eliminating Information islands, Right Sizing Documentation, Promoting Collaboration and Role-Based Non-Consensus, and short Feedback Loops to Minimize Inventory to the Agile Project Management Office. As a fan of pragmatic Agile, Terry always tries to determine if we can deliver value as soon as possible through iterations. As a practical Project Manager, Terry is known to challenge assumptions and strive to strike the balance between the theoretical and real world approaches for both Traditional and Agile approaches. Terry is a fan of AWE (Agile With Estimates), the Green Bay Packers, Winnipeg Jets, and asking why?

4 thoughts on “Why I believe in Estimates #Agile #Science #NoEstimates”

  1. It is my view that the argument that Estimates are “project theories” is fundamentally flawed.
    First of all Estimates are not experiments in most cases, they are blurted out at the request of a manager and never followed up again until it is too late.
    I am not in agreement that the analogy holds at all.

    Having said that, there are some good points in your blog post:
    1) estimation process can generate information that is useful to some people
    2) Estimates make you consider different design options (perhaps too early?)


  2. Vasco,

    Estimates as you describe them as example of bad management. If they are blurted out, they are not estimates, they a bad guesses and not allowed where we work. The are not fundamentally flawed and I’d suggest are considered so only when the underlying process of probabilistic estimating is not understood, either with intent, or lack of exposure to those principles. Start with the book below.

    If the estimates are not followed up, again “bad management.” Fix the bad management first.

    We make estimates every month for both the Estimate to Complete (ETC) and the Estimate at Completion (EAC) that are submitted to our customer and used internally for budgeting, resource planning and overall program performance forecasting.

    Estimates are in fact experiments. Using past performance, how can we best adjust our stochastic models to improve the confidence of future forecasts. See “Probability Methods for Cost Uncertainty Analysis: A Systems Engineering Perspective,” Paul Garvey, CRC Press. This includes, cost, schedule, and technical performance estimating techniques.

    These estimates are developed bottom up once a year and modeled with Reference Class Forecasting quarter using past performance. Stochastic networks (the integrated master schedule) and the correlation and auto correlation between “work packages” are used in a System Dynamic model for cost, schedule, and technical performance are developed that state the “probability of program success.”

    So the description in this post is EXACTLY what we do using the steps above, even if they are paraphrased a bit from our actual “Program Performance Management Process Directive.”

    All estimates are used in the “analysis of alternatives” early in the program – through the preliminary design review where the needed capabilities are baselined. From that point on, actual products are being built, integrated, or acquired and the estimates are then used to forecast where problems will appear in the future.

    Estimates are not about forecasting a future we want to achieve, we have that in the Master Plan and Schedule. Estimates are about forecasting where trouble will appear and how to void it.

    When the conversation starts with a “Dilbert boss” example, it pretty much ends the conversation in our domain with the suggestion – “don’t do stupid things on purpose.”


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.