10 Reasons Why Software Project Estimates Fail

Think about the web and software projects you’ve completed. How many were delivered on time and on budget? How many estimates were accurate? IT projects are notorious for over-running, and here are several reasons why it occurs…

1. The project is poorly scoped
How can you estimate time on a project when you don’t know what that project is? It’s rare to find a client who appreciates exactly how their system should work.

Almost every large project I’ve undertaken has requested “flexibility”. In other words, the client wants the system to handle anything they want at any future point in time — even though they have no idea what those features might be. Flexibility is not a requirement!

2. Development time is estimated by non-programmers
If you’re not a programmer, don’t guess at development times. A project is doomed the moment a manager writes their own fictional estimate. At best, they’ll be completely incorrect. At worst, the programmers will be tempted to prove them wrong.

3. Developer estimates are too optimistic
Developers think in terms of coding hours. Time passes quickly when you’re in the zone and it’s difficult to assess your own speed. Appreciating the speed of other developers is impossible.

Many developers are over-optimistic. They tend to forget the softer side of the development process, such as project management, collating requirements, discussions with colleagues, absences, PC problems, etc.

4. The project is not adequately dissected
Be wary if the development estimate for an individual feature exceeds a week. That chunk should be sub-divided further so the developer can analyze a complex problem in more detail.

5. Estimated time is used
Give a programmer 5 days to complete a task and it’ll take 5 days. Software development is infinitely variable and any code can be improved. If a developer takes 3 days to finish the task, they’ll spend the remaining time tweaking it or doing other activities.

Unfortunately, this results in a situation where estimates become the minimum number of development days. The actual delivery time can only get worse.

6. More developers != quicker development
A 100-day project will not be completed in 1 day by 100 developers. More people results in an exponential increase in complexity. See Why Larger Teams Do Not Result in Faster Development…

7. The project scope changes
This is perhaps the most irritating problem for a developer. A feature is changed or added because customer X has requested it or the CEO thinks it’s a cool thing to do.

Is the impact of that new feature documented?…

8. Estimates are fixed
Estimates should be continually assessed and updated as the system development progresses. Programmers often believe they can make up lost time — it rarely happens.

9. Testing time is forgotten
It’s impossible for a developer to adequately test their own code. They know how it should work, so they consciously or sub-consciously test in a specific way. In general, you can expect to spend another 50% of the development time on testing and debugging.

10. Estimates are taken too literally
Non-programmers rarely appreciate the complexity of software development yet few businesses plan for schedule slippages. The project often sits at the bottom of a huge unstable tower of other activities, such as literature printing, marketing, distribution, etc.

Development hold-ups can cause a costly chain reaction of delays. Unfortunately, it becomes easy to blame the programmer at the bottom of the pile. That’s doesn’t bode well for future projects — the programmer will either refuse to provide estimates or inflate them dramatically.

Have you encountered other reasons why project estimates fail?

Free book: Jump Start HTML5 Basics

Grab a free copy of one our latest ebooks! Packed with hints and tips on HTML5's most powerful new features.

  • martinduys

    Nice post; but the real question for me is what do you do about this? How do you solve these problems? Given all these things happen, how do you allow for them in a way that you can sell to the client?

    • http://www.optimalworks.net/ Craig Buckler

      That’s the million-dollar question! Many companies use agile development methods, but I don’t think they solve the issues — they often result in a series of smaller over-runs.

      Perhaps the first step is to admit that estimates are just that: estimates. They are not a delivery date guarantee.

      • Cornel Alexa

        Golden word Craig!

        They are just “Estimates” ! Basically more or less guessing based on a set of data you have at a given moment in time, in a given context and lots of assumptions. If any part of this data is changing the initial estimate is gone and you have to start over.
        Seems that doing it on smaller periods of time (one or two weeks) seems to get some better results, but even so, is it still a mater of talent to get the right info and right resources.

        All the bests!

      • Adam Sanderson

        I’m a dev at LiquidPlanner and have really found that two things really help: ranged estimation and frequently updating your estimates.

        Ranged estimates are a nice way to communicate the uncertainty in a project or a feature. I find that it’s a lot more effective to tell a person who wants something done that it will take 2-28 days instead of just saying it will be 15 days. Alternatively if I say it’s going to take 14-16 days, you know I’m pretty dead certain about when it will get done (this never happens).

        Keeping your estimates up to date, is something of a no brainer, but you’d be amazed how many people don’t.

  • Alfred Low

    I keep going back to this illustration, which sums it up quite nicely:
    http://www.projectcartoon.com/cartoon/2

    Unfortunately, on a practical level, I think Craig Buckler has the right of it – realise that estimates are estimates.

  • http://www.mikehealy.com.au cranial-bore

    The other thing I find tricky is when a feature is seemingly simple (even to the programmer), but it has unexpected reach into other parts of the app, and is a lot more complex that would seem. The end result still seems minor, but it took 12 hours.

    • http://www.optimalworks.net/ Craig Buckler

      I totally agree. The easy stuff always takes far longer than you expect. Sometimes it’s because you don’t look at the problem in enough detail and sometimes it’s unexpected issues like cross-browser compatibility. Either way, few things are as simple as they initially appear.

  • kBailey

    I hate to bring up the old fall back position, but one of the biggest issues facing software development projects is that we’re never working from the same blueprint. Rarely do we get the opportunity to do the -same- thing twice, to readjust and reaffirm our estimates and arrive at something that can be delivered consistently and predictably.
    What we get instead is, “It’s just like that other project we did, except…” And the exception(s) end up being far more complicated than anyone anticipated.
    Some say this is because software development is so relatively immature as an industry, but I don’t think time will help. The near infinite variability of what we build will always doom us to uncertainty.
    How’s that for a positive outlook?

    • http://twitter.com/JeffRatcliff Jeff Ratcliff

      KBailey, It’s 11 months later so I don’t know if you’ll ever read this, but I wanted to tell you that your comment was one of the most insightful things I’ve read in a long time. You’re right on both counts: we almost never build something similar enough to provide good estimates and the uncertainty is caused by the nature of the work – not by the alleged immaturity of the profession.

  • silvia

    Great post. We tend to write our estimates with a high and low range. That helps to a certain extent. We also breakout PM (project mgmt) and debugging/QA time. PM time we put a number to but debugging and QA always has a tbd asterisk to it since it can be the slippery slope. We also have regular check-ins on scope, and costs.

    It’s our job to educate the client to all the above nuances but in a more palatable delivery. We’ve found that most appreciate our candor and assistance. They tend to feel more comfortable and empowered once they understand the process.

    And then sometimes you just have to pad the time and hope for the best!

  • MAPstr

    David’s Laws of Specification:
    1. The specification is always wrong.
    2. Insofar as the specification is right it is too vague to be useful.
    3. Insofar as the specification is right and specific it almost certainly doesn’t meet the system requirements.
    4. Insofar as the specification is right and specific and meets the system requirements when you get the specification it won’t when you finish the application.
    http://www.clarionmag.com/col/97-11-cad.html

  • sebastian

    herpadurp. Just read Mythical Man Month

  • gorlok

    I amazing when projects don’t fail :P

  • Jean-Nicolas

    All so true. I hang my self the first project I work on… Meaning I said that it would take not too long but at the end it took allot longer than I ever estimated. The other problem is people don’t seem too understand what a programmer does. For example, I did a reunion and some people ask me what I did exactly. I had too tell them 3 times before they understood anything.

    Another project I work on, people got mad at me because it took allot of time too get what too them seemed simple allot of time.

    That’s the problem, is that I have to educate them all the time.

    Also, I want to add business plan are also useless (only in certain situations they are useful). I think that you need a crystal ball to make a business plan and a estimate.

    It’s hard for people in science and in tech too explain what we do because there is so many layers of complexity compare too other industry. I think this is the first thing that needs too be said to a client.

  • FB

    Evidence-Based Scheduling from FogCreek takes into account for Risk and you can find out how realistic the completion date is, so you can tell your boss with a straight face: “Yes, we can ship on time. With 10%, 50% or 90% probability.” For more can be found here.. http://www.fogcreek.com/FogBugz/docs/70/topics/schedules/Evidence-BasedScheduling.html

  • Joe Programmer

    What’s also interesting about estimates, is I saw a paper once that said basically you’ll get 2 very different estimates out of someone based on how you ask the question.

    “How long will it take to do this thing?”

    is very different than

    “How long did it take you to do something like this last time?”

    The second type of question will get more accurate answers (they may also be more precise as well…). I’ve noticed this myself as a programmer, and will often try to rephrase requests for estimates so that they sound more like the latter.

  • Bill

    Another simple thing that people often forget in estimates is to pad for sick days and vacation days. Especially if it’s a long project (6 months or a year or more) with many developers – it’s unlikely that nobody is going to take time off or get sick at any point during the project.

  • Selvakumar Manickam

    I havemanaged various projects and I could almost never meet the project estimates. My conclusion is very simple, any project involves people, and people factor is something very volatile and very difficult to be managed or forecasted :).

  • Bros

    You have to status that why non-programmers is not suitable to do project manager .

  • Dan

    I posted a related article about this a while back, on some specific categories of features that are often the most poorly estimated chunks of a project.
    http://otherthanthink.blogspot.com/2010/03/top-software-estimating-mistakes.html
    Aside from the the excellent Mythical Man Month that was already mentioned, the best advice I’ve found on how to mitigate these risks has been in these books:
    1. Waltzing with Bears (DeMarco)
    2. Software Estimation (McConnell)

  • http://www.altimawebsystems.com altima

    In our business, web development/interactive design, the big problem that many clients try to save budget on the analysis and jump into design and development w/out making good specs (while same time requiring fixed price quote).

  • ChrisFlink

    Great article. This is an ongoing issue at the web development company I work for and I am trying to make some improvements in the reliability of the estimates. These estimated are not only used for planning purposed but also for the cost estimation so the importance is even greater!

  • James

    I love this article! I once did a web database project for a local medical group, and everytime I’d show them some new progress, they’d have a new ‘feature’ they want added.
    “You want what? Ok…where does that put us with the original estimates?”
    By the time I was done, they were fairly upset with the cost of things.
    Note to self: Get everything in writing from the beginning, finish work on that plan, and THEN move on to new additional features.

  • Sphamandla

    The thing about development projects is that you need to have reliable programmers who can get the job done in time as expected but sometimes people don’t understand the concept of testing and debugging. So it is essential that the project manager draws up a good documentation that specifies the specifications and reqiurements of a software project,which clearly identifies the system reqiurements and scope with a cleared signiture from the client incase they deside to change the system’s functionality tomorrow

  • Dan

    Development teams are generally better at estimating relative complexity, rather than how long something might take. If you can quantify complexity, and track how much of that gets completed over time, you’re able to predict project timelines with at least somewhat better confidence. Many agile methods employ this technique, via point estimation. It’s also the basis for the predictive aspect of Pivotal Tracker (http://www.pivotaltracker.com).

  • http://www.nrzee.com ali0482

    This is perhaps the most irritating problem for a developer. A feature is changed or added because customer X has requested it or the CEO thinks it’s a cool thing to do.

    Is the impact of that new feature documented?…

  • Loda

    The biggest problem we got is the point 7.

    and also the point 7b:

    *we depend on some client’s program/server/data that are not documented or well implemented. *

    (read : we must do our program AND debug the server/data of the client)

    for the anecdote, we made some “syncro” of data send by a client by fax. It was a printed excel sheet. (They did not knew where was the .xls)

    This is always a surprise and a big loose of time.

  • webmage

    Another problem: The increasing administrative overhead during schedule slippages because management wants more frequent status reports (or worse, have more meetings), which means that the developers have less time for development, more interruptions after which they take some minutes to ‘get back into the flow’, and a dramatic decrease in general productivity.

  • Pzol

    Did you ever hear about agile Software development???

    • http://www.optimalworks.net/ Craig Buckler

      I have — see the second comment down.