Field stories: Estimation is hard. Tips for better software project estimation.

Of all the things that engineers do, ‘Estimation’ is one of the most difficult skills to master. Mostly because it involves people, cross functional team members and dependent teams, and technical surprises, but there are ways of improving your estimation skills, if you understand and practice the craft right. These are my tips to improve your estimation skills —

1. Consider that estimations can be for bandwidth or for calendar date. Explicitly mention the choice you are making.

This is one of the attribute that leads to eventual mismatch in the expectations between the consumer of your product and you. While you might have mentioned ‘this will take one month’ and meant to say, if a developer is working on 40 hours a week, for 4 weeks, it will take one month to complete. The other party would have understood it as, ‘this will be done by one month from now’.

While it is easy to estimate ‘bandwidth’ instead of time to market, the latter is all that some stakeholders care or understand.

How to tackle this

a. Mention to your stakeholders that bandwidth estimation is easier and can be done faster, and timeline estimate involve more time and you will require extra time for that.

b. Always be explicit and mention if you are estimating ‘bandwidth’ or ‘timeline’.

c. It is the estimator’s responsibility to find out all possible scenario for bandwidth of persons on a project could be taken away.

In a team with good culture, individual availabilities are bottoms up shared and are usually most accurate.

2. Always share the estimate details over emails or even better — on wikis.

Verbal estimates are lies, and slack estimates are forgotten. I know I’m using a very harsh word ‘lie’ here but the reason for that is everytime verbal estimates are disputed, you will be ‘liar’ from someone’s point of view.

Wikis are far better ways of sharing estimate, because after your make progress on your project, you will get a higher accuracy estimate of when you can complete the project

Ensure that the email in which you are talking about estimation, includes all the artifacts and research work you have done to come to the conclusion. Anyone changing the artifact, should automatically lead to re-visit (not necessarily change) in the estimates.

3. Never use the buffer!

This is perhaps most controversial take. But I believe ‘buffer’ is a dirty word. Never use buffers in your estimation.

How to avoid using buffer in the presence of uncertainties?

This is a topic of a research paper by itself, but these are some of the things that I use to avoid using buffer

a. Believe to begin with, that any buffer is ‘dirty’. Also don’t just remove the word ‘buffer’ from the communication and think internally or in a private sheet with buffer in mind. Remove it everywhere.

b. Always associate your estimates with confidence intervals. E.g. ‘I’m 50% sure that this can be done in a weeks time’ or ‘Im 99% certain this can done in two weeks.’

c. Promise your stakeholders that you will keep them updated with your view on the certainty of your estimate, if and when your confidence to deliver on a given date goes up or down.

d. Learn from the past deliverables timelines and retrospective meetings. This would be the reason to document and share the retrospective meetings.

4. Escalate and communicate risks ASAP you discover them

“It is ok to update estimate, but never ok to miss an estimate. “

Whenever you come across data or situation that can alter the estimate, immediately communicate to stakeholders. In that communication include how the new thing might affect the timeline, and what you are doing to mitigate it.

5. Accurate estimation is only at the end of design

In my organization, we have two kinds of estimation a. t-shirt sizes b. actual-estimates.

I have often repeated the line that t-shirt sizes could be 100% wrong. There could be projects that legal will never approve or computationally too expensive to be ever done. On the other extreme we have discovered that some things could be done by simple configuration changes so work is close to zero.

Once your design document is ready, get it reviewed by all the right engineering, legal and product folks including all the dependent services/component owners. Only then you can give a very high-confidence estimation. All estimation before this point are “guess-works”.

A senior engineer is someone who can identify all the stakeholders from whom s/he should get the document reviewed to get accurate estimate. An efficient engineer is one who can prune this list with their experience in domain and technology.

6. Make all your assumptions explicit

No assumption is small enough to skip the estimation email/wiki page.

While you don’t have to produce a legal document, but it is important to identify list everything that can affect the project timeline, because having that list will make your stakeholders appreciate the dependencies, complexities and uncertainties you are dealing with.

a. Migrations of are hidden villains of missed timelines. Try identify all the dependent service migrations if you can.

b. Get a written in email assurance from your dependent service that they will complete the work that you are dependent on.

c. Call out how many people will be working on the project. We live in a world with 20% attrition, which means one in five is leaving the company every single year.

d. Call out what is priority of this project w.r.t. rest of the projects that you or your team is working on. Overflow from a sister project, with higher priority can also cause delays. I generally call out P0 as projects that will not displaced by other projects and P1 as something that may be displaced.

7. Be willing to say no to giving the estimation.

Don’t buckle under pressure to give an estimate if you don’t have confidence or full understanding. Fall back on rule #5 if anyone wants a accurate estimate.

It is far better to go out as obnoxious ‘Sorry, I can’t estimate on the fly.’ than as a person who doesn’t know what they are talking.

8. Align with stakeholders: It is ok to be wrong in few projects, as long as on an average you catch up with your estimation over the quarter or half.

Rarely anyone will expect every estimate to be accurate, even though they will not admit it during the discussion. The reason is admitting that estimation could be missed becomes signal that ‘it is ok if estimations are missed’.

Always keep track of your estimations. If you always underestimating then try to course correct, not by adding buffer but by doing retrospections on what caused delays.

If you have never missed an estimate, then you still haven’t gotten estimation art correct, and you can compact your future estimate more.

9. Two layer estimates

There are always some projects, admittedly rare in product companies, which have hard deadlines. If a project takes its time, it takes its time.

The exceptions of course are when you are working on legal/compliance work or its a make or break competition with the competitor. In such scenarios, when you absolutely can’t afford miss the estimates, I add reverse of the buffer. I will have even tighter internal schedule, just so that any surprises could be ironed out before the external un-buffered schedule.

10. Understand and leverage the ‘project management triangle’

from: https://en.wikipedia.org/wiki/Project_management_triangle

Understanding this ‘project management triangle’ will give you obvious options of trade-offs and a common ground to discuss changes with your stakeholders.

Since I have repeated stakeholders words so many times, it is important to mention that for every project a very clear list of R.A.C.I. (responsibility assignment matrix).

My final comment on this topic is, estimation is more of an art than science and more your consciously practice, you will get better at it. So I wish you all the best!