How to Create Better Estimates and Set Realistic Deadlines

The human brain is usually biased towards positivity, which is why we tend to ditch information that contradicts our optimistic beliefs when we plan.

Next, if we add to our biases the external pressure of teammates, clients, and managers, things will easily go downhill.

We end up setting unrealistic deadlines and budgets and accept only minimal changes to the initial plan. But this is costing us, companies, and society so much.

But, when there’s a will, there’s a way. The will to adapt and change can lead to better ways.

Raise your hand if you’ve ever:

• planned to renovate something in your house, and it took way longer than expected?
• had sleepless nights because you thought you’d finish a school project faster, and you ended up running out of time?
• worked on a project at work that was not delivered on time?
• missed a flight because you thought there was enough time to catch it?

I hate to be the one telling you this, but we are all optimistic about deadlines, and this phenomenon has been observed under the term “optimism bias.”

Even Nobel prize winners such as Daniel Kahneman experienced this bias and later coined the “planning fallacy” in 1977, which is a specific variant of optimism bias.

It begs the question, is there a way to avoid the planning fallacy and set more realistic deadlines?

Of course.

And I’ll show you how in this article.

What’s behind “optimism bias” and the “planning fallacy”? 

Optimism bias is a cognitive bias that leads someone to believe they are less likely to experience a negative event. Sounds familiar?

Several factors influence optimism bias:

• desired end states. People have dreams and desires that they are emotionally attached to. They make people feel positive about their goals and focus less on any negative outcomes or setbacks. If you think about it, this is often a good thing. Since antiquity, man has always had the dream of flying. Hundreds of failed attempts happened until the first controlled, sustained flight of a powered, heavier-than-air aircraft occurred in 1903. The heroes were the Wright Brothers. If people gave up on their dreams, our planet, and lives would be so different.
• cognitive mechanisms. This refers to the mental shortcut that we use when estimating probabilities. This means that, eventually, people compare a given scenario to some preconceived notion or bias.
• information bias. This refers to the fact that people know more about their own personal information and own risk but way less about others.
• overall mood. People with a negative and sad mood are less optimistic. Such a mental state enhances the memories of negative events, thus leading to negative judgment. The same goes for positive moods.
• the power that fosters selective attention. People in a position of power typically display a narrower focus of attention consistent with constructs such as goals and needs. On the other hand, people with less power pay more attention to secondary information, as per “The Situated Focus Theory of Power” by Ana Guinote. The two perspectives combined can benefit both parties; however, things can go wrong if a proper tradeoff between perspectives fails.

Optimism bias influences decision-making and forecasting. It affects budgets, completion times, and even benefits—which can be overestimated.

Planning fallacy refers to the tendency of individuals to underestimate the time, resources, and effort needed to achieve a goal. Interestingly, when outside observers predict task completion times, they tend to be more pessimistic.

Planning fallacy is also related to wishful thinking, meaning that people think tasks will be finished quicker and easier because they just want it to happen like that.

Methods to fight planning fallacy  

1. The segmentation effect 

The segmentation effect refers to the fact that the time needed for a task is significantly smaller than the sum of the time needed for its individual sub-tasks. This idea was tested in a 2008 study by Darryl K. Forsyth and Christopher D. B. Burt.

In this study, the authors found that participants allocated more time to complete a task when it was segmented into subtasks compared to when it was presented as a single task. Participants provided more accurate estimates of the time required to complete a task when asked to provide a best-case and worst-case estimate in addition to a single estimate.

The authors concluded that task segmentation can reduce the planning fallacy bias by increasing the accuracy of time estimates, recommending that people break down tasks into subtasks to improve their time estimates.

Project management tools such as Paymo can help you properly structure your tasks.

Set subtasks in Paymo to tackle poor time estimates.

2. Implementation intentions 

These concrete plans accurately show how, when, and where someone will perform a task or sub-task. This helps people visualize all possible outcomes while making them more aware of what’s at stake.

You can also use the Eisenhower Matrix method to classify tasks according to urgency.

The matrix is a visual to-do list that prioritizes tasks across four quadrants: Do, Schedule, Delegate, and Delete:

 

I find the Eisenhower matrix a useful prioritization system.

3. Reference class forecasting 

Instead of relying solely on personal experiences or intuition, people can examine similar projects or tasks and analyze their actual completion times to generate more accurate estimates.

By considering a broader range of outcomes, they can mitigate the effects of the planning fallacy.

4. “Superforecasting” or how to harness the future with insight 

Now that we know how to plan better, let’s go more in-depth about estimates and deadlines.

But before diving in, I’ve got some trivia about estimates and forecasts.

Did you know that, generally, the accuracy of our predictions is no better than chance? This means that flipping a coin is just as good as our best guess, experts included!

Yes, you heard that right—Philip Tetlock reached this conclusion in a famous experiment.

The participants were tasked with evaluating the likelihood of three possible results: the continuation of the current situation, an increase in something (such as political freedom or economic growth), or a decrease in something (such as repression or recession).

Interestingly, the experts fared worse on the first rating scale than a scenario where they had simply assigned an equal probability (33%) to each of the three outcomes.

In other words, individuals who dedicate their lives to studying global conditions proved to be less effective forecasters than random chance, as even dart-throwing monkeys would have evenly distributed their selections among the three options.

Tetlock states that “In this age of academic hyperspecialization, there is no reason for supposing that contributors to top journals—distinguished political scientists, area study specialists, economists, and so on—are any better than journalists or attentive readers of the New York Times in ‘reading’ emerging situations.”

Shocking, isn’t it?

Now, is everyone wrong about forecasting?

The answer is clearly no.

Enter superforecasters!

They are the people who can make predictions far more accurately than the rest of us.

Within “The Good Judgment Project,” Tetlock and his team discovered that essential characteristics of a superforecaster include collaborative teamwork, adopting a probabilistic mindset, using diverse information sources, and displaying a readiness to acknowledge errors and adapt their strategies accordingly.

Now, this might sound sad. Who do we trust now?

But there’s some light at the end of the tunnel: while not everyone can become a superforecaster, we can all improve our judgment.

So, how do superforecasters get more accurate in their predictions?

• They don’t ask themselves, “Which countries will no longer exist because of rising sea levels in 2050”. This would be a too-far-fetched assumption. This is why they narrow their timeframe and get closer to the present;
• They break problems into chunks. They also try to see what it’s unseen and unknown;
• Superforecasters know that “history repeats itself.” This is why they try to learn from similar situations;
• Incorporate new beliefs and other sides of the story. A cool exercise here is to kill your best idea. You need to become your own critic;
• Diminish uncertainties. Try to avoid as many “maybes” as possible;
• Keep score of successes and failures. Learn from them;

If you want to dig deeper into this topic, read Tetlock’s book Superforecasting, The Art and Science of Prediction.

How to get better at setting estimates and deadlines 

In this discussion, I’m starting with the following assumptions:

• We’re bad at forecasting and setting estimates (dates, budgets, number of people involved, etc.);
• We’re not good at setting up deadlines;
• We are overly optimistic when planning;

I’m not saying it—science is.

In the earlier section, I’ve given you some hints on how we can improve our forecasting and planning abilities.

Let’s dig some more.

Avoid anchoring in the original plan 

We all hate admitting when things go wrong, and changing plans doesn’t make us happy.

But superforecasters are the ones that can easily change plans. They adjust their view to the new amount of information they receive. In short, they adapt.

Anchoring is another type of cognitive bias that affects the way we plan. We tend to rely too much on the initial information that we have.

Face the social pressure 

Managers, organizations, and clients often push for projects to finish quickly.

Executives might prefer optimistic deadlines.

Keep track of your time

Studies show that people underestimate their task completion times. It’s best to rely on historical data when making estimates, and that data can be from your personal time logs, which is why time tracking is crucial to accurate time estimates.

It’s also easy—and free!—to adopt an automatic time tracker: let it run in the background, and at the end of the work day, you link timestamps to task entries:

Convert timestamps into task entries in a few clicks with Paymo Track.

Schedule “buffer time” 

Remember the optimism bias. Also, keep in mind to expect the unexpected. And leave room for that.

Or you could learn from others as well. At Tidaro, we took four months to rebrand, although we expected three months. We postponed some small tasks that were not needed after the rebrand. So we had room to adapt and prioritize.

Establish mini-milestones 

Use tools such as Paymo to set up various types of milestones and also to monitor progress. This way, you can properly adjust your estimates along the way. Gantt charts are best for visualizing task dependencies, timelines, and milestones:

Gantt chart in Paymo.

Handle important tasks first thing in the morning  

Now, a deadline may be realistic, but procrastination creeps in.

What should you do?

Meet the “Eat the frog” method by Brian Tracy, the frog being something you do not love doing.

The idea is to resist the urge to do the easiest things first. That’s not the way to go.

You should pull up your sleeves and take the bull by the horns first thing in the morning.

Minimize distractions  

“Oh… I’m craving another snack.”
“Let me just clean up my inbox today.”
“Just a little sneak peek into my Instagram account.”
“Hmmm, this is an interesting article.”

Sounds familiar?

It’s so easy to get distracted, especially if you are stressed and overwhelmed. These distractions can add up and make you miss a deadline. But there are ways to minimize them.

For example, you could block time for work in your calendar. Then you could snooze notifications.

You can set up a special time of day for verifying your emails. You could schedule a specific day for meetings to focus better on the rest of the days.

Conclusions

It’s time to bid farewell to the land of unrealistic expectations and say hello to the realm of practicality and success.

If we accept that we have an innate tendency to underestimate the time needed for tasks, we can avoid falling into the planning fallacy trap. Remember, folks – it’s better to overestimate and pleasantly surprise ourselves than to scramble at the last minute like headless chickens.

To conquer the planning fallacy, we must break down projects into manageable chunks and factor in the unexpected. Let’s face it; life loves throwing curveballs our way, so let’s be prepared for those pesky detours and allow for some buffer time.

Also, asking for an extension is always better than sheepishly delivering subpar results.

So, let’s conquer the planning fallacy, one deadline at a time.

And let’s be optimistic but not embrace optimism bias.

First published on July 10, 2023.