"I've done it a hundred times"

Here's the puzzle. If the estimate is wrong, and you've walked that route a hundred times, why haven't you learned? You have a hundred data points and you still say fifteen minutes.

Buehler, Griffin and Ross looked at this too. When their participants were asked why they'd missed past predictions, they put the blame on outside causes that were specific to the occasion and unlikely to come round again. The essay was late because of a bout of flu, or a computer problem, or a friend visiting. Each explanation was true. And each one let the person set that occasion aside as a special case, so that when they made the next prediction, the past had nothing to teach them. Their studies also found that when people were told to connect their predictions with their past experience of similar tasks, the optimism went away. The information was there. It just wasn't being consulted.

You do the same. Last Tuesday you were late because of the keys. The Tuesday before, because of the phone call from your mother. The one before that, because of the wet path. Each one was a one-off, and so the walk still takes fifteen minutes, and the one-offs, which happen every week, live in a separate file that the estimate never looks at.

Memory adds a second problem, and it's a surprising one. Michael Roy, Nicholas Christenfeld and Craig McKenzie reviewed the evidence in 2005 and argued that people don't only ignore the past when predicting. They misremember it. When people are asked how long something took last time, they tend to give a figure that's shorter than it really was. So even on the occasions when you do consult memory, memory hands you a shortened version. You remember the walk as fifteen minutes because that's roughly how long it felt, and the ten minutes of finding keys and coats didn't feel like part of the walk, so they aren't in the memory of it.

Put the two together and you get a person who's been late to the same meeting many times and still expects to be on time. Experience can't fix the estimate on its own, because the lessons are filed as exceptions and the durations are stored short.

There's an odd detail in the 1994 studies that I find useful. When people predicted how long someone else's task would take, they weren't optimistic. If anything they were pessimistic, and they made more use of past experience. So you can already estimate accurately. You do it for other people. If a colleague said the walk across the park took fifteen minutes and they had to leave at quarter to, you'd probably say, "Leave at twenty to, you know what you're like." The knowledge is there. It's only your own trips that get the best-case treatment.