BookYol

Concept

Illusion of Validity

The illusion of validity is the confidence people feel in a judgment because the evidence tells a coherent story, even when that kind of evidence has been shown to predict the outcome poorly; the feeling of certainty measures the story's coherence, not its accuracy.

Daniel Kahneman named the illusion after his own experience as a young psychologist in the Israeli army. He and his colleagues watched officer candidates tackle a group exercise in the field and felt sure they could see who would lead and who would not. Months later the reports from officer school arrived: their forecasts were barely better than blind guesses. Then the next batch of candidates walked onto the field, and the team felt exactly as sure as before. Knowing the record had not touched the feeling. In Thinking, Fast and Slow Kahneman explains why. Confidence is produced by System 1 from the coherence of what it sees and the ease of building a story from it, and because what you see is all there is, the missing evidence and the noise of the real world never enter the story. The same pattern appears wherever people make confident predictions in an unpredictable setting: picking stocks, hiring after one interview, forecasting next year's sales. Subjective confidence is a feeling, not a measurement. The useful question is not 'how sure am I?' but 'how often have judgments like this one turned out right?', and answering it takes a written record of predictions and outcomes.

A real-life example

Aslı runs a restaurant in İzmir and hires cooks after a single trial shift; she has always felt she knows within an hour who will last. For one year she writes her verdict on each trial in a notebook before deciding. When she compares the notebook with who actually stayed and did well, her confident calls turn out right about as often as her hesitant ones. She now adds a second trial shift and a call to a former employer.

How to use it

  1. 1Before an important prediction, write it down together with how sure you feel; confidence you have not recorded cannot be checked later.
  2. 2Ask how predictable your field is and how often judgments like yours have proved right, not how convincing this particular case looks.
  3. 3When the story feels complete, list what you have not seen; the more coherent it feels on little evidence, the more caution it deserves.