Noise
Daniel Kahneman, Olivier Sibony and Cass R. Sunstein
Noise, by Daniel Kahneman, Olivier Sibony and Cass R. Sunstein, argues that human judgment fails in two ways: bias, which pushes judgments in one direction, and noise, the unwanted scatter among judgments that should be identical. Noise is widespread and rarely measured. The authors' remedy is to audit it and to practise "decision hygiene": independent judgments, judgments broken into parts, and the outside view first.
Two underwriters at the same firm price the same policy. Two doctors read the same scan. Each answer looks reasonable on its own, and nobody ever puts them side by side. If someone did, they would often find a gap far larger than anyone expected, which means the outcome depends partly on whose desk the file landed on. Noise is about that hidden lottery and how to shrink it.
The book in essence
Kahneman's Thinking, Fast and Slow was largely about bias: errors that bend judgment in a predictable direction. In Noise, published in 2021, he and his co-authors turn to the other half of error. Their image is a target: one team's shots cluster tightly but off-centre (bias), another's scatter around the bull's-eye (noise). Organisations have spent years hunting bias and have barely looked for noise, although measuring it needs no known right answer, only several judgments of the same case.
The authors break system noise into three parts. Level noise is the difference in people's average severity; pattern noise is each person's particular reaction to particular cases; occasion noise is the same person judging the same case differently at another time. Because each judgment is usually made once, by one person, all three stay out of sight.
Their remedy is deliberately unglamorous. A noise audit measures the problem. Decision hygiene reduces it: aggregate independent judgments, structure complex judgments into separate assessments, hold back intuition until the facts are in, and think statistically about similar cases first. Like handwashing, it prevents errors you never get to see.
Who it's for
- Managers who own recurring judgments: hiring, pricing, credit, performance reviews, grading
- Readers of Thinking, Fast and Slow who want the other half of the story of error
- Anyone designing a process whose fairness depends on consistent decisions
Who it's not for
- — Readers looking for a light introduction: the book is long, and its statistical chapters ask for patience
- — Those who want one quick fix: the remedies are routines that pay off across many decisions
Key lessons
01
Bias and noise are two different errors
A judgment can miss in two ways. Bias pushes many judgments the same way, like a rifle that always shoots to the left. Noise scatters judgments that should match, like shots spread around the centre. The two need different remedies, and noise is the one organisations rarely see, because each case is usually judged once. Yet measuring noise requires no right answer: several judgments of the same case are enough. If they disagree widely, at least some of them are wrong, whatever the truth turns out to be.
Two loan officers at the same bank reviewed the same small-business application on the same afternoon. One recommended approval with a modest limit; the other declined. Neither knew the other had seen the file. Only when a manager happened to compare their notes did the bank realise that the applicant's answer had depended on which officer was free.
02
Run a noise audit
The authors' main diagnostic is simple. Choose a judgment your organisation makes again and again and prepare a handful of realistic cases. Give the same cases to several of the people who normally decide them, ask each to judge alone, and compare. The question is not who is right but how far apart they land. The book describes audits at an insurance company where executives expected a modest gap between two underwriters' quotes and found a real difference several times larger. A useful measure is the typical gap between two judgments of the same case, as a share of their average.
A recruitment agency gave five consultants the same three anonymised CVs for one role and asked each to recommend a starting salary. Nobody conferred. For the same candidate, the highest and lowest recommendations differed by more than a quarter. The partners had assumed their team was consistent; the audit took an hour and changed how they priced every placement.
03
Judge alone, then pool
Averaging independent judgments cancels part of their errors, so a group's average is less noisy than a typical single judgment. The condition is independence, and discussion usually destroys it: the first confident or senior voice pulls later speakers towards it, and a group can end up more extreme and more certain than its members were alone. The remedy is a sequence. Each person writes a judgment and one reason before any discussion; the judgments are revealed together, ideally without names; then discussion focuses on the differences.
A product committee used to open budget reviews with the director's view, and the room usually agreed. They changed one thing: everyone wrote a number on a card before speaking, and the cards were turned over at once. The spread surprised them, and the discussion that followed was about the evidence behind the highest and lowest cards, not about who had spoken first.
04
Break the judgment into parts
One overall impression is noisier than several separate assessments. The book's clearest case is the job interview: unstructured conversations predict performance poorly and interviewers disagree widely, while structured interviews do better. The general method, which the authors call the mediating assessments protocol, has three moves. Decide in advance the few assessments the decision depends on. Rate each one separately, on facts and comparisons. Delay the overall judgment until all the ratings are in: intuition is welcome, but at the end, not at the start.
A hospital's panel for hiring nurse managers used to discuss each candidate freely and then vote. They agreed on four qualities in advance, gave each interviewer two of them, and scored answers against written descriptions of a strong response. The final discussion began only once all the scores were on the table, and a quiet candidate who had scored well throughout was hired.
Try this today
Pick one judgment your team makes often. This week, give the same case to two colleagues separately and write down how far apart their answers are.
Quick check
In the book's terms, what is noise?
Why do the authors ask people to write down their judgments before a group discusses them?
Selected quotes
“Wherever there is judgment, there is noise—and more of it than you think.”
“To understand error in judgment, we must understand both bias and noise.”