Concept
Calibration
Calibration is the match between your stated probabilities and reality: when you say 70 percent, being right about 70 percent of the time. Most people who say 90 percent are right about 70 percent of the time, and only a written record reveals it.
Tetlock's superforecasters were distinguished less by knowing more than by knowing how much they knew. A calibrated forecaster's 60 percents come in at 60 and 90 percents at 90; an overconfident one's 90s come in at 70. Calibration cannot be felt from the inside; it exists only if you write probabilities down, date them and mark them later against what happened. The Brier score does this formally, but the habit matters more than the formula. Once you see that your 90s are really 70s, you adjust, and the adjustment is the whole improvement.
A real-life example
A team logs its delivery estimates for a quarter and finds that dates it was 90 percent confident about were met 60 percent of the time. From then on the team quotes ranges and adds a buffer, and the customers stop being surprised.
How to use it
- 1Write every probability as a number with a date.
- 2Mark it later against what happened and note which direction you were off.
- 3After ten or more, look for the pattern and adjust your numbers.