Dr. Wallace's Favorite Mental Models

Survivorship Bias

Why the success stories you can see are never the whole story, and how to find the ones that quietly disappeared.

30-second overview
Survivorship bias happens when you judge a situation only by the examples that made it through, while the examples that failed or disappeared are left out of view. The failures did not get quieter. They just stopped showing up in the data you can see.

Why it matters

  • It makes risky strategies look safer than they are.
  • It makes advice from "winners" look more reliable than it is.
  • It hides in plain sight, because the missing cases are, by definition, missing.
Mental Model Compass

Where this model earns its keep

Two questions decide how much this model matters: how much you are learning from a filtered group of "winners," and how much is riding on the conclusion.

High stakes Low stakes Full population known Only winners visible Sweet spot
Sweet spot: High stakes, and you're only seeing the winners. Copying "top performer" habits, studying market leaders, following advice from famous founders.
Worth a check: Lower stakes but still a filtered sample. Reading a glowing product review or a single success testimonial.
Low value here: You already have the full population, wins and losses both. The bias has nothing to hide behind.
Progressive learning

Build the idea in three steps

During World War Two, engineers studied bombers that came home and mapped where the bullet holes clustered: wings, tail, fuselage. The natural conclusion was to reinforce those spots.

Statistician Abraham Wald pointed out the flaw. The planes in the study had survived their damage. The bombers that took hits to the engine and cockpit mostly did not come back, so they were never in the sample. Wald recommended armoring the places with the fewest hits on returning planes, because that is where a hit was fatal.

Survivorship bias is not about anyone lying or cherry-picking on purpose. It is a structural filter: whatever process you are studying quietly removes some cases before you ever get to look at them.

The fix is always the same question: "What would I need to see to know I'm looking at everyone, not just the ones who made it?" If you cannot answer that, treat the pattern as a hypothesis, not a conclusion.

Three moves handle most cases:

  1. Name the filter. What process separated the visible group from the invisible one? (Getting funded, staying in business, publishing a book, returning from a mission.)
  2. Estimate the denominator. Even a rough guess at "how many tried" changes the story more than a precise count of "how many succeeded."
  3. Compare rates, not counts. Five happy customers mean little without knowing how many customers there were in total.

Quick check: A business book profiles ten companies that took a big risk and won big. What's the first question to ask?

Diverse examples

The same pattern, five settings

Investing

A fund company reports its average 10-year return using only funds still open today. Funds that closed after poor performance are dropped from the record, quietly boosting the reported average.

Hiring

A team studies its five best salespeople and builds a training program around their habits, without checking whether average and struggling reps share those same habits too.

Publishing

"I dropped out and built a billion-dollar company" is a compelling story precisely because almost everyone who drops out to build a company does not end up on a magazine cover.

Product reviews

People who had a bad experience with a product and returned it rarely leave a review. The five-star average reflects the buyers who kept it, not everyone who bought it.

Health and fitness

A diet plan looks impressive when you only meet the people still following it a year later. The people who quit after two weeks are not in the testimonial pool.

Old buildings

"They built things to last back then" ignores that the poorly built structures from the same era have long since collapsed and been forgotten.

Common mistakes

Where people misuse this model

✕
Blaming every gap on survivorship bias. Sometimes data is simply missing for an unrelated, boring reason. Check for a real filtering process before assuming one.
✕
Overcorrecting into cynicism. Dismissing every success story on principle is as lazy as accepting every one uncritically. The goal is a sharper question, not a blanket veto.
✕
Confusing it with confirmation bias. Confirmation bias is about which evidence you go looking for. Survivorship bias is about evidence that never had a chance to reach you in the first place.
✕
Forgetting to ask "compared to what." A count of successes means little without a sense of the full attempt pool it came from.
When not to use it

Boundaries worth respecting

!
When you already have complete data, wins and losses both, there is no hidden group left to account for.
!
When you are looking at a single, well-documented case rather than a pattern drawn from many cases, there is no comparison population to reconstruct.
!
When the "missing failures" you suspect were never systematically recorded anywhere, in which case the honest move is to say the evidence is incomplete, not to assume a specific hidden pattern.
Related models

Models that travel well together

Selection Bias

The broader family this model belongs to: any time the way a sample was gathered shapes what it can tell you.

Confirmation Bias

A different failure mode: seeking evidence that fits what you already believe, rather than missing evidence you never saw.

Base Rate Neglect

Ignoring how common something is overall in favor of a vivid, specific example.

Regression to the Mean

Extreme results tend to be followed by more ordinary ones, which can look like decline even when nothing changed.

Worked application

Walking through a real decision

Scenario: leadership wants to build a training program around the habits of the sales team's five top performers.

1
Name the filter. These five reps survived a process that others did not: some reps with different habits left the company, moved teams, or simply had an average year and were never studied.
2
Estimate the denominator. The team has 40 reps. Before copying the top five, ask how many of the other 35 already do some of the same things without the same results.
3
Compare rates, not stories. If 30 of the 40 reps also "build relationships before pitching," that habit is not what separates the top five. Something else is.
4
Look for the ones who left. Check whether any departed reps had similar habits to the top five but did not get similar results, perhaps because of territory, timing, or luck.
5
Revise the conclusion. The training program should teach the habits that actually separate high performers from everyone else, not just the habits every top performer happens to share.
Try it yourself

Build your own check

Describe a success story, dataset, or piece of advice you're looking at. This builds a short prompt you can hand to an AI assistant, or just work through yourself.

Learn it→Apply it→Save as skill
Save as skill

Take this model with you

This is a portable, plain-language version of the model you can save, copy into an AI assistant, or hand to a teammate. It works the same way whether you paste it once or keep it as a standing instruction.

Loading preview...

Apply to my problem

Turn this into your own prompt

Describe what you're facing right now. This combines the skill above with your situation into one prompt, ready to paste into an AI assistant.