Dr. Wallace's Favorite Mental Models

The Wisdom of Crowds

Many independent, diverse guesses, averaged together, are often more accurate than any single expert. The catch: the moment people start copying each other, the crowd becomes a herd and the magic disappears.

The 30-second version

What it is

Every individual guess contains some signal and some error. If the errors point in different directions, they cancel when you average them, and the signal is what remains.

In 1906, at a livestock fair in Plymouth, England, the statistician Francis Galton collected 787 paper tickets from people guessing the dressed weight of an ox. The middle guess was 1,207 lb. The real weight was 1,198 lb. The crowd's median missed by under 1%, and later analysis of the same tickets put the mean at 1,197 lb. Galton published the result in Nature in 1907 under the title "Vox Populi." James Surowiecki gave the idea its modern name in his 2004 book The Wisdom of Crowds.

The idea is not that crowds are always smart. It is that crowds are smart only under specific conditions. Surowiecki named four:

1

Diversity

People bring different information, experience and ways of reasoning, so their mistakes differ.

2

Independence

Each person forms a view before seeing anyone else's. Copying makes errors line up instead of cancel.

3

Decentralization

People can draw on local, specialized knowledge rather than one shared source.

4

Aggregation

Some mechanism, such as an average, a median, a market price or a vote, turns private judgments into one answer.

One-line test Before trusting a group answer, ask: "Did these people get to their answers separately, from different starting points?"

Mental Model Compass

Where this model sits

The horizontal axis asks whether judgments are formed independently or under social influence. The vertical axis asks whether a model mainly helps you get better answers or spot a failure. Choose a model to compare it with the Wisdom of Crowds.

Mental Model Compass A two-axis map. Left means independent judgment, right means social influence. Top means a tool for better answers, bottom means a warning about failure. The Wisdom of Crowds sits top-left. Law of Large Numbers and Bayesian Updating are also top-left. Social Proof is in the middle right. Groupthink and the Abilene Paradox are bottom-right. The Keynesian Beauty Contest is right of center near the middle. Independent judgment Social influence Tool for better answers Warning about failure
Build it step by step

How the crowd gets smart, and how it gets dumb

Step 1 of 5

Every guess = truth + signal + noise

Imagine each person's estimate as the true answer plus an error. Some people guess high, some low. Nobody is reliably right, but nobody needs to be.

If the errors are random and independent, averaging 100 guesses shrinks the typical error of the average to roughly one tenth of a typical individual's error (error shrinks with the square root of the crowd size).

Step 2 of 5

The diversity prediction theorem

The complexity scientist Scott E. Page proved a simple identity for averages:

Crowd error = Average individual error − Diversity of estimates

Here error is measured as squared distance from the truth, and diversity is how spread out the guesses are around the crowd's own average. It is arithmetic, not opinion. A crowd's average can never be worse than its average member, and it gets better the more the members disagree for good reasons.

Quick check: two teams have the same average individual error. Team A's guesses are tightly clustered; Team B's are spread out. Whose average is more accurate?

Step 3 of 5

Correlated errors do not cancel

If everyone read the same misleading report, or listened to the same confident boss, their errors all lean the same way. Averaging a thousand copies of the same mistake gives you the same mistake, now with false confidence.

This is why crowd size cannot fix a shared bias. Adding more people from the same background mostly adds more of the same error.

Step 4 of 5

Social influence turns a crowd into a herd

In a 2011 experiment at ETH Zurich (Lorenz, Rauhut, Schweitzer and Helbing, PNAS), 144 participants estimated factual quantities over five rounds. When they could see others' answers, their guesses converged, but accuracy did not improve. Worse, the truth often drifted to the edge of the group's range, and people felt more confident anyway.

Quick check: after people saw each other's estimates, what happened to their confidence?

Step 5 of 5

The practical rule

Collect first, discuss second. Get each person's private estimate in writing before any conversation. Make sure the people really differ in what they know. Aggregate with a simple rule (median is robust to outliers). Only then discuss, ideally starting with the people whose estimates were furthest from the middle, because they may hold information the rest of the group lacks.

You will try this rule in the simulator under "Apply it."

Across domains

Real examples

Statistics · 1906

Galton's ox

At the West of England Fat Stock and Poultry Exhibition in Plymouth, 787 fairgoers paid sixpence each to guess an ox's dressed weight. Galton, who doubted ordinary people's judgment, found the median guess (1,207 lb) within 9 lb of the true weight (1,198 lb).

Lesson: Butchers, farmers and curious townspeople had different information and guessed privately on tickets, so their errors canceled.

Naval search · 1968

The lost submarine USS Scorpion

When the submarine vanished in the Atlantic, Navy scientist John Craven had experts assign probabilities to several scenarios for what went wrong. The team combined these into one probability map of the sea floor, a technique now called Bayesian search. The wreck was found southwest of the Azores, a few hundred yards from the highest-probability cell. The popular story that experts bet bottles of whisky comes from the book Blind Man's Bluff; the 1971 technical paper on the search does not mention wagering.

Lesson: Pooling diverse expert judgments with a formal method beat any single expert's favorite theory.

Elections · 1988–2004

Prediction markets vs. polls

Joyce Berg, Forrest Nelson and Thomas Rietz compared the Iowa Electronic Markets, where traders buy and sell contracts on election outcomes, with 964 polls across five U.S. presidential elections. The market price was closer to the final result 74% of the time (International Journal of Forecasting, 2008).

Lesson: A price is an aggregation mechanism. Traders with money at stake reveal private information independently.

Lab experiment · 2011

When the crowd stops being wise

In the ETH Zurich study, participants who could see others' estimates converged toward each other without getting more accurate. The researchers named three effects: the social influence effect (less diversity, no better accuracy), the range reduction effect (the truth slides to the edge of the group's range) and the confidence effect (people feel surer anyway).

Lesson: This is the failure case, and it is just as important as the success stories.

Everyday work

Planning Poker in software teams

Agile teams estimating the effort of a task often reveal estimate cards at the same moment, rather than talking first. The ritual exists to protect independence: nobody can anchor on the senior engineer's number before committing to their own.

Lesson: The big outliers then start the discussion, which surfaces hidden risks.

Television

"Ask the Audience"

Surowiecki reports that on Who Wants to Be a Millionaire?, the studio audience's top choice was right about 91% of the time, compared with roughly 65% for the contestant's hand-picked expert friend.

Lesson: A large, varied group voting privately (each with a keypad) can beat one well-chosen expert on general-knowledge questions.

Worked application

Forecasting demand at a small company

Illustrative scenario Lantern & Pine Coffee Co. is a fictional roaster. Before ordering green beans, its manager asks staff to forecast next quarter's bag sales. The true demand (unknown at the time) turns out to be 1,200 bags. The owner, who speaks first in meetings, believes it will be 1,500.

The steps

  1. Frame one clear question. "How many bags will we sell between 1 Jan and 31 Mar?" A number, a date range, no ambiguity.
  2. Pick diverse estimators. Include baristas, the wholesale rep, the web-shop lead and the bookkeeper, not just the sales team.
  3. Collect privately. Everyone submits a number on a form before the meeting.
  4. Aggregate with a rule. Take the median, and note the spread.
  5. Discuss the outliers. Ask the highest and lowest estimators what they know.
  6. Decide and record. Order against the median, with a buffer sized by the spread. Compare with actual sales next quarter.

What to watch in the simulator

Turn the owner speaks first switch on and lower independence: the estimates bunch up around 1,500 and the crowd's error grows, even with a large team.

Turn on same-background team: everyone shares a blind spot. Watch how adding more people barely helps.

The numbers under the chart check the diversity prediction theorem live: crowd error always equals average individual error minus diversity.

How many people submit a forecast.
100% = everyone writes their number privately. 0% = everyone mostly adopts the anchor they heard.
Owner speaks first (anchor at 1,500)
Same-background team (shared +180 bag blind spot)
Misapplications

Common mistakes

Thinking "more people" is the same as "a wiser crowd"

A thousand people who all read the same headline make one guess, a thousand times over. Fix: Spend your effort on diversity and independence before you spend it on headcount.

Discussing before estimating

The first number spoken in a meeting anchors everyone, especially if a senior person says it. Fix: Collect written, private estimates first, then reveal them all at once.

Treating majority opinion or popularity as crowd wisdom

Likes, bestseller lists and "everyone says so" are usually cascades: each person reacting to earlier people. That is Social Proof, not independent aggregation. Fix: Ask whether each judgment was formed before seeing the others.

Asking the crowd a question nobody has information about

Averaging ignorance gives you averaged ignorance. The ox-weighers had seen cattle before. Fix: Make sure most members hold at least some relevant, partial knowledge.

Using the mean when a few wild answers dominate

One joke answer of 1,000,000 can wreck an average. Fix: Use the median or a trimmed mean, as Galton did, and report the spread alongside it.

Ignoring the outliers once you have the average

The person furthest from the middle may know something nobody else does, such as a supplier problem or a competitor's launch. Fix: After aggregating, ask the outliers to explain their reasoning before you finalize.

Boundary conditions

When not to use it

Deep technical questions

If a question needs specialized training to answer at all, such as a drug interaction or a structural load calculation, a qualified specialist beats a lay crowd. Use a crowd of experts, or just the expert.

Guessing what others will guess

When people are rewarded for predicting the crowd rather than reality, as in speculative bubbles, aggregation amplifies the error. That is the Keynesian Beauty Contest, not crowd wisdom.

Questions of value, rights or ethics

"What is the right thing to do?" is not an estimation problem with a hidden true number. Averaging opinions here produces a compromise, not a correct answer.

When independence cannot be protected

In a live group chat, a public vote or a meeting where the boss speaks first, assume herding. Either change the process or discount the result.

Creative or one-off design choices

Averaging ten logo sketches gives you mush. Crowds are good at estimating and choosing between options, not at authoring a single coherent vision.

When the crowd can be gamed

Anonymous online ratings and small markets can be manipulated by a few motivated actors. Check whether anyone has a reason to flood the system.

Try it yourself

Build a prompt for your own situation

Fill in the fields. The prompt updates as you type, ready to paste into any AI assistant.

Your prompt


        
        
Save as skill

Take this model with you

This Markdown file gives an AI assistant a compact, reusable version of the model. Paste it into custom instructions, a project knowledge file, or anywhere your assistant reads standing guidance.

Skill preview


        

Apply to my problem

A more action-focused prompt: describe a decision you face this week, and get back a concrete plan for running it through a crowd.


        
        
Further reading

Sources

  • Galton, F. (1907). "Vox Populi." Nature, 75, 450–451.
  • Surowiecki, J. (2004). The Wisdom of Crowds. Doubleday.
  • Page, S. E. (2007). The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies. Princeton University Press.
  • Lorenz, J., Rauhut, H., Schweitzer, F., & Helbing, D. (2011). "How social influence can undermine the wisdom of crowd effect." PNAS, 108(22), 9020–9025.
  • Berg, J., Nelson, F., & Rietz, T. (2008). "Prediction market accuracy in the long run." International Journal of Forecasting, 24(2), 285–300.
  • Richardson, H. R., & Stone, L. D. (1971). "Operations analysis during the underwater search for Scorpion." Naval Research Logistics Quarterly, 18(2), 141–157.
  • Sontag, S., & Drew, C. (1998). Blind Man's Bluff: The Untold Story of American Submarine Espionage. PublicAffairs.