Dr. Wallace's Favorite Mental Models

The Law of Large Numbers

Learn it → Apply it → Save as Skill

A small sample is mostly noise. A large one is mostly signal. As independent observations pile up, their average drifts closer to the true underlying rate. The model is a reason to ask how big your sample really is before you trust what it seems to show.

Why This Library Is Different

Learn it. Apply it. Save as Skill.

The Dr. Wallace's Favorite Mental Models is designed to turn useful ways of thinking into reusable tools. Each model gives you a fast visual overview, deeper interactive learning, practical application exercises and an AI-ready Markdown skill you can save for future use.

Instead of merely reading about mental models, you can gradually build your own AI skills library: a collection of portable reasoning skills that can help ChatGPT, Claude, Grok and other AI assistants apply the mental models you value.

30-Second Understanding
Trust grows with N.

A single outcome, or a handful of them, can look like a trend while being almost pure chance. The more independent, comparable observations you collect, the closer their average settles near the true long-run rate.

Core question: Is this pattern driven by the real underlying rate, or by a sample too small to tell the difference?

Same coin, two samples

70%
51%

Important: both bars describe the same fair coin. The small sample (10 flips) just happened to run hot. The large sample (10,000 flips) is close to the true 50%.

Mental Model Compass

Use it whenever a result might just be noise.

The model is most useful when a number looks meaningful but the sample behind it is small, new, or volatile.

InterpretPrimary use case
BeginnerDifficulty
3 minUnderstand
10–30 minApply with data
StatisticsA/B testingQuality controlInsurance & risk poolingPolling & surveysPerformance reviews
Model at a Glance

Define → Sample → Compare → Trust Proportionally.

The law describes convergence, not a guarantee. It tells you how much weight a sample size can bear.

01Define

Name the true rate or average you are trying to estimate: a conversion rate, a defect rate, a win rate, a score.

02Sample

Collect independent, comparable observations of that same underlying process, not a mix of different ones.

03Compare

Weigh your sample size against the stakes. A tiny sample can still be enough for a low-stakes, cheap-to-reverse call.

04Trust proportionally

Let confidence grow gradually with N. No finite sample gives certainty, only decreasing uncertainty.

When Should I Use This?

Recognize the situation.

Open a use case to see what the model changes.

Product & Growth
A test "wins" after a handful of visitors

Check how many visitors and conversions actually sit behind the headline percentage. A 2-of-10 result and a 200-of-1,000 result can show the same rate with very different reliability.

Operations & Quality
Defect rate jumped this week

A production run of 20 units can swing wildly by chance alone. Compare the week's volume to your typical batch size before declaring a process has gotten worse.

Insurance & Finance
Pricing a pool of unpredictable individual risks

Insurers can price uncertain individual outcomes because, pooled across enough comparable policyholders, the average claim rate becomes predictable even though any one claim is not.

People & Management
One great or terrible quarter from an employee

A single quarter is a sample of one. Look at a longer run of comparable periods before treating it as evidence of a lasting change in performance.

Research & Surveys
A poll of 40 people suggests a shift in opinion

Small polls have wide margins of error. Ask how large the sample is relative to the size of the shift being claimed before treating it as real movement.

Personal Decisions
One bad outcome from a normally sound process

A well-reasoned decision can still produce a bad result once. Judge the process across a longer run of similar decisions rather than discarding it after one unlucky draw.

Worked Example

Six visitors converted. Is the new button better?

This is an illustrative example designed to teach the method.

N=20
20%
N=100
14%
N=500
11.6%
N=2K
10.4%
N=10K
10.1%
N=50K
10.0%
SITUATIONNew checkout button, 4 of 20 visitors convert (20%)
BASELINEOld button's long-run rate is 10%
SAMPLE CHECK20 visitors is far too few to separate luck from a real lift
AS N GROWSBy 10,000 visitors the rate has settled near 10.1%
INSIGHTThe early "win" was mostly noise, not a better button
NEXTKeep the test running until the sample matches the size of the effect you need to detect
Do not conclude: "the button change did nothing." The honest conclusion is narrower: at 20 visitors there was not enough data to tell a real improvement from ordinary variation.
History & Accuracy

A three-century-old theorem with a very modern name.

Expand for the distinction between the original proof and the everyday phrase.

What did Jacob Bernoulli actually prove?

Bernoulli proved a rigorous version of the law for repeated binary trials (like coin flips) in his book Ars Conjectandi, published posthumously in 1713. He considered it important enough to call it his "golden theorem."

Who actually coined the name "law of large numbers"?

The French mathematician Siméon-Denis Poisson named it la loi des grands nombres in 1837, over a century after Bernoulli's proof. The name stuck even though the underlying mathematics is older.

What is the difference between the "weak" and "strong" law?

The weak law says the sample average becomes increasingly likely to sit close to the true average as the sample grows, with occasional larger deviations still possible at rare moments. The strong law says the average converges to the true value with near-certainty over the long run. For everyday use, the practical takeaway is the same either way: more good data means less room for chance to mislead you.

Common Mistakes

Simple does not mean safe to misuse.

Trusting a small sample

This is sometimes called the "law of small numbers": treating a handful of observations as if they carried the statistical stability of thousands. Small samples are far noisier than intuition expects.

The gambler's fallacy

Believing an independent future event "owes" a correction for past results, as if a coin that landed heads five times must be due for tails. The law describes long-run averages, not a short-term balancing force. See the related Gambler's Fallacy model.

Pooling non-comparable data

Mixing different populations, time periods, or conditions into one "large" sample can hide the true rate rather than reveal it. More rows are not more evidence if they are not measuring the same thing.

Confusing average with guarantee

The law governs where the average settles. It says nothing about what happens on any single draw, which can still be far from typical.

Ignoring dependence between observations

The convergence assumes roughly independent observations. Correlated samples, such as repeated measurements from the same source, shrink uncertainty far more slowly than the count alone suggests.

Peeking and stopping early

Checking a test repeatedly and stopping the moment it looks favorable inflates the chance of mistaking noise for a real effect, even if the final sample size looks adequate on paper.

When Not to Use It

Some situations resist convergence.

Use another lens when...

the underlying process is not stable: a rate that is genuinely drifting over time will not converge to one fixed number no matter how much data you collect.

Use severity analysis when...

a single rare event would be catastrophic. A low long-run average rate is no comfort if you cannot survive the one time it happens.

Use individual judgment when...

fairness, law, or ethics requires each case to be evaluated on its own facts rather than smoothed into a group average.

Counterpoint: averaging can also hide real differences between subgroups. Ask what the chosen population leaves out before you average across it.
Combine It With

One model rarely does the whole job.

Try It Yourself

Build a sample-size reality-check prompt.

This browser tool structures your thinking. It does not send your information anywhere or perform AI analysis.

Your AI-ready prompt

Complete the fields, then build your prompt.
Save as Skill

Turn this mental model into a reusable AI skill.

Save as Skill converts the model into portable Markdown instructions that teach an AI when and how to apply the Law of Large Numbers, what questions to ask, what evidence to seek, what mistakes to avoid, and how to structure its analysis.

Build your own Mental Models Skills Library

Save the skills you find useful and organize them into your own reusable library. The Markdown is intentionally model-neutral, so it can be used wherever an AI system accepts uploaded files, project knowledge, custom instructions or reusable text context. Platform capabilities differ, so this is a portable skill file rather than a claim of one-click installation.

Go Deeper

For readers who want the nuance.

Does more data always mean more certainty?

Yes, but with diminishing returns. Typical statistical uncertainty shrinks with the square root of the sample size, so cutting your margin of error in half generally takes about four times as much data, not twice as much.

What if the thing I'm measuring is changing over time?

The law assumes a stable underlying process. If the true rate itself is drifting, such as a market shifting or a team's skill improving, more historical data does not converge on a single fixed truth. In that case, recent, comparable data matters more than sheer volume.

How is this different from the Central Limit Theorem?

The Law of Large Numbers says the sample average gets closer to the true average as the sample grows. The Central Limit Theorem separately describes the shape of that remaining uncertainty, which tends to look like a bell curve regardless of the underlying data's shape. Together they are why confidence intervals and significance tests work at all.