Statistics & Data
The Data That Lies While Telling the Truth
A hospital can have worse survival rates than its rival in every single department, and still be the better hospital overall.
The Idea
Simpson's paradox is one of those statistical phenomena that should permanently change how you read a data summary. It occurs when a trend that appears consistently across several groups of data disappears — or outright reverses — when those groups are combined. The aggregate tells a story that is the exact opposite of what every individual subset is saying, and both are, in a narrow technical sense, correct. The mechanism is surprisingly simple once you see it. When groups are different sizes, or when a lurking third variable is distributing unevenly across them, the combined average gets pulled toward whichever group dominates the pooled dataset. The minority group's signal gets swamped. You end up with a number that is arithmetically accurate but logically misleading — a perfectly valid statistic pointing you firmly in the wrong direction. What makes this more than a statistical curiosity is how often it quietly distorts real decisions. Medical treatments, hiring patterns, educational interventions, sports rankings — anywhere you aggregate across meaningfully different subgroups, the paradox is waiting. The troubling part is not that the data is wrong. The troubling part is that nothing flags it as suspicious. It looks like any other summary statistic. The paradox doesn't announce itself. You have to already be looking for it, which means the biggest danger is the confident analyst who isn't.
In the World
The most consequential real-world instance of Simpson's paradox appeared in a 1973 study of graduate admissions at UC Berkeley. On the surface, the numbers looked damning: the university admitted around 44 percent of male applicants and only 35 percent of female applicants. Discrimination seemed straightforward to demonstrate. But when researchers Bickel, Hammel, and O'Connell broke the data down by individual department, the pattern reversed. In most departments, women were being admitted at equal or higher rates than men. How? Women were disproportionately applying to the most competitive, oversubscribed departments — humanities and social sciences — where admission rates were low for everyone. Men were applying in larger numbers to departments with much higher acceptance rates, like engineering. So when you pooled everything together, men appeared to benefit from an advantage that, at the departmental level, mostly didn't exist. The aggregate number was real. The discrimination it implied was largely an artifact of a lurking variable: choice of department. The Berkeley case is now a standard teaching example, but its deeper lesson is easy to miss. The researchers didn't set out looking for Simpson's paradox. They found it because they were rigorous enough to disaggregate. Most analyses don't go that extra step. And the consequences of stopping at the summary — in policy, in law, in medicine — can be profound.
Why It Matters
Knowing Simpson's paradox exists doesn't just make you better at reading studies. It changes the question you ask first. Instead of 'what does the data show?', you start asking 'what is this data pooling across, and does that pooling make sense?' That is a fundamentally different, and more honest, relationship with numbers. It also gives you a principled reason to be sceptical of clean, confident summaries — not in a nihilistic way, but in a specific, directed way. When someone presents an aggregate figure to support a claim about groups, about treatments, about performance, the useful response is not to distrust the number but to ask about the composition underneath it. What are the subgroups? Are they the same size? Is there a variable — age, severity, prior experience — that is distributed unequally across them? Most misinformation doesn't come from fabricated data. It comes from real data, summarised carelessly or strategically. Simpson's paradox is a reminder that aggregation is not a neutral act. Every time you collapse detail into a single number, you are making a choice about what to preserve and what to lose — and that choice shapes what the number appears to say.
A Question to Ponder
Is there an area of your life — your work, your health, your understanding of a social issue — where you have been trusting a summary statistic without ever asking what subgroups it might be quietly averaging over?
Get a new one of these every morning.
Start learning with Thinkable