Reproducibility Crisis
When Science Can't Repeat Itself
More than half of psychology studies that researchers tried to replicate in 2015 simply failed to reproduce — and that wasn't the scandal; the real scandal is how unsurprised the insiders were.
The Idea
Science's most powerful feature isn't the experiment — it's the idea that any experiment worth believing should produce the same result when someone else runs it. Reproducibility is the immune system of scientific knowledge. So when a coordinated effort called the Reproducibility Project attempted to replicate 100 published psychology studies in 2015, and only 39 held up, it sent a legitimate tremor through the research world. But the crisis isn't simply about fraud or sloppiness, though both exist. It runs deeper. The structure of academic publishing quietly rewards novelty over confirmation. Journals hunger for surprising positive results; they have little appetite for 'we tried this and it worked exactly as expected' or, worse, 'we tried this and nothing happened.' This creates a filter that inflates the literature with findings that are statistically fragile. There's also a practice called p-hacking — not quite cheating, but not quite clean either. Researchers collect data, test multiple hypotheses, and report only the one that crossed the significance threshold, without disclosing the others. The result looks like a clean finding. It often isn't. Add small sample sizes, underpowered studies, and unconscious experimenter bias, and you get a body of published science that is, in patches, a hall of mirrors — each paper reflecting what researchers hoped to find rather than what reliably exists. The question isn't whether science is broken. It isn't. The question is which parts of the edifice are load-bearing and which are decorative.
In the World
In 2011, social psychologist Daryl Bem published a paper in one of the most prestigious journals in his field. It reported statistically significant evidence that people could sense future events — precognition. The methodology was rigorous by conventional standards. The p-values cleared the accepted threshold. It passed peer review. The paper became a slow-motion crisis. Not because anyone seriously believed in precognition, but because if you applied the standard tools of psychological research honestly, you could produce evidence for something impossible. Several teams tried to replicate Bem's results. They couldn't. But getting those negative results published proved surprisingly difficult — journals weren't interested in 'we ran the study and found nothing.' The Bem episode crystallised something many researchers had privately sensed: the machinery of publication was miscalibrated. It was optimised to generate interesting claims, not reliable ones. What followed was arguably the most honest period of self-examination in modern social science. The Center for Open Science launched large-scale replication efforts. Preregistration — where researchers publicly log their hypotheses and methods before collecting data, removing the temptation to reframe after the fact — started gaining traction. Some journals began accepting 'registered reports,' committing to publish a study regardless of outcome, before the results were known. The Bem paper didn't break psychology. It exposed a vulnerability that, once named, could be addressed. That is, for what it's worth, a very scientific response.
Why It Matters
Most of us will never run a study or read a methods section. But we absorb scientific claims constantly — through news headlines, health advice, policy debates, workplace training programmes built on findings about human motivation or memory or bias. If some of that research is built on sand, it matters practically, not just philosophically. Knowing about the reproducibility crisis changes how you hold scientific findings. Not with cynicism — 'science is all wrong' is as confused a response as 'science says so, therefore it's true.' The more useful posture is calibrated scepticism: single studies are hypotheses, not verdicts. Replicated findings across independent labs carry real weight. Effect sizes matter as much as significance. And the fields that have most actively grappled with reproducibility — biology, psychology, nutrition science — are now producing more trustworthy work precisely because they were forced to look hard at themselves. The crisis, counterintuitively, might be the best evidence that science works. It found the bug, named it, and started patching it.
A Question to Ponder
If a finding you've built a belief around — about willpower, sleep, or how people make decisions — turned out not to replicate, how would you actually change your behaviour, and what does your answer reveal about how you really use scientific evidence?
Get a new one of these every morning.
Start learning with Thinkable