Artificial Intelligence & Machine Learning
The Curve That Built Modern AI — and the Wall It Just Hit
For five years, the entire artificial intelligence industry ran on a single, almost absurdly simple belief: make it bigger, and it gets smarter.
The Idea
In January 2020, a team at OpenAI led by Jared Kaplan — with co-authors including Dario Amodei and Sam McCandlish — published a paper that turned AI development from craft into engineering. Their finding was striking in its tidiness: a language model's performance improves as a smooth power law across three variables — the number of parameters, the size of the training dataset, and the compute budget spent on training. What made this remarkable was the range over which it held: more than seven orders of magnitude. You could plot a curve using small models and predict, with reasonable confidence, how a model ten thousand times larger would perform before you built it. Architecture barely mattered. Attention heads, layer depth, network width — these were details. What dominated was sheer scale. This turned model-building from guesswork into forecasting, and it had an immediate consequence: spend more, get more. The race to pour compute and data into ever-larger models was not irrational hubris — it was the logical response to a reliable mathematical law. But two cracks appeared quickly. First, DeepMind's 2022 Chinchilla paper showed that the original Kaplan laws had led labs to build models that were too large for the data they were trained on — undertrained giants, in effect. The optimal ratio, Chinchilla argued, was roughly twenty tokens of training data per model parameter. Second, and more fundamentally, the gains from simply adding parameters have been visibly flattening at the frontier. As datasets of high-quality public text approach exhaustion, and as each incremental performance improvement requires exponentially more compute, the brute-force era is running into hard physical and economic limits.
In the World
The clearest illustration of what comes after scaling arrived from an unexpected direction. In January 2025, the Chinese lab DeepSeek released a reasoning model called R1 that matched OpenAI's o1 in performance on a range of benchmarks, reportedly at a fraction of the training cost — achieved on export-restricted chips that lacked the raw compute of the hardware powering frontier Western models. The demonstration was stark enough to rattle markets and force a public reckoning inside the industry: if algorithmic efficiency could substitute for raw compute, the "just spend more" playbook had limits that money alone could not solve. OpenAI's response — and the broader industry's — pointed toward a different kind of scaling law. Rather than making models larger during training, reasoning models like o3 and o4-mini scale at inference time: they are given more compute to think through a problem before answering. OpenAI noted during o3's development that large-scale reinforcement learning exhibited the same more-compute-equals-better-performance trend they had seen in pre-training — but this time applied to how long the model reasons, not how many parameters it has. On the American Invitational Mathematics Examination, an upper-tier high school competition, test-time scaling helped push performance from roughly 12 percent — the ceiling for a standard large language model — to over 85 percent for reasoning models. By 2026, a McKinsey survey found that roughly one in five organisations was already limiting AI use due to operating costs. The bottleneck had shifted from building models to running them.
Why It Matters
Scaling laws matter beyond the lab because they shaped an entire decade of investment logic. Hundreds of billions of dollars flowed into AI infrastructure on the implicit assumption that the performance curve would keep climbing if you kept feeding it. The fact that this assumption is now under serious pressure does not mean AI stops improving — it means the nature of the improvement changes, and that change has consequences. If the next gains come from reasoning at inference time rather than from training ever-larger models, the economics shift. Efficiency and architecture begin to matter more than raw compute budgets. Smaller, smarter, more specialised systems may outperform monolithic ones on the tasks that actually matter to most users. The lab with the most chips wins a different game than the lab with the best algorithms. For anyone trying to understand where AI is actually heading, the lesson is this: the story has never really been about intelligence emerging from size. It was about a convenient proxy — scale — that worked extraordinarily well for a while, and is now giving way to harder, subtler questions about how systems reason, generalise, and learn efficiently. That is a more interesting problem. It is also a less predictable one.
A Question to Ponder
If raw scale can no longer be relied on to deliver the next leap in AI capability, what does it mean that so much of the world's AI infrastructure was built around the assumption that it could?
One reader's Thinkable for Tech Tuesday — a random draw from the library.
Start learning with Thinkable