Large Language Models
The Autocomplete That Learned to Reason (Sort Of)
The most powerful text-generating systems in the world were not designed to understand language — they were designed to predict the next word, and something unexpected fell out of that.
The Idea
At their core, large language models do one thing: given a sequence of text, they estimate which token — roughly, which word or word-fragment — is most likely to come next. That's it. The architecture, called a transformer, does this by learning statistical relationships between words across enormous bodies of text, building internal representations of how language patterns cluster and co-occur. There is no explicit grammar module, no knowledge database, no reasoning engine deliberately wired in. What makes this strange and worth sitting with is that capabilities researchers didn't specifically build for — logical inference, translation, basic arithmetic, analogical reasoning — appear to emerge as the models scale up. Not gradually, but sometimes abruptly, at certain thresholds of size and training data. This phenomenon, called emergent capability, is genuinely not well understood. It's as if you kept adding musicians to an orchestra and, past a certain headcount, they spontaneously started playing jazz. The implication that unsettles many researchers is this: we don't fully know what these systems have learned, or why they can do what they can do. The model is not following rules we gave it. It has compressed patterns from human writing into billions of numerical weights — and whatever understanding, or simulation of understanding, lives in that compression is not directly inspectable. We can probe its behaviour from the outside, but the inside remains, for now, largely opaque.
In the World
In 2022, a team at Google Brain published a paper tracking over 100 distinct capabilities across a family of language models as they scaled from small to large. Most abilities improved smoothly and predictably. But a handful — including multi-step arithmetic, answering questions about fictional scenarios, and a task called 'word in context' — showed something different: near-zero performance at smaller scales, then a sharp, sudden improvement at a particular size threshold. The team called these 'emergent abilities' and noted they had no good mechanistic explanation for them. The paper triggered a productive argument in the field. Some researchers pushed back, arguing the apparent jumps were artefacts of how performance was being measured — that smoother, continuous improvements would appear if you used more granular metrics. Others maintained that something genuinely discontinuous was happening inside the model at certain scales. The debate has not fully resolved. What made the original finding so striking wasn't just the capabilities themselves — it was the implication that we might not be able to predict in advance what a larger model will be able to do. You could, in principle, train a system and discover it has acquired a new skill only after the fact, by testing for it. For a technology being deployed at global scale, that is a remarkable admission of uncertainty from the people building it.
Why It Matters
This isn't just a technical footnote. The opacity of large language models sits at the centre of almost every serious debate about how to use them wisely — in medicine, in law, in education, in democratic processes. When a system does something impressive, neither its creators nor its users can always explain why it worked. When it fails — confidently generating false information, reproducing biases embedded in its training data, or breaking down on problems that seem simpler than ones it solved easily — the failure is equally hard to trace. Understanding this changes how you interact with these tools. It's not that they're unreliable and therefore useless — they're often extraordinarily useful. It's that their reliability is domain- and context-dependent in ways that aren't always obvious from the outside. Treating the output as a first draft to interrogate rather than a final answer to accept isn't just good practice; it's an accurate response to what these systems actually are. Being a thoughtful user of this technology starts with resisting the temptation to anthropomorphise it — to assume that fluency means understanding.
A Question to Ponder
If a system can produce outputs that are functionally indistinguishable from reasoning — solving novel problems, drawing inferences, catching its own errors — does it matter whether anything like understanding is actually happening inside it?
Get a new one of these every morning.
Start learning with Thinkable