Neural Networks
The Machine That Learns by Getting Things Wrong
Every time a neural network makes a mistake, it rewires itself — and that single loop of failure and correction is responsible for nearly every AI breakthrough of the last decade.
The Idea
Neural networks are not programmed in the conventional sense. You do not write rules and hand them to the machine. Instead, you expose the network to enormous amounts of data and let it discover patterns by repeatedly failing to predict the right answer, then adjusting itself in response to that failure. The mechanism driving this is called backpropagation — a process where the error at the output gets traced backwards through the network, and each connection gets nudged, very slightly, in a direction that would have made the error smaller. Do this millions of times, across millions of examples, and something remarkable emerges: a system that can recognise faces, translate languages, and generate prose, not because anyone told it how, but because the structure of its errors gradually shaped it toward competence. What makes this genuinely strange is that no one fully understands what the network has learned. The knowledge is not stored in any legible rule; it is smeared across billions of numerical weights — the strengths of connections between artificial neurons. The network becomes capable without becoming explainable. This is not a bug people are working around. It is the fundamental architecture of the most powerful AI systems in existence today, and it raises a question that engineers, philosophers, and regulators are all circling: can you trust a system whose reasoning you cannot read?
In the World
In the early 2010s, a team at the University of Toronto — Geoff Hinton, Alex Krizhevsky, and Ilya Sutskever — entered a prestigious annual computer vision competition called ImageNet, where systems compete to correctly label photographs from a set of a thousand categories. The competition had been running for years, and progress was incremental: teams would scrape out a percentage point of improvement with elaborate hand-crafted algorithms. Then, in 2012, Hinton's team submitted a deep neural network they called AlexNet. It did not edge ahead. It demolished the field. The previous best error rate was around 26 percent. AlexNet achieved 15.3 percent — a margin so large that many competitors initially assumed a mistake had been made. The victory announced something important: the era of hand-engineered features was over. You no longer needed domain experts painstakingly telling the system what to look for in an image. The network found it on its own, through scale, data, and the loop of error correction. Hinton would later receive the Nobel Prize in Physics in 2024, a recognition that this method of learning — messy, iterative, opaque — had quietly become one of the most consequential ideas in modern science. Nearly every system you interact with today that seems to understand you — your search engine, your voice assistant, the autocomplete on your phone — descends directly from the approach AlexNet vindicated that year.
Why It Matters
Understanding the basic logic of how neural networks learn changes how you interpret the AI tools already woven into your day. When a recommendation algorithm surfaces something eerily accurate, or a translation stumbles in a weirdly consistent way, you are seeing the residue of its training — the shape of its errors, frozen into weights. That reframe matters because it shifts the question you ask. Instead of 'is this AI smart?', you start asking 'what was it trained on, and what errors did it learn from?' Those are questions with real answers, and they point toward real consequences: biased data produces biased systems, not because anyone intended it, but because the failure signal that shaped the network reflected the world's existing patterns. The opacity of the learned weights is also worth holding onto. The fact that capability and interpretability have, so far, moved in opposite directions — the more powerful the model, the less readable its reasoning — is not a temporary engineering problem. It is a feature of the architecture itself, and it is the reason that questions about AI accountability are genuinely hard, not just politically contentious.
A Question to Ponder
If a system learns entirely from its mistakes but cannot explain what it has learned, at what point — if any — does trusting it become reasonable?
Get a new one of these every morning.
Start learning with Thinkable