Artificial Intelligence & Machine Learning
The Snake That Eats Itself: What Happens When AI Learns from AI
A model trained to describe medieval architecture, fed only on its own outputs for nine generations, eventually produced nothing but lists of jackrabbits.
The Idea
Every large AI model is built on a vast diet of human-generated text, images, and code — the accumulated record of human thought scraped from the web. But the web is no longer purely human. As AI-generated content floods the internet, the training data for future models is increasingly made of outputs from previous models. This is the setup for what researchers call model collapse: a degenerative feedback loop in which each generation of AI learns from a slightly corrupted version of the last. The mechanism is subtle but consequential. When a model generates text, it does so by sampling from a probability distribution — most outputs cluster near the centre, while rare, unusual, or nuanced outputs live in the 'tails'. When those outputs become training data for the next model, the tails shrink. Each generation loses a little more of what was uncommon, specific, or strange. The model becomes more confident, more average, and less representative of the original range of human expression. After enough iterations, the outputs converge toward a narrow, repetitive mean — statistically smooth but intellectually impoverished. A landmark 2024 paper published in Nature by Ilia Shumailov and colleagues at Oxford, Cambridge, Toronto, and Imperial College demonstrated this empirically, showing that the effect appears universally across model types: language models, image generators, and simpler statistical models all exhibit the same collapse when trained recursively on their own outputs. Their term for it — model collapse — has since entered the vocabulary of AI research as one of the field's most pressing structural concerns.
In the World
The medieval architecture experiment from the Shumailov paper makes the problem vivid. Researchers started with real, human-written text about the subject, trained a model on it, then used that model's outputs to train the next model, and repeated the process. By the ninth generation, the model had drifted so far from its origins that it was generating lists of jackrabbits. No single step looked catastrophic — the drift was gradual, cumulative, and entirely invisible if you only looked at one generation at a time. This isn't a controlled-lab curiosity anymore. Research published in 2026 estimated that roughly 24 percent of all crawlable web pages already contain AI-generated or AI-edited text — up from near zero before 2022, and growing at an accelerating rate. By April 2025, over 74 percent of newly published web pages contained detectable AI-generated content. Models being trained today are already ingesting significant quantities of synthetic material, often without any mechanism to distinguish it from human-written text. The picture is complicated further by a 2025 finding sometimes called 'strong model collapse': even mixing a small fraction of synthetic data into a predominantly real dataset can stall the gains that come from scaling — the usual assumption that more data means better performance. As little as one synthetic sample in a thousand, in certain configurations, can bend the scaling curve in the wrong direction. The question of how to maintain a living, human-generated signal in training data — and how to verify what is real — has become one of the most active areas in AI research.
Why It Matters
Model collapse is easy to dismiss as a technical problem for the engineers building these systems. But its implications reach further than the lab. If the richness, diversity, and strangeness of human language and thought are gradually smoothed away through recursive training, the AI tools that hundreds of millions of people use to write, reason, and create will quietly converge toward a narrower and narrower idea of what good output looks like. The models would not be wrong, exactly — they would just be relentlessly average. There is also a feedback loop that concerns researchers at the level of culture rather than just computation. As AI-generated content fills the web, human writers, artists, and thinkers face economic and competitive pressure. If the fraction of genuinely human-generated content online continues to shrink, future models will have an ever-thinner real-world signal to anchor them. The antidote the research suggests is straightforward in principle: keep real human data in the mix, accumulate rather than replace, and verify the provenance of training material. But doing this at scale, in a world where synthetic and human content are increasingly indistinguishable, is a harder problem than it sounds.
A Question to Ponder
If AI systems gradually lose access to the full range of human thought — the rare ideas, the unusual phrasings, the minority perspectives — what does that mean for the people who rely on those systems to help them think?
One reader's Thinkable for Tech Tuesday — a random draw from the library.
Start learning with Thinkable