Surveillance States
The Camera That Knows You're Nervous Before You've Done Anything Wrong
China's surveillance network doesn't just watch you — it predicts you, and the distinction matters more than most people realise.
The Idea
Most discussions of surveillance states get stuck on the obvious: cameras, tracking, data collection. What makes the current generation of systems genuinely different is the shift from observation to inference. The older model was reactive — catch someone doing something, then prosecute. The emerging model is predictive: identify behavioral signals that correlate with future non-compliance, and intervene before any act occurs. China's Social Credit System is the most-cited example, though it is less a single unified system than a patchwork of local government programs and private platforms. What they share is the attempt to convert behavior — purchasing habits, online speech, debt repayment, public conduct — into a numerical proxy for trustworthiness. Cameras equipped with emotion-detection software have been trialled in schools and detention centers, claiming to read anxiety or deceit from micro-expressions. The philosophy underneath this is Benthamite surveillance logic taken to its digital extreme: if people believe they are always being watched, they will self-regulate. Except the new version doesn't even need belief. It just needs data. The panopticon assumed a human guard. The algorithmic version removes the guard and replaces them with a pattern-matching engine that never sleeps, never gets bored, and never forgets a face. What makes this intellectually unsettling isn't just the authoritarianism — it's the epistemology. These systems treat correlation as cause, and proxy scores as reality. A low score doesn't mean you are untrustworthy; it means the model thinks you are. That gap is where injustice lives.
In the World
In 2018, a man named Ao Hui became one of the first widely-reported cases of China's facial recognition infrastructure being used to intercept someone in real time. He was located and arrested at a pop concert attended by roughly 60,000 people in Nanchang — identified within minutes of arrival by cameras cross-referencing a national database. Authorities described it as a triumph. Ao was wanted on economic crime charges, so the mechanics worked exactly as designed. But the same infrastructure was simultaneously being used in Xinjiang in ways that attracted a very different kind of attention. Researchers and journalists documented a system in which Uyghur residents were flagged algorithmically for behaviors as ordinary as using a VPN, owning a compass, or having relatives abroad. The outputs fed a detention system that by some credible estimates held over a million people without trial. The Xinjiang case is instructive precisely because the technology was not malfunctioning. It was doing what predictive surveillance systems are designed to do — surface people who deviate from a behaviorally-defined norm. The problem is that the norm was politically constructed, and deviation from it was treated as proto-criminality. No act required. No harm done. Just a pattern the model had learned to distrust. This is what separates contemporary surveillance states from their 20th-century predecessors: the Stasi relied on informants and paper files. The new version runs at scale, at speed, and with the false authority of mathematics.
Why It Matters
Most people in democratic countries follow this topic as a kind of horror story about elsewhere. That framing is increasingly hard to sustain. The tools — facial recognition, behavioral analytics, emotion detection, predictive policing software — are commercially available and politically attractive in many contexts well beyond authoritarian states. Several European cities trialled facial recognition in public spaces before legal challenges slowed deployment. Predictive policing software has been used across cities in the United States and United Kingdom, with documented racial bias baked into the training data. The question worth sitting with isn't 'could this happen here' — versions of it already have. The more useful question is about the epistemological bargain being struck when a government or institution outsources judgment to a model. When a system flags you as high-risk, who is accountable for that designation? What evidence can you contest, and with what mechanism? Surveillance states don't announce themselves. They emerge through incremental decisions — each individually defensible, collectively transformative. Understanding the architecture helps you see the trajectory earlier, which is the only point at which course corrections are genuinely possible.
A Question to Ponder
If a system can predict your behavior more accurately than you can explain it yourself, does that predictive power constitute knowledge about you — or something more troubling than knowledge?
Get a new one of these every morning.
Start learning with Thinkable