The Ethics of Technology
Who Dies When the Brakes Fail? The Hidden Moral Architecture of Self-Driving Cars
Before a single autonomous vehicle ships, someone — an engineer, a product manager, a philosopher hired by a tech company — has already decided whose life is worth more than whose.
The Idea
The trolley problem is philosophy's most famous thought experiment: a runaway trolley is heading toward five people; you can pull a lever to divert it onto a track where it will kill one. Do you act, and if so, are you responsible for that death? It was always meant to be a probe for moral intuitions, not a practical dilemma. Then we started building cars that drive themselves. Autonomous vehicles must be pre-programmed with decision hierarchies — not because crashes are common, but because the software needs to resolve edge cases in milliseconds, long before any human could react. That means designers must encode trade-offs. Does the car prioritise the passenger? Pedestrians? The person who darted into the road versus the one using the crossing correctly? An elderly pedestrian versus a child? What makes this genuinely thorny is not the decision itself but the act of making it in advance, at scale, and in code. When a human driver swerves and causes harm, we treat it as a tragedy. When a manufacturer programs a vehicle to make the same choice across a million cars, it starts to look like policy — and policy implies intent, accountability, and the possibility of being wrong in systematic ways. The deeper discomfort is that these trade-offs can't be avoided by simply not choosing. Refusing to encode a preference is itself a preference. Every line of code, or its absence, is a moral position.
In the World
In 2016, researchers at MIT launched the Moral Machine — an online experiment that presented millions of participants with variations on the autonomous vehicle trolley problem. Should the car spare a doctor or a criminal? An athlete or an overweight pedestrian? Five elderly people or one child? Over 40 million decisions were collected from more than 230 countries. The results were uncomfortable in several ways. Preferences were not universal — they clustered by region, culture, and economic context. People in more individualistic cultures tended to prioritise the young over the old more strongly than those in collectivist societies. Across the board, humans preferred sparing more lives over fewer, sparing the young over the old, and sparing humans over animals. But within those broad tendencies, the variation was significant enough to make any single global standard look like a cultural imposition. What MIT's experiment actually demonstrated was that there is no neutral answer — and no answer that will feel fair to everyone. Mercedes-Benz briefly made headlines when an executive appeared to suggest their cars would always prioritise the passenger. The backlash was swift. Volvo said they'd try to avoid the situation entirely through better sensor systems, which is philosophically honest but sidesteps the question rather than answering it. Regulators have largely not caught up. The EU's AI Act touches on high-risk systems, but the specific moral calculus inside crash-avoidance algorithms remains, for now, mostly unlegislated — and mostly invisible.
Why It Matters
This isn't a thought experiment that stays safely in philosophy seminars. Autonomous vehicles are already on public roads — in San Francisco, Phoenix, parts of China — and their numbers are growing. The decisions baked into their code will affect real people before any democratic process has properly weighed in on what those decisions should be. There's a broader principle here worth carrying into how you think about technology generally: the most consequential ethical choices often get made not in boardrooms or legislatures, but in product specs and default settings. The people writing those specs are rarely malicious — they're usually just trying to ship — but that doesn't make the choices less real or less worth scrutinising. Asking 'who decided this, and on what basis?' is one of the more useful questions you can bring to any technology that affects your life. Autonomous vehicles make that question unusually visible. Most of the time, it's buried much deeper.
A Question to Ponder
If you had to choose the moral framework — utilitarian, precautionary, passenger-first — that governs the car you or someone you love rides in, which would you pick, and could you live with it being applied to everyone else too?
Get a new one of these every morning.
Start learning with Thinkable