Decision Theory / Preference Formation
You Didn't Choose That — It Chose You
Most of what you call a preference was installed in you before you were old enough to question it.
The Idea
Decision theory has a quiet assumption buried at its foundation: that you arrive at your choices with preferences already formed, stable, and genuinely your own. The math of rational choice — weighing options, maximising utility — only works if the thing doing the weighing is a coherent self with coherent desires. But preference formation, the process by which you come to want what you want, is far messier and stranger than the models admit. Psychologists and philosophers have accumulated substantial evidence that preferences are not discovered, they are constructed — often on the fly, often under the influence of whoever is watching. What you claim to want shifts depending on how a question is framed, what options sit nearby, and what you imagine others will think of your answer. The preference feels real and internal. The process generating it is largely external and situational. What makes this genuinely unsettling is not just that we are influenced — everyone grants that. It is that we are influenced without noticing, and then we confabulate. We experience the preference first, then produce a reason for it afterward, and mistake the explanation for the cause. This is what the philosopher Jonathan Haidt called the 'emotional tail wagging the rational dog.' The decision-making machinery you trust most — your considered judgment — is often a post-hoc press office spinning a story about something that was already decided downstream.
In the World
In the 1970s, psychologists Amos Tversky and Daniel Kahneman ran a deceptively simple experiment. They asked people to spin a wheel — rigged, though participants didn't know it — that landed on either 10 or 65. Then they asked: what percentage of African nations belong to the United Nations? People who'd seen 65 guessed far higher than those who'd seen 10. The number was arbitrary, completely irrelevant, and yet it colonised their judgment. They had a preference for a particular answer, and that preference was shaped by a random spin of a wheel seconds earlier. This is anchoring, and it turns up everywhere preferences form under uncertainty — salary negotiations, property valuations, medical diagnoses. But it points to something deeper than a cognitive bias catalogue entry. It suggests that the moment you ask someone what they want, you are not taking a reading of a pre-existing internal state. You are participating in the construction of that state. Perhaps the most striking version of this comes from work on 'choice-supportive bias': after making a decision, people reliably remember the chosen option as better than they rated it before choosing, and the rejected option as worse. We don't just choose what we prefer. We learn to prefer what we chose. Preferences and choices are not cause and effect — they are in a continuous loop, each reshaping the other.
Why It Matters
If your preferences are partly constructed by context, framing, and accident, then the question 'what do I actually want?' becomes one of the most important questions you can ask — and one of the hardest to answer honestly. This is not cause for paralysis or nihilism. It is cause for a particular kind of alertness. When you feel a strong pull toward something, it is worth pausing to ask: where did this preference come from? Was it formed in conditions I'd endorse on reflection — calm, informed, free from social pressure — or was it shaped by something I wouldn't choose as an influence if I saw it clearly? Mindful practice has something useful to offer here, not because meditation dissolves preferences, but because it creates enough space between stimulus and response to notice the machinery working. You start to catch the moment a preference assembles itself, rather than simply receiving it as given. The goal isn't to have no wants. It is to want with more awareness — to be the author of your preferences where you can, rather than the vehicle for ones that were handed to you without your knowledge.
A Question to Ponder
Is there a preference you hold firmly that, on reflection, you cannot trace back to a moment of genuine, uninfluenced choice — and what would it mean to examine it honestly?
Get a new one of these every morning.
Start learning with Thinkable