The original claim from Charles Goodhart in 1975:
Any observed statistical regularity tends to collapse once pressure is placed on it for control purposes.
Marilyn Strathern’s reformulation of Goodhart’s Law in 1997:
When a measure becomes a target, it ceases to be a good measure.
Both explain that a proxy that tracked something you cared about stops tracking it once you start optimizing on the proxy itself.
Goodhart’s Law and the heuristics-and-biases program in cognitive psychology describe the same pattern at different levels of analysis: a proxy that once tracked what you cared about stops tracking it once you start optimizing the proxy itself.
Most people read Stathern’s version as regarding institutions: KPIs that get gamed, test scores that stop measuring learning, citation counts that stop measuring contribution. It also functions at the scale of a single person, on a millisecond timescale, without anyone optimizing for it.
Kahneman called the cognitive version attribute substitution. When the mind faces a hard target question - is this person competent? Is this investment good? - it silently substitutes an easier proxy question: do they sound confident? Does this feel familiar? - and answers the proxy. The proxy once correlated with the target. Once it becomes the sole basis for judgment, the correlation can break, and you don’t notice, because you’re no longer looking at the target - that’s Strathern.
This reframing matters because it clarifies what counts as a heuristic, what counts as a bias, and what lived experience is actually doing.
A heuristic is a proxy with ecological validity - it works because it exploits reliable structure or patterns in a specific environment:
- The recognition heuristic tracks importance - you’ve heard of one brand on the shelf and not the other, so you reach for the one you know. This evolved from environments where important things get talked about.
- The availability heuristic tracks frequency - the plane crash on the news, the shark attack you remember. This came from environments where exposure matched occurrence.
- The representativeness heuristic tracks probability - the profile of someone quiet and bookish matches a librarian, not a farmer. This came from environments where surface features correlate with the underlying type.
- The affect heuristic tracks value - a technology that makes you uneasy feels riskier than one that doesn’t. This came from environments where unease was earned by real consequence.
- The anchoring heuristic tracks reasonable estimates - a jacket marked down from 200 feels like a bargain. This came from environments where the first number came from someone informed, not someone trying to move you.
A heuristic is a good measure until the environment shifts, or until somebody starts gaming it. Adversaries exploit proxies that used to track something but became the target of optimization - phishing, advertising, and propaganda are all deliberate Goodhart’s Law attacks on cognitive proxies.
Biases are heuristics that outlived their environment. The negativity bias was a good measure of survival risk on the savanna. But when you apply it to a social media feed engineered to maximize engagement, the proxy becomes actively anti-correlated with what it was meant to track.
On Feedback
Experience can fix this or worsen bias, depending on feedback quality and analysis.
Gary Klein’s naturalistic expertise - chess masters, firefighters, neonatal nurses - sharpens heuristics in environments with tight, honest feedback. Proxies get pulled back toward the target every time they drift - there is ground truth.
Phil Tetlock’s experts in low-feedback domains- geopolitics, stock-picking, and political punditry- show the opposite: experience accumulates confidence without accuracy.
The same mechanism can lead to wisdom or delusion depending on whether the environment punishes proxy drift - ‘twenty years of experience’ can mean twenty years of corrected error, or twenty years of unchallenged proxy use - they look identical from the outside.
Strangely, System 2 thinking doesn’t necessarily escape this either; it just relocates it. Explicit metrics - KPIs, GDP, test scores, citation counts - when analyzed, can become System 2’s heuristics. Entire job roles are built around creating, facilitating, and analyzing metrics in a thoughtful System 2 way. But without security modeling and weighting for qualitative analysis, they are more Goodhart-vulnerable, not less. Legibility to the thinker is also legibility to the optimizer - making the proxy explicit is what makes it gameable. The institutional version of attribute substitution is the analytics dashboard. Such dashboards corrupt downstream judgment in exactly the way the brain’s silent substitutions corrupt individual decisions - you believe you have all the information you need to make a reasonable judgment. Still, you lack the relevancy realization and security modeling of the metrics. Without the willingness to look at the qualitative, you won’t have them until disaster hits or a threshold of critical feedback becomes unignorable.
On Shorthand & Jargon
Firstly, language is metaphor, and when heuristics are collapsed into or related to a single term, there is a risk that heuristics learned without their history are already decoupled. Words like synergy, alignment, leverage, or domain-specific terminology emerged from specific situations as compressed labels for dynamics someone had actually observed. Unfortunately, when people use these words too often without deeper consideration or reference, or pick them up through cultural osmosis, the secondhand acquisition travels as a label, with the underlying learning of why it matters in the first place forgotten. The recipient holds a token they can pattern-match on but can’t unpack. When the token shows up, they have no way to check whether the underlying thing is actually present. Substance and rhetoric become indistinguishable because the discriminator was either never installed or forgotten. It’s alright if you know a word and its sentiment; it’s great if you know its definition and when to use it with precision - know its semantic utility; it’s incredible when you have a lived experience associated with the semantics. Otherwise, the term is reduced to rhetoric.
This is the structural condition behind recent research on bullshit-receptivity, which showed that people who score high on susceptibility to corporate-speak have a defect of derivation. They are running on imported caches of terms with no fallback. When asked to evaluate a claim, they can’t fall back to the underlying dynamics, because the underlying dynamics never accompanied the words. The remedies usually proposed for this - plain language norms, clarifying-question rituals, leadership ‘critical thinking checks’-treat symptoms. The deeper issue is selection. Organizations that reward rhetorical fluency at promotion gates accumulate people whose competitive advantage is rhetorical fluency.
The cleanest way to reframe System 1 and System 2 once you’ve read all this is: it isn’t just fast versus slow; it’s cached versus generated versus experienced. A cached judgment or term is reliable only if you cached it yourself in a high-feedback environment, or if you can re-derive it on demand. The buzzword-fluent employee fails both tests - the cache was loaded by exposure, not experience, and there is no derivation procedure to fall back on. The expert firefighter passes both - feedback shaped the cache, and they can reconstruct, in slow time, why the cache is correct. The label “intuition” applies to both groups, but they got there by doing fundamentally different things. The latter is far better off leading, judging, and arriving at a consensus with the team around them.
The unifying claim under all of this is simple. We rarely act on the thing we want to act on. We often act on representations of it. Goodhart’s Law is the observation that representations decouple from referents under optimization pressure. The heuristics-and-biases program is the same observation about cognition. The mechanism is identical; only the optimizer changes - natural selection, deliberate gaming, or organizational incentive. Once the proxy is what’s being optimized, the signal becomes diffuse.
The practical question this leaves you with is uncomfortable. For any judgment you trust yourself to make: do you have feedback that would tell you when your proxy stopped tracking? If not, you should assume it has, and you haven’t noticed.
Appendix: Evaluation as the Goodhart Site
The argument has an obvious application to one place where these failures concentrate: performance evaluation.
An evaluator without domain competence has the same problem as the buzzword-fluent employee - they are running on imported caches. They pattern-match on what competent work looks like with no derivation procedure underneath. When the work in front of them looks competent, they can’t check whether it is. Their proxies - charisma, confidence, output legibility, deck quality - are all they have. So their proxies are what get optimized against.
This selection mechanism is what makes theatre dominate. Specialists optimize for what their evaluators can grade. Evaluators who can only grade legibility get work optimized for legibility. The substance underneath isn’t penalized when it’s missing, because the evaluator can’t see it missing. Over a long enough timespan, the people who survive are the ones who learned to perform competence at the resolution the evaluator can detect. The people who actually do the work either leave or learn to package it the same way. The Goodhart attack runs continuously, without anyone deciding to attack.
The fix isn’t more rigor from the evaluator’s existing toolkit - that’s just sharpening the proxy. The fix is making sure the evaluator has at least one domain where they have generated, not cached, precise terminology and judgment. Evaluate Your Evaluator makes the case in detail: a founder who has built something hard can read the artifacts the people they evaluate produce, and has specialized long enough to know what mastery costs. Those three are the conditions for derivation. Without at least one, the evaluator is entirely a cache.
Footnotes
- This could be why people did the The Custody Illusion.
- This is also precisely the problem with AI Code Agent evaluation, and whether or not AI can write good code. The software industry seriously needs to consider whether we are even reading code anymore, or if we are reading its legibility.