The original claim from Charles Goodhart in 1975:
Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes
Marilyn Strathern’s reformulation of Goodhart’s Law in 1997:
When a measure becomes a target, it ceases to be a good measure.
Both explain that a proxy that tracked something you cared about stops tracking it once you start optimising on the proxy itself.
Goodhart’s Law and the heuristics-and-biases program in cognitive psychology are the same observation at different levels of analysis. Which isn’t usually how either gets taught.
Most people read Stathern’s version as regarding to institutions: KPIs that get gamed, test scores that stop measuring learning, citation counts that stop measuring contribution. It’s also functioning at the scale of a single person, on a millisecond timescale, without anyone optimising for it at all.
Kahneman called the cognitive version attribute substitution. When the mind faces a hard target question - is this person competent?, is this investment good? - it silently substitutes an easier proxy question - do they sound confident?, does this feel familiar? - and answers the proxy. The proxy was once a measure that correlated with the target. Once it becomes the sole thing you actually decide on, the correlation can break and you don’t notice, because you’re no longer looking at the target - that’s Strathern.
This reframing matters because it adds nuance to what counts as a heuristic, what counts as a bias, and what lived experience is actually doing.
A heuristic is a proxy with ecological validity - it works because it exploits reliable structure or operational patterns in a specific environment:
- The recognition heuristic tracks importance - you’ve heard of one brand on the shelf and not the other, so you reach for the one you know. This evolved from environments where important things get talked about.
- The availability heuristic tracks frequency - the plane crash on the news, the shark attack you remember. This came from environments where exposure was proportional to occurrence.
- The representativeness heuristic tracks probability - the profile of someone quiet and bookish matches librarian, not farmer. This came from environments where surface features correlate with the underlying type.
- The affect heuristic tracks value - a technology that makes you uneasy feels riskier than one that doesn’t . This came from environments where unease was earned by real consequence.
- The anchoring heuristic tracks reasonable estimates - a jacket marked down from 200 feels like a bargain. This came from environments where the first number came from someone informed, not someone trying to move you.
The heuristic is a good measure until the environment shifts, or until somebody starts gaming it. Adversaries exploit proxies that used to track something but became the target of optimisation - phishing, advertising, and propaganda are all deliberate Goodhart Law attacks on cognitive proxies.
Biases are heuristics that outlived their environment. The negativity bias was a good measure of survival risk on the savanna. But when it applied it to a social media feed engineered to maximise engagement and the proxy is now actively (adversarially) anti-correlated with the thing it was meant to track.
On Feedback
Experience can fix this or worsen bias, depending entirely on feedback and analyse quality.
Gary Klein’s naturalistic expertise - chess masters, firefighters, neonatal nurses - sharpens heuristics in environments with tight, honest feedback. Proxies get pulled back toward the target every time they drift - there is ground truth.
Phil Tetlock’s experts in low-feedback domains, geopolitics and stock-picking and political punditry, show the opposite: experience accumulates confidence without accuracy.
The same mechanism can facilitate wisdom or delusion depending on whether the environment punishes proxy drift - ‘twenty years of experience’ can mean twenty years of corrected error, or twenty years of unchallenged proxy use - they look identical from the outside.
Strangely, System 2 thinking doesn’t necessarily escape this either, it just relocates it. Explicit metrics - KPIs, GDP, test scores, citation counts - when analysed can become System 2’s heuristics. Entire job roles are based around creating, facilitating and analyzing metrics in a thoughtful system two way. However without a notion of security modelling and weighting for qualitative analysis, they are more Goodhart-vulnerable, not less. Legibility to the thinker is also legibility to the optimiser - making the proxy explicit is what makes it gameable. The institutional version of attribute substitution is the analytics dashboard. Such dashboards corrupt downstream judgment in exactly the way the brain’s silent substitutions corrupt individual decisions - you believe you have all the information you need to make a reasonable judgment but you lack the relevancy realization and security modelling of the metrics, and without the willingness to look in the qualitative you won’t have them until disaster or a threshold of critical feedback is no longer ignorable.
On Shorthand & Jargon
Firstly, language is metaphor, and when heuristics are collapsed into or related to a single term there is a risk that the heuristics acquired without their genealogy are already decoupled. Words like synergy, alignment, leverage, or domain specific terminology, emerged from specific situations as compressed labels for dynamics someone had actually observed. Unfortunately when people use these words too often without deeper consideration or reference, or they picked them up through cultural osmosis, the acquired secondhand travels as a label with the underlying learning of why they are important in the first place, forgotten. The recipient holds a token they can pattern-match on but can’t unpack. When the token shows up, they have no way to check whether the underlying thing is actually present. Substance and rhetoric become indistinguishable, because the discriminator was either never installed or forgotten about. It’s alright if you know a word and it’s sentiment, it’s great if you know it’s definition and when to use it with precision - know it’s semantic utility, it’s incredible when you have a lived experience associated to the semantics. Otherwise, the term is reduced to rhetoric.
This is the structural condition behind the recent research on bullshit-receptivity, where it was shown that people who score high on susceptibility to corporate-speak have a defect of derivation. They are running on imported caches of terms with no fallback. When asked to evaluate a claim, they can’t fall back to the underlying dynamics, because the underlying dynamics never accompanied the words. The remedies usually proposed for this - plain language norms, clarifying-question rituals, leadership ‘critical thinking checks’ - treat symptoms. The deeper issue is selection. Organisations that reward rhetorical fluency at promotion gates accumulate people whose competitive advantage is rhetorical fluency.
The cleanest way to reframe System 1 and System 2 once you’ve read all this is: it isn’t just fast versus slow, it’s cached versus generated versus experienced. A cached judgment or term is reliable only if you cached it yourself in a high-feedback environment, or if you can re-derive it on demand. The buzzword-fluent employee fails both tests - the cache was loaded by exposure, not experience, and there is no derivation procedure to fall back on. The expert firefighter passes both - feedback shaped the cache, and they can reconstruct, in slow time, why the cache is correct. The label “intuition” gets applied to both groups but they got to where they are by doing fundamentally different things. The latter is far better off leading, judging and arriving a consensus with the team around them.
The unifying claim under all of this is simple. We rarely act on the thing we want to act on. We often act according to representations of it. Goodhart’s Law is the observation that representations decouple from referents under optimisation pressure. The heuristics-and-biases program is the same observation about cognition. The mechanism is identical; only the optimiser changes - natural selection, deliberate gaming, or organisational incentive. Once the proxy is what’s being optimised, the signal becomes diffuse.
The practical question this leaves you with is uncomfortable. For any judgment you trust yourself to make: do you have feedback that would tell you when your proxy stopped tracking? If not, you should assume it has, and you haven’t noticed.
Appendix: Evaluation as the Goodhart Site
The argument has an obvious application to one place where these failures concentrate: performance evaluation.
An evaluator without domain competence has the same problem as the buzzword-fluent employee - they are running on imported caches. They pattern-match on what competent work looks like with no derivation procedure underneath. When the work in front of them looks competent, they can’t check whether it is. Their proxies - charisma, confidence, output legibility, deck quality - are all they have. So their proxies are what get optimised against.
This is the selection mechanism that makes theatre dominate. Specialists optimise for what their evaluators can grade. Evaluators who can only grade legibility get work optimised for legibility. The substance underneath isn’t penalised when it’s missing, because the evaluator can’t see it missing. Over a long enough timespan, the people who survive are the ones who learned to perform competence at the resolution the evaluator can detect. The people who actually do the work either leave or learn to package it the same way. The Goodhart attack runs continuously, without anyone deciding to attack. The fix isn’t more rigour from the evaluator’s existing toolkit - that’s just sharpening the proxy. The fix is making sure the evaluator has at least one domain where they have generated, not cached, precise terminology and judgment. Evaluate Your Evaluator makes the case in detail: a founder who has built something hard, can read the artifacts the people they evaluate produce, and has specialised long enough to know what mastery costs. Those three are the conditions for derivation. Without at least one of them, the evaluator is entirely a cache.
Footnotes
- This could be why people did is the The Custody Illusion.
- This is also precisely the problem with AI Code Agent evaluation, and whether or not AI can write good code. The software industry seriously needs to consider if we are even reading code anymore, or if we are just reading the legibility of it.