Goodhart’s law says that when a measure becomes a target, it ceases to be a good measure. Put a bonus on a number and people will move the number. Sometimes they move it by doing the work better. When that’s hard, they move it by changing what gets counted, and the number keeps rising while the thing it was meant to track stands still or gets worse.
The Goodhart’s law examples below are not obscure. A bank fired more than 5,300 employees for gaming its sales goals. Ambulance services rewrote response times to hit eight minutes. Hospital schedulers kept veterans off the official waiting list so that the waits looked short. In every case the people at the top had been warned about what was happening, often years earlier.
That’s the part that interests me. Managers know Goodhart’s law; plenty could recite it. They attach bonuses to single numbers anyway, because a number is cheap and judgment is expensive.
Where does Goodhart’s law come from?
Charles Goodhart was a monetary adviser at the Bank of England when he presented a paper on British monetary policy at a Reserve Bank of Australia conference in Sydney in 1975. His point was about money supply targets: a statistical relationship that holds while nobody leans on it tends to collapse once policymakers use it for control (Goodhart, reprinted in 1984).
The version everyone quotes came later. In 1997 the anthropologist Marilyn Strathern, writing about audits in British universities, put it this way: “When a measure becomes a target, it ceases to be a good measure.” Her example was the degree grade: the more a good grade is expected, the less it tells you about the student. She credited the label “Goodhart’s law” to the educationalist Keith Hoskin.
The American social scientist Donald T. Campbell had made the same point about social programs. In a 1976 paper, he argued that the more a quantitative indicator is used for social decisions, the more it attracts “corruption pressures” and the more apt it becomes “to distort and corrupt the social processes it is intended to monitor” (Campbell, 1976). That is Campbell’s law, and I find it the more useful of the two, because it names both kinds of damage: the number gets corrupted, and so does the work.
His examples could have been written last week. Police judged by the share of crimes they solved let burglars confess to extra burglaries, often ones they hadn’t committed, in return for lighter sentences. Contractors paid by pupils’ test-score gains taught the answers to the test items. An employment office that rated staff by people placed found them concentrating on the easiest cases.
I’ve covered the social media version in Participation is not engagement. Here the reward is real money.
Wells Fargo: eight rhymes with great
For years Wells Fargo told investors that its edge was cross-selling, and it published a “cross-sell metric” in its annual reports to prove it. The goal was eight products per household. In his 2010 letter to shareholders, the chief executive, John Stumpf, explained why eight: it rhymed with “great.”
In the branches, the strategy turned into sales goals for employees, and the goals came with pressure. The bank had its own word for what followed. According to the statement of facts Wells Fargo accepted as true in 2020, “gaming” was a term generally known at the company, meaning manipulating or misrepresenting sales to meet goals, earn incentive pay or avoid being reprimanded or fired. Employees used customers’ identities without consent to open accounts and cards nobody had asked for.
From 2002 to 2016 the bank opened millions of unauthorized or fraudulent accounts and products, and between 2011 and 2016 it fired more than 5,300 employees for sales ethics violations. In 2016 the Consumer Financial Protection Bureau fined it $100 million over roughly 1.5 million deposit accounts and 565,000 credit card applications that may not have been authorized. In 2020 the bank agreed to pay $3 billion to settle with the Justice Department and the Securities and Exchange Commission.
What makes it a textbook case is what the managers knew. The statement of facts says that as early as 2002, when a group of employees was fired from a branch in Fort Collins, Colorado, for sales gaming, the community bank’s senior leaders learned that the misconduct was happening, that it was growing and that the sales goals and pressure were its cause. The goals stayed for another 14 years. The bank dropped product sales goals for branch employees only in 2016, the year the scandal became public.
Hospital waiting-time targets: 24 days on paper, 115 in real life
In 2014 the inspector general of the Department of Veterans Affairs examined the VA health care system in Phoenix. For a sample of 226 new patients, official data showed an average wait of 24 days for a first primary care appointment; the inspectors found 115 days. Another 1,700 veterans waiting for appointments weren’t on the official electronic waiting list at all, so officially their wait hadn’t started.
The report tied the numbers to pay: the understated waits appeared in the Phoenix leadership’s performance appraisals, one of the factors considered for awards and salary increases. Schedulers elsewhere described being told to “fix” appointments that exceeded the 14-day goal. Nobody could claim surprise, either. The report reprinted an April 2010 memo from a senior VA health official warning that some facilities were using “gaming strategies” to improve their access scores.
England ran a similar experiment with ambulances. From 2001, services were expected to reach 75% of immediately life-threatening calls within eight minutes, and in England the result fed a public “star rating.” When the Commission for Health Improvement looked at how response times were distributed, around a third of trusts showed sharp jumps at the eight-minute mark: calls that took slightly longer had been “corrected” to eight minutes or less. In 2007 the Audit Commission found that staff at one service had manually changed the times of nearly 600 of the most urgent calls, noting that data which is easy to manipulate gives people “both the motive and opportunity” to do so.
Now the fair hearing for targets. The same review estimated that the doctored times flattered performance by at most 6%, while reported performance had improved by up to 20% since 1999. English ambulances really did get faster, and only England, where missing the target was public, met it. A 2009 study of the four-hour target in English emergency departments by Steven Kelman and John Friedman likewise found large improvements in waiting times and no evidence of the dysfunctions critics had predicted.
So targets work, and that’s exactly why they’re dangerous. A target can improve the work and corrupt the measurement at the same time, and from the dashboard you can’t tell how much of each you’re getting.
Police statistics, test scores and clicks
Campbell called crime rates “very corruptible indicators,” and Britain supplied a modern example. In January 2014 the UK Statistics Authority stripped police-recorded crime figures of their National Statistics status over doubts about their reliability. That April a committee of MPs concluded that “the subordination of data integrity to target-chasing” had become ingrained, and said police and crime commissioners should be told not to set performance targets based on police data. In November the inspectorate of constabulary estimated that 19% of crimes reported to the police, more than 800,000 a year, were not being recorded.
Schools did it with test scores. In Atlanta, according to the 2013 indictment, a school that met 70% of its annual targets earned a bonus for every employee. A state investigation in 2011 had found cheating on standardized tests in at least 44 schools, with nearly 180 educators accused, and in April 2015, 11 former educators were convicted of racketeering. The superintendent, Beverly Hall, who had long denied knowing about the cheating, died before she could be tried.
Online, pay for clicks and you get headlines that hold back the story until you click. In August 2014 Facebook said it would demote “click-baiting” and use the time people spent reading after the click as a signal. Market research has its own version: pay per completed questionnaire and you get people who are very good at completing questionnaires, for about $3 an hour.
The cobra effect, the nail factory and the rats of Hanoi
Talks about Goodhart’s law tend to include two stories. In one, a Soviet nail factory with a quota by weight makes a few huge nails, and with a quota by number makes millions of tiny ones. In the other, a bounty on cobras in colonial Delhi leads people to breed cobras, and when it is cancelled they release them. The economist Horst Siebert named the “cobra effect” after it in a 2001 book.
I couldn’t find a primary source for either story, so I treat both as parables, not history.
There is a documented version, though. In 1902 the French authorities in Hanoi, alarmed that the city’s rats could carry plague, paid a few cents for every rat tail delivered to the police. The historian Michael Vann, who found the files in the French colonial archives in Aix-en-Provence, describes officials discovering that people were cutting off the tails and letting the rats go to breed, and that some were farming rats. History didn’t need the cobras.
Why do managers keep doing it?
None of this is news to management. In 1975 Steven Kerr published “On the folly of rewarding A, while hoping for B” in the Academy of Management Journal (Kerr, 1975), a catalog of organizations that pay for one behavior while hoping for another. In the early 1980s W. Edwards Deming put it into his 14 points for managers: “Eliminate management by numbers, numerical goals.”
The managers’ side deserves a hearing. In 1994 Richard Boettger and Charles Greer argued that organizations often reward A while hoping for B for good reasons, because they must serve contradictory goals at once. And a manager who replaces targets with “judgment” may simply be replacing a number with his own favorites.
Still, I think the usual reasons are less sophisticated. A number is cheap: it scales to thousands of employees, needs no conversation and is easy to defend in front of a board. Judgment is slow and arguable, and it makes the manager responsible for the verdict. A bonus on a number hands that responsibility to a spreadsheet, and when the number turns out to be gamed, the people who gamed it are the first to go. It’s the incentive version of Submariners in space: leaders announce a new direction and leave the rewards where they were.
How do you avoid Goodhart’s law?
You can’t repeal it, but you can make it expensive to exploit. Here’s what I’d do:
- Use several measures that pull against each other. Robert Kaplan and David Norton’s balanced scorecard paired financial measures with customer, process and learning measures in 1992 for this reason. Accounts opened belong next to accounts used; response times next to patient outcomes. But a scorecard with a bonus on every line is just four targets.
- Check the number against something the measured can’t touch. England and Wales count crime twice: in police records, and in a face-to-face survey of about 31,000 people that includes crimes never reported to the police.
- Audit the numbers, and look at their shape. The ambulance corrections showed up as a spike at eight minutes. The MPs who examined police data called the end of regular external audits in 2007 a mistake and asked for annual ones.
- Rotate what you reward. A target people can study for a decade is one they will learn to beat.
- Keep measurement and pay apart where you can. Use the numbers to start a conversation in a review, not to settle a bonus automatically. Wells Fargo’s own fix was to scrap product sales goals altogether.
- Leave room for judgment. Jerry Muller argues in The Tyranny of Metrics (2018) that metrics work best as a complement to judgment based on experience. My test: if no manager will defend a rating face to face, it shouldn’t carry money.
My take
Goodhart’s law isn’t an argument against measuring. It’s an argument against pretending that a measure stays the same thing once you pay people to move it. The day a number gets a bonus attached, it stops being an instrument and becomes a negotiation, and the people closest to the number always negotiate better than the people who set it.
What bothers me isn’t that employees game targets. The bankers, schedulers and ambulance controllers did what the system paid them to do. What bothers me is that the people who built those systems had been warned, in memos, audits and dismissals, and kept them running because the numbers looked good.
So my rule is simple. Measure lots of things, pay for few of them, check the ones you pay for against something nobody can edit, and assume that a number which has risen smoothly for three years is telling you about the people who report it before it tells you anything about the work.
Questions people ask
What is Goodhart’s law in simple terms?
Once people are rewarded for hitting a number, the number stops being a reliable measure, because they can often move it more easily than they can improve what it was meant to track. The popular wording, ‘When a measure becomes a target, it ceases to be a good measure,’ comes from the anthropologist Marilyn Strathern in 1997.
What is a real example of Goodhart’s law?
Wells Fargo set a goal of eight products per household and pushed its branch employees with aggressive sales goals. Between 2002 and 2016 staff opened millions of unauthorized or fraudulent accounts, and in 2020 the bank agreed to pay $3 billion to settle criminal and civil investigations.
What is the difference between Goodhart’s law and Campbell’s law?
Goodhart’s law (1975) comes from monetary policy: a statistical relationship tends to break down once it is used for control. Campbell’s law (1976) adds that a social indicator used for decisions gets corrupted and also corrupts the work it measures, as when teachers teach to the test.
Is the cobra effect a true story?
I couldn’t find a primary source for the Delhi cobra bounty, so I treat it as a parable. A similar episode is documented: in 1902 the French authorities in Hanoi paid a bounty for rat tails, and people cut off the tails and let the rats go to breed.