Goodhart's Law Isn't as Useful as You Might Think - Commoncog

-->

Close

0" class="border-t border-slate-100 px-4 pb-2 sm:px-7"><br>Enter at least 2 characters.

Searching…

= 2" class="divide-y divide-slate-100">

No results found.

This is part of the Operations topic cluster, which belongs to the Business Expertise Triad.

Business Thinking

Goodhart's Law Isn't as Useful as You Might Think

By Cedric Chin

Table of Contents

The thought of business school make you go ‘eww’?

You’re in good company.

9,000+ investors and operators read Commoncog to sharpen their business acumen ... WITHOUT going back to school.

Sign up for our newsletter and get a weekly dose of good business thinking (no BS guaranteed):

Subscribe

This is Part 1 of the Becoming Data Driven in Business series.<br>Goodhart’s Law is a famous adage that goes “when a measure becomes a target, it ceases to be a good measure.” If you’re not familiar with the adage, you can go read all about its history on Wikipedia, and perhaps also read the related entry on the ‘cobra effect’ (which includes a litany of entertaining perverse incentive stories, of which the eponymous cobra anecdote is merely one):<br>The British government, concerned about the number of venomous cobras in Delhi, offered a bounty for every dead cobra. Initially, this was a successful strategy; large numbers of snakes were killed for the reward. Eventually, however, enterprising people began to breed cobras for the income. When the government became aware of this, the reward program was scrapped. When cobra breeders set their now-worthless snakes free, the wild cobra population further increased.<br>But I’m here to tell you that Goodhart’s Law is not as useful as you might think.<br>At some level, this is self-evident. Goodhart’s Law is about as pithy and about as practicable as “the only certainty in life is death and taxes” and “hell is other people.” It is descriptive; it tells you of the existence of a phenomenon, but it doesn’t tell you what to do about it or how to solve it.<br>Thankfully, it turns out that there has been a fair amount of work on solving for Goodhart’s Law at the organisational level. Note that there is a variant of Goodhart’s Law that concerns broad social policy; the ideas here probably won’t work for that. But if you’re an operator, like I am, and you’re interested in solutions at the company level, this is going to be right up your alley.<br>A brief note so you know the source of these principles: many of these ideas were worked out by W. Edwards Deming and his colleagues in the 50s through to the 80s, as part of a body of work known today as ‘Statistical Process Control’ or, more broadly, ‘Continuous Improvement’. I’ve talked a little about how I fell into this rabbit hole in the recent past; the short version is that I did some work for Colin Bryar to explicate Amazon’s Weekly Business Review process for his company’s clients, and during that project I discovered that many of the ideas in the WBR were actually taken from the field of Statistical Process Control. As a result, I started digging into SPC to see what other principles or ideas might be applicable to business.<br>One of the more interesting things about the WBR is that the folks at Amazon have developed a number of ways to solve for Goodhart’s Law. We’ll use the set of practices around the WBR as an example in a bit. But the main idea that I want to highlight here is that the WBR’s practices came from a body of work; that body of work offers us a bunch of principles to use in our own contexts.<br>I’ll start with the principles, and then articulate one instantiation of those principles with the Amazon WBR as an example.<br>A More Solvable Version of Goodhart’s Law<br>To get at the principles, it’s useful to talk about formulations of Goodhart’s Law that are more useful than the original form.<br>There’s a fairly interesting paper by David Manheim and Scott Garrabrant titled Categorising Variants of Goodhart’s Law that lays out four categories of the phenomenon. I summarised the paper a number of years back, in which I talked about some of their proposed solutions for each of the categories. I do recommend the paper if you’d like a more general take on various forms of Goodhart’s Law — which is useful if you’re into, say, AI alignment research. But I did not think highly of the solutions — they seemed too academic, too theoretical, for my taste.<br>Thankfully real world organisational solutions are much simpler. The first step is to turn Goodhart’s Law as a narrower, more actionable formulation. The one that I like the most is from Deming contemporary Donald Wheeler, who writes, in Understanding Variation:<br>When people are pressured to meet a target value there are three ways they can proceed:

1) They can work to improve the system<br>2) They can distort the system<br>3) Or they can distort the data<br>Let’s demonstrate this by example. Say that you’re working in a widget factory, and management has decided you’re supposed to...