Chapter 1: Inadequacy and Modesty | Inadequate Equilibria

Inadequate Equilibria<br>-->

Chapter 1

Inadequacy and Modesty

This is a book about two incompatible views on the age-old question: “When should I think that I may be able to do something unusually well?”

These two viewpoints tend to give wildly different, nearly cognitively nonoverlapping analyses of questions like:

My doctor says I need to eat less and exercise, but a lot of educated-sounding economics bloggers are talking about this thing called the “Shangri-La Diet.” They’re saying that in order to lose weight, all you need to do is consume large quantities of flavorless, high-calorie foods at particular times of day; and they claim some amazing results with this diet. Could they really know better than my doctor? Would I be able to tell if they did?

My day job is in artificial intelligence and decision theory. And I recall the dark days before 2015, when there was plenty of effort and attention going into advancing the state of the art in AI capabilities, but almost none going into AI alignment: better understanding AI designs and goals that can safely scale with capabilities. Though interest in the alignment problem has since increased quite a bit, it still makes sense to ask whether at the time I should have inferred from the lack of academic activity that there was no productive work to be done here; since if there were reachable fruits, wouldn’t academics be taking them?

Should I try my hand at becoming an entrepreneur? Whether or not it should be difficult to spot promising ideas in a scientific field, it certainly can’t be easy to think up a profitable idea for a new startup. Will I be able to find any good ideas that aren’t already taken?

The effective altruism community is a network of philanthropists and researchers that try to find the very best ways to benefit others per dollar, in full generality. Where should effective altruism organizations like GiveWell expect to find low-hanging fruit—neglected interventions ripe with potential? Where should they look to find things that our civilization isn’t already doing about as well as can be done?

When I think about problems like these, I use what feels to me like a natural generalization of the economic idea of efficient markets. The goal is to predict what kinds of efficiency we should expect to exist in realms beyond the marketplace, and what we can deduce from simple observations. For lack of a better term, I will call this kind of thinking inadequacy analysis.

Toward the end of this book, I’ll try to refute an alternative viewpoint that is increasingly popular among some of my friends, one that I think is ill-founded. This viewpoint is the one I’ve previously termed “modesty,” and the message of modesty tends to be: “You can’t expect to be able to do X that isn’t usually done, since you could just be deluding yourself into thinking you’re better than other people.”

I’ll open with a cherry-picked example that I think helps highlight the difference between these two viewpoints.

i.

I once wrote a report, “Intelligence Explosion Microeconomics,” that called for an estimate of the economic growth rate in a fully developed country—that is, a country that is no longer able to improve productivity just by importing well-tested innovations. A footnote of the paper remarked that even though Japan was the country with the most advanced technology—e.g., their cellphones and virtual reality technology were five years ahead of the rest of the world’s—I wasn’t going to use Japan as my estimator for developed economic growth, because, as I saw it, Japan’s monetary policy was utterly deranged.

Roughly, Japan’s central bank wasn’t creating enough money. I won’t go into details here.

A friend of mine, and one of the most careful thinkers I know—let’s call him “John”—made a comment on my draft to this effect:

How do you claim to know this? I can think of plenty of other reasons why Japan could be in a slump: the country’s shrinking and aging population, its low female workplace participation, its high levels of product market regulation, etc. It looks like you’re venturing outside of your area of expertise to no good end.

“How do you claim to know this?” is a very reasonable question here. As John later elaborated, macroeconomics is an area where data sets tend to be thin and predictive performance tends to be poor. And John had previously observed me making contrarian claims where I’d turned out to be badly wrong, like endorsing Gary Taubes’ theories about the causes of the obesity epidemic. More recently, John won money off of me by betting that AI performance on certain metrics would improve faster than I expected; John has a good track record when it comes to spotting my mistakes.

It’s also easy to imagine reasons an observer might have been skeptical. I wasn’t making up my critique of Japan myself; I was reading other economists and deciding that I trusted the ones who were saying...