The Sweet Lesson - josh
josh
SubscribeSign in
The Sweet Lesson<br>What 70 years of machine learning research reveals about why you don't know what you want...and what to do about it
josh<br>Jul 06, 2026
Share
You don’t know what you want. Seventy years of AI research hints at why.<br>In 2019, Richard Sutton distilled those seventy years into a short essay called The Bitter Lesson. This would go on to be considered one of the most important thought-pieces in machine learning.<br>In it, he observes that again and again, researchers baked human knowledge into AI systems: clever features, heuristics, and theories of how vision, chess, or language must work. And again and again, these hand-crafted theories got beaten by dumber methods that just searched and learned at a large enough scale.<br>The lesson is bitter because it directly attacks our oldest pride: the idea that we know best.<br>I’ve come to believe this lesson generalizes to life. Your preferences, the career you’re chasing, the type you swipe for, the city you’re sure you’d love, are hand-crafted features too. And on top of that, they’re crafted from a pretty small sample: our parents, our hometown, a few books, movies, and, of course, ~algorithms~<br>Unfortunately, these preferences are usually wrong, but Sutton’s essay shows us how machines overcome this.<br>I hope this post can help you do the same.<br>I thought I knew what I wanted.
Do you think you know what you want in a partner? The research says that you probably don’t.<br>Eli Finkel and Paul Eastwick ran a series of speed-dating studies where they asked people what they wanted in a partner. They asked how much they valued attractiveness, earning potential, warmth, and then watched who they actually liked after four-minute conversations.
As I’m sure you can guess, their stated preferences predicted almost nothing. Men who swore looks mattered and women who swore status did were both wrong the moment they made contact with actual humans
I think this study is especially important to reflect on in our digital age, where human contact is becoming increasingly rare. Thanks to Tinder, Hinge, Bumble, and whatever else is out there, our chances at love are increasingly filtered through thin digital proxies: photos, prompts, height, job titles, vibes, and whatever else the algorithm decides to surface... And whether we swipe right is solely based on our human biases.<br>Could someone you swiped left on for a superficial criterion have built an amazing life with you? The evidence suggests: more often than we’d like to admit.<br>Now, you may be thinking that this is just speed-dating, but it seems to generalize! It seems that we’re consistently bad at knowing what will make us happy. Let’s take a look at another study:<br>Dan Gilbert and Tim Wilson spent a career documenting what they call the impact bias: we systematically mispredict how future events will make us feel. They asked assistant professors how they would feel if they were denied tenure. Their response? Devastated!<br>However, years later, the denied ones were about as happy as the tenured ones. They imagined one future would define them, but life kept generating other sources of meaning.<br>Take a chance.
So, if we don’t know what will make us happy, what should we do? Well, Sutton’s lesson is to search and learn. To stop over-prescribing our lives, to open ourselves to the myriad of possibilities, and to learn the unknown. Let’s look at one more study.<br>Steven Levitt, the Freakonomics economist, ran what may be the wildest life-advice study ever. He found tens of thousands of people currently undergoing a major life decision (quitting a job, ending a relationship, moving), and had a website flip a coin for them.<br>If it landed on heads, they would make the change, and if it landed on tails, they wouldn’t.<br>Luckily, enough people listened to the almighty coin that Levitt was able to measure outcomes. Six months later, the people who made the change reported being substantially happier, even on the irreversible decisions!<br>I know this because a version of Levitt’s coin rewrote my life. Six years out of college, I was in Chicago and all my closest friends had moved away. I wanted something new but had no idea what... So I made a bold choice: I googled the best city in America. The internet said Austin. I moved there knowing nobody, knowing nothing about the city, never having visited.<br>That choice produced nearly everything I now care about. A friend pulled me into outdoor climbing, which snowballed into a summit on Kilimanjaro. I found friends I’ll keep until I die. I found a dog who taught me why we call them “man’s best friend.” I met the love of my life, and now we share an address. I got a master’s at UT Austin because I passed by a billboard and decided to apply.
None of it was on my list. None of it could have been.<br>Maybe I just got lucky. But Levitt’s data suggests my luck wasn’t an outlier: when you’re genuinely on the fence, exploration usually wins.<br>The takeaway, from...