The Optimal Amount of Slop is Non-zero - Doug Slater

Table of Contents

Looks can be deceiving

Matching rigor to risk

Too much rigor

Too little rigor

The right amount of rigor

Slop I have shipped

Objections

Vibe coding doesn't imply slop

LLMs can review the code, so I don't have to

What to do now

References

Regretting that code you vibed? Learn when skipping human review is and isn't a smart move.

Rigor should be proportional to risk.

My regular readers might be shocked at the title of this post. If you've read my other posts, such as AI: Accelerated Incompetence or LLMs are not Bicycles for the Mind, you might expect that I would more readily miss my son's birthday than ship unreviewed LLM code. You would not be far from wrong: there are just a few narrow situations where I have. Today you'll learn about those and along the way my decision criteria for skipping code review.

note

Definitions

agentic coding: An LLM edits, runs, and tests code for you in a loop

vibe coding: Accepting LLM-generated code without reading it

slop: Low-quality, high-quantity AI-generated content

Looks can be deceiving

Month by month, I encounter more people who have discovered agentic coding and have come to trust it so much that they are now unbothered to outsource not just software implementation but also verification to it. Just yesterday I chatted with a dev who says he's stopped reviewing code. He lets a team of LLM agents do it for him. I felt disappointed because he should understand this vexatious property of software, that externally observable appearance and behavior gives very little signal about internal quality. A program that does everything expected of it can still be riddled with quality issues. It works today but will break when revised and the world around it changes.

As a daily user of Claude Code, I can attest that when given clear requirements and context, it regularly generates software that actually does what I asked. However, across hundreds of sessions, the code has not once been what I would call good, even after adversarial LLM review.

Closed-source software is both an experience good and credence good1. We've all bought some downloadable software or subscribed to a SaaS. Before you bought, you evaluated whether the software works well for your needs, but there was no way for you, a prospective customer, to evaluate the quality of its implementation. You can only evaluate on externals. If there's a security flaw, you can't discover it. Certifications like SOC 2 exist to rebalance this information asymmetry between the developer and the customer.

If you, a developer, outsource reading the code to an LLM, then you discard your information advantage and bring no more value over a nonpractitioner.

Here's how we know software is a credence good: give an exec a slick-looking prototype, and they'll be ready to write a check for millions. Really all you've done is given them a poster for a movie that doesn't even have a premier date yet. This is why good prototypes deliberately look unfinished.

Prodent mockups and prototypes look like pencil sketches on purpose because no exec says "ship it now."

Programmers have a capability the general population doesn't: to review the code their LLM generates. That's a valuable advantage, but internal code quality is paid for in the scarce currencies of time and attention. When is the effort worthwhile?

What we're looking for is the right risk-rigor ratio.

Matching rigor to risk

In any situation, when deciding how much rigor to exercise, we have to consider the possible costs of things going wrong. If they're low enough, we don't bother to exercise the rigor, but if they are high enough, we should. Let me tell you two stories that demonstrate getting this wrong.

Too much rigor

Imagine a dystopian future where hamburgers are extremely valuable. Crime rings regularly steal, launder, and resell them. When you walk into McDonald's, you pass through a metal detector and are subjected to a brisk frisking. When you order a hamburger, the cashier sternly asks to see your government-issued photo ID. In this dire world, such extreme measures are necessary to maximize McDonald's profits by protecting from loss. In the real world, this story is a laughable fiction because the rigor far outweighs the risk. Burgers would cost ten times more, and McDonald's wouldn't sell very many.

At a sufficient level of risk, such drastic security measures land fully inside the Overton window: they are routine at every commercial airport in the world.

Too little rigor

Let's flip over to a story that demonstrates the exercise of too little rigor. The movie The Invention of Lying (2009)2 takes place in a world where nobody has ever told a lie. The main character Mark Bellison, played by Ricky Gervais, is down on his luck: he's about to be evicted because he can't afford his rent. Defeated and expecting to become homeless, he saunters in to his local bank branch and asks...