Chat, we've made a terrible mistake... | sgnt.ai

I’ve spent the last few days realizing a genius idea I had to save some money with an aggressive Go-service-based caching layer over a service I need to pay for. Shower thought to detailed planning with an agent to large but fast build-out with many layers of code-review and hardening. It is well-engineered, edge-cases have been discovered and fixed and regression-tested by myself and agent-based reviews. #winning.

But during some exploratory testing of it, I suddenly had the all-too-familiar chill of realizing there was a fundamental design problem in the whole thing. Subtle but deadly thing I hadn’t considered about the service I was caching.

As I started to type it into Claude, I got as far as “We’ve made a terrible mistake” before accidentally hitting enter. No worries! Claude immediately identified the exact terrible mistake before I had to tell it what it was, despite our having gone through many, many rounds of declaring the codebase essentially perfect, with additional sign-off from Codex and Grok. (We were caching IP information at a prefix level, which was mostly fine, but for several categories I cared about, it was far too coarse. Yes, we can do “hosting IP” at the prefix level, but “VPN exit point” is often per-IP.)

What?

Claude, my friend, if this was such an obvious and fundamental mistake, how could we never have identified it during the many rounds of design planning we did? Why did neither of the other LLMs — who we pulled in many times for their opinion during the design phase — pick this up?

Claude’s opinion, with which I think I agree, is we started with a clever solution, and so never really needed to take a step back and look at the problem more widely. We were too intent on building, and our implementation was great, other than the fundamental design flaw. The models were all in “build this well” mode, and not “are we solving the right problem” mode. The second we switched to “how have we fucked up”, Claude had no trouble in figuring it out.

Would a human have caught this? Well, I did, eventually, so I guess the better question is “would another human have caught this before the tokens were expended”? One saving grace here of not involving other humans is that not too much time was wasted: a service that would have taken at least a week to get built was built (well, other than the above) in a work day.

I am now going back to all codebases I’ve worked on recently, claiming a “terrible mistake”, and seeing if Claude can find one…

Share

Read Next<br>LLM spambots liked my Show HN post more than real people did<br>July 21, 2026<br>I posted my AI agent jail to Show HN. Nobody upvoted it, but within fourteen hours thirteen LLM-written sales pitches found the two email addresses on the page.

Read Post

Get these articles sent to you<br>If you liked it, you might like other stuff I write

Email Address

Subscribe