Our product now improves itself: here is how

SubscribeSign in

Our product now improves itself: here is how<br>Everyone building self-improving products obsesses over the coding agent. The layer that actually decides whether your product gets better is the one nobody talks about.

Dan Leshem<br>Aug 11, 2026

Share

Last week one of our engineers at Stencil, came to me with a gnarly bug: builders were losing conversation context mid-project. The agent would just... forget. Real frustration, the churn-inducing kind. He congratulated me for catching it.<br>I didn’t catch it. Discovery caught it. An agent that reads our chat history fished the pattern out of roughly 600 messages, matched it across eight different users , and filed the ticket with the evidence attached.<br>Thanks for reading Night Shift! Subscribe for free to receive new posts and support my work.

Subscribe

None of those eight users reported it. It never hit Slack. Nobody complained. They just quietly had a worse day, eight separate times, and the only thing in the building that noticed was a cron job.<br>That’s the moment this piece is about.<br>“What’s next” was always the game

Stencil is a vibe-coding platform — domain experts describe a micro SaaS, and an agent builds it. So I think about self-improving products for a living, and here’s my claim: the discovery layer is the most critical layer of a self-improving product. Not the coding agent. Not the merge gate. Discovery.<br>The reasoning is almost embarrassingly simple. Implementation is solved — I’ve written before about loops that ship code and the merge gate that reviews it . Point a coding loop at a well-specified problem and it grinds it into dust. But who points it?<br>The hardest part of getting any product to maturity was never writing the code. It’s deciding what deserves to be built next. And the raw material for that decision is all there, piling up: analytics, session recordings, error tracking, support tickets, chat history, feedback widgets, competitor moves. The problem isn’t that you’re working on the wrong things. The problem is capacity. Processing all of that signal is a full-time job for a team you don’t have, so almost all of it goes unprocessed — and your backlog fills with whatever happened to be loudest, instead of high-quality work items backed by evidence.<br>Your agents aren’t limited by what they can build. They’re limited by what you can feed them.

That’s the gap nobody warns you about: coding agents multiplied your build capacity overnight, and your signal-processing capacity stayed human-sized . The memory-loss bug sits in message 412 of the chat history — not ignored, just never read.<br>You can hold six threads. Maybe.

The deeper reason discovery is the critical layer isn’t about products at all. It’s about your head.<br>It’s not that I can’t investigate a bug — I do, daily, with Claude. But a chat with Claude is synchronous . I’m holding the thread. And a human holds three of those at once, maybe six on a great day. That’s the real ceiling on how fast a small team improves a product: not typing speed, not agent quality — the number of open threads a brain can carry.<br>Discovery inverts the ownership. The loop asks the questions. The loop holds the memory. The loop connects yesterday’s weak signal to today’s. My involvement compresses to reading a finished report — and when I’ve reviewed it, I’m *done*. Feedback goes back into the loop, the loop carries it, and it leaves my head entirely.<br>I call this file and forget , and I’ve come to believe it’s the actual mechanism behind agentic leverage.<br>True parallelism isn’t running ten agents at once. It’s being able to close a thread the moment you’ve applied judgment, knowing something with perfect memory is holding it.

Continuity lives in the loop, not in your cognitive load. That’s what lets three people operate like thirty.<br>The machine

What actually runs at Stencil today: the agents, the bar their output has to clear, and the loop they run around the clock.<br>Meet the Rangers

At Stencil we call our discovery agents Rangers — rangers protect the park. Always on guard, gathering intel, proactive. Special forces for your product. (PostHog calls the same pattern scouts, and their write-up is excellent; Credit where due — this pattern is being convergently evolved all over right now.)<br>A Ranger is a scheduled agent with one signal source and a skill — a markdown file that tells it what matters in that source, what’s noise, and how hard to squint. Here’s the roster running today, every one of them live:<br>Session replay — where users rage-click and stall

AI chat history — what people ask the agent for, and what frustrates them (this one caught the memory bug)

Error tracking — exceptions, bursts, regressions

Support tickets — the highest-value words a user ever writes

Meeting notes — decisions and complaints that never make it to a tracker

Feedback widget — the in-app suggestion box

Product changelog — recent PRs, so Rangers know what just...