Building Pi With Pi | Armin Ronacher's Thoughts and Writings

Armin Ronacher's Thoughts and Writings

blog<br>archive<br>projects<br>travel<br>talks<br>about

Building Pi With Pi

written on May 24, 2026

Pi is now part of Earendil, but in the important sense it is<br>still Mario’s project. He has been living with its<br>issue tracker longer than I have, and he has been exposed to the weirdness of<br>the new form of agent traffic in Open Source projects for longer too. This post<br>is mostly a reflection of my own experience after spending more time in the<br>tracker, using Pi to work on Pi, and watching what I have learned about it so<br>far.

Slop Issues

Unsurprisingly, we are using Pi to build Pi. That sounds like a cute dogfooding<br>thing but it really helps understand what we do. An interesting effect of<br>building with agents is that it changes the role of the issue tracker a tiny<br>bit. The issue descriptions are not just messages from a user to a maintainer<br>because we also use them as inputs for prompts in Pi sessions. It is something<br>I might hand to my clanker1 and say: "understand this, reproduce it, inspect<br>the code, and propose a fix."

That means the shape of the issue matters in a new way. A bad issue was always<br>annoying, but at least a lot of issues were vague. Now we are also dealing with<br>a class of issues that are 5% human and 95% clanker-generated and largely<br>inaccurate shit. A bad issue that contains a plausible but wrong diagnosis<br>creates extra work.

The most frustrating failure mode right now is that people submit issues that<br>are not in their own voice. They contain an observed problem somewhere, but it<br>has been thrown into a clanker and the clanker reworded it and made a huge mess<br>of it. Typically, it was prompted so badly that the conclusions produced are<br>more often than not inaccurate but always full of confidence. The result is<br>complete guesswork on root causes, fake-minimal repros, suggested implementation<br>strategies, analogies to adjacent but often the wrong code, and long lists of<br>error classes that might or might not matter.

That is worse than no diagnosis.

I don’t want to point to specific issues because I really do not want to bad<br>mouth anyone, but it is frustrating. It is also frustrating because when I give<br>that issue to Pi, Pi sees the wrong diagnosis too. It does not treat the issue<br>body as a rumor. It treats it as evidence. It will happily go down the path<br>that the issue already prepared for it, because the prose is confident and the<br>code references look plausible. We use a custom slash command called /is,<br>which specifically has this instruction in it:

Do not trust analysis written in the issue. Independently verify behavior and<br>derive your own analysis from the code and execution path.

Unfortunately, it does not fully work, because when humans first throw their<br>issue through the clanker wringer, their clanker expands scope almost<br>immediately. What was once a very narrow and fact based bug observation, turns<br>into a much expanded surface area full of hypotheses. So at least personally, I<br>increasingly want issue reports to be condensed to what the human actually<br>observed:

I ran this command.

I expected this to happen.

This happened instead.

Here is the exact error or log.

That is enough. If you used an LLM to understand the problem, great, maybe<br>leave it as a follow-up comment. But the issue and the issue text should be<br>something you own. If you do not know the root cause, say that. I too can<br>operate a clanker, and I would rather do this myself than use your slop. If<br>your repro is a guess, say that. If the only hard fact is one stack trace, give<br>me the stack trace and stop there.

Slop Begets Slop

That we’re seeing issues full of slop is just a result of the present day<br>quality of these machines. Sadly, their failures in creating good issues<br>extend to a lot of code that is generated. Not all of it, but a lot of code.<br>Over and over I keep running into them over-engineering the hell out of issues<br>and implementations.

If you tell them that "this malformed session log crashes the reader," the<br>clanker<br>will often add a tolerant reader. Then it will add a fallback, then maybe a<br>migration, then more debug output, then a test for all of this. None of this is<br>necessarily wrong in isolation, but it can be the wrong move for the system.

At Pi’s core is a rather well-designed session log with invariants that must be<br>upheld. The clanker’s present-day behavior is to just assume that no such<br>invariants exist, and instead to make the system work with all kinds of<br>malformedness, blowing up the complexity in the process.

Almost always, the correct fix is not to handle the bad state, but to make the<br>bad state impossible. This matters a lot for persisted data such as Pi session<br>logs. They are opened, branched, compacted, exported, shared, and analyzed.<br>The goal here is to never write bad session data. Yet if you just let the<br>clanker roam freely, it will attempt to handle every case of bad data in the<br>session log...