How I use Agents to Extract Hidden Feedback from Github and Improved Rector | Tomas Votruba
Home
Tools
PHPStan Rules Beyond Core
Reach me:
tomas.vot@gmail.com
How I use Agents to Extract Hidden Feedback from Github and Improved Rector
2026-07-14
I was randomly drifting through Github trending PHP repositories, and checking how popular projects use rector.php. A lot of them use it extensively, with prepared sets and even levels.
What I didn't like to see is the skip section. It's like a baseline in PHPStan, but I take it really seriously. In 90 % of cases, it means something is broken in Rector and the user had to skip it to avoid unwanted changes over and over.
Feeling the frustration, I thought: Is there something we can do about it?
"Skip" over Bug Report
So how did it start? 3 weeks ago, I opened the rector.php of the mautic/mautic project (versioned here) and quickly scanned the skip section. I noticed the rules are applied = wanted, but skipped for a file here and there.
I asked Honza, Mautic project merge lead and a friend from Prague PHP meetups I organized in pre-covid times, if he would be interested in a couple of tiny fixes here and there. He said "yes", so I gave it a go.
Before making any changes in the Mautic project itself, I thought, let's figure out if AI agents can figure out why the files are skipped. If it looks like a bug, propose a PR, make it pass and merge it.
My first prompt sounded something like this:
Hey, can you check this https://github.com/mautic/mautic/blob/135aabc643f0ac65e7fa105ea70665c063608cf4/rector.php<br>ignore section of Rector, and check particular rule + files combinations?<br>Figure out, if that's a bug or valid ignore, and if a bug, propose a PR in<br>a particular repository and try to fix it.
Once done, open it in my Chrome browser.
Agent work has started...
What is amazing about open-source is that every file is available to anyone. I don't have to copy file contents and write them into the terminal (using Claude Code currently) so the agent has the very important context.<br>I can just tell him (yes, agent Smith) to fix the file and thanks to the FQN path, he can access it. E.g. this part from withSkip().
// handle later, case by case as lot of chnaged code<br>RemoveAlwaysTrueIfConditionRector::class => [<br>// watch out on this one - the variables are set magically via $$name<br>// @see app/bundles/FormBundle/Form/Type/FieldType.php:99<br>__DIR__.'/app/bundles/FormBundle/Form/Type/FieldType.php',<br>],
There is:
FQN Rector class rule - so the agent knows where to fix it
a link to the file
and we're lucky - there is also a comment someone wrote after debugging the issue!
That's all the agent (or me) needs to figure out a proper fix.
To be honest, the first prompt took a while to figure out. The agent would have to learn about Rector, about rector.php config, most likely explore the documentation to figure out what all the available options there are, then figure out that I talk about the ->withSkip() and nothing else.
I was getting impatient, so I wrote a bit more narrow prompt:
The rule RemoveAlwaysTrueIfConditionRector breaks https://github.com/mautic/mautic/blob/7.x/app/bundles/FormBundle/Form/Type/FieldType.php<br>file
Figure out why, make test fixture, fix via PR, open new Chrome tab once CI passes
The agent understood this prompt much more. It took ~20 % of the original time to get the success, and I could also just copy-paste a template:
The rule breaks file<br>Figure out why, make test fixture, fix via PR, open new Chrome tab once CI passes
Tip: if a too wide prompt takes too long, try to narrow it down to its essence. What, where, how, when. I like to first shot and merge, because it works best for me. Rector's CI setup is very strong, fast and is only getting better. That's why it fits nicely.
Tip 2: I use the caveman extension not only to save tokens (never had a problem), but to save token input to my own eyes.
It costs much less attention to read: "bug found, making PR",
than: "I've found the bug in the file you've provided. I'm trying to figure out what rule is doing wrong in there. Downloading the file locally into and exploring ways to fix it."
If you can, use Caveman to extend your attention span by saving tokens throughput to your brain.
Discovering new Feature missing in Rector
It took roughly rector.php and extract a good-enough bug report from 2 lines.
Suddenly, what took a human reporter at least 20-30 minutes to figure out (sometimes more) - the rule, the minimal reproducer, and create a Rector demo link where we could actually see the error. Then it took us a couple of days (or weeks) to process, 10-20 minutes to look at and make a fix,<br>took 2 minutes . Full circle.
This opened a door: what else can we extract from this config?
Here is an interesting section:
// Avoiding breaking BC breaks with forced return types in public methods<br>ReturnTypeFromReturnNewRector::class =>...