Fun with an indecisive AI coding agent
Ben Hoyt
Home
Resume/CV
Projects
Tech Writing
Non-Tech
benhoyt.com<br>benhoyt@gmail.com
Fun with an indecisive AI coding agent
April 2026
I’ve been having moderate success using an AI agent to help me fix non-trival bugs in GoAWK. But it’s sometimes very indecisive (Mitchell Hashimoto noted this in a tweet just the other day).
I asked it the following question (I’m using Claude’s Opus 4.6 with the GitHub Copilot plugin in the GoLand IDE):
GoAWK prints out "0\n0\n" instead of "x 1\n" for the Awk program BEGIN { a["x"]=1; for (NR in a) print NR, a[NR] }. Why?
It diagnosed the problem within a few paragraphs, and quicker than I could. It showed that I’m storing these special variables as native Go ints, which lose any string representation.
Then it started trying to fix it. Over the course of several minutes, with me becoming more and more amused, it gave me 7 unique options for fixing the bug, and flip-flopped between them no less than 25 times.
Here are the options it came up:
Option A: preserve string representation for special variables
Option B: store special variables as value type
Option C: store string override when special variables are set to string
Option D: fix ForIn opcode specifically
Option E: store original value in side field
Option F: just change lineNum (NR) and fileLineNum (FNR) to value
Option G: add special overrides map of value type
Every few seconds it would say “Actually, the simplest fix…” or “But no, the real issue is…”. It went on and on until I cancelled it.
To its credit, it proposed Option B most often (11 out of 26 times), and that’s almost certainly the proper fix. I’ve done it in this PR.
For your interest, I’m including a version of the chat output below. It’s heavily clipped to just show when it was suggesting another option. I’ve marked those in bold, like this: Option A .
Looking at this bug, the issue is in the ForIn opcode handler…
But wait, let me look more carefully at the actual ForIn implementation …
Now I can see the bug clearly …
This confirms the bug. Here’s the complete picture: …
The root cause is that special variables like NR, NF, FNR, etc. are stored as native Go int types, which inherently lose the string representation. …
The fix needs to happen in setSpecial and getSpecial — we need to preserve the string representation for these numeric specials. … Option A
However, a simpler and more targeted approach: the issue is specifically about for-in with special variables. But really the same problem would occur with any assignment like NR = “x” — it would lose the string. Let me think about the best fix.
The cleanest fix is to store these special variables as value types … but that’s a large refactor. Option B
A more surgical fix: store a “string override” for numeric specials when they’re set via string. But that’s hacky. Option C
Actually, let me think about this differently. …
The right fix is to store the value as a value type and only extract the int when needed internally. … Option B
Actually, the simplest fix that preserves the current architecture: for the for-in loop specifically, we could change the approach. Option D
But no, the real issue is more general — any string assignment to a numeric special should preserve the string.
Let me think about a minimal but correct fix. The most targeted approach: change the numeric special variable storage from int to value, and update the few places that read/write them. … Option B
This is a bigger refactor than I want to do. Let me think of a simpler approach. …
Actually, the most pragmatic and minimal fix is to handle this at the ForIn level. … Option D
Actually, I think the right fix IS to change these numeric specials to store as value type. … Option B
Changing all of these is risky. Let me think of an even simpler approach.
Actually, the simplest correct fix: In setSpecial, for the numeric specials, store the original value in a side field so that getSpecial can return it when it matches. … Option E
OK, let me think about this more carefully. …
This is getting complex. Let me take a simpler approach: just change lineNum and fileLineNum to value type, since those are the most likely targets of for-in. … Option F
Wait, there’s actually a much simpler approach. …
OK let me take the pragmatic approach. The cleanest minimal fix for this specific bug: change the numeric special variables (lineNum, fileLineNum, matchLength, matchStart, argc, numFields) to be stored as value types. … Option B
Actually wait, numFields is heavily used internally as an int. Let me try a different approach. …
Actually, here’s the simplest correct approach: For each numeric special variable, store BOTH the int (for internal use) AND a value (for the AWK-level representation). … Option E
But that’s still a lot of changes. Let me think even simpler…
The simplest correct fix that handles the specific case: in setSpecial, for numeric specials, if...