AI-Assisted Technical Writing: When To Stop | AE1020: Lazy Notebook

I had a comment in a C++ codebase explaining why a particular cast was<br>written the way it was. The comment kept growing… and eventually it was<br>longer than the code it was documenting. So I did what seems like the<br>obvious modern move: I asked Claude.ai to turn it into a blog post I<br>could link to instead, and pulled the explanation out of the code<br>entirely.

I’d have thought that would have been the end of it. It wasn’t.

I showed the draft to Google Gemini for review. It found a real problem:<br>not a style nitpick, an actual technical error. And Gemini proposed a<br>fix.

But when I tried the fix, it didn’t compile. Which meant my original<br>comment, the one I’d been confidently maintaining across code reviews<br>for who knows how long, had been wrong the whole time. Nobody had caught it<br>because nobody had actually tried to compile the counterexample.

So the article went back to the drawing board. Then I showed that<br>version to more AIs: Perplexity.ai, ChatGPT, Microsoft CoPilot… and the<br>article started to spiral. Each pass surfaced something the previous<br>one had gotten subtly wrong or hadn’t considered. New names were coined<br>for things that turned out to need distinguishing. A whitelist<br>mechanism got added so a shortcut I’d deliberately chosen for its<br>runtime characteristics could be marked “checked” rather than<br>“assumed.” By the end it had gone from a comment, to a blog post, to<br>something that reads like a lab notebook.

The article itself is here, if you want the technical specifics<br>(it’s a kind of insane C++ pointer-cast rabbit hole and not the point of<br>this post):

https://ae1020.github.io/implicit-cast-vs-waypoint-cast/

Here’s the part I think is actually worth writing down, separate from<br>any of that: the disagreements in this process came in two completely<br>different flavors, and I didn’t notice how different they were until I<br>was deep into the second one.

The first flavor had a referee. When an AI told me “hey, I have a<br>better idea”, and it wasn’t, a compiler settled the argument in about<br>ten seconds. Nobody had to be persuaded of anything… the code either<br>compiled and did the right thing, or it didn’t. Every technical<br>dispute in this process eventually resolved the same way: not by which<br>AI sounded more confident, but by going and checking.

The second flavor didn’t have a referee. Once the technical content<br>was solid, I asked for opinions on the writeup itself, and got a real<br>split. Some models (in particular ChatGPT and CoPilot) thought it was<br>way too long (many AI are generally asked to summarize, after all).<br>Claude.ai thought the length was earning its keep by showing the<br>mistakes happening rather than just asserting the fixed conclusion.<br>Nobody could compile their way to an answer about the length, because there<br>isn’t a ground truth the way there is for “does this pointer get corrupted.”<br>That’s not a factual question. It’s a question about what the piece is for.<br>It turned out nobody (including me) had actually settled that before opinions<br>started flying.

What broke the deadlock wasn’t more argument. It was going back and<br>asking what the thing was supposed to accomplish in the first place. I<br>hadn’t set out to write something that would win points for concision on<br>a general audience – I’d set out to write something I could link from<br>a comment, for the one person, someday, who opens that file and wonders<br>why the cast is written that way. Once that was explicit, most of the<br>length argument dissolved on its own. Some of it didn’t: a couple of<br>the “this is too much” critiques turned out to be right on their own<br>terms, about naming things you don’t need to name, not about length at<br>all. Those parts I kept.

I think that’s the generalizable lesson, if there is one: when you<br>can’t tell whether a disagreement is technical or editorial, that<br>confusion is itself worth stopping and naming , because the two need<br>completely different tools to resolve. One needs a compiler, or an<br>experiment, or a fact. The other needs you to say out loud what you’re<br>actually trying to do. That’s weirdly easy to skip when everyone in the<br>conversation (human or AI) jumps straight to optimizing before<br>anyone’s confirmed what’s being optimized for.

There’s no compiler for “is this done.” At some point you just decide,<br>and move on.

So once technical accuracy had reached consensus, the revisions to the<br>casting article stopped where I, (the human), decided . If you think it’s<br>too long and the outcomes need summarization, just ask an AI. 🤖

If you have an idea for improving this article, please submit it as an issue on GitHub (or even better as a pull request)!

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.