Robot Comment Classifier

Robot comment classifier

Here’s a comment I read in some code I was working on.

Tagger walks the entire collection to locate flagged clusters. Skip it altogether when the cached, dirty-tracked flag count says there are none (the usual case): nothing needs marking, so the walk is pure waste. When the count is stale (e.g. right after an edit) or non-zero, we fall through to the real lookup, so stale marks never show. Same eventually-consistent signal the “Flags” badge relies on.

One thing in this comment caught my mind: the small parenthetical that says “the usual case”. If that were true, it would be important information! However, this comment is generated by a robot, which has no sense of what counts as the usual case in this domain. But by generating that comment, the robot tricks every future reader (both human and robot) into believing in a property of the system that does not exist.

Since I knew, in this case, that the comment was generated by a robot and that the property it contained was hallucinated, I could fix the comment. But that’s not always so easy. Here’s another comment.

AcmeRate’s live currency conversion (in our per-line totals) are incompatible with connection pooling: on a reused connection the rate lookup runs against the previous session’s locale and returns figures in the wrong currency. So fall back to a fresh connection when the order has a foreign-currency line (and therefore conversion), and keep pooling otherwise so large domestic orders stay fast. sessionInit runs before the request config is assembled, so localeState is already populated when this is read.

This comment implies that it is important that we “keep pooling” so that “large domestic orders stay fast”. If a human wrote this comment, I would assume they had thought carefully about it, and determined that the optimisation must be kept in. But if it’s written by a robot, it sounds more like it defensibly kept something in the code that it has no business deciding about, because it doesn’t know what it’s doing.11 The optimisation was useless. Only a little domain experience is needed to know that nearly every order contains a foreign currency, and large domestic orders are fast even without pooling.

The robot comment classifier

To aid my intuition, I wrote a classifier for source code comments. When fed the opening example of this article, it correctly predicts the comment was generated by a robot with almost certainty. The second example is also predicted to be robot-generated with very high confidence.

I wish at this point I could say “and it achieves 98 % accuracy!” like some of the llm detection papers out there do, but no. Some other spot checked examples include:

Provenance Classification Confidence

Robot Misclassified 53 %

Robot Correct 67 %

Robot Correct 78 %

Robot Correct 79 %

Robot Correct 92 %

Robot Correct 93 %

Robot Correct 99 %

Human Correct 55 %

Human Misclassified 71 %

Human Correct 74 %

Human Correct 85 %

Human Correct 90 %

Human Correct 96 %

Human Correct 97 %

Mixed Robot 54 %

Mixed Robot 74 %

Mixed Human 76 %

Mixed Human 86 %

As you can see, misclassifications happen often enough to show up in this short table, but it is encouraging that the misclassifications generally happen at lower confidence levels.22 The mixed-provenance comments were originally generated by a robot, but a human – that would be me – found the comments so terribly written they had to be rewritten. The rewritten comments retained some of the robot-generated structure, but they also have elements of human style. It makes sense that these comments don’t get super high confidence levels either.

I don’t think the accuracy can go beyond the roughly 80 % this model gets, for two reasons.

Code comments are often short, around 20 words or less. There’s only so much signal that can be extracted from that little data.

The training data is inaccuractely labeled. See the methodology notes further down for more on that.

This model is good enough to be an additional piece of evidence when I need it, but I wouldn’t trust it alone.

How do detect differences in language

The most interesting bit is probably not the classifier itself, but what it looks at to distinguish robot-generated comments from human ones. It’s worth knowing that while this is trained on a wide range of human authors, I suspect the vast majority of robot-generated comments in the training data come from Anthropic models, so the features we see below don’t really distinguish robots-in-general from humans as much as they distinguish Anthropic llm models from humans. This is an important point we’ll get back to later.

I’m not versed in computational linguistics, so for this project I did some cursory reading and picked up three basic ways to decompose texts to try to extract style...

Robot Comment Classifier

Related Articles

Incident with Github.com

(no title)

Scientists reverse brain aging, with a nasal spray

NHS apologises and admits Palantir have access to identifiable patient data

Cursor Origin