Can you tell AI code from human code — and does it matter?

The habit I couldn't unsee

Earlier this year I was building a website and an automated mailer system with AI assistance. Nothing exotic — forms, a bit of PHP, the usual. Somewhere in the second week I noticed something about the way the assistant worked: it never, not once, edited a line of code in place. Every change, however small, was the same move — delete the block, write a new block. Ask it to fix a one-character comparison bug and you don't get a one-character fix; you get a fresh function.

Humans don't work like that. When you or I maintain code we rename a variable, tighten a condition, thread a parameter through three call sites. We repair. The model regenerates. And once you've seen it you see it everywhere — watch your own diffs for a day and tell me I'm wrong.

I've spent twenty years measuring code churn in large codebases, so my reflex when I notice something is to count it. That reflex took me somewhere I didn't expect, and this article is the route: first into the question everyone asks (can you detect AI-written code?), then into the question almost nobody asks (does it even matter who wrote it?), and finally to the one that I now think is the only question with teeth.

First question: can you actually detect AI code?

There's a real literature on this now, and I went through it properly — the survey is written up here, with links to every source. Short version: four families of technique. Stylometric classifiers trained on engineered features (best multilingual result I found: 84.1% F1 across ten languages). Zero-shot probability-curvature methods descended from DetectGPT. Perplexity scoring. And fine-tuned neural classifiers — one modified CodeT5 paper reports above 97% on its own dataset.

Those numbers sound like a solved problem. They aren't, and the caveats aren't small print — they're the story:

The strongest signals are the shallowest. One controlled study found comment style moved the decision boundary more than AST structure did. Comments are also the first thing a human touches. Reformat, rename, strip the comments — the classifier folds.

Detectors rot. Train on 2024 model output, test on 2026 model output, watch accuracy drift. The target moves every release cycle; the detector doesn't.

Almost nothing is purely one or the other. Real repositories are human code touched by models and model code touched by humans. One dataset effort had to invent a third class — "machine-refined" — and notes the pure cases are rare. The binary question dissolves on contact with an actual codebase.

So my answer to "can you tell?" after actually reading the evidence: sort of, sometimes, with confidence that degrades under trivial edits and across model generations, on a question that's mostly a spectrum anyway. A per-file verdict of "an AI wrote this" is a probability dressed up as an accusation. Treat these things as pointers, not verdicts — and anyone selling you certainty here is selling astrology with a confusion matrix.

What you can measure: the shape of the diff

Here's the thing though. My mailer observation — delete-and-regenerate, never repair — isn't a property of the code's style. It's a property of the process. And process leaves fingerprints in version history that no amount of prompt engineering can wipe, because they're made of arithmetic, not vibes.

Classify every churned statement between two snapshots of a codebase as added, deleted, or changed in place. Then ask one question: what fraction of all that churn was in-place editing? Call it REWORK. No ML anywhere in the measurement — it's a diff and a division.

I measured nine codebases. The full write-up, with every commit hash so you can re-run every public row yourself, is here. The result surprised me more than anything I've measured in years:

CodebaseWhat it isREWORK

curlhand-maintained C, est. 199816.7%<br>Expresshand-maintained JavaScript, est. 201016.7%<br>PostgreSQLhand-maintained C, est. 199617.1%<br>PostgreSQL stable branchesbug-fix-only maintenance17.6–18.0%<br>deno / esbuild / vite / Kubernetespre-AI hypergrowth windows, 2015–20211.6–7.2%<br>An agent-built application65.9% of commits carry AI-agent trailers0.19%

Three established projects — different languages, different decades, churn volumes three orders of magnitude apart — cluster within half a point of each other. Whatever humans maintain, about one churned statement in six is an in-place edit. Even PostgreSQL's bug-fix-only branches, which I probed expecting pure repair work to score much higher, sit in the same band: it turns out even bug fixes are add-dominant, because fixes bring their guard code and tests with them.

The agent-built application edits one statement in five hundred. That's a ninety-fold collapse against the established cluster. The delete-and-regenerate habit I spotted in my mailer diffs, showing up in six figures of churn as plain arithmetic.

Now the honest part, because the...