LLMs for Technical Editing: The Good, the Bad, and the Ugly | Tech Stackups

Skip to main content<br>The experiment​

With the existence of Opus 4.8 and the limited re-release of Fable to the global public, you may be thinking that it’s possible to completely replace your writers and editors with AI.

It’s certainly possible, but it would also be the most inefficient, self-sabotaging decision you could make if you want people to actually care about your content and connect with your brand.

That said, I’ll admit that I have a bias – I'm an editor who finds the corporate obsession with AI counterproductive.

Nevertheless, some people are still convinced that AI can replace editors, and I'm going to show you why that isn't true. In the interests of remaining objective, I've used AI to edit an already published article seeded with errors.

By the end of this experiment, we'll be able to tell where Claude falls between two extremes: Can it replace editors entirely, or is it just fancy (and sometimes incorrect) autocomplete?

I seeded our published AX article<br>with 23 errors of varying severity:

Error typeCountExampleHomophones & near-homophones4"you can er on the side of longer docs"Grammar4"How to testing your AX" (a heading)Consistency4"optimise" in an otherwise US-English articlePunctuation3A deleted period creating a run-onLogic3The article's own framework defined backwardsTypos2"a devv prompts the agent"Doubled word1"short and and snappy"Verbatim duplication1An entire paragraph pasted twice, back to backStructure1A transition paragraph moved two sections too early<br>Then I asked both Opus 4.8 and Fable to evaluate the error-ridden article, using prompts from<br>our editing prompt library.

NB: I asked Claude to identify the errors first before suggesting fixes.

(I'd also recommend reading this article to see how Opus 4.8 fared on editing the clean version of the AX article.)

The Good: Holding your piece together​

Okay, as much as I hate to say it, Claude did really well with flagging structural and logical errors. If there was a mismatch in your headings, contradictions in content, or even missing information, both Opus and Fable were good about flagging these errors and suggesting appropriate fixes.

1. It caught a framework contradiction​

The article defines three hurdles – discovery (does the agent know about you?), onboarding (can it sign up?), and usage. I seeded this sentence, which swaps the first two labels:

Does the agent sign up with minimal help from its handler (discovery?) Does it know about you (onboarding)?

Both models caught it.

Fable:

This directly contradicts the framework the article itself set up two sections earlier, where discovery is "the agent should know that you provide a solution" and onboarding is the sign-up flow. The one conceptual takeaway a reader is meant to walk away with is stated backwards.

Catching this requires holding definitions from two sections earlier in mind and checking a later sentence against them. Simple pattern-matching tools wouldn't be able to do that.

2. It caught a heading arguing against its own section​

I inverted a heading to read "Signing up is usually still more about AX than DX" – directly above body text explaining that it's still a human who visits your sign-up page.

Fable:

If a human does the signing up, that's the developer's experience being tested,<br>not the agent's — the heading should say sign-up is still more about DX than AX.

3. It caught a claim refuted by its own example​

I swapped the subjects in this sentence, so that the example now proves the opposite of the claim:

Note that humans generally do much longer web searches than agents. While a human would have searched for something like Steel captcha, Claude does Steel.dev solve captcha session config Python SDK 2025.

Opus, correctly and bluntly:

The human query is shorter; the agent's is longer. The example proves the<br>opposite of the claim.

4. It did 30 minutes of consistency checking in one pass​

Both models swept the piece for mechanical inconsistencies in a single prompt: a British "optimise" in a US-English article, one curly apostrophe among dozens of straight ones, spaced en dashes in a document that uses em dashes, "head-less" versus "headless", and lowercase "captcha" against uppercase "CAPTCHA".

The Bad: What Claude missed​

It may surprise you to learn that where Claude struggled the most was with picking out typos and simple grammatical errors.

1. Both models looked straight at an error and flagged the wrong thing​

I changed "it's" to "they're" in this sentence, breaking the pronoun agreement:

Skyscanner has a simpler CAPTCHA but they're not part of Steel's automatic solving, so it took longer.

Both models flagged a typographical error, but neither spotted the pronoun mismatch.

Fable:

Line 248: "they're" uses a curly apostrophe; every other contraction in the file uses straight apostrophes.

Opus:

The document is straight quotes/apostrophes throughout...