The Engineer and the Autocomplete: What LLMs Can't Do

toadi1 pts0 comments

LLMs vs. Human Engineers: Who Really Understands the Code?

SubscribeSign in

LLMs vs. Human Engineers: Who Really Understands the Code?<br>What separates a software engineer from an LLM isn't raw output — it's causal ownership. Here's why that distinction matters more as AI tools get better.

Geert Theys<br>Jun 02, 2026

Share

Neither, straightforwardly. But the question reveals something important.<br>What the Human Engineer Actually Does<br>A good software engineer isn’t purely “understanding” either. A lot of day-to-day coding is pattern matching. Recognizing this problem as an instance of a known shape, reaching for a familiar solution. Experienced engineers are fast partly because they’ve internalized so many patterns they don’t have to reason from first principles every time.<br>So the human is already doing some prediction. The difference is it’s grounded in a causal model they can interrogate.<br>When something breaks in a novel way, the human can form a hypothesis about why, design an experiment to test it, update their mental model, and reason about implications for other parts of the system.<br>This is the loop LLMs can’t fully close. They can simulate the loop, sometimes convincingly, but they don’t have persistent stakes in being right. They don’t own the system.

Subscribe

What the LLM Actually Does Well<br>The LLM wins on a few real things:<br>Surface area. It has “seen” more code, more patterns, more solutions than any human ever will.

Speed on known territory. For well-trodden problems, it’s faster than searching docs or memory.

No ego. It doesn’t get attached to its own solutions or resist refactoring.

Tirelessness. It doesn’t have bad days, doesn’t lose focus at hour six.

These are real advantages, not trivial ones.<br>The Core Asymmetry<br>The human engineer builds a model of the system. Not just what it does, but why it’s structured that way, what assumptions are baked in, what will break under what conditions. This model lives outside the code itself.<br>The LLM has no persistent model of your specific system. Every context window is a fresh start. It can read your code and form a temporary, in-context model, but it evaporates. It has no memory of the last three times this bug appeared, no intuition built from watching this codebase age. I’ve been trying to solve exactly this problem.<br>This is why LLMs are much better at local tasks (write this function, fix this bug, explain this snippet) than global ones (why is our architecture making this class of problems hard, what will this change break in six months).<br>The Honest Answer<br>For a greenfield, well-specified, common-pattern problem: the LLM is probably faster and good enough.<br>For a complex, legacy, poorly-documented system with subtle invariants and accumulated decisions: the human engineer isn’t just better. The LLM is actively dangerous, because it will produce confident, plausible, wrong answers about things it can’t actually know. I’ve felt this firsthand.<br>The best current outcome is probably the human engineer using the LLM. Offloading the pattern-matching and boilerplate while retaining ownership of the causal model. The human provides the understanding that makes the LLM’s predictions safe to use. I’ve written about what that looks like in practice.<br>The risk is engineers who mistake the LLM’s fluency for understanding, and stop building the mental models themselves. Then you lose the thing that made the combination work.

Share

Discussion about this post<br>CommentsRestacks

TopLatestDiscussions

No posts

Ready for more?

Subscribe

© 2026 Geert Theys · Privacy ∙ Terms ∙ Collection notice<br>Start your SubstackGet the app<br>Substack is the home for great culture

This site requires JavaScript to run correctly. Please turn on JavaScript or unblock scripts

human engineer model llms code system

Related Articles