Software Engineering is the new Manufacturing Engineering

Sign in<br>Subscribe

Why does vibe-coding seem so mesmerizing and even magical at first--yet just weeks into a project, predictably fail to deliver the low-risk, dependable software a business actually needs?<br>I believe LLMs have pushed a large portion of the software engineering discipline from the domain of "code as craft" into the domain of manufacturing engineering--in other words: making stuff, but with statistical control.<br>This is transforming what we think of as "coding" from a handmade trade (like a ceramic potter) to an industrial process. It's akin to engineering a rocket part with a high-speed CNC machine: the tool is incredibly powerful, but without strict gauges to catch when a cut drifts out of spec, it will quickly ruin the entire part.<br>LLMs' probabilistic machinery has made the discipline of Statistical Process Control relevant to all software engineers who use AI for coding:<br>An LLM is, at its core, a statistical machine. But it is a rose with thorns: the very ability that enables it to create magic also enables it to create chaos (the service is down!), and also entropy ("tech debt"). The chaos and entropy can grow so large that vibe coders who once felt euphoric soon feel frustrated and even discouraged as the LLM won't make the code do what they want any more.<br>By "statistical machine", I mean that an LLM is a regression machine: in everything it does, it regresses to the mean over the inputs that it's trained on. If you take no code at all as a starting point, and tell it what to do, it will produce code! Mediocre code, but probably functioning code. If you take well-engineered code as a starting point, and tell it what to do, it will produce, over time, mediocre code.<br>Regression to the mean feels very different, as an experience, depending on your perspective and where you begin--whether you begin from above or below the line of "LLM averageness."<br>If you are kind of new to producing working software--if your ability to code, alone, would produce below-average code (for example, if someone had asked you, "Do you know how to code?" a few years ago and your answer had been, "No") then the experience of an LLM in the domain of software engineering will be magical. You will produce things the likes of which you never thought possible. You will write an app that serves the exact itch you had always wanted to scratch, and it will be glorious. If you are a non-engineering CEO, it will appear at first that software engineers are wasteful, lavish, slow-moving resources.<br>If you are skilled at producing working software--if your ability to code is above the mean (for example, you are a software engineer of several years) then the experience of an LLM in the domain of software engineering will be frustrating. Sometimes it will pleasantly surprise you. Often it will make poor decisions--sometimes remarkably stupid ones. As someone whose job it is to reduce risk to the business in the long run (i.e. an engineer), exclusive use of an LLM to produce code looks foolhardy.<br>I've chosen coding as one skill. But feel free to expand this general principle--choose any skill domain that LLMs are currently disrupting--generating art, producing film, storytelling, copywriting, email correspondence, etc.--and apply the same rules. If you are below the mean, it will feel like magic. Above the mean, frustration. A regression-to-the-mean-machine pulls you up if you're below, and down if you're above.<br>The New Discipline: Statistical Software Manufacturing<br>So where is this going? I think what we're seeing is the emergence of a new discipline, which I'll call Statistical Software Manufacturing (SSM). The motivation behind it is to take advantage of the insanely powerful abilities of LLMs, without their adverse effects--like modern manufacturing uses Statistical Process Control to reject widgets below a certain quality threshold. We aren't there yet, but I'm seeing glimmers of this new direction opening up.<br>In SSM, as in manufacturing, we can introduce Control Limit Violations (CLVs). An example of a CLV in manufacturing would be "the target diameter is >10mm." In SSM, the equivalent might be measuring the Functional Core / Imperative Shell metric in typescript: by traversing the Abstract Syntax Tree, a CLI tool like fcis (linked above) can deterministically flag when an LLM interleaves database calls or other I/O with pure math.<br>Then, if an LLM generates a new feature or fixes a bug, we don't accept its output until we measure it against FC/IS scores. And if it's measured to introduce a lower FC/IS score than we can tolerate, just like manufacturing engineering, we "rework" (explain to the LLM how it failed, and feed it back through), or "scrap" (discard, and start fresh).<br>Software engineers are already familiar with several other Control Limit Violation classes:<br>unit tests<br>linting<br>type checking<br>In each case, these deterministic measurements help trip a CLV,...