If AI Accelerates Your Software Factory, Where’s the ROI?Skip to content<br>We’re hearing a lot lately about the “Software Factory,” an iterative process for turning ideas into shipped changes: products, features, bug fixes, and optimizations. It’s a reductive but useful framing for how software gets built. And now the factory is getting robotic arms. Coding agents let teams ship more software, faster, without necessarily sacrificing quality.<br>I’ve felt the benefits firsthand while engineering at Firetiger, in the weeds helping companies apply AI to level up their production software and operations. Coding agents automate tedious work and allow us to ship more code than we ever imagined was possible. They’re probably the largest and most valuable application of “AI” discovered so far (besides ads…).<br>But talk to executives at software companies paying for coding agents rather than selling into the AI boom, and the story is murkier. Product and engineering productivity is accelerating, token spend doubly so, while most other business indicators show only marginal improvement, if any! Are our glorious Software Factories, despite producing incredible amounts of code, actually producing more “business value”? Where is the ROI?<br>Here’s an uncomfortable theory: perhaps software production was never the binding constraint on most companies’ success, even before product and engineering teams gained these new capabilities.<br>“No shit,” some of you muttered. But believe me, from what I’m seeing on the factory floor, this is not as obvious as it sounds! Those of us working inside the Software Factory typically forget to step outside, smell the flowers, chat with the truck drivers, and consider how the rest of the business works. From the production line, more software can look a lot like more business value. But if we’re going to use a manufacturing analogy, we might as well consider some of the fundamental ideas from industrial engineering and operations management. In the physical world, a factory is just the means of production. Its output still has to be distributed, marketed, sold, and used by real customers with limited attention. The Software Factory is not exempt.<br>Companies pay for coding agents to implement changes faster. They are also trying new AI-enabled software development tools to discover, validate, and safely ship changes faster. But none of this necessarily increase a company’s capacity to metabolize change as fast as they increase software production. The company must still decide which changes are worthwhile, get employees and customers to adopt them, understand whether they helped, and turn the added capacity into company progress.<br>And damn it, change is hard. One big reason is that every change requires the company to make tradeoffs, often without a clear objective function. A code change may improve performance while increasing cost. It may increase engagement without improving customer satisfaction. It may generate revenue while creating complexity that makes the next change harder. A/B experiments help only when we can isolate a change and measure its outcome. And sometimes the objective function isn’t even financial! A company may decide to build a rickety homegrown Slack agent not because it will produce obvious returns, but because doing so yields capability, promotions, ownership, and camaraderie among the team.<br>Being good at making these tradeoffs is a big part of what people mean by having “taste.” As product strategy gets cheaper to execute, deciding what is worth doing matters more. But taste is not enough, because you can’t make every great change at once.<br>Jeff Wilke made this point to Jeff Bezos at Amazon. Bezos alone, Wilke told him, had enough good ideas to destroy Amazon if he released them faster than the organization could absorb them. Every new idea created work-in-process, backlog, and distraction. The insight was to rate-limit the work while developing Amazon into an organization that could metabolize more.<br>Every organization has a saturation point: a limit on how much change it can process at once. This is the logic behind the Theory of Constraints, and it’s interesting to apply it to our messy companies. A business is obviously more complicated than a serial production line. It has many interconnected processes, and what counts as the bottleneck depends on what you’re trying to accomplish. But it’s still a system, and making one part move faster will not make the whole process move faster if the bottleneck is elsewhere. You might just cause a pile up of inventory somewhere down the line.<br>And to be clear, we’re not talking about the heaping inventory of unreviewed agent pull requests. We’re talking about how an engineer can create a bug fix before anyone has decided whether the bug even matters. A product team can ship a feature before sales can sell the last one, support can explain it, customers can adopt it, or the market can respond to it....