AI Broke Software's Best Trick · Ardonio Ltd.Software ate the world, per Andreessen. And it did so on one particular economic pillar: zero marginal cost. In simple terms: the cost of producing an extra copy of your application is effectively zero. Compared to the cost of coding the application, shipping a few floppy disks is effectively zero cost. These days, SaaS software has replaced that shipping cost with compute cost. But the relative equation holds. One extra user of your product carries a negligible cost compared to creating that product.<br>genAI is different . Every user, every question/response, every nano banana image, … every interaction carries a significant direct cost. And the more a user engages, the higher the cost to the company. And it turns out that providing that service is pretty expensive, since you need high end datacenter hardware to run the computations that produce the output.<br>The traditional “scale your way out of it” doesn’t work if you’re not profitable on a per-user basis. It just digs the hole faster.<br>Oops…<br>And of course, nobody sat around wanting “AI”. The models are only interesting to the extent we can do something of value with them. For some that might be entertainment (like asking a model how to get a peanut butter sandwich out of the VCR in the style of the King James bible … we’ve all tried that, right?). For others that might look more productive, like trying to get a handle on sales data. And whether or not that task ends up being done reliably really depends on the task in question.<br>Broadly speaking, we currently use AI on high value and low value tasks.<br>High ROI tasks . Think about coding, complex agent workflows, … . The sort of task where the cost and token burn is probably justifiable. And exactly the sort of tasks where new frontier models are still meaningful steps forward. The difference between what Fable (when it was available) could code vs Opus 4.8 is significant.<br>Low ROI tasks . Summarizing a meeting, drafting an email from your bullet points, scheduling a task through calendar access. The ROI on “draft this email” is hard to justify at unsubsidized LLM pricing. I’m not saying there’s no value to it, but rather that the economics break. For $20 / month I’ll happily have claude write and rewrite summaries. At $200 / month … nah, my trusty manual notes in obsidian will do.<br>A broken model in three parts
Link to heading<br>First, as I just mentioned, cost grows in line with usage AND current subscriptions don’t cover cost. And yes, the saas subscription pricing is normally set so that a few net-loss power users get subsidized by a long tail of net-profitable users. SemiAnalysis and others have pointed out that a user using 25% of their token limits is at best a negative 25% margin. Or the more aggressive side of the calculus by Ed Zitron, 3 power users need at least 2000 low-end users to break even. Econ101: this is not how you get to a profitable product.<br>The second thread I want to quickly pull is amortization. Training a frontier model carries a cost in the 100s of millions to produce the asset (and let’s assume it’s all capex for simplicity), and projections are that this will keep growing as bigger models need more compute time to train and more data to ingest. On the flip side, let’s look at the lifetime of these models. Flagship models, major versions, ship roughly every 12-18 months with minor upgrades in between. And when a new model comes out the early adopters switch, and the laggards wait for it to become the default setting, which is typically quite quickly as well. So let’s say that the first 25% (power users) switches more or less right away, and the remaining gets swept up over the next 6 months as it becomes the default model. So you’ve got a 100+ million asset with, generously a 2yr shelf life, and you’re stacking a new asset of that size every year-ish. Sure, technically it will get written off over a longer life than its real practical use but in reality every model has about a year to make a profit.<br>But hey, token costs are coming down. This is the third place the economics break. Token costs are objectively coming down, however increasingly reasoning models consume more tokens to get to an answer (and FYI, reasoning tokens are considered output tokens; the expensive kind). Tokens are a poor proxy measure anyway of course, see all the “tokenmaxxing” backlash. The thing that really matters is task completion, the economically relevant result . And as models get more capable between reasoning and longer context windows, and we start sticking them in skills and workflows which burn more tokens as well, we’re actually driving the bill “per result” up, despite token costs coming down right now.<br>Solving the economics
Link to heading<br>Since we’re in a world where the users...