Buy tokens, rent GPUs, or own the rack? - Flowstate

Menu

Until recently, self-hosting a frontier model wasn’t a serious question. Open-weight models sat a generation behind the proprietary ones, so there was no calculation to make.

Then the gap closed. Not to zero, but to about thirty Elo. Kimi K3 shipped with open weights and sits second on the frontend code arena at 1,682, behind Claude Opus 5 Max on 1,712 and ahead of everything else1. Its weights fit, on paper, on a single eight-GPU node. You can download the model tonight for nothing.

The box is another matter. Assuming you can make the capital investment. And assuming, once you’ve done the arithmetic, it turns out to be worth doing at all.

So I modelled four ways to run the same workload: a rack in your office, owned hardware in a colocation facility, rented dedicated GPUs, and pay-per-token APIs. I’ll call that last one the meter throughout.

The answer isn’t “buy the rack”. Under about 44 engineers the meter stays cheapest. Between roughly 44 and 95, rented GPUs win. Above that owning starts to pay, and by 750 engineers it beats the meter roughly six to one.

The number that decides it is utilisation. A rack bought on a three-year schedule costs the same at three in the morning whether it’s serving tokens or warming the building.

K3 is only the worked example here. The same method applies to whatever ships next.

Four ways to buy inference

OptionYou ownYou rentYou pay forOffice rackGPUs, power infrastructure, coolingnothingCapex, electricity, an electrician, floor space, peopleColo rack (your hardware, someone else’s building)GPUsSpace, power, coolingCapex, rack rental per kW2, peopleDedicated / rented GPUsnothingWhole GPUs by the hourReserved hours, peopleThe meter (per-token API)nothingnothingTokens

As you move down that table your fixed commitment falls and your flexibility rises. What you give up is control, and at sufficient scale, unit economics. Finding where that trade flips is the whole exercise.

Before the spreadsheet, though, a more basic question: can you actually put one of these machines in an office?

Can we?

In season two of Silicon Valley, Gilfoyle’s servers catch fire. The team piles too much load onto Anton, it maxes out the amperage, blows the main breaker and burns. Everyone remembers it as a gag about hubris. Here, we’re treating it as commentary on electrical engineering.

Anton, moments after the main breaker gave up. Silicon Valley, HBO.

Apologies if you haven’t seen Silicon Valley. It’s a modern classic, ahead of its time.

K3 is 2.8 trillion parameters, 104 billion active per token, 896 experts, quantised to MXFP4 straight out of training3. That is 1,390GB of weights on paper. An eight-GPU B200 node gives you 1,440GB, so on paper it fits with 50GB to spare. Reports of the actual checkpoint put it nearer 1.56TB once everything that is not a quantised weight is counted, in which case it does not fit at all. This is the r/selfhosted wet dream: the second-best coding model on earth, humming away in a cupboard, yours.

Then you read the spec sheet.

Spec, one DGX B200ValueWhat it means in an officePower draw14.3 kW4Two full induction hobs, every ring, permanentlyUK ring main7.4 kWOne server wants two entire circuits to itselfHeat output~48,800 BTU/hrThree to four domestic aircon units, running flat outAirflow2,145 CFM4Not a cupboard. A plant roomWeight130 kg4Before the rack, PDUs and coolingTwo nodes per rack2x 380V three-phase 32A4You are calling an electrician, not buying an extension lead

One inconsistency to own: the power, weight and airflow figures are NVIDIA’s for the DGX B200, its own integrated appliance, while the $450,000 I model is a street price for an 8-GPU HGX B200, the board an OEM builds a server around. A DGX lists nearer $515,000. I use the cheaper number with the pricier machine’s power draw, which flatters self-hosting on capex and penalises it on electricity.

Anton didn’t blow that breaker because a writer wanted a fire in act three. It blew because that’s what happens when you hang industrial load off domestic wiring.

One node is the charitable case, by the way, and it’s charitable in a second way too. Those 50GB of headroom leave nothing for KV cache, the working memory a model needs to hold a long conversation, so a real production deployment needs more accelerators than this. Moonshot recommends 64 or more5. That’s eight of these boxes, 114 kilowatts , in your office. Less a server room than a micro nuclear reactor waiting for someone to announce a Series A around it.

I’ve modelled the single node anyway, because it is the cheapest thing that could plausibly hold the weights. Be careful with that, though: it is not a worst case. Staff and electrical work are close to fixed, so more nodes spread them and the economics improve. A one-node result is the hardest test self-hosting faces, not the easiest.

Then there’s supply. Blackwell allocation goes to the biggest buyers first, so getting...