A Measured Decomposition of the Trillion-Parameter Serving Step, Brainsless Research Lab

Contents

The record

Run it yourself

The verify tax

Fovea

The exchange rate

What the proxy missed

Limits

Full text

The record, plainly

On four B200s we serve Kimi-K2.6 — a trillion-parameter open model — at 511.6 tokens per second: a record set at 505.9 (depth 7) and raised at depth 6 by blind re-runs of the unmodified public release, measured the way the public leaderboard measures the companies whose business is serving it: 10,000-token inputs, 2,048-token outputs, temperature 0.6, sixteen novel prompts per cell, prompts hash-pinned before results existed. Standings at the record's pin: us 505.9 from four stated GPUs, Crusoe 438.1 from hardware they do not state, Fireworks 381.2, CoreWeave 261.8 from a 72-GPU rack, Nebius 222.4, Together 218.2, and the rest of the fourteen-provider board below; at the raised cell's pin the leader read 451.0, with 511.6 standing 13.4% above it. Above every GPU provider the leaderboard measures for this model; per accelerator, roughly three times NVIDIA's disclosed trillion-parameter configuration; and 505.9 was the first single-stream GPU reading above 500 on this model.

The number was located by measurement, and the measurement is the story of this report. We decomposed the serving step piece by piece — the bare forward pass costs 6.6ms at 10k context, verifying a drafted token costs 0.397ms, doubling the node to eight GPUs changed nothing — and the decomposition said the step was mostly engine overhead. Moving the same model, the same public draft head, and the same protocol to the engine the physics pointed at removed 2.1 milliseconds per step and put the cell 15.5% above the best GPU provider median the leaderboard reports. When we replayed the leaderboard's own measurement path against a provider's production API, it read a third of that provider's board median (the full text prints the replication and its artifact); the standings above use the board's numbers as published. And the record itself replicates: three blind re-runs of the unmodified release landed every n=16 cell inside the stated node envelope and raised the record to 511.6 (depth 6) — with first-eight medians to 538 and individual requests to 568. Replication moved this record up, not down, and a record whose replications read above it is a floor. The rest of this page is that science, in order.

511.6 tok/s<br>the record: set 505.9 (depth 7), raised by blind re-runs of the release (depth 6, n=16) — artifacts: brl11_stage600r.json, brl11_repl_r1.json

1st / 1st<br>above every GPU provider median the leaderboard reports for this model; first per accelerator among disclosed counts (127.9 tok/s per GPU, ~3× the best)

6.6ms<br>the bare forward pass at 10k context — the physics floor of one step; verification adds 0.397ms per drafted token, measured

τ 5.51<br>accepted draft length per step on the record-setting cell (505.9, depth 7), cross-validated on two engines (4.825 / 4.815 at equal depth, same head)

Run it yourself

The record cell re-runs from the public release with one command:

modal run stage600_r.py

against a rented 4×B200 node — about $15 and forty minutes at the published rate. The runner rebuilds the sixteen prompts from source, asserts their SHA-256 against the pin printed in this report, fail-closes on any mismatch, and writes the same per-request artifact schema every table here is built from: raw output text, per-token timestamps, engine counters. The release carries every session runner, the exact engine versions and flags, and the artifacts behind each table.

brainsless/the-decoding-step

The verify tax

We measured both live drafter families on the same 1T target, same session, same novel prompts — the block-diffusion family guesses better in every domain and still loses the throughput race (the full text carries the table). On an ordinary dense model that result would be impossible: verifying nine tokens costs about the same as verifying four, because the model's weights stream through the GPU once per step either way. A mixture-of-experts model with 384 experts breaks that arithmetic. Each verified token routes to its own eight experts, so a step with more tokens touches more experts, and every touched expert's weights must be read from memory. Dividing each measured cell's acceptance by its throughput gives the implied cost of one step: 8.0–9.3ms for EAGLE's four-token verify, 11.5–16.6ms for DFlash's nine-token verify. On general chat the guessing advantage is a factor of 1.16 and the step-cost penalty is a factor of 1.8. The product decides the row.

That is the verify tax, and it is the number to check before believing any speed claim about giant MoE serving. Concurrent work measured the same expert-union effect on models up to 45B and built draft-length controllers on it; our measurement puts it at 1T, where it is strong enough to reverse the published ordering. It hits before the first...