Route the Work, Not Just the Data: GPUs, CPUs, and the Rise of AI-Native Storage
Paul Woll 路 Triuna Labs Research 路 August 18, 2026
馃 Route the Work, Not Just the Data: GPUs, CPUs, and the Rise of AI-Native Storage
When we think about Large Language Models, we tend to picture GPUs.
That makes sense. Modern generative AI would not exist at its current scale without them.
But GPUs also expose one of AI's increasingly important architectural problems:
The fastest place to compute is not the cheapest place to keep data.
An NVIDIA B200 GPU has 180 GB of HBM3e and can move data through that memory at up to roughly 8 TB/s .
At the other end of the hierarchy, Micron began shipping its 245.76 TB 6600 ION SSD in May 2026 .
One drive can hold more than a thousand times as much data as a single B200's HBM.
But flash operates nowhere near HBM's bandwidth or latency.
That enormous gap between hundreds of gigabytes of extraordinarily fast memory and hundreds of terabytes of comparatively inexpensive persistent storage creates a fascinating architectural question:
What if storage stopped being merely the place where AI data waits for the GPU?
What if some AI workloads were processed near the storage itself, while an intelligent storage tier decided what actually needed to reach expensive GPU memory?
That may sound futuristic.
It actually follows a research path stretching back almost three decades.
And LLMs may provide one of the strongest reasons yet to pursue it.
Here is the thesis of this article, stated plainly:
An LLM request is not one monolithic computation. It is many kinds of work, and only some of it needs a GPU. As model state outgrows GPU memory, the winning architecture will route each operation to the cheapest tier that can perform it (GPU, CPU, or increasingly intelligent storage), and the cost that decides the route is data movement. The next major optimization is not making the GPU faster. It is reducing how much data has to reach it in the first place.
Everything that follows is the evidence: what already ships, what research demonstrates, what I measured on my own hardware, and what remains genuinely speculative.
HBM wins on bandwidth. Flash wins on capacity. AI architecture increasingly lives in the gap.
鈿欙笍 First: What Is an LLM Actually Doing?
An LLM is not searching a giant database for a sentence matching your prompt.
At a simplified level, it repeatedly performs enormous amounts of numerical computation.
Your text is divided into tokens .
Those tokens are converted into numerical vectors and passed through many layers of a neural-network architecture called the Transformer .
The Transformer was introduced in the landmark 2017 paper Attention Is All You Need.
Two operations are especially important.
Attention
Attention helps the model determine which previous tokens matter when interpreting the current token.
A simplified version creates three representations:
Query: What am I looking for?
Key: What information do I represent?
Value: What information should be passed forward?
Queries are compared against keys.
Attention scores are calculated.
Those scores determine how strongly different values influence the next representation.
Feed-Forward Networks
Each Transformer layer also contains large learned matrices that transform the token representations.
Across billions of parameters, this creates an enormous amount of multiplication and accumulation.
Eventually, the model produces a probability distribution over possible next tokens.
It selects a token according to the decoding strategy, appends it to the sequence, and performs the process again.
And again.
And again.
That repeated numerical workload helps explain why GPUs became so important.
馃М Why GPUs Are Better Than CPUs for LLMs
CPUs are extraordinary general-purpose processors.
They are designed for workloads such as:
Operating systems
Application logic
Branching
Databases
Networking
Scheduling
Serial dependencies
Irregular computation
Many different instruction types
A CPU's strength is flexibility.
An LLM workload is different.
Huge portions of it repeatedly ask something closer to:
Can you multiply these enormous arrays of numbers as quickly and in as much parallelism as possible?
GPUs were built for parallelism.
Modern AI GPUs contain thousands of execution units plus specialized Tensor Cores designed for matrix operations using formats such as FP16, BF16, FP8 and increasingly lower-precision representations.
They also sit beside extraordinarily fast High Bandwidth Memory.
NVIDIA lists a B200 at up to roughly 8 TB/s of HBM bandwidth per GPU .
A high-performance PCIe Gen5 SSD such as Micron's 9550 reaches roughly 14 GB/s of sequential read bandwidth .
Those are completely different performance classes.
There is no plausible architecture in which NAND flash simply becomes a drop-in substitute for GPU HBM.
But that is not the interesting question.
The interesting question...