Unified Memory, Explained: Why Mini PCs Can Run 70B Models a Big GPU Can't (and Where They Slow Down)
Skip to content
Subscribe
Dark
Put two machines on a desk, each about $2,000. One is a tower with an NVIDIA RTX 5090: 32GB of the fastest consumer memory ever shipped, 1,792 GB/s. The other is a mini PC the size of a paperback, an AMD Ryzen AI Max+ 395 "Strix Halo" box with 128GB of soldered memory at roughly 256 GB/s. Now ask each one to run a 70-billion-parameter model.<br>The RTX 5090 cannot. A 70B model at a sensible 4-bit quant needs about 40GB, and 40 will not fit in 32. The little mini PC loads it without complaint, then answers at the pace of a slow reader. That paradox is the entire mini PC category in one image: these boxes can hold models that a much faster GPU cannot, and they pay for it in speed. Understanding why comes down to one idea, unified memory, and two numbers that pull in opposite directions.<br>We have not benchmarked these boxes ourselves. What follows synthesizes vendor specs, the inference literature, and owner-measured numbers, all linked at the end.<br>What "unified memory" means<br>In a normal desktop, the CPU has its own system RAM and the graphics card has its own separate VRAM, and data shuttles between them over the PCIe bus. A model has to fit inside the GPU's VRAM to run on the GPU, which is why a 24GB card sets a hard 24GB ceiling no matter how much system RAM you bolt on.<br>A unified-memory machine throws out that split. The CPU, the integrated GPU, and the NPU all share one single pool of soldered LPDDR5X memory. There is no separate VRAM, so almost the whole pool can be handed to a model. Buy the 128GB configuration and you have something close to 128GB of "VRAM" for a model to live in, for around the price of one mid-range graphics card. Apple has built Macs this way for years; AMD's Strix Halo, NVIDIA's DGX Spark, Intel's Core Ultra, and Qualcomm's Snapdragon X all now do the same. That is why mini PCs suddenly entered the local-LLM conversation at all. Capacity, cheaply.<br>The two numbers that decide everything<br>A machine's fitness for local LLMs comes down to two specs that people constantly confuse:<br>Capacity (how many GB of memory): decides whether the model loads at all. This is where unified-memory mini PCs win.<br>Memory bandwidth (how many GB per second): decides how fast it generates text once loaded. This is where they lose to real GPUs, badly.<br>Here is the current landscape. Bandwidth figures are theoretical peaks (real delivered bandwidth runs lower, a point we return to). Capacity is the top usable unified configuration.
Machine (SoC)Memory busPeak bandwidthMax unified memory
Intel "Lunar Lake" (Core Ultra 200V)128-bit LPDDR5X-8533~137 GB/s32 GB<br>Qualcomm Snapdragon X2 Elite128-bit LPDDR5X~152 GB/sup to 128 GB<br>Intel "Panther Lake" (Core Ultra X)128-bit LPDDR5X-9600~154 GB/s96 GB<br>Qualcomm Snapdragon X2 Elite Extreme192-bit LPDDR5X~228 GB/sup to 128 GB<br>AMD Ryzen AI Max+ 395 "Strix Halo"256-bit LPDDR5X-8000256 GB/s128 GB<br>NVIDIA DGX Spark (GB10)256-bit LPDDR5X-8533273 GB/s128 GB<br>Apple M4 (Mac mini)128-bit120 GB/s32 GB<br>Apple M4 Pro (Mac mini)256-bit273 GB/s64 GB<br>Apple M4 Max (Mac Studio)384 to 512-bit410 to 546 GB/s128 GB<br>Apple M3 Ultra (Mac Studio)1024-bit819 GB/sup to 512 GB<br>for contrast: discrete GPUs (much faster, far less capacity)<br>RTX 3090 / 4090384-bit GDDR6X936 / 1,008 GB/s24 GB<br>RTX 5090512-bit GDDR71,792 GB/s32 GB<br>RTX PRO 6000 Blackwell (~$8,500)512-bit GDDR71,792 GB/s96 GB
Sources: vendor spec sheets, Chips and Cheese, NotebookCheck, TechPowerUp. Bandwidth = theoretical peak.<br>Read the table as two stories. Down the mini PC rows, capacity climbs to 128GB (and 512GB on the M3 Ultra) while bandwidth stays between about 120 and 270 GB/s. Down the GPU rows, it is the reverse: 900 to 1,800 GB/s of bandwidth, but capacity capped at 24 to 32GB until you reach an $8,500 workstation card. A mini PC and a used RTX 3090 are not two points on one scale. They are opposites.<br>Why bandwidth, not compute, sets the speed<br>To see why the fast GPU and the roomy mini PC behave so differently, you need the roofline model (Williams, Waterman, and Patterson, 2009), the standard way to reason about performance. It says a workload is limited either by how fast the chip can compute or by how fast it can move data from memory, whichever runs out first. Which one bites depends on how much math you do per byte you read.<br>Text generation does very little math per byte. To produce one token, the machine has to read essentially the entire model out of memory once, then do a small amount of arithmetic on it. So the clock is set by memory bandwidth, not compute. Apple's own machine-learning team puts it plainly in their MLX write-up: "Generating subsequent tokens is bounded by memory bandwidth, rather than by compute ability." The academic version is the same. Pope et al.'s "Efficiently Scaling Transformer Inference" (2022) models generation latency as dominated by the time to stream weights from...