The Path To AGI Runs Through HBM - by Lawrence Lundy-Bryan
State of the Future
SubscribeSign in
The Path To AGI Runs Through HBM<br>Moore Than More w/ SEMRON, Issue 1 : The bet is that HBM will continue to rise in price. Will it? Answers at the end.
Lawrence Lundy-Bryan<br>Jul 16, 2026
Share
Gut morgan, I’m trying a new experiment. Sponsored posts. Because gotta get that bread, man. I’ve tried to make the sponsor, someone I would/will personally invest in, I need to really believe it, you know? They will get some editorial input but I am going to do my god damn best to not sell out and shill. I am independent-minded and do my own research. Call me out if I balls it up and lose your trust. Because what I am, without your trust. At that point, I am just a man. And what good is that to anyone?<br>So as Packy says: Today’s Weekly Dose is brought to you by… SEMRON
For those who have been reading this newsletter for a while, I want to take you back to November 2023, to a piece I wrote called “Can we make enough AI chips?” I modelled two scenarios. In the second, the pessimistic one, I wrote:<br>“In scenario two, an under-capacity scenario, demand massively outstrips the semiconductor industry’s ability to scale. This is the world of 2023, but worse. HBM3 continues to be sold out, and CoWoS cannot keep up; even hyperscalers like Amazon and Microsoft struggle to make enough chips. Nvidia, AMD and Broadcom/Google can’t meet demand, and prices rise.”
Hell of a call, that. Easy to cherry pick the right calls. What about all the crypto stuff… yea well we don’t talk about that. Like Bruno.<br>So, it’s 2+ years later and some of you may know Memory is Kind of a Big Deal Again. SK Hynix sold out. Not a product line, but a year. 2026. Its entire memory capacity for the year. Money left on the table, poor planning you might say.<br>“Betting that Nvidia, AMD, and TSMC have their numbers wrong is a tough bet. But what if they have?” Lawrence, Nov 2023
SK makes DRAM, the commodity memory in your computer box. Stack a tower of them and wire it straight into an AI accelerator and you get HBM, high-bandwidth memory, which is what a GPU needs to read a model super fast.<br>A standard 16Gb DDR5 chip sold for circa 7 bucks last September was up at about 27 in December. And re margins, SK Hynix ran at a 72% operating margin in Q1 2026, with $25bn in that quarter, which was more than in FY2024. Its HBM orders now run 3 years past what it can build. And it keeps coming. Upward revisions every quarter from the datacentre buildout means more HBM. Yes I know SK stock taking a battering and memory trade is reversing but that’s a story of HBM4 ramp and competition, not a change in structural demand for HBM<br>But this is not the memory trade story. And we are not traders here. We are founders and VCs; here to create value! Build the future, not trade the present. So we ask: how do we build around HBM? Some answers have been around for a decade: Graphcore: SRAM? Groq - just do loads of SRAM?; Cerebras’s idea was to turn it up to 11 and build even bigger chips with even more SRAM. And now, the new upstarts with your in-memory computes, your analog in memory computes, your weights hard wired into silicon, or using commodity NAND to store weights. No idea is a bad idea until you hit the fab.<br>You can think about it like this: you have loads of money and join the queue for HBM or you find a way around HBM. Pick your fighter.<br>But, as SEMRON keep telling me, everyone already knows this, wen can you tell everyone that SEMRON solves this? Patience my sweet silicon kings. First, we must answer: why now? Not the lame: CMOS is running out of road. Or because photonics is better than copper. Or because von-neumann architecture is inefficient. These are lazy strawmans. I want one of those proper steelmen….
Subscribe
HBM drives AI scaling
We are going to start from the beginning. Because we must. Because if you skip to the end or ask Claude to give you the actionable insight, YOU DO NOT REALLY UNDERSTAND. Eat your broccoli. I’ll give you some olive oil and salt.<br>So, a processor with nothing to read from does no work, it draws power. So giving it stuff to read quickly is important. Every inference run reads the model’s weights out of memory, so feed the beast, the beast is hungry.<br>We are smart, we found answers. First, multicore. When single threads stopped speeding up. Do more cores. Then GPUs, when the parallel work outgrew the CPU. Then HBM, just stack memory on top of each other so you get feed memory to processor faster. Your DRAM Tower gets a 1,024-wire bus straight into it, and pull a 1 TB/s from a single stack. TSMC’s CoWoS packaging bolts several of those towers onto a silicon interposer around the processor, and it adds up: an H200 reads at 4.8 TB/s, a Blackwell B200 getting on for 8 TB/s. I can keep going…<br>HBM doesn’t solve the memory wall
But, did you notice something in the last paragraph? Or did you just skim read. Go back. “Stack memory”. This is a...