To study how chips really work, MIT researchers built their own operating system | MIT CSAIL

Skip to main content

MIT CSAIL

Search

Back to News

May 19 '26

To study how chips really work, MIT researchers built their own operating system

Written By

Rachel Gordon

When security researchers want to understand what a modern processor is really doing with the kind of detail that determines whether attacks like Spectre and Meltdown are possible, they usually run their experiments on top of an operating system that was never built for the job. They open up macOS or Linux, patch the kernel by hand, and hope the modifications hold. The approach is unstable, hard to reproduce, and on Apple's platforms, slated for deprecation.<br>A team at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) decided to build something different. Fractal, a new operating system kernel written from the ground up, treats the hardware itself as the object of study. Its first major use, a deep look at the branch predictors (CPU's way of guessing what code to run next before it knows for certain), so it doesn't have to waste time waiting to find out) inside Apple's M1 processor, has already turned up findings that prior work missed, including the first evidence that a class of speculative attack known as “Phantom” affects Apple Silicon.<br>"We're using hardware in ways it wasn't designed for," says Joseph Ravichandran, the MIT PhD student who led the project. "It's not even obvious that this is a possible thing you could do with the hardware. But we found a way to pull all these different primitives off. It's like a microscope. If you've got a hand magnifying glass, you can see a little bit. But if you had an electron microscope, now we're really talking. That's what Fractal is. The electron microscope of operating systems."<br>A clean room for chip research<br>The core problem Fractal solves is one that researchers have worked around for years. Modern processors keep state in many internal structures: branch predictors, caches, translation lookaside buffers, and more. To study how those structures behave across the boundary between user code and kernel code, two domains the chip is supposed to keep isolated, researchers need to run nearly identical experiments on each side of that boundary. On a general-purpose operating system, that is very difficult. The system itself manages privilege levels, address spaces, and scheduling, and it injects its own activity into every measurement.<br>Fractal inverts the model. It boots directly on bare metal, with no other software running, and exposes primitives that let a single experiment switch privilege levels at runtime while executing the same instructions in the same address space. The team calls the underlying technique multi-privilege concurrency, and it relies on a new construct they introduced: the outer kernel thread, which sits inside a user process's memory but executes with kernel privileges.<br>The result is an experimental setup with almost no background noise. Where measurements taken under macOS or Linux are blurred by interrupts, scheduler activity, and address-space management, Fractal produces flat baselines and clean signals.<br>What Fractal found on the M1<br>Apple's M1 implements an ARM specification called CSV2, which is supposed to prevent code running in one privilege level from steering speculation in another. Using Fractal, the MIT team confirmed that the protection works for the execute stage of indirect branch prediction: a user-mode program cannot make the kernel speculatively execute a chosen target through the indirect branch predictor.<br>But the team also found something the chip's designers may not have intended. The CPU still fetches the target into the instruction cache before the protection kicks in. That fetch is observable through a side channel, which means user code can still influence what the kernel pulls into its caches across the privilege boundary. The same pattern appeared between processes assigned different address space identifiers.<br>The team also produced the first evidence that Apple Silicon exhibits Phantom speculation, a class of misprediction previously demonstrated only on AMD and Intel processors. In Phantom, ordinary instructions, including a no-op, can be misinterpreted by the CPU as branches, triggering speculative behavior the program never asked for. On the M1, Fractal showed that Phantom fetches succeed across both privilege levels and address spaces, though the execute phase remains blocked.<br>A separate Fractal experiment overturned a finding from earlier work on the M1's conditional branch predictor, which had reported that cross-privilege training worked on Apple's performance cores but not its efficiency cores. The Fractal team showed that the conditional branch predictor has no privilege isolation at all, on either core type, and that the earlier result was likely an artifact of macOS quietly migrating threads between cores during...