Cache-Conscious Data Layout in Rust: Field Zoning, False Sharing, and the 128-Byte Rule | Ruminations of a Programmer

Cache-Conscious Data Layout in Rust: Field Zoning, False Sharing, and the 128-Byte Rule

Cache-Conscious Data Layout: Field Zoning, False Sharing, and the 128-Byte Rule

Part 1 of Low-Level Systems Design in Rust - a series on writing high-throughput, low-latency systems code, using a single-producer / single-consumer (SPSC) ring buffer as the running example.

Part 0 - Architectural Decomposition made the highest-leverage decision (remove contention structurally, so every writer gets its own ring) and holds the full series index and guiding principles.

This post starts the micro-level work: given one such SPSC ring, how should it be laid out in memory? The patterns aren't specific to ring buffers - they apply to any structure more than one core touches.

The thing nobody tells you about multi-threaded structs

When you write a single-threaded data structure, the only layout question that matters is "does it fit in cache." When you write a multi-threaded one, a second, relevant and a bit subtle question appears:

For each field, which core touches it, and how often?

Getting this wrong will make you pay in a way that no profiler flags as a single hot line. Two cores writing to different fields that happen to share a 64-byte cache line will quietly serialize each other through the hardware coherence protocol - a pathology called false sharing . The code looks lock-free. The benchmark says otherwise.

This post is about designing the layout deliberately. It comes in two parts:

Field zoning - grouping fields by (write owner, frequency) so that one core's hot writes don't evict another core's hot working set.

Alignment and padding - the mechanics that make zoning real: why #[repr(C)] is load-bearing, why the magic number is often 128 and not 64, and why adding prefetch hints can make things worse.

The running example is a single-producer / single-consumer (SPSC) ring buffer. One core (the producer) appends, another core (the consumer) drains. They coordinate through two monotonic cursors - tail (where the producer writes next) and head (where the consumer reads next).

Though the following discussion and the design principles are general enough, you can refer to this ring buffer implementation as a reference point that honors these guiding principles.

Part 1 - Field zoning: design based on "who touches what"

The cache line - commonly 64 bytes on x86-64 and many AArch64 cores - is the unit of currency for inter-core communication. Coherence traffic is accounted per line, not per byte or per field. So the first design move for any shared structure is to sort its fields into zones by access pattern:

ZoneFieldsWrite ownerCross-core access<br>Producer-hottail, cached_headproducerconsumer samples tail<br>Consumer-hothead, cached_tailconsumerproducer samples head<br>Coldclosed, config, metricslifecycle/observabilityrare, not hot-path polling

"Producer-hot" does not mean "producer-only." The producer owns writes to tail, but the consumer still reads tail when its cached view runs dry. The important distinction is write ownership : the producer is the core that keeps making the line exclusive, so the fields near that write must be chosen deliberately.

First, the vocabulary, since the rest of the post leans on it. The hot path is the code that runs on (almost) every operation - here, the send and receive loops that each core executes millions of times a second. Hot fields are the ones those loops touch, like tail, head, and the two cursor caches. The cold path , by contrast, is everything that runs rarely - construction, shutdown, an occasional metrics read - and cold fields are the ones only the cold path touches, like config and metrics. "Hot" and "cold" are about frequency of access, and they're what the zones in the table above are sorted by.

The rule is then simple to state: place each zone on its own set of cache lines. A hot field written by one core must not share a line with hot fields another core reads or writes frequently - a producer store to tail should not invalidate the line holding the consumer's head or cached_tail, and vice versa. Cold fields, since nobody contends over them on the hot path, can stay packed together at the back.

Here is a ring buffer that encodes those zones directly in its declaration. The type parameter A is just a pluggable allocator for the backing buffer, ignore it if you like - what matters is the order and grouping of the fields:

#[repr(C)]<br>pub struct RingT, A: BufferAllocator = HeapAllocator> {<br>// PRODUCER HOT<br>tail: CacheAlignedAtomicU64>,<br>cached_head: CacheAlignedUnsafeCellu64>>,

// CONSUMER HOT<br>head: CacheAlignedAtomicU64>,<br>cached_tail: CacheAlignedUnsafeCellu64>>,

// COLD<br>closed: AtomicBool,<br>metrics: Metrics,<br>config: Config,<br>buffer: UnsafeCellA::BufferT>>,<br>A few things are worth reading carefully here, because each one is a deliberate choice rather...