Pony's Arena Allocator - Pony

Skip to content

Initializing search

GitHub

Use

Documentation Generation

LLM Skills

Development

Debugging

Performance

Working with the Compiler

Cross-Compilation

Ecosystem

Build and Release Tools

Contribute

Workflow

Working with the Compiler

Project Operations

Infrastructure

Pony Development Sync

Last Week in Pony

Resources

Community

Events

Stay Informed

Blog

Categories

FAQ

What’s next

Pony's Arena Allocator

Here’s how it was. We’d been running stress tests on the TCP system in ponyc every day for a really long time. Those tests hadn’t failed in forever. There’s two ways you can look at “no tests are failing.” Either your system is rock solid or your tests aren’t covering enough of the codebase. I’d long felt that the lack of stress test failures was the result of “tests aren’t good enough.”

A few months back, I did something about that. I took the TCP stress tests that did the same thing on every run and added some “swarm testing goodness” — randomizing what they were doing and how they were being run, from run to run. A lot more code was getting exercised, and out popped a bunch of bugs.

This post is about one of those “bugs.”

Under certain usage patterns, memory grew without bound. A little investigation showed that it wasn’t a bug in the TCP code itself. The test was triggering an interesting collection of edge cases in the Pony runtime’s allocator.

There were three reasons the old allocator could hold on to memory in a way that could lead to the degenerate case I was seeing with the failing stress test instance.

In Pony, a message is allocated by its sender and freed by its receiver. About half of all frees land on a different thread than the one that did the allocation. A large block freed on the wrong thread stayed reserved, never reclaimed. A workload that passed blocks between threads reserved fresh address space for every block it allocated and freed, without bound. At ten thousand blocks, it was still climbing, about 4.3 MiB of address space per block.

Once memory was carved into 32-byte slots, it held 32-byte objects forever. The allocator never returned it for a different size.

Two adjacent free blocks were never merged. The old allocator kept a sorted free list, and mixed-size workloads slowed to a crawl walking it. 0.49 seconds to allocate at 4,000 blocks. 39.6 seconds at 16,000. 64,000 blocks did not finish in nine minutes.

All three came from the same place: the old allocator used a global pool with per-size-class free lists shared by every thread. No thread tracked which memory it allocated.

The fix for all three: make each thread track the memory it allocates — what’s in use, what’s free, when a region is empty. That tracking is what the old allocator lacked. With it, the allocator can merge freed memory, return empty regions to the OS, and reuse memory across threads.

The New Allocator¶

I built a new allocator, inspired by snmalloc’s region-based design. Memory is organized in two tiers: large regions requested from the OS, split into smaller arenas. Each arena belongs to one thread. That thread is responsible for all the bookkeeping in its arenas — tracking what’s allocated, what’s free, and when memory can go back to the OS. Cross-thread frees are batched and routed to the owning thread rather than handled globally.

Regions and arenas¶

The allocator requests memory from the OS in large aligned chunks called regions — 256 MiB on 64-bit machines, 64 MiB on 32-bit. Regions are shared by every thread and never unmapped. By not unmapping regions, a thread walking the region list has a memory safety guarantee that the rest of the design depends on.

Threads take arenas from regions. An arena is 8 MiB on 64-bit, 2 MiB on 32-bit, and each is bound to a thread. The owning thread is responsible for maintaining the arena’s bookkeeping. When an arena empties, its physical pages go back to the OS, but its address space stays parked in the region for reuse.

Freeing memory starts with finding which arena it belongs to. An arena’s starting address is aligned to its own size. On 64-bit, every arena starts on an 8 MiB boundary. Mask the low bits of any pointer and you get the arena’s base address. One instruction. No memory read. Nice and fast. We like nice and fast over here in Ponyland.

Units, slabs, and the bitmap¶

An arena is divided into 16 KiB units. A slab is one or more contiguous units serving one size class. There are 16 size classes, from 32 bytes up to 1 MiB, each a power of two. For small classes, multiple objects fit into a single unit — a 32-byte class fits 512 objects per unit. Large classes span multiple units.

Free or used is one bit per unit in a bitmap. An 8 MiB arena has 512 units, which fits in 8 bitmap words on a 64-bit machine.

Two...