Why I reimplemented LVM (with worse guarantees)<br>For AI agents: a site index is available at /llms.txt. Append .md to any documentation, blog, changelog, or customer URL on this site to fetch its markdown source.

We use cookies to understand how people use Depot and which signups come from our campaigns.<br>Reject cookiesAccept cookies

⚡️ Announcing Depot Metal<br>⚡️ Announcing Depot Metal<br>Depot

Sign InGet started

Menu

Blog > engineering<br>Why I reimplemented LVM (with worse guarantees)<br>Written by<br>Héja Péter (Vau)

Published on<br>13 August 2026

Share<br>Share on TwitterShare on LinkedInShare on Hacker News

Stay in the loop<br>Get notified when we ship new posts.<br>Subscribe

Wait, you rewrote LVM, and made it worse?!

This is the question I heard in my head when I started reimplementing LVM, which has been rock-solid for two decades. But I couldn't use vanilla LVM2 for our hypervisors running microVMs: it simply does too much and assumes too little about our workload.

Running microVMs is weird

The whole story starts with us moving towards microVMs on bare-metal hypervisors to run the Sandboxes that host our customer workloads. This means we are running expensive bare-metal machines, where we should not waste a single second between customer workloads - so we went down to sub-second microVM launches.

Doing this sub-second on local SSD is already a nice challenge, but we do this with networked storage with pretty good performance, so we had to (re)invent a couple of tools for ourselves.

Note : This move also means that we are auto-scaling expensive bare-metal machines, so every second we spend on booting our hypervisors and getting them prepared to launch microVMs counts.

What is LVM even doing?

What we love about LVM is that it is super simple and, because the data-plane is the Linux kernel's device-mapper infrastructure, it uses as little magic as possible with good performance.

However, LVM is meant to be rather safe and give guarantees, such as your data not vanishing if you crash while creating and deleting volumes. Even more, if you accidentally delete, you have a really good chance of recovering it!

This is at the cost of global locking (well, volume group-wide), which is acceptable at one operation a week. But an operation holding the VG-wide lock for roughly 100ms on an operation you want to do up to 200 times a second in parallel is hardly doable. It makes you wonder if you are even using the right tool.

LVM guarantees vs guarantees we actually need (and want to afford)

Locking and data-safety are really good features, but if your workload fails performance targets because of it, you cannot use the solution. This was the case for us with LVM.

One place where we really want a fast solution is assembling block devices to be consumed by our hypervisor to launch microVMs. Basically, it has to take network block devices from a remote host, add dynamically sized caches on top of them, and produce a single block device.

Zooming into this post's most important bit: we need a very quick way to allocate variable amounts of RAM from a big pool, then create a cached block device via device-mapper.

Crash safety does not give us a lot in this specific scenario. A hypervisor crash results in both the failure of an ephemeral microVM workload (therefore it is discarded) and the loss of volatile memory (which results in data loss anyway), so there is either no value or no data to protect from crashes.

We need pretty good performance and high reliability, so we use the same device-mapper infrastructure that is behind LVM. We just assemble it differently.

What the solution looks like

As the first layer, we have networked storage, that provides us various block devices. This is currently out of scope for this post, but we have our own storage agent and storage daemon which materialize variable sized block devices.

The second layer is the memory block device, which the Linux kernel provides. There are multiple mechanisms and patterns on how to achieve this, but the end result is essentially the same: a pool of memory, managed by our storage agent.

Once we have this memory pool, an in-process allocator in the storage agent manages it. The allocator holds the mappings, the slices, and everything else in memory, and can handle allocations of custom sizes and shapes.

Because this critical path is in-process, and we don't have to open a disk or do any "real" IO operation, the allocation is incredibly fast. This allocation is the only point where we need any kind of locking, and it is only for an in-memory mapping.

Since the mapping ensures that there is no race on using the memory block device, we can enjoy the kernel's asynchronous device-mapper creations, resulting in roughly 100x speedup compared to using LVM for the same use case.

Even though the first prototype still used dmsetup, it was already fast enough for our boot-time target. It just wasn't exactly elegant.

What if storage agents crash?

This is the cool...