We Rebuilt the Linux MicroVM Stack on Apple Silicon

signa111 pts0 comments

We rebuilt the Linux microVM stack on Apple Silicon – Encore Blog

← All articles<br>We rebuilt the Linux microVM stack on Apple Silicon<br>What it takes to boot the same microVMs on Apple's hypervisor, and the one capability Apple will not let you have.

Aug 18, 2026<br>15 Min Read<br>Ivan Cernja

← All articles<br>Aug 18, 2026<br>We rebuilt the Linux microVM stack on Apple Silicon<br>What it takes to boot the same microVMs on Apple's hypervisor, and the one capability Apple will not let you have.

Ivan Cernja<br>15 Min Read

Encore builds and deploys backend applications, and since mid-2022 every one of those builds has run inside a Firecracker microVM. Firecracker strips the emulated hardware down to what a Linux kernel needs, giving each build the isolation of a virtual machine with startup close to the cost of a container.

Firecracker drives KVM, so it needs a Linux host with /dev/kvm, which no Mac has, and most engineers at Encore develop on a Mac. The maintainers have no plans to close that, given they turned down a working proof of concept built on Apple's Virtualization.framework and said they do not plan to support macOS any time soon.

So for four years, working on the build system meant working on it somewhere else. We wanted to run the same build system on our laptops while keeping Firecracker in production, so we built crackling , a single microVM API that drives Firecracker on Linux and Apple's hypervisor on macOS; booting the same images on both required rebuilding much of the Linux image toolchain to run on macOS.

Four years of developing on a shared remote machine

We onboarded each engineer with a script you ran once. It SSHed into the shared build machine as root, pulled your public key from https://github.com/.keys and created you a user, then added you to the kvm and docker groups so you could reach the hypervisor and run containers. It copied the VM images into your ~/images and hard-linked the firecracker binary into your ~/binaries, since every user needed it under their own tree on the one box. You ended up with a personal environment in a datacentre, reachable over Tailscale, sitting next to everybody else's.

Getting a change onto that environment took a second script, which read your username and your port out of the CUE config, from a gitignored per-engineer file, because we all shared that host and had to agree not to collide. Binaries were the easy half: we cross-compiled with GOOS=linux GOARCH=amd64, rsynced the results across, and counted the transferred files to work out whether anything needed restarting.

Images were the hard half, because Firecracker boots a block device and Docker produces layers. We could not find an existing tool that converted Docker layers into a block device Firecracker could boot, so we built the conversion ourselves, half on your laptop and half over SSH:

# tools/dev-builder/deploy-dev-builder.sh<br>docker save -o "$imagesdir/$name.tar" "$docker_image"<br>tar -C "$layersdir" -xf "$imagesdir/$name.tar" # explode the layers<br>tar -C "$dst/" -xf "$imagesdir/$name.tar" manifest.json

rsync -azP $layersdir ${username}@builder:~/images/<br>rsync -azP "$dst" ${username}@builder:~/images/

ssh ${username}@builder -- \<br>"bash -l -s squash_layers \"images/${outputdir}\" \"images/${name}\"" $scriptpath

That last line pipes a shell function into a login shell on the far end and runs it there. The bash was later rewritten in Go, but the pipeline and the host did not change. We had squash_layers re-extract every layer in manifest order, delete the .wh..wh..opq whiteout markers with find because tar will not apply them for us, write a hardcoded /etc/resolv.conf since the VM had no DNS otherwise, pull the image's environment variables out of the Docker config with jq, and finally call mksquashfs to produce something Firecracker could boot. We keyed a cache on the Docker image id so the whole path could be skipped when it matched. It ran whenever you had changed anything in the image, which was most of the time if you were working on the guest side.

Restarting took a third SSH, to kill your container and start its replacement:

docker run --privileged \<br>-v ~/socks:/var/lib/buildsvc/socks:rw -v ~/logs:/tmp/encore-builds:rw \<br>-v ~/.keys:/.keys:ro -v ~/binaries:/usr/local/bin:ro \<br>-v ~/images:/usr/lib/buildsvc/images:ro \<br>--env-file service-envs \<br>--device /dev/kvm --device /dev/net/tun \<br>-p $port:9060 --name "${username}-builder" -d -t buildsvc-tester

Firecracker is running inside a Docker container there, so we had to pass --privileged, /dev/kvm and /dev/net/tun through, because the process inside that container was going to create tap devices and boot virtual machines of its own.

Firecracker expects those tap devices to attach to a host bridge, and inside a Docker container there is no host bridge. So we built one, in a shell script that ran before buildsvc, the build service itself, came up:

# tools/dev-builder/container/start.sh<br>ip link add docker0 type bridge<br>ip link set eth0 master...

firecracker apple images linux docker builder

Related Articles