A modern Proxmox Docker architecture with disposable VMs, VirtIO-FS, and ZFS state separation - du.nkel.dev

du.nkel.dev

Architecture Overview

Creating a qcow2 VM template

Create the Tooling VM

Customize the VM Image

Create the template, destroy the tooling VM

Persistent vs. ephemeral data

VirtIO-FS and ID-Mapping

VFS ID-Mapping with /etc/fstab

Mounting to the VM

Boot Conditions

Rootless Docker Service Isolation

VM configuration with ephemeral Docker disk

Creating Service Accounts (Rootful vs. Rootless)

Configuring Rootless Namespaces

Automation, Reverse Proxy, and Organization

Git-Based Configuration Tracking

Nginx VM Setup and SSL

Automated Updates via Cron

Conclusions

A modern Proxmox Docker architecture with disposable VMs, VirtIO-FS, and ZFS state separation

Published: 2026-05-18<br>Revised: 2026-06-07

TL;DR

The Linux kernel's security model is constantly evolving. In 2026, my Docker-in-LXC nesting became increasingly fragile and needed a replacement. Here I describe a state of the art architecture for Proxmox. The post outlines deploying lightweight VMs via cloud-init linked clones, isolating services in rootless Docker namespaces, and using VirtIO-FS with native VFS idmapped mounts for ressource efficient ZFS storage passthrough.

Motivation<br>Back in 2021, I documented my approach to running Docker inside unprivileged LXC containers on Proxmox. At that time, my hypervisor was quite resource-restricted (a 2013 Xeon with 32GB of memory). This made unprivileged LXC the obvious choice. It worked well for years (and it continues to work, largely - read on).

The Linux kernel has moved on. With the adoption of cgroup v2, stricter AppArmor profiles, and tighter UserNS restrictions, running Docker inside LXC slowly turns into a battle against the kernel's security model.

In 2023, I wrote about a successor system architecture for running Mastodon in rootless Docker behind an Nginx proxy. I have used this setup extensively for cloud VMs and at work. The system concept utilizes the improved isolation of VMs while preserving resource efficiency of isolating individual services within unprivileged, rootless Linux namespaces.

The logic behind is that not every 20MB application requires the memory overhead of its own dedicated VM. Conversely, mixing unrelated services (like Nextcloud and Immich) into the same Docker daemon is a security anti-pattern. If a dependency chain attack compromises a library in your photo indexing app, that attacker should not also gain read/write access to your entire Nextcloud document repository. Enforcing the principle of separation of concerns becomes even more critical today because of the rise of dependency chain attacks.

What those previous posts did not cover is how to use this new architecture with the hypervisor's storage layer.

If your storage sits directly on the Proxmox Hypervisor (e.g. attached as a JBOD), there is a need for a clean way to share specific paths into isolated VMs. In a homelab, VirtIO-FS bind-mounts are preferable to NFS sharing. NFS is often slow and difficult to configure with user and permission mapping. Imagine, for example, one system needs read/write access (e.g., Nextcloud handling automatic photo uploads from client phones), while another system strictly needs read-only access (e.g., Immich indexing those same photos), all with custom uid- and guid-mapping for each services, - this is the sweet spot I am exploring with this blog post.

This guide outlines the architecture concept from the ZFS and Hypervisor configuration, up to the application layer. To understand this setup, a grasp of persistent versus ephemeral data is good to have, along with an understanding of how to layer down these components from the hypervisor into a nested rootless Docker environment. If you don't know what I am talking about, you may still find individual parts interesting for copying or best practice. Pick what is relevant to you.

Specifically, I address the following documentation and concept gaps:

Ephemeral storage: ZFS snapshots can easily grow in size when used with Docker. I show how to separate the persistent data from ephemeral, disposable Docker OS Image content found in ~/.local/share/docker. This separation keeps backups small.

VFS ID-mapping: Network file systems can cause UID/GID mismatches and add network overhead. With VirtIO-FS, I use the Linux kernel's Virtual File System to translate the hypervisor's UID to the guest's unprivileged UID. This avoids exposing the host file structure. I utilize the X-mount.idmap fstab option for this. Documentation on this specific implementation is not easy to find. It builds upon the idmapped mounts feature introduced by Christian Brauner in Linux 5.12 1 and its later integration with util-linux v2.39 into the standard mount utility 2.

Linked clones: I initially considered template creation unnecessary. A manual VM installation only takes about 30 minutes. However, treating VMs as disposable...