ZeroFS vs. Amazon S3 Files — ZeroFS

← Back to blog<br>ZeroFS vs. Amazon S3 Files

July 11, 2026·<br>Pierre Barre·<br>8 min read

Amazon S3 Files and ZeroFS expose POSIX filesystems backed by object storage, but the shared interface hides opposite bucket layouts. The choice turns on the role of the bucket: if files must remain ordinary S3 objects, S3 Files preserves that identity; if the bucket can be an internal persistence layer, ZeroFS trades direct S3 access for packing, compression, and client-side encryption.

Storage layout

The defining property of S3 Files, which is built using Amazon EFS, is that images/cat.jpg on the mount corresponds to the same key in the bucket, with changes flowing in both directions. Active data and metadata reside in a low-latency tier that AWS calls “high-performance storage.”

That one-file, one-object identity is deliberately absent from the ZeroFS storage format. Metadata lives in an LSM tree; file contents are split into extents, compressed and encrypted, then packed into immutable segment objects. An S3 client sees an opaque internal layout rather than the mounted files.

flowchart TB<br>subgraph CLIENTS["Client Layer"]<br>NFS["NFS Client"]<br>P9["9P Client"]<br>NBD["NBD Client"]<br>WEB["Web Browser"]<br>end<br>subgraph CORE["ZeroFS Core"]<br>NFSD["NFS Server"]<br>P9D["9P Server"]<br>NBDD["NBD Server"]<br>WEBUI["Web UI"]<br>VFS["Virtual Filesystem"]<br>SEG["Segment store: compressed, encrypted file-data frames"]<br>SLATE["LSM tree: metadata and 32-byte extent pointers"]<br>CACHE["Local Cache"]<br>NFSD --> VFS<br>P9D --> VFS<br>NBDD --> VFS<br>WEBUI --> VFS<br>VFS --> SEG<br>VFS --> SLATE<br>SEG --> CACHE<br>SLATE --> CACHE<br>end<br>subgraph BACKEND["Storage Backend"]<br>SEGOBJ["Immutable segment objects"]<br>SSTS["Metadata SSTs and manifest"]<br>S3["S3 Object Store"]<br>CACHE --> SEGOBJ<br>CACHE --> SSTS<br>SEGOBJ --> S3<br>SSTS --> S3<br>end<br>NFS --> NFSD<br>P9 --> P9D<br>NBD --> NBDD<br>WEB --> WEBUI

ZeroFS keeps file data and filesystem metadata on separate paths until both reach the object store.

Both mounts use the client page cache. The write-path row below starts after the client sends the write to the server.

Amazon S3 FilesZeroFS

Storage modelAWS-managed high-performance storage synchronized with a bucket; internal layout not documentedAn LSM tree and immutable data segments on object storage<br>Object layoutOne file maps to one S3 objectMetadata in an LSM tree; file-data frames packed into segments<br>Write path after client cacheNFS write to high-performance storage, durable immediately; asynchronous S3 export after write inactivityOver 9P, fsync uploads data segments and flushes LSM metadata to object storage before returning success<br>Cold read and read-aheadLinux NFS read-ahead, directory metadata import, optional small-file import, or direct S3 readLSM lookup, then adaptive object and cross-segment frame prefetch<br>S3 API access to filesYes; filesystem changes appear after asynchronous exportNo; reading files requires ZeroFS and the encryption password<br>Client interfaceNFS 4.1/4.29P or NFS for files; NBD for blocks<br>Object-store choiceAmazon S3Amazon S3, S3-compatible stores, Azure Blob, or Google Cloud Storage<br>Cost modelS3 plus high-performance storage, file access, and synchronization chargesObject storage and requests, plus the compute and cache running ZeroFS

Object interoperability

Keeping that one-to-one mapping means a file written through the mount eventually becomes a normal S3 object. Export begins after 60 seconds without a write, so continued writes postpone S3 visibility. Once export completes, existing tools can read the object with GetObject, and S3-side changes flow back into the filesystem. If both sides modify the same file before synchronization, S3 wins and the file-side version moves to lost+found.

A ZeroFS pathname cannot be fetched with the S3 API, and a segment cannot be scanned as Parquet. In exchange, small files need neither one data object nor one PUT each: their extents are compressed, encrypted, and packed together. The bucket and raw local cache contain ciphertext, so mounting or recovery requires ZeroFS and its encryption password. The encryption documentation lists the structural metadata that remains visible.

If other applications need immediate S3 visibility after a filesystem write, neither model provides it: S3 Files exports asynchronously, while ZeroFS never exposes mounted files through the S3 API.

Cold access

The first S3 Files access can trigger an import: listing a directory loads every object's metadata and asynchronously copies files below the import threshold, 128 KiB by default, into high-performance storage. AWS says a first listing of 1,000 objects may take several seconds. Larger files stay in S3, and reads of at least 1 MiB go directly to S3.

ZeroFS instead populates a local RAM and disk cache from reads and uploads. Newly sealed segments enter the cache from bytes already in hand, so read-after-write needs no GET; the cache is not write-back, and writes still reach object storage when segments are sealed and metadata...