Managing File Descriptors for Durability and Performance<br>Managing File Descriptors for Durability and Performance<br>Jul 26, 2026

In the previous article we described the block-based layout behind ReductStore’s storage engine: append-only blocks, descriptors, and a WAL for crash recovery. In this article, we move to a lower level and examine how ReductStore manages file descriptors to balance durability, write performance, and remote-storage costs.

The cost of durability

The textbook lifecycle of a file is the same across operating systems and languages:

open → write → close<br>This works for the vast majority of applications, but not for databases. When close() returns, the kernel has accepted the data into its page cache, but nothing guarantees the data has reached persistent storage. A power failure at this point can lose everything written since the last flush.

The standard fix is to call fsync (or fdatasync) before closing:

open → write → fsync → close<br>fsync asks the kernel to write modified file data and the required metadata to the storage device, then waits for the operation to complete. Subject to the guarantees of the filesystem and storage device, a successful fsync makes the file contents durable across a power loss.

The problem is cost. fsync can be an expensive system call because it stalls the calling thread until the storage device confirms the write. This can take from tens of microseconds on a fast NVMe drive to tens of milliseconds on a spinning disk or network-attached volume. Percona published a detailed benchmark that shows how much the results vary across devices.

For a storage engine that writes many small records per second, calling fsync after every write would significantly reduce throughput. But skipping it risks data loss after a host failure. This tension between durability and write throughput is one of the two problems that shape how we manage file descriptors in ReductStore.

When close means upload

The second problem appears when ReductStore runs on S3-compatible object storage mounted as a local filesystem through a FUSE driver, such as s3fs, goofys, or mountpoint-s3.

FUSE drivers emulate filesystem operations on top of an object store. They let applications use familiar operations such as open, write, and read, but their behavior is not identical to that of a local filesystem. Many object-storage FUSE drivers finalize or upload a modified object when its write handle is flushed or closed. The exact behavior depends on the driver: it may upload the complete file, use a multipart upload, or maintain a local cache before sending data to remote storage.

For a storage engine, this can create a serious problem. Closing a sealed block may reasonably finalize one remote object. Closing an active block between writes, however, may repeatedly upload its current contents. Reopening it later may also require another download before the next update. The amount of extra work depends on the driver’s caching and upload strategy.

The cost scales with block size and write frequency. With a driver that uploads complete modified files, repeatedly closing a preallocated 64 MB block could transfer that block many times before it is sealed. This is too expensive for a write-heavy workload.

This means file descriptor lifetime directly affects both network cost and write latency when running on FUSE-mounted storage. We cannot treat close() as a cheap cleanup operation.

These two problems are related but distinct. On a local filesystem, close() does not guarantee durability; the engine must synchronize data explicitly. On some FUSE-mounted object stores, closing or flushing a modified file may trigger expensive remote work. Descriptor lifetime and synchronization policy must therefore be managed together, without leaking descriptors or exhausting system resources.

Caching file descriptors

The first issue to address is descriptor lifetime. Instead of closing a block file after each operation, we keep its descriptor open and reuse it across reads and writes. However, keeping every block file open indefinitely is not practical:

The operating system limits the number of open file descriptors per process. On many Linux systems the default is 1024. The limit is tunable, but it is never infinite, and a storage engine managing thousands of blocks can easily exceed it.

Some filesystem operations require the file descriptor to be closed first. You cannot delete or rename an open file on every platform.

Writers, readers, and compaction tasks all need access to descriptors, so the program needs a shared data structure for them.

Since we already need a shared structure, we can make it a cache with two eviction rules:

Capacity limit: when the number of open descriptors exceeds a configured maximum, evict the least-recently-used entry.

Idle timeout: mark a descriptor for eviction when it has not been accessed within a configurable period.

Every access refreshes the entry’s timestamp, so actively...