SteadQ - the filesystem is a message queue — DAVID TORCIVIA

Skip to content

SteadQ - the filesystem is a message queue

19 August 2026<br>· 19 min read

I've spent the last few weeks obsessing about an idea that is either very old Unix wisdom, or a symptom of spending too much time around software: files should be true .

By "true" I mean authoritative . Not a serialization of some state that actually lives in a database, not an export from someone's pet application, or any other kind of convenient human-readable shadow. The files themselves should be the thing.

If some program disappears, if I delete a bunch of databases and come back to it in five years with nothing but my trusty shell, I should still be able to get a good sense of what's going on.

This started, innocently enough, with me tinkering around with this website. Every dev writes a static site gen at some point and it was my time. I rewrote the entire app in Go (a program with the uninspired name of dtcom), with the obvious idea that all of the posts should be in Markdown, the site configuration can be a simple YAML file, and then some HTML templating will stitch it all together.

But it's 2026 and just because something is simple and authoritative doesn't mean it can't have nice modern features. I use a SQLite database, because some things (such as admin backend, search, basic analytics, etc) are just plain easier in a relational database. The crucial difference is that these SQLite databases don't hold any authoritative state. The site continues to work if you delete the database files, and even recreates it if you remove all generated HTML - everything can be built from those authoritative markdown (and YAML) files.

When it comes to actually editing posts, every path edits Markdown files. Log into the administrative interface and start writing, use the REST API or MCP server (why not let agents interact directly), open the relevant file up in vim and start hacking away on it - every path leads to the same underlying set of files.

There's something deeply satisfying about having only one version of truth in a piece of software, and not trying to keep multiple data stores coherent.

At the same time I have been working on a far more elaborate project called VOT (Verified Object Transport - a way of transferring immutable objects, and having full assurance that what you receive is really, absolutely the same object) which takes this idea further. In particular, objects don't really change: rather you create a new object with some relation to an old one (such as "this is version 2 of it"), identify this new object cryptographically, and publish the result somewhere where you know that nobody can overwrite what's already there (while producing evidence as to how everything relates back).

After staring at these two projects, I started to wonder just how far this idea goes. In particular - could the file system itself be used as a work queue?

Note that I'm not talking about storing payloads for jobs in files and then keeping pointers to them (along with state) somewhere else like a database. The idea is simpler and far more stupid:

Could we build an entirely broker-less queue out of just files?

Files would represent jobs ready to be worked, and others could try to take ownership by renaming them atomically. If a job completes successfully, you'd create some kind of durable record (such as an acknowledgement), or if it fails then you maybe retry later, up to a certain number of attempts. If something dies while working on the job (such as power being cut), then you'd like to know about it so someone else can retry the job later.

This has been solved many, many times already, with very good software implementations. Whether it's Redis (which I love and use almost everywhere), RabbitMQ, SQS or Postgres, there are a lot of choices here.

This approach has precursors on the file side too; unsurprisingly, it closely resembles Dan Bernstein's maildir format. A message is a file - the application creates it in the maildir's tmp/ directory and then atomically renames it into new/. That rename is the commit point, after which the message has been delivered exactly once; no locking or dedicated daemon required. It was safe by the standards of its time. CERN's dirq library and other simple tools like nq extended this pattern to general-purpose job directories. But they're limited; maildir only solves the problem of delivering a message once and nothing more. None of them address what should happen when things go wrong: if the process working on that file dies, how can you tell? How many times has the job been attempted already? Was it ever actually finished? SteadQ takes that single atomic rename and tries to drive a full-fledged job queue off of it, which is where all the generation counters and leases and durability barriers come from.

Because on any single machine we already have something that coordinates access to files by multiple processes,...