Biff 2.0 sneak peak<br>HomeDocsNewsGitHub
Biff 2.0 sneak peak<br>Jacob O'Bryant | 20 Apr 2026
I have for the past year or two been working on some large Biff changes, such as those discussed in<br>Structuring large Clojure codebases with Biff<br>and Biff support for XTDB v2 is in pre-release. Now that<br>coding agents have gone mainstream (and in particular, now that I personally have started using them<br>heavily), I've had a few more ideas for changes I'd like to make to Biff. And also thanks to coding<br>agents, I've actually been able to make consistent progress instead of my Biff development time<br>being bottlenecked by how late I can stay awake on weekend nights after my kids are sleeping. So we,<br>fingers crossed, are getting close to some major Biff updates, and I figure I may as well slap a 2.0<br>label on it.
Here's what I've got in the works.
SQLite will be the default database
This is the biggest change. Biff will retain first-class support for XTDB, but it'll also have<br>first-class support for SQLite, and I'll update the starter project to use SQLite by default. There<br>will still be a (non-default) starter project that uses XTDB.
Biff has used XTDB since its (Biff's) initial release in 2020, back when the database was still<br>called Crux. About a year ago I started working on migrating Biff from XTDB v1 to XTDB v2, which<br>brings a whole new architecture, including column-oriented indexes that make analytical queries<br>faster. Besides writing some Biff-specific helper code for XTDB v2, I migrated<br>Yakread (a 10k-LOC article recommender system) to v2 and did a bunch of<br>benchmarking for Yakread's queries. (A big thank you to the XTDB team who responded to lots of my<br>questions during this time and also made a bunch of query optimizations!)
Long-story short: despite the optimizations, I had trouble getting Yakread's page load times to be<br>as quick as I wanted. For the particular queries Yakread runs—which are mostly row-oriented—I've<br>generally found v2's performance to be slower than v1. There is also a larger per-query latency<br>overhead, perhaps another design tradeoff of the new architecture (you can still run v2 as an<br>embedded node within your application process, but it’s designed primarily to be run on a separate<br>machine like more traditional networked databases).
I also will admit that before this benchmarking exercise I had not actually used SQLite much, and I<br>was unaware of how ridiculously fast it is. And one of the main downsides of SQLite when compared<br>to XTDB—that SQLite is a mutable database—is mitigated by Litestream, which streams<br>changes to object storage and lets you restore from (and even run ad-hoc<br>queries on) historical snapshots saved with 30-second<br>granularity.
I could see myself switching back to XTDB at some point in the future. It's still the early days for<br>v2 and the XTDB team is doing lots of work, including on query performance. And SQLite's speed comes<br>with tradeoffs:
Scaling beyond one machine is an unsolved problem. Litefs can let you put SQLite nodes in a<br>cluster where writes get forwarded to a single leader and changes are streamed to the other nodes.<br>However, to use it with Litestream, you have to disable automatic leader<br>failover. So you basically<br>have to choose between HA or PITR.
SQLite only supports a few basic datatypes: ints, floats, strings, and blobs (byte arrays). A<br>large part of my work in integrating SQLite into Biff has been to set up automatic data type<br>coercion so you can use richer types (UUID, boolean, instant, enum, map/set/vector) in your schema<br>without having to do manual coercion when reading and writing.
Litestream's snapshots-at-30-second-granularity is fine for recovering from bad transactions like<br>a DELETE FROM without the WHERE, but it's less helpful than XTDB/Datomic for the<br>debugging-weird-production-issues use case: you can't include a transaction ID or similar in your<br>production logs and then re-run queries with 100% confidence that the results you're seeing are<br>what the application saw when it e.g. threw an unexpected exception.
I was chatting with Jeremy from the XTDB team last week, and he mentioned they've been working on<br>having XTDB ingest changes directly from Postgres. It sounds like it shouldn't be much work to make<br>that work with SQLite too, which means that you could stick an XTDB node alongside your<br>SQLite-powered Biff app and then get more granular historical queries. Maybe XTDB could be a<br>replacement for Litestream?
That could get even more interesting if eventually we can do the inverse as well, where data from<br>our immutable XTDB log could be sent both to a bitemporal index for historical queries and also to<br>SQLite "indexes"/databases for the application servers to use. That would solve the HA problem too.
Anyway. However it happens, I'm looking forward to the glorious future when we finally have an<br>inside-out database<br>that's fast for all query shapes, highly available, models time correctly, and can even do advanced<br>things like let you put a...