pqi: Making libpq a Choice, Not a Requirement – Functional programming debugs you
You are using an outdated browser. Please upgrade your browser to improve your experience.
Functional programming debugs you
Nikita Volkov
Consulting Software Architect
Consulting<br>Email<br>Twitter
Github<br>Stackoverflow
pqi: Making libpq a Choice, Not a Requirement
For years, every serious Haskell PostgreSQL driver has been chained to the same C library - libpq - with no clean way out. I tried to cut it loose twice, and both times I failed at the finish line. They say the third time’s the charm. I hope it will be - not because I’ve tried harder, but because I’ve changed the goal.
I’m Nikita Volkov, author of hasql. This is the story of that reframe, and of pqi, the library it produced.
2014: hasql, tied to C
hasql came out in 2014. It was fast, type-safe, SQL-first - and, like everything else in the ecosystem, it stood on postgresql-libpq, a binding to the C libpq library.
That dependency is invisible right up until it isn’t. You feel it when libpq isn’t on the build machine. You feel it in CI. You feel it in a minimal production container, when you try to cross-compile, or when you just want a static binary and the linker has other ideas.
Two attempts that stalled
I tried. Twice.
The first attempt was a pure-Haskell wire protocol written directly inside hasql. I got it working, but I couldn’t make it beat libpq on performance. The one place it clearly won was pipelining, because libpq had none at the time - and pipelining was the reason I’d started the chase in the first place. “Wins on one axis, loses on the rest” wasn’t enough to justify forcing every hasql user onto it, so I shelved it.
The second attempt, around 2021, went the other way: I led with optimisation. I wrote supporting C to bundle alongside the Haskell and pushed until I was outperforming libpq across a range of benchmarks. I was almost there.
Then two things happened. libpq shipped its own pipelining in version 14 - defeating my original motivation entirely. And I switched my own machine to ARM, where my recent performance gains against libpq started to become less pronounced. Once more, not satisfied enough to ship. Once more, shelved.
Both times I was chasing the same target: be faster than libpq. Both times the target moved, and I lost the race at the finish line. (The full story of those two attempts probably deserves a post of its own.)
The demand never went away
Shelving a project twice doesn’t shelve the need behind it. Questions about the fate of native Hasql persisted with a stable frequency over the years.
Eventually I realised I’d been answering the wrong question the whole time.
The reframe: libpq can be a choice
I had been treating this as a duel - my implementation versus libpq, winner takes the ecosystem. That framing is what kept killing the project. The moment you set out to replace libpq, you’ve signed up to beat it on every axis, forever, on every architecture.
But a driver doesn’t need to declare a winner. It can let the user declare one.
If swapping the C-backed implementation for a pure-Haskell one is a one-line change in your dependencies - and nothing in your driver code changes - then the performance question stops being existential. Need the battle-tested C path? Use it. Need a static binary with no libpq in sight? Swap the adapter at no cost to your codebase. The driver author writes against one interface and supports both, the user picks.
That’s pqi: a driver-agnostic interface to the libpq API, plus interchangeable adapters behind it.
How it works
pqi is an ecosystem of libraries:
pqi - the interface. It reproduces the postgresql-libpq API surface, but reifies the connection (and its results) as a type class rather than a concrete type. Code written against it runs unchanged on any adapter.
pqi-ffi - a thin adapter over postgresql-libpq. Battle-tested, production-safe, the default.
pqi-native - a pure-Haskell adapter that speaks the PostgreSQL frontend/backend wire protocol directly. No C. Experimental.
You choose the adapter by choosing the connection type. Everything else is written against the IsConnection constraint, so it doesn’t change:
import qualified Pqi<br>import qualified Pqi.Ffi<br>import qualified Pqi.Native
-- C-backed (battle-tested, requires libpq)<br>connection Pqi.connectdb conninfo :: IO Pqi.Ffi.Connection
-- Pure Haskell (experimental, no C dependency)<br>connection Pqi.connectdb conninfo :: IO Pqi.Native.Connection
Same code, same API, different transport.
How LLMs changed the equation
Reframing the goal solved the strategy, not the work. A correct PostgreSQL wire-protocol implementation takes a lot of sweat - exactly the kind of thing I’d burned out on twice. A complete implementation for the libpq API surface is even scarier.
My final realisation happened after getting used to LLMs and studying their patterns. You see, LLMs operate the better the clearer you define the...