The arena has to outlive the process

The arena has to outlive the process

July 2026 · the growordie team · ops & state

For the first weeks of growordie, every time I shipped a fix I committed a tiny act of vandalism. The deploy would kill the server, and with it the entire arena: everyone mid-run got booted, and worse, their run never passed through the death screen, so the score they'd fought for was never even saved. I was punishing my most engaged players for the crime of being online while I was improving the game. This is how I stopped doing that, and why the fix is a boring one built out of parts I already had.

The short version: the live state of the world now survives the code I deploy on top of it. It takes a quick trip through Postgres between the old process and the new one, and your browser quietly reconnects and picks its snake back up. If you want the wider tour of the server this all lives inside, there's a write-up on the whole architecture. This one zooms in on the deploy.

Live capture a real deploy, nothing staged

An actual deploy, recorded end to end: the snake is weaving along, then a SIGTERM freezes the arena and the "Updating the arena" banner drops in, then the new process rehydrates the snapshot and the browser reconnects onto the same snake and keeps going. Same run, same length, a two-second gel where a death screen used to be.

01 The problem: one machine, all of it in RAM

02 The insight: state has to outlive the code

03 The sequence, step by step

04 The reconnect: rebind, don't respawn

05 The honest asterisks

06 What real zero-downtime looks like

01 · One machine, all of it in RAM

growordie is an authoritative server, which is the honest way to build a game where people can lie to you. Your browser doesn't decide anything; it just sends "I want to turn left," and the server runs the real simulation 30 times a second and streams the world back. I like this design. But it has a consequence that took me a while to feel in my gut: the truth of the game, every snake's position, length, body, kills, all of it, lives in plain JavaScript objects, in the RAM of exactly one process, on exactly one small machine. There is no second copy. The Map of snakes is the world.

That's completely fine while the process is alive. It's a catastrophe the instant the process dies. And a deploy is me killing the process on purpose. Fly sends the old machine a signal, the machine stops, that Map evaporates, and everyone connected to it is standing on a floor that just vanished. Two separate things break at once, and for a while I only noticed the loud one.

The loud break is obvious: the arena resets. Everyone's snake is gone, the socket drops, you're staring at a dead screen. Annoying, but you could imagine shrugging it off. The quiet break is the one that actually made me feel bad. A run's score only gets written to the leaderboard when the snake dies, through the normal death path. A deploy doesn't kill your snake, it kills the whole server out from under it, so your run never reaches that code at all. You didn't die. You were disappeared. Your 60-meter run, the best one you'd had all day, simply never happened as far as the database was concerned.

wiped<br>every live snake, on each deploy

unbanked<br>runs never hit the death path

machine holding the entire world

02 · State has to outlive the code

Once I framed it as "the world dies when the process dies," the shape of the answer was almost embarrassing. The world can't be allowed to depend on the process staying alive. It has to be able to step outside the process I'm about to kill, wait for the new one, and step back in. The code is the disposable part. The state is not.

I did briefly go down the fun rabbit hole. The proper answer to "keep game state alive across a code swap" is often the BEAM: Elixir or Erlang, with the world living in a supervised process or an ETS table that a hot code reload can slide underneath. I genuinely considered a rewrite. I talked myself out of it for two honest reasons. First, my ceiling in this game isn't CPU, it's bandwidth, the cost of streaming everyone the world, and switching runtimes does nothing for that. Second, BEAM hot code reloading is a beautiful trick that is also famously fiddly to get exactly right in production, and "fiddly, in production, on the thing that must not break" is not a trade I wanted. I don't need the world to survive a reload in place. I just need it to survive being written down and read back.

And I already had the place to write it down. Postgres is right there, holding accounts and the leaderboard. It's durable, it's shared, and crucially it's a thing that keeps existing while both the old machine and the new machine are alive at the same moment. So that's the whole trick: at shutdown the old process serializes the human snakes into one row in Postgres, and at boot the new process reads that row back. Postgres is the stepping stone the arena hops across, from the dying...