Resident: vibe coding firmware (our new sandbox library for ESP32 devices) (Interconnected)
Interconnected
A blog by Matt Webb
About
Archive
Work
Subscribe for $0
RSS feed
(What is a feed?)
Unoffice Hours
Book a call
(What is this?)
a.k.a. genmon
Bluesky
X/Twitter
Insta
Mastodon
Building the AI clock
Check out Poem/1
Resident: vibe coding firmware (our new sandbox library for ESP32 devices)
14.14, Wednesday 20 May 2026
Link to this post
We’re open sourcing Resident, our library for running AI-authored code on microcontrollers – with no compile step and no firmware flashing.
It’s our twist on vibe coding firmware, built for instantly loading new device functionality coded by end users. It’s aimed at device developers, like us. We use Resident in all our work.
(We = Inanimate.)
Resident gives you a code sandbox on ESP32 devices and a driver API to provide hardware control and events. So an end user can push an app over Wi-Fi that instantly turns their clock into an interactive pill timer (for example) but the app can’t run probes on the home Wi-Fi network.
It comes bundled with a set of Claude skills. I told Claude about the capabilities of a compatible dev kit and asked it to push a simple app. Here it is:
At Inanimate, we believe that on-device sandboxes are an essential low-level primitive for AI agents in the real world.
The announcement is over on Lab Notes but I want to take a moment to connect this to some previous themes…
Look, I want to bring software into my room.
Like: if I’m working with Claude Code and I step away from my keyboard, it should be able to give me updates on what it’s working on by taking over my desk clock, and ping its permission requests to the gumstick device I have in my pocket for me to accept/decline while I’m making tea. Untether me from my desk! (btw I built this, it was awesome.)
Or - more prosaically - why can’t I yell at my stove as I’m leaving the house in the morning oh I forgot I’m roasting a chicken that needs to be ready for 7pm and have it look up timings and push me a notification when it’s pre-heating, and show a custom basting timer app on its 14-seg LED display.
Why can’t I point at a lamp and say ‘on’ and the light come on? – I was asking in 2020.
So half of the solution here is AI: LLMs are super good at translating intent into action. I use regular language and the computer will Do What I Mean (2025).
But the other half is a problem: how does this actually work? Where does the software run?
Every device needs to be able to run user code, that’s the answer.
I’ve been bouncing off this since 2023 when I put an LLM in charge of a smart home (using tech that we now call agents, but then was my implementation of the ReAct pattern from a paper out of Princeton and Google Research).
In a nutshell, let’s say you press a button on your desk clock and your AI agent has to decide in the moment what to do with that. It won’t work, it’s too slow. Cognitively, an interface has to respond inside 150ms or it is no longer instant – but the AI is a network hop away.
Ok so let’s remove the network hop. Let’s pretend we have edge AI: a GPT-4-equiv AI as a component in every physical thing; ubiquitous intelligence is coming and a Feynmann-level light switch could guess your intentions pretty well, that was was my guess in 2023.
It’s not an outrageous extrapolation! Taalas is baking LLMs into silicon and delivers 17k tokens per second per user on Llama 3.1 8B (try it here, e.g. 4,000 words on Hamlet as a space opera, it’s wild it’s so instant). So GPT-4-equiv is a matter of time.
Even with that speed, it turns that inference is not enough. To Do What I Mean, you still need to import user context for personalisation and grounding. But memory, Gmail, Wikipedia and the rest are still mostly in the cloud. That network hop again.
So AI can’t be inside the event loop, not if you want really great on-device interactions.
Instead – let the AI write device code. That’s the approach we’ve found.
Take that toy example of a roast chicken basting timer on the display of my stove. My AI agent should be able to dynamically write code for that interactive app, and have that app code executed on the stove itself.
I’m not saying that the AI needs to vibe-code firmware.
Firmware is the code that runs on device microcontrollers: you author it and compile it and you flash it and from that point on, it never changes. AI agents can vibe firmware (very happily). But I’m not sure I want to load code onto my stove that has boundless control over the heating element and the network stack and whatever other low-level capabilities are managed by the firmware itself. We’ve already had smart fridges sending spam email (BBC News, 2014), no more thanks.
Instead of firmware, the AI can run code in sandboxes.
Cloudflare made the case for sandboxes in March this year when they introduced dynamic workers:
Last September we introduced Code Mode, the...