Running NAM A2-Lite natively on an ESP32-S3 · Taurus<br>Running NAM A2-Lite natively on an ESP32-S3
An ESP32-S3 is a two-core microcontroller you can buy for a few dollars. It runs<br>a Neural Amp Modeler A2-Lite capture in single-precision float, live, with a<br>guitar going in through a USB interface — and the samples it produces are<br>bit-identical to the same capture rendered on a desktop. Not close. All 720,000<br>samples of the test render match, both files have the same hash, and subtracting<br>one from the other gives digital zero.
It is also a whole pedal now: a 480×480 touch screen with seven effects around<br>the amp, all of it on the same two cores.
As far as we can tell this is the first time a NAM A2 model has run on this chip<br>at all. The version that ships is the floating-point one, which is not what we<br>set out to build.
The obvious plan was to stop using floats
The S3 has 128-bit vector instructions and they are excellent — at integers. The<br>chip has a floating-point unit too, but no vector path into it, so float goes one<br>multiply at a time while integers go sixteen.
So the obvious move is to stop using floats and run the network in 16-bit<br>integers instead. That worked: about 20% faster than a well-optimised float<br>build, comfortably inside two cores, and it sounded fine.
It also sounded slightly hissy in the gaps between notes.
The hiss was not ours
The test material is fifteen seconds of a dry guitar riff, with 28 dB between its<br>peak and its average level.
That DI is never digitally silent. In its quietest passages it sits around −67<br>dBFS, which is the room and the pickups of the original take. Then put a 5150<br>with an overdrive in front of it — about 42 dB of gain — and that noise comes up<br>with everything else.
Measured across exactly those quiet windows, the integer engine, the float<br>three-channel engine and the full eight-channel float engine all put out the same<br>level, within 0.7 dB of each other, all carrying the same tiny DC offset, which is<br>the model's own output bias. The hiss was in the recording, and the amp was doing<br>what an amp does to it.
We also swept a proper gate in front of the amp to see whether gating would help,<br>and it does not: the best it ever bought was under a decibel, and past that it<br>started eating the playing. Quantisation error does not live in silence. It lives<br>in mid-level sustain and decay — inside the notes — where no threshold can reach<br>it. A gate is still worth having for the amplified room noise. It is not a fix for<br>arithmetic.
What actually fixed the integer engine
Three changes, and the order they happened in matters, because two of them look<br>worthless on their own.
Every quantisation scale had been rounded down to a power of two and padded ,<br>which throws away most of a bit per layer. Only one of those shifts is a hardware<br>shift and actually has to be a power of two; the rest are ordinary multiplies and<br>can take their exact values. On its own this moved the measurement by 0.2 dB. It<br>matters only because it stops wasting the range the next change needs.
Then each layer got its own exponent , so the integer grid slides up when the<br>signal drops instead of sitting frozen at whatever the loudest possible passage<br>would require. This is where the 6 dB came from, after two failures worth<br>recording:
Driving that exponent from the loudest sample in the current block made it<br>dramatically worse — and the reason is a nice trap. The convolution does not<br>read the block. It reads up to 1,200 samples of history. A quiet block right after<br>a loud one has a tiny peak and an enormous convolution result, so scaling up on<br>the block's peak guarantees you clip. The exponent has to follow a sliding peak<br>over the history the layer will actually read, with the current block only ever<br>allowed to pull it down — the direction that cannot overflow.
And without a retreat rule, a layer can park at an exponent that clips on every<br>single block with nothing to push it back. Adding "if the output gets within 15%<br>of full scale, drop a step" took the clipping count from 188,508 to a few<br>thousand, and tightening that ceiling took it the rest of the way to zero.
Finally the input moved up one bit. A guitar DI never exceeds full scale, so<br>half the input range was reserved for headroom that never gets used. The output<br>deliberately did not move, because an amp's output does pass unity on<br>transients, and clipping those is far worse than a bit of noise floor.
That last one was the biggest single jump of the three, and only because the other<br>two came first: while the layers were still noisy, a wasted bit on the input was<br>buried underneath them. Together the three took the quantisation error 26 dB down,<br>for about 10% more CPU — and the version that made the noise measurement worse<br>along the way was a log2 call, which cost 13 points of CPU to produce<br>bit-identical output. An integer count-leading-zeros does the same job for free.
Where the integer engine landed, and why it stopped mattering
At the end...