Agentic AI at Two Different Scales: Nanbeige4.2-3B and Laguna S2.1

The Kaitchup – AI on a Budget

SubscribeSign in

Agentic AI at Two Different Scales: Nanbeige4.2-3B and Laguna S2.1<br>The Weekly Kaitchup #152

Benjamin Marie<br>Jul 25, 2026

Share

Hi everyone,<br>In this edition of The Weekly Kaitchup, I’m looking at two of the week’s most interesting releases for agentic workloads: Nanbeige4.2-3B and Laguna S 2.1.<br>The Kaitchup – AI on a Budget is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Subscribe

Agentic workloads include multi-step reasoning, tool use, interaction with external environments, and tasks that may require many actions before completion.<br>Nanbeige/Nanbeige4.2-3B

poolside/Laguna-S-2.1

Despite this shared focus, they operate at very different scales.<br>Nanbeige4.2-3B is a compact dense model with approximately 4 billion total parameters and 3 billion non-embedding parameters. It is intended to make capable agentic behavior practical on consumer and workstation hardware.<br>Laguna S 2.1 is a 118-billion-parameter Mixture-of-Experts (MoE) model. It activates approximately 8 billion parameters for each token.<br>The two models represent different approaches to agentic AI. Nanbeige uses repeated computation over a relatively small set of weights, while Laguna uses sparse access to a much larger pool of learned parameters.<br>Nanbeige4.2-3B

While Nanbeige4.1 used a basic Llama architecture, Nanbeige4.2-3B uses a Looped Transformer.<br>Instead of containing a large number of unique transformer layers, the model has 22 physical decoder layers that are executed twice . The same weights are reused during the second pass, giving the model the computational depth of approximately 44 layer executions without storing 44 independent layers.<br>This design reduces weight memory, but it does not reduce inference computation to that of a normal 22-layer model. Each token must still pass through the transformer stack twice.<br>The model has 48 attention heads with a dimension of 128, and eight key/value heads. This is a lot. For instance, Qwen3.6 35B A3B has only two KV heads.<br>Looping introduces a further memory consideration. Unless the implementation explicitly shares KV-cache state between the two passes, each loop may require separate key/value entries. Nanbeige’s default configuration does not appear to enable loop-level KV sharing.<br>Moreover, I think models using the Looped Transformer should be better named. When you use a 3B model, you don’t expect it to be nearly as slow as a 6B model. small MoE models, like Qwen3.6 and Gemma 4, show the number of active parameters in their name, like 26B-A4B. For Looped Transformers, we should adopt a similar convention, like 3B-2P (for “2 passes”), for instance.<br>Nanbeige4.2-3B’s memory consumption

A 16GB GPU should be able to run the model, without quantization, at moderate context lengths with careful settings. A 24GB GPU provides more room for longer prompts, greater concurrency, and runtime overhead.<br>Based on the model’s 22 physical layers, two loop passes, eight key/value heads, 128-dimensional heads, and BF16 KV values, a conservative estimate places KV-cache consumption at approximately 176 KiB per cached token for each active sequence.<br>I show in the following article how to estimate the KV cache memory consumption:

The KV-Cache of Small MoEs: Qwen3, Qwen3.5/3.6, GLM 4.7 Flash, and Nemotron 3 Nano Compared<br>Benjamin Marie<br>Mar 18

Read full story

At 16K tokens, this would require roughly 2.75 GiB of KV-cache memory. Using the complete 256K context could require roughly 44 GiB of KV-cache memory for a single active sequence.<br>This is very large. Almost twice as much as what Qwen3.6 27B would consume for the same context length.<br>Using Nanbeige4.2-3B with vLLM

At release, Nanbeige provided a dedicated vLLM branch for the model:<br>git clone -b nanbeige42 https://github.com/Nanbeige/vllm.git<br>cd vllm<br>pip install -e .The model can then be served through vLLM’s OpenAI-compatible API:<br>vllm serve Nanbeige/Nanbeige4.2-3B \<br>--host 0.0.0.0 \<br>--port 8000 \<br>--enable-auto-tool-choice \<br>--tool-call-parser nanbeige \<br>--reasoning-parser nanbeige

#use --host 127.0.0.1 if you are running it locallyThe reasoning and tool-call parsers are important for agent frameworks. They allow vLLM to separate reasoning content and structured tool calls from ordinary assistant text.<br>Nanbeige’s chat template exposes two controls.<br>The enable_thinking option determines whether the current response contains an explicit reasoning phase. The preserve_thinking option determines whether reasoning content from previous assistant turns remains in the conversation history.<br>For normal chat and question answering, preserved reasoning can usually be disabled. For multi-turn tool use, office workflows, and coding agents, the developers recommend retaining earlier reasoning content but I don’t understand how this can work well. Reasoning traces can be very...