What Is Soofi S 31.6B? German Open MoE ExplainedTLDRSoofi S 31.6B is a hybrid Mamba-MoE with 3.2B active params, trained on ~27T tokens by a German consortium on Deutsche Telekom's Munich cloud.<br>Meet Soofi S 31.6B, the German 27T-Token Open MoE

Soofi S 31.6B is an open-weights foundation model with 31.6 billion total parameters and 3.2 billion active per token, developed by a German consortium and trained entirely on Deutsche Telekom's Industrial AI Cloud in Munich. It uses a hybrid Mamba-2 plus granular Mixture-of-Experts architecture, was pretrained on roughly 27 trillion tokens with a deliberate German-English focus, and supports a 1 million token context window. The model was released as a base checkpoint (no instruction tuning) alongside its full pretraining report on arXiv and weights on Hugging Face.

Key Takeaways

Soofi S 31.6B has 31.6B total parameters with only 3.2B activated per token via a 128-expert MoE (6 active + 2 shared).

The architecture stacks 52 layers: 23 Mamba-2 blocks, 23 MoE blocks, and 6 GQA attention blocks, adopting NVIDIA's Nemotron 3 Nano recipe unmodified.

Pretraining ran on ~26.68–27T tokens across a three-phase Warmup-Stable-Decay schedule, with German upweighted to 15.32% during the anneal phase.

Training consumed ~253,000 GPU-hours on up to 512 NVIDIA B200 GPUs at Deutsche Telekom's Munich site, powered by 100% renewable energy.

Reported aggregate scores are 70.1 English and 79.1 German, leading fully open baselines and beating Olmo 3 32B and Apertus 70B on those suites.

Weights, intermediate checkpoints, full data inventory, and eval code are gated-open on Hugging Face; the final commercial license is still being finalized.

What Is Soofi S 31.6B?

Soofi S 31.6B (officially Soofi S 30B-A3B) is a base language model built by the Soofi consortium, a group coordinated by the KI Bundesverband and including Fraunhofer IAIS/IIS, DFKI, TU Darmstadt, University of Würzburg, ellamind, and Merantix Momentum. The project was funded with roughly €20 million from Germany's IPCEI-CIS program and positioned as a sovereign European foundation for industrial and agentic use. It is a raw pretrained checkpoint without supervised fine-tuning or alignment, released alongside instruct and thinking previews codenamed Isar and Rhine.

The model surfaced publicly through an official pretraining report on arXiv (2607.09424, submitted July 10 and revised July 13, 2026) and a coordinated announcement by Dr. Michael Fromm, Head of Pretraining Data, on July 11. Coverage from Marktechpost and The Decoder followed on July 13 and 15, framing the release as the strongest fully open German-English open-weights model to date.

Soofi S 31.6B at a Glance

FieldDetailDeveloperSoofi consortium (KI Bundesverband-coordinated; Fraunhofer IAIS/IIS, DFKI, TU Darmstadt, Uni Würzburg, ellamind, Merantix Momentum)TypeBase foundation model (hybrid Mamba-2 + granular MoE + GQA)Total parameters31.6BActive parameters per token3.2BArchitecture referenceNVIDIA Nemotron 3 Nano, adopted unmodifiedLayers52 total (23 Mamba-2, 23 MoE, 6 GQA)Experts128 experts, 6 active + 2 shared per tokenTraining tokens~26.68–27TContext window1M tokens (Phase 3)Knowledge cutoffEnd of 2025Training computeUp to 512 NVIDIA B200 GPUs, ~253,000 GPU-hoursTraining locationDeutsche Telekom Industrial AI Cloud, Munich (100% renewable)Training windowMarch 24 – May 13, 2026ModalityText (German-English focus)WeightsGated preview on Hugging Face (Soofi-Project/Soofi-S-Base)License"Other" (custom, not yet finalized)Official APINot yet confirmedCompanion releaseIsar (instruct), Rhine (thinking), GGUF + FP8 builds; Soofi L 120B-A12B in training<br>How Soofi S 31.6B Works

Soofi S 31.6B is what the consortium calls a Hybrid Mamba-MoE stack : 52 layers alternate 23 Mamba-2 state-space blocks with 23 granular MoE blocks, and only 6 layers use grouped-query attention. That ratio is why the model can hold a 1M-token window without paying the full quadratic cost of attention across every layer.

The MoE routing is what the report labels a Granular MoE Layer — 128 experts per MoE block, with 6 experts activated plus 2 shared per token. That gives a 3.2B active-parameter budget while the total sits at 31.6B, and the architecture is copied unmodified from NVIDIA's Nemotron 3 Nano recipe. Pretraining followed a three-phase WSD Schedule (Warmup-Stable-Decay): ~20T breadth tokens in phase one, ~6.58T high-quality anneal tokens with German pushed to 15.32% in phase two, and ~0.1T long-context tokens in phase three to lock in the 1M window.

Two efficiency claims stand out. The report says decode throughput per GPU runs 8–9× dense 14–24B models at a 40k-token context and batch size 32, and stays flat from 4k up to 256k. Second, the whole training run consumed roughly 253,000 GPU-hours on renewable-powered infrastructure — an unusually detailed sovereignty-and-sustainability disclosure for a model of this size.

What You Can Do With Soofi S 31.6B

The consortium is...