Neutrino-1 8B · Fermion Research<br>Run Neutrino-1 8B

Overview

Neutrino-1 8B is an 8.19B-parameter decoder-only transformer that ships as one 3.88 GB file. Every one of its 252 transformer linears is stored in a proprietary ternary-family weight format eight times smaller than fp16; the weights stay bit-packed at rest and are decoded inside the matrix kernels, so nothing in the decode path is stored as fp16 or fp32 weight material.<br>Small weights change the serving economics. Single-stream decode is bound by how many bytes move per token, so a 3.88 GB working set decodes at rates a 16 GB fp16 artifact cannot reach on the same memory system, and the whole model fits beside its KV cache on an 8 GB GPU or a 16 GB laptop. The same container serves every platform below without conversion.

Architecture

A dense decoder-only transformer. Grouped-query attention holds the KV cache at a quarter of the query width, 144 KiB per token at fp16, so a 4k-token session costs 0.60 GB of cache beside the 3.88 GB of weights.<br>Base modelApache-2.0, Alibaba CloudQwen3-8B<br>Parameters6.95B coded projection weights, 1.24B int8 embedding, 0.3M norm8,190,735,360<br>Decoder layers36<br>Hidden width4,096<br>Feed-forward widthgated (SwiGLU), three linears per layer12,288<br>Attentiongrouped-query 4:1, head width 12832 query heads, 8 key-value heads<br>KV cachefp16; 0.60 GB at 4k context, 4.83 GB at 32k144 KiB per token<br>Position encodingapplied across the full 128-wide headrotary, base 1,000,000<br>Normalizationplus per-head query/key RMSNorm inside attentionRMSNorm, eps 1e-6<br>Context length40,960 tokens<br>Vocabulary151,936<br>Embeddingsinput embedding and output head are separate tensorsuntied<br>Geometry as read from the shipped container's header.Where the bytes live<br>Only the transformer linears carry the coded format. The two embedding tensors stay int8 because their rows are read one token at a time, not multiplied against the full activation stream, and the normalization weights are too small to be worth coding. A third of the file is vocabulary.<br>252 transformer linearsthe coded ternary-family lane, 67.2% of the file: query, key, value, and output projections plus the gate, up, and down feed-forward linears, 7 per layer, 72,351,744 bytes per layer2,605 MB<br>Token embeddings32.1% of the file: two untied int8 tensors of 151,936 × 4,096, input embedding and output head, one scale per row1,245 MB<br>Per-row metadata0.6%: row dimensions, scales, and row sums25 MB<br>Normalization weights145 tensors, kept float32: four per layer plus the final norm1.2 MB<br>Container header60 bytes<br>Byte budget of the 3,875,404,812-byte container, by tensor class.Inside the coded lane<br>Across the 6.95B coded weights, 62.63% sit at zero and the remainder splits 18.68% plus to 18.69% minus: sign-balanced to a hundredth of a point with no constraint asking for it. The balance is not uniform in depth. The gate and down feed-forward projections spike to 70 to 72% zeros in layers 1 through 3 while all four attention projections hold within about one point of 62% at every depth: the early feed-forward blocks shed weights the network does not need, and attention keeps a constant code density from layer 0 to layer 35.<br>gateupdownquery, key, value, output<br>60%65%70%08162435<br>60%65%70%08162435<br>Share of coded weights at zero, per projection, across all 36 layers.

Format

One container, three doors. The download is a coded transport of the container, not a compressed copy of an fp16 model: it expands bit-exactly to the file every runtime executes, and that one file is what runs on a datacenter GPU and on a laptop alike.<br>Downloadcoded transport, 2,559,822,594 bytes; expansion is bit-exact2.56 GB<br>On diskone container, 3,875,404,812 bytes3.88 GB<br>Distribution surfaces<br>pip engine<br>24.9 tok/s on an Apple M5, CPU only, 9 threads

The one-command door: pip install fermion-research downloads the container and the platform-matching native binary. CPU runtimes for macOS arm64 and Linux x86-64, with a bit-exact torch reference path underneath.

GGUF pack + CUDA fork<br>30.7 tok/s on an NVIDIA L4, 4.68 GiB at 4k context

The llama.cpp door: the container converted to GGUF with our weight types, loaded by our public llama.cpp fork. Runs llama-completion and llama-bench, with full CUDA offload.

MLX pack<br>33.7 tok/s on a base M5 MacBook

The Apple-silicon door: Python-native runtime with custom Metal kernels. The container is memory-mapped and the packed planes are decoded inside the GEMV kernels.

Evaluation

The release battery runs on the shipped container with thinking disabled, so every grade below is the artifact you download and not a research checkpoint. Protocol rides every row: shot count, grading mode, and item count.<br>MMLU5-shot, all 57 subjects, 14,042 items72.1<br>MMLU-Reduxgenerative, re-annotated subset, thinking off67.8<br>IFEval, prompt-strictgenerative, thinking off77.2<br>IFEval, instruction-strictsame run, per-instruction grading80.2<br>IFEval, prompt-loosesame run, loose extraction76.3<br>BFCL v3macro over 13 subsets,...