DFlash 2: Keep Drafting Parallel

tosh1 pts0 comments

DFlash 2: Keep Drafting Parallel — Inco AI<br>Skip to contentInference is the bottleneck of the agent era. Agents read, plan, and call<br>tools, often for hours or days. They consume tokens at a rate chat never<br>approached. Every one of those tokens takes a full forward pass over the<br>model. At Inco AI, we are building the inference stack scaled to the token<br>economics of tomorrow. This post is a sneak peek.

Our team released DFlash in January; it<br>now runs in SGLang, vLLM, TensorRT-LLM, and llama.cpp. NVIDIA measured<br>up to 15× throughput<br>with it on Blackwell GPUs; Google reported<br>3× more tokens per second<br>on TPUs; CoreWeave's production Kimi K2.7 Code endpoint, the fastest for<br>that model on Artificial Analysis,<br>runs DFlash by default. The ecosystem now builds on it:<br>NVIDIA,<br>Red Hat, and<br>Modal have all published<br>DFlash drafters; Meta<br>(Muse Glimmer),<br>Poolside (Laguna),<br>Xiaomi<br>(MiMo-V2.5-Pro),<br>and NVIDIA<br>(Nemotron 3.5 Lightning)<br>ship official drafters with their own models. On Hugging Face, DFlash models have<br>been downloaded more than 3.5 million times (as of August 2026).

Speculative decoding is a core piece of the modern inference<br>stack.1 A small draft model guesses a block of tokens,<br>and the target model verifies the whole block in one forward pass. Good guesses<br>turn one pass into several tokens; bad ones just get thrown away. For years, though, the draft itself stayed autoregressive :<br>one token at a time. DFlash made it one-pass too: the entire block, every<br>position, predicted in parallel .

DFlash 2 drafting for Qwen3.8-27B on an Apple M5 Max with oMLX, side by side with autoregressive decoding.<br>DFlash 2 pushes parallel drafting one step further: over 20% more output from<br>every verification pass, for around 1% added cycle latency , with the output<br>provably unchanged. Across benchmarks the gain runs 16–25%. With the<br>Qwen3.8-27B drafter released today, SGLang serves at 2.7–3.4× the<br>throughput of autoregressive decoding at batch size 1. Predicting every<br>position independently leaves headroom in two places: choosing the right<br>tokens and holding accuracy to the end of the block. DFlash 2 recovers<br>both without giving up the one-pass design.

Run It Now

DFlash 2 already runs in the mainstream inference engines:

SGLangvLLMllama.cppoMLX<br>pip install "sglang[all] @ git+https://github.com/sgl-project/sglang.git#subdirectory=python"

python -m sglang.launch_server \<br>--model-path Qwen/Qwen3.8-27B \<br>--speculative-algorithm DFLASH \<br>--speculative-draft-model-path incoai/Qwen3.8-27B-DFlash2 \<br>--speculative-num-draft-tokens 8

pip install -U "vllm @ git+https://github.com/vllm-project/vllm.git@refs/pull/52816/head"

vllm serve Qwen/Qwen3.8-27B \<br>--speculative-config '{<br>"method": "dflash",<br>"model": "incoai/Qwen3.8-27B-DFlash2",<br>"num_speculative_tokens": 7<br>}'

git clone https://github.com/ggml-org/llama.cpp.git<br>cd llama.cpp<br>git fetch origin pull/27342/head:pr-27342<br>git switch pr-27342

# NVIDIA CUDA<br>cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON<br>cmake --build build -j

# Apple Silicon<br>cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=ON<br>cmake --build build -j

./build/bin/llama-server \<br>-hf ggml-org/Qwen3.8-27B-GGUF:Q4_K_M \<br>-hfd incoai/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M \<br>--spec-type draft-dflash \<br>--spec-draft-n-max 7

:first-child]:mt-0 [&>:last-child]:mb-0 [&_li:last-child>p:last-child]:mb-0">Download and install the prebuilt oMLX with DFlash 2 support.<br>To run Qwen3.8-27B with DFlash 2:

Open the oMLX Model Downloader and download:

mlx-community/Qwen3.8-27B-4bit

incoai/Qwen3.8-27B-DFlash2

Open the Model Manager and edit mlx-community/Qwen3.8-27B-4bit. Configure DFlash with the following settings:

DFlash : enabled

Draft model : incoai/Qwen3.8-27B-DFlash2

Draft quantization : enabled

Runtime block size : 5

Verify mode : dflash

Save the settings and load the target model.

The Right Tokens Are Already There

DFlash predicts every position independently, in parallel. Each pick is<br>plausible on its own. Yet nothing makes them fit together, and an<br>incoherent block is cut short at verification.<br>Recent methods such as Domino and<br>DSpark buy coherence with sequential<br>heads that rewrite each position's full-vocabulary distribution.<br>But is that costly autoregressive correction really necessary?

No. The evidence is already in DFlash's own candidate lists. Take the<br>first position: DFlash's top pick is right 85.4% of the time, but the<br>right token is in its top 16 candidates 99.5% of the time. Even when the top pick<br>is wrong, the right token is usually on the list.

[role=checkbox]]:translate-y-[2px] whitespace-nowrap text-[0.7rem] font-medium tracking-wider uppercase text-left">Metric[role=checkbox]]:translate-y-[2px] whitespace-nowrap text-[0.7rem] font-medium tracking-wider uppercase text-right">0[role=checkbox]]:translate-y-[2px] whitespace-nowrap text-[0.7rem] font-medium tracking-wider uppercase text-right">1[role=checkbox]]:translate-y-[2px] whitespace-nowrap text-[0.7rem] font-medium tracking-wider uppercase...

dflash qwen3 model tokens draft right

Related Articles