Running Qwen3.8-27B on DGX Spark | Kubesimplify

On this page (15)What Qwen3.8-27B actually is<br>What works on the Spark on day zero<br>The llama.cpp path (fastest way to first token)<br>The vLLM path (best throughput)<br>The long-context concurrency wedge (day-zero honesty)<br>NVFP4: the best-numbers recipe<br>Spark Arena<br>The Ollama path (same-day support)<br>Vision test: it read its own benchmark chart<br>Unified memory sequencing lesson<br>Three days later: MTP arrives on vLLM, and finds a cliff<br>SGLang joins the matrix (and teaches the scariest lesson)<br>The 75 tok/s post, reproduced<br>Wrapping up<br>Links<br>Qwen announced the 3.8 family on August 3: Qwen3.8-Max, the 2.4T flagship, plus a promise that open weights were coming "next week". The Max weights (2.4T-A95B) landed August 12. The one everyone actually wanted for local inference, the 27B, went quiet. Trackers even reported it as delayed with no new date.

It dropped on August 14. I had it running on the DGX Spark within the hour, so here is the full recipe, what worked on day zero, what did not, and the numbers. Let's get into it.

Here is exactly what everything below was run on:

ComponentWhat I ranBoxDGX Spark, GB10, 128GB unified memoryOS and driverDGX OS, driver 580.159.03, kernel 6.17.0-1018-nvidiaTest datesAugust 14-15, 2026 (the MTP, SGLang, and DSpark runs came on August 17)llama.cppbuild b10423vLLMspark-arena nightly 0.27.2rc1, plus stable v0.27.1 for the DSpark runOllamav0.32.12SGLanglatest-cu130Load-test toolllama-benchy 0.4.0FP8 checkpointQwen/Qwen3.8-27B-FP8 at 017b9c7a (the launch upload, unchanged since)GGUF quantunsloth/Qwen3.8-27B-GGUF at 4604b899NVFP4 quantunsloth/Qwen3.8-27B-NVFP4 at 60e813d4 (day zero), 7d6f8d4d (MTP runs)<br>unsloth updated both quants after launch, so I re-benchmarked the updated NVFP4 revision: 11.9 t/s single-stream, within 4% of the numbers below, nothing material changed.

Every measurement in this post came from one of these four commands, so you can rerun any table row yourself:

# llama.cpp raw numbers (pp512/pp2048/tg128/tg32 tables)<br>docker run --rm --gpus all -v $HOME/models/qwen38:/root/.cache/huggingface \<br>--entrypoint /app/llama ghcr.io/ggml-org/llama.cpp:server-cuda \<br>bench -m path-to>/Qwen3.8-27B-UD-Q4_K_XL.gguf -fa 1 -p 512,2048 -n 128,32

# vLLM and SGLang numbers (all pp2048/tg128 tables, any depth/concurrency)<br>uvx llama-benchy@0.4.0 --base-url http://spark-ip>:8000/v1 --model served-model> \<br>--pp 2048 --tg 128 --depth 0 16384 32768 --concurrency 1 2 5 10 \<br>--enable-prefix-caching --save-result results.csv --format csv

# Ollama numbers (from Ollama's own eval counters, temperature 0, 3 runs)<br>curl -s http://127.0.0.1:11435/api/generate -d '{"model":"qwen3.8:27b",<br>"prompt":"","stream":false,"options":{"temperature":0,"num_predict":200}}' \<br>| jq '{prompt_tok:.prompt_eval_count, prompt_ns:.prompt_eval_duration, gen_tok:.eval_count, gen_ns:.eval_duration}'

# Edit-heavy vs fresh-generation workload comparison (DSpark section)<br>python3 edit_bench.py http://127.0.0.1:8002/v1 qwen3.8-27b # from 0xBakeer's repo, bench/Copy

What Qwen3.8-27B actually is #

Reading the config before running things saves a lot of confusion, and this one is interesting:

27B parameters, and it is NOT a MoE. 64 layers with a hybrid attention pattern: every 4th layer is full gated attention, the other 48 layers are Gated DeltaNet (linear attention). Same hybrid lineage as Qwen3.5/3.6.

Native vision language model. There is a 27-layer vision encoder in the checkpoint, images and video in, text out.

262,144 token native context, extensible to 1M with YaRN.

Thinking mode on by default ( blocks), with recommended sampling temp 1.0 / top_p 0.95 / top_k 20. Non-thinking: temp 0.7 / top_p 0.80.

Apache 2.0.

Architecture class is Qwen3_5ForConditionalGeneration (model_type: qwen3_5). This detail matters: it is the same architecture family the inference engines already support, which is why day-zero support mostly just works.

Qwen's own (vendor-reported, so calibrate accordingly) numbers for the 27B: SWE-bench Pro 61.7, LiveCodeBench v6 90.3, Terminal Bench 2.1 at 73.0, GPQA Diamond 89.2, OSWorld-Verified 84.3.

What works on the Spark on day zero #

PathStatus on day zerollama.cpp + unsloth GGUFWorks (stock release build b10423, vision mmproj included)vLLM + official FP8 checkpointWorks, on both the stable v0.27.1 release and the spark-arena nightly (recipes below)OllamaWorks, needs v0.32.12 (released the same day)SGLangWorks on upstream latest; the older pinned dev container silently produced garbage (see below)<br>The llama.cpp path (fastest way to first token) #

Let's start with the fastest way to first token. One command, and the server pulls the GGUF straight from Hugging Face:

docker run -d --name qwen38-llamacpp --gpus all -p 8091:8091 \<br>-v $HOME/models/qwen38:/root/.cache/huggingface \<br>--entrypoint /app/llama-server \<br>ghcr.io/ggml-org/llama.cpp:server-cuda \<br>-hf unsloth/Qwen3.8-27B-GGUF:Q4_K_XL \<br>--port 8091 --host 0.0.0.0 -ngl 99 -c 32768 -fa onCopy

Two...