Qwen3.8-27B on One RTX 3090

">

01Test environment

ComponentVersion / spec

OSDebian GNU/Linux 13 (trixie), kernel 6.12.101+deb13-amd64<br>DesktopKDE Plasma on Wayland (kwin_wayland)<br>CPUAMD Ryzen 9 5950X — 16C/32T, Zen 3<br>RAM31 GiB<br>GPUNVIDIA GeForce RTX 3090, 24576 MiB (GA102, sm_86)<br>Driver610.57.04 · CUDA UMD 13.3<br>PCIegen 4 x16 under load (gen 2 at idle is power saving, not a fault)<br>Power limit380 W (VBIOS default 370, max 380), pinned via systemd<br>StorageSamsung SSD 970 PRO 1TB NVMe<br>Inferencellama.cpp b10217-ddd4ec142 (llama.app installer)<br>Model Aunsloth/Qwen3.8-27B-GGUF · UD-Q4_K_XL · 17,923,394,624 B · rev f1bfb127<br>Model BAtomicChat/Qwen3.8-27B-GGUF · AD-Q4_K_M · 17,120,781,792 B<br>Vision proj.mmproj-F16.gguf · 927,607,488 B<br>Harnessesopencode 1.18.18 · Hermes v0.20.2 (2026.8.16) · atomic-agent v0.2.1 *

* atomic-agent does not self-report a version. v0.2.1 is inferred from the latest<br>GitHub release of AtomicBot-ai/atomic-agent (2026-08-14) at install time (2026-08-16).

The architecture is what makes everything else possible

The GGUF reports general.architecture = qwen35, and it is a hybrid SSM + attention<br>model, not a plain transformer. Only one layer in four keeps a KV cache. That single fact is why a 27B model<br>gets a 131K window on a 24 GB card.

block_count 65 full_attention_interval 4 ← hybrid<br>attention.head_count 24 attention.head_count_kv 4<br>attention.key_length 256 attention.value_length 256<br>embedding_length 5120 context_length 262144<br>nextn_predict_layers 1 ← MTP heads ship with the model<br>general.sampling temp 1.0 · top_k 20 · top_p 0.95

02What these tests are, and what they are not

Most "which model / which setting" advice in circulation is eyeballed — side-by-side screenshots judged by<br>taste — or copied from someone running different hardware. This benchmark only asks questions with a<br>verifiable answer :

Reasoning problems: every answer computed by a reference implementation before<br>the model saw the question. Several sets are validated against exhaustive brute force.

Coding tasks: scored by hidden test suites the model never sees, each suite first<br>validated against a reference solution.

Performance: taken at thermal steady state after discarding warm-up runs, never cold.

Tool calling: scored on selecting the right tool, extracting the right arguments, and —<br>critically — declining to call when no tool applies.

What is deliberately not measured: anything judged by taste. Web-design quality, prose<br>style, "which output looks nicer". Those matter, but a benchmark that scores them is reporting the author's<br>preferences with extra steps.

Difficulty was escalated because the model kept passing

The first problem sets were too easy — everything scored 100%, which measures the test, not the model.<br>Difficulty was raised three times. The progression is itself a result:

SetCharacterScore

Easy — 8 short problemsarithmetic, counting8/8<br>Hard — 5 multi-stepsimulation, search, DP5/5 with reasoning<br>Brutal — 5 heavy200-term recurrence, 8×8 grid DP4/5<br>Frontier — 7 categories × 3 repstraps, bug-hunting, knapsack, probability20/21

Instrumentation scope

Telemetry depth is not uniform. A single long-lived nvidia-smi process<br>sampled temperature, power, clocks and utilisation every 2 s with native timestamps, at 0.0% measured<br>CPU overhead — no per-sample process spawning, and nothing called inside the timed path. Later probes<br>record start/end timestamps per row and are joined against that log offline. Earlier probes carry wall-clock<br>and token counts only.

03The vision crash — start here

Any request containing an image killed the server instantly. Text-only requests ran indefinitely.

ggml-cuda.cu:106: CUDA error<br>CUDA error: the requested functionality is not supported<br>current device: 0, in function ggml_cuda_mul_mat_cublas_impl at ...:1548<br>cublasGemmEx(..., cu_data_type_a, ..., cu_data_type_b, ..., cu_compute_type,<br>CUBLAS_GEMM_DEFAULT_TENSOR_OP)

What it is not

Not the mmproj dtype. The repo ships BF16 and F16 projectors. Both abort identically.

Not memory. Same abort at -c 16384 and -c 131072, ~2 GB free at<br>abort. The status is NOT_SUPPORTED, not OOM.

Not image size. Aborts with and without --image-min-tokens, with<br>--image-max-tokens 512, at 640×480 and 1920×1080.

Not the installer. A third-party report in the upstream issue hit the same abort with a<br>nixpkgs build. Reinstalling llama.cpp another way does not fix it.

The fix

export GGML_CUDA_CUBLAS_COMPUTE_TYPE=fp32

Value1920×1080 screenshotLatency

fp32works — reads the error text verbatim, with its line number3.4 s<br>fp16abort—<br>unsetabort—

fp16 does not help; only fp32 does. Measured cost: none.<br>66.11 tok/s with it against 65.28 without, same seed, thermally equalised. Prefill over a 128,290-token<br>prompt was unchanged at 705 tok/s.

The CPU fallback (--no-mmproj-offload) also avoids the abort and is unusable: 28.6 s at 640px<br>— and the model hallucinated the contents, because the text is illegible at that scale — rising to<br>over 300 s at 1080p, against...