We Tested Qwen3.8 27B: How Much GPU and VRAM Do You Really Need? | Hardware Corner

Select Page

Table of Contents

We Tested Qwen3.8 27B: How Much GPU and VRAM Do You Really Need?

By

Allan Witt |<br>Updated: August 17, 2026

Qwen3.8 27B is another model in the 27B class that looks particularly interesting for local inference. We wanted to find out what it actually takes to run it on consumer hardware, especially at longer context lengths.

This article is based on our own llama.cpp benchmark results. We are not evaluating model intelligence, coding quality, or reasoning accuracy here. The focus is hardware: VRAM usage, context scaling, prompt processing speed, and token generation speed.

We tested the model on several NVIDIA GPUs, including the RTX 3090, RTX 4090, RTX 5090, and a dual RTX 5060 Ti setup. We also have results from an Apple M5 Max system with 128 GB of unified memory.

The testing environment used Ubuntu 24.04, NVIDIA driver 595.58.03, CUDA 12.8, and llama.cpp (MTP disabled) build 153d324bc (10364). All GPU tests used CUDA with full model offload and Flash Attention enabled.

One important detail is that llama.cpp reports the model architecture internally as qwen35 in the benchmark output. The benchmark data below is labeled as Qwen3.8 27B because that is the model we tested.

What Qwen3.8 27B Requires

The Q4_K Small build used in our tests is 16.68 GiB and contains 27.32 billion parameters.

That makes VRAM the first practical limit. The model itself fits on a 24 GB GPU, but the usable context length depends on the additional memory required by the KV cache and runtime overhead.

Our VRAM measurements show a fairly predictable increase as context grows.

Context Length<br>Measured VRAM Usage

4k<br>18 GB

8k<br>18 GB

16k<br>19 GB

32k<br>20 GB

64k<br>22 GB

128k<br>26 GB

256k<br>34 GB

The important number for a single 24 GB card is 64k. At that context length, our measured usage was about 22 GB, leaving limited headroom for the operating system, CUDA runtime, and other allocations.

At 128k, the requirement rises to about 26 GB, so a 24 GB GPU is no longer enough for a fully GPU-resident configuration.

A 32 GB card such as the RTX 5090 can handle 128k in our testing. At 256k, the measured requirement rises to around 34 GB, which moves the model beyond the capacity of a 32 GB consumer GPU.

This gives Qwen3.8 27B a relatively simple hardware profile: 24 GB is the practical starting point for long-context use, while 32 GB makes 128k context possible without offloading.

What GPU Do You Need for Qwen3.8 27B?

For basic 4k to 32k local inference, a 24 GB GPU is the most straightforward option. The RTX 3090 remains particularly interesting because its large VRAM pool allows the model to run without compromises at these context sizes.

For longer agentic workloads, 64k is the practical target on a 24 GB card. Our measurements show 22 GB at 64k, which leaves some operating margin.

The RTX 5090 is the more capable single-GPU option. Its 32 GB of VRAM is enough for the measured 128k configuration, although the model is not especially fast at that context length compared with shorter contexts.

The dual RTX 5060 Ti configuration is another interesting route. Two 16 GB cards provide enough aggregate VRAM to run the model at larger context lengths, although multi-GPU inference introduces its own scaling and interconnect considerations.

The key difference between these systems is not simply whether the model loads. It is how much context you can run while keeping the entire workload in GPU memory.

Dual RTX 5060 Ti Performance Results

We tested two RTX 5060 Ti cards with 15,849 MiB of VRAM each. Combined, the setup provides roughly 32 GB of physical GPU memory, which is enough to reach the larger context sizes in this test.

The RTX 5060 Ti setup is substantially slower than the RTX 3090 and RTX 4090 for token generation, but it demonstrates that a lower-cost multi-GPU configuration can handle the model.

Context<br>Prompt Processing (t/s)<br>Token Generation (t/s)

4k<br>1167.70<br>22.47

8k<br>1116.94<br>22.03

16k<br>1032.66<br>21.57

32k<br>904.12<br>20.43

64k<br>721.46<br>18.50

128k<br>512.63<br>15.55

At 4k context, generation is around 22.5 tokens per second. That falls to 15.6 tokens per second at 128k.

The setup is therefore more interesting as a capacity-per-dollar solution than as a high-speed inference platform. If the goal is simply getting a 27B model and a large context window into a relatively affordable system, two 16 GB cards can make sense.

RTX 3090 Performance Results

The RTX 3090 is still one of the most interesting cards for local LLM inference because of its 24 GB VRAM capacity and relatively low second-hand pricing.

For Qwen3.8 27B, that 24 GB capacity is enough to run the model up to 64k context in our testing.

Context<br>Prompt Processing (t/s)<br>Token Generation (t/s)

4k<br>1308.05<br>40.31

8k<br>1235.01<br>39.76

16k<br>1136.84<br>38.77

32k<br>977.25<br>37.02

64k<br>766.59<br>33.95

The RTX 3090 starts at just over 40 tokens per second and remains at...