I Rented a 96 GB GPU and Took Uncensored Qwen3.8 From 44 to 125 tok/s | Aseem Shrey · security × AI

The whole experiment in one image: Qwen3.8 27B, one 96 GB GPU, 125 tok/s, 518 agent calls, and $1.42 of Verda instance charges.

tl;dr

I rented an RTX PRO 6000 Blackwell with 96 GB of VRAM and self-hosted orcarouter/Qwen3.8-27B-Uncensored-FP8 on that remote GPU through vLLM.

It was genuinely refreshing. I gave it the Ox Alpha investigation and it worked on the task. No sermons. No ritual begging. No pretending a terminal command was morally complicated.

I used it as the autonomous orchestrator behind my Ox Alpha investigation. Across OpenCode and Pi, it made 518 model calls over a 2 hour 22 minute window and processed 60.8 million logical input tokens.

Verda billed $1.42 for the recorded 1 hour 30 minute RTX PRO 6000 run. The instance page shows $0.945/hour spot pricing . Coupon code KAITCHUP-50 gives new users $50 in free credit .

The 96 GB card gave me the easiest path to FP8, 262K context, DFlash2, and a large KV cache on one GPU. Later measurements showed that 48 GB is enough if I reduce the maximum context to roughly 214K . The exact 262K setup does not quite fit a real 48 GB ceiling.

I looked into FreeToken but did not test it. Its interesting memory tricks are built around sparse MoE expert offload. This Qwen checkpoint is dense, so it was the wrong model shape for that experiment.

Uncensored does not mean smarter, correct, or safe. It means I get to put the controls in the execution environment instead of negotiating with the model every time I ask it to use a tool.

Here is the token-speed summary. I kept output length in the table because an 80-token run and a 512-token run are not the same benchmark.

Serving pathOutput tokensKV cacheMedian output speedComparisonPlain vLLM80BF1644.29 tok/sbaselineMTP380BF1676.21 tok/s+72.1% on the same short testMTP3512BF1681.62 tok/slonger-output referenceDFlash2, lookahead 7512FP8124.99 tok/s+53.1% observed versus the MTP3 profile; KV format differed

Copyable setup prompt

The deployment repository is private. Instead of giving you a clone command that will fail, this is the prompt I would give a coding agent on a fresh GPU machine:

Set up a reproducible Qwen3.8 27B inference server on this machine.

Hardware and host:<br>- Ubuntu 24.04, CUDA 12.8, Docker, and one RTX PRO 6000 Blackwell with 96 GB VRAM<br>- keep Docker layers, Hugging Face cache, vLLM cache, metrics, and run configs under /mnt/qwen<br>- do not expose port 8000 publicly

Model and serving profile:<br>- model: orcarouter/Qwen3.8-27B-Uncensored-FP8<br>- served model name: qwen38-27b-uncensored-fp8<br>- DFlash2 draft model: incoai/Qwen3.8-27B-DFlash2<br>- pin vLLM to commit b389ac29465b33f9e9c534df221ea3c129e9793f, which contains DFlash2 PR 52816<br>- DFlash lookahead: 7 tokens<br>- FP8 KV cache<br>- maximum model length: 262144<br>- GPU memory utilization: 0.90<br>- maximum sequences: 256<br>- enable prefix caching, Qwen3 reasoning parser, automatic tool choice, and qwen3_coder tool parser

Create the Dockerfile and shell scripts needed to build and run this pinned image. Ask me for my gated-model Hugging Face token interactively, save it to /mnt/qwen/secrets/hf-token with mode 600, and never print it. Record the full launch configuration and timestamp for every run.

Before declaring success, verify the GPU with nvidia-smi -L, check /v1/models, inspect the container logs, and send a real /v1/chat/completions request that must return QWEN_READY. Also create a stop script that removes the container and reminds me to stop the rented GPU.

Show me every file you create and every command I need to run. Stop and explain the error if the pinned vLLM build or either gated model cannot be fetched.

The verification flow, including a real prompt, is near the end.

timeline

DateWhat happenedWhat I learnedAug 20Got Qwen3.8 27B FP8 running on Google Vertex AIThe model worked, but getting GPU capacity was the first boss fight. Baseline generation was 42.53 tok/s.Aug 21Moved experiments to Lightning AILightning was much easier for iteration. Plain vLLM measured 44.29 tok/s and MTP reached 81.62 tok/s on the longer test.Aug 21Added the DFlash2 draft modelMedian generation reached 124.99 tok/s. That was 53% above my MTP profile, but the KV format also changed.Aug 21-22Used the model to investigate Ox AlphaIt stayed on task across hundreds of tool calls and long, repetitive agent contexts.Aug 22Rebuilt the stack on Verda and ran a proper matrixDFlash7 with FP8 KV cache became the balanced configuration. DFlash9 was faster, but not enough to pass my promotion threshold.Aug 22Tested memory limits and prefix cachingA repeated 96K prefix went from 17.754 seconds to first token down to 0.841 seconds. A 48 GB deployment looked viable around 214K context, not the full 262K.

Three days of unnecessary GPU drama. I went from a 42 tok/s baseline to a 125 tok/s agent stack, then used it on Ox Alpha.

Why I wanted a model without policy brakes

I was...