Can I Run This Model?

The job

Model kind

Language<br>Image & video

Model

Params (B)

Layers

Task

Inference<br>QLoRA<br>Full tune

Context

2K<br>4K<br>8K<br>16K<br>32K<br>64K<br>128K

KV cache

FP16<br>Q8

Resolution

Frames

16<br>33<br>49<br>81<br>121

Text encoder precision

FP16 — as shipped<br>FP8 — the usual compromise

Offload the text encoder Encode the prompt, free the encoder, then load the model. ComfyUI does this by default and it is usually what saves you.

Weight precision

Concurrent seqs

Job length (h)

The GPU you have

Verdict

— GB VRAM NEEDED

0 GB

Link to this exact result

Copy

Where the memory goes

ComponentSizeShare

Peak100%

Rentable GPUs that fit

ConfigurationProviderVRAM$ / hrYour job

The math, so you can check it

Everything here is arithmetic you can reproduce, not a benchmark. Language estimates land within roughly 10–15% of what nvidia-smi reports for llama.cpp at these settings.

vLLM works differently and the number above is not what it will show you. vLLM is budget-driven rather than demand-driven: it claims gpu_memory_utilization of the card up front (0.90 in Runpod's guide, 0.92 is vLLM's own current default), subtracts model weights, non-Torch memory and the profiled activation peak, and hands the entire remainder to the KV cache as paged blocks of 16 tokens. So it never asks whether your context fits — it asks how many tokens the leftover can hold. Two consequences worth knowing: your usable VRAM is about 10% less than the sticker figure, and raising Max Model Length does not reserve more cache, it just lets one request consume more of a fixed pool. The deploy panel works out that pool and the resulting concurrency for you.

LANGUAGE<br>weights = params × bytes_per_weight<br>kv_cache = 2 × layers × kv_heads × head_dim × context × kv_bytes × seqs<br>overhead = 0.8 GB (CUDA context + runtime) + 5% of weights<br>QLoRA = 4-bit base + adapters/optimizer + checkpointed activations<br>Full tune = 16 bytes/param (fp16 weights + grads + Adam states + fp32 master)

IMAGE & VIDEO<br>model = params × bytes_per_weight<br>encoder = text_encoder_params × encoder_bytes ← the forgotten one<br>work_set = base × (pixels ÷ 1024²) × batch × (frames ÷ 16)^1.3<br>offloaded = max(encoder, model + vae + work_set) + 0.8 GB<br>resident = encoder + model + vae + work_set + 0.8 GB<br>The text encoder is the trap. Flux and SD 3.5 ship with T5-XXL, which is 4.7B parameters — 9.1 GB in fp16, before the image model loads at all. Offloading it (what ComfyUI does by default) is usually the difference between fitting and not.

The working-set term is the roughest number on this page. It is calibrated against reported figures for SDXL and Flux at 1024&thinsp;&times;&thinsp;1024, and video scaling is superlinear because 3D attention sees every frame at once. Treat video numbers as a starting point; block-swapping and tiled VAE decode beat them substantially.

Multi-GPU adds about 10% for tensor-parallel buffers, applied only when a job is actually sharded. MoE models hold every expert in memory but activate a few per token — memory tracks total params, speed tracks active params.

Prices were captured in August 2026 and drift constantly. Runpod figures are on-demand Secure Cloud, billed per second; Community Cloud and spot are cheaper. Lambda figures are published on-demand rates for single-GPU instances. Vast.ai is a marketplace where hosts set their own prices, so those rows are marked ~ and should be read as indicative — interruptible instances there run 30–50% below on-demand, at the cost of being interrupted.

One bias to correct for: Runpod has the most rows here because it publishes a full price list I could transcribe; Vast.ai has three because those are the only rates I could source with confidence. Vast.ai not appearing beside a given card does not mean it costs more there — it usually means I had no number. Check it yourself for anything above a couple of dollars an hour.