Deepseek V4 Flash 0731 latency numbers from nine providers | Oskari Mantere

DeepSeek V4 Flash 0731 is available from many inference providers, and I needed to evaluate which ones could handle our production traffic.

I sent 30 streaming requests concurrently to each provider across four workloads: 120 per provider and 1,080 total requests. Decode speed was measured as the combined rate of the 30 decode requests.

This is a one-time snapshot. DeepInfra felt faster earlier in the day, but got slower as US traffic was starting to ramp up at the time when I was doing this test.

Results

Provider<br>Decode tok/s<br>TTFT p50 / p99<br>Total p99<br>Cache hit<br>Errors

Scaleway<br>1,650<br>733 / 818 ms<br>18.62 s<br>80.2%<br>18/120

TensorX<br>1,192<br>873 / 1,630 ms<br>25.77 s<br>73.4%<br>1/120

Fireworks<br>2,333<br>1,072 / 2,592 ms<br>13.17 s<br>17.6%<br>0/120

Baseten<br>3,980<br>774 / 2,979 ms<br>7.72 s<br>78.7%<br>0/120

Azure<br>596<br>618 / 3,029 ms<br>51.55 s<br>57.7%<br>0/120

DeepInfra<br>685<br>1,466 / 4,663 ms<br>44.85 s<br>86.6%<br>0/120

DigitalOcean<br>1,280<br>985 / 7,412 ms<br>24.00 s<br>26.5%<br>0/120

Nebius<br>2,542<br>948 / 7,563 ms<br>12.08 s<br>0.0%<br>0/120

Lyceum<br>934<br>8,275 / 9,354 ms<br>32.89 s<br>96.0%<br>0/120

Azure wasn’t running the new DeepSeek V4 Flash 0731 checkpoint, so its results aren’t directly comparable with the others.

Scaleway’s 18 failures were HTTP 429s from a token-per-minute quota which we need to increase. Other than the rate limits, successful requests to them had the best TTFT distribution. TensorX had one transient HTTP 502. Otherwise, all the providers were solid.

subscribe via RSS