Evaluating DeepSeek V4 Pro 0813 on Hack The Box Challenges | TheArtificialQ Blog

The release of DeepSeek V4 Pro back in April got a lot of attention - so DeepSeek decided to release it again! The new version is called DeepSeek V4 Pro 0813 , and it is not exactly clear what changed. Reading its release page, you can almost feel the marketing team’s desperation to put at least something there. The few concrete details are that it uses the same core V4 Pro architecture with approximately 1.6T total and 49B active parameters, adds a DSpark speculative-decoding module (whatever that is), and supports a 1M-token context.

DeepSeek reports large improvements across several agent benchmarks, although those are the developer’s own results. I had finished testing the previous Pro version, now called DeepSeek V4 Pro 0423, shortly before the new version appeared (sic!). I therefore ran 0813 through my HTB-Challenger BenchmarkThe HTB-Challenger Benchmark evaluates LLMs’ ability to find and exploit security vulnerabilities. It tests models against selected Hack The Box challenges of varying difficulty and measures their performance. For more information, visit the HTB-Challenger Benchmark page.<br>to see whether it performed better on cybersecurity tasks.

This blog post is part of a series of tests for the HTB-Challenger BenchmarkThe HTB-Challenger Benchmark evaluates LLMs’ ability to find and exploit security vulnerabilities. It tests models against selected Hack The Box challenges of varying difficulty and measures their performance. For more information, visit the HTB-Challenger Benchmark page.<br>See the benchmark results page for all results and the benchmark methodology to learn how the benchmark is calculated.

The new model achieved a slightly higher benchmark score than the old one, 38.8% versus 36.2%, and solved nine challenges instead of eight. Given the small test set and normal run-to-run variability, I would still call the performance result a draw.

The clearest improvement was efficiency. The median run used far fewer steps (12.5 versus 62.5), tokens (0.25M versus 1.94M), and money ($0.20 versus $0.35) than the older model. That is a good reason to upgrade if you still use the older version.

If you are not specifically committed to DeepSeek, however, there are better choices for this kind of cybersecurity work. GPT-5.6 Luna Pro scored 51.3% at a lower median cost of $0.10, while Grok 4.6 and GPT-5.6 Sol achieved much higher scores at broadly similar median costs.

Cost vs. Benchmark Score

The highlighted point is this model. Models closer to the upper-left achieve a higher benchmark score at a lower median cost per challenge.

Overall benchmark results

Number of challenges: 16

Number of solved challenges: 9

Number of false positives: 0

Runs where the model gave up: 0

Runs that reached the step or cost limit: 2

Runs where the model got stuck: 5

Benchmark score: 38.8%

Metric<br>Per challenge (median)<br>Total

Model steps<br>12.5<br>456

Model cost<br>$0.20<br>$7.97

Duration<br>00:07:17<br>06:46:42

Number of input tokens<br>0.24M<br>16.32M

Number of output tokens<br>0.01M<br>0.98M

Number of read_file tool calls<br>1.0<br>121

Number of write_file tool calls<br>0.5<br>43

Number of execute_command tool calls<br>13.0<br>434

Number of web_search tool calls<br>0.0<br>12

Results by challenge difficulty

All resource-usage metrics are medians per challenge.

Metric<br>Very Easy<br>Easy<br>Medium<br>Hard

Results

Number of challenges

Number of solved challenges

Number of false positives

Runs where the model gave up

Runs that reached the step or cost limit

Runs where the model got stuck

Benchmark score<br>94.4%<br>73.1%<br>49.2%<br>0.0%

Median per challenge

Model steps<br>10.5<br>12.5<br>27<br>19

Model cost<br>$0.05<br>$0.22<br>$1.08<br>$0.40

Duration<br>00:03:09<br>00:09:19<br>00:34:12<br>00:10:44

Number of input tokens<br>0.07M<br>0.18M<br>0.47M<br>0.75M

Number of output tokens<br>0.00M<br>0.02M<br>0.11M<br>0.02M

Number of read_file tool calls<br>0.5<br>1.0<br>1.5<br>0.0

Number of write_file tool calls<br>0.0<br>4.0<br>1.0<br>0.0

Number of execute_command tool calls<br>10.0<br>8.5<br>22.5<br>13.0

Number of web_search tool calls<br>0.0<br>1.0<br>0.0<br>0.0