GDP.pdf: Can Frontier Models Master the Documents that Run the World?
Blog<br>benchmarks<br>Expert Workforce<br>Products<br>Research<br>Careers<br>Contact
Login
Menu
Close
THE SURGE AI BLOG
GDP.pdf: Can Frontier Models Master the Documents that Run the World?
July 8, 2026<br>April 14, 2026
No items found.
TL;DR
One of our expert Surgers, an ER physician who helps us teach AI to triage medical cases, recently asked a frontier model to determine the correct treatment for a patient with an acute pulmonary embolism. It's the kind of case that comes up every day in medical departments around the world, and it required understanding a PDF of 2026 treatment guidelines.<br>Every single frontier model failed.<br>So we built GDP.pdf: a benchmark of real professional documents across ten domains. No frontier model cleared 25%. Before we trust enterprise agents with the workflows that run the economy, they have to master the paperwork that sustains it.
See the Benchmark
Read the Paper
HuggingFace Dataset
Table of contents
Case Study Llama<br>Analysis of Individual writings
Appendix
The unglamorous lifeblood of the economy<br>Parsing PDFs isn't the sexiest area of AI research. It doesn't produce viral videos, code flashy apps, or generate splashy headlines. But PDFs are the unglamorous lifeblood of the global economy, capturing every medical record, earnings report, contract, and invoice.<br>They’re also the lifeblood of AI agents. If we expect autonomous agents to genuinely transform day-to-day work, they have to natively master these formats: reading them, organizing them, cross-referencing dense data, and accurately filling them out.<br>When models fail at this level, the consequences are serious:<br>Finance . A model transposes two numbers from a quarterly earnings table, and a fabricated margin profile circulates in a buy-side memo.<br>Legal . A model hallucinates the location of a liability cap in a commercial lease, leading to catastrophic legal advice.<br>Healthcare . A model pulls the wrong row from a drug interaction chart, creating a life-threatening patient safety hazard.<br>Every one of these happened in our testing.<br>Measuring the essential: GDP.pdf<br>To measure the unsexy-but-essential work that keeps the economy moving, we built GDP.pdf . Its public set is available on HuggingFace here.<br>GDP.pdf is an expert multimodal and reasoning benchmark. It consists of 100 real-world prompts and PDFs pulled directly from professional workflows across ten domains: Finance, Healthcare, Legal, STEM/Research, Engineering, Construction, Manufacturing/Supply Chain, Insurance, Real Estate, and HR.<br>Every task requires parsing, understanding, and synthesizing complex PDFs: interpreting a multi-page dosage table, isolating an indemnification clause buried in nested exhibits, reconciling revenue figures across quarterly filings.<br>The result: Every frontier model scored under 30% .
GDP.pdf Benchmark Scores
GDP.pdf Benchmark Scores
Rank<br>Model<br>Score<br>(100% rubrics)<br>Mean criteria<br>pass rate
GPT-5.5 (xHigh reasoning)<br>25%<br>76.76%
Claude Opus 4.8 (Adaptive Max)<br>23%<br>76.94%
Claude Opus 4.7 (Adaptive Max)<br>21%<br>76.24%
Gemini 3.1 (Pro)<br>17%<br>73.07%
Gemini 3.5 Flash<br>14%<br>72.95%
Kimi K2.6<br>12%<br>69.10%
Gemini 3 Flash<br>10%<br>64.83%
Grok 4.3 (High)<br>8%<br>57.03%
Mistral Large 3<br>2%<br>50.98%
NVIDIA Nemotron 3 Nano Omni<br>2%<br>42.25%
Nova 2 (Pro)<br>2%<br>40.03%
Into the real world<br>At Surge, we often build benchmarks for the ceiling. Hemingway-bench measures their progress toward the Booker Prize. CoreCraft measures their ability to run a chaotic startup. Riemann-bench tests whether models can solve moonshot mathematics. We care about what's possible at the frontier.<br>We built GDP.pdf because real-world economic utility matters just as much. A model that can theorize about the Riemann hypothesis but gets lost in the fine print of a commercial lease is simply an intelligent liability.<br>Before we trust AI agents to manage the high-stakes workflows that drive the economy, they need to be able to master the complex paperwork that sustains it.<br>View the full GDP.pdf benchmark results and failure examples here. The public set can be found on Huggingface here.
Follow us
/surge-ai
@hellosurgeai
Deeper Instructions, Stronger Generalization: Training on ComplexConstraints
HANDBOOK.md: Can Agents Follow 100-Page Company Policies?
Anthropic cited GDP.pdf and Riemann-bench in their Fable 5 and Mythos 5 system card
ComplexConstraints: A Benchmark for Entangled Instruction Following
Microsoft used Surge human evaluations to benchmark MAI-Thinking-1
Cross-Benchmark Generalization for Long-Horizon Agentic Tasks
Antidote: Optimizing for You
GDP.pdf: Can Frontier Models Master the Documents that Run the World?
Riemann-bench: A Benchmark for Moonshot Mathematics
EnterpriseBench: CoreCraft – Measuring AI Agents in Chaotic, Enterprise RL Environments
Hemingway-bench: Because Good Writing Isn't a Checklist of Vibes
Building AdvancedIF: Evolving Instruction Following Beyond IFEval and “Avoid the Letter...