Live Web Search Benchmarks: Pick the Right Engine, Depth, and Model for Your Agent — OpenRouter BlogLive Web Search Benchmarks: Pick the Right Engine, Depth, and Model for Your Agent<br>Ayush Patel ·8/12/2026
We benchmark all the combinations to find strengths and weaknessesWe benchmark all the combinations to find strengths and weaknessesSearch budget matters more than any other factorYour worst-case cost scenario is driven by your failure rateWhile the engine matters, the model matters moreTry it on your own workloadBenchmarking methodologyFAQ
Web search is table stakes for most LLM requests in order to overcome knowledge cutoffs. Labs and search providers are evolving fast to make search more effective and efficient, leaving all of us with a set of tricky decisions: take the native search some labs build in, or wire up a third-party engine like Exa, Parallel, or Perplexity? Is one search enough, and if not how long do I let the agent keep searching? Are more search turns worth the quality they buy?
We built live leaderboards to help you decide the best search configuration with data. See the data on our new Benchmarks page.
We benchmark all the combinations to find strengths and weaknesses
When setting up a search request, you have four decisions:
Model. Writes the exact query that gets submitted to the search engine and processes the results.
Engine. You can choose a specific engine or rely on the bundled engines offered by some labs. On OpenRouter, we offer Exa, Parallel, and Perplexity, alongside the native engines from labs like OpenAI, Anthropic, and Google.
Search method. Either you can perform the search before calling the model and pass the results in as context, or you can equip the model with a web search tool that it calls at its discretion.
Search budget. If you choose the search tool method, you can also give the model a budget for how many searches it’s allowed to do. This enables models to adjust the query if it doesn’t like the results or to do follow-up searches. Our runs use 1, 5, or 25 turns.
To comprehensively understand web search performance, we regularly run four benchmarks across multiple models, engines, and search configurations:
BrowseComp: hard fact-finding that takes real browsing
DeepSearchQA: multi-hop research questions
WideSearch: broad “fill in the whole table” collection
HLE: expert exam questions with search
Each page ranks configurations by quality, value, and speed, so you can make decisions on the factor most important to your workload. The leaderboards are live, so the numbers move as new runs land and new models and engines are added. Today’s leader is not guaranteed to be tomorrow’s. We won’t spend much time on today’s leaders in this post as we expect that to change over time. Instead, let’s look into what the data tells us about how to make a decision for your workload.
Search budget matters more than any other factor
Increasing the engine budget up from one turn improves quality more than any other single change you can make. To illustrate, here was our initial run of BrowseComp on Perplexity across three different budgets:
Model, with Perplexity1 turn5 turns25 turnsClaude Opus 5, high35.8% ($0.14)66.5% ($0.51)89.0% ($0.99)GPT-5.6 Sol, high46.3% ($0.20)65.2% ($0.29)82.4% ($0.50)GPT-5.6 Luna, extra-high33.7% ($0.02)57.0% ($0.04)74.0% ($0.10)<br>This pattern holds up across all providers we measured:
These runs cover BrowseComp only, using the server tool with ten results per search, no page fetching or code execution, and the latest qualifying run per configuration.
Increasing search depth is the cheapest way we’ve found to increase quality. Increasing from 1 turn to 25 roughly doubles the score while costing only 2.5-7 times more per question.
You may assume this universally slows down response time, but that’s not always the case. For example, Luna took 140 seconds per question at 1 turn and 111 seconds at 25. Of the 35 configurations we ran at both 1 and 5 turns, over a third were slower with fewer turns. All were OpenAI models. These models deal with the constrained search budgets with extra reasoning.
On the other hand, search depth can be detrimental to costs on easier tasks. For example, on HLE, GPT-5.6 Sol with Perplexity scored similarly between 1 turn and 25 turns, for triple the cost. If your searches tend to be simple, it may still be worth keeping your budget limited.
Your worst-case cost scenario is driven by your failure rate
The other situation where an expanded budget is detrimental is when the model is failing to find an answer. We found that models would exhaust their budget attempting to find an answer even though they would eventually fail.
Suite (25-turn budget)Avg searches when correctAvg searches when incorrectBrowseComp10.319.7DeepSearchQA11.720.1HLE5.27.5WideSearch17.623.4<br>The deepest attempt we recorded, 81 searches on a WideSearch table, was still graded incorrect. If your workload has a high failure rate,...