Poetiq | Benchmarks Are Dead (for us)
get in touch<br>hello@poetiq.ai
join our team<br>join@poetiq.ai
Benchmarks Are Dead (for us)
Our RSI loop now fully automatically builds its own state-of-the-art harness<br>for any task it encounters.
July 15, 2026
Mainstream AI research is fixated on a capital-intensive strategy — intelligence via<br>incremental weight updates, costing billions in compute. At Poetiq, we believe that<br>intelligence does not need to live solely within model weights. Instead, we treat the LLM<br>as a single component of a larger, self-improving reasoning architecture. This<br>self-optimizing architecture leads to a Recursive Self-Improvement (RSI) loop that is<br>designed to automatically construct entire harnesses (consisting of code, prompts, tools,<br>and search strategies) for any benchmark it encounters.
At Poetiq, we're building a Metasystem capable of autonomously generating<br>the harnesses and auxiliary data required to solve any challenge. Setting state of the art<br>(SOTA) results on newly introduced benchmarks is now a routine procedure for us. However,<br>benchmarks only serve as proxies for real world challenges; their diagnostic utility<br>plummets once our system automatically masters them. Conventional, static, benchmark<br>frameworks are inadequate for evaluating systems with true RSI capabilities.
The Benefits of our RSI Loop
Autonomous SOTA Generation: With zero human intervention, our<br>Metasystem independently built harnesses that set SOTA milestones across six diverse<br>benchmarks — including long-context retrieval, agentic tool use, complex system<br>planning, and competitive mathematics. This includes outperforming a SOTA benchmark<br>result from Muse Spark 1.1 within 48 hours of publication.
Disruptive Performance: We frequently secure these SOTA outcomes<br>without using the benchmark's leading model. In multiple instances, our system overtook<br>existing benchmarks held by Anthropic's Claude Fable 5 using a previous generation<br>model.
A Model-Agnostic, Compounding Moat: By decoupling reasoning mechanics<br>from underlying weights, our design is completely model-agnostic. As proprietary and<br>open-source models advance, our Metasystem utilizes them as tools, continuously<br>compounding their value.
The Path That Brought Us Here
In the past year, we've been deliberate about which benchmarks we attempt. We picked our<br>initial three to match critical categories of tasks for LLMs:<br>Reasoning — synthesizing provided information in inventive ways.<br>ARC-AGI<br>(followup) is the premier<br>test of this.
Retrieval — testing the limits of the breadth of knowledge embedded in<br>a model's weights.<br>Humanity's Last Exam<br>audits this across a vast spectrum of disciplines.
Coding — the most pervasive commercial application of AI, melding<br>reasoning and retrieval with specialized procedural logic.<br>LiveCodeBench Pro tests coding ability while minimizing the risk of model memorization of answers.
We set SOTA on all three with no fine-tuning of weights and no privileged access. Across the<br>board, we see gains on all proprietary and open-source models. With each test, our<br>Metasystem constructed and optimized increasingly large portions of its own harness,<br>widening its capabilities through RSI.
The loop is now good enough that it no longer needs us to do more than pick the benchmark.<br>No hand tuning or customization needed.
The Results: Brand New SOTA Benchmarks
In the following table, we demonstrate the efficacy of our automatic<br>Metasystem on six highly complex benchmarks. Each benchmark was chosen to<br>target new and diverse tasks of types that our system had not previously seen. The same<br>Metasystem built every harness automatically. It designed the code, prompts, and<br>hyperparameters. For half the benchmarks, SOTA was achieved even with one-generation older<br>models than the current SOTA holder.
Summary of Results
Each harness, created by Poetiq's Metasystem, improved across all models (both proprietary<br>and open-source). Below, we give our results and the models that our system used to reach<br>it.1<br>(Note: While we did not use Anthropic's Claude Fable 5 as our underlying model, we<br>routinely outperformed its capabilities.)<br>Benchmark Category Underlying models used by Poetiq Poetiq Prev. SOTA Prev. model ArXivMath Competition math GPT-5.5 89.2 87.5* Claude Fable 5 SciCode Scientific coding Gemini 3.1 Pro 61.5 60.2 Claude Fable 5 Haladir — Challenge Set Long horizon planning Gemini 3.1 Pro 0.47 0.28 Claude Fable 5 Haladir — Full Set Long horizon planning Gemini 3.1 Pro 0.69 0.33‡ Claude Opus 4.8 MCP-Atlas Agentic tool use Muse Spark 1.1 & Gemini 3.5 Flash 89.8 88.1 Muse Spark 1.1 MRCR v2 Long-context Gemini 3.1 Flash-Lite 99.26 97.3 GPT-5.4 Toolathlon Agentic tool use Gemini 3.5-Flash 59.26 56.5† Gemini 3.5 Flash
Scores are percentages except Haladir, whose native score is 0–1. * Model released after<br>our results were reached. † Prior best on the standard, unmodified benchmark. Per<br>Anthropic's own note, its reported Claude Fable 5 (61.7) and Opus 4.8...