GRID — An agent with a spreadsheet engine beats one without

27. July 2026<br>An agent with a spreadsheet engine beats one without

Our results are now on the official SpreadsheetBench leaderboard: 91.25% on the verified 400 set, using Claude Sonnet.<br>The rest of this post is how we got here: what SpreadsheetBench is, what six months of running the benchmark taught us - and why an agent with a spreadsheet engine beats an agent without one.<br>We're also releasing @grid-is/agent-tools: the tools our spreadsheet agent uses to read, write, and understand workbooks - complementing our free-for-evaluation GRID engine.<br>What is SpreadsheetBench?<br>SpreadsheetBench is a benchmark for evaluating how well LLMs solve real spreadsheet tasks. It was introduced in a 2024 NeurIPS paper by researchers from Renmin University of China, Tsinghua University, and Zhipu.AI (Zeyao Ma, et al.). The dataset holds 912 tasks scraped from real Excel forum threads. Scoring is done by comparing cells: the cells your agent produced against the cells an expert says should be there.<br>In late 2025 the researchers released a 400-task subset in which every task has been reviewed by experts, the "verified 400", in collaboration with Shortcut (see blog). Their objective was to create a subset of tests with fewer correct interpretations, optimized for automated evaluation. By removing overly ambiguous and simple tasks they created, in our opinion, a subset which gives a much cleaner signal, and it's the set all our published numbers refer to.<br>Why we ran it<br>We build a spreadsheet engine. This is a tricky thing to do, because a modern spreadsheet is a combination of decades of features piled on top of one another, none particularly well documented. Unless you spend your days knee-deep in a workbook, you may not have thought about the engine underneath. You just use Excel, and the calculation engine is invisible.<br>When you've built something invisible, it's hard to tell people what you've done. You can say "We've written a spreadsheet engine, from scratch, that runs the calculations itself, manages the dependency graph, handles the awkward edges of Excel semantics, and exposes itself cleanly to code and to language models", and people will nod politely and then ask you if you have any interesting hobbies.<br>So how exactly should we show off our engine? We have demos, we have case studies, standard stuff. And then we discovered SpreadsheetBench. 912 real-world problems, nabbed from Excel forums, scored by comparing cells to cells. The tasks are the sort of thing a colleague sends you at half past four on a Friday: combine these two sheets by matching on text in two columns, oh and by the way there's a totals row, oh and by the way new rows will be added later, so please make it survive that. A great test for an engine.<br>This would help us test our hypothesis: an agent with a spreadsheet engine is better than an agent without. If you ask an LLM to solve a spreadsheet problem, it will often reach for the OpenPyXL Python library. But OpenPyXL is designed to read and write .xlsx files, not to provide a spreadsheet engine. It can't run calculations, it can't run formulas, it can't pivot tables. Instead, the LLM needs to write Python code to calculate values, and at that point a spreadsheet problem becomes a programming problem. It tries to solve the programming problem and hope the translation was lossless. It usually isn't.<br>Meanwhile our engine, sitting quietly on the other side of the room, can just evaluate the formula. It can solve the spreadsheet problem. We think an engine provides a good foundation for LLMs to solve the sort of problems people have in spreadsheets.<br>There was another reason too. We wanted to see our own engine and harness work on problems we hadn't chosen. It is very easy, when you build something, to end up testing it on the things it happens to be good at. SpreadsheetBench would help us get out of our comfort zone and find out if our spreadsheet engine could stand up against unforeseen problems.<br>How we benefitted<br>Our first tests, in December 2025, ran against the online MCP API we were hosting at the time, and they were surprisingly slow. That turned out to be the token-hungry nature of spreadsheets, not the service. So we quickly built an agent backend with the engine running directly inside it: a fuller exercise in what we believed our customers would want to build, and a faster way to iterate.<br>We'd been developing that engine for more than seven years, but we had only just taught it to save modified .xlsx files. Until then it had powered our first product, Calculator Studio, which turned spreadsheets into visual, interactive calculators that read and calculated across more than 200,000 workbooks but never wrote changes back to a file. Durable editing was brand new.<br>The first tests bore that out. They surfaced plenty of weaknesses in our export path (enough, we reckon, to account for something like 10–15% of the failures) and left us around...