When Smaller Models Win - by Sruly Rosenblat
Asimov’s Addendum
SubscribeSign in
When Smaller Models Win<br>Chess, LoRA, and the case for specialized AI
Sruly Rosenblat<br>Aug 21, 2026
Share
IBM’s Deep Blue and Garry Kasparov (Wikipedia)<br>Even the Best AI Models Can Suck at Chess
The launch of ChatGPT had an interesting effect on the online chess discourse.1 Chess has already long been conquered by machines. As early as 1996 a computer (IBM’s Deep Blue) was able to beat the human world champion, grandmaster Garry Kasparov, in a game watched by over six million people.2 The world was shocked that a machine took on the best player and won a game, but chess engines didn’t stop evolving there. Since the 90s, they have gotten better while the machines needed to run them have become much smaller. Today Stockfish is widely considered much stronger than any human player. It has run on consumer hardware since its launch in 2008, and by 2014 it was beating some of the world’s top grandmasters.<br>It came as a surprise to many, therefore, that modern LLMs, trained on a vast portion of the internet and requiring supercomputers to run , couldn’t help but cheat on almost every move. There are endless videos showing how just a few moves into a normal chess game, ChatGPT and some of its competitors would gladly throw the rules out the window to escape a checkmate or gain an advantage.<br>But in truth, this isn’t surprising. The LLMs were not trained with chess in mind. Sure, they may have seen countless chess games scattered throughout the internet, but the vast majority of their parameters and training compute were devoted to capabilities that are completely useless once you put a chessboard in front of them.3 Stockfish on the other hand uses a tree search algorithm that is built to be good at chess. If you want a chess engine, you use a chess engine.4<br>Smaller Models Are Sometimes Better
While chess is a particularly potent example of a large language model losing to a much smaller specialized system, it’s far from unique. In a 2025 position paper, Nvidia researchers argued that small models5 (which it defines as models under 10 billion parameters) are the future of agentic AI and that they “provide significant benefits in cost-efficiency, adaptability, and deployment flexibility.”
Subscribe
Just because a larger model can do a job does not mean that a small model fine-tuned for that specific task can’t do it better and more cheaply. There are countless examples of smaller models doing just that. LiteResearcher is a 4B model that beat out Claude Sonnet 4.5 on some search benchmarks. Terminus-4B allows larger models to save compute by handing off terminal execution to a smaller model without suffering capability loss. The Docling family of open-source models start at just 258 million parameters and allow for fast extraction of PDFs to text without having to feed 100-page PDFs into an expensive LLM. Researchers also trained a small 4B model to outperform even the GPT 5 series of models in a few social negotiation situations such as negotiating salary or bargaining for a purchase. Each wins, not by raw intelligence but because it is built or fine-tuned for a narrower, more specific purpose.<br>A frontier model may know how to do all of these jobs, but that doesn’t mean it’s the right tool for the job. Large models are expensive and unpredictable, and doubly so when it comes to agentic tasks which can span several turns and hundreds of thousands of tokens.<br>Nvidia draws the line for small models at 10 billion parameters, but the more important boundary for developers may be whether a model is small enough to run yourself. There is still a whole class of models that are not necessarily small but are still small enough to fit on one consumer GPU (at least when quantized). This includes models like Qwen 3.8 27B, gemma 4 26b and GPT-OSS 20B. These models are very capable even without specialization and rank very highly on benchmarks (with Qwen sometimes outranking top models from a few months ago), but can still be easily run on premises without spending thousands of dollars on GPUs.<br>Share<br>The ability to run smaller specialized models adds more than just efficiency; it provides a more realistic opportunity for a developer to train and fine-tune their own model, and to host the model locally or in the cloud instead of relying on the model provider to do so for it. This in turn can provide developers with more control over how tokens are used, how outputs are structured, and how each part of the pipeline can be improved individually – instead of assuming an improvement in the most popular benchmarks will lead to every task improving. And as noted above, smaller models can be easier to fine-tune, thereby creating a more specialized AI. As my colleague Ilan Strauss has noted, specialization is a powerful economic force.<br>How Do You Train It, and Where Does It Run?
The strongest argument for using an off the shelf generic chat model is...