I Benchmarked Local LLMs on the Laptop I Already Have :: mamonas.devm.
mamonas.devAug 10, 2026tech8 min read<br>I Benchmarked Local LLMs on the Laptop I Already Have<br>My last post was about companies discovering their AI bills and installing caps. A reasonable reaction to that post is the one the local-LLM crowd has been giving all year: stop renting tokens, the open models are good now, just run them on your own machine.<br>The benchmarks backing that advice have a hardware problem. The writeups that measure anything run on Mac Studios or 128GB M5 Max laptops, and the headline stories are worse. DeepSeek V4 Flash “runs on a MacBook” if the MacBook is a 128GB build that costs about $5,000. Kimi K3 has open weights that need eight H100s just to load. One guide this summer was literally titled “Open Weights You Can’t Run.”<br>My laptop is a 2021 M1 Pro with 16GB of RAM. It’s almost five years old, and I suspect it’s a lot closer to the median developer machine than anything in those benchmarks. So I benchmarked what it can actually do.<br>The Setup⌗<br>I tested five models, all pulled through Ollama, all picked to fit in 16GB:<br>qwen3.5:4b and qwen3.5:9b , Alibaba’s current small models, both of which think before answering by default<br>gemma4:e4b-it-qat and gemma4:12b-it-qat , Google’s current small family, in the quantization-aware builds<br>qwen2.5:7b , a late-2024 model, as the baseline for whether a year of progress shows up at this size<br>The current models everyone actually talks about didn’t make the list, and that’s a finding on its own. glm-4.7-flash and qwen3-coder both sound small and both are 19GB downloads. Qwen3.8-27B, announced the week I ran this, needs about 17GB and its weights weren’t even downloadable yet, they’re promised for the week this post goes out. On 16GB of RAM the list of models you can actually run is much shorter than online discussions suggest.<br>The tasks are 21 problems of the kind that come up in a data engineer’s week, in six categories: writing DuckDB SQL against a seeded database, fixing buggy Python files with a failing test suite, extracting JSON from messy text like Airflow logs and billing emails, predicting what tricky Python snippets print, answering quick questions I’d normally google, and finding facts planted in a 7,400-token runbook. Everything is graded by code, not by me reading the answers: generated SQL gets executed and compared row by row against a reference query, bug fixes have to pass pytest, extractions are checked field by field. Each model also has a temperature setting that controls how much randomness goes into its answers, I set it to zero and ran everything twice, so each model gets 42 runs.<br>What Passed⌗<br>modelsizepass ratetotal wall timegemma4:12b-it-qat7.2GB38/42 (90%)123 mingemma4:e4b-it-qat6.1GB36/42 (86%)18 minqwen3.5:9b6.6GB34/42 (81%)114 minqwen3.5:4b3.4GB26/42 (62%)125 minqwen2.5:7b4.7GB22/42 (52%)3 minTwo things surprised me here. The first is how much these models get right. Gemma 4’s 12B answered 90% of the suite correctly, including 6/6 bug fixes that had to pass an actual test suite and 8/10 SQL questions graded by execution. The 2024 baseline managed 52% on the same tasks and went 0 for 6 on predicting what Python code prints, so the models did improve over the year, at least between those two.<br>The second surprise is the wall-time column. The old qwen2.5 ripped through the whole suite in 3 minutes. The models that beat it needed one to two hours for the same 42 runs, and almost all of that gap is reasoning tokens.<br>% passed50607080900306090120qwen2.5:7bgemma4:e4bqwen3.5:9bgemma4:12bqwen3.5:4bwall-clock minutes for the full 42-run suiteAccuracy against time for the whole suite. Only gemma4:e4b is both fast and accurate.The Reasoning Tax⌗<br>The current generation thinks before answering, and on a laptop that thinking is where most of the time goes.<br>My first run capped generation at 3,072 tokens per task, which is not much considering how many tokens something like Claude uses on a single request. The qwen3.5 models hit that cap in 48 out of 84 runs, spending the entire budget on reasoning and never producing an answer. Both scored around 40-57%. I doubled the budget to 8,192 and reran: both gained exactly ten passes. The ability was there, the models just needed thousands of extra tokens to reach the answer. Some runs blew through the doubled budget too. qwen3.5:9b spent eight minutes and 34,000 characters of thinking on one SQL question and still ran out of room.<br>The gemma models think too, usually less, but not always. On one run, gemma4:12b spent 14 minutes reasoning about which numeric chmod mode corresponds to rwxr-xr–, produced 19,000 characters of thinking, and answered 750. The answer is 754.<br>On the API this same behavior shows up as a line item, the reasoning tokens get billed like any others. Locally it shows up as time spent waiting.<br>Would You Stop Googling...