Qwen 3.8 27B vs Nemotron 3.5 vs Muse Glimmer on an RTX 5090 - KGP Talkie<br>Englishहिंदी
Topics You Will Master<br>How to build a local benchmark whose answers are checked by running code, not by reading it<br>Why two models can tie on score and still be completely different to work with<br>How a generation token cap silently turns into a failure, and how to tell that apart from a wrong answer<br>How to pick between Qwen 3.8, Nemotron 3.5 Lightning, and Muse Glimmer for your own GPU and workload
Easy questions cannot separate local models any more. Ask a modern 27B model to write a palindrome checker and all of them get it right, so the score tells us nothing.
In this blog, we will push three of them until they break. We built sixteen hard problems where every answer can be checked exactly, ran each model alone on one RTX 5090, and measured not just who was correct but how each one failed.
udemyBestseller<br>Deep Agent - Multi Agent RAG with Gemini and Langchain<br>Build real-world AI agents and deep research systems using Google Gemini, LangChain v1, MCP, and modern RAG techniques.
Enroll on Udemy →30 day refund, lifetime access
Three Different Bets on the Same Size Class
The three models we tested are close in size and very different inside.
Nemotron is the odd one out. It is a sparse mixture of experts, the biggest on paper at 32.9B, and it carries a 1M token context window. It also has the narrowest embedding of the three at 2,688. It pays for that ambition in memory: 25.4GB on the card while running, against roughly 17GB for the other two.
Muse Glimmer is a dense 27.9B model with the widest embedding at 6,656. It is the only one of the three that does not expose a separate thinking field, so whatever reasoning it does comes out inline in the answer.
Qwen 3.8 sits in the middle. It is a dense 27.3B model with a 256K context and a vision projector built in. It is the smallest of the three and it still takes image input, which Nemotron cannot do at all.
Our Setup
We kept the hardware and settings identical for every model so the numbers stay comparable.
GPU: RTX 5090, 32GB VRAM
Runner: Ollama 0.32.12 on Windows 11
Context window: 65,536 tokens for every task
Temperature: 0.2, one run per task
Quantization: Q4_K_M for all three models
Generation cap: 32,768 tokens
One rule mattered more than the rest. Before any model loaded, every other model was unloaded and the script polled Ollama until the GPU reported nothing resident. No model ever shared VRAM with another, so every speed number is a clean solo run.
Note
We tested the 4-bit builds that Ollama ships, not the original release weights. For local use through Ollama, this is what you would actually run.
The Sixteen Problems
Every problem has one exact answer that a script can check. We grouped them into three families.
Math has six problems: the digit sum of 1 + 11 + 111 and so on out to 300 ones, the last three digits of the power tower 7^(7^7), the expected number of rolls to get three sixes in a row, domino tilings of a 4 by 10 board, arrangements of MISSISSIPPI with no two S letters touching, and an expected value draw problem.
Code has six problems: an expression evaluator that handles **, //, and % with correct right associativity and no eval, counting inversions in O(n log n), an O(1) LRU cache, fixing a closure late binding bug, comparing version strings, and predicting the exact output of a tricky Python snippet.
Reasoning has four problems: a 5 by 5 logic grid, a formatting task where six sentences must spell PYTHON and each must be exactly seven words, a JSON conversion with a discount and GST to compute, and a 64K document with three chained facts and deliberate decoy notes.
Nothing here was graded by eye. Code answers ran in a subprocess against test cases, with size and time gates so a correct but slow O(n squared) answer still fails. We also checked the graders against known wrong answers before the run, including a check that they reject an inversion counter that is too slow and an LRU cache that never tracks recency.
We did not trust our own arithmetic either. Every expected answer was computed by a separate script. The 4 by 10 tiling count came out as 18,061, which matches OEIS sequence A005178. That number matters later.
The Scoreboard
Muse Glimmer takes the top score, and the other two tie behind it.
Here, we can see a one task gap between first and second. That gap is the least interesting thing in the whole run, because a tie hides the fact that these models failed in completely opposite ways.
The matrix uses three states instead of two. A green check means solved, a red cross means a wrong answer, and the third symbol means the model was still reasoning when it hit the token ceiling. That third state is the one most benchmarks throw away.
Qwen 3.8 Is the Efficiency Pick
Qwen finished all sixteen problems in 6.3 minutes. Nemotron took 8.3 minutes. Muse Glimmer took 16.2 minutes, more than two and a half times...