Slop Machine | cmart's blog

Slop Machine

Jul 12, 2026

I, a human, write every word of this blog in my own voice. Nonetheless, I built an AI Slop Machine.

Your browser does not support the video tag.

Background

(Boring! Skip to the pretty chart)

American and Chinese AI labs continue to make increasingly strong open-weights large language models (Qwen, Gemma, gpt-oss, and others). You can self-host these LLMs on consumer-attainable computers and GPUs. They’re pretty good at agentic knowledge work tasks, and when you use them, several of the things that people say are bad about AI don’t apply.

A tech company never handles your prompt or response data. It stays on your own computer.

You aren’t stuck on a ride of subscription-with-mystery-quota versus premium API pricing. Once you buy the hardware, it’s free to download the models and use them forever.

The ways that you access self-hosted LLMs are generally free of advertising or manipulation of your attention.

Once you have a copy of the model, its maker cannot withdraw your access to it.

You aren’t causing additional demand for AI data centers.

Yes, these models are trained at large computational expense, but the cake is done baking by the time you download a copy.

The electricity you’ll use is at most a few hundred watts while it’s actively working, similar to a video game console.

So, I’ve watched my friends with higher-spec Macbooks running increasingly-capable AI at usable speeds, bringing chunky power bricks to the cafe and using agents on hot, whirring laptops.

Meanwhile, my daily laptop (a 13th-gen Intel Framework 13) is not good at this. As a party trick, it fits decent LLMs in its 48 GB of system RAM, but the ~50 GB/s combined bandwidth of two DDR4 memory modules is a severe bottleneck. With an empty context window, I get single-digits of tokens per second from mixture-of-experts models (like Gemma 4 26B-A4B or Qwen 3.6 35B-A3B), and it slows down sharply as the context window grows. If I try a stronger dense model (like Qwen 3.6 27B or Gemma 4 31B), I measure throughput in seconds per token, unusably slow for interactive work. So, after leaving a job at which I deployed LLMs on a $15m pile of hardware, I’ve been using AI services hosted by companies for day-to-day tasks, mostly via Kagi Assistant and OpenRouter.

Also meanwhile, AI labs and neoclouds are supply-squeezing the RAM and GPU markets, making LLM-capable computers (nay, all computers) more expensive. This may not be an intentional strategy to make people more dependent on Codex and Claude subscriptions, but the economics sure point in that direction. If the AI bubble doesn’t pop then the shortage will continue raising hardware prices for years. So, if I wanted to do productive things with strong LLMs on my own computer, it was time to buy more computer.

LLM Selection

Given the pace of LLM evolution and obsolescence, it seems stupid to buy hardware sized to run one specific model, but I sorta did. Qwen 3.6 27B came out in April. It’s a Chinese model, with all the quirks and biases that entails. But if you believe LiveBench, Qwen 3.6 27B performs better than much larger models across several categories of task. Better than DeepSeek V3.2! To find an unambiguously more capable open-weights model on this benchmark, you need to reach for DeepSeek V4 Flash, which requires 10 times more VRAM (holding weight precision constant), and its overall score is only a little better. Other unpolluted, private benchmarks also show good things for this model. Also, I think it’s fair to say that the /r/localllama community really likes working with Qwen 3.6 27B. It behaves well in an agent harness, and malformed tool calls are rare in my experience.

Being a dense model, Qwen 3.6 27B is computationally intensive to run compared to mixture-of-experts models of similar overall size. But dense models get you the most ‘intelligence’ for a given amount of VRAM, and VRAM is the most limited aspect of ‘affordable’ GPU hardware these days. This model tolerates quantization down to ~4 bits per weight quite well, which makes it even more VRAM-friendly.

I already had good experiences harnessing cloud-hosted versions of this model. But if I want an alternative, Gemma 4 31B from Google is almost as good on benchmarks, and Google fixed the obnoxious corporate training manual personality that plagued Gemma 3.

I usually prefer slower-smarter tokens over faster-stupider tokens, because progress is often limited by me reading and understanding what the agent is doing, and ‘more code’ is often a liability. But if you want a faster model at similar size, you can use Qwen 3.6 35B-A3B or Gemma 4 26B-A4B, both sparse mixture-of-experts which accept an ‘intelligence’ penalty to use less computation per token.

So, any computer that will fit Qwen 3.6 27B will also fit good alternatives, with...