Self-Hosting My Own LLMs
Self-Hosting My Own LLMs
Originally published 2026-07-05.
At the beginning of 2026 I decided I wanted to self-host all the software that<br>would give me a "ChatGPT"-like experience, but fully under my own control and<br>ownership. I had a few reasons. Some were the usual ones — privacy and total<br>ownership of my own data. Some were pure curiosity: LLMs seemed like something<br>approaching black magic, and I wanted to look behind the curtain. And some were about<br>self-sufficiency — what happens if OpenAI or Anthropic decide to start being less<br>generous with their technology? This post explains the tools I eventually settled<br>on, and how I configured them in a way that suited my needs.
The Big Why
At a time when ChatGPT, Claude, Gemini and plenty of others are being handed to<br>users for free, why go to the time and expense of recreating something that’s<br>admittedly inferior? I’ll go through a few reasons, but the main one comes down to<br>data sovereignty. I wanted full ownership and control over my own data. Full stop.
I know I’m an outlier here. But it’s the same reason I still pay — in 2026, even! —<br>for an email service instead of relying on Gmail. Throughout 2025 I found myself<br>having more and more conversations across ChatGPT, Claude and other services, and I<br>realized those conversations were deeply personal and valuable to me, even when I<br>wasn’t chatting about anything especially private. I wanted to be able to look back<br>and easily find a conversation I’d had months (or eventually years) earlier. I know<br>ChatGPT, Claude and Gemini all offer that kind of history — but I’d grown uncomfortable<br>with the idea that I was building up a library of my own conversations inside someone<br>else’s walled garden. Technically, they own that data, and therefore my conversations.<br>I felt very little agency over something that felt like it should be entirely mine.<br>So it was settled: I would use open-source tools to host my own chat interface —<br>and if at all possible, power it with large language models running on my own<br>machine.
There were other reasons, too. I like tinkering, and I wanted to see how the LLM<br>sausage gets made. Chatting with ChatGPT online felt mysterious and hard for me to<br>fathom — how did this new technology actually work? There’s no better way to learn<br>than to set it up locally and watch all the moving parts.
Lastly, there’s the self-sufficiency angle. We’ve already seen powerful<br>closed-source models kept behind tightening restrictions and steep prices —<br>Anthropic’s Mythos and Fable, OpenAI’s GPT-5.6. And even when the models aren’t<br>formally restricted, plenty of people swear up and down that the big providers<br>quietly throttle them at busy times. Whether that’s actually true is more or less<br>unknowable — but there’s an antidote either way: use open-weights models that<br>aren’t so directly under any one company’s control. A local model, running on my<br>own hardware, is something that neither the government nor big business can so<br>easily restrict or take away.
The Cost
Before I begin, I should give some context about the sort of hardware I have.<br>It’s 2026, and hardware — memory especially — is suddenly expensive, because<br>we’re in the thick of a generative-AI bubble. People are paying scalper’s prices<br>for the hardware that lets you run models locally. Pretending this is all doable<br>without dropping a modest amount of money would be disingenuous. So what am I<br>running, and what did it cost?
I’m running what most people would call an enthusiast’s setup: some decent<br>hardware, but nowhere close to a high-end build. I started this with a custom AMD<br>desktop I built right before COVID, paired with a video card I picked up last<br>year. In 2026 it’s not remotely cutting edge, and that’s fine.
I built it around AMD’s Ryzen 9 5900X — a 12-core beast at the time — and loaded<br>it up with about 80 GB of DDR4. I really wish I’d bought more memory back when it<br>was cheap a couple of years ago, but so does everyone else. Even so, this is a<br>very solid (if not cutting-edge) machine.
Last summer I walked into Micro Center and walked out with a refurbished NVIDIA<br>RTX 3090 Ti for $850 plus tax. I remember wondering at the time whether it was a<br>dumb purchase. I’m now sleeping very easy with that acquisition — it turned out to<br>be a great buy. The 3090 Ti and its 24 GB of VRAM do the majority of the heavy<br>lifting in my system, and that 24 GB is the single most important number in this<br>whole story: it decides which models fit, and shapes nearly every configuration<br>choice I make later on. Luckily I’d already put a largish 850-watt power supply in<br>the machine, so I didn’t need to upgrade that either.
So how much would this setup cost to assemble today? This got me curious about my<br>original build cost. Back in Chicago I actually lived within walking distance to a<br>Micro Center, a deeply expensive convenience for me. I did my whole build using<br>Micro Center, and I thought it would be fun to pull up my original<br>receipts…mostly...