Building a 1-Outlet, 4-GPU Workstation — Joe Barrow

local compute

Building a 1-Outlet, 4-GPU Workstation

As a grad student, the one thing I desperately wanted was a GPU workstation. As a gainfully employed adult I can finally make that happen.

By Joe Barrow 2026-05-18

TL;DR

I built a 4x 3090 Ti rig so you don’t have to. Unless you want to, in which case you should probably read the whole post!

I have spent the past few months piecemeal building a multi-GPU workstation, ensuring it could run on a single US wall outlet.<br>This rig allowed me to train some DETR models on CommonForms, a 1.3B param VLM on CommonForms, and to OCR the first million pages of laws for the LOCUS-v1 dataset.

The full constraints for the build were:

it must run on a single, standard US wall outlet – 15A, 120V; this one is non-negotiable, I rent an apartment and cannot modify the wiring or add a new high power circuit;

it must not be a nuisance; I don&rsquo;t live alone, so it can&rsquo;t be too loud and/or hot;

it must fit within a reasonable budget; we may have different definitions of reasonable, and I&rsquo;ll detail the costs later on. I could have built a much more impressive rig subject to the first two constraints for $60k, but that falls outside of my definition of reasonable; and

it should have enough VRAM and speed to comfortably SFT and RL good vision language models; this is the whole reason I&rsquo;m doing this, after all.

My workhorse. Currently it&rsquo;s a 4x3090Ti machine. Using this, I can OCR around 250k pages a day, and do a full finetune of Qwen-3 VL 2B at pretty respectable batch sizes.

The Plan

My plan for this was simple: buy a server motherboard with enough PCIe x16 slots, a CPU with enough PCIe lanes, as much cheap DDR4 RAM as I could load into it, a bunch of cheap SSDs.<br>Then power-limit the GPUs to a reasonable wattage, and go on my merry way running experiments.

Getting Punched in the Face

Quote

Everybody has a plan until they get punched in the face.

Mike Tyson

Unfortunately, Mike Tyson is right.<br>While trying to build my workstation, I had to change plans a few times.

A Global Compute Crunch

If you know anything about the price of computer parts from the last year, you would know what happened to that plan.<br>RAM quintupled in price, SSDs tripled, and used GPUs skyrocketed.

Buying used, cheap server parts was no longer an option.<br>It took a lot more time and patience to find decent prices on components, trawling eBay, r/homelabsales, and r/hardwareswap.<br>It became really hard to determine what was a good price for a component, so I had to resort to thinking about prices in terms of the Secretary Problem.<br>Given the most recent sales price, and the upper limit of what I was willing to pay, was this a good price?

Taking Risks on Parts

GPU scams on eBay are so interesting to me, because the account “earringprincess” who has 2 reviews that both say “great earrings” is selling an 8xH100 node at 90% less than legitimate sellers.

Is anyone really dropping $20k without, like, basic due diligence?<br>— Joe Barrow (@barrowjoseph) March 11, 2026

Pretty much every part of this build is a secondhand part, save the first 3090 Ti.<br>Secondhand parts come with unique risks.<br>Is this drive shot? Does the GPU work? Is the whole account a scam?

I was pretty fortunate here, though once I ordered a 3090 Ti and received a 3090.<br>The seller was very responsive, and it resolved with me getting a refund.<br>But the risk is part of the game.

The 3090 I received, instead of the 3090 Ti I ordered.

Power-Limiting is Not What it Seems

If you want to run off of a single outlet, power-limiting is your friend.<br>Unfortunately, it&rsquo;s a bad friend, and a liar no less.<br>Running nvidia-smi -pl 250 does not set a hard cap at 250W, it sets a cap at an average of 250W over a millisecond period.<br>Your GPUs can have transient spikes within that millisecond.<br>Which, if they line up, can trigger your PSU&rsquo;s overcurrent protection (OCP).

To avoid this, you need to limit clock speeds, not just power-limit.<br>Something like nvidia-smi -lgc 210,1500 can avoid triggering OCP.

My Setup

I ended up going with the following:

Motherboard ASRock Rack ROMED8-2t; from all of my research this seems to be a pretty popular choice for home GPU clusters. A lot of people on r/localllama who do multi-3090 setups use it, I think the tinybox red might use it, and James Betker (nonint) used it for his clusters. People complain about issues like being unable to update the firmware. Thus far, I have not tried. My only complaint is that not all error codes are documented in the manual, so if something goes mildly wrong you are on your own to debug. It has a whopping 7 16x PCIe slots, meaning it&rsquo;s expandable far beyond 4GPUs without bifurcation!

CPU Epyc 7532; these are currently quite cheap on eBay, and give you 32 cores/64 threads. If you can efficiently parallelize tasks, that&rsquo;s… not bad. Even better, you get 128 PCIe lanes, enough...