When is NVLink Worth It?
When is NVLink Worth It?
Nvidia doesn’t support NVLink on consumer GPUs anymore.<br>The last one that had it available was the RTX 3090.
Right now in April 2026, NVLink bridges are pretty expensive,<br>$200-$400 depending on the exact size and whether you’re able to snipe<br>one at retail. Since I have two 3090s, I wanted to test how much benefit<br>NVLink actually provides for AI workloads.
The Tests
I ran a few tests to measure the effect that adding NVLink would have<br>on AI-related workloads:
Model Inference with layer split and tensor<br>parallelism. I used llama-bench with Llama<br>3.3 70B and Gemma 4<br>31B. Layer split does very little inter-GPU communication, while<br>tensor parallel inference must sync at every layer.
Model Training using DDP for TinyLlama<br>1.1B and FSDP for Qwen2.5 3B. DDP only<br>syncs once to compute gradients at each step, while FSDP requires<br>constant data transfers.
With these, I felt I had a good mix of tests that would be affected<br>to different degrees by inter-GPU bandwidth.
Hardware
CPU<br>AMD Epyc 7532 (32 Cores, 128 PCIe lanes)
Motherboard<br>Supermicro H12SSL-i, 5 PCIe 4.0 x16 Slots
RAM<br>8 Channels, 32GB DDR4-3200, 256GB total
GPU<br>2x EVGA RTX 3090, installed directly in slots 3 & 6, both<br>running at full x16 bandwidth.
OS<br>Ubuntu 24.04, Nvidia driver 595
Initial Results
The performance on all layer split tests was unaffected. For model<br>inference with tensor parallelism, NVLink boosted prompt processing by<br>about 30%, while token generation speed was exactly the same.
Token generation speed being the same on tensor parallel runs was<br>unexpected. I thought there would be a large difference since it needs<br>to sync activations frequently. Although token generation speed is<br>usually the headline number, I still care a lot about prompt processing<br>speed since most of my usage patterns, such as coding agents, use very<br>long context windows.
The uplift for FSDP training is huge, a nearly 3x improvement. DDP<br>training was not affected at all.
However, I consider this data to be somewhat misleading, as the<br>non-NVLink tests were not actually measuring GPU data transfer over PCIe<br>like I initially thought.
In all of my subsequent test configurations, inference with layer<br>split, tensor parallel token generation, and DDP training were<br>completely unaffected (within 3%/noise). I will omit any more<br>charts/data for those tests and show only tensor parallel prompt<br>processing and FSDP training . All data available here.
Pitfalls With<br>Disabling NVLink For Tests
In my first test, I used NCCL_P2P_DISABLE=1 for my<br>non-NVLink test. I had also set NCCL_DEBUG=INFO so I could<br>verify that the GPUs were using the channel I expected. I noticed lines<br>at test startup like this:
gpu:11419:11419 [0] NCCL INFO Channel 03 : 0[0] -> 1[1] via SHM/direct/direct<br>This means that instead of using P2P (Peer-to-Peer over PCIe), the<br>GPUs were falling back to SHM (Shared Host Memory), which is a full<br>round trip to main memory and incurs a higher performance penalty.
It’s not actually possible to specifically disable NVLink while it’s<br>installed and force the GPUs to use P2P. NCCL_P2P_DISABLE=1<br>disables P2P over PCIe as well and transfers go over SHM.
I reran the tests by physically unplugging the NVLink bridge.
SHM vs P2P, Nvidia’s<br>Restrictions
Even after unplugging the NVLink bridge, NCCL was still logging that<br>it was using SHM. Nvidia disables P2P entirely on consumer GPUs,<br>presumably for product segmentation. This is an artificial restriction<br>by Nvidia at the driver level; the RTX 3090 and other consumer GPUs are<br>capable of doing it. Some clever people have figured out how to bypass<br>this restriction by patching the drivers, and the solution is pretty<br>easy to install: open-gpu-kernel-modules.
I installed the patch and tested it again. I confirmed the NCCL log<br>messages indicating that it was using P2P this time:
gpu:11090:11152 [1] NCCL INFO Channel 00/0 : 1[1] -> 0[0] via P2P/direct pointer<br>Results Using Actual P2P<br>Communication
Enabling P2P almost entirely makes up the difference between SHM and<br>NVLink. Prompt processing speeds are basically identical and FSDP<br>training reaches over 90% of the performance of NVLink.
So why would anyone drop $300 on an NVLink<br>bridge?
Consumer Platforms
NVLink doesn’t really have much of a benefit for my specific setup. I<br>already have a server CPU & motherboard (Epyc 7532 + Supermicro<br>H12SSL-i), which comes with multiple full PCIe Gen4 x16 slots. I can<br>install multiple GPUs and each of them gets full x16 bandwidth. However,<br>the average consumer is probably not using a server platform at<br>home.
Consumer CPUs don’t have support for a lot of PCIe lanes. As a<br>result, the motherboards for these CPUs don’t have many x16 slots. Some<br>motherboards advertise 2 x16 slots, but what they actually mean is that<br>they have two x16 sized slots, but only one of them has full<br>x16 bandwidth. Furthermore, there’s a caveat that if you install a card<br>into the second x16 sized slot, it will...