MediumZero-Copy Go: Why io.Copy Is Lying to You (sendfile/splice) | by Cheikh seck | Jul, 2026 | Dev GeniusSitemapOpen in appSign up<br>Sign in

Medium Logo

Get app<br>Write

Search

Sign up<br>Sign in

Dev Genius

Coding, Tutorials, News, UX, UI and much more related to development

Member-only story

Zero-Copy Go: Why io.Copy Is Lying to You (sendfile/splice)

Your HTTP middleware is probably copying every byte your AI gateway serves. Here’s the throughput you’re leaving on the table — measured, not guessed.

Cheikh seck

6 min read·<br>21 hours ago

Listen

Share

I run an inference gateway in front of our model servers. Last month we had a p99 latency spike during a traffic burst — requests that normally returned fast were stalling, and the CPU on the gateway boxes were pegged while the GPUs sat idle.<br>My first instinct, like an idiot, was to blame the model. “The 70B is slow under load,” I told my team. We spent two days tuning batch sizes and KV-cache before I finally looked at the gateway itself.<br>The model wasn’t the problem. The bytes were.<br>Every response body — multi-megabyte tensors, streamed tokens, the works — was being copied from kernel space into a userspace buffer and back out again, because of a logging middleware wrapper I’d written three months earlier and never thought about again. The gateway was a very expensive, very slow photocopier.<br>This is the io.Copy lie. And if you serve any kind of high-throughput traffic — AI inference, file downloads, video — it's costing you real money.<br>Press enter or click to view image in full size

Published in Dev Genius<br>32K followers<br>·Last published 21 hours ago

Coding, Tutorials, News, UX, UI and much more related to development

Written by Cheikh seck<br>313 followers<br>·19 following

Get your personal AI intern: https://liteagent.cloud/

Help

Status

About

Careers

Press

Blog

Store

Privacy

Rules

Terms

Text to speech