Your GPUs aren’t the problem: AI’s real bottleneck is data delivery | TechCrunch

SearchSubmit

Site Search Toggle

Mega Menu Toggle

Topics

Latest

AI

Amazon

Apps

Biotech & Health

Climate

Cloud Computing

Commerce

Crypto

Enterprise

EVs

Fintech

Fundraising

Gadgets

Gaming

Google

Government & Policy

Hardware

Instagram

Layoffs

Media & Entertainment

Meta

Microsoft

Privacy

Robotics

Security

Social

Space

Startups

TikTok

Transportation

Venture

More from TechCrunch

Staff

Events

Startup Battlefield

StrictlyVC

Newsletters

Podcasts

Videos

Partner Content

TechCrunch Brand Studio

Crunchboard

Contact Us

Image Credits: Shutterstock

Sponsored

Your GPUs aren’t the problem: AI’s real bottleneck is data delivery

This article is presented by TC Brand Studio. This is paid content, TechCrunch editorial was not involved in the development of this article. Reach out to learn more about partnering with TC Brand Studio.

Sponsored by:

As enterprises race to scale AI, the biggest obstacle to performance and ROI may be the infrastructure moving data, not the hardware processing it.

Enterprise AI has entered a new phase. For the past 18 months, organizations have spent aggressively on GPUs, large language models (LLMs), and AI tooling. Now the focus is shifting from experimentation to operationalization. With that comes a sharper emphasis on ROI.

According to a 2025 IDC Spotlight report, organizations are moving away from one-off AI deployments and toward repeatable, scalable architectures designed to support production workloads. As AI becomes embedded across the business, performance, security, reliability, and operational consistency are becoming just as important as model innovation.

And moving data between storage and compute is becoming increasingly complex. As AI environments grow more distributed, ensuring that data reaches compute resources quickly, securely, and reliably has emerged as a critical infrastructure challenge. These developments are forcing CIOs to confront a difficult question: How do you turn AI investment into measurable business value?

When AI projects underperform, many tech leaders assume they need more compute. They add GPUs, expand clusters, or look for a better model. But according to infrastructure teams operating large-scale AI environments, the problem often lies elsewhere.

That’s because the GPUs aren’t starving for compute. They’re starving for data.

In truth, expensive compute resources are only as effective as the systems feeding them. If data can’t move efficiently, securely, and consistently between storage and compute, even the most powerful GPU clusters end up waiting. And idle GPUs are among the most expensive assets in the data center. The cost of disruption is rising too. According to the Uptime Institute’s Annual outage analysis 2025, more than half of organizations say their most recent significant outage cost more than $100,000 and one in five report costs exceeding $1 million.

Look below the waterline

To understand why AI initiatives stall, it helps to rethink the traditional infrastructure model.

Nirav Shah, senior vice president of product marketing at F5, compares modern AI infrastructure to an iceberg. Above the waterline sits everything executives can see: LLMs, AI applications, orchestration frameworks, and increasingly expensive GPU clusters. This visible layer receives most of the attention, and often most of the investment.

Below the waterline lies the infrastructure that determines whether those investments actually deliver value in production: storage, networking, traffic management, security controls, and the systems responsible for moving data between storage and compute.

“Everybody is focused on the visible 10%,” Shah says. “But it’s the other 90% that determines whether those investments actually work.”

That’s where many organizations discover their real bottleneck.

Modern AI systems depend on enormous volumes of unstructured data stored in Simple Storage Service (S3)–compatible object environments. Training, fine-tuning, retrieval-augmented generation (RAG), and inference workloads all rely on a continuous flow of data between storage systems and GPU environments. When that pipeline becomes constrained, GPU utilization drops.

“The symptom looks like a compute problem. The root cause is often data starvation,” says Mark Menger, solutions architect at F5.

And unlike traditional enterprise applications, AI workloads amplify small infrastructure weaknesses. A latency spike, throughput blockage, or traffic surge that might go unnoticed in a conventional environment can have an outsize impact on AI performance.

That’s why many organizations are beginning to look beyond GPUs and focus instead on the storage-to-compute boundary.

Moving from tight coupling to loose coupling drives flexibility

Many of today’s bottlenecks can be traced back to architectural decisions that made perfect sense before AI arrived....