Query Latency in the Age of AI Agents - Cube BlogJuly 15, 2026Engineering<br>Pavel Tiunov<br>PhD, Co-founder and CTO at Cube

Alexandr Romanenko<br>Principal Software Engineer at Cube

Table of Contents

Before it all started

Why AI needs even more speed and throughput

Why keep caching in the semantic layer

Why have a separate caching layer external to the warehouse

Cube Store architecture

What we are working on

Subscribe to Newsletter<br>Get Cube updates to your inbox for building better data products.

Subscribe

Every generation of analytics has had the same problem underneath it: the query a person wants to run is more expensive than the latency they are willing to wait for. The tools change, the storage changes, the query language changes, but that gap between "what I asked" and "how fast I need it" has never gone away. AI agents are now the largest consumers of analytical queries we have ever had, and they have made that gap wider, not narrower. This post is about how the industry has closed that gap over the last thirty years, why the answer keeps coming back to pre-computed aggregates, and how we built Cube Store to serve them.

Before it all started

Online Analytical Processing (OLAP) systems solved the latency problem in the 1990s by pre-computing. A MOLAP (multidimensional OLAP) engine took a fact table, picked a set of dimensions, and materialized the aggregates for combinations of those dimensions into a cube stored in a dedicated multidimensional structure. When a user sliced revenue by region and month, the answer had already been computed at load time, so the query was a lookup rather than a scan. Essbase, and later SQL Server Analysis Services, were built on this idea. It worked because the set of questions was known in advance and the cube was shaped to match.

The problem is combinatorial. A cube over n dimensions has 2^n possible aggregation groupings, and materializing all of them is not feasible past a handful of dimensions. The foundational treatment of this is Harinarayan, Rajaraman, and Ullman, Implementing Data Cubes Efficiently (1996), which frames the aggregates as a lattice and asks a greedy question: given a fixed storage budget and a known query workload, which views should you materialize so that the rest can be answered cheaply from them? That paper is thirty years old and it still describes the exact decision every pre-aggregation system makes today. The key input to the decision is the workload. You cannot pick the right views to materialize without knowing which queries you need to answer.

As data grew, the pure MOLAP cube ran into scaling limits, and the industry moved analytical modeling into ROLAP engines and, later, into cloud data warehouses. The warehouse pitch was that elastic, columnar, massively parallel compute would be fast enough that you would never need to pre-compute anything again. In practice the pre-computation never disappeared. It reappeared as materialized views, as aggregate tables built by transformation jobs, as BI extracts pulled into a tool's own in-memory engine, and as query result caches bolted onto the warehouse. The vocabulary changed from "cube" to "rollup" to "materialized view," but the mechanism is identical: compute the expensive aggregate once, ahead of time, and serve the interactive query from it.

Why AI needs even more speed and throughput

Two things changed when agents started querying data on behalf of users, and both push latency in the wrong direction.

The first is expectation. People now interact with analytics through a chat box, and they carry over the latency budget of a consumer product. A dashboard user tolerated a spinner because they had learned that "the query is running." An agent conversation sets a different baseline: a few hundred milliseconds feels responsive, a couple of seconds feels broken. The tolerance for a cold analytical query that takes eight seconds to plan and scan is much lower when the surface looks like a messaging app.

The second is volume, and it is the larger effect. A human exploring data issues queries one at a time, thinks about the result, and issues the next one. An agent in exploration mode does not. To answer one question it will fan out: profile a few columns, check cardinalities, try a grouping, notice something, drill in, verify the number a different way, then reformulate. A single user-facing task routinely turns into tens of queries, and multi-step agent workflows push that into the hundreds. This is the usual amplification pattern that shows up whenever a machine takes over a task a human used to do by hand, and it applies directly to query load. The system now has to serve many more queries per user session, and because those queries run in a chain, the tail latency of each one adds up into the total time the agent takes to answer. That means the store has to hold up on throughput and on p99 latency together, not trade one for the other.

Why keep caching in the semantic layer

The...