Red vs Blue (vs Yellow) — benchmarking low latency, high concurrency analytics engines | by Daniel Hunt | Jun, 2026 | MediumSitemapOpen in appSign up<br>Sign in

Medium Logo

Get app<br>Write

Search

Sign up<br>Sign in

Red vs Blue (vs Yellow) — benchmarking low latency, high concurrency analytics engines

Daniel Hunt

9 min read·<br>2 days ago

Listen

Share

Low latency, high concurrency analytics has been in the spotlight in the past few weeks, with benchmarks flying left right and centre.<br>In particular, ClickHouse has published a number of benchmarks across the TPC-H and ClickBench datasets, and Databricks joined the fray this week with some vague (and as we’ll see, misleading) bench-marketing for a product they’re yet to release.<br>Snowflake’s Interactive Warehouses have been conspicuously absent from many of these comparisons, so I thought I’d dig in and see if I could replicate some of the results myself, and in the process provide the transparency for anyone else looking to do the same.<br>And how do they perform? Well, while Snowflake generally doesn’t recommend synthetic benchmarks and I’d encourage you to always test things for yourself, my curiosity got the better of me.<br>The results below speak for themselves:<br>Snowflake outperforms the Reyden engine by up to 3x on TPC-H, based on numbers shared in the keynote. This, in spite of claims during that same keynote that “Vendor 1 (interactive)” failed to run this same benchmark. I’d encourage you to try for yourself with the code.<br>Snowflake outperforms both ClickHouse (1.2–5.6x) and Databricks (28–35x) across all scales of CostBench, a significantly more difficult benchmark on cost/performance (as defined in the official CostBench repo by ClickHouse).<br>Press enter or click to view image in full size

Figure 1: A graphic from the Databricks keynote this week, updated with accurate numbers for Snowflake’s Interactive Warehouses in blue, which are 1.4x to 3x faster than the claimed Databricks Reyden performance at larger scale on the 22 query TPC-H benchmark. Note: “Vendor 2” numbers are sourced from Databricks, but as the benchmark details weren’t disclosed we can’t replicate those results. (code)Press enter or click to view image in full size

Figure 2: On “CostBench”, 43 queries against the ClickBench dataset, Snowflake Interactive delivers 1.2–5.6x better price performance than ClickHouse, and 28–35x better price performance than Databricks. Here, the 100B scale represents 18TB of raw data, i.e. 18x larger than the above TPC-H 1TB scale. We have used the optimal price-performance configuration for all engines at each scale. (code)Press enter or click to view image in full size

Figure 3: Drilling into “CostBench” at the largest 100B scale, we see the significant performance and cost advantages of Snowflake Interactive over traditional query engines (note the logarithmic scales). Here, Interactive was ~40% faster than ClickHouse, albeit at a ~30% premium. (code)For the uninitiated, Snowflake Interactive Warehouses became Generally Available in December 2025 as a new Snowflake compute architecture, ideal for workloads that require a combination of low latency and high concurrency, such as observability, real-time dashboards and data-powered APIs.<br>These warehouses run against Interactive Tables, which can be created from standard Snowflake surfaces including open table formats such as Apache Iceberg, and exist within the same consistent Snowflake RBAC governance model. They also inherit the majority of features from Snowflake’s regular tables and warehouses — search optimisation, auto-scaling, replication, cross-AZ redundancy, strong performance on joins and complex queries, etc.<br>With that out the way, let’s dig a bit deeper. You can also get stuck straight into the code if you prefer.

Deep Dive 1: TPC-H (code)<br>The TPC-H benchmark entails running 22 queries at different scales, such as SF10 (10GB) and SF1000 (1TB). As it happens, I had recently run benchmarks for Interactive Warehouses at the SF10 and SF100 scale, after ClickHouse published a series of blogs ranking various engines by their cost-performance on this benchmark, and was planning to publish my working.<br>Conveniently for me, Databricks then used the same benchmark in their keynote this week, but added the SF1000 scale (1TB) and SF1 scale (a measly 1GB). I added SF1000 scale to my testing, and was particularly interested in this result as the Databricks keynote claimed that “Vendor 1 (interactive)” failed to run at this scale, something I was certain would not be true in my testing, having already completed the significantly larger 18TB ClickBench run using Snowflake Interactive Warehouses.<br>How the benchmark was run<br>All TPC-H numbers below follow the standard protocol used in other blogs. The 22 queries run at SF10 (10GB), SF100 (100GB) and SF1000 (1TB); the result cache is disabled; each query gets three untimed warm-up passes and then three hot runs, and the fastest hot run of each query is summed for the...