Your Parquet Column Indexes Are Being Ignored on EMR and Glue | Dustin Smith's Online Resume<br>Skip to main content<br>Menu

I was benchmarking data-clustering layouts on Delta tables. Locally, everything was skipping beautifully, box queries touched about 10% of rows. Then I ran the identical benchmark on EMR 7.13 and those queries scanned 93% of the table. Same code, same data shape, same queries.<br>Either my layouts were broken on EMR, or something in the read path was. This is the investigation that followed. It ends with a four-environment A/B test proving that the AWS Spark runtime&rsquo;s vectorized parquet reader, on both EMR 7.13 and Glue 5.0, performs row-group pruning but never applies parquet column indexes . Stock Apache Spark 3.5.6, the exact version EMR ships, applies them fully against the same files.<br>A Quick Refresher: The Three Tiers of Parquet Pruning<br>TierMechanismGranularityFileDelta/table-format stats, footer min/maxentire filesRow groupparquet footer statistics~tens of MBPageColumn indexes (parquet-format 2.5+)~1 MB or less<br>Column indexes are the finest tier, per-page min/max stored in the footer. On a well-clustered table this is where most of the win lives; pages are tight, so a 5% range query really does read ~5% of the data. My numbers said this tier was missing on EMR, but benchmark numbers come with a hundred confounders, so I needed to isolate it.<br>Ruling Out the Suspects<br>Is the writer broken? I copied the EMR-written parquet files to my laptop and read them with stock Spark. Page skipping worked (8.6% of rows scanned), so the files carry valid column indexes. Writer exonerated.<br>Is it a configuration problem? One cheap cluster, four variants of the same query:<br>VariantRows scanned (fraction)EMR defaults0.939spark.sql.parquet.columnIndex.enabled=true forced0.939spark.sql.parquet.filterPushdown=true forced0.939spark.sql.parquet.enableVectorizedReader=false0.079<br>No documented conf restores the behavior, but the last row is the evidence. Disabling the vectorized reader falls back to the slow row reader, which always evaluates column indexes, and full page skipping returns. At 9 to 14x the wall-clock, that is ground truth, not a workaround. (This sweep ran on my original table with file-sized row groups, which is why no-skipping degrades to 0.939; the probe below uses a rebuilt table with more files. Compare within each table, not across.)<br>Is it the environment? One probe script, run unmodified in three AWS environments against byte-identical files in the same bucket. It runs ten staggered range queries over the sort column and reads the scan node&rsquo;s actual output-row count from the executed plan. A counter, not a timing, so hardware differences cannot move it.<br>The Test Environments<br>Every number in this post comes from one of these setups. Bookmark this table; the prose refers back to it instead of re-describing hardware.<br>LabelHardwareRuntimeUsed forLaptopApple silicon, local NVMestock Apache Spark (local mode)writer check, MWEEC2 single node1x m7g.2xlarge (8-core Graviton), tuned s3a (connection pool 128, threads 64; the tuning alone halved wall-clocks)stock Spark 3.5.5/3.5.6, later 4.0.4 and 4.2.0the probe, the reader survey, the Spark-version ladderEMR diag cluster2x m7g.xlarge (~4 executor cores)EMR 7.13 (ships Spark 3.5.6)the verdict, the config sweepGlue2x G.1X DPUsGlue 5.0the verdictEC2 fleetm7g.xlarge driver + 4x m7g.8xlarge workers (128 executor cores)stock Spark 4.2.0 standalone (4.1.3 + Analytics Accelerator for the Iceberg leg only)the runs at 2.75 billion and 27.5 billion rowsEMR fleetm7g.xlarge master + 4x m7g.8xlarge core nodes (128 cores, matching the EC2 fleet)EMR 7.13the matched comparisons<br>The Verdict<br>Environmentvectorized (default)row reader (control)EC2, stock Apache Spark 3.5.6 0.086 0.086EMR 7.13 (ships Spark 3.5.6)0.287 0.086Glue 5.0 0.287 0.086<br>Stock Spark&rsquo;s vectorized reader hits the ground truth exactly. EMR and Glue read 3.3x more rows, byte-identical to each other, which points at a shared AWS runtime lineage. And 0.287 is exactly what row-group pruning alone predicts for this file geometry. As a control, 80 range queries over unsorted columns read ~1.000 in every environment under both readers; the probe is consistent.<br>The Fork You&rsquo;re Forced Into<br>Wall-clock for the same probe queries, both reader modes (warm medians). Hardware differs across rows (see the environments table), so read each row&rsquo;s ratio, not the columns:<br>Environmentvectorized ONvectorized OFF (row reader)Cost of getting page skippingEC2, stock Spark 3.5.x (tuned s3a)4.9s6.9snone; ON already skips (0.086)EMR 7.130.88s12.3s14x Glue 5.00.88s9.9s11x<br>On EMR and Glue you pick one, fast decode that reads 3.3x more rows than it needs, or full skipping at 11 to 14x the wall-clock. Note the perverse detail in the EMR row, where the row reader scans 3.3x fewer rows yet takes 14x longer. Decode efficiency dominates I/O at this scale, which is why &ldquo;just disable vectorization&rdquo;...