Let's Take a Look at... Lower Java Tail Latencies With ZGC - Gunnar Morling

Gunnar Morling

Random Musings on All Things Software Engineering

Let's Take a Look at... Lower Java Tail Latencies With ZGC

Posted at Sep 17, 2025

java

performance

gc

Table of Contents

ZGC Allocation Stalls

Summary

In the "Let’s Take a Look at…​!" blog series I am exploring interesting projects, developments and technologies in the data and streaming space. This can be KIPs and FLIPs, open-source projects, services, relevant improvements to Java and the JVM, and more. The idea is to get some hands-on experience, learn about potential use cases and applications, and understand the trade-offs involved. If you think there’s a specific subject I should take a look at, let me know in the comments below.

Java 25 was released earlier this week,<br>and it is the first Java release with long-term support (LTS) which ships with Generational ZGC as the one (and only) flavor of the ZGC garbage collector.<br>ZGC itself is a relatively new concurrent collector, originally added in Java 11.

The high-level intuition on concurrent garbage collectors (another example being Shenandoah) is that they move as much of their work as possible from the application’s threads to separate GC threads.<br>That way, they essentially do away with GC pauses, which used to plague Java users in the past in the form of high tail latencies of their applications.<br>ZGC pushes down GC times in application threads down to the sub millisecond range, making GC pauses practically a non-issue for the vast majority of use cases.<br>Of course, there is no free lunch: by running the GC logic in separate threads, concurrent collectors require more CPU resources,<br>thus reducing the overall throughput of the system.

So far, I haven’t had the chance to gather some hands-on experience with ZGC yet;<br>hence, I set out to run some comparisons of ZGC and G1, which is Java’s default garbage collector since version 9.<br>Now, ZGC oftentimes is associated with large heaps of hundreds of gigabytes and beyond,<br>but I was curious whether it would also be beneficial for a typical microservice deployment with just a few gigabytes.<br>Furthermore, I was eager to learn about the performance characteristics using the default settings,<br>i.e. I’m not too interested in fine-tuning specific garbage collectors.<br>In practice, most folks don’t bother doing so for running their applications either.<br>Hardly anyone has the time or interest to find optimal settings,<br>which may be obsolete very soon anyways when details of the workload change, or a new Java version with changes to the GC behavior gets released.<br>So arguably, in most cases the performance with default settings matters more than a theoretical peak performance achievable only with highly tuned settings.

I started by benchmarking a sample microservice built using the Quarkus framework,<br>returning some data from a Postgres database.<br>Using Vegeta as a load generator,<br>I created a moderate load of 1,000 requests per second.<br>The test ran on a Hetzner CCX43 instance, using four of its 16 exclusive CPU cores and four GB of RAM.<br>Here are the request latencies from running the test for two minutes with each collector, discarding the first 30 seconds of each run to exclude any warm-up effects.<br>It’s not a super-scientific benchmark by any means, but good enough to show some interesting results (click to enlarge):

While latencies are practically identically up to the 99th percentile, the p999 and p9999 latencies show quite an advantage for ZGC.<br>Let’s try and find out whether indeed GC pauses explain the difference.<br>Examining the actual request latencies in the Vegeta plot show that there a several significant outliers with G1:

Whereas the runtimes look much more homogenous with ZGC:

In order to verify whether the G1 outliers actually were caused by GC pauses, I enabled JDK Flight Recorder while running the tests.<br>And indeed there we can observe GC pauses of more than 20 ms at the respective offsets in the JFR recording:

With ZGC on the other hand, the longest GC pause time observed is ~50 microseconds:

That’s pretty neat: solely by using ZGC as the garbage collector, we could improve tail latencies of this example service substantially, without any sort of tuning.<br>Note you may potentially get better results out of G1 too by playing with JVM options such -XX:MaxGCPauseMillis, but the much lower tail latencies you get from ZGC with default settings are what make it very appealing.<br>Results may look different for your specific workloads, but it’s definitely worth giving ZGC a try.<br>Chances are you may see some really nice benefits, without a lot of effort.

Garbage collections are note the only cause for JVM pauses.<br>Other examples include the deoptimization of compiled methods and the creation of heap dumps.<br>These, and other operations, require all threads to come to a JVM savepoint, which may take some time.<br>This post by Zac Blanco discusses potential causes for JVM...