Balancing Playwright Test Shards | Flakiness.io BlogGo back<br>Balancing Playwright Test Shards<br>Andrey Lushnikov·3 Aug, 2026
Summary
WordPress’s Gutenberg project cut its Playwright test runtime by 34% — from 35 minutes to 23 — simply by balancing its test shards. With Playwright 1.62, you can now apply the same technique to your own test suite.
Table of contents
Open Table of contents
Before and after
Getting started
Generating historical test-duration data
1. Simple but slow
2. Fast but involved
3. Fast and simple
What you should see
Debugging
Wrapping up
Before and after
WordPress/Gutenberg is the project behind the WordPress block editor. It’s a large, actively developed open-source codebase that runs about 2,000 Playwright tests on every pull request.
The tests were split across eight shards, but the end-to-end run still took about 35 minutes on average. Here’s a typical example:
Based on the work across all shards, the theoretical minimum was about 22 minutes .
After the change was merged, duration-based shard balancing became the default for Gutenberg’s end-to-end test runs. They now complete in about 23 minutes on average — just one minute above that minimum.
Here’s a typical example:
Getting started
Playwright 1.62 shipped a new low-level API that allows custom reporters to assign tests to shards.
The @flakiness/playwright reporter now uses this API to implement shard balancing based on historical test-duration data. The reporter happily coexists with your existing reporters, so you can use it solely to balance shards.
Free and open source
@flakiness/playwright is self-contained and MIT-licensed. You do not need a Flakiness.io account to use the reporter.
Under the hood
The algorithm uses longest-processing-time-first scheduling and supports all Playwright suite settings, including parallel and serial modes and the fullyParallel option. Its source code is well tested, well documented, and available on GitHub.
To balance your shards, follow these steps:
Use Playwright Test 1.62 or later and Node.js 20 or later.
Install and configure @flakiness/playwright as one of your reporters.
The reporter is a regular npm package:
npm install -D @flakiness/playwright<br>Next, add it to the reporter array in playwright.config.ts:
import { defineConfig } from "@playwright/test";
export default defineConfig({<br>reporter: [<br>["@flakiness/playwright"],<br>],<br>});
Generate test-duration data and store it in a timings.json file. There are a few ways to do this, covered below.
Configure @flakiness/playwright to use the test-duration file for shard balancing.
In your playwright.config.ts, add the following to the reporter options:
import { defineConfig } from "@playwright/test";
export default defineConfig({<br>reporter: [<br>"@flakiness/playwright",<br>shardBalancing: { timingsFile: "./timings.json" },<br>},<br>],<br>],<br>});
That’s it! Now, whenever you run Playwright with --shard, the shard balancing algorithm will distribute tests among shards using the historical durations stored in timings.json.
Generating historical test-duration data
To balance shards efficiently, we need historical test-duration data. In this guide, we’ll store this information in a timings.json file.
Tip
timings.json should be committed to the repository — treat it similarly to package-lock.json. For large projects, timings.json can reach 500 KB in size, but it compresses extremely well and won’t add much storage overhead to your Git repository.
There are three ways to generate this data, pick one that suits you best. All three use the flakiness-playwright-timings utility bundled with the Flakiness reporter.
1. Simple but slow
The easiest approach is to run the full test suite and generate a report:
npx playwright test<br>The Flakiness reporter will automatically generate a Flakiness JSON report and save it to ./flakiness-report/report.json. This file stores all the information about your test run, including test durations.
Did you know?
These Flakiness reports can be viewed as rich, interactive HTML reports:<br>npx flakiness show ./flakiness-report/report.json
This report can be used directly as shardBalancing.timingsFile, but it contains much more than test durations.
It’s better to distill it:
# Build timings.json containing test durations only<br>npx flakiness-playwright-timings build -o timings.json ./flakiness-report/report.json<br>This will create a new timings.json file that contains only test names and their durations.
While this method works, it requires you to run all your tests, which might take hours on your local machine.
2. Fast but involved
Why run all your tests locally if your CI pipeline already runs them across multiple shards?
Instead, we can configure CI to upload all the flakiness reports as artifacts. We can then download them and use the flakiness-playwright-timings utility to combine them into a single timings.json file.
For example, if you’ve downloaded the reports from all your shards as...