Keeping 170+ Components Accessible: How We Made Atomic’s VPAT a Build Artifact
✨ Watch the masterclass now : GenAI vs. Agentic AI: which is best for you?
Coveo Labs<br>Keeping 170+ Components Accessible: How We Made Atomic’s VPAT a Build Artifact
Re-testing 170+ components for accessibility on every release would be prohibitively expensive—so we asked AI to do it instead.
By Yassine Lakhdar
Jul 14, 2026<br>12 min read
Get the latest articles delivered to you.
Yes! I want to receive communications about educational resources, product news, upcoming events, and services from Coveo Solutions Inc. and affiliates. I understand that I can unsubscribe at any time.
Subscribe
Coveo Atomic is our open-source library of web components for building Coveo-powered search interfaces. It has 170+ components, and we ship new versions constantly. Keeping all of them accessible is one challenge. Proving it, on every release, in a document a procurement team will read, is another. That’s what this post is about.<br>That document is called an Accessibility Conformance Report (ACR). If you sell software to a large enterprise or a public-sector buyer, someone will eventually ask for one: a structured report showing how your product holds up against WCAG, Section 508 in the US, or EN 301 549 in Europe.<br>Note:<br>“VPAT” and “ACR” aren’t the same thing. A VPAT (Voluntary Product Accessibility Template) is the empty form, published by ITI. Fill it in with your product’s results and the finished document is an Accessibility Conformance Report (ACR), the version you hand to a buyer.<br>Most people say “VPAT” for both. I’ll say “ACR” when I mean the completed report from here on.
The usual way to produce an ACR is to bring in a third party, run an audit once, and publish a PDF. That PDF is accurate on the day it’s signed, but every release that touches UI without re-testing may drift a little further from reality. For a library that ships weekly, that drift adds up fast. So we built a pipeline that produces the ACR the same way we produce any other build artifact: from real test results, checked by CI, and republished to our CDN on every release.<br>We also tried to hand the hardest part, the judgment calls automated tools can’t make, to AI. That experiment didn’t survive, and scrapping it shaped everything else. More on that below.<br>The Cost Problem<br>Re-testing 170+ components for accessibility on every release would be prohibitively expensive. So we made the report a by-product of the build instead: the accessibility tests already run in CI on every change, and the ACR is generated from their results. Keeping it up-to-date costs nothing extra.<br>Automation carries everything it can. A person steps in only for the criteria a machine can’t judge.<br>The Sweet Spot: Three Signals and a Worst-Wins Rule<br>Automated accessibility testing has a ceiling. It can verify a large portion of the WCAG success criteria: axe-core will reliably flag a missing alt attribute or text that fails a contrast ratio. But it can’t tell you whether there’s a keyboard trap three steps into a flow, or whether a screen reader announcement makes sense to the person hearing it.<br>So full automation isn’t realistic, and a fully manual audit doesn’t scale to the entire list of components. The useful work sits in between. We resolve each criterion from up to three signals, with one override on top:<br>Static (axe-core) : Catches the low-hanging fruit at scale. We run axe-core against every Storybook story to flag structural issues (missing roles, broken labels, contrast failures) across all components with zero human effort.<br>Interactive : Verifies keyboard behavior axe-core can’t see. These tests are automated too; the difference is that they drive the component. Scripted checks follow the WAI-ARIA Authoring Practices Guide (APG), the W3C’s catalog of how each UI pattern should behave, to confirm components actually work the way a keyboard user expects.<br>Manual : Covers the criteria that need human judgment, things like whether a screen reader announcement makes sense, or whether a flow is understandable without vision. Results are stored in a single JSON file, not scattered per component.<br>AI (dropped): We tried having a model judge the criteria axe-core can’t reach. It would have increased our maintenance load and made our CI slower and non-deterministic, so we dropped it. The full story is below.<br>Overrides : Lets engineering document intentional, by-design exceptions with a written reason. Used sparingly for cases where a criterion does not technically apply to the library.<br>An override always wins, and it always carries a written reason. When there’s no override, the verdict is the worst of whatever signals we have:<br>┌──────────────────────────────────────────────────────────┐<br>│ How we resolve each criterion: │<br>│ │<br>│ 1. Override exists? → use it (authoritative) │<br>│ 2. Otherwise, take the → worst result wins: │<br>│ worst across: │<br>│ • manual audit │<br>│ • interactive test │<br>│ • static...