What broke when I stopped trusting my own validator — InvoiceHubEngineering · July 2026<br>What broke when I stopped trusting my own validator<br>Real bugs, real commits — what building an independent validation harness actually found in InvoiceHub's own EN 16931 engine.<br>I build InvoiceHub — an API that validates, generates, and converts EU e-invoices: UBL 2.1, UN/CEFACT CII, Germany's XRechnung, the Factur-X hybrid PDF format. Every document it hands back has already been checked against the official CEN EN 16931 Schematron. That's the entire pitch, really: you get pre-validated output, not “probably fine” output.<br>Here's the part of that pitch I hadn't sat with properly: the code that generates a document and the code that validates it live in the same package, written by the same person, on the same afternoons. If I get one assumption wrong about, say, how XRechnung wants its settlement block structured, there's a decent chance I get it wrong the same way in both places. The validator doesn't catch the generator's mistake — it shares it. And it'll keep passing every document I throw at it, because I'm not in the business of generating documents specifically designed to break my own rules.<br>So over the last couple of days I built a validation harness whose entire job is to stop me from grading my own homework: run real output through tools I didn't write, and see whether they agree with me. A few things came out of that which are more interesting than “everything passed eventually,” so here's the honest version, bugs included.<br>First, a number I'd had wrong for a while<br>Before reaching for any external tool, I wanted to answer something simpler: does the Schematron I've got compiled actually contain every rule from the official CEN artifact, or some subset that happens to look complete? I already had a regression test for this — a hardcoded count of distinct rule IDs baked into the compiled ruleset, 955, meant to catch the day someone (me) accidentally ships a truncated build.<br>Turns out the regex behind that count was itself incomplete. It matched BR-* and UBL-CR-*/UBL-SR-* rule identifiers but missed UBL-DT-* datatype rules entirely — and the official conformance corpus tests those directly (UBL-DT-01.xml, UBL-DT-06.xml, UBL-DT-07.xml are real files in it). Widening the pattern brought the true count to 979. The old baseline wasn't catching a regression for two years; it was quietly undercounting by 24 rules the whole time. Not dramatic, but exactly the kind of thing you only find by going and building the thing that's supposed to find it.<br>The bug I found by staring at a suspiciously round zero<br>Building the coverage report itself hit a stranger problem first. Saxon-JS — the engine that actually runs the Schematron transforms — mutates its own compiled stylesheet object the first time you use it, attaching live parentNode references for its internal tree. That's fine at runtime. It is not fine if, later in the same process, you call JSON.stringify() on that same object to count rule IDs in it, because it's now circular and JSON.stringify just throws.<br>My rule-counting function had always called JSON.stringify on demand and never once broken — because I'd never called it after running an actual transform in the same process, until I put a coverage-report test in the same file as 1,161 corpus tests that exercise the engine first. Fix was boring once I found it: snapshot the JSON text once at module load, before Saxon-JS has touched anything, and count against that frozen string instead of the live object.<br>Right after that fix, a second bug showed up immediately, and it's the one I'm least proud of. The official CEN test corpus isn't a folder of standalone invoices — each file is a wrapper containing several blocks, each one a deliberately partial invoice fragment plus an assertion about which single rule it's meant to trip. To report which rules the corpus actually exercises, I parse a tag out of each file. My header-boundary search was xml.indexOf(' — which is also, of course, a prefix of , the very first tag in the file. So it found the wrong occurrence every single time, truncated the search before it ever reached , and returned nothing. Silently. On all 279 files.<br>Nothing crashed. A test just printed rules_exercised_by_corpus=0 for a corpus I'd already run 1,128 real assertions against successfully — and the only reason I caught it is that zero looked too clean to be true.<br>What the “official” corpus does and doesn't actually cover<br>With both of those fixed, the real numbers: my UBL Schematron has 979 distinct rules, of which CEN's own corpus exercises 201. My CII Schematron has 806 rules, of which the same corpus exercises two . That's not my engine's fault — CEN's corpus is overwhelmingly built out of UBL fixtures, and it's an honest gap in what an “official” test suite actually proves, not something I get to paper over just because it's not mine. It's also the reason the next problem didn't show up in this corpus...