PDF Field Detection: What Broke Across 23 Real Forms | pdf2fill

The result

0.8288 micro F1, with a 0.7018 document average

Across 23 documents, the answer key contains 2,130 fields. The detector returned 1,837 predictions: 1,644 true positives, 193 false positives, and 486 false negatives. A match required at least 0.50 intersection over union (IoU) on the same page.

Precision0.8949 1,644 / 1,837 predictions matched

Recall0.7718 1,644 / 2,130 annotated fields found

Micro F10.8288 harmonic mean from pooled counts

Macro F10.7018 mean of 23 per-document F1 scores

Government and tax subset0.5557 F1

Seven documents, 324 annotated fields, 162 matches, 97 false positives, and 162 misses. Comb conventions, boxed blanks, and other page geometry all contribute. On two measured comb cases, correcting the answer key from one field per cell to one usable field per comb changed HMRC from 0.90 to 0.51 F1 and IR330 from 0.10 to 0.12. Those two shifts don't explain the whole subset.

Micro F1 pools every field, so a 356-field document can move it sharply. Macro F1 gives each document equal weight. The 0.1270 gap between them is the warning label: performance varies more by document than the headline number suggests.

Method

What counted as a detected field

We drew a rectangle around each fillable region on the pages a user would complete. Instruction-only pages weren't annotated. The evaluator paired predictions and truth rectangles one to one on the same page, then counted a pair as a match when its IoU reached 0.50.

IoU is the overlapping area divided by the combined area. A detection can be in roughly the right place and still miss when its rectangle is too large, too small, or split differently.

The score is geometry-only. Field type is ignored, and F1 is the harmonic mean of precision and recall. We omit type accuracy because eight documents still carry generic, unreviewed type labels.

The corpus is a committed regression set rather than an unseen test. 20 documents are development documents. The other three form a committed validation subset, which means they are visible to the test gate. A separate unseen holdout exists outside the repository, but it hasn't been scored for this article.

Several official PDFs were flattened before annotation. In normal use, pdf2fill reads existing AcroForm fields directly when they are present. This benchmark removes that easier path and measures only the fallback detector for digital pages made from text, lines, vector shapes, and embedded raster elements.

Failure map

A blank line can use six visual primitives

A person sees a place to type. A PDF may encode that place as underscores, a horizontal rule, a bordered rectangle, nine adjacent comb cells, a box nested inside a larger border, or a font glyph that only looks like a checkbox.

Four abstract field geometries from the test work. These drawings contain no source-form text or imagery.

Rules and underscores were the easy half

The original detector was strongest on underscore runs and ordinary ruled blanks. Documents built around those marks commonly scored from the high 0.70s into the 0.90s. That success didn't transfer cleanly to bordered cells and combs.

Some checkboxes were hidden inside bigger contours

An early contour pass kept only the outside edge of a connected shape. Small checkboxes nested inside a form border disappeared before classification. Reading the contour hierarchy recovered 46 true positives on the development documents, with no new false positives, in a round that moved overall F1 from 0.7312 to 0.7510.

Higher resolution couldn't turn a diamond into a square

One 356-field patient intake document contained 251 checkbox annotations. The raster checkbox pass found none at 150, 225, 300, or 450 DPI. At 300 DPI, an isolated diamond covered 60.9% of its bounding box, below the square detector's 85% extent floor. Of the 251 checkbox annotations, 248 sat on lozenge characters in the text layer.

An allow-listed glyph pass later recovered all 251 on that targeted document, plus 32 more across two other documents. The full-corpus change added 283 true positives and four false positives, with no per-document F1 regression. That document was a development case, so it isn't independent evidence of generalization.

Comb fields forced an annotation decision

A row of nine character cells can be annotated as nine boxes or as one field with nine positions. The product needs the second version: one place to type the full value. We changed the answer-key convention to one field per contiguous comb run, even though the detector still emitted individual cells. That made the answer key more useful and made the weak score more visible.

Labels can live inside the box

Some intake forms print a caption inside the same bordered area where the answer belongs. A filter designed to reject boxes containing text then rejects the input itself. The geometry alone can't reliably tell whether the text is a label, a...