GitHub - digailab/PCB-Bench: Official implementation of PCB-Bench (ICLR 2026): A benchmark for evaluating LLMs on printed circuit board placement and routing reasoning. · GitHub
/" data-turbo-transient="true" />
Skip to content
Search or jump to...
Search code, repositories, users, issues, pull requests...
-->
Search
Clear
Search syntax tips
Provide feedback
--><br>We read every piece of feedback, and take your input very seriously.
Include my email address so I can be contacted
Cancel
Submit feedback
Saved searches
Use saved searches to filter your results more quickly
-->
Name
Query
To see all available qualifiers, see our documentation.
Cancel
Create saved search
Sign in
/;ref_cta:Sign up;ref_loc:header logged out"}"<br>Sign up
Appearance settings
Resetting focus
You signed in with another tab or window. Reload to refresh your session.<br>You signed out in another tab or window. Reload to refresh your session.<br>You switched accounts on another tab or window. Reload to refresh your session.
Dismiss alert
{{ message }}
Uh oh!
There was an error while loading. Please reload this page.
digailab
PCB-Bench
Public
Notifications<br>You must be signed in to change notification settings
Fork
Star<br>13
main
BranchesTags
Go to file
CodeOpen more actions menu
Folders and files<br>NameNameLast commit message<br>Last commit date<br>Latest commit
History<br>32 Commits<br>32 Commits
Task1-Text-Text-QA_evaluation
Task1-Text-Text-QA_evaluation
Task2-Image-Text-QA_and_CQ_evaluation
Task2-Image-Text-QA_and_CQ_evaluation
Task3-PCB_design_understanding
Task3-PCB_design_understanding
figs
figs
README.md
README.md
index.html
index.html
View all files
Repository files navigation
PCB-Bench
PCB-Bench: Benchmarking LLMs for Printed Circuit Board Placement and Routing (ICLR 2026)
📄 [OpenReview]: https://openreview.net/forum?id=Q5QLu7XTWx&referrer
🌐 [Project Page]: https://digailab.github.io/PCB-Bench/
PCB-Bench is the first comprehensive benchmark designed to systematically evaluate (multimodal) large language models (LLMs/MLLMs) in the context of PCB placement and routing . It addresses the lack of standardized benchmarks and high-fidelity datasets for real-world PCB engineering reasoning by integrating text , images , and real PCB design artifacts into a unified evaluation framework.
What's Included
PCB-Bench spans three complementary task settings and corresponding datasets:
1) Text-based reasoning (Text-to-Text QA & CQ)
~1,800 expert-written free-form QA instances
Each QA has a corresponding single-choice question (CQ) version
Total ~3,700 questions (QA + CQ)
Covers component placement , routing strategies , and design rule compliance
Covers both macro-level (global design principles) and micro-level (fine-grained implementation details), across placement and routing, with topic labels (e.g., signal integrity, EMI/EMC, power planning, differential pairs, DFM, etc.).
2) Multimodal image-text reasoning (Image-and-Text QA/CQ)
~500 problems requiring joint interpretation of PCB layout images + technical prompts
Includes choice questions , cloze-style fill-in-the-blank , and free-form QA
Covers visual-semantic subtasks such as component identification, functional block recognition, trace reasoning, via presence checking, differential-pair continuity analysis, etc.
3) Real-world PCB design comprehension (PCB Design Understanding)
174 complete real-world PCB projects collected from OSHWHub (operated by JLCPCB) (https://oshwhub.com/)
Each design includes artifacts such as schematics , placement/routing files , design descriptions , component libraries , and EDA software screenshots
Task setting: given a standalone EDA editor screenshot (no extra text/schematic provided), models generate a free-form description of the board’s function/structure/application scenario, assessing structured visual interpretation of professional PCB artifacts.
Task Formulation
PCB-Bench is organized into three task settings aligned with real engineering workflows:
Task 1: Text-to-Text QA & CQ
Evaluate PCB placement/routing knowledge via both open-ended generation and objective multiple-choice selection.
Task 2: Image-and-Text Multimodal QA & CQ
Answer questions based on PCB layout images together with textual prompts.
Task 3: PCB Design Understanding (Screenshot-to-Description)
Describe full-board PCB screenshots from EDA tools using free-form functional/structural descriptions.
Evaluation Protocol
All models are evaluated under a unified zero-shot setting across tasks (each instance is answered independently, without demonstrations or fine-tuning).
Metrics
Choice Questions (CQ): Top-1 Accuracy
Free-form QA: BERTScore and Sentence-BERT (SBERT) similarity for semantic consistency with reference answers
Task 3 (Design Understanding): additionally report Precision / Recall / F1-score to capture complementary aspects of prediction quality
Models Evaluated in the Paper
The paper benchmarks a diverse...