PatentScore: Multi-dimensional Evaluation of LLM-Generated Patent Claims - ACL AnthologyPatentScore: Multi-dimensional Evaluation of LLM-Generated Patent Claims<br>Yongmin Yoo,<br>Qiongkai Xu,<br>Longbing Cao
Correct Metadata for
Use this form to create a GitHub issue with structured data describing the correction. You will need a GitHub account.<br>Once you create that issue, the correction will be reviewed by a staff member.<br>⚠️ Mobile Users: Submitting this form to create a new issue will only work with github.com, not the GitHub Mobile app.<br>Important : The Anthology treat PDFs as authoritative. Please use this form only to correct data<br>that is out of line with the PDF. See our corrections<br>guidelines if you need to change the PDF.<br>Title<br>Adjust the title. Retain tags such as
Authors<br>Adjust author names and order to match the<br>PDF.<br>Add AuthorAbstract<br>Correct abstract if needed. Retain XML formatting tags such as . You may use ... for bold , ... for italic, and ... for URLs.
Verification against PDF<br>Ensure that the new title/authors match the snapshot below. (If there<br>is no snapshot or it is too small, consult the PDF.)<br>Authors concatenated from the text boxes above:
ALL author names match the snapshot above—including<br>middle initials, hyphens, and accents.<br>Create GitHub issue for staff review
Abstract<br>High-stakes texts such as patent claims, medical records, and technical reports are structurally complex and demand a high degree of reliability and precision. While large language models (LLMs) have recently been applied to automate their generation in high-stakes domains, reliably evaluating such outputs remains a major challenge. Conventional natural language generation (NLG) metrics are effective for generic documents but fail to capture the structural and legal characteristics essential to evaluating complex high-stakes documents. To address this gap, we propose PatentScore, a multi-dimensional evaluation framework specifically designed for one of the most intricate and rigorous domains, patent claims. PatentScore integrates hierarchical decomposition of claim elements, validation patterns grounded in legal and technical standards, and scoring across structural, semantic, and legal dimensions. In experiments on our dataset which consists of 400 Claim1, PatentScore achieved the highest correlation with expert annotations (r = 0.819), significantly outperforming widely used NLG metrics. This work establishes a new standard for evaluating LLM-generated patent claims, providing a solid foundation for research on patent generation and validation.
Anthology ID:2025.emnlp-main.1564Volume:Proceedings of the 2025 Conference on Empirical Methods in Natural Language ProcessingMonth:NovemberYear:2025Address:Suzhou, ChinaEditors:Christos Christodoulopoulos,<br>Tanmoy Chakraborty,<br>Carolyn Rose,<br>Violet PengVenue:EMNLPSIG:Publisher:Association for Computational LinguisticsNote:Pages:30727–30746Language:URL:https://aclanthology.org/2025.emnlp-main.1564/DOI:10.18653/v1/2025.emnlp-main.1564Bibkey:yoo-etal-2025-patentscoreCite (ACL):Yongmin Yoo, Qiongkai Xu, and Longbing Cao. 2025. PatentScore: Multi-dimensional Evaluation of LLM-Generated Patent Claims. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 30727–30746, Suzhou, China. Association for Computational Linguistics.Cite (Informal):PatentScore: Multi-dimensional Evaluation of LLM-Generated Patent Claims (Yoo et al., EMNLP 2025)Copy Citation:BibTeX<br>Markdown<br>MODS XML<br>Endnote<br>More<br>options…PDF:https://aclanthology.org/2025.emnlp-main.1564.pdfChecklist:<br>2025.emnlp-main.1564.checklist.pdf<br>PDF<br>Cite<br>Search
Checklist
Fix data
Export citation
BibTeX<br>MODS XML<br>Endnote<br>Preformatted<br>@inproceedings{yoo-etal-2025-patentscore,<br>title = "{P}atent{S}core: Multi-dimensional Evaluation of {LLM}-Generated Patent Claims",<br>author = "Yoo, Yongmin and<br>Xu, Qiongkai and<br>Cao, Longbing",<br>editor = "Christodoulopoulos, Christos and<br>Chakraborty, Tanmoy and<br>Rose, Carolyn and<br>Peng, Violet",<br>booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",<br>month = nov,<br>year = "2025",<br>address = "Suzhou, China",<br>publisher = "Association for Computational Linguistics",<br>url = "https://aclanthology.org/2025.emnlp-main.1564/",<br>doi = "10.18653/v1/2025.emnlp-main.1564",<br>pages = "30727--30746",<br>ISBN = "979-8-89176-332-6",<br>abstract = "High-stakes texts such as patent claims, medical records, and technical reports are structurally complex and demand a high degree of reliability and precision. While large language models (LLMs) have recently been applied to automate their generation in high-stakes domains, reliably evaluating such outputs remains a major challenge. Conventional natural language generation (NLG) metrics are effective for generic documents but fail to capture the structural and legal characteristics essential to evaluating complex high-stakes documents. To address this gap, we propose PatentScore, a multi-dimensional...