[2510.18245] Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs

-->

Computer Science > Machine Learning

arXiv:2510.18245 (cs)

[Submitted on 21 Oct 2025 (v1), last revised 13 May 2026 (this version, v3)]

Title:Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs

Authors:Song Bian, Tao Yu, Shivaram Venkataraman, Youngsuk Park<br>View a PDF of the paper titled Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs, by Song Bian and 3 other authors

View PDF<br>HTML (experimental)

Abstract:Scaling the number of parameters and the size of training data has proven to be an effective strategy for improving large language model (LLM) performance. Yet, as these models grow increasingly powerful and widely deployed, the cost of inference has become a pressing concern. Despite its importance, the trade-off between model accuracy and inference efficiency remains underexplored. In this work, we examine how key architectural factors, hidden size, the allocation of parameters between MLP and attention (mlp-to-attention ratio), and grouped-query attention (GQA), influence both inference cost and accuracy. We introduce a conditional scaling law that augments the Chinchilla framework with architectural information, along with a search framework for identifying architectures that are simultaneously inference-efficient and accurate. To validate our approach, we train more than 200 models spanning 80M to 3B parameters and 8B to 100B training tokens, and fit the proposed conditional scaling law. Our results show that the conditional scaling law reliably predicts optimal architectural choices and that the resulting models outperform existing open-source baselines. Under the same training budget, optimized architectures achieve up to 2.1% higher accuracy and 42% greater inference throughput compared to LLaMA-3.2.

Comments:<br>32 pages, 27 figures

Subjects:

Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Cite as:<br>arXiv:2510.18245 [cs.LG]

(or<br>arXiv:2510.18245v3 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2510.18245

Focus to learn more

arXiv-issued DOI via DataCite

Journal reference:<br>ICLR 2026

Submission history<br>From: Song Bian [view email]<br>[v1]<br>Tue, 21 Oct 2025 03:08:48 UTC (236 KB)

[v2]<br>Sun, 1 Mar 2026 01:23:07 UTC (633 KB)

[v3]<br>Wed, 13 May 2026 04:16:31 UTC (633 KB)

Full-text links:<br>Access Paper:

View a PDF of the paper titled Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs, by Song Bian and 3 other authors<br>View PDF<br>HTML (experimental)<br>TeX Source

view license

Current browse context:

cs.LG

next >

new<br>recent<br>| 2025-10

Change to browse by:

cs<br>cs.AI

References & Citations

NASA ADS<br>Google Scholar

Semantic Scholar

export BibTeX citation<br>Loading...

BibTeX formatted citation

&times;

loading...

Data provided by:

Bookmark

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower (What are Influence Flowers?)

Core recommender toggle

CORE Recommender (What is CORE?)

IArxiv recommender toggle

IArxiv Recommender<br>(What is IArxiv?)

Author

Venue

Institution

Topic

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

Which authors of this paper are endorsers? |<br>Disable MathJax (What is MathJax?)