[2605.31464] GPU Forecasters: Language Models as Selective Surrogates for Kernel Runtime Optimization
-->
Computer Science > Machine Learning
arXiv:2605.31464 (cs)
[Submitted on 29 May 2026]
Title:GPU Forecasters: Language Models as Selective Surrogates for Kernel Runtime Optimization
Authors:Zaid Khan, Justin Chih-Yao Chen, Jaemin Cho, Elias Stengel-Eskin, Mohit Bansal<br>View a PDF of the paper titled GPU Forecasters: Language Models as Selective Surrogates for Kernel Runtime Optimization, by Zaid Khan and Justin Chih-Yao Chen and Jaemin Cho and Elias Stengel-Eskin and Mohit Bansal
View PDF
Abstract:GPU kernels are the workhorse of modern deep learning, and optimizing them (via evolutionary search or coding agents) usually requires repeated measurement on target hardware. While these measurements provide the ground-truth signal necessary for kernel search, they are costly, because each evaluation of a kernel requires compilation and repeated execution on a GPU. As improvements in LLM inference reduce the cost of writing novel kernels and LLM-driven searches scale to large search budgets, on-device evaluation becomes a bottleneck. To address this, we study how LLMs can serve as selective GPU surrogates for kernel evaluation, by forecasting the performance of proposed kernels. A useful surrogate should be accurate, and it should be selective, by knowing when it could be wrong, and deferring to the GPU. To evaluate surrogates, we measure whether their forecasts are accurate, calibrated, and practically useful for recovering fast kernels under limited GPU-measurement budgets. Next, we study whether reinforcement learning can improve forecast accuracy and confidence calibration. Our experiments demonstrate that LLMs can accurately forecast relative kernel performance, that their utility can be improved through reinforcement learning. Used inside a kernel search, the surrogate lets the search consider several times as many candidates under the same GPU evaluation budget, and that leads to finding faster kernels than an equal-budget baseline. These results suggest that LLMs can play a broader role in kernel optimization, by acting as virtual models of a GPU rather than solely as kernel generators for search.
Comments:<br>Code: this https URL
Subjects:
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Cite as:<br>arXiv:2605.31464 [cs.LG]
(or<br>arXiv:2605.31464v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2605.31464
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history<br>From: Zaid Khan [view email]<br>[v1]<br>Fri, 29 May 2026 15:56:08 UTC (1,342 KB)
Full-text links:<br>Access Paper:
View a PDF of the paper titled GPU Forecasters: Language Models as Selective Surrogates for Kernel Runtime Optimization, by Zaid Khan and Justin Chih-Yao Chen and Jaemin Cho and Elias Stengel-Eskin and Mohit Bansal<br>View PDF<br>TeX Source
view license
Current browse context:
cs.LG
next >
new<br>recent<br>| 2026-05
Change to browse by:
cs<br>cs.AI
References & Citations
NASA ADS<br>Google Scholar
Semantic Scholar
export BibTeX citation<br>Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
IArxiv recommender toggle
IArxiv Recommender<br>(What is IArxiv?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .
Which authors of this paper are endorsers? |<br>Disable MathJax (What is MathJax?)