[2608.05446] EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents
Skip to main content
Search arXiv
Press Enter to search · Advanced search
-->
Computer Science > Machine Learning
arXiv:2608.05446 (cs)
[Submitted on 5 Aug 2026]
Title:EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents
Authors:Xuying Ning, Dongqi Fu, Tianxin Wei, Hanqing Zeng, Yuanchen Bei, Bingxuan Li, Zihao Li, Qifan Wang, Xiang Shen, Yifan Wu, Jiayi Liu, Hong Li, Yinglong Xia, Xiangjun Fan, Hanghang Tong, Jingrui He<br>View a PDF of the paper titled EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents, by Xuying Ning and 15 other authors
View PDF<br>HTML (experimental)
Abstract:Long-horizon LLM agents increasingly rely on external execution support to maintain state, track progress, invoke tools, verify outcomes, and reuse experience across interactions. However, effective harness use raises two coupled challenges: state formation from noisy interaction traces and runtime control over external-state access. Existing agents usually handle both through prompts, heuristics, or domain-specific conventions, leaving the external workspace and its usage policy manually engineered. To address this, we study the problem of harness policy learning, where agents learn harness policies offline and deploy them to construct and update external harness state online during runtime task execution. We introduce EvoHarness-RL, which exposes Belief, Progress, and Experience (BPE) as policy-facing harness state. Supervised harness fine-tuning teaches the base agent the harness action space and how to construct useful external state, while cost-aware GRPO explores coordination policies to selectively read, update, and consolidate that state during long-horizon interaction. Instantiated on ALFWorld with a Qwen3-8B LLM, EvoHarness-RL reaches 96.9% success and reveals two key dynamics: harness annealing, where training internalizes recurring harness-use patterns into the model policy and shifts the agent from frequent harness calls toward selective external-state access, and harness evolution, where progress updates and experience consolidation refine the harness into a compact, task-adaptive state substrate. These results suggest that long-horizon agents benefit from trainable policies for constructing and coordinating with external harness workspaces, beyond simply adding stronger tools or larger memories.
Comments:<br>Accepted to LLA@COLM 2026
Subjects:
Machine Learning (cs.LG); Computation and Language (cs.CL)
Cite as:<br>arXiv:2608.05446 [cs.LG]
(or<br>arXiv:2608.05446v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2608.05446
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history<br>From: Xuying Ning [view email]<br>[v1]<br>Wed, 5 Aug 2026 22:29:20 UTC (1,730 KB)
Full-text links:<br>Access Paper:
View a PDF of the paper titled EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents, by Xuying Ning and 15 other authors<br>View PDF<br>HTML (experimental)<br>TeX Source
view license
Current browse context:
cs.LG
next >
new<br>recent<br>| 2026-08
Change to browse by:
cs<br>cs.CL
References & Citations
NASA ADS<br>Google Scholar
Semantic Scholar
export BibTeX citation<br>Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
IArxiv recommender toggle
IArxiv Recommender<br>(What is IArxiv?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that...