[2608.19197] SPADE: Self-Play in Adaptive Synthetic Executable Environments

Skip to main content

Search arXiv

Press Enter to search · Advanced search

-->

Computer Science > Computation and Language

arXiv:2608.19197 (cs)

[Submitted on 19 Aug 2026]

Title:SPADE: Self-Play in Adaptive Synthetic Executable Environments

Authors:Bo Liu, Simon Yu, Yiding Jiang, Ao Qu, Andrew Zhao, Zichen Liu, Junsu Kim, Zijian Zhou, Seungone Kim, Tongzheng Ren, Mickel Liu, Hanfei Yu, Zhaorun Chen, Weiyan Shi, Paul Pu Liang, Luke Zettlemoyer, Yejin Choi, Natasha Jaques<br>View a PDF of the paper titled SPADE: Self-Play in Adaptive Synthetic Executable Environments, by Bo Liu and Simon Yu and Yiding Jiang and Ao Qu and Andrew Zhao and Zichen Liu and Junsu Kim and Zijian Zhou and Seungone Kim and Tongzheng Ren and Mickel Liu and Hanfei Yu and Zhaorun Chen and Weiyan Shi and Paul Pu Liang and Luke Zettlemoyer and Yejin Choi and Natasha Jaques

View PDF<br>HTML (experimental)

Abstract:Continuous self-improvement requires an ever-expanding pool of self-generated, diverse, adaptive goals. For language agents, existing training environment pools (hand-curated, statically synthesized, or frozen-verifier) keep the goal distribution fixed as the learner scales. We introduce SPADE (Self-Play in Adaptive Synthetic Executable Environments), a self-play RL framework in which a single LLM plays two roles: an Environment Designer that writes complete, long-horizon training environments as executable code with an OpenAI Gym-style reset()/step() interface, and a Reasoning Agent that learns to act in them. Each is a stateful, multi-turn environment (state transitions, reward functions, and verification code), so one interface spans reasoning problems and multi-step agentic tool use. The Reasoning Agent's regret is estimated using the gap between its reward with and without privileged hints; in optimizing this regret signal the Environment Designer learns to target environments at the edge of the agent's capabilities while keeping them feasible. Through extensive experimentation, we find several components critical to success: grounding the Environment Designer on documents sampled from a large pretraining corpus, and giving it an accumulated environment memory. Scaling to 30B-parameter models, SPADE improves over the strongest fixed-environment baseline by +5.3 on average across eight held-out math, science, code, and reasoning benchmarks, and lifts the tool-use setting by +5.7 on BFCL-v4 multi-turn and +13.9 on ACEBench-Agent; on the games setting, the margin over the strongest baseline grows with model scale. By making environment design itself a learnable component, SPADE takes a concrete step toward open-ended self-improvement.

Comments:<br>Work in progress. Project page: this https URL ; Code: this https URL

Subjects:

Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

Cite as:<br>arXiv:2608.19197 [cs.CL]

(or<br>arXiv:2608.19197v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2608.19197

Focus to learn more

arXiv-issued DOI via DataCite (pending registration)

Submission history<br>From: Bo Liu [view email]<br>[v1]<br>Wed, 19 Aug 2026 17:58:56 UTC (2,026 KB)

Full-text links:<br>Access Paper:

View a PDF of the paper titled SPADE: Self-Play in Adaptive Synthetic Executable Environments, by Bo Liu and Simon Yu and Yiding Jiang and Ao Qu and Andrew Zhao and Zichen Liu and Junsu Kim and Zijian Zhou and Seungone Kim and Tongzheng Ren and Mickel Liu and Hanfei Yu and Zhaorun Chen and Weiyan Shi and Paul Pu Liang and Luke Zettlemoyer and Yejin Choi and Natasha Jaques<br>View PDF<br>HTML (experimental)<br>TeX Source

view license

Current browse context:

cs.CL

next >

new<br>recent<br>| 2026-08

Change to browse by:

cs<br>cs.AI

References & Citations

NASA ADS<br>Google Scholar

Semantic Scholar

export BibTeX citation<br>Loading...

BibTeX formatted citation

&times;

loading...

Data provided by:

Bookmark

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower (What are Influence Flowers?)

Core recommender toggle

CORE Recommender (What is...