GitHub - JeanKaddour/sokoban_speedrun: RL models to play Sokoban. The fastest recipe wins. · GitHub
/" data-turbo-transient="true" />
Skip to content
Search or jump to...
Search code, repositories, users, issues, pull requests...
-->
Search
Clear
Search syntax tips
Provide feedback
--><br>We read every piece of feedback, and take your input very seriously.
Include my email address so I can be contacted
Cancel
Submit feedback
Saved searches
Use saved searches to filter your results more quickly
-->
Name
Query
To see all available qualifiers, see our documentation.
Cancel
Create saved search
Sign in
/;ref_cta:Sign up;ref_loc:header logged out"}"<br>Sign up
Appearance settings
Resetting focus
You signed in with another tab or window. Reload to refresh your session.<br>You signed out in another tab or window. Reload to refresh your session.<br>You switched accounts on another tab or window. Reload to refresh your session.
Dismiss alert
{{ message }}
JeanKaddour
sokoban_speedrun
Public
Notifications<br>You must be signed in to change notification settings
Fork
Star<br>27
main
BranchesTags
Go to file
CodeOpen more actions menu
Folders and files<br>NameNameLast commit message<br>Last commit date<br>Latest commit
History<br>192 Commits<br>192 Commits
assets
assets
llm
llm
non_llm
non_llm
.gitignore
.gitignore
README.md
README.md
make_record_report.py
make_record_report.py
plot_track_train_solve_rate.py
plot_track_train_solve_rate.py
View all files
Repository files navigation
Sokoban Speedrun
Fastest recipes to RL models to solve Sokoban to a held-out target on one node:
LLM Track : RL-fine-tune Qwen3-4B-Instruct-2507 from 57% to >80% held-out pass@1 on one 8xH100 .
Non-LLM Track : train a from-scratch agent on a single H100 .
Play Sokoban if the task is unfamiliar.
LLM Track
World record history
Record time (mm:ss)<br>Description<br>Date<br>Log<br>score<br>Contributors
48:53<br>GRPO, LR 1.6e-6 annealed, 75 steps<br>2026-06-29<br>llm/records/2026-06-29_01<br>0.834<br>@JeanKaddour
36:53<br>steps + LR-decay horizon 75 → 60<br>2026-06-29<br>llm/records/2026-06-29_02<br>0.807<br>@dexhunter
35:29<br>earlier stop: 54 steps<br>2026-07-02<br>llm/records/2026-07-02_01<br>0.829<br>@dexhunter
33:40<br>Weco advantage shaping, 52 steps<br>2026-07-02<br>llm/records/2026-07-02_02<br>0.835<br>@dexhunter
26:27<br>rollout budget 5632 → 4800 tokens, 48 steps<br>2026-07-02<br>llm/records/2026-07-02_03<br>0.815<br>@lorenzflow
25:51<br>GRPO → CISPO, same 48-step recipe<br>2026-07-14<br>llm/records/2026-07-14_01<br>0.804<br>@lorenzflow
19:20<br>earlier stop: 35 CISPO steps<br>2026-07-15<br>llm/records/2026-07-15_01<br>0.824<br>@lorenzflow
Rules
Fastest wall-clock run wins: one run on one 8xH100 node, from training step 1 through the final training update.
Score: the lower 95% bootstrap CI of pass@1 on llm/datasets/sokoban_eval.jsonl — a record must score > 0.80 .
Eval: 8 completions/puzzle, 12,288 tokens, temperature 0.8, top-p 0.95, seed 12345.
Fixed: model, train set, eval set, reward function, hardware.
Open: RL algorithm, loss, schedules, engine, parallelism, domain-agnostic rewards, prompt.
Not allowed: Sokoban-specific hints, heuristics, or few-shot examples.
Verification: Rerun with a second seed; both runs must score above the target. The score column reports the worse of the two runs.
Running
/step_000051">cd llm<br>uv sync<br>NODE_GPUS=8 uv run torchrun --standalone --nproc_per_node=3 -m speedrun<br>uv run python -m eval_speedrun --eval-checkpoint outputs/run>/step_000051
Non-LLM Track
This track uses PufferLib's Boxoban environment; the initial PPO implementation was forked from pufferlib/torch_pufferl.py.
World record history
Record time (mm:ss)<br>Description<br>Date<br>Log<br>score<br>Contributors
22:24<br>cnn-mingru h256<br>2026-06-21<br>non_llm/records/2026-06-21_01<br>0.718<br>@JeanKaddour
21:00<br>same recipe as #1, earliest clearing checkpoint<br>2026-06-29<br>non_llm/records/2026-06-29_01<br>0.701<br>@JeanKaddour
15:55<br>torch.compile + steps-matched-anneal<br>2026-06-30<br>non_llm/records/2026-06-30_01<br>0.706<br>@JeanKaddour
14:42<br>anneal horizon tuned 1300→1200 steps<br>2026-07-02<br>non_llm/records/2026-07-02_01<br>0.709<br>@JeanKaddour
12:38<br>conv-free shift + pooled-global encoder (sgpm2), 950-step anneal<br>2026-07-02<br>non_llm/records/2026-07-02_02<br>0.715<br>@srijanpatel
Rules
Fastest wall-clock run wins: one run on a single H100, from training step 1 through the final training update.
Score: the lower 95% CI of the held-out solve rate — a record must score > 0.70 .
Eval: official DeepMind Boxoban test split unfiltered/test.
Open: policy architecture, RL algorithm, optimizer, schedules, implementation.
Verification: Rerun with a second seed; both runs must score above the target. The score column reports the worse of the two runs.
Running
cd non_llm<br>uv sync<br>uv run python speedrun.py
Submitting a record
Each track's assemble_record.sh (llm/, non_llm/) turns a finished run into a record dir: it collects the log, eval JSON, and source snapshot, builds the report, pins the top-level speedrun.py, runs verify_record.py, and adds or refreshes the record's...