What Broke & Why: Practical Lessons from Reinforcement Learning Post-Training | Zenodo
Skip to main
You are using an outdated browser. Please upgrade your browser to improve your experience.
Published July 1, 2026
| Version 1.0
Book
Open
What Broke & Why: Practical Lessons from Reinforcement Learning Post-Training
Authors/Creators
Verma, Luv
Description
A practitioner's field guide to reinforcement learning post-training on small GPU budgets (one to eight card, H100-80GB). Written failure-first: each lesson comes from a training run that broke in a specific way, with every load-bearing number tied to a logged evaluation condition or artifact rather than asserted.
The book is organized in three layers. The Journey is the sequence of programs actually run — math, search, entropy, mixture-of-experts, SWE-bench, and distillation — each of which failed in a way that forced the next. The Science is what those runs proved about how RL changes a model. The Reference is a symptom-indexed field guide for debugging a failing run, including a catalog of failure modes with their fixes.
Topics include verifiable-reward RL, reward design and reward hacking, entropy collapse and training stability (Clip-Cov, GSPO), correctness-gated rewards, MoE routing under RL, evaluation discipline and small-eval noise, the rollout engine, and disaggregated inference. The work is independent and grounded throughout in real training logs and experimental artifacts.
Files
RL_Book.pdf
Files<br>(3.1 MB)
Name<br>Size
Download all
RL_Book.pdf
md5:901bb271a46580b279b4164be177f6a6
3.1 MB
Preview
Download
13
Views
Downloads
Show more details
All versions<br>This version
Views
Total views
13
13
Downloads
Total downloads
Data volume
Total data volume
9.3 MB<br>9.3 MB
More info on how stats are collected....
Versions
External resources
Indexed in
OpenAIRE
Communities
Keywords and subjects
Keywords
RLHF
Reinforcement Learning
post-training
GRPO
GSPO
reward modeling
LoRA
entropy collapse
Details
DOI
DOI Badge
DOI
10.5281/zenodo.21115798
Markdown
[](https://doi.org/10.5281/zenodo.21115798)
reStructuredText
.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.21115798.svg<br>:target: https://doi.org/10.5281/zenodo.21115798
HTML
Image URL
https://zenodo.org/badge/DOI/10.5281/zenodo.21115798.svg
Target URL
https://doi.org/10.5281/zenodo.21115798
Resource type<br>Book
Publisher<br>Zenodo
Languages
English
Rights
License
Creative Commons Attribution 4.0 International
The Creative Commons Attribution license allows re-distribution and re-use of a licensed work on the condition that the creator is appropriately credited.
Read more
Copyright
© 2026 Luv Verma. Licensed under CC BY 4.0.
Citation
Export
Technical metadata
Created
July 1, 2026
Modified
July 1, 2026
Jump up
This site uses cookies. Find out more on how we use cookies
Accept all cookies<br>Accept only essential cookies