Miles v0.1: Production-level Post-training - LMSYS Org<br>Projects<br>Blog<br>About<br>Donations<br>Contact

‹ Back to Blog‹ Back to BlogContents<br>The Miles RL Loop<br>Rollout<br>Fast Agentic Rollout by SGLang<br>Fully Async RL<br>Agentic Environments<br>Token-In-Token-Out (TITO)<br>Efficient Rollout Routing Replay (R3)<br>Training<br>Low-precision Training<br>Memory Efficiency & Disk Offload<br>Two Training Backends<br>Weight Update<br>Verified Day-0 Model Support<br>Other Post-Training Recipes<br>LoRA RL<br>On-Policy Distillation (OPD)<br>Zero-KL Alignment<br>Code Quality Principle<br>Miles-Diffusion<br>Multi-Hardware Support<br>Example: Training GLM-5.2 on Terminal-Use Tasks with 64 NVIDIA GB300 GPUs<br>References<br>Acknowledgement

Miles v0.1: Production-level Post-training<br>Miles TeamAugust 18, 2026<br>We present Miles v0.1, a full-stack production-ready system for frontier post-training, the successor to our first Miles release [1]. Building upon slime's [16] clean design, Miles optimizes every stage in the RL training loop around a simple principle: verified, clean, and customizable everywhere . With accuracy, efficiency, reliability, and scalability as first-class goals, Miles aims to make frontier-scale RL accessible to researchers and developers alike. In this blog post, we will walk through Miles end-to-end.

The Miles RL Loop

An RL training job in Miles is a loop over the following stages:

Rollout — SGLang engines generate trajectories. In agentic RL, each multi-turn rollout session interacts with its own isolated environment that executes actions and produces the reward.

Training — completed trajectory groups are consumed by the trainer (NVIDIA Megatron-LM or FSDP), which computes the RL loss and updates the policy.

Weight update — new weights are synchronized back to the rollout fleet with minimal interruption to in-flight rollouts.

In what follows, we will go through each element in the loop to highlight how we make Miles accurate, efficient, reliable, and scalable.

Figure 1. The Miles fully async RL loop.

Rollout

Every rollout in Miles is generated by SGLang. Building upon SGLang's native inference efficiency, Miles unlocks the full agentic training workflow: multi-turn sessions, tool execution, sandboxed environments, and token-faithful trajectory capture.

Fast Agentic Rollout by SGLang

Miles provides fast agentic rollout by natively integrating SGLang [15], which is optimized for long, multi-turn generation. Agentic trajectories vary widely in length, and each new turn reuses most of the previous context. By default, Miles uses the SGLang router, which keeps all turns of a session on the same SGLang engine and DP rank (if DP attention is enabled) to reuse the cached prefix, while assigning new sessions to the least-loaded rank so that a few long trajectories do not overload part of the fleet. The SGLang router also reserves KV-cache capacity for long sessions beforehand. These features ensure a balanced and stable rollout concurrency, keeping cache-hit rate high in agentic training.

Fully Async RL

For long-context, tool-use, and agentic workloads, rollout time is mainly determined by a handful of stragglers. A synchronous schedule exacerbates the straggler issue, because the trainer has to sit idle until the slowest trajectory in the batch returns, and the rollout engines have to wait until the optimizer finishes model update. Miles' fully asynchronous RL eliminates such mutual-blocking by allowing rollout engines to generate persistently: rollout generation stays continuously in flight while the trainer consumes completed groups and updates the model. Neither side is blocked by the other.

Figure 2. How fully async RL handles the long tails. Different colors indicate different weight versions; the green fragments represent tool-call time.

Scheduling operates at sample granularity: each completed trajectory immediately frees a slot, keeping generation concurrency stable despite large differences in trajectory length. Completed groups enter a bounded data buffer that decouples rollout throughput from training cadence. This buffer also forms a customizable policy boundary where users decide which samples to keep, retry, discard, or reject as stale without modifying scheduling or execution.

To make evaluation asynchronous, Miles provides three evaluation modes, distinguished by where model weights come from.

Shared-engine evaluation uses the rollout fleet and temporarily pauses new submissions, making it suitable for small debug sets without extra GPUs.

Dedicated evaluation uses a separate GPU fleet loaded from checkpoint snapshots, allowing training and rollout to continue uninterrupted.

External evaluation passes a checkpoint directory to any user-provided evaluator, including non-SGLang services.

All three modes associate results with the checkpoint and training step that produced them. If evaluation finishes several steps later, Miles reports that lag rather than misattributing the result; failures skip the evaluation point instead of terminating...