Agent-Assisted SGLang Development: An Initial Exploration - LMSYS Org<br>Projects<br>Blog<br>About<br>Donations<br>Contact

‹ Back to Blog‹ Back to BlogContents<br>1. TL;DR<br>2. Why SGLang Is a Good Fit for Agent-Assisted Development<br>3. From Prompt Engineering to SKILL: Protocols and Examples<br>3.1 Current Skill Stack<br>3.2 Recent Optimization and Workflow Examples<br>4. Profiling, Review, and Loop Engineering<br>4.1 Humanize/RLCR: Adding External Review to the Loop<br>4.2 SGLang SOTA Performance Loop (Loop Engineering)<br>4.3 Codex Goal: A Lower-Cost Loop Implementation<br>5. KDA-Based CUDA Kernel Optimization for SGLang Systems<br>6. Practice Rules<br>7. Acknowledgments<br>8. References

Agent-Assisted SGLang Development: An Initial Exploration<br>SGLang TeamJuly 2, 2026<br>SGLang development increasingly goes beyond isolated code changes. The same repository now spans LLM serving, distributed runtime, GPU kernels, diffusion pipelines, model-specific execution paths, and production incident handling. In the past, many of these workflows depended on individual developer memory: how to launch a certain model, how to read a profile trace, which log to add first when debugging a CUDA crash, or which benchmarks a performance PR should include. As agent tools mature, this experience can be turned into executable SKILL.md files, scripts, benchmark contracts, and review loops.

Around SGLang agent development, a set of skills has already emerged for both LLM and diffusion work:

SGLang .claude/skills is maintained inside the SGLang repository and captures repo-level development workflows such as CUDA crash debugging, kernel integration, tests, CI, profiling, production triage, and source-tree conventions.

SGLang diffusion .claude/skills focuses on diffusion-specific workflows, including adding new diffusion models, benchmarking and profiling denoise paths, tuning performance options, and validating quantized pipelines.

BBuf/AI-Infra-Auto-Driven-SKILLS covers cross-framework serving benchmarks, capacity planning, profile and pipeline analysis, model compute simulation, SGLang human-style review, production incident triage, SOTA loops for SGLang and other open-source inference frameworks, and model PR history.

kernel-design-agents is the KDA project and the winning solution for the MLSys 2026 FlashInfer Kernel Contest.

BBuf/KDA-Pilot applies KDA-style agent kernel workflows to SGLang. Its public B200 diffusion summary now tracks 10 SGLang kernel tasks. Most rows come from KDA-Pilot's public benchmark ledger, while residual_gate_add uses the B200 speedup reported by the merged SGLang integration PR after the original task baseline moved. KDA-Pilot-derived work has now landed in three SGLang integration PRs.

Viewed together, these efforts point to the same direction: the value of agents comes from procedural engineering knowledge, including executable steps, reproducible experiments, and reviewable evidence.

1. TL;DR

Agents are most useful in SGLang when they can keep moving along a well-defined workflow. Benchmarking, profiling, kernel API logging, adding diffusion pipelines, production incident replay, and SOTA loops can all be encoded as skills.

An SGLang skill is an executable development procedure. In debug-cuda-crash, sglang-diffusion-benchmark-profile, and llm-torch-profiler-analysis, the important content is preflight checks, hard failure gates, artifact contracts, reproduction commands, and result formats.

Profile evidence is central to performance work. The SGLang profiler skills produce fixed kernel tables, overlap-opportunity tables, and fuse-pattern tables. KDA-Pilot extends this into same-ABI baseline/candidate comparison, real workloads, correctness gates, NCU evidence, and per-shape results.

Long-running optimization has started to move into Loop Engineering. The SGLang SOTA Performance Loop decomposes "chasing SOTA" into fair benchmarking, gap decision, profiling, patching, and revalidation. Humanize/RLCR adds external review, while Codex Goal can run the same loop with lower coordination overhead.

Review becomes more important. Agents can run more experiments, but they also generate more changes that look plausible and still need careful review. Developers increasingly define problems, choose evidence, design workflows, and decide whether results are ready for production paths.

2. Why SGLang Is a Good Fit for Agent-Assisted Development

SGLang is a high-performance serving framework for LLMs and multimodal models. As model families and hardware paths expand, several recurring problems show up in development:

LLM paths are complex. A single performance issue may cross the Python runtime, scheduler, CUDA graph, Triton/CUDA kernels, FlashInfer/FlashAttention, distributed collectives, and model-specific wrappers.

Diffusion paths are also complex. A slower denoise pass may involve pipeline/stage partitioning, DiT blocks, attention backends, torch.compile graph breaks, CFG/SP parallelism, VAE, or custom fused kernels.

Validation is expensive....