Linting SKILL.md: A Study Report

Developer ToolingLintingAgent Skills

Do SKILL.md Linters Matter? An Empirical Study

This report evaluates whether recently proposed<br>SKILL.md best practices translate into measurable improvements in agent<br>performance. After implementing a linter based on these recommendations<br>and applying it to Garry Tan's GBrain repository, I observed consistent<br>improvements in skill-routing accuracy across all 18 benchmark runs,<br>ranging from +1.3 to +7.6 percentage points.

ZY

Zhuohao Yang

8 min read · July 30, 2026

Chart 1: Full-bench routing accuracy.

A recent paper,<br>From Anatomy to Smells: An<br>Empirical Study of SKILL.md in Agent Skills (Hong, Imani &<br>Ahmed, UC Irvine), proposes a set of best practices for writing<br>SKILL.md files — the documentation files that tell an agent<br>what a skill does and when to invoke it. Does following them actually<br>change agent behavior, or does it just make the files look tidier? To<br>find out, I implemented a linter that enforces the<br>paper's recommendations and tested it on real-world data rather than an<br>idealized lab setup.

The test subject is<br>Garry Tan's GBrain<br>repository, which contains 53 skills as of July<br>2026.

Method

Fully evaluating agent performance is difficult and often subjective,<br>so I narrowed the focus to routing — whether the model picks the<br>right skill for a prompt — rather than full execution.

To start, I ran the linter<br>nikhail<br>against the GBrain repository, checking all 53 skills, and got this<br>report:

$ uvx nikhail path/to/gbrain/skills --html gb_report.html<br>rules violated:<br>S005 body-size-budget: SKILL.md body over 500 lines defeats<br>progressive disclosure<br>S006 referenced-files-exist: Files referenced from SKILL.md must<br>exist in the package<br>S007 no-orphan-files: Bundled files never mentioned in SKILL.md are<br>dead weight the agent will never use<br>S008 parseable-frontmatter: SKILL.md must begin with valid YAML<br>frontmatter<br>S009 trigger-guidance: Description should state when to use the<br>skill, not only what it does<br>S011 no-markup-in-description: Description must not contain XML/HTML<br>tags (prompt-injection surface)<br>S014 third-person-description: Description should be written in<br>third person, not first person<br>S016 no-rigid-command-sequence: Long fixed command sequences<br>over-prescribe execution; state the goal or ship a script<br>S018 has-example: A substantial skill should include at least one<br>concrete example

path/to/gbrain/skills/academic-verify/SKILL.md<br>- warning S007 bundled file 'routing-eval.jsonl' is never<br>referenced in SKILL.md — the agent has no instructions for when<br>or how to use it<br>205 info S006 referenced file 'skills/perplexity-research/SKILL.md'<br>resolves only from 'gbrain/' (2 level(s) above the skill package)<br>— the reference breaks if the skill is distributed standalone<br>...

The linter runs a series of deterministic checks and reports every rule<br>violation it detects. For example, S005 flags oversized<br>skill bodies, while S009 flags descriptions that lack<br>trigger guidance. This guidance is critical because the router depends<br>on the description to determine when a skill should be invoked.

What was fixed

Because at routing time most models look only at the title and the<br>description to decide which skill to invoke, I applied fixes to the<br>description strings alone, leaving the rest of each skill package<br>unchanged.

For each flagged skill, a fresh agent session (Claude Fable 5)<br>was spawned with that skill's full original text plus its linter<br>findings, and asked to apply the fixes to the description string<br>only.

The routing test bench

Each test case presents a user prompt alongside the catalog of all 52<br>skill descriptions and checks whether the model routes to the expected<br>skill — or correctly declines to pick one. Grading is exact-match and<br>binary. The frozen intent set has 323 cases from four<br>sources:

110 routing-eval cases — the human-written<br>routing-eval.jsonl fixtures that ship inside the GBrain<br>repo itself.

104 generated easy cases — paraphrases that stay<br>close to each skill's own vocabulary.

104 generated hard cases — paraphrases that<br>deliberately avoid the skill's vocabulary, the way a user who has never<br>read the skill would phrase the ask.

5 null cases — prompts that should route to<br>no skill at all, so the bench also penalizes over-eager<br>selection.

The generated cases were authored by Claude Fable 5. Each<br>configuration ran k = 5 trials per case,<br>comparing the original descriptions (Arm B) against the linter-fixed<br>ones (Arm C) across multiple models and reasoning-effort levels.

Results

Routing accuracy improved in every configuration tested, by<br>1.3–4.9 percentage points depending on the<br>model.

All raw test data is in<br>bench/results,<br>and a sample per-run summary shows the level<br>of detail each run records. Across all 12 benchmark runs,<br>Arm C beat Arm B every single time . Three patterns<br>stand out:

Weaker routers gain more. The cheapest model<br>tested (GPT-5.4 nano) gained 4.9 points. A strong model can often...