The State of AGENTS.md: Scoring the Agent-Makers' Own Instructions — Franz Paul<br>Skip to content

← back to writing7 July 20269 min read

The State of AGENTS.md: Scoring the Agent-Makers' Own Instructions

agents-mdinstruction-qualityai-agentstoolingschliff

16 repos. About 1.46 million GitHub stars between them. Zero A grades.

AGENTS.md has quietly become the cross-tool standard for repo-level agent instructions — a single file that Cursor, Codex, Copilot, Claude Code and a growing list of others read on startup. So I went looking for how the AI-agent ecosystem itself uses the convention it created. I swept 36 of the best-known AI coding-agent and agent-framework repositories, found 16 with at least 20k stars shipping a root AGENTS.md, and scored every file with a deterministic engine — no LLM judge, same file same score on every machine, reproducible against the exact commit SHAs at the bottom of this post.

The result: mean 70.0, median grade C, not a single A. The repo whose team popularized the convention ranks 13th of 16. And the best AGENTS.md I could find anywhere belongs to a library with 170 stars.

This is the same scorer I used to grade my own merged MCP contribution and the one behind the case for static instruction-file security. It is open source, MIT, stdlib-only: the rubric is a dict you can read, not a hidden prompt.

The ranking#

The set is every well-known AI coding-agent or agent-framework project at or above 20k stars that ships a root AGENTS.md, each scored at its HEAD on 2026-07-07 with released schliff==8.5.0. The AGENTS.md rubric is three dimensions: structure (weight 0.4), operational coverage (0.4), and efficiency (0.2).

#RepoStarsScoreGradestructureop. coverageefficiency1block/goose51k84.2B9575812langchain-ai/langchain141k81.0B8585653ComposioHQ/composio29k79.6B8575784sst/opencode183k77.8B9565695QwenLM/qwen-code26k77.8B8575696n8n-io/n8n195k77.8B8575697langgenius/dify148k75.2B9055868Significant-Gravitas/AutoGPT185k74.2C9545919All-Hands-AI/OpenHands80k72.4C80755210Kilo-Org/kilocode26k71.6C85753811zed-industries/zed †87k70.0C95457012browser-use/browser-use103k69.4C75754713openai/codex96k66.4C85555214crewAIInc/crewAI55k55.2D80158615huggingface/smolagents28k50.0D75010016RooCodeInc/Roo-Code24k37.0E75035

Seven B, six C, two D, one E. No A, no S. † zed ships AGENTS.md as a symlink to its .rules file; the score is for the resolved content, and the symlink itself is a finding — a tool that fetches the raw AGENTS.md gets the six-character string .rules, not instructions.

One repo was excluded as a non-independent data point: OpenInterpreter’s AGENTS.md is byte-identical to openai/codex’s — the same git blob, since the project is a codex derivative for open models. Counting it twice would have been dishonest.

The standard-bearer paradox#

openai/codex — the repo whose team did the most to popularize AGENTS.md — scores 66.4, a C, 13th of 16. The file is not lazy: 22.5 KB across 28 headings, genuinely rich in review conventions and TUI style rules. But the operational-coverage dimension asks a narrow question: does this file equip an agent to operate the repo? Test and lint commands are there (just test, just fix). A build command is not. It appears nowhere in the file — for a Rust and Bazel monorepo. There are no commit or PR conventions either. I filed the gaps as an issue with an offer to fix.

The pattern generalizes across all sixteen. Structure is uniformly strong — 75 to 95, because headings are easy. What separates the top of the table from the bottom is operational coverage: the real setup, build, and test commands, and the gotchas an agent cannot cheaply rediscover at runtime.

Famous frameworks, empty operations#

Three files earn a D or E, and each survived me reading the raw Markdown by hand — these are not scorer artifacts:

crewAI (55.2, D): 1 KB of contribution platitudes — “follow best practices”, DRY, YAGNI. The only concrete commands are for the docs site (mintlify dev). Nothing tells an agent how to set up, build, or test the actual Python framework. Operational coverage: 15 of 100.

smolagents (50.0, D): the entire file is 157 bytes — four bullet points of generic advice (“Be Pythonic”, “follow OOP principles”). Operational coverage: 0.

Roo-Code (37.0, E): 445 bytes — one genuinely useful UI-state gotcha and nothing else. No setup, no build, no test. Operational coverage: 0.

A high structure score with zero operational coverage is the signature of a file written for a human skim, not for an agent that has to run the project.

The best file in the sample has 170 stars#

Not one of the sixteen giants reaches an A. The best AGENTS.md in my entire sample — the corpus below plus these sixteen — belongs to maxcountryman/underway, a Postgres-backed job queue with 170 stars: 91.0, grade A . It is 1,728 tokens covering setup, build, test, lint, and project structure as copy-pasteable commands.

block/goose (84.2, B) shows the same shape among the big repos: 4.8 KB, almost...