Measuring whether Claude.md changes what the agent does

DevAutomationCC1 pts0 comments

trigger-tree — gate docs and measure instruction adherence

🌳">

🌳 trigger-tree

Gate documentation discovery in CI.

Measure whether the CLAUDE.md you pay for on every request changes observable behavior.

100% local · zero model tokens · no cloud, no analytics vendors

CI gate · day one

Deterministic and telemetry-free. A regression fails the PR with the affected file and a suggested fix.

Instruction adherence · a few sessions

Per directive: opportunities, followed evidence, uncertainty, and the recurring cost of always-loaded context.

tt watch — live doc-discovery

instruction adherence · 2 observable · 1 unobservable

route-telemetry 7/8 followed · 88% · warming

checks-before-commit 5/6 followed · 83% · warming

~190 tokens/session · unobserved ≠ violated

every flash is a real doc read rippling up the tree · ← older prompts · → newer prompts · this demo is the plugin

The demo above is a faithful mock.<br>This is the real thing — an actual terminal recording of tt watch --demo:

real footage — tt watch --demo

The dashboard's written counterpart:<br>view an example insights report — same synthetic demo data, full format.

🔥 Heat & cold maps

Attention cools with a 30-day half-life while lifetime reads stay visible. Focused top 10 live; exact 7/30/90-day windows in insights. Captures rg/grep/find and expanded Bash reads.

📈 Proof, not vibes

/tt suggestions proposes fixes with evidence. /tt note marks the change; the trend shows whether it worked.

🔒 Local by design

Telemetry stays local and gitignored. Before setup, prompts are hashes and new adherence capture is off. Setup makes each privacy choice explicit. Stdlib only, zero network calls.

🅰️ A grade your PO gets

Coverage, verified folder-router reachability, and search concentration distilled into one documentation health score — trackable sprint over sprint.

⚡ Zero model tokens

Hooks run shell-side in milliseconds. Measuring costs nothing — not in tokens, not in latency, not in context.

🤝 Claude Code · Codex

Native lifecycle hooks feed one git-root dataset on both agents. Python CI covers macOS, Linux, Windows and Python 3.10–3.14; native Windows hook launch uses Claude's documented exec form but is not exercised end-to-end in CI.

Gate your CI on discoverability

The structural gate works<br>immediately: router coverage, orphaned docs, folder entry points, and watch scope.<br>It needs no telemetry and nothing leaves your runner.

- uses: Hedde/trigger_tree@v1.25.1 # GitHub Actions<br># GitLab CI: pip install trigger-tree<br># tt gate --code-quality gl-code-quality-report.json<br>Commit a baseline once (tt gate --update-baseline).<br>A PR that makes docs harder to discover fails with the exact file and fix. The gate<br>checks wiring, not whether an instruction changed behavior. Discoverable never means discovered;<br>behavior is proven by telemetry, not by the gate.

docs discoverability100%

Does your CLAUDE.md actually work?

Injected instructions are already loaded,<br>so trigger-tree measures a narrower question: when a user-confirmed directive applied,<br>did its deterministic probe observe the requested behavior?

DirectiveOpportunitiesFollowedRateConfidence<br>Route telemetry questions8788%warming<br>Run checks before committing6583%warming<br>Preserve stdlib-only runtime———unobservable

Your always-loaded context is ~190 tokens per session.<br>2 directives (~34 tokens) have not been triggered in the 12 sessions where their<br>capture was active (of 12 recorded over 9.8 days).

Unobserved means evidence was not captured; it does not mean<br>violated. Zero opportunities is never-triggered, not 0% — and only counted over<br>sessions where that probe was actually being captured. Rates below five usable<br>opportunities are provisional. Measurement is deterministic and zero-token; only<br>model-assisted manifest authoring uses tokens, and the user confirms every probe.

Honest time to value

CI gate Immediate. No telemetry required.

Instruction adherence After several applicable opportunities—usually a few working sessions.

Heat and health Directional during warm-up; strongest after roughly a month.

Why this matters

In AI-assisted development, documentation is the steering wheel:<br>it tells the assistant how your team builds software. But a rule that is never<br>read protects nothing — an unread guardrail fails silently, and you only notice when the AI "ignores" a<br>convention it simply never found. trigger-tree measures what actually gets read, flags what never does, and<br>proves whether your fixes worked. Documentation becomes monitored infrastructure instead of a hopeful artifact.

And this metric exists nowhere else: agent observability platforms measure<br>tokens and traces — none measure which of your docs the assistant actually read .<br>Anthropic's own guidance says to keep CLAUDE.md lean; trigger-tree provides evidence for what to review,<br>what to protect or reroute, and what to rescue.

One signal, with its evidence attached

measuring…

This repository’s current local snapshot ·...

gate tokens tree telemetry claude trigger

Related Articles