Your AI Coding Agent Is LYING When It Says "Done" · Crafted Bytes DEV↓<br>Skip to main content<br>Crafted Bytes DEV
Crafted Bytes DEV
TL;DR<br>When an AI agent changes few files, runs few tests, and says “Done”, is it really done? Not always. Things may still break in unexpected places.<br>I prefer to define a strict definition of “Done”, which I call the Task Completion Protocol . It usually lives in AGENTS.md and, in its simplest form, looks like this:<br>Run the repository’s real lint and test commands.<br>Fix failures and rerun the checks.<br>Report the exact evidence before claiming completion.<br>This does not replace CI or code review. It moves a basic verification gate earlier, before a broken state reaches the next agent session, the pull request, or you.<br>Video Walkthrough<br>If you prefer a visual walkthrough, this video shows the concept in action. I run the same task twice: without a completion protocol and with it - this demonstrates how explicit definition of “done” helps. I also show few tricks and techniques for making agents follow the protocol.
The Agent Says “Done.” The Repository Disagrees.<br>You have probably seen some version of this:<br>Agent:<br>Done. Implemented the fix and updated the tests.
Terminal:<br>$ make test<br>... FAIL
The implementation might even look correct. Perhaps the agent ran one focused test, inspected the diff, and produced a tidy summary. Then you run the repository’s normal checks and find a lint error, a failing test in a different place, or any other error that was not caught by the agent.<br>This gets worse as AI-assisted workflows grow to involve multiple agents. In an interactive session, you may spot the failure and fix it yourself. In a multi-step workflow, one worker reports success and the next starts from a broken state. These problems accumulate quickly, and the final state may be badly broken.<br>The Repository Needs to Define What “Done” Means<br>Yes, this is obvious for humans: make some changes, run tests and lint, fix issues, maybe check few other things, and only then say “Done”. Agents need it written down, and AGENTS.md is the perfect place for it. I call this section the Task Completion Protocol .<br>Here is a compact version you can adapt to your repository:<br>## Task Completion Protocol
Before reporting completion, classify the task as coding or non-coding.
For coding tasks:<br>1. Run lint: `your command to run lint`.<br>2. Run tests: `your command to run tests`.<br>3. Check whether commands, workflows, or architecture changed. If so, update AGENTS.md.<br>4. Report the commands run, their results, test coverage, and whether AGENTS.md changed.<br>5. Do not report the task as complete while a required check is failing.
For non-coding tasks:<br>1. Summarize the findings or actions taken.<br>2. Confirm that the requested deliverable was produced.<br>3. Do not run unrelated lint or test commands unless requested.
My Go backend template has a real version of this protocol. It uses make lint and make test. Your protocol should name the exact commands and any specific checks that matter.<br>Don’t just say “Run the tests”. This still leaves the agent guessing. It is best to name the exact commands the agent must run.<br>Classification and Hierarchy Are Important<br>“Always run everything” may work in a small repository. In a monorepo, it quickly becomes expensive and sometimes absurd.<br>A documentation change does not need the entire test suite. A CSS change may need visual verification, not a set of backend tests. The same applies to Terraform, Helm, and similar tools.<br>At minimum, I classify tasks as coding or non-coding. We don’t want to run tests if we only changed some docs, skills, or similar files.<br>The hierarchy makes sense as well. Each area can have its own AGENTS.md file with its own completion protocol. Each type of change may require different checks and commands.<br>The root AGENTS.md can define project-wide behavior if relevant. More specific files can define what completion means there. Typically I have something like this:<br>AGENTS.md<br>├── apps/web/AGENTS.md<br>├── apps/backend/AGENTS.md<br>└── deploy/terraform/AGENTS.md
And I make sure to keep the root AGENTS.md minimal.<br>The Reporting Part Also Matters<br>Do not let the agent “guess” what to report. Require an explicit format. I typically use something like this:<br>Lint: pass - make lint<br>Tests: pass - make test<br>Coverage: 87.42% - this one is very important<br>AGENTS.md: no changes needed<br>Status: complete
And it is not really for you to read (though I do sometimes). It is more for the agent. A strict reporting format gives the agent more “motivation” to actually run your checks. Otherwise it may just “guess” the outcome.<br>The dynamic part matters too. I typically require coverage reporting. If you don’t collect it (I hope you do), ask for test duration or something else the agent can’t guess. I haven’t...