How to Make an Agent Know It Screwed Up | Rafał Opiłowski

AI agents write code fast. They also leak null through every layer, create 2,000-line router files, leave bugs in production, and generate dead code once the happy path compiles.

This isn’t malice; LLMs optimize for plausible output, not production durability. You won’t stop them from making mistakes, but you can build a system where the agent finds out it screwed up before a human has to tell it.

I’ve implemented features and maintained code quality through an AI agent on a production TypeScript monorepo (10+ packages, 3+ apps). These are the layers that made it work.

Layer 0: Documentation

Write down what, how, and why; otherwise the agent guesses.

Agent context windows can’t hold the full codebase, so layered documentation delivers just-in-time context: project overview, package-specific rules, task specs. Write it once; the agent stops asking “what’s the structure?” on every task.

Layered Agent Instructions

A root-level AGENTS.md covers project overview, tech stack, filesystem structure, build system, and scripts. Each app and package has its own file with domain conventions: the API package documents router patterns, the database package documents schema, the frontend package documents components.

Files concatenate from root downward (OpenAI’s discovery pattern 1), deeper files override shallower ones. An agent gets project context plus localized rules without reading every file.

Layer 1: Test-Driven Development

Tests first, then code. An agent can’t fake coverage when CI enforces thresholds.

Agents make tests pass, but they don’t know which tests to write. The configurations below remove that choice: passing tests mean working code, not well-mocked code.

Base Test Configuration

A shared config centralizes test settings: global utilities, dependency optimization, and coverage thresholds. In CI, the floor is 80% across branches, functions, lines, and statements.

Tests live next to the code they test (__tests__/ directories co-located with source), so the agent finds them without guessing where test files are stored.

In-Memory Database Tests

Integration tests use PGLite2, an in-memory WebAssembly Postgres that runs in-process. No Docker, no network setup. A setup helper creates a fresh database, applies the production schema, and returns a Drizzle ORM client identical to production.

Benefits over traditional mocking:

Same schema as production, so no drift between test and real queries

Real PostgreSQL constraints, triggers, transactions, and locking all work as expected

No mocked query builders, so the agent can’t write a query that passes tests but fails in prod

Network-Layer API Tests

For client-side code, tests use Mock Service Worker (MSW3) to intercept HTTP requests at the network layer. The test server uses production router definitions. The agent tests against real request/response cycles, not mocked function returns.

This catches mistakes module-level mocks miss:

Request serialization errors (wrong content-type, malformed body)

Response parsing failures (wrong shape, missing fields)

Error handling paths (HTTP 400, 401, 403, 500)

Caching and retry behavior

The full client stack (transport, serialization, error handling, caching) runs unmodified. Only the network hop is replaced.

Layer 2: Spec-Driven Development 4

The best time to catch a mistake is before the agent writes any code.

Give an agent a vague prompt and it guesses scope, usually wrong. A spec forces thinking before typing:

Scope boundaries prevent feature creep. The agent reads “out of scope: bulk operations” and doesn’t spend cycles building pagination when all that’s needed is a text input.

User scenarios encode edge cases the agent doesn’t know to ask about. What happens when the search returns zero results? What’s the error message for a network timeout?

Acceptance criteria are the test the agent can verify against. No “looks done.” The spec says exactly what done means.

The Workflow

The screenshots below span different projects within a similar tech stack. The workflow uses NeoLab’s Context Engineering Kit.

Human writes → Initial prompt (captured verbatim in draft/)

Agent expands → Structured spec with scope, scenarios, ACs

Human reviews → Validates direction, adjusts scope

Agent implements → Follows spec, verifies against ACs

Spec graduates → Moved to done/ as documentation

The spec is the contract. It stops the agent from building the wrong thing.

Layer 3: Architectural Isolation

Some bug classes shouldn’t be possible. Architecture, not code review, enforces that.

When business logic tangles with infrastructure, agents inject database calls into validators, API calls into pure functions, and framework imports into domain models. Physical separation prevents this because you can’t import what isn’t there.

Monorepo Structure

Everything in one repo. The agent gathers cross-domain context in a single workspace instead of hopping...