I Stop LLMs Drifting in Production Codebases

How I Stop LLMs Drifting In Production Codebases - Scott Spence

Skip Navigation Scott Spence 🐱 Choose a theme: Select a themeacidaquaautumnblackbumblebeebusinesscmykcoffeecorporatecupcakecyberpunkdarkdimdraculaemeraldfantasygardenhalloweenlemonadelightlofiluxurynightnordpastelretrosunsetsynthwavetealixvalentinewinterwireframe

Posts Tags Speaking Contact

How I Stop LLMs Drifting In Production Codebases June 21, 2026 • 18 min read pimy-pisvelteguidenotes

Hey! Thanks for stopping by! Looks like this post was updated 7 days ago. Just bear in mind it was originally posted 1 month ago. If there's anything in here which doesn't make sense, please get in touch.

The problem I care about with LLM coding is drift: one plausible shortcut gets copied, then future sessions start treating it as how the app works. I’ve used AI coding tools every day as a team lead on large private client codebases, across features, bug fixes, refactors, reviews, and more recently a finance workflow processing more than $20m a month in capital. When the app has audit trails, permissions, calculations, generated documents, client workflows, and operational handoffs, “prompt harder” is not an engineering system. What works for me is guardrails in the repo: checks, docs, lint rules, tool-call blockers, and handoff validation that stop the model drifting before the mistake becomes normal. That starts with establishing good patterns first. LLMs are pattern followers before they are engineers. They look for the nearest example that appears to work, then continue from there. If the repo is full of clean service boundaries, narrow components, cited docs, targeted invalidation, and boring validation commands, the model has something useful to copy. If the repo is full of shortcuts, the model will copy those too, just faster. For me, drift looks like this: one lazy $effect turns into three more lazy $effects a route imports the database directly because it saw another route do something similar broad cache invalidation appears because it is easier than understanding the dependency graph demo data, placeholder IDs, and temporary glue quietly become production-shaped behaviour the next session treats all of that as precedent This post is the practical version of that system. I’ll cover the layers I use, why they exist, where they should be strict, where they should start as warnings, and a few examples toward the end that you can steal without building my exact setup. Prompting harder is not the solution For a long time the only thing stopping the model from making a mess was me typing “please follow the architecture” again. That is not a workflow. That is me supervising a process, and I have already done enough of that. LLMs are very good at local optimisation. They look at the current file, the nearest import, the most obvious pattern, and the shape of the request. Then they move fast. That is useful when the local pattern is good, and dog shite when the local pattern is temporary, legacy, copied from a spike, or just wrong. So I moved the guardrails to places the model cannot politely forget: the tool boundary, before code is written lint rules, while the diff is still small architecture checks, before bad imports become normal docs retrieval, before the model invents business rules validation scripts, before a handoff says “done” That is the difference between advice and pressure. Advice says “try not to use $effect”. Pressure says “this .svelte file was not modified because the write contained $effect; rewrite it with $derived, an event handler, an action, or a lifecycle API”. One of those scales, the other becomes review debt. The receipts I queried my Pi session history for one private production codebase where I’ve been deliberately building this system. Across 350 sessions: boundary checks appeared in 199 sessions boundary checks failed in 39 sessions boundary checks passed in 65 sessions after fixes LSP diagnostics appeared in 224 sessions Svelte checks appeared in 208 sessions local docs retrieval appeared in 271 sessions project skill files appeared in 278 sessions Those numbers are not a benchmark. They’re just receipts that the rails were in the normal path. The useful bit is that the failures were real. The checks caught things like: route files importing database access directly instead of calling a server service or command browser-side route code mutating domain objects directly route files generating business identifiers locally instead of using server commands or database identifiers demo data and placeholder identities leaking into production-shaped paths remote function files putting schemas and handlers in the wrong place route components and state modules getting too large and mixed concern broad cache invalidation instead of targeted invalidation file naming and identifier naming drift unexplained Svelte effects That is what I want from AI-assisted development. Not a model that never makes mistakes, because that does...

I Stop LLMs Drifting in Production Codebases

Related Articles

(no title)

Scientists reverse brain aging, with a nasal spray

AI has torched the market for junior programmers

Al Vigier: Canada's AI strategy shouldn't include secret Palantir bills

Google will expand age checks on Android worldwide till the end of the year