Code Factories Without Quality: The AI Development Blind Spot | QA Wolf
Hot takes<br>Code Factories Without Quality: The AI Development Blind Spot<br>Nishant Shukla
August 20, 2026
Share
Key Takeaways
- Code multiplied with AI. Verification didn't. That gap is where the incidents live.<br>- Coverage isn't verification. 100% line coverage still ships broken user flows. E2E catches what coverage misses.<br>- Verification has to be a pipeline. It must scale like code, runs on every deploy, fixes its own flakes.
Zapier. Nubank. Goldman Sachs. Three companies with great engineering and risk management teams – and all three are now handing coding tasks to AI agents with minimal human intervention.<br>The workflow of code factories is deceptively simple: a developer hands off a task and the agent plans, codes, reviews, tests, debugs, and ships it end-to-end.<br>Zapier deployed 800+ AI agents internally, with 89% AI adoption across the entire organization. Nubank pointed autonomous AI agents at one of the most critical projects in the company, migrating its core ETL. Goldman Sachs, meanwhile, is piloting autonomous software engineers on its own codebase.<br>The appeal of code factories is obvious: velocity. Code generation at scale. 10x speed or more. Entire projects shipped without a human hand touching the keyboard.<br>But every code factory is quietly skipping or underinvesting in the same step: testing.<br>Agents are generating production code faster than anyone can verify it. Generation capacity increased 10x. Verification capacity didn’t. So, the code ships untested or undertested making code factories not just about velocity, but about velocity at all costs, quality be damned.<br>And when it breaks in production (it will), there's no feedback loop to make the next generation any better. The whole system is designed to ship more slop, faster.<br>The unspoken assumption: AI code is production-ready
Every code factory is built on an assumption that nobody says out loud: generated code is production-ready code.<br>It's not. But the assumption persists because QA is rarely prominently in the equation.<br>Instead, QA is being treated as a downstream checkbox or something you can solve with minimal coverage. In a code factory, where velocity is the core value, QA is the friction—the thing preventing you from shipping even faster.<br>The reasons are complex but because it’s seen as slow, it gets deprioritized or automated away with shallow test coverage that passes but doesn't truly verify much.<br>And teams often know this but still ship code they know is likely to break because they feel they have no other choice. Fast slop is now seen as better than slightly slower quality.<br>The real problem is that verification velocity hasn't yet matched generation velocity. AI code review tools are helping with one side of verification. They're catching some bugs, suggesting improvements, flagging obvious issues. But they're still letting bugs through.<br>What's needed now is an increase in testing velocity.<br>The cost of undertesting in code factories
A code factory might say they ship with 80% coverage and call that success. But coverage like that typically measures lines executed, not whether user flows actually work. You can have 100% line coverage and still ship code that fails under load, handles errors wrong, or leaves data in an inconsistent state.<br>The code factory speeds up the rate at which bad code hits production.<br>And the results are brutal. Incident rates are climbing as code generation scales. Teams shipping AI-generated code are seeing a 30% increase in change failure rates and a 23.5% increase in incidents per pull request.<br>More code shipped faster without proper verification means more bugs in production, more customer-facing failures, more firefighting.<br>A bug caught at the PR stage cost minutes to fix. A bug caught in staging costs hours. A bug caught in production can cost days, angry customers, incident responses, and cascading failures across dependent systems. The cost of fixes post-deployment dwarfs the cost of catching them before they ship.<br>Good testing allows you to ship fast without shipping slop
The real problem is that code factories are optimized for generation speed, not verification speed. QA is positioned as sometimes a blocker to velocity, when it should be seen as the enabler of it. Because if you have good automated test coverage in place, code factories become something you can actually trust to ship fast.<br>The problem is that you can't hire your way to 10x verification since human test creation is linear, whereas agent generation isn't. You also can't let agents grade their own homework because then tests that inherit the code's blind spots just automate your false confidence.<br>The tests that actually catch these failures – end-to-end coverage of real user flows – are historically the slowest to build and most painful to maintain. That's the trap: the verification your code factory actually needs is the exact verification that's...