AI Makes Mistakes. That Means Process Design Matters More Than Ever. | by Mary Olowu | May, 2026 | MediumSitemapOpen in appSign up<br>Sign in
Medium Logo
Get app<br>Write
Search
Sign up<br>Sign in
AI Makes Mistakes. That Means Process Design Matters More Than Ever.
Mary Olowu
6 min read·<br>Just now
Listen
Share
Why the companies that get real value from AI will not be the ones with the cleverest prompts, but the ones with the strongest workflows, controls, and execution discipline.<br>Press enter or click to view image in full size
For years, business software sold a simple promise:<br>take messy human work, turn it into a repeatable workflow, and reduce avoidable errors.<br>That promise was never perfect, but the operating model was clear. Humans introduced variation. Software enforced sequence, validation, approvals, and audit trails.<br>Generative AI changes that model.<br>It is excellent at handling ambiguity. It can read unstructured text, summarize exceptions, classify messy inputs, and generate plausible next steps faster than most teams can. That is why adoption has moved so quickly.<br>According to Stanford HAI’s 2026 AI Index, 88% of surveyed organizations reported using AI in at least one business function in 2025, and 79% reported regular generative AI use in at least one business function. But the same report also notes that scaled AI-agent deployment remained in the single digits across nearly all business functions.<br>That gap matters.<br>It suggests many organizations are willing to use AI, but are not yet willing to hand it full operational control.<br>That restraint is rational.<br>The same 2026 AI Index reports that documented AI incidents rose from 233 in 2024 to 362 in 2025. It also highlighted a new accuracy benchmark in which hallucination rates across 26 top models ranged from 22% to 94%.<br>Those are not edge-case numbers. They are a warning about the shape of the technology.<br>The core issue is not that AI is bad. It is that AI is probabilistic. It can be useful, impressive, and wrong in the same workflow.<br>That means the old enterprise question, “How do we automate more of this process?” becomes a sharper one:<br>How do we redesign the process so AI can add judgment without being allowed to quietly break execution?<br>That is the real management problem now.<br>The New Primitive<br>A lot of business process design used to assume this:<br>manual work creates inconsistency, so we should replace as many manual steps as possible with deterministic systems.<br>AI changes the failure mode.<br>Now the risky step is often not the human. It is the machine-generated interpretation that sounds correct, moves quickly, and can slip past review because it looks fluent.<br>So the new primitive is not “AI replaces process.”<br>It is this:<br>AI makes mistakes. Processes, controls, and deterministic automation are how you make those mistakes survivable.<br>That is a much more useful frame for executives, operators, and business process owners.<br>It also fits what official guidance is already saying. NIST’s Generative AI Profile warns that generative AI opportunities, risks, and long-term performance characteristics are typically less well understood than non-generative tools, and may require additional human review, tracking, documentation, and management oversight. Its AI Risk Management Framework centers four functions: govern, map, measure, and manage.<br>In other words, the institutions thinking most seriously about AI risk are not telling organizations to prompt better. They are telling them to build operating discipline around the technology.<br>Where AI Actually Belongs in a Process<br>The mistake many teams make is trying to treat AI like deterministic business logic.<br>That is the wrong abstraction.<br>AI is strongest at interpretation:<br>reading messy intake<br>extracting signal from documents<br>translating natural language into structured candidates<br>proposing classifications, summaries, or actions<br>It is much weaker as the final authority for:<br>policy enforcement<br>permissions<br>financial commitments<br>compliance decisions<br>irreversible system changes<br>That suggests a better architecture for business processes:<br>AI interprets.<br>Rules validate.<br>Humans approve exceptions or high-risk actions.<br>Deterministic systems execute and record.<br>This looks simple, but it changes everything.<br>Instead of asking whether AI can do the workflow, process owners should ask which layer of the workflow it should own.<br>Usually, the answer is not the final one.<br>A More Useful Operating Model<br>In practice, strong AI-enabled processes tend to separate work into three layers.<br>1. Interpretation Layer<br>This is where AI creates value.<br>Examples:<br>classify inbound support requests<br>summarize claims or dispute documents<br>extract fields from emails, PDFs, or contracts<br>draft recommended next actions<br>The output here should be treated as a proposal, not truth.<br>2. Control Layer<br>This is where the business decides what must be true before action is allowed.<br>Examples:<br>is the customer eligible?<br>is the refund above a threshold?<br>does the...