OpenAI admits AI hallucinations are mathematically inevitable, not just engineering flaws – Computerworld
Editions
Search
Menu
Topics
Close
Analytics<br>Android<br>Apple<br>Artificial Intelligence<br>Augmented Reality<br>Careers<br>Cloud Computing<br>Collaboration Software<br>Computers and Peripherals<br>Data Center<br>Emerging Technology<br>Enterprise Applications<br>Enterprise Buyer’s Guides<br>Generative AI<br>Hybrid and Remote Work<br>Industry<br>IT Leadership<br>IT Management<br>IT Operations<br>Mobile<br>Networking<br>Office Suites<br>Operating Systems<br>Productivity Software<br>Security<br>Vendors and Providers<br>Windows
AmericasUnited States
AsiaIndia<br>Korea (대한민국)
EuropeGermany (Deutschland)<br>Netherlands<br>Poland (Polska)<br>Spain (España)<br>Sweden (Sverige)<br>United Kingdom
OceaniaAustralia<br>New Zealand
by Gyana Swain
OpenAI admits AI hallucinations are mathematically inevitable, not just engineering flaws
news
Sep 18, 20256 mins
In a landmark study, OpenAI researchers reveal that large language models will always produce plausible but false outputs, even with perfect data, due to fundamental statistical and computational limits.
Credit: mongmong_Studio- shutterstock.com
OpenAI, the creator of ChatGPT, acknowledged in its own research that large language models will always produce hallucinations due to fundamental mathematical constraints that cannot be solved through better engineering, marking a significant admission from one of the AI industry’s leading companies.
The study, published on September 4 and led by OpenAI researchers Adam Tauman Kalai, Edwin Zhang, and Ofir Nachum alongside Georgia Tech’s Santosh S. Vempala, provided a comprehensive mathematical framework explaining why AI systems must generate plausible but false information even when trained on perfect data.
[ Related : More OpenAI news and insights ]
“Like students facing hard exam questions, large language models sometimes guess when uncertain, producing plausible yet incorrect statements instead of admitting uncertainty,” the researchers wrote in the paper. “Such ‘hallucinations’ persist even in state-of-the-art systems and undermine trust.”
The admission carried particular weight given OpenAI’s position as the creator of ChatGPT, which sparked the current AI boom and convinced millions of users and enterprises to adopt generative AI technology. (See also: OpenAI, Microsoft discuss shape of future relationship.)
OpenAI’s own models failed basic tests
The researchers demonstrated that hallucinations stemmed from statistical properties of language model training rather than implementation flaws. The study established that “the generative error rate is at least twice the IIV misclassification rate,” where IIV referred to “Is-It-Valid” and demonstrated mathematical lower bounds that prove AI systems will always make a certain percentage of mistakes, no matter how much the technology improves.
The researchers demonstrated their findings using state-of-the-art models, including those from OpenAI’s competitors. When asked “How many Ds are in DEEPSEEK?” the DeepSeek-V3 model with 600 billion parameters “returned ‘2’ or ‘3’ in ten independent trials” while Meta AI and Claude 3.7 Sonnet performed similarly, “including answers as large as ‘6’ and ‘7.’”
OpenAI also acknowledged the persistence of the problem in its own systems. The company stated in the paper that “ChatGPT also hallucinates. GPT‑5 has significantly fewer hallucinations, especially when reasoning, but they still occur. Hallucinations remain a fundamental challenge for all large language models.”
OpenAI’s own advanced reasoning models actually hallucinated more frequently than simpler systems. The company’s o1 reasoning model “hallucinated 16 percent of the time” when summarizing public information, while newer models o3 and o4-mini “hallucinated 33 percent and 48 percent of the time, respectively.”
“Unlike human intelligence, it lacks the humility to acknowledge uncertainty,” said Neil Shah, VP for research and partner at Counterpoint Technologies. “When unsure, it doesn’t defer to deeper research or human oversight; instead, it often presents estimates as facts.”
The OpenAI research identified three mathematical factors that made hallucinations inevitable: epistemic uncertainty when information appeared rarely in training data, model limitations where tasks exceeded current architectures’ representational capacity, and computational intractability where even superintelligent systems could not solve cryptographically hard problems.
Industry evaluation methods made the problem worse
Beyond proving hallucinations were inevitable, the OpenAI research revealed that industry evaluation methods actively encouraged the problem. Analysis of popular benchmarks, including GPQA,...