A Deep Dive into Hallucinated Citations: DeepMind Paper Edition - VerusCite Blog
August 23, 2026
A Deep Dive into Hallucinated Citations: DeepMind Paper Edition
I periodically run papers that come across my desk through VerusCite to check their<br>bibliographies for hallucinations. One recent example was<br>How LLMs Distort Our Written Language,<br>a paper with several authors affiliated with Google DeepMind.
The expanded abstract from How LLMs Distort Our Written Language.
You can see the<br>full VerusCite report here.<br>I figured it would be worthwhile to walk through the types of hallucinations it found,<br>as it highlights common examples of AI hallucinations in peer-reviewed papers.
Error 1: A real paper number with invented metadata
First, the bibliography cites<br>NBER Working Paper 34255. The paper number and<br>first author are correct, but the title is slightly off and every author after the first one is<br>wrong.
This is a hallmark of AI-hallucinated citations. It assembles plausible subcomponents, but the<br>citation as a whole is incorrect.
VerusCite lists another citation as a minor error. The cited title is<br>Labour MPs accused of using AI to write parliamentary speeches, while the<br>actual headline<br>is Labour MPs accused of using ChatGPT to write speeches.
I think it is possible there was some sort of AI prompt to “replace ChatGPT with AI,” or<br>something along those lines. (For folks who use AI for writing, keep a separate bibliography<br>file so an inline editing prompt cannot rewrite the references directly.)
While this example could arguably be called a hallucination as well, I am OK with leaving it as<br>a minor error. I intentionally instruct VerusCite to be more lenient with online articles, since<br>their titles can change.
Error 2: Invented authors
The second clear hallucination follows the same general pattern. The citation includes a<br>real arXiv identifier, but fills in an altered<br>title and the wrong authors.
The next two errors are the same story. For error 3, the title and arXiv identifier point to<br>The Lock-in Hypothesis: Stagnation by Algorithm,<br>but the cited authors are incorrect. Interestingly, the correct author list includes Max<br>Kleiman-Weiner, who is also an author of How LLMs Distort Our Written Language.
Error 5: The same paper appears once wrong and twice right
Error number five is especially interesting. The bibliography cites the correct paper title,<br>Generative AI enhances individual creativity but reduces the collective diversity of novel content,<br>but attaches an invented author list and incorrect publication details.
But the same bibliography also cites the correct authors, Anil Doshi and Oliver Hauser, in both<br>the working-paper and published versions. Even in documents with some AI-hallucinated<br>references, the majority tend to be correct.
VerusCite has a mechanism to identify duplicate citations. Unfortunately, this one sneaks<br>through because the titles differ too much between the hallucinated and correct entries. But<br>you can also see that VerusCite correctly handles the legitimate working-paper and published<br>versions, linking to the appropriate record for each.
What should we make of these errors?
These bibliography errors do not determine whether the paper’s actual findings are accurate.<br>(Inline citations are<br>often pretty superficial.)<br>But to me, they are a strong signal that the paper’s references were not reviewed with enough<br>care.
I want people to use AI responsibly. In a bit of irony, VerusCite currently uses Gemini 3.7<br>Flash to make these determinations, with web search provided through Perplexity. The point is not<br>that AI tools never make mistakes—the missed duplicate above is one example. The point is to put<br>an explicit verification step into the workflow.
Citing papers that actually exist is necessary, although not sufficient, for an AI-assisted<br>paper to be accurate.
Back to all posts