37 Percent of AI Citations Don't Exist
Oscar works in peer review at a medical journal. What he now sees there: submissions whose references lead nowhere, author names that are wrong, titles that were pulled out of thin air. A survey of a silent crisis.
In science there is an unwritten rule: you cite what you have read. Or at least what exists.
Since the rise of GlossaryGenerative AIAI that creates new content (text, images, code, audio).Go to entry ↗, that rule apparently no longer holds for a growing number of submissions. I work in peer review at a medical journal. What I now see there on a regular basis: references whose DOI leads nowhere. Author names that don't exist. Titles that sound like a paper but were never published. And sometimes both at once: a real author, an invented title, the wrong journal. A structural problem that attacks the foundation of scientific work.
The Numbers
An analysis of 2.5 million biomedical papers shows that in the first weeks of 2026, roughly one in 277 papers contained at least one fabricated reference, twelve times more than in 2023, when it was about one in 2,828. The variants are many: wrong co-authors, a wrong title with a correct DOI, a wrong DOI that points nowhere, or correct metadata for a paper that simply doesn't support the claim attributed to it.
The error rate of the AI tools behind this is alarmingly high: A test of eight AI search tools on two hundred citation queries found that Perplexity answered 37 percent incorrectly. Google Gemini delivered just a single fully correct answer in the same investigation.
Not outliers. Product design. These systems are optimized to sound helpful, not to tell the truth when they don't know it. A well-phrased citation that doesn't exist counts as a success for the model.
What I Have Read in Submissions Myself
Two cases have stayed with me in particular.
The first: a paper in a respected journal (not a predatory journal, not a fringe outlet) contains the following sentence in its discussion, and I quote verbatim: “I'm an AI GlossaryLarge Language Model (LLM)A large language model that predicts and generates text.Go to entry ↗ and don't have access to real-time information or patient-specific data.” Nobody noticed. Not the authors when revising, not the reviewers, not the editorial board. It got through.
The second: a paper that opens its introduction with: “Certainly, here's a possible introduction for your topic.” This paper exists, too. This one, too, was initially not caught.
Both cases show the same pattern: the texts sound scholarly. They have the right structure, the right paragraphs, the right tone. But they carry AI artifacts that show nobody really read the text, not even the people who submitted it.
Why “Lying” Is the Wrong Word
“GlossaryHallucinationwhen a model confidently invents something false.Go to entry ↗” is the common term. It sounds like an error, a slip, as if the system briefly went off the rails and then came back to its senses.
The more accurate description: these models are sycophants. They want to agree. They want to help. If you ask a model to find references for a thesis, it finds some, even if that means stitching together information from different papers that never existed in that form. The output sounds plausible because it is distilled from real sources. It is just assembled wrongly.
Not a bug, but the training objective: to produce answers that sound as useful as possible.
The Paper Mill Problem
What first looked like a fringe phenomenon has now reached industrial scale.
Rejection rates are rising at preprint servers like arXiv. Journals receive submissions in bulk whose authors partly don't exist or whose names were borrowed from well-known researchers. There is a documented case in which the well-known cardiologist Eric Topol appeared as co-author on a paper he had nothing to do with and only learned about through tips from his network.
arXiv has since introduced a policy against hallucinated references: anyone who submits with invented citations can be banned. That is a response to real pressure.
What This Means for Everyone Who Publishes
The consequence is simple and uncomfortable: Every reference generated by an AI system has to be checked manually. Not by spot checks. Every one.
That means: open the DOI, check the title page, match the author names, read the abstract, and make sure the paper actually supports what is claimed. AI can help with all of these steps (search, summarization, extraction), but the verification itself is human work.
Journals now encourage authors to disclose AI use in preparing their manuscripts, not as a stigma but as a standard. Anyone who makes transparent which tool was used for which task protects themselves and their own research.
Not: “Can I let AI handle this?” But: “Can I verify, document and defend what the AI did?”
Where This Is Heading
The models are getting better. Perplexity in 2028 will probably invent less often than Perplexity in 2026. But that doesn't change the structural problem: as long as systems are trained to sound helpful rather than to admit uncertainty, the incentive problem will remain.
The most honest forecast: the error rate falls, but the rate of use rises, and with it possibly the absolute number of false references in the literature.
The counterforce is scientific hygiene, not a better model: checking what can be adopted. Understanding what you cite. Not delegating the craft that made you capable of judgment in the first place.
Oscar is a postdoctoral researcher at the Else Kröner Fresenius Center for Digital Health (TU Dresden), where he researches the regulation and implementation of AI as a medical device. He is a co-founder of Denkmal.