Bixonimania’—the fake illness that AI fell for | Scientific American

May 22, 2026<br>Add Us On GoogleAdd SciAm<br>This researcher made up a disease to test AI. It failed miserably

How an experiment involving a made-up skin condition exposes the risks of increasingly popular AI medical advice

By Rachel Feltman, Sushmita Pathak & Alex Sugiura<br>J Studios/GettyImages

SUBSCRIBE TO Science Quickly<br>Apple | Spotify | YouTube | RSS

Rachel Feltman: For Scientific American&rsquo;s Science Quickly, I&rsquo;m Rachel Feltman.<br>Have your eyes ever felt sore and itchy after spending too much time staring at a screen? You might have a condition known as bixonimania—or at least that&rsquo;s what several popular AI-powered chatbots might have told you if you&rsquo;d asked last year.<br>Millions of people around the world turn to AI chatbots for medical advice every day, often as a supplement to a doctor&rsquo;s visit but also sometimes in place of it. That can lead to dangerous consequences and in rare cases, even death.<br>On supporting science journalism<br>If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.<br>Our guest today is Almira Osmanovic Thunstr&ouml;m. She&rsquo;s a researcher at the University of Gothenburg in Sweden and at the Sahlgrenska University Hospital, Center for Digital Health and Chalmers Industriteknik. She&rsquo;s also the creator of bixonimania. She says this totally made-up disease reveals some very real problems with the way we train and use large language models.<br>Feltman: Thank you so much for coming on to chat with us today.<br>Almira Osmanovic Thunstr&ouml;m: Thank you so much for inviting me.<br>Feltman: So you recently did an interesting project involving AI. Can you tell us a little bit about how you came to this idea?<br>Osmanovic Thunstr&ouml;m: I work many different jobs, but one of them is in academia. I was having lectures for students and telling students how systems that create large language models work and demonstrating where the data comes from. And it was interesting how few of them, or how few even people within AI, understand how large language models are built.<br>So I really wanted to have a clear case that leaves breadcrumbs throughout the whole system to show both how data is processed, how data is churned out and how the prediction model and training model works when it comes to distributing information. And most of my students are in medicine, so they&rsquo;re either medical students or psychologists or working with health. So it was quite easy to use that as a target for creating this project where I show you go from just a loose [Laughs], a loose mention of a condition to it being a full-blown disease in the large language models.<br>Feltman: So walk us through the process here.<br>Osmanovic Thunstr&ouml;m: Well, to start off with, I knew that most of data that these commercial large language models—and, quite clearly, all language models, even the noncommercial ones—are built on is Common Crawl. It is a nonprofit organization that crawls the Internet for written and digitized information and has done so since 2007. And this large repository is what is used to create the algorithm that—and the reasoning behind what information is fed into, for example, ChatGPT. And that is where it starts.<br>So knowing that anything that goes in there will come out as information, and humans are in the loop and sift out data, but those humans are not always able to sift out data, especially if it looks credible ...<br>Feltman: Mm.<br>Osmanovic Thunstr&ouml;m: So creating something that looks credible enough for an AI and credible enough for a human eye that wouldn&rsquo;t care to look deeply into it, I knew that I had to create, to start off with, a fake university. Universities are highly ranked as sources of information. I knew I had to create a researcher because humans and not companies [Laughs] are more valued as information sources, especially if [they] belong to a credible institution.<br>But I also know that sprinkling little words in, for example, blogs or social media is also picked up &rsquo;cause those are open sources being crawled. So I knew that I had to sort of put the word out there in several different sources for it to seem credible for the AI system.<br>Feltman: Yeah, and did anything surprise you about how this played out, or, or did it all proceed as you had expected it to?<br>Osmanovic Thunstr&ouml;m: In a sense, yes, &rsquo;cause I didn&rsquo;t think that preprints, which are academia&rsquo;s sort of tabloids [Laughs] &rsquo;cause anything can end up there, would be weighed into the database as seriously as it was in the context of what kind of information is used for training medical information.<br>So I thought that this preprint would not make it into large language models. I was convinced that perhaps the word...