AI Fiction Is Easy to Detect Because It's Stupid and Bad, Research Finds

Account

Log in

Subscribe

Navigation

Home

About

RSS

Support/FAQ

Podcast

FOIA Forum Archive

Merch

Advertise

Privacy

Contact Us/Tips

Follow us

Twitter<br>Bluesky<br>Mastodon<br>Instagram<br>TikTok<br>Facebook<br>RSS

Advertisement

&bull;

Go ad free

News<br>AI Fiction Is Easy to Detect Because It's Stupid and Bad, Research Finds

Matthew Gault

Jul 10, 2026<br>at 2:32 PM

ChatGPT uses too many dream sequences and Gemini won’t stop describing characters.

Photo by Ed Robertson / Unsplash

Fiction written by artificial intelligence is easy to detect because it struggles with complex story structure and tends to moralize in clunky ways, according to a preprint study from researchers at University of Maryland, College Park and Google DeepMind. They found that AI fiction has tells that go beyond stereotypical overuse of em-dashes and other obvious AI tropes and have more to do with the formulaic nature of the text itself.<br>“AI stories over-explain themes and favor tidy, single-track plots while human stories frame protagonists’ choices as more morally ambiguous and have increased temporal complexity,” the study, which looked at more than 50,000 AI-generated short stories, found. “Claude produces notably flat event escalation, GPT over-indexes on dream sequences, and Gemini defaults to external character description. We find that AI-generated stories cluster in a shared region of narrative space, while human-authored stories exhibit greater diversity. More broadly, these results suggest that differences in underlying narrative construction, not just writing style, can be used to separate human-written original works from AI-generated fiction.”

Basically, AI-generated fiction sucks and at the moment is easy to detect. The typical method of detection involves looking for stylistic markers such as an abundance of em-dashes, the overuse of the word “delve,” or an obsession with goblins, but this project tried something different. “The idea for this project came because we are hoping to eventually move past plain text detection, into some sort of space where we can separate human ideas from AI-generated ideas,” Jenna Russell, a University of Maryland researcher and one of the study’s authors, told 404 Media. Russell is also an intern at the AI-detection company Pangram.<br>Russell and her team decided to attempt to detect what she called “narrative features” in AI- generated fiction. The detector is called StoryScope and it builds on NarraBench, a 2025 benchmark that suggested a taxonomy of narrative features in fiction. StoryScope looked at how fiction handled plot development, character descriptions, setting, and temporal structure to determine if something was written by a human or an AI.<br>“It was my first attempt at getting 'under the surface' and focusing more on ideas,” Russell said. “We wanted to see how close to typical AI-detection we could get by only relying on the narrative features, to understand if this sort of structural difference really even exists. This method also adds some interpretability to detection, which is an open question in the field. Using narrative features, we can point to certain tangible features (such as the number of subplots included in a story). I think this is why it's struck a chord recently, people can really say ‘ah these are some of the underlying traits of how AI writes fiction.’”<br>To test StoryScope, the researchers selected 10,272 human-written stories then reverse engineered them into writing prompts using Gemini 2.5. Then it took those thousands of prompts and fed them into Gemini 3 Flash, DeepSeek V3.2, Claude Sonnet 4.6, Kimi K2.5, and GPT 5.4. All of the data — including the prompts and the resulting AI stories — are available on Hugging Face.<br>To source the stories, the researchers used the Books3 dataset — a database of 183,000 books collected from pirated ebooks. The dataset is the subject of several lawsuits and has been used to train an unknown number of LLMs. The StoryScope study included more than 10,000 of some of the most famous short stories ever written, many of them pulled from popular anthologies. There’s Joyce Carol Oates, Stephen King, Louis L'Amour, Charlotte Perkins, and Harlan Ellison. All have been rendered down to their base elements by AI and then regurgitated into a different LLM to see if it can replicate them.<br>Russell told me the dataset was controversial. “Hence why we do not release it to the public,” she said.<br>The study itself contained a disclosure. “We acknowledge the copyright issues related to the Books3 dataset and do not endorse its use for model training or commercial text generation,” it said. “The use of the dataset in our paper is restricted to academic purposes only and is meant to understand the narrative differences in human-written and AI-generated text to help inform discussions on AI-detection, authorship, and copyright policy.”<br>The various AIs, of course, can’t...