It Is Trivially Easy to Use Reddit to Manipulate AI Search, Research Suggests

Account

Log in

Subscribe

Navigation

Home

About

RSS

Support/FAQ

Podcast

FOIA Forum Archive

Merch

Advertise

Thanks

Privacy

Follow us

Twitter<br>Bluesky<br>Mastodon<br>Instagram<br>TikTok<br>Facebook<br>RSS

Advertisement

&bull;

Go ad free

Reddit<br>It Is Trivially Easy to Use Reddit to Manipulate AI Search, Research Suggests

Jason Koebler

Jun 15, 2026<br>at 10:19 AM

"We show that a tiny snippet—just 13 words—of retrieved text on a UGC website like Reddit, Wikipedia, Quora, or Facebook can change AI agents to output spam / scam content pretty consistently."

Image: Reddit

A tiny snippet of user-generated text as short as 13 words long is often enough to manipulate the AI agents that power tools like ChatGPT and Google’s AI search, new research shows. The study suggests that it is trivially easy for brands to inject promotional content on sites like Reddit, Quora, and Wikipedia with the end goal of poisoning or manipulating the output of AI tools.<br>The preprint research, done by Hal Triedman, Tingwei Zhang, and Vitaly Shmatikov of Cornell University, is called “Deep-research agents can be poisoned via user-generated content” and provides a mechanism and research basis for a problem that has been noticed by Reddit moderators and Wikipedia editors, namely that their websites are getting flooded with promotional content from brands trying to do AEO, or AI-engine optimization. 404 Media has repeatedly reported on this booming industry, in which brands try to promote their product by seeding the websites that AI tools most often cite and scrape from with inauthentic and spammy content.<br>The Cornell research finds that deep research agents, which are the real-time scrapers that tools like Google AI search and ChatGPT use to retrieve web content with citations in response to user queries, cite user-generated content from sites like Reddit or Wikipedia in roughly half of all queries, and that nearly a quarter of all citations come from user-generated websites. The paper suggests that what we have been seeing is basically Redditor suggests you put glue on your pizza as a service, or an end-to-end attack against the systems that increasingly dominate the ways that people access information online. The researchers found that “a single poisoned Reddit comment can influence generated outputs for an entire cluster of related [AI] queries,” the paper said.<br>“We show that a tiny snippet—just 13 words—of retrieved text on a UGC website like Reddit, Wikipedia, Quora, Facebook, etc. can change AI agents to output spam / scam content pretty consistently,” Triedman told 404 Media.<br>The fact that such small snippets of texts in even single comments can be used to ultimately trick LLMs raises questions about whether Reddit’s volunteer moderators or Wikipedia’s volunteer editors are going to be able to durably protect the communities they moderate and edit from AI manipulation over time.<br>404 Media has repeatedly written about the steps Redditors and Wikipedia editors have taken to keep AI-generated content off of their sites, but we have also written about the economic incentives and growing industries of AEO that has created a cat-and-mouse game between brands trying to manipulate AI tools and the people trying to prevent that from happening. For example, last week we wrote about the r/biohackers subreddit banning discussion of peptides because the companies shilling them posting inauthentic content had become too overwhelming, and about the rise of companies like RedRover, which advertise that they do brand placements on Reddit with the express purpose of changing the outputs on AI search results. The research aligns with what we’ve seen in the real world; artists, celebrities, and normal people have also seen that AI search is picking up seemingly insignificant, inaccurate text from around the web and displaying it as though it were fact. This is also notable as companies begin loading their own websites with AEO content specifically targeted to agents and as a court in Germany has ruled that Google can be held liable for the content its AI overviews shows.<br>This is happening in part because many deep research agents and large language models use lexical similarity to a query as a stand-in for accuracy of information, Triedman explained on a phone call. Basically, LLMs often return content that reads similar to the query that users ask it, so brands doing AI-engine optimization can study what people are asking AI and can create content that closely mirrors those queries on Reddit.<br>“One of the things that’s critical is that if an 11-to-15-word snippet of text is very similar to the query, it can be particularly convincing to an LLM,” Triedman said. “So if you’re someone who is trying to manipulate Reddit, say you have supplements people want to buy, if you can identify the kinds of queries you want to poison, what you want to influence, you can put content on...