Hackers can use 9 of the most popular AI tools to assemble massive botnets - Ars Technica

Skip to content

AI

Biz & IT

Cars

Culture

Gaming

Health

Policy

Science

Security

Space

Tech

Forum

Subscribe

Story text

Size

Small<br>Standard<br>Large

Width

Standard<br>Wide

Links

Standard<br>Orange

* Subscribers only

Learn more

Pin to story

Theme

Search

Sign In

Sign in dialog...

Text<br>settings

Story text

Size

Small<br>Standard<br>Large

Width

Standard<br>Wide

Links

Standard<br>Orange

* Subscribers only

Learn more

Minimize to nav

In the brief history of AI security, the prompt injection has quickly become the top threat. Large language models are inherently unable to distinguish between legitimate instructions provided by users and malicious ones sneaked into emails, source code, and other third-party content the models are processing. This makes it trivial to surreptitiously inject malicious commands that the LLM readily follows.

With no way to enforce this crucial boundary between trusted and untrusted sources, AI engine developers are left to erect elaborate guardrails designed to mitigate the damage rather than solve the root cause.

To date, most prompt injections have fallen into a class known as push, in which each potential victim is targeted. For example, the adversary injects malicious instructions into an individual email or calendar invitation. Because the injection must then be sent (or pushed) to each specific target, the scale of the attack is limited, hampering mass exploits that hit the Internet at large.

Meanwhile, pull-based attacks, in which an LLM actively seeks out the adversarial prompts planted on websites, remain limited. With no way to lure large numbers of LLMs to a malicious site, these sorts of attacks don’t scale either.

Enter HalluSquatting

Now, researchers have devised a pull-based attack that changes all that. A new attack the researchers have named HalluSquatting has the potential to assemble massive botnets, perform large-scale DDoSes, and infect devices at scale, a first for prompt-injection attacks. The attack works against AI coding assistants and agents, including Cursor, Cursor CLI, Gemini CLI, Windsurf, GitHub Copilot, Cline, OpenClaw, ZeroClaw, and NanoClaw, which are all susceptible. In the normal course of performing day-to-day activities, these assistants and agents routinely pull code and other resources from repositories and registries.

The HalluSquatting threat model.

Credit:<br>Spira et al.

The HalluSquatting threat model.

Credit:

Spira et al.

Short for adversarial hallucination squatting, HalluSquatting is built on an LLM’s inherent tendency to hallucinate the resource identifiers hosted in repositories and registries. It works against coding agents and assistants, which commonly access high-privilege command lines to run code from third-party resources. By predicting the identifiers LLMs are most likely to hallucinate and then registering and seeding them with instructions to install reverse shells or other malicious wares, the attack can indiscriminately infect massive numbers of devices without having to target each one.

“The scalable property of the attack enables the attacker to compromise a large number of users with minimal effort by targeting popular resources, thereby maximizing the likelihood that the squatted resource will be retrieved,” the researchers wrote in a paper published Wednesday. “By exploiting integrated shells and terminals of agentic applications to run scripts and code, attackers can effectively ‘infect’ many independent agentic applications by embedding instructions to install reverse shells in the resources the attackers register.”

With the ability to take control of distributed devices at scale, HalluSquatting has the potential to achieve various objectives not previously possible with prompt injections. Large ransomware campaigns and large botnets for use in DDoSes or cryptocurrency mining are two such examples.

The “squatting” part of the name is an invocation of “typosquatting,” in which a domain, repository package, or other resource identifier closely mimics the name of a popular one in hopes of luring potential users to visit or install it. Typosquatting first gained widespread attention in 2016 when a college student uploaded 214 booby-trapped packages to the PyPI, RubyGems, and NPM repositories that closely mimicked names of legitimate packages. The result: The imposter code was executed more than 45,000 times on more than 17,000 separate domains, and more than half were given all-powerful administrative rights. Typosquatting attacks have flourished ever since.

LLMs don’t know how to say “I don’t know.”

The starting point for HalluSquatting is the inability of LLMs to accurately identify the location of a resource specified by the user. When a developer, for instance, instructs a coding agent to clone a popular new repository, the LLM hallucinates its correct location up to 85 percent of the time. When...