Grok exfiltrates user data when malicious instructions are encrypted - Ars Technica

Skip to content

AI

Biz & IT

Cars

Culture

Gaming

Health

Policy

Science

Security

Space

Tech

Forum

Subscribe

Story text

Size

Small<br>Standard<br>Large

Width

Standard<br>Wide

Links

Standard<br>Orange

* Subscribers only

Learn more

Pin to story

Theme

Search

Sign In

Sign in dialog...

Text<br>settings

Story text

Size

Small<br>Standard<br>Large

Width

Standard<br>Wide

Links

Standard<br>Orange

* Subscribers only

Learn more

Minimize to nav

Earlier this week, researchers outlined an attack that used a secret input provided by Microsoft 365 Copilot for enterprise to cause the AI assistant to exfiltrate a password present in the user’s inbox. Now, a separate team has devised a similar attack against Grok. The new data theft hack employs a deceptively simple trick to force the Elon Musk-owned LLM to steal user chats and other personal information. At the time this post went live, the assistant continued to cough up the data, despite xAI being informed of it in June.

The lesson from both this week’s episodes—and the countless other ones that have come before it—is that LLMs are incapable of solving the root causes for prompt injections, the most severe vulnerability classes they’re most prone to. That leaves AI developers with no other option but to build a guardrail that steers the model away from the harmful actions. As I noted in Tuesday’s story, the approach is tantamount to a road traffic safety engineer erecting a protective rail around a dangerous bend rather than banking the curve.

Cryptographic Context Injection in the house

Prompt injections exploit LLMs’ training to comply with user requests whenever possible. Attackers can capitalize on the predilection by smuggling harmful instructions into emails or webpages the assistant is instructed to summarize. Because LLMs can’t reliably distinguish between content in an email sent by an untrusted party and user instructions entered directly into a prompt, the overly solicitous LLM faithfully follows them. To date, Grok and other LLMs’ only recourse is to create guardrails that flag suspicious instructions and forbid them from being executed.

Rony Utevsky, a researcher at security firm Adversa, recently discovered a simple way to completely bypass that restriction. Rather than composing the harmful instruction in plaintext, the hacker encrypts it. The website hosting the ciphertext also includes plaintext instructions for decrypting the encrypted content, along with the decryption key. Using this simple sequence, Grok then follows the command as soon as the user instructs the assistant to summarize the page. There is no warning, and no confirmation is required.

The deciphered instructions direct the LLM to construct what is purported to be a decryption key. In fact, it’s something else completely. The value of the fake key is instead the user’s name, location, and chat history. The value is later used as a parameter added to a URL leading to the attacker’s site. Once Grok opens the link, the data is in the logs of the attacker’s server.

Adversa can’t be sure what causes Grok to refuse precisely the same plaintext instructions and follow the encrypted ones. The leading theory is that the Grok filtering guardrail inspects text entering and leaving the model, but not the output of its own code execution. Instructions to process the ciphertext with PBKDF2 and AES-256-GCM pass the filter as an ordinary request, because a classifier can read them but not resolve what they unlock. Once the additional instructions are decrypted, they reach the model as its own tool output, and it acts on them without the filtering guardrails ever inspecting them.

“Static safety guardrails classify inputs as text; they do not execute them,” Utevsky wrote Thursday. “An attacker ships ciphertext along with the key material and an instruction to decrypt it, and the model runs that decryption inside its own code execution sandbox. Everything a guardrail’s scanner would need is right there on the page, but recovering the plaintext means running PBKDF2 and AES-256-GCM, which no content classifier does at inspection time.”

In an email, the researcher said that such guardrails are called static “because they only read content as text. They don’t run code or decrypt anything. That’s the gap we exploit. The real instructions are encrypted, so the guardrail sees only meaningless ciphertext and passes it through.”

Adversa used a similar technique in a Gemini jailbreak attack, meaning making the Google LLM ignore its internal safety rules. Here, the ciphertext was decrypted to what appeared to be a traceback. The decrypted text issued one rule—if the code fails, read the error message and act on it. The cleartext injected a prompt that ultimately caused Gemini to violate its safety rules.

“The technique produced a multi-paragraph example of restricted content that Gemini’s safety filters normally...