The asymmetry problem : AI safeguards are mainly annoying to the good guys
HuggingFace just experienced a very common problem with cybersecurity safeguards in general audience tooling : it only hinders the good guys on the defence side.<br>In their July 2026 Security incident disclosure, when they tried to leverage their AI-assisted tool set for forensics analysis, they quickly hit a brick wall due to the model provider's safeguard :<br>When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident responder from an attacker. We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure. This had a second benefit: no attacker data, and none of the credentials it referenced, left our environment.
This experience points to a gap worth planning for. We do not know which model powered the attacker's agents, whether a jailbroken hosted model or an unrestricted open-weight one; either way, the attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried. The practical lesson for defenders: have a capable model you can run on your own infrastructure vetted and ready before an incident, both to avoid guardrail lockout and to keep attacker data and credentials from leaving your environment. This is not an argument against safety measures on hosted models, and we are sharing this feedback with the providers concerned.<br>This is not the first time we encounter this problem in real incidents. We're also seeing exploit code injecting forbidden content in their comments hoping to shut down automated, AI-assisted analysis :<br>The _index.js payload begins with a large JavaScript block comment containing fake system instructions and policy-triggering content. Because it is inside a comment, it does not affect JavaScript execution. The runtime skips it. The real malware begins after the comment with a try{eval(...)} wrapper around a large character-code array and a ROT-style substitution function.
This header appears designed for AI-mediated analysis, not for Node, Bun, or Python. It attempts to derail scanners or analyst copilots that feed the beginning of a file to a language model without clearly isolating the content as untrusted data. In weak pipelines, this can cause refusal behavior, prompt confusion, context pollution, or premature classification before the scanner reaches the actual malware.
This is not a magical bypass against static detection. YARA rules, entropy checks, AST parsing, string extraction, deobfuscation, and behavioral rules still work. But it is a practical anti-analysis trick against naive LLM-first triage systems.<br>I don't think there is any magical solution to this issue. Those safeguards do have benefits by removing the lowest hanging fruits, demanding some effort from attackers. This is also not specific to LLM guardrails. We will have uncensored/abliterated LLM providers the same way we have bulletproof hosting providers, anti-cheat bypass, DRM removal tools, automated CAPTCHA solvers or underground zero-days reseller.<br>Those tools are available to everyone, but the good guys usually can't use them as they have to abide by the laws while cybercriminals are well past that.
Read more
Five-Eyes joint-statement on AI
The Five Eyes recently published a joint statement highlighting the urgency of raising our security posture to face AI-assisted cyberattacks. Their practical actiond broadly align with my post-mythos article and add more specific recommendations about stepping up incident management processes.
Practical actions<br>These actions are not new, but
Role-confusion
This recent paper on role-confusion nicely explains how modern LLMs distinguish information like system prompts, user queries, chain of thoughts and tool callings using role tags. It shows how weak this system is at ensuring the segmentation and hierarchy of those different roles. Even if the API has safeguards
(PR)MFA is the baseline, not a step-up
It is now realistic to deploy user-friendly, phishing-resistant, passwordless authentication across the board. Yet many cybersecurity professionals kept the old habit of seeing MFA as a necessary inconvenience we should use sparingly, based on risky use cases. Let's change that.
SociaLLM Engineering: Old tricks, AI agents are the new victims
In this article, I’m calling SociaLLM Engineering the use of social engineering techniques against LLM agents to manipulate their decision-making and induce unauthorized actions.