SociaLLM Engineering: Old tricks, AI agents are the new victims

Spectacular “social engineering” attack involving AI agents, powered by Large Language Models (LLM), are multiplying as companies are progressively rolling them out in replacement to traditional Customer Services staff. Customer Service bots have existed long before the advent of AI agents, but they usually had very limited capabilities. They acted as a glorified search engine for a customer knowledge base or an alternative UI for a feature already available as self-service to the customer. This new generation of bot goes beyond that, giving them access to sensitive information and backend capabilities that were only accessible to human, trained personnel.<br>In this article, I’m calling SociaLLM Engineering the use of social engineering techniques against LLM agents to manipulate their decision-making and induce unauthorized actions. This could be revealing sensitive information, similar to phishing attacks (or should I say PhAIshing?) or broader social engineering attacks like pretexting and impersonation. You might have seen other terms qualifying those families of attacks, like manipulated delegation or more simply, prompt injection. I would argue that SociaLLM engineering goes beyond injecting extra instruction and leverage the implicit social context understanding of large language models to shape its behaviour. For example by acting as an authority figure or making it “think” its current perceptions are distorted from reality.<br>What SociaLLM Engineering looks like<br>SociaLLM engineering is at first glance very similar to a traditional social engineering attack targeting human personnel. In traditional Social Engineering, the attacker will reach out using an official channel, like emails, chat, phone or in person. They will invent some fantasy to manipulate the victim into doing an action they wouldn’t have done otherwise. It exploits human weaknesses like trust authority figures, fear and urgency, curiosity, empathy, reciprocity, etc.The most common outcomes are taking over an account (Phishing), leaking sensitive information, or getting money (ex: Business Email Compromise or BEC).<br>SociaLLM engineering will act the same, this time inserting the attacker’s fantasy into the LLM’s context window. This could be directly via user inputs like when AI agents acts as chatbots, or indirectly via media processed by the model such as webpages, pictures, calendar invites and emails, etc.<br>In traditional social engineering, Customer Service staff are among of the most targeted employees alongside finance or HR teams as:<br>They act as a gatekeeper to valuable information or capabilities (e.g.: Recovering accounts, sending money, …)<br>They are often not among the most tech-savvy employees<br>They routinely interact with persons outside the company (clients, suppliers, applicants…)<br>Those interactions very often involve opening attachments (KYC documents, CVs, Invoices…)<br>It’s also an area companies often see as “cost centres”, which they are aggressively trying to reduce. This lack of investment combined with their sensitivity make them targets of choice for an attacker,<br>Therefore, it’s not surprising to see the same happening with AI Customer Service agents being prime targets of SociaLLM Engineering attacks. Other agent systems both with a high level of autonomy and significant processing untrusted content, such as AI browser or Agentic coding are also very vulnerable.<br>Let’s go through a few occurrences of SociaLLM Engineering that made the news:<br>Instagram account takeover<br>Between April and May 2026, attackers exploited Instagram’s AI-assisted account recovery system to compromise more than 20,000 accounts. Some high-profile victims included Barack Obama’s White House accounts or Sephora’s brand.<br>This account takeover attack is as easy as it can get. The technique involved leveraging commercial VPN to appear in the same geographic location as the target’s Instagram account to pass heuristic checks, start the account recovery procedure claiming you’ve been hacked, then instructing the AI agent to send recovery codes to an email address you controlled. Due to some other shortcomings, the provided email address was never checked against the one registered in the account, the agent complied without question. The only accounts protected from this attack were those with 2FA enabled as the technique only allowed to change the password.<br>While this outcome could have been achieved without the presence of AI agents through social engineering, it would never have reached this order of magnitude without triggering internal reviews. AI agents will gladly perform the same dangerous tasks thousands of times in a row without raising an eyebrow on the pattern or learning from past occurrences, highlighting the importance of keeping a “human in the loop” for sensitive operations:<br>This incident maps directly to OWASP LLM06:2025 Excessive Agency; an AI agent granted capabilities,...