AI agent suggested installing a malware package. Engineer almost took its advice

Jump to main content

Search

REG AD

SECURITY

AI agent suggested installing a malware package. Engineer almost took its advice

Fortunately, the company had a policy of checking source code on GitHub first

Avram Piltch

Avram<br>Piltch

US EDITOR

US editor

Published<br>thu 20 Aug 2026 // 08:00 UTC

PWNED Welcome back to PWNED, the column where we make fun of those who are security self-owned, so hopefully you don’t do the same. This week, we have a story that’s hot off the presses about a company almost sabotaging its security by using AI for programming.<br>Have a story about someone leaving a gaping hole in their network? Share it with us at pwned@sitpub.com. Anonymity is available upon request.<br>Our tale of machine learning malfeasance comes courtesy of Sergiy Fitsak, managing director of Softjourn, a consulting and software development company. He reminds us that, when it comes to AI, don’t trust: verify.

REG AD

During the course of business, one engineer asked an AI agent to recommend a package that they needed for a common task. The agent came back with the name of a legitimate-sounding package, which was formatted like a familiar library.

REG AD

At many organizations, this would have been the end of the story. The developer would have taken the AI agent’s advice and downloaded and installed the recommended package.<br>However, at Softjourn, the company has a policy which they actually followed: double-check any software recommendations made by AI to make sure they are legit. The developer skimmed the recommended package’s source code on GitHub and noticed that it had few downloads and had just been created a few days earlier. In other words, it was suspicious.<br>According to Fitsak, attackers have found a way to exploit package names hallucinated by AI models.<br>“The problem is that AI models sometimes invent package names that sound plausible but don't exist, a pattern security researchers have started calling 'slopsquatting,'” he told us. “Attackers have caught on and now register real packages under those exact invented names, betting that a developer under deadline pressure will install first and check later.”<br>If Softjourn hadn’t been so careful, they could have installed a malware package. We don’t know the exact payload, but this malware package could have given crims a backdoor into their systems and the ability to steal data or wreak other havoc.<br>“We caught it because we'd already built a habit of verifying download counts and reviewing source code on GitHub before installing anything an AI recommends, even when it looks routine,” Fitsak said. “It takes a few extra minutes. Skipping that step once is how a team ends up explaining a supply chain compromise instead of shipping a feature on time.”

MORE CONTEXT

Passwords stored in public Google Doc then showed up in search results

IT department put sticky notes on the laptops to help employees log in

Headteacher had the most guessable username-password combo you could imagine

Talking smack about a doctor got him access to private medical files

The lesson here is a very simple one: Don’t trust the package names that AI agents recommend. Have a human check the supply chain. And always have a human in the loop so they can take the time to stop and approve any outside code that comes into a project. ®

security<br>ai and ml<br>pwned

REG AD

AI AND ML

OpenAI glitch locks out vetted cyber researchers – and some can't get back in

Affected users say support cannot restore their previous approval or override the new decision

AI AND ML

Software development and tech services in the cross-hairs as AI marches on

Forrester warns of disruption to long-established tech activity

devops

Platform Engineering 2.0: your platform was built for a different era. AI just exposed it

PARTNER CONTENT: Platform engineering won the argument. Now it has to grow up fast and evolve for the AI era.

SCIENCE

No lift for Swift as NASA abandons orbital rescue

LINK will attempt a rendezvous, but its failed reaction wheels rule out grabbing and boosting the observatory

COLUMNISTS

Be a hater all you want, AI's here to stay

The good news? One of the worst bits, tech giants controlling it all, might soon be over

SECURITY

AI agent suggested installing a malware package. Engineer almost took its advice

Fortunately, the company had a policy of checking source code on GitHub first

MOST POPULAR

AI and ml

Google buys crashed airline Spirit’s data at auction, because AI

AI and Ml

Excel's Copilot function is headed for the Recycle Bin

ofbeat

NASA estimates the size of the hole SpaceX made in the moon

security

Expired credit cards revived by researchers to make unauthorized payments

SAAS

GitHub blames 8-hour outage on autoscaling fail and VS Code retry storm

DEVOPS

GitHub has Issues as repo downloads hit 50% error rate

AI

SYSTEMS

AMD inches closer to its goal of making AI suck less ......