Ugvx - Developers, It's a good time to evaluate your threat model
Developers, It's a good time to evaluate your threat model
Notice<br>Standard disclaimer applies. These are my personal views, not those of any employer.
Your threat model is stale
The world is changing. We know this. As developers we live this. New tools, new patterns, new problems. We build solutions<br>for problems. And we use our experience with our tools to achieve that. But one area that we often forget is our own<br>threat model. Now this is not limited to just developers, but anyone who often runs new software on their hardware, whether<br>it's a phone, server, or workstation. If you install software (and that includes new versions of libraries for development) or<br>run agentic solutions that do that for you, your threat model is likely in desperate need of review. If you don't have a<br>threat model, it's time to think about it.
The bugpocalypse
AI started off as a slow burn in terms of value, but in recent months, it's hit an inflection point. Bugs. AI, especially<br>the frontier models, are brilliant at picking up bugs. This is not a bad thing, but it has uncovered some critical vulnerabilities<br>in core software we use. It's invalidated assumptions that many had around security. The assumption that open source is secure because the code<br>is public and reviewed, along with the assumption that premium closed-source hardware and software providers rapidly capture vulnerabilities. Privilege<br>escalation is the current flavour with not just Linux hit with a series of vulnerabilities (Copy Fail, Dirty Frag, Fragnesia), but also Apple with an M5<br>kernel memory corruption exploit. The tools we use every day are<br>vulnerable, and we are in a liminal space while all projects and solutions find means to handle the increasing rate of bug and vulnerability<br>resolution.
Now some have the assumption that these are coming from Mythos, and therefore should be handled<br>internally by the organisations that provide the software. There is some truth to that. But the capability gap<br>between restricted and public models is not as wide as it might seem. With the right harness, public models, whether<br>frontier or open source, can action the same research. (HTTP Terminator) If not now, then very soon.<br>Who knows what has been found yet not disclosed, or by whom, and to what end?
Accelerated cascading compromise
There is a common belief in tech enthusiast circles. One does not need anti-virus software. Careful internet usage is all you<br>need to remain safe on the internet. This is while zero days keep popping up with little to no user interaction required to<br>infect a system.<br>Now this suggestion completely ignores the developer angle. Developers pull new software onto their machines<br>every day. New feature needs a new library? Pull from npm/PyPI/Cargo/etc. New version? Pull again. Need a provider for Terraform?<br>Pull some more code. Developers are always pulling unfamiliar code onto their machines. Restricting to only known packages<br>does not work as there are too many transient packages to consider, and there is a wave of worms hitting popular libraries<br>on a regular basis. (Mini Shai-Hulud hits TanStack)<br>It's not much of a stretch for an organised group to prepare worms that leverage privilege escalation bugs like Dirty Frag. Pair<br>that with a previously compromised CI/CD pipeline and you get a very damaging attack against organisations. One that could be incredibly<br>persistent at a global scale. Agentic engineering will accelerate this, where patches on popular open source solutions will be pulled,<br>evaluated, and exploited by agents before the maintainers have had a chance to respond.
Notice<br>The PC Security Channel on YouTube demonstrates real execution of prevalent malware in the wild. It's a practical resource for understanding how these attacks actually work beyond the theoretical.
As developers, we can't just stop using libraries. Even if AI was good enough to replace libraries with<br>bespoke solutions, it still introduces its own security risks and would be a maintenance nightmare. On top of that, the<br>development process is changing. The world is moving towards an agentic approach to engineering. And with that, autonomous<br>agents download libraries, consume websites (which could contain prompt injections), and make decisions without any user<br>interaction. It's clear that the current agentic tools we use don't have the best protections in place, often relying on<br>system prompts to limit the functionality of an agent. Agentic engineering broadens the attack surface of security vulnerabilities<br>on top of the general increased risk due to the environment we work in today.
Things to consider
This is not a framework. The environment we work in is changing rapidly. At best, these are suggestions that can be applied<br>to help limit one's exposure risk. It will not eliminate all risk, and one should evaluate the environment on a regular basis.<br>These are the thoughts of a developer with an...