Brendan Keaton's Blog
Self-Service Ransomware as Security Against Local AI Tools<br>Read Time: 7 minsPosted: 2 weeks ago
“The only way these things in our world ever get cared for is if there is someone out there that has a sense of responsibility for maintaining the world we live in… and takes it upon themselves to do something about it.” - Philosophize This [168], Ethics of Care<br>Note:<br>This application is a file-encryption experiment for the local AI agent threat model. The full working code is available at the GitHub link below. It is built using self hosted Supabase (Postgresql, Kong, etc), Nextjs, Tauri (Rust), and FastAPI (Python). Generative AI was used for some sections of code (particularly most of the frontend for the webapp and some of the fastAPI endpoints). This project is currently unaudited and provided as-is (MIT License). See the repo for the full disclaimer<br>github.com/BrendanKeaton/annex
Local AI software can read your files, API keys, images, and any other sensitive information on your device.<br>Some attempts have been made at minimizing the effects of this (see .llmignore); however, this is ultimately a handshake agreement between the tools and you. There is nothing fundamentally stopping a bad actor agent, prompt injection attacks, or even a good agent over-indexing from leaking secrets or sensitive data . More secure solutions have been offered, such as developing in VMs or working under different user spaces; however, Mythos has shown this can be broken out of by a powerful enough model according to Anthropic's red team.<br>Over the course of the last few months, I have worked on a possible solution. In short, the tool is self-service ransomware that encrypts your files while agents work. The basic flow is as follows:<br>A user selects files that AI should not have access to or be able to edit, such as .envs, testing files, data files with PII, etc.<br>Before starting an agent, start an Annex “session”. This encrypts all of the files at rest with AES-256-GCM, including the file names. It usually takes a few seconds to a minute depending on the size and count of the files.<br>Use your AI tools normally. All of your sensitive files are encrypted entirely and cannot be read by AI tools.<br>At the conclusion of the agent running, end the session, and all of your files are decrypted.<br>The full flow and code includes a number of additional safeguards against prompt injection attacks and bad actor AI tools:<br>Starting a Session<br>Note:<br>The first step in the process is during sign up. Users create a 6 digit PIN and are then provided 6 “recovery seeds”. These are not account recovery keys; more on this below.
After a user selects all of the files they want to be protected for this “session”, they click a button to initiate. This sends a request from the client to the backend for a key that will be used in the AES-256-GCM encryption. Unlike other encryption softwares, this key is not random and is deterministically created with 4 inputs:<br>The current UTC date<br>An incrementing session count<br>One of the six recovery codes provided at sign up<br>A once-generated secret , stored only on the FastAPI server that never leaves<br>These four are then concatenated into a single string to be a seed for SHA256.<br>session key derivation
Below is a graphic, explaining the flow of the creation of a session key.<br>Key Generation FlowTauri Client [Rust]FastAPI Server [Python]Database [Postgresql][1] User initiates session, requestfor key sent to backend[2] Deterministically create aSHA256 key for AES-256-GCMencryption (explained above)[3] Store session key in Postgresql[4] Confirm storage of session key[5] Return session key to Tauri ClientLocal encryption cannot begin until keystorage is confirmed from Postgresql<br>Key Generation Flow<br>Tauri Client → FastAPI Server<br>[1] User initiates session, request for key sent to backend
FastAPI Server<br>[2] Deterministically create a SHA256 key for AES-256-GCM encryption (explained above)
FastAPI Server → Database<br>[3] Store session key in Postgresql
Database → FastAPI Server<br>[4] Confirm storage of session key
FastAPI Server → Tauri Client<br>[5] Return session key to Tauri Client
Local encryption cannot begin until key storage is confirmed from Postgresql
Note:<br>Having deterministically created, derivable keys may feel a bit like an anti-pattern for an encryption tool; however, because the threat model is exclusively for local AI agents, I believe it is a safe-trade off for the ability to recover files in the event of a database or logic failure. Even then, for a targeted, professional attack, the bad actors would still need to gain access to the fastAPI server secret, a user's recovery codes, a user's pin, and the device itself.
Encrypting Selected Files<br>Upon receiving the session key, the desktop application can begin encrypting files. Each file goes through the pipeline below:<br>File Encryption Pipeline[1] Collect all file statsBefore encryption begins, all protected file paths are gathered, sizes summed,...