A novel method for input privacy from LLMs | Sameer Wagh %F0%9F%94%90"><br>One of the most interesting privacy technologies that I have come across is called Stained Glass Transform (SGT). This was invented by folks at Protopia AI (their team includes my talented friend and collaborator Sid Roy and in this blog I am looking into their technical paper [1]. The problem it addresses is one that anyone building on cloud LLM APIs encounters: you want the model’s intelligence, but you don’t want the LLM provider to see your prompt/data.<br>The problem<br>When you call any hosted LLM endpoint (think ChatGPT, Claude.ai, OpenRouter, HuggingFace), you hand your prompt in the clear to a third-party server, which stores it in their database logs. This is a major concern given the increasingly personal nature of prompts and the mechanics of the data economy. A mechanism to let users benefit from LLMs while preserving the privacy of their input is therefore critical.<br>Existing solutions<br>There are a few different ways to resolve this prompt privacy challenge.<br>$\textbf{Local hosting.}$ Host the model yourself so the prompt never leaves your environment. Ollama makes this straightforward, letting you run Llama, Mistral, Gemma, and other open-weight models on consumer hardware with a single command. The obvious limitation is compute: a capable model needs a GPU with sufficient VRAM. Beyond that, you forfeit all the infrastructure that comes for free with hosted endpoints: load balancing, auto-scaling, automatic retries, hardware maintenance, and the operational overhead of keeping a model server healthy in production.<br>$\textbf{Fully Homomorphic Encryption (FHE).}$ FHE allows computations directly on encrypted data so your prompt is encrypted on-device and the server processes it without ever decrypting it. This Belfort Labs demo is a live in-browser experience that gives a feel for what FHE-based inference looks like in practice. On the open-source side, Zama’s Concrete ML is the leading library tackling the underlying hard cryptographic engineering. The downsides are steep: FHE inference is slower than plaintext, LM endpoints need significant re-engineering to operate over encrypted arithmetic (plaintext-ciphertext), and key management at scale is a non-trivial operational challenge.<br>$\textbf{Trusted Execution Environments (TEEs).}$ TEEs (e.g. Intel SGX/TDX, AMD SEV, Confidential Containers) create hardware-isolated enclaves where code and data are hidden even from the host OS and cloud provider. This can be used to perform two-sided privacy where the server cannot see the user’s prompt and the model provider’s weights can simultaneously remain confidential. In practice, the user must still trust the hardware vendor’s attestation, GPU TEE support (needed for performant inference) is relatively new (NVIDIA Hopper is the first generation with production-ready confidential computing), and trust questions around the TEE hosting entity can undermine the privacy guarantees entirely.<br>Stained Glass Transform (SGT)<br>The Stained Glass Transform is a novel solution to the same problem with a well-studied and rigorous notion of privacy. The solution involves sending obfuscated embeddings instead of raw text to the LLM provider and letting the provider’s endpoint do the rest.<br>In other words, it moves the initial preparatory stages used by all LLMs (tokenization and embedding) to the user’s side. Using a trained machine learning model (their secret sauce), the embedding (and thus the prompt) is obfuscated. The key insight, however, is that this obfuscated prompt provides two empirically validated guarantees:<br>$\textbf{(Utility preservation)}$ The LLM output on the obfuscated prompt is close to the LLM output on the raw text.<br>$\textbf{(Privacy guarantee)}$ The raw text prompt is hard to reverse-engineer from the obfuscated embeddings.<br>Viewing note This interactive walkthrough is optimized for laptop-sized displays and mobile devices in portrait orientation. Other viewports — including landscape mobile and tablet — remain functional but may exhibit reduced layout fidelity.
Conventional Flow
Private Flow
User Prompt "How does a black hole form?"
Embedder<br>d = 2,048
Embedder<br>d = 2,048
Stained Glass Transform<br>obfuscate embeddings<br>using a trained model
LLM<br>LLaMA 3.2 1B · frozen
LLM<br>LLaMA 3.2 1B · frozen
Response (conventional)
Response (prompt private)
Token Embeddings<br>d = 2,048 dims · 7 tokens<br>LLaMA 3.2 1B
Training the SGT<br>The SGT paper is well-written and in this post I have simply followed their approach. While I describe my implementation choices such as architecture (which may not be fully detailed in the paper for IP reasons), I encourage the reader to refer to the paper for further details. The high-level idea is that you run a small local network (called the SGT) that takes the embedding sequence and replaces it with a perturbed version. Thus, the server never sees tokens or raw embeddings; it only ever processes the scrambled version....