The Session You Cannot Take With You | EARENDIL

The Session You Cannot Take With You

Date:Thu, 30 Jul 2026

From:Earendil Engineering rfc@earendil.com>

To:You

Subject:The Session You Cannot Take With You

The original promise of an inference API was wonderfully simple: send some<br>input, receive some output. If you kept both, you had the conversation. You<br>could inspect it, archive it, replay it, or give it to a different model.

That abstraction was never completely true. Prompt caches<br>live on somebody else's GPUs, tokenization differs<br>between models, and sampling is not reproducible (and quite intentionally so).<br>But the semantic record of a session in the form of a transcript could still<br>belong to the user. A transcript should contain the instructions, messages,<br>tool calls and tool results. Another sufficiently capable model might not<br>continue identically, but it could understand what happened and take over.

Inference APIs are frustratingly moving away from that property, at least<br>somewhat. They increasingly return a mixture of text and provider-bound state<br>that is very intentionally non-portable.

reasoning tokens that are billed to the user but returned only as opaque,<br>encrypted blobs, with useless summaries at best

web searches where the model sees source material the client never sees

compacted context that only the original provider can decrypt

subagent instructions and messages hidden from the application running the<br>agents in the form of encrypted payloads

file, vector-store, container, and cache references that cannot be resolved<br>anywhere else.

response and conversation state that is entirely keyed by IDs that are<br>stored fully on the provider's servers

Each feature comes with a basic justification that's trivial for a provider to<br>come up with, along with good arguments for why this is good for the user. Together<br>all of these things change the ownership reality of an AI session: the<br>transcript on your machine is no longer your session but a partial view of a<br>session whose operational state belongs to an inference provider and not you.

We are not fans of this direction, and we want to talk a bit about what it means<br>to you, as a user, and what it means to us, as people developing tools in this<br>space.

A Practical Test for Session Ownership

By a portable session we do not mean that switching from one model to another<br>must produce the same next token. That's a given because models have different<br>capabilities, trained personalities, context windows, and ways of working with<br>tools. And well, it's all quite nondeterministic anyway. Portability means<br>something more modest:

const transcript = session.export();<br>revokeCredentials(oldProvider);<br>session = newProvider.continueFrom(transcript);

The archive should contain enough intelligible information for another model to<br>continue the work. It should not require the old provider to dereference an ID,<br>decrypt a blob, remember a search result, or reconstruct a summary.

This gives us five useful tests:

Inspection: Can the user see what the model saw, what tools did, and what<br>agents told each other?

Export: Is the session self-contained, apart from ordinary artifacts that<br>can also be downloaded?

Replay: Can another implementation reconstruct a semantically equivalent<br>context?

Audit: Can a human explain why the system took an action after the fact?

Deletion: Can the user identify and remove every server-side copy on which<br>the session depends?

A response ID is not a transcript, a ciphertext the user cannot decrypt is not<br>user-controlled state, a list of citations is not the evidence that was placed<br>in the model's context by a search result.

Encryption for Whom?

The naming and marketing around these features can be misleading.<br>encrypted_content sounds like a privacy feature under the user's control.<br>Usually it is a capsule that the client cannot read and only the provider can<br>open. The provider chooses the keys, decrypts the content for its own models,<br>and defines where the data can be replayed.

A better term is provider-sealed state .

Provider sealing can have a real privacy benefit. OpenAI, for example, can<br>return encrypted reasoning to a client using store: false, then decrypt it in<br>memory on the next request without persisting the intermediate state. That is<br>better than requiring server-side conversation storage, particularly for Zero<br>Data Retention customers. But, remember, there is not really anything that<br>needs encryption to begin with!

Most importantly, this encryption does not hide the data from the inference<br>provider; it hides it from you.

Stored Conversations Turn a Transcript into a Pointer

OpenAI's Responses API stores responses by default. Its documentation says<br>response objects are retained for at least 30 days by default; items attached to<br>a Conversation are not subject to that 30-day TTL. store: false is available<br>and should be used, as it makes it work more like completions: the data is not<br>stored on OpenAI's servers.

The...