The Context Bill — LLM memory on TemporalStore

LLM Context Management

TemporalStore: a disruptive open-source engine that can cut LLM token cost and improve answer quality.

One open-source, Rust-native temporal engine that shrinks the context you send each turn, sharpens the answers, and remembers across every session, device, and agent on your team.

Open source · Apache-2.0<br>Rust-native temporal engine<br>~9 min read

The short version<br>If you run LLM agents in production and pay per token, replaying a growing conversation into every<br>prompt can quietly become a big chunk of the bill — sometimes rivaling inference itself.<br>TemporalStore replaces that with a small, ranked,<br>source-backed ContextPack : on a real agent history it answered every question better<br>while sending a tiny fraction of the tokens. And because memory lives in the engine — not the prompt —<br>it persists across sessions, devices, agents, and your whole team , something<br>point-in-time retrieval over one transcript can't do. Open source, self-hostable. Come build it with us →<br>github.com/matrixarkai/TemporalStore.

The usual approach: append everything, every turn

The typical way to give an agent memory is the simplest one: on each new turn, append<br>all of the prior context — the earlier prompts and the model's own intermittent output —<br>back into the next input window, and send it again.

It works, so it ships. But it has two quiet problems.

Most of what you re-send is noise. A lot of that appended history — especially the<br>intermittent output: tool results, logs, stack traces, file dumps — was useful for a single step<br>and then just sits in the window, re-sent every turn, diluting the signal the model actually needs now.<br>You pay for it on every call, and it crowds out the few lines that matter.

And nothing is learned. It's all trapped in one session — there's no cross-session<br>memory, and no long-term profile that gets better as the history deepens. The agent never<br>accumulates what it figured out about the user, the project, or the decisions already made; when the<br>window rolls or the session ends, it starts from zero.

So you pay more (re-sending a growing, noisy transcript) and remember less (nothing survives the<br>session). That's the trap.

APPEND EVERYTHING, EVERY TURN<br>RANKED CONTEXTPACK

prompt<br>intermittent output (noise)

turn 1turn 20turn 40

re-sent every turn · mostly noise · needed fact buried<br>$$$ grows · one session — nothing learned

events + tools

once

TemporalStore<br>denoise · rank · profile

~1.3k tokens

noise dropped · facts kept<br>+ cross-session profile, improves with depth

the model sees only what matters — source-backed

Appending everything re-sends a growing, noisy transcript — mostly intermittent output — and forgets between sessions. TemporalStore denoises and ranks it into a small, source-backed ContextPack, plus a cross-session profile that improves as history deepens.

The "solution" that becomes its own stack

So you add retrieval. Reasonable instinct. But watch what "agent memory" quietly turns into in production:

a vector database to store, version, and semantically search the embeddings,

a summarization / extraction pipeline to compress old turns into something retrievable,

and a memory service to glue it together, decide scope, and rank — usually a bespoke one.

Three systems, and you're still replaying transcripts on the side. Three things to run, monitor,<br>and keep consistent — all to answer one small question on every turn: what does this agent actually<br>need in front of it right now?

The bet behind TemporalStore is that agent memory is one temporal problem — and deserves one temporal engine, not a stitched-together stack.

It's open source (Apache-2.0), Rust-native, runs from a single Docker command, and ships with<br>reproducible benchmarks instead of adjectives. Here's how it works, and why the numbers land.

vector DB (+ embeddings)<br>summarize / extract<br>memory glue service

One temporal engine<br>ingest · rank · ContextPack<br>one temporal index · one durability path

The memory stack, consolidated: a vector DB, a summarization pipeline, and their glue collapse into one temporal engine.

The core idea: send a ContextPack, not a transcript

The design draws one clean line. Your model reasons and extracts; the engine remembers and serves.

# the flow — you own the model, TemporalStore owns the memory<br>raw query / event / resource + hints<br>→ extract intent, time, filters, entities (your runtime, or MatrixArk)<br>→ store compact serving records (TemporalStore)<br>→ retrieve a token-budgeted ContextPack (TemporalStore)<br>→ final LLM = local context + ContextPack<br>→ the answer becomes tomorrow's memory<br>Notice what TemporalStore doesn't do: it never calls an LLM or an embedding model.<br>Your harness — or a managed runtime like MatrixArk<br>(matrixark.ai) that sits on top — produces the vectors and structured<br>intent. TemporalStore stores them and does the fast, bounded work at serving time.

That work produces the whole point of the system:...