Engineering · July 11, 2026 · 7 min read

Deterministic context packs and how prompt caching compounds

Prompt caching can cut the cost of repeated context by an order of magnitude — but only if the context is byte-for-byte identical across calls. In a multi-agent system, that's exactly the property most memory layers quietly destroy.

The cache is a prefix match

Prompt caching works on one rule: it's a prefix match. The provider caches the tokens up to a breakpoint, and any byte change anywhere in that prefix invalidates everything after it. Identical prefixes across calls make repeated context cost a fraction; change one byte and you pay full price again.

For a single agent with a stable system prompt, that's easy to get right. For a fleet of agents pulling context from shared memory, it's easy to lose by accident.

How memory layers bust the cache without telling you

The context an agent receives is assembled from memory, and assembly is where determinism goes to die:

• Volatile fields. A timestamp, a freshly generated request id, or a "retrieved at" note baked into the block changes the bytes every call. Cache read: zero.
• Unstable ordering. Re-ranking by a score that shifts, or serializing a set/map without a fixed key order, reorders the same facts into different bytes — same information, different prefix, no hit.
• Per-agent noise. Interpolating the agent's name or run id into the shared preamble gives every agent a different prefix, so nothing caches across the fleet.

None of these throw an error. The usage meter just quietly reports cache_read_input_tokens: 0, and the bill grows with every agent you add.

Deterministic packs, by construction

The fix is to make the context block a pure function of its inputs: the same query, scope, and covered sequence always produce the same bytes. Concretely — a fixed field order, no volatile values inside the cached region, and stable serialization — then expose a hash so you can verify it:

const pack = await lore.pack({
  query: "current state of auth work",
  scopes: { team: "platform" },
  tokenBudget: 2000,
});

pack.packHash    // identical inputs -> identical hash -> identical bytes
pack.savedTokens // what the cached prefix bought you, in tokens

Volatile facts — the freshness lag, the timestamp your app wants to show — live outside the cached prefix, appended after the breakpoint, so they never invalidate the shared block. The stable part caches; the changing part rides after it.

Why it compounds in a fleet

Determinism turns caching from a per-agent win into a fleet-wide one. When five agents on the same team request the same scoped context, a deterministic pack gives them the same cacheable prefix — so the second through fifth read it from cache instead of paying to build it again. The savings don't add up linearly; they compound with every agent and every turn that shares a prefix. This is the mechanism behind the claim that a coordination layer makes the token bill go down as you add agents, not up.

Measure it, don't assume it

Determinism is verifiable, so verify it. If cache_read_input_tokens is zero across calls that should share a prefix, something volatile slipped into the cached region — diff the bytes of two packs and you'll find the timestamp or the unsorted map. A pack_hash turns that from an archaeology dig into a one-line check.

The one-line version

Prompt caching rewards byte-for-byte repetition, and shared memory is where that repetition either compounds or quietly dies. Make packs deterministic, keep volatile data after the cache breakpoint, expose a hash to check — and adding agents stops multiplying your token bill.

Deterministic packs are how Lore, an open-source coordination memory layer, makes savings compound. See how the write path is built, how it fits context engineering for teams, or the repo on GitHub.