Storage Isn't the Hard Part Anymore. Deciding What Stays Relevant Is.
Every memory system solves storage. Almost none solve what happens to memories after they're stored.
Real-World Failure
A legal research assistant ships to a mid-size law firm. Lawyers use it daily across months-long cases. The agent accumulates memory fast — case facts, client preferences, jurisdiction nuances, procedural history.
Four months in, a senior partner notices the agent is surfacing arguments from a motion that was withdrawn two months ago. The motion was superseded by an amended filing. The memory system stored the original. It stored the amendment. Both are active. Both surface on retrieval. The agent has no way to know which is current.
The partner catches it before it reaches a brief. But the near-miss triggers a firm-wide review. The question isn't whether the memory was stored correctly — it was. The question is why the system had no mechanism to govern which memories remained relevant as the case evolved.
Nobody had an answer.
Why It Happens Technically
Relevance governance is the problem of determining, at any point in time, which stored memories should influence agent behavior — and which should not. It sounds like a retrieval problem. It isn't. It's a lifecycle problem, and the distinction matters architecturally.
Retrieval systems are optimized to surface memories that are semantically similar to a query. Given a question about a legal argument, a retrieval system returns memories about legal arguments — ranked by similarity score. This is correct behavior. The retrieval system is doing exactly what it was designed to do.
The problem is that semantic similarity and current relevance are orthogonal properties. The withdrawn motion is highly semantically similar to queries about the case's legal strategy. It will surface. The retrieval system has no concept of "withdrawn" — that's a lifecycle state, not a semantic property. Similarity search cannot distinguish between a current argument and a superseded one if both are present in the store with similar embeddings.
Governance requires answering questions that retrieval cannot:
Should this memory still be active? Not "is this memory similar to the query?" but "does this memory still represent valid state?"
Has this memory been implicitly superseded? Not by an explicit contradicting fact, but by an event that changed the context in which it was relevant.
What is the relative authority of competing memories? When two memories conflict — original motion vs amended filing — which takes precedence, and how does the system express that precedence at retrieval time?
These are policy questions, not similarity questions. A retrieval system answers similarity questions. Relevance governance requires a policy layer that retrieval systems don't have and aren't designed to provide.
Why Common Approaches Fail
The importance score pattern. Teams assign importance scores at write time and use them to weight retrieval. Higher importance scores surface more frequently. This helps prioritize memories at the moment of extraction — a critical fact scores higher than a passing observation. It doesn't help with relevance decay. A memory with a high importance score from six months ago retains that score indefinitely. Importance at write time is not relevance at read time. These are different properties that many implementations collapse into one.
The recency weighting pattern. Teams apply time decay to retrieval scores — newer memories surface more readily than older ones. This approximates relevance governance for facts that change over time but penalizes stable memories that are permanently relevant. A constraint established once and never revisited — "client will not accept settlements under $2M" — decays on the same schedule as a transient observation. Recency is not relevance.
The manual deletion pattern. Teams build admin interfaces for explicitly deleting stale memories. This works for known staleness — when a team member identifies a bad memory and removes it. It doesn't work for the class of memories that become irrelevant through accumulated context changes rather than a single identifiable event. Governance by manual deletion is reactive and incomplete by design.
All three patterns treat relevance as a property that can be assigned at write time or managed by time. Relevance is actually a relationship between a memory and the current state of the world — and that relationship changes continuously.
Mental Model: Relevance Is a Relationship, Not a Property
Here's the reframe.
A memory doesn't have relevance. A memory has relevance to a context — and that context changes.
The withdrawn legal motion is highly relevant in the context of the original case strategy. It is irrelevant — actively misleading — in the context of the amended strategy. The memory didn't change. The context did. Relevance governance is the problem of tracking that relationship as context evolves.
This reframe has a specific architectural implication: relevance cannot be computed at write time and stored as a property of the memory. It has to be computed at read time, relative to current context. A memory's relevance score at retrieval time should be a function of:
SEMANTIC SIMILARITY → how related is this memory to the current query
LIFECYCLE STATE → is this memory still active, or has it been superseded or resolved
CONTEXTUAL AUTHORITY → in the current context, does this memory take precedence over competing memories
TEMPORAL CONFIDENCE → how recently has this memory been confirmed as still valid
These four signals combine into a retrieval score that reflects current relevance — not just historical importance or semantic similarity.
The practical version of this model looks like a policy layer that sits between the memory store and the retrieval output:
Memory store → all stored memories, with lifecycle state and metadata
Governance policy → rules about which memories are active in which contexts, how conflicts are resolved, what decay rates apply to which memory types
Retrieval output → memories that pass the governance filter, ranked by combined relevance score
Without the governance policy layer, retrieval output is raw similarity search over all stored memories regardless of lifecycle state. This is the default for most memory implementations — and it's the root cause of the withdrawn motion surfacing alongside the current one.
The governance policy doesn't have to be complex. It starts with a single question applied to every retrieved memory: given everything the system currently knows, should this memory be influencing the agent's response right now? Answering that question correctly is the whole problem.
Production Implications
Legal, medical, and compliance domains. Any domain where information has explicit lifecycle events — motions are withdrawn, diagnoses are updated, regulations are amended — requires governance that tracks those events and suppresses superseded memories. In these domains, surfacing a superseded memory isn't a UX issue. It's a liability. The cost of a governance failure is measured in professional consequences, not user satisfaction scores.
Long-running agent relationships. An agent operating with a user over months or years accumulates memories across multiple life phases. Goals set in phase one may be irrelevant in phase three. Constraints established early may have been lifted. Preferences change. Without governance, the agent's behavior is shaped by the full accumulated history — including the parts that no longer apply. The longer the relationship, the more governance matters.
Multi-agent systems with shared memory. When multiple agents share a memory store — a support agent and a sales agent sharing user context, or a team of research agents sharing a knowledge base — governance becomes a coordination problem. Which agent's writes take precedence? How are conflicts between agents resolved? What memories are scoped to which agents? Without a governance layer, shared memory stores accumulate conflicts that no individual agent can resolve.
Open Problems and Tradeoffs
Governance policies are domain-specific. There's no universal relevance governance policy that works across legal research, healthcare, customer support, and personal assistance. Each domain has different memory lifecycles, different conflict resolution rules, different decay rates. This means governance policies have to be configured per deployment — which adds engineering overhead and requires domain expertise that most teams building general-purpose agents don't have.
Implicit superseding is still unsolved. Explicit superseding — a new fact directly contradicts an old one — is tractable. Implicit superseding — a sequence of events collectively renders a memory irrelevant without any single event directly contradicting it — is not. The withdrawn motion example is implicit: no single memory says "disregard the original motion." The governance system has to infer irrelevance from context, which requires LLM inference and can be wrong.
Governance adds retrieval latency. Applying a policy layer to every retrieval — checking lifecycle state, computing contextual authority, applying decay functions — adds latency to the read path. For high-frequency retrieval in latency-sensitive applications, this overhead is real. Teams have to decide how much governance fidelity they can afford at their latency budget.
How Memgram Approaches This
Memgram's memory types — fact, preference, goal, skill, event, context, constraint — are the foundation of lifecycle-aware governance. Each type carries different decay expectations and conflict resolution semantics. Structured facts in the graph layer have explicit superseding: when a new value arrives for the same predicate, the old value is marked is_current: false with a superseded_at timestamp.
The PipelineTrace makes write-side governance decisions visible — what was stored, what was dropped, what was superseded, and why. When a governance decision is wrong, the trace shows where it happened.
Relevance governance is not fully solved in any system. But making the governance decisions traceable is the precondition for doing anything about it.
