Semantic Memory: Facts That Survive

Enterprise problem: Long-term memory must be a governed retrieval system, not a transcript archive: a mailbox owner can reserve mornings for focus yet a fresh run may choose 09:00 when that fact vanishes, while storing everything creates privacy, relevance, and cross-tenant leakage risk.

Whole-course context: The incoming evidence is Day 02’s authorized, idempotent model–tool trace; today adds durable factual continuity to that loop without yet learning from past outcomes or changing operating procedures.

Today’s slice: Normalize an explicit preference into a governed semantic-memory record, isolate it by authenticated tenant and user, and retrieve only active facts that can change Alice’s scheduling decision.

End-of-day evidence: Two correlated executions prove the preference write and later after-10:00 meeting choice, while a cross-tenant probe returns zero records and a supersession probe returns only the active fact.

Still unsolved: Episodic examples, approved procedural rules, background extraction, large-scale vector infrastructure, and full production operations remain deferred.

A flawless tool loop still makes the wrong decision when relevant context is absent, because a model cannot reason over information the application did not supply. The smallest memory-aware model is past fact → governed store → relevant retrieval → current decision.

• General rule: Persist normalized knowledge that future decisions need; do not confuse persistence with replaying every message.
• Simple example: Store “The user avoids meetings before 10:00,” not three turns of acknowledgement.
• Realistic example: A fresh Alice request retrieves the preference before checking calendar availability.
• Failure mode: A larger context window postpones forgetting but increases noise, cost, retention exposure, and contradictory instructions.
• Decision rule: Use semantic memory for durable facts, preferences, relationships, and constraints. Avoid it for temporary dates, task-local reasoning, secrets outside product purpose, or untrusted claims without authority.

Semantic memory answers “what is true for this user now?” It does not yet answer “what happened in a similar case?” or “how should the system behave?” Those are episodic and procedural memory, introduced later.

Using one undifferentiated store for execution progress and durable knowledge turns every thread into an accidental archive. Checkpoints and semantic memories can share database technology, but their keys, lifecycle, retrieval, and purpose differ.

Vector similarity is not access control: searching globally and filtering afterward can expose another tenant’s record to the model or reranker. The authenticated runtime must constrain the eligible set before ranking.

def memory_namespace(tenant_id: str, user_id: str):
    return (
        "email-assistant",
        tenant_id,
        user_id,
        "semantic",
    )

• General rule: Derive application, tenant, user, and memory type from trusted execution context.
• Simple example: User 123 cannot search user 456’s tuple even with an identical query.
• Realistic example: Alice’s email body may suggest a preference, but it cannot select the mailbox owner’s namespace or authorize a write.
• Failure mode: The model passes tenant_id as a tool argument and prompt injection requests another customer’s memories.
• Decision rule: Apply identity and policy filters in the database/query layer before approximate nearest-neighbor ranking; then test cross-tenant retrieval with a zero-result release gate.

A content-only record cannot reveal who asserted it, whether it is active, or how to rebuild its vector. Governed metadata makes a fact challengeable, correctable, and deletable.

from datetime import datetime, timezone
from uuid import uuid4

memory = {
    "memory_id": str(uuid4()),
    "type": "preference",
    "content": "The user avoids meetings before 10:00.",
    "source_ref": "thread-892:message-14",
    "confidence": 1.0,
    "status": "active",
    "created_at": datetime.now(timezone.utc).isoformat(),
    "embedding_model": "text-embedding-version",
}

• General rule: A durable memory needs content plus provenance, authority, confidence, lifecycle status, scope, and embedding version.
• Simple example: An explicit owner statement has stronger authority than an inferred pattern from one email.
• Realistic example: The 10:00 preference links to its source, remains active, and can later be superseded by a noon boundary.
• Failure mode: Contradictory strings are silently edited in place, destroying history and returning both as equally current.
• Decision rule: Preserve immutable identity and provenance; change lifecycle state and link replacements rather than overwriting history.

Direct model access to database internals turns untrusted email content into durable product state. A tool should accept a proposed fact while trusted context and policy decide where and whether it is stored.

from langchain_core.runnables import RunnableConfig
from langchain_core.tools import tool
from uuid import uuid4

@tool
def manage_memory(content: str, config: RunnableConfig) -> str:
    """Store an explicit, durable user fact or preference."""
    tenant_id = config["configurable"]["tenant_id"]
    user_id = config["configurable"]["user_id"]
    namespace = memory_namespace(tenant_id, user_id)
    memory_id = str(uuid4())
    store.put(namespace, memory_id, {
        "content": content,
        "status": "active",
    })
    return f"Stored memory {memory_id}"

The teaching code shows the interface, not the entire policy. Before persistence, production code checks source authority, sensitivity, durability, contradiction, and consent. Explicit “remember this” requests can use the hot path; inferred facts should normally enter review or background processing.

Injecting every memory makes the prompt slower and more contradictory, while an overly small or poorly filtered set misses the fact that changes the action. Retrieval should optimize for decision value within an authorized scope.

@tool
def search_memory(
    query: str,
    config: RunnableConfig,
    limit: int = 5,
) -> list[str]:
    """Retrieve active memories relevant to the current decision."""
    namespace = memory_namespace(
        config["configurable"]["tenant_id"],
        config["configurable"]["user_id"],
    )
    results = store.search(namespace, query=query, limit=limit * 2)
    active = [
        item for item in results
        if item.value.get("status", "active") == "active"
    ]
    return [item.value["content"] for item in active[:limit]]

• General rule: Filter by scope and lifecycle, rank for relevance, and return only facts that can materially change the decision.
• Simple example: “meeting-time preferences” returns the 10:00 boundary, not an unrelated writing-style preference.
• Realistic example: The constraint narrows Alice’s available slots before the write tool selects 14:00.
• Failure mode: Top-k is increased to hide poor recall, flooding the prompt with stale or contradictory facts.
• Decision rule: Start with a small labelled evaluation set and tune filters, query construction, top-k, and reranking against recall@k, ranking quality, false positives, and zero cross-tenant leakage.

A well-designed store has no product value until retrieved facts influence a bounded action, and a memory write has no governance value unless its use is traceable. Add search and management as peer tools, then record which memory IDs affect the final decision.

tools = [
    write_email,
    check_calendar_availability,
    schedule_meeting,
    search_memory,
    manage_memory,
]

agent_model = model.bind_tools(tools)

For the recurring request, the loop searches “meeting-time preferences,” retrieves the active after-10:00 constraint, checks availability under that constraint, and chooses 14:00. Memory changes the action rather than decorating the reply.

The action for today is to run the paired evidence test: write the explicit preference in one execution, invoke a fresh Alice request in another, and verify the cited memory ID and compliant slot. Then attempt a cross-tenant search and a noon supersession; block release unless leakage is zero and only the new active rule is returned.

Durability without scope and lifecycle creates persistent risk, while retrieval without decision impact creates expensive decoration. Semantic memory is useful only as governed product state connected to observable action.

• Semantic memory stores normalized durable facts and preferences, not complete transcripts.
• Checkpoints preserve execution progress; semantic memory serves future executions and threads.
• Tenant and user authorization must constrain eligible records before vector ranking.
• Memory records need provenance, confidence, status, sensitivity, timestamps, and embedding version.
• Retrieval should return the smallest active set that can change the current decision.

A memory demo is incomplete if it shows only a successful search. Complete the write, use, isolation, and lifecycle probes before adding episodic or procedural memory.

• [ ] I can distinguish a checkpoint from semantic memory.
• [ ] I can normalize a conversation into one durable preference with provenance.
• [ ] I can explain why the runtime, not the model, supplies tenant and user scope.
• [ ] I can supersede a preference without silently rewriting history.
• [ ] I can prove a fresh scheduling run used the right memory and leaked zero cross-tenant records.