Semantic Memory: Facts That Survive

Imagine the user says, “I reserve mornings for focused work; do not schedule meetings before 10:00.” A week later, a fresh email asks the agent to schedule Alice tomorrow. Graph state from the earlier interaction is gone, so the tool loop may choose 09:00 even though its reasoning is otherwise flawless.

This is a context-availability failure. The model cannot apply information that the system did not retrieve. Making the context window larger only delays the problem: full histories become costly, noisy, and difficult to govern. The agent needs a separate, durable memory plane.

Semantic memory stores normalized knowledge: facts about people and projects, preferences, relationships, and stable constraints. Instead of preserving a transcript, it records the useful proposition: “The user avoids meetings before 10:00.”

A graph checkpoint persists the progress of one thread: which nodes ran, which messages exist, and where execution can resume after failure. Semantic memory crosses threads and requests. The two may use the same database, but they have different keys, lifecycles, and retrieval rules.

Checkpoint question: What happened in this execution?

Semantic-memory question: What durable fact should future executions know?

Conflating them creates an accidental transcript archive and makes deletion, retention, and relevance much harder.

A store must isolate application, tenant, user, and memory type before similarity ranking. Filtering after vector search risks returning another user’s fact to the model.

def memory_namespace(tenant_id: str, user_id: str):
    return (
        "email-assistant",
        tenant_id,
        user_id,
        "semantic",
    )

The namespace is not cosmetic folder organization. It is part of the authorization model. The authenticated runtime supplies these identifiers; the email body and model never get to choose them.

A production memory needs enough metadata to be challenged, updated, and deleted. Content alone cannot reveal where the claim came from or whether it remains active.

from datetime import datetime, timezone
from uuid import uuid4

memory = {
    "memory_id": str(uuid4()),
    "type": "preference",
    "content": "The user avoids meetings before 10:00.",
    "source_ref": "thread-892:message-14",
    "confidence": 1.0,
    "status": "active",
    "created_at": datetime.now(timezone.utc).isoformat(),
    "embedding_model": "text-embedding-version",
}

Provenance supports correction and audit. Confidence distinguishes an explicit user statement from an inferred pattern. Status allows supersession without silently rewriting history. The embedding model identifier makes future re-indexing possible.

The agent should not manipulate storage internals. A memory-management tool validates a candidate and places it in the authenticated namespace. The simple version below demonstrates the boundary; Day 6 will add policy and background review.

from langchain_core.runnables import RunnableConfig
from langchain_core.tools import tool
from uuid import uuid4

@tool
def manage_memory(content: str, config: RunnableConfig) -> str:
    """Store an explicit, durable user fact or preference."""
    tenant_id = config["configurable"]["tenant_id"]
    user_id = config["configurable"]["user_id"]
    namespace = memory_namespace(tenant_id, user_id)
    memory_id = str(uuid4())
    store.put(namespace, memory_id, {
        "content": content,
        "status": "active",
    })
    return f"Stored memory {memory_id}"

Not every sentence deserves persistence. Temporary dates, untrusted claims from an external sender, secrets, and sensitive data outside the product’s purpose should be rejected or expire quickly.

Memory search converts a natural-language query into an embedding, ranks candidates inside the authorized namespace, filters to active records, and returns a small top-k set.

@tool
def search_memory(
    query: str,
    config: RunnableConfig,
    limit: int = 5,
) -> list[str]:
    """Retrieve active memories relevant to the current decision."""
    namespace = memory_namespace(
        config["configurable"]["tenant_id"],
        config["configurable"]["user_id"],
    )
    results = store.search(namespace, query=query, limit=limit * 2)
    active = [
        item for item in results
        if item.value.get("status", "active") == "active"
    ]
    return [item.value["content"] for item in active[:limit]]

The goal is not to find every remotely related record. It is to retrieve the smallest set that materially changes the current decision. Precision protects attention, cost, and safety.

manage_memory and search_memory become peers of the email and calendar tools. The model can retrieve context, perform an operational action, or record an explicit preference.

For the next scheduling request, the agent searches for “meeting-time preferences,” retrieves the after-10:00 constraint, checks availability, and chooses a compliant slot. Memory has changed an action rather than merely decorating a response.

Test a small labelled set of requests with known relevant memories. Measure recall at k, ranking quality, false-positive rate, and—most importantly—cross-tenant leakage, which must be zero. Include stale and contradictory memories. An answer can sound reasonable while retrieval silently returns the wrong person’s preference.

Semantic memory provides continuity, but it does not teach triage from experience. Knowing “Alice runs Project North” is different from remembering that messages like Alice’s weekly FYI were previously ignored. That is the role of episodic memory.

• Semantic memory stores normalized durable facts and preferences, not complete transcripts.
• Checkpoints preserve one execution; long-term memory serves future executions and threads.
• Tenant and user filtering must happen before vector ranking.
• Memory records need provenance, confidence, lifecycle status, and embedding version.
• Retrieval should return the smallest active set that can change the current decision.

• [ ] I can distinguish a checkpoint from semantic memory.
• [ ] I can normalize a conversation into one durable preference.
• [ ] I can explain why the runtime, not the model, supplies the namespace.
• [ ] I can name four metadata fields needed beyond memory content.
• [ ] I can define a retrieval test that detects cross-tenant leakage.