04

Prove Recall and Erasure

Finish Maya's support agent with hybrid memory, measurable retrieval, and deletion evidence.

Choose hybrid only when roles differ

One database or SDK rarely solves every memory job cleanly. Maya's agent needs a semantic preference across tickets and checkpointed progress inside one ticket, so use Mem0 for approved durable facts and LangGraph persistence for transient workflow recovery.

def prepare_support_context(tenant_id: str, user_id: str, ticket_id: str, question: str):
    scope = f"{tenant_id}:{user_id}"
    facts = memory.search(question, user_id=scope)
    checkpoint_config = {"configurable": {"thread_id": f"{scope}:{ticket_id}"}}
    return {"facts": facts, "checkpoint_config": checkpoint_config}

This function makes two lifetimes explicit. Never promote checkpoint transcripts into profile memory without a separate approval path.

Test relevance and isolation

Retrieval quality is a product property, not a demo. Keep a small fixed evaluation set with expected relevant facts, expected absent facts, and a threshold you can regress-test.

def test_maya_memory_is_relevant_and_isolated():
    maya = memory.search("best channel for Maya update", user_id="acme:maya")
    assert any("email" in hit["memory"].lower() for hit in maya)

    stranger = memory.search("best channel for Maya update", user_id="other-tenant:maya")
    assert not any("email" in hit["memory"].lower() for hit in stranger)

Record backend version, fixture version, score, and failures. This proves this test case only; it does not prove universal safety or relevance.

Erase from every owned store

Deletion is incomplete if one copy remains in a vector index, checkpoint store, cache, or log. Build an inventory of every store your design owns, then delete by tenant-qualified reference and verify no later search returns the fact.

from typing import Protocol

class MemoryBackend(Protocol):
    def search(self, query: str, *, user_id: str) -> list[dict]: ...
    def delete_user(self, *, user_id: str) -> None: ...

def erase_user(tenant_id: str, user_id: str):
    scope = f"{tenant_id}:{user_id}"
    memory_backend.delete_user(user_id=scope)
    checkpoint_backend.delete_thread_prefix(f"{scope}:")
    audit_log({"event": "memory.erase", "scope": scope})

erase_user("acme", "maya")
assert memory_backend.search("Maya email preference", user_id="acme:maya") == []

MemoryBackend is an adapter contract, not a claim that every provider exposes the same delete call. Map it to each selected provider's documented deletion API; backups, legal holds, and immutable audit logs can have separate retention rules. Explain those exceptions to users instead of promising immediate universal erasure.

Operate memory as governed data

Memory is user data with an AI-shaped retrieval path. Assign an owner, review write policy, measure retrieval, restrict tenant access, set retention, and keep deletion receipts.

ControlEvidence
Tenant isolationCross-tenant fixture returns no Maya fact
Write policySource and policy version stored with each fact
RelevanceVersioned retrieval fixture passes threshold
ErasureDelete receipt plus post-delete empty search
Incident responseMemory ID and backend reference in audit event

Capstone: Maya asks for an update; agent recalls only her approved email preference, resumes only her ticket, and can produce an erase receipt on request. Start with explicit records, add products only for a demonstrated job, and keep policy plus proof in your application boundary.