The Background Learning Loop

Enterprise problem: A mailbox owner needs Alice’s handled request to improve future decisions, but synchronous extraction, redaction, embedding, deduplication, and review make the reply slow and can couple a successful email action to a failing learning subsystem.

Whole-course context: The hosted workflow already emits authenticated routes, tool outcomes, accepted episodes, and exact procedure versions; today consumes those immutable references after the user-visible result is durable.

Today’s slice: Split serving from learning with an outbox-backed event, durable queue, idempotent worker, memory-policy gate, supersession, and lifecycle evidence owned by the provider control plane.

End-of-day evidence: The Alice request completes once, a duplicate learning event produces no duplicate memory, an accepted candidate becomes active, a poisoned procedural candidate is denied, and every transition shares immutable event and trace references.

Still unsolved: Production database and vector-engine selection, cache topology, infrastructure as code, CI/CD rollout, disaster recovery, and fleet-wide SLO operations remain deferred.

Doing optional learning work before replying makes customer success depend on slow, retryable processing. The serving path should finish the authorized job; the learning path should improve future context afterward.

Rule: Publish learning only after the visible outcome is durable, and use an outbox when committing the outcome without the event would violate requirements. A simple example replies first and embeds later. In Alice’s case, the calendar result and response commit with an outbox row, then the worker creates the accepted episode. The failure mode is a dual write where the reply commits but the process crashes before queue publication. Decision rule: use a transactional outbox whenever “result committed, learning event lost” is unacceptable; allow best-effort publication only when missed learning is explicitly tolerable.

async def publish_learning_event(result, context):
    await memory_jobs.publish({
        "event_id": context.event_id,
        "tenant_id": context.tenant_id,
        "user_id": context.user_id,
        "thread_id": context.thread_id,
        "procedure_versions": result.procedure_versions,
        "route": result.classification,
        "tool_outcomes": result.tool_outcomes,
        "feedback_ref": result.feedback_ref,
        "content_ref": result.redacted_content_ref,
    })

Copying raw MIME or attachments into every event expands exposure and deletion work. Prefer opaque protected references plus the minimum routing and provenance fields the worker needs.

Queues can deliver the same event more than once, and extraction models can suggest unsafe memories. The worker must make duplicate processing harmless and treat extraction as a proposal, not a write decision.

Rule: Deduplicate at event level and again at candidate meaning/version level. A simple replay sees the completed event_id and returns. For Alice’s request, one event may propose an accepted scheduling episode and an attacker-authored procedure; policy stores the first and rejects the second. The failure mode is recording completion before persistence, losing candidates after a crash, or recording it too late and duplicating them. Decision rule: claim atomically, make candidate writes idempotent, and mark completion only after every decision commits; retry transient failures and dead-letter exhausted jobs for review.

async def process_memory_event(event):
    if await processed_events.exists(event["event_id"]):
        return

    source = await content_store.read(event["content_ref"])
    candidates = extract_candidates(redact(source), event)

    for candidate in candidates:
        decision = memory_policy(candidate, event)
        if decision == "store":
            await persist_active(candidate, event)
        elif decision == "review":
            await review_queue.publish(candidate.model_dump())
        else:
            await audit_rejection(candidate, event)

    await processed_events.record(event["event_id"])

def memory_policy(candidate, event):
    if candidate.origin == "external_email" and candidate.type == "procedural":
        return "reject"
    if candidate.sensitivity in {"secret", "restricted"}:
        return "reject"
    if candidate.type == "procedural":
        return "review"
    if candidate.confidence < 0.85 or contradicts_active_memory(candidate):
        return "review"
    return "store"

Thresholds require calibration on labelled candidates. Prefer high precision: missing one convenience memory is usually cheaper than persisting a false belief or attacker instruction.

Preferences change and malicious content can target future calls, so an append-only pile of equally active vectors is unsafe. Preserve history while making retrieval eligibility and deletion behavior explicit.

Rule: Create a new record and supersede the old one atomically; never silently rewrite learned content. A simple change from “meetings after 10:00” to “after 09:00” leaves only the new record active. The realistic failure is retrieving both constraints equally or embedding “remember forever” from Alice’s quoted text as policy. Decision rule: contradictions and high-impact procedures go to review; external procedural candidates are rejected; retrieval includes only authorized active records.

async def supersede_memory(old_id, new_memory):
    async with database.transaction():
        old = await memories.lock(old_id)
        await memories.update(old_id, {
            "status": "superseded",
            "superseded_by": new_memory.memory_id,
        })
        await memories.insert({
            **new_memory.model_dump(),
            "status": "active",
            "supersedes": old_id,
        })

Deletion is a workflow, not one row removal: invalidate retrieval and caches immediately, then remove vectors, replicas, blobs, and derived episodes according to policy while retaining only legally required minimal audit evidence.

A background architecture can hide stale queues and low-quality memories unless the event is observable from publication through retrieval. Measure the loop as a decision system, not only as background worker uptime.

Without explicit ownership and evidence, “background learning” can mean lost events, duplicate memories, or delayed poisoning. Keep serving fast and learning governed.

• Commit the user-visible result before optional learning, using an outbox when event loss is unacceptable.
• A queue delivers work; it is not long-term memory or an authoritative store.
• Make event handling and candidate writes idempotent because redelivery is normal.
• Extraction proposes; policy stores, reviews, or rejects based on authority, sensitivity, confidence, provenance, contradiction, and impact so memory poisoning cannot turn untrusted content into durable authority.
• Supersede rather than rewrite, and make retrieval invalidation the first step of deletion.

The next action is to prove one event through happy, duplicate, poisoned, and recovery paths before increasing worker concurrency. Use the Alice interaction as the shared fixture.

• [ ] Commit Alice’s visible result and outbox row atomically, then confirm the response returns before worker completion.
• [ ] Relay the event with tenant/user scope, procedure versions, feedback/content references, and immutable event/trace IDs.
• [ ] Process the event twice and verify one accepted memory, one response, and one tool side effect remain.
• [ ] Reject the external procedural-injection candidate while storing or reviewing the safe accepted-outcome candidate.
• [ ] Supersede a conflicting active preference and prove retrieval returns only the new record.
• [ ] Simulate worker failure, retry, dead-letter, and replay while observing queue age and terminal evidence.
• [ ] Exercise deletion across active retrieval, cache, vector, blob, replica, and derived-record locations.