Revocation, Audit, and Observability

Enterprise problem: Removing a user or connector in an admin screen does not stop cached decisions, live sessions, subscriptions, signed URLs, child agents, or queued jobs, and fragmented logs cannot prove when access actually ended.

Whole-course context: The incoming evidence is an immutable deployment with separate provider grants, workload/network controls, generated-app roles, and per-call decision IDs; today makes changes to that authority converge measurably.

Today’s slice: We design versioned revocation, immediate high-risk rechecks, cancellation propagation, cross-boundary lineage, redacted audit events, metrics, and security information and event management (SIEM) export across all three authorization planes.

End-of-day evidence: A revocation run records the request time, first confirmed denial, every convergence leg, unaffected positive controls, residual limits, complete actor chain, policy versions, and immutable trace and event identifiers.

Still unsolved: The adversarial threat suite, release-blocking acceptance thresholds, staged production rollout, rollback drills, and final retirement evidence remain deferred.

Thesis: Revocation is not a row update and audit is not a log pile; authority is safely removed only when every controllable path converges to denial and one redacted lineage proves when, where, and under which version that happened.

Smallest complete mental model: change authoritative state, drive and measure convergence, then reconstruct the decision from immutable evidence.

Boundary: This day covers controllable provider, runtime, and generated-app sessions, caches, streams, jobs, links, runs, and evidence. It cannot recall bytes already downloaded or screenshotted, and it does not pretend that one plane's revocation silently removes grants owned by another.

The customer outcome is a reliable, reviewable implementation of 11 revocation audit and observability. This day introduces the mechanism before policy detail and evidence review; it does not repeat the same customer stories in prose, tables, and diagrams.

Authorization policy service owns decisions; the connector/runtime gateway owns enforcement. Compute: API and isolated worker processes. Storage: PostgreSQL is authoritative for grants and state; Redis is a versioned cache; vault owns secrets; object storage retains redacted evidence only.

Implement the day’s boundary with a current, explicit decision before privileged compute or a downstream call. Bind every effect to a tenant, subject or workload, deployment, resource, and short-lived evidence ID; a cache or model response never grants authority.

Teams often promise “instant revocation” without identifying paths, or collect every payload in case it helps an investigation. The first promise is untestable and the second creates a new secret store, so the design must trade bounded freshness, availability, evidence detail, privacy, and retention explicitly.

A revocation pipeline can report healthy while one subscription or queued side effect survives, so verification must begin from a known authorized Acme run and observe every leg. Build one vertical revoke-and-investigate path before adding broad dashboards.

1. Enumerate the HelixWorks Supplier Onboarding Agent's provider membership, deployment, connector, approval, app audience, session, stream, signed-link, run, child-run, and job authorities; assign each an owner and denial objective.
2. Change one authority and monotonic version transactionally with an outbox event; make each consumer idempotently reject stale versions and report acknowledgment plus first denial.
3. Recheck high-risk Slack execution immediately before broker access, cancel parent and Compliance child runs, close affected app streams, and terminally deny stale jobs while leaving one unrelated Acme grant active.
4. Emit redacted events with stable session, run, child, intent, decision, approval, protocol, connector, downstream, event-offset, and policy-digest identifiers.
5. Probe every leg until its deadline; record request time, first confirmed denial, retries, unaffected positive control, and irrecoverable residual.
6. Query from the app trace to the Slack result, export a signed bounded set to the SIEM, then inject a missing lineage edge and secret canary to prove quarantine and repair.

Falsifiable evidence: the design fails if an old version authorizes, a high-risk job reaches the broker after revocation, a child survives parent cancellation, an unaffected grant is destroyed, a required path misses its objective, raw secret material reaches the ledger, or an incident query cannot join actor to downstream result.

Generic revocation objectives hide the slowest path, so choose one Acme membership removal during an active Campaign Launch run and write the path inventory before changing code. Name the expected denial deadline for the next tool write, child run, app request, subscription, signed URL, and queued job, plus the positive control that must remain allowed.

Run the scenario in a production-like environment and keep the signed convergence report and redacted lineage query. The next threat-testing stage may proceed only when every controllable path either meets its objective or has an explicit owner-approved residual; a missing acknowledgment or join remains a failing result.

The evidence is falsifiable and observable: the claim fails when any controllable path remains usable after its deadline, any unaffected positive control is denied, or the signed lineage cannot connect the revocation event to every terminal outcome.

Follow revocation from transaction to projection. workers/evidence-projector/src/application/project-evidence.ts makes PubSub delivery idempotent:

const event = eventEnvelopeSchema.parse(rawEvent);
const claimed = await this.inbox.claim(
  'evidence-projector.v1',
  `${event.tenantId}:${event.idempotencyKey}`,
);
if (!claimed) return false;

await this.projection.append(this.toEvidence(event));
return true;

The broker commits CapabilityRevoked.v1 to its outbox; a relay publishes it; this worker interprets it and appends one tenant-scoped record. CPU validates and de-duplicates, storage holds inbox and projection state, and one record after two deliveries proves convergence. Outbox, relay, inbox, and projector retain one responsibility (SRP); ports use IoC/DI; the shared envelope avoids duplicate integration logic (DRY/PubSub).

pnpm --filter @helixworks/evidence-projector test
pnpm smoke:product

Deliver the revocation twice, then decide with the revoked capability. Expect one evidence row and allowed: false; an unrelated grant must allow. A duplicate row, post-revocation allow, leaked credential reference, or damaged positive control falsifies the claim.