07

Publish Events Without Losing Business State

Make HelixWorks workflows reliable when a database commit succeeds but a network call, event delivery, or consumer fails.

The enterprise problem and today’s slice

Enterprise problem: Supplier onboarding crosses independently deployed services. A process can approve a blueprint, start a run, call a connector, and publish evidence. A partial failure must not lose state or repeat a supplier-side effect.

Whole-course context: Days 1–6 established the product path. Today replaces fragile synchronous assumptions with versioned events, transactional outboxes, idempotent consumers, dead-letter queues, and explicit compensation.

Today’s slice: Persist BlueprintApproved.v1 beside the approved blueprint, relay it to EventBridge, consume it through SQS, and make duplicate delivery harmless.

End-of-day evidence: One approval survives a relay outage, two deliveries produce one projection, and a failed supplier action produces a compensating record instead of invented success.

Still unsolved: The event path is reliable, but connector authorization, environment promotion, and full recovery drills arrive on later days.

Repository lab

Use the public HelixWorks service-kit at course-v0.1.0. The immutable tag keeps this lesson's line-by-line behavior stable while develop continues evolving.

export interface OutboxRepository {
  append(record: OutboxRecord): Promise<void>;
  pending(limit: number): Promise<readonly OutboxRecord[]>;
  markPublished(id: string, publishedAt: Date): Promise<void>;
  markFailed(id: string): Promise<void>;
}

export interface InboxRepository {
  claim(consumer: string, idempotencyKey: string): Promise<boolean>;
}

The interfaces are dependency-inversion boundaries: application code owns the contract; PostgreSQL, EventBridge, and test doubles implement it. This keeps the use case single-purpose and the transport replaceable.

from commit to recoverable workflow

One durable fact

The business record and its publication intent commit together.

Never publish an event as the only proof that the business transaction happened.

Asynchronous publication

A retryable relay separates database availability from broker availability.

Mark an outbox record published only after the broker accepts it.

At-least-once consumption

The consumer claims an idempotency key before applying the event.

Assume every event can arrive twice, late, and out of order.

Failure containment

Poison messages stop blocking healthy traffic and become visible work.

Bound retries; route exhausted work to a DLQ with correlation and causation identifiers.

Business recovery

Technical retry and business compensation become different decisions.

Retry only idempotent technical work; model irreversible business recovery explicitly.

Implementation and verification

  1. Run pnpm test --filter @helixworks/service-kit.
  2. Stop the broker after the database commit and verify the pending outbox record remains.
  3. Restore the broker and flush the relay twice.
  4. Verify the evidence projector claims the idempotency key once.
  5. Force a connector timeout and inspect the compensation and DLQ evidence.

Practical next action and falsifiable evidence

The claim is falsified if an approval disappears during broker downtime, duplicate delivery creates duplicate evidence, or a failed external effect is reported as completed. Passing evidence includes the committed outbox row, broker event ID, inbox claim, duplicate suppression result, and compensation or DLQ record under one correlation ID.