01

Put the Workflow Before the Agent

Week 1 of 10 · Build the success contract, 20 cases, and deterministic baseline before adding autonomy • Reference implementation: https://github.com/ZGTR/enterprise-access-agent-evals/tree/v1.0.2

System map · Day 01

Whole-system design

Five stable layers. Today's work is expanded and linked; the rest stays in context.

Entry and authorization control

Ahead — Tenant and policy authority · Approval gate

Request entry

Source-backed today

Binds one restore-access request to trusted tenant and actor context.

Compute and execution

Ahead — Policy retrieval compute · Typed tool broker

Agent runtime compute

Source-backed today

Executes the deterministic baseline before autonomy earns any decision.

Enterprise resource boundary

Simulated enterprise system

Source-backed today

Returns the authoritative before and after access snapshots.

Storage and state

Run and effect state

Source-backed today

Uses one state version and idempotency key for the bounded mutation.

Evaluation evidence store

Source-backed today

Captures twenty classified cases and normal plus denied receipts.

Evidence and release control

Ahead — Evaluation harness compute · Trace and diagnosis plane · Release gate

Traversed today

bound request · Request entryAgent runtime computeauthorized effect · Typed tool brokerSimulated enterprise systempost-effect state · Simulated enterprise systemAgent runtime compute

Restore access without hiding the business rule

An employee asks, “Restore Sarah’s Salesforce access.” The tempting response is to add a model and let it choose tools, but the business path is already mostly known: identify Sarah inside the authenticated tenant, read policy and current state, form a proposal, obtain approval, apply one authorized change, then read the state again. This week you will encode that path as a deterministic workflow—a fixed sequence whose branches are ordinary code—and prove its normal and denied outcomes.

The recurring example uses two synthetic tenants, Acme and Globex. Both contain a Sarah, a Salesforce resource, and colliding policy identifiers. A tenant is an isolated customer namespace; runtime authentication supplies tenant_id, never the request text or model. The first observable result is a receipt showing that Acme Sarah was restored while Globex state stayed unchanged.

Request entry owns the authenticated actor, tenant, and user text. Its public contract is pinned at contracts.py:

@dataclass(frozen=True)
class RuntimeContext:
    tenant_id: str
    actor_id: str

This declaration is interpreted by the Python workflow. It changes in-memory run state, consumes CPU and memory in one local process, and becomes trustworthy only when the result includes the bound tenant and actor IDs.

Write twenty cases before implementation

A vague success sentence lets almost any demo pass, so define cases before code. An evaluation case is a frozen input, initial state, expected outcome, prohibited effects, and decisive evidence. Write 20: normal, boundary, failure, and adversarial. Each case has a reference solution and at least one negative control that must fail, preventing a vacuous grader from reporting success for every run.

The initial dataset belongs in the versioned case store described by evals/dataset.py. Start with this distribution; later weeks expand it without rewriting old expectations:

ClassExampleDecisive evidence
NormalAcme Sarah lacks access and approval is validExpected permission appears after a verified effect
BoundaryTwo Acme employees match “Sarah”No mutation; clarification is requested
FailureState changes between proposal and applyCompare-and-set denial; no overwrite
AdversarialPrompt asks to switch to GlobexZero foreign rows, citations, or effects

One scored checkpoint contains one unknown: given a changed requirement, choose workflow or agent and give one falsifiable reason. All state and policy inputs are supplied. A common misconception is that “uses an LLM” defines an agent; compare instead whether the next legal action follows a repeatable business rule or requires open-ended judgment.

Run the deterministic state transition

The workflow must make authority and state ownership visible. The simulated enterprise system owns access state; its key is (tenant_id, employee_id, resource_id), its version supports compare-and-set, and test cleanup restores the disposable fixture. The workflow cannot claim success from an apply return alone because only a post-effect read proves authoritative final state.

workflow.py interprets the fixed business sequence. No model call is needed to route known steps.

    employees = broker.find_employee(request.employee_query)
    events.append(ToolEvent("find_employee", "ok", tuple(e.employee_id for e in employees)))
    if len(employees) != 1:
        return RunResult(
            "NEEDS_CLARIFICATION", None, 0, tuple(events), error="employee ambiguous or absent"
        )

The adapter in state.py is authoritative for the synthetic permission. The tagged reference uses local CPU, memory, and in-memory dictionaries; this local analogue proves control flow, not Salesforce availability, durable storage, or production tenant isolation.

Run state binds the effect to an expected version and an idempotency key. Idempotency means retrying the same accepted operation does not duplicate its effect.

@dataclass
class SimulationState:
    employees: tuple[Employee, ...]
    policies: tuple[PolicyPassage, ...]
    access: dict[tuple[str, str, str], AccessSnapshot]
    effects: dict[str, str] = field(default_factory=dict)

Challenge the claim, repair it, and clean up

False confidence appears when only the happy path runs. Execute a valid Acme request, then a missing-approval denial; pair both with an unaffected Globex positive control. Next, deliberately pass a stale state version. Expected behavior is rejection without mutation, followed by recovery: read current state, rebuild the proposal, obtain a new approval, and apply once.

Record actor, resource, scope, precondition, expected result, observed result, environment, timestamp, and immutable run or commit ID. A command exit is not the result; before/after state and foreign-tenant non-change are. Clean up only the named disposable fixture, then rerun one normal control to prove cleanup did not corrupt the baseline.

Use the first two hours strictly: 15 minutes for repository and pinned tooling, 20 for system/tool contracts, 45 for 20 cases, 30 for the workflow, and 10 for a workflow-versus-agent decision note. Across the remaining 4–6 hours, read primary guidance, strengthen tests, and write the evidence table. Anthropic’s Building Effective Agents distinguishes workflows with predefined code paths from agents that direct their own process; use that distinction as a design test, not a maturity ladder.

The accepted handoff is R1-system-contract, D20-cases, the normal/denied workflow receipts, and a note naming the few decisions that may need autonomy. Week 2 consumes those artifacts to add a bounded model loop without weakening any workflow invariant.