Bound the Agent Loop and Its Tools
Week 2 of 10 · Add decide → act → observe only where Week 1 found open-ended judgment • Public source target: https://github.com/ZGTR/enterprise-access-agent-evals/tree/v1.0.2
System map · Day 02
Whole-system design
Five stable layers. Today's work is expanded and linked; the rest stays in context.
Entry and authorization control
Covered — Request entryAhead — Tenant and policy authority
Approval gate
Source-backed today
Binds approval to the exact proposal digest and policy version.
Compute and execution
Ahead — Policy retrieval compute
Agent runtime compute
Design target · not proved
Runs decide, act, observe under maximum-step and output budgets.
Typed tool broker
Source-backed today
Validates all five typed tools without treating descriptions as authority.
Enterprise resource boundary
Simulated enterprise system
Source-backed today
Accepts one authorized effect and exposes post-effect state.
Storage and state
Covered — Run and effect state
Evaluation evidence store
Source-backed today
Adds ten malformed, exhausted-budget, and approval cases.
Evidence and release control
Ahead — Evaluation harness compute · Trace and diagnosis plane · Release gate
Traversed today
Replace one decision, not the safety contract
Week 1 produced R1-system-contract, 20 frozen cases, and a deterministic restore-access receipt. The remaining problem is language variation and occasional ambiguity: code can enforce the sequence, but it cannot cheaply enumerate every way a person describes an employee or resource. This week you add an agent loop, a bounded cycle in which a model chooses a legal next action, receives the observed result, and either continues or stops.
Autonomy changes who selects the next tool; it does not create authority. Tenant scope, approval verification, compare-and-set, idempotency, and post-effect inspection remain ordinary code. The observable result is that the agent handles a paraphrased request while the same missing-approval and foreign-tenant denials still pass.
The tagged runtime in agent.py proves a deterministic provider seam and a hard step floor; a provider-backed decide → act → observe loop remains a design target rather than a claimed implementation:
if self.budget.max_steps < 6:
return RunResult("STOPPED", None, 0, (), error="step budget exhausted")
return restore_access(self.provider.normalize(request), context, state, approval)
The Python process interprets this loop, model inference or a deterministic double supplies the decision, and the broker changes only validated software state. CPU, memory, network, and model tokens are bounded resources; a terminal receipt and step count prove the stop rule ran.
Give the model five narrow handles
A broad “manage access” tool hides policy, state, approval, and mutation inside one opaque effect. Narrow tools make each boundary independently testable. A typed tool has a named input/output schema that rejects missing, extra, or malformed fields before business execution.
The five typed Python method boundaries are fixed in tagged tools.py. Turning those signatures into provider-facing JSON-schema validation is part of this week’s implementation exercise, not a capability claimed by the offline tag:
| Tool | Model may supply | Runtime always supplies | Effect |
|---|---|---|---|
find_employee | query | tenant | read candidates |
search_policy | query | tenant, as-of version | read citations |
inspect_access | selected employee/resource | tenant | read state/version |
propose_access_change | bound evidence references | tenant, request digest | pure proposal |
apply_access_change | proposal/approval references | tenant, state version | one mutation |
Tool descriptions help selection; they are not permissions. The broker resolves trusted context separately and returns a non-enumerating absence for foreign IDs. In your provider-facing adapter, test malformed JSON, unknown fields, a fabricated tenant_id, and an unavailable tool. Each must stop before adapter code.
Bind approval to one proposed effect
Human approval is useless if it means “do something for Sarah later.” The approval must bind tenant, actor, employee, resource, action, proposal digest, policy version, and expiry. This binding makes later substitution or replay observable and rejectable.
The receipt data type belongs in contracts.py; enforcement below is byte-for-byte from tools.py:
if approval.proposal_digest not in (None, proposal.digest):
raise AuthorizationError("approval proposal mismatch")
if approval.policy_version not in (None, proposal.policy_version):
raise AuthorizationError("approval policy mismatch")
Only after that assertion may state.py accept the effect. Then inspect_access reads the authoritative state again. If the tool returns success but the read remains unchanged, the business outcome fails.
Exhaust budgets safely, then recover
Agent loops can repeat plausible actions forever, inflate cost, or amplify a transient failure. Set maximum steps, elapsed time, tool output bytes, and optional provider cost before execution. Offline CI uses a deterministic model double; provider-backed experiments abort before a call that could exceed the remaining cap.
Add ten cases to the frozen dataset: malformed arguments, duplicated proposal, unavailable tool, ambiguous Sarah, missing approval, expired approval, stale state, repeated observation, max-step exhaustion, and false success without verification. For every denial, run an unaffected positive control using a fresh approved Acme request.
Recovery is bounded: malformed arguments may be repaired once from the validation error; stale state requires a new inspection, proposal, and approval; budget exhaustion stops with no mutation and a resumable evidence receipt. Cleanup deletes only the named disposable run and restores its fixture, then repeats the normal control.
The scored checkpoint supplies one observation and all schemas; the only unknown is the next legal tool or stop reason. Feedback contrasts “tool sounds relevant” with “tool is legal in this state.” Decline prompts that simultaneously change tool design and model behavior because two unknowns make failure diagnosis ambiguous.
Spend 1–1.5 hours reading Anthropic’s tool-writing guidance and the MCP introduction, 4–5 hours implementing and testing, and 1 hour writing the result table. End with R2-agent-v1, D30-cases, schemas, budget receipts, and post-state proof. Week 3 consumes those artifacts to make tenant isolation impossible by construction.