Closed-Loop Evaluation for Enterprise Agents

Build one Enterprise Access Agent across ten cumulative weeks: deterministic baseline, bounded tools, tenant-safe retrieval, causal traces, a 100-case suite, independent graders, repeated-run reliability, adversarial gates, and a closed deployment loop.

1. 01Put the Workflow Before the AgentWeek 1 of 10 · Build the success contract, 20 cases, and deterministic baseline before adding autonomy • Reference implementation: https://github.com/ZGTR/enterprise-access-agent-evals/tree/v1.0.2Published 02 Sept 20264 sections
2. 02Bound the Agent Loop and Its ToolsWeek 2 of 10 · Add decide → act → observe only where Week 1 found open-ended judgment • Public source target: https://github.com/ZGTR/enterprise-access-agent-evals/tree/v1.0.2Published 02 Sept 20264 sections
3. 03Filter Tenant Policy Before RankingWeek 3 of 10 · Make cross-tenant retrieval and effects impossible by construction • Public source target: https://github.com/ZGTR/enterprise-access-agent-evals/tree/v1.0.2Published 02 Sept 20264 sections
4. 04Trace the First Causal FailureWeek 4 of 10 · Trace every causal boundary and diagnose 20 runs from first divergence • Public source target: https://github.com/ZGTR/enterprise-access-agent-evals/tree/v1.0.2Published 02 Sept 20264 sections
5. 05Freeze a Portable Evaluation HarnessWeek 5 of 10 · Expand to 75 cases and run dataset → solver → scorers from one command • Public source target: https://github.com/ZGTR/enterprise-access-agent-evals/tree/v1.0.2Published 02 Sept 20264 sections
6. 06Separate Outcomes, Trajectories, and JudgesWeek 6 of 10 · Implement independent graders and calibrate subjective judgment against 30 blind labels • Public source target: https://github.com/ZGTR/enterprise-access-agent-evals/tree/v1.0.2Published 02 Sept 20264 sections
7. 07Measure Retrieval Before GenerationWeek 7 of 10 · Inject difficult policy documents, reach 85 cases, and locate RAG failure at its boundary • Public source target: https://github.com/ZGTR/enterprise-access-agent-evals/tree/v1.0.2Published 02 Sept 20264 sections
8. 08Distinguish Capability from ReliabilityWeek 8 of 10 · Run every one of 85 cases five times and keep consistency, cost, and latency separate • Public source target: https://github.com/ZGTR/enterprise-access-agent-evals/tree/v1.0.2Published 02 Sept 20264 sections
9. 09Make Safety a Hard Release GateWeek 9 of 10 · Reach 100 cases with 25 adversarial attacks and zero tolerated critical failures • Public source target: https://github.com/ZGTR/enterprise-access-agent-evals/tree/v1.0.2Published 02 Sept 20264 sections
10. 10Close the Evaluation and Deployment LoopWeek 10 of 10 · Compare v1/v2, choose promote/hold/rollback, and convert failures into regressions • Public source target: https://github.com/ZGTR/enterprise-access-agent-evals/tree/v1.0.2Published 02 Sept 20264 sections