From Prompt to Executable Specification

A generated application is working only relative to an explicit set of expectations and observations. A page that renders once is evidence of one successful observation; it is not proof that the code is correct for every user, input, tenant, browser, failure, or future change.

No team can guarantee zero defects in a non-trivial application. Tests examine selected behaviours, reviews sample code and design, and production monitoring observes conditions that actually occur. Unknown inputs, changing dependencies, misunderstood requirements, and untested interactions leave residual risk, meaning risk that remains after controls have been applied. The honest objective is therefore evidence-backed assurance: enough reproducible evidence to justify a decision while stating what the evidence does not cover.

For this course, a coding agent is a software system that can inspect files, propose edits, run tools, and react to their results. It should not decide for itself that its output is ready. Readiness comes from an executable contract, independent checks, and a named human or policy authority that accepts the residual risk.

A useful working claim is narrow and falsifiable: “At commit 4f2a, in the clean test environment, an active member of Tenant Alpha could create and complete a todo, while a member of Tenant Beta was denied access to it.” It names the version, environment, behaviour, and denial boundary. A vague claim such as “the dashboard works” cannot tell a checker what to observe or a release owner what risk is being accepted.

Workboard is the single application we will develop: a multi-tenant todo dashboard where an organization is a tenant, its members use boards, and each board contains todos. Multi-tenant means one running service holds data for multiple organizations while keeping each organization’s identity, authorization, and data isolated.

The initial product brief is deliberately small:

• An active member can list boards within the member’s current tenant.
• An editor can create a todo with a title and due date on an accessible board.
• An editor can mark an open todo complete.
• A viewer can read todos but cannot create, edit, or complete them.
• The dashboard can filter todos by open or completed state.
• Requests must never expose whether another tenant’s board or todo exists.

Three actors make the policy concrete: owner, editor, and viewer. The owner manages membership and all board content; the editor manages todos; the viewer reads permitted boards. Anonymous sharing, enterprise sign-on, and automated provisioning are outside this first slice. Declaring those exclusions prevents the agent from inventing half-designed features and lets later controls be added against a stable core.

The application boundary is equally explicit. The browser renders the dashboard; an application programming interface, or API, receives operations; a policy check decides whether the principal may act; and a database persists tenant-scoped records. Email delivery, billing, mobile clients, and offline editing are not part of this product brief.

This scope is a control, not merely project management. Every unrequested feature adds code, dependencies, permissions, and states that need evidence. A narrow vertical slice gives the coding agent a tractable target and gives reviewers a clear basis for rejecting attractive but unsupported additions.

An executable specification describes expected behaviour in a form that a deterministic tool can check. Deterministic means the same input and environment produce the same verdict, so success does not depend on whether the generator feels satisfied with its own output.

Each important expectation becomes an AcceptanceCase with this minimal shape:

type AcceptanceCase = {
  id: string;
  actor: string;
  given: string[];
  when: string;
  then: string[];
  oracle: string;
  evidenceRequired: string[];
};

An oracle is the rule or mechanism that determines pass or fail. It may be an exact return value, a database query, a Hypertext Transfer Protocol (HTTP) status and response schema, an accessibility assertion, or a measured duration below a threshold. “The agent reviews the result” is not a strong oracle because the same probabilistic system that produced the code may repeat its own mistake.

Here is a specification for the first useful Workboard action:

id: WB-TODO-001
actor: active editor in Tenant Alpha
given:
  - board alpha-planning belongs to Tenant Alpha
  - the editor has access to alpha-planning
when: the editor creates a todo titled Prepare launch notes
then:
  - the API returns status 201
  - the response contains a new todo in the open state
  - the persisted todo carries Tenant Alpha and alpha-planning identifiers
  - the todo appears in the open dashboard filter
oracle: API assertions plus a tenant-scoped database query and browser assertion
evidenceRequired:
  - integration test report
  - browser trace
  - database isolation assertion

Notice what this does not prescribe: framework, component names, table names, or Cascading Style Sheets (CSS) architecture. A good specification fixes observable behaviour and safety boundaries while leaving implementation choices open unless a constraint truly matters. This separation lets the agent choose an implementation without silently redefining success.

A domain invariant is a rule that must remain true across every valid state change, regardless of which screen, API endpoint, job, or tool caused the change. Invariants give generated code a stable model and expose contradictions before implementation spreads them across many files.

Workboard’s minimal domain schema is:

Entity	Required fields	Important relationship
Tenant	id, name, state	Owns boards and memberships
Membership	tenantId, principalId, role, state	Connects one principal to one tenant
Board	tenantId, id, name	Belongs to exactly one tenant
Todo	tenantId, boardId, id, title, state, dueAt, completedAt	Belongs to one board in the same tenant

The first invariants are stronger than UI rules:

1. Every board belongs to exactly one tenant.
2. Every todo’s tenantId equals its board’s tenantId.
3. Resource identity is tenant-scoped, so the safe lookup key is the pair (tenantId, resourceId), not resourceId alone.
4. Only an active membership can authorize an action.
5. A todo state is either open or completed.
6. Completing a todo sets completedAt; reopening it clears completedAt.
7. Repeating the same completion command does not create a second event or corrupt the record.

The seventh rule is idempotency: repeating an operation has the same externally visible effect as performing it once. It matters because browsers, networks, tests, and agents retry. If the implementation increments a completion counter or emits duplicate notifications on retry, the happy path may pass while the system is still wrong.

Keep invariants in the product specification and encode them again at independent boundaries: type constraints, domain functions, database constraints, and tests. Repetition here is intentional defense in depth. A UI restriction alone is not an invariant because a direct API call can bypass it.

Acceptance criteria turn each requirement and invariant into a concrete observation. A strong set includes successful use, denied use, boundary values, retry behaviour, and failure recovery, because generated applications often look convincing on the single path they were prompted to demonstrate.

The initial Workboard acceptance matrix is:

Case	Given and when	Expected result	Primary oracle
WB-TODO-001	Active Alpha editor creates a valid todo	Created once in Alpha and shown as open	API, database, browser
WB-TODO-002	Alpha editor completes an open todo twice with one idempotency key	One completed record and one logical transition	Domain and integration assertions
WB-AUTH-001	Alpha viewer attempts to complete a todo	Denied with no data change	HTTP status plus before/after database state
WB-TENANT-001	Beta member requests an Alpha todo identifier	Denied without revealing existence	Response comparison and query log
WB-VALID-001	Editor submits a blank or over-limit title	Stable validation error and no write	Schema assertion plus row count
WB-FILTER-001	Member selects the completed filter	Only accessible completed todos appear	Browser DOM assertion

Each row should have one authoritative oracle and may have supporting evidence at other layers. A unit test is well suited to a state transition; it cannot prove that the browser wires the correct tenant into a real request. A browser test shows the user path; it may miss a corrupt database constraint. The specification names both when both claims matter.

Negative criteria deserve equal precision. “User cannot access another tenant” should specify which actor, target, API, status, response body, side effect, and audit observation are expected. The oracle must also check that denial does not leak the target’s title, owner, timing, or existence through different error messages.

Functional behaviour says what Workboard does; non-functional requirements set measurable limits on qualities such as speed, accessibility, security, reliability, and compatibility. Calling them budgets makes failure visible: a build spends only up to a defined amount of latency, errors, or resource use.

Use provisional targets that can be tightened with real measurements. The Web Content Accessibility Guidelines (WCAG) define testable accessibility criteria:

Quality	Initial Workboard budget	Evidence
API latency	95th percentile under 300 ms for list/create/complete in the reference load test	Timestamped load-test report
Browser performance	Largest Contentful Paint under 2.5 s at the chosen test profile	Browser performance trace
Accessibility	WCAG 2.2 Level AA target; keyboard completion flow; named controls; announced status changes	Automated scan plus manual checklist
Security	No cross-tenant access in the defined abuse cases; no high-severity finding accepted without an exception	Security test and exception record
Reliability	Retried create/complete operations remain idempotent; defined error response on dependency failure	Fault-injection test
Compatibility	Current stable releases of the organization’s supported browsers	Versioned browser test report

A percentile describes a distribution: the 95th percentile latency is the value at or below which 95 percent of measured requests complete. It is more useful than an average that can hide a slow tail. The reference load profile must name dataset size, concurrency, geography, hardware, and warm-up rules; otherwise the number cannot be reproduced.

Automated accessibility tools find only some problems. Keyboard order, meaningful labels, focus after a dialog closes, and whether a completion status is understandable need human checks. Likewise, a security scanner does not establish tenant isolation. The budget identifies the claim; the evidence plan combines tools appropriate to that claim.

An evidence ledger is a durable index linking each criterion to the gate that checked it and the artifact that records the observation. It prevents a green badge from becoming an unsupported summary and makes evidence traceable to an exact commit and environment.

Use this minimal EvidenceRecord; its artifact URI is a Uniform Resource Identifier that locates stored evidence:

type EvidenceRecord = {
  criterionId: string;
  gate: string;
  result: 'pass' | 'fail' | 'not-run';
  artifactUri: string;
  commitSha: string;
  environment: string;
  observedAt: string;
};

For WB-TODO-001, separate records might point to the integration report, a browser trace, and a tenant-scoped database assertion. Store large logs and screenshots in an artifact repository; the ledger holds stable addresses and metadata. A missing artifact, unknown commit, or mismatched environment is a gap even if the result field says pass.

The ledger should also record failures. Failed evidence explains why a change was revised, helps detect recurring weaknesses, and stops teams from presenting only favourable runs. Evidence expires when its assumptions change: a dependency upgrade, policy change, new browser version, schema migration, or altered acceptance case can require a fresh observation.

More evidence narrows uncertainty but never removes it entirely. The ledger supports a release decision; it does not transform sampled observations into a universal guarantee. That distinction is the foundation for every later assurance, security, and operational control applied to Workboard.

These official primary sources provide the risk and secure-development foundations behind the specification and evidence approach. Versions and status are stated so a reader can detect future drift; all links were accessed on 2026-07-26.

• NIST AI Risk Management Framework 1.0 — released 2023-01-26 for voluntary use; NIST notes that revision work is under way. Its Govern, Map, Measure, and Manage functions support explicit context, measurement, ownership, and residual-risk decisions.
• NIST AI 600-1, Generative Artificial Intelligence Profile — final NIST publication, published 2024-07-26 and updated 2026-04-08. It applies AI RMF risk-management practices to generative-AI-specific risks across the lifecycle.
• NIST SP 800-218, Secure Software Development Framework 1.1 — final publication, February 2022. It provides outcome-focused secure-development practices that can be integrated into an existing software lifecycle rather than treated as a final scan.
• NIST SP 800-218A — final SSDF community profile, July 2024. It extends secure-development considerations to generative AI and dual-use foundation models.

This section condenses the specification discipline into decisions you can apply before allowing a coding agent to modify a repository. The central idea is to replace a broad promise with narrow, reproducible claims and explicit residual risk.

• “Working” is relative to declared requirements, environments, and observations; zero-defect guarantees are impossible for non-trivial software.
• Workboard stays one bounded multi-tenant todo dashboard with explicit actors, data boundaries, and excluded features.
• An executable specification fixes observable behaviour and safety boundaries without prematurely fixing every implementation detail.
• AcceptanceCase records the actor, setup, action, expected results, oracle, and required evidence.
• Domain invariants must hold across UI, API, jobs, retries, and direct data access, especially tenant-scoped identity and todo state transitions.
• Happy paths, denial paths, validation failures, retries, and isolation claims each need an authoritative oracle.
• Performance, accessibility, security, reliability, and compatibility become enforceable only when their budgets and test conditions are measurable.
• EvidenceRecord binds a result to a criterion, gate, artifact, commit, environment, and time; it supports assurance without claiming perfection.

Use this checklist to decide whether Workboard is specified well enough for controlled implementation. Every checked item should point to text, a schema, an acceptance row, or another inspectable artifact rather than an unwritten assumption.

• [ ] The product brief names Workboard’s actors, first vertical slice, system boundary, and excluded features.
• [ ] “Working” is expressed as falsifiable claims with explicit environments and residual risk.
• [ ] Tenant, membership, board, and todo entities have required fields and relationships.
• [ ] Tenant isolation, role permissions, todo transitions, and idempotency are written as invariants.
• [ ] Every important requirement has an AcceptanceCase with a deterministic primary oracle.
• [ ] The matrix covers success, denial, invalid input, retry, filtering, and cross-tenant isolation.
• [ ] Performance and compatibility test profiles are reproducible rather than implied.
• [ ] Accessibility and security targets combine automated checks with suitable human or adversarial checks.
• [ ] The evidence ledger binds results to artifact URIs, commit SHAs, environments, and observation times.
• [ ] The release claim states evidence gaps and residual risk instead of promising defect-free software.