Proving the Generated Application Works

Validation asks whether an artifact satisfies specified requirements in a stated environment. No single check answers every question, so Workboard uses layers whose different failure-detection strengths combine into a more credible assurance case.

A formatter proves consistent syntax layout but says nothing about tenant isolation. A type checker rejects incompatible values but may accept wrong business logic. Unit tests cover selected transitions, while browser tests cover integrated application programming interface (API) paths; each can miss behaviour the other sees. Production probes observe real deployment conditions but arrive too late to be the first defense.

The layers should be independent enough that one mistake is unlikely to fool all of them. The code-generating system may run the gates, but it does not define their expected values during the same run, delete inconvenient assertions, or decide that its own prose is equivalent to a passing oracle. Gate definitions come from the executable specification and reviewed policy.

This is an evidence stack, not a guarantee pyramid. Higher layers cover broader integration and usually cost more to run; lower layers give fast, precise feedback. A release decision considers the whole stack, known gaps, failed or flaky results, and residual risk.

Static gates inspect source and build outputs without driving Workboard through a user workflow. They run first because they are fast, deterministic, and good at rejecting malformed changes before expensive environments are started.

• Formatting produces a canonical code layout, reducing review noise and preventing style drift.
• Linting applies rules for suspicious or forbidden source patterns, such as unhandled promises or unsafe imports.
• Type checking verifies that statically declared values and interfaces are used consistently.
• Build validation proves that the pinned toolchain can resolve dependencies, compile, bundle, and generate required assets in a clean environment.

Pin the package manager, lockfile, runtime, environment, and build command. Regularly begin from a clean checkout and empty dependency cache because a warm machine can hide undeclared inputs. Give build warnings an explicit policy so important changes cannot disappear into noise.

A fail-fast sequence can remain simple:

gates:
  - id: format.check
    commandRef: format-check-v3
  - id: lint.source
    commandRef: lint-v7
  - id: types.application
    commandRef: typecheck-v4
  - id: build.clean
    commandRef: production-build-v5
policy:
  stopOnFailure: true
  requireCleanWorkspace: true
  attachLogs: always

commandRef is a reviewed mapping in the trusted runner, not command text supplied by the agent. Every result records tool versions, exit code, commit SHA, environment image digest, duration, and log artifact URI. Passing output from a different commit is stale evidence.

Static gates cannot prove runtime authorization, correct database queries, browser usability, or performance. Keep their claims narrow: they establish that the source meets defined static rules and produces a build under the recorded conditions.

A unit test checks a small piece of code in isolation, while a property-based test generates many inputs and checks a rule that should always hold. Together they make Workboard’s domain invariants precise without requiring a browser or network for every case.

The todo state function should cover at least these examples: completing an open todo records one completion time; completing an already completed todo with the same operation key changes nothing; reopening clears the completion time; a viewer cannot request the transition; and an inactive membership cannot act. Use a fixed clock and deterministic identifiers so failures reproduce exactly.

Example tests are necessary but may miss combinations. Property-based tests can generate valid titles, state sequences, operation keys, and roles, then assert invariants such as:

• A todo state is always open or completed after any valid sequence.
• completedAt exists exactly when the state is completed.
• Repeating one operation key does not produce a second logical transition.
• A denied actor leaves the todo byte-for-byte unchanged.
• A todo’s tenantId never changes during a state transition.

Record the random seed and smallest failing input. Shrinking reduces a generated failure to a minimal example. Run stable seeds on every change and a larger rotating set on a schedule, then keep valuable failures as regression examples.

Avoid replacing real logic with permissive test doubles. A fake authorization service that always returns allow cannot prove denial behaviour, and a repository stub that ignores the tenant key can make unsafe code look correct. Unit boundaries should reveal domain mistakes quickly, while integration tests independently exercise the real seams where isolation can fail.

An integration test exercises multiple real components together, such as the API handler, authorization policy, database driver, schema, and transaction. This layer matters because generated code can be correct inside each function yet unsafe at the boundary between them.

Start an isolated instance of the production database engine, apply the same migrations used for release, and seed two synthetic tenants. Tenant Alpha and Tenant Beta deliberately receive boards and todos with the same local identifiers. This arrangement catches a query that looks up todo-7 alone instead of the compound key (tenantId, todoId).

For WB-TODO-001, the integration test authenticates an active Alpha editor, sends the create request, verifies its Hypertext Transfer Protocol (HTTP) status and schema, and reads the committed row through an independent tenant-scoped query. It also checks the board relationship, initial open state, and operation key. A repeated request with the same idempotency key returns the original logical result rather than a duplicate row.

Then exercise denial and failure:

Boundary case	Expected observation
Beta principal requests Alpha todo with the same local id	Denial response with no Alpha fields and no mutation
Alpha viewer attempts completion	Denied and database state unchanged
Missing or inactive membership	Fail closed before resource mutation
Todo references a board in another tenant	Database constraint or transaction rejects it
Database write fails after validation	Stable error contract and no partial record
Two completion requests race	One valid final state and no duplicate logical event

Inspect more than HTTP status. Compare response bodies for leakage, count rows before and after denial, verify transaction rollback, and inspect audit output where the specification requires it. Reset the database between cases or use isolated transactions so tests remain order-independent.

An end-to-end test, often shortened to E2E, drives the application through its external interface with real integrated components. For Workboard, a browser test proves that routing, rendering, controls, API calls, persistence, and visible state cooperate for an actual user flow.

The primary flow signs in as an active Alpha editor, opens the planning board, creates “Prepare launch notes,” marks it complete, filters to completed todos, and verifies the item remains visible with its completed status. Select controls by accessible role and name rather than fragile Cascading Style Sheets (CSS) classes. Wait for user-visible state or a named network response, not arbitrary sleeps.

Denial paths deserve their own browser and direct-API coverage. Sign in as an Alpha viewer and verify the completion control is absent or disabled, then call the API directly and require denial with no data change. Sign in as Beta, navigate to a guessed Alpha URL, and require a neutral response that does not reveal whether the board exists. Test list, search, export, attachment, and subscription paths as they enter scope; hiding a button never constitutes authorization.

Capture a browser trace, named-state screenshots, console errors, failed requests, and the browser version. A passing Document Object Model (DOM) assertion alongside an uncaught exception is not clean evidence; define which console and network conditions fail the gate.

Visual regression testing compares a rendered screenshot with a reviewed baseline, while accessibility testing checks whether people using keyboards and assistive technology can perceive and operate the interface. These are separate claims: a pixel-perfect page can be unusable, and an accessible semantic tree can still have a severe layout defect.

Create baselines for the empty board, populated list, completed filter, validation error, narrow viewport, and high zoom. Pin fonts, viewport, colour mode, browser, animations, and fixtures. A reviewed tolerance may absorb rendering noise, but must not hide a missing button or tenant name. Inspect and explain every update.

Target Web Content Accessibility Guidelines, or WCAG, 2.2 Level AA for Workboard. Automated rules should check semantic names, form labels, colour contrast where determinable, invalid markup, and common ARIA misuse. ARIA means Accessible Rich Internet Applications, a set of attributes that can communicate roles, states, and relationships to assistive technology when native HTML alone is insufficient.

Manual and scripted keyboard checks should prove that a user can create, complete, and filter todos without a pointer; focus is visible and follows a sensible order; validation errors identify the affected field; and completion updates are announced without unexpectedly moving focus. Test at 200 percent zoom and a narrow viewport for reflow. Inspect the accessibility tree with a representative screen reader for the critical flow.

Automated accessibility scans do not establish WCAG conformance by themselves. Record which success criteria were evaluated automatically, manually, or not yet covered. Treat uncovered criteria as evidence gaps, not passes.

Performance testing measures whether Workboard stays within its declared latency and rendering budgets under a reproducible workload. Compatibility testing repeats critical behaviour across the supported browsers, viewports, and other environments instead of assuming one successful run generalizes everywhere.

For API load tests, record dataset size, request mix, concurrent users, geographic location, database tier, cache state, warm-up, duration, and software digest. Report distributions such as median, 95th, and 99th percentiles together with error rate and throughput. Averages alone can hide a small but important group of very slow requests.

For the browser, measure the agreed user-centred metrics on the board route with fixed network and CPU profiles. Keep asset-size budgets for initial JavaScript, styles, images, and fonts, because generated UI changes can add a large dependency while leaving functional tests green. Compare against both the absolute budget and a reviewed regression threshold.

Run the critical create, complete, filter, validation, and denial flows on the organization’s supported browser matrix. Use current stable browser versions recorded in the artifact, and include mobile or tablet viewports only when they are part of the product contract. Compatibility is a declared support policy, not a promise that every historical browser behaves identically.

Performance failures should fail closed when the budget is a release criterion. If measurement infrastructure is unavailable, record not-run; never convert absence into a pass. A time-bound exception needs an owner, reason, compensating control, expiry, and follow-up evidence requirement.

A validation matrix maps every acceptance case to the independent gates needed to support its claims. It prevents teams from accumulating many tests while leaving a critical requirement, denial path, or non-functional budget without an oracle.

An initial slice of the Workboard matrix is:

Criterion	Static	Unit/property	Integration	Browser	Visual/accessibility	Performance
WB-TODO-001 create todo	Types/build	Title and initial-state rules	API, persistence, tenant key	Create and open filter	Form, error, empty/populated states	Create latency
WB-TODO-002 complete once	Types	Transition and idempotency	Retry and race	Complete and filter	Status announcement	Complete latency
WB-AUTH-001 viewer denied	Lint policy	Role decision	No mutation	Hidden control plus direct denial	Disabled-state semantics	Not applicable
WB-TENANT-001 isolation	Query-rule scan	Tenant invariant	Alpha/Beta same-id test	Guessed route denial	Neutral error state	Denial timing review
Accessibility budget	Build	Component semantics	Not applicable	Keyboard flow	WCAG 2.2 AA checklist	Reflow responsiveness

For every matrix cell that applies, create an EvidenceRecord containing criterionId, gate, result, artifactUri, commitSha, environment, and observedAt. The continuous-integration system, or CI, must fail closed when a required record is failed, stale, missing, produced for another commit, or created by an unapproved environment.

Flaky tests, which alternate between pass and fail without a relevant code change, are not trustworthy passes. Quarantine may protect the main signal temporarily, but it must create a visible gap with an owner and expiry. Keep raw artifacts for both passing and failing runs so a summary cannot hide console errors, skipped tests, changed baselines, or partial execution.

These primary standards anchor Workboard’s security and accessibility criteria and the secure-development process around its tests. Versions and status were verified on 2026-07-26, and the guidance is paraphrased rather than treated as a substitute for application-specific threat analysis.

• OWASP Application Security Verification Standard 5.0.0 — latest stable ASVS release, published 2025-05-30. It supplies versioned, testable web-application security requirements; include the version in requirement references because identifiers can change.
• W3C Web Content Accessibility Guidelines 2.2 — W3C Recommendation dated 2024-12-12. Level AA is a defined conformance target, but evaluation still requires applicable success criteria, accessibility-supported techniques, and human judgement where automation is insufficient.
• NIST SP 800-218, Secure Software Development Framework 1.1 — final publication, February 2022. It frames verification as integrated secure-development practice and emphasizes preserving evidence, addressing vulnerabilities, and improving root causes.

This section summarizes how layered validation turns plausible generated code into a reviewable release package. Each gate makes a limited claim, and the matrix shows how those claims combine without pretending to eliminate residual risk.

• Run fast format, lint, type, and clean-build gates before expensive environments, and bind results to the exact commit and toolchain.
• Use unit examples and reproducible property tests for Workboard’s state, tenant, role, and idempotency invariants.
• Exercise the real API, policy, migration, database, and transaction boundaries with Alpha/Beta same-id fixtures.
• Drive both successful user journeys and direct denial attempts end to end; UI hiding is not authorization.
• Keep visual regression and accessibility evidence separate, reviewed, and reproducible.
• Target WCAG 2.2 Level AA with automated checks plus keyboard, zoom, focus, status, and assistive-technology evaluation.
• Measure performance distributions under a named workload and test only a declared compatibility matrix.
• Map every acceptance case to applicable gates and fail closed on failed, missing, stale, skipped, or unapproved evidence.
• Preserve raw artifacts and make flaky or uncovered checks visible gaps.

Use this checklist to review the validation system for the generated Workboard slice. A checked item should have a named gate, deterministic oracle, current artifact, and clear statement of what it does not prove.

• [ ] Formatting, linting, type checking, and production build run in a pinned clean environment.
• [ ] Gate records include tool versions, commit SHA, environment digest, exit code, duration, and artifact URI.
• [ ] Unit tests cover todo transitions, role denial, tenant stability, and idempotency.
• [ ] Property-test failures record their seed and smallest failing input.
• [ ] Integration tests use the production database engine, real migrations, and isolated Alpha/Beta fixtures.
• [ ] Cross-tenant, viewer, invalid-input, dependency-failure, retry, and race cases prove no unintended write.
• [ ] Browser tests cover create, complete, filter, validation, and direct denial without arbitrary sleeps.
• [ ] Visual baselines are reviewed at named states and pinned rendering conditions.
• [ ] Accessibility evidence combines automated rules with keyboard, focus, zoom, status, and assistive-technology checks.
• [ ] Performance evidence records workload, environment, percentiles, errors, throughput, and asset budgets.
• [ ] The supported browser and viewport matrix is explicit and versioned.
• [ ] Every applicable validation-matrix cell has current evidence or a visible, owned, expiring gap.