07

Secure Delivery, Promotion, Canary, and Rollback

Turn Workboard's tested source into one identifiable artifact, admit it through evidence-based policy, and move it safely to production.

Separate environments and identify the deployable artifact

A deployment environment is an isolated place where software runs, such as development, staging, or production. Separation matters because a coding agent may experiment freely in development while production keeps different identities, data, network routes, approvals, and secrets; passing one boundary must never imply permission to cross the next.

For Workboard, the unit that crosses boundaries is an immutable container image. Immutable means its bytes do not change after publication. Identify it by a cryptographic digest such as sha256:..., which is a fingerprint of those exact bytes, rather than a movable tag such as latest. A tag may be convenient for people, but the release decision and deployment manifest must pin the digest.

Create a release manifest that connects the artifact to its evidence:

releaseId: workboard-2026-07-26.3
artifactDigest: sha256:6e6f...93ab
sourceRevision: 8bc1f5d
builderIdentity: ci://workboard/release-builder
sbomUri: evidence://workboard/6e6f/sbom.spdx.json
provenanceUri: evidence://workboard/6e6f/provenance.intoto.jsonl
gateResultsUri: evidence://workboard/6e6f/gates.json

The accompanying ReleaseDecision has the course contract { artifactDigest, gateResults, provenance, exceptions, approvers, decision }. Each environment verifies the manifest and digest before starting the image. This gives operators a precise answer to “what is running?” and prevents a registry tag change from silently substituting different bytes.

Environment isolation reduces blast radius; digest identity prevents ambiguity. Neither says the code is correct. They create evidence about where execution happened and which bytes moved, leaving product correctness to independent validation gates.

Keep secrets out of generated code and build output

A secret is a credential that grants authority, such as an API token, signing key, database password, or encryption key. Generated code, prompts, test fixtures, logs, container layers, source history, and build provenance are all poor places for secrets because they may be copied widely and retained long after a release.

Give the coding agent only short-lived credentials for the minimum development resources it needs. The release builder should obtain an ephemeral workload identity at run time; it should not receive a long-lived production credential. Production injects its own secrets when the verified image starts, from an approved secrets manager, so the same artifact can move between environments without embedding environment-specific values.

Workboard's rules are concrete:

LocationAllowedForbidden
Source and agent contextSecret references and test placeholdersLive tokens, passwords, private keys
Build environmentShort-lived package-read credentialProduction database or identity-provider credentials
Image and SBOMPublic configuration and dependency metadataSecret values in layers, labels, or source maps
RuntimeNarrow, environment-scoped secret mounted in memoryShared cross-environment credential

Run secret scanning before commit, during the build, and against the finished image. Also prevent accidental disclosure: redact command output before it enters the agent transcript, do not pass secrets as command-line arguments, and configure crash reports to exclude them. If a secret is found, stop the release, revoke and rotate it, then rebuild from a clean revision. Removing the visible string is insufficient because copies may remain in history, caches, or logs.

Verify dependencies, SBOMs, signatures, provenance, and correctness

Software supply-chain assurance asks what entered a build and how the output was produced. Several controls answer different questions; combining them into one “secure” badge hides gaps, so Workboard records each result separately.

Dependency policy and resolution. Allow packages only from approved registries or a controlled mirror. Commit the package-manager lockfile, require a frozen or immutable install mode, verify registry integrity metadata where the ecosystem supports it, and reject an install that would rewrite the lock. Disable dependency install scripts by default because they execute publisher-controlled code during installation; grant a narrowly reviewed exception only when a named package genuinely requires one. Pin build tools and base images by version and digest, and make a clean environment prove that resolution is reproducible.

Inventory. A software bill of materials (SBOM) is a machine-readable inventory of components and relationships in the artifact. Generate it from the resolved build, including transitive dependencies, versions, package identifiers, and hashes where available. An SBOM helps answer exposure questions, but inventory alone does not say whether a component is vulnerable, malicious, legally acceptable, reachable, or correctly configured.

Vulnerability and license review. Scan the resolved dependencies and base image against current vulnerability data, then apply an explicit rule that considers severity, exploitability, exposure in Workboard, available fixes, and compensating controls. Review licenses against the enterprise's allow, review, and deny lists. A numeric severity or repository score is decision input, not an automatic truth; uncertain or disputed findings need a recorded owner and rationale.

Signer and integrity evidence. Verify the artifact signature against an allowed signer identity, expected repository or workflow, and any required transparency record. A valid signature supports two claims: the artifact bytes have not changed since signing, and the approved signer produced a signature over those bytes. It does not establish that the bytes are safe, bug-free, or fit for Workboard.

Provenance. Provenance is an attestation describing how an artifact was built. Verify the attestation's authenticity, then inspect its predicate rather than accepting a signed envelope alone: its subject digest must equal the proposed artifact digest, its builder identity must be approved, its source repository and exact revision must match the release, and its build type and external parameters must match the expected release workflow. Reject a well-signed attestation whose predicate describes different bytes or an unexpected build.

SLSA v1.2 is an Approved specification with separate Build and Source tracks. The Build track provides levels for provenance authenticity and build isolation; the Source track provides levels for the trustworthiness and completeness of how a source revision was created, from version control through stronger history, continuous controls, and two-party review. Claim only the track and level actually assessed. SLSA v1.2 Build Level 3 isolation is not the same as a hermetic build, and the specification explicitly does not make “no network access” a Build Level 3 requirement. Workboard pursues hermeticity—declared inputs, controlled network access, and reproducible resolution—as an additional design goal.

Correctness gates. Run source-level acceptance, type, and unit checks on exact revision R, then run integration, browser, tenant-denial, accessibility, performance, and security checks against a deployment of digest D built from R. Require the running service to report D and bind every result to both the revision and digest; do not reuse a source-only pass as evidence for different deployed bytes. Provenance can show that the approved builder produced D from R, while the independent gates provide evidence that the resulting Workboard artifact behaves as required. Even together they bound residual risk rather than prove perfection.

Turn enterprise policy into fail-closed gates

A policy gate is an automated decision that admits or rejects a release from declared evidence. “Fail closed” means missing, stale, unreadable, or contradictory evidence produces a denial, which prevents an unavailable scanner or malformed attestation from becoming an accidental approval.

Workboard's production gate evaluates the digest-keyed ReleaseDecision, not a branch name or a screenshot. It requires successful correctness gates, approved dependency and license results, an SBOM, a valid artifact signature, verified provenance predicate, secret-scan evidence, required approvers, and no expired exception. Every result includes its tool or policy version and observation time so an old pass cannot masquerade as current evidence.

An exception is a time-bounded, accountable decision to accept one known policy deviation—not a switch that disables the gate. Record the affected digest and rule, risk explanation, compensating control, owner, approvers independent from the requester, expiry, and remediation ticket. Scope it to the smallest artifact and environment, surface it in the release decision, and automatically fail after expiry. Emergency approval may change who can decide or how quickly, but it must preserve the evidence trail.

Build once and promote the same artifact

Promotion is the act of allowing an already-built artifact into a more sensitive environment. Building once matters because rebuilding for staging or production creates new bytes whose test, signature, and provenance evidence may no longer apply.

The Workboard release service copies or grants access to digest D across registries without invoking the compiler or package manager again. Development, staging, and production each resolve D, verify its digest and evidence, and attach environment-specific configuration and runtime secrets. The production deployment then records both D and the configuration version, because identical code with different policy or configuration can behave differently.

Before promotion, compare the proposed digest with the digest that passed staging. Reject mutable tags, locally built images, manual file copies, and any “equivalent” rebuild. If a platform must transform an artifact, such as signing a platform-specific package, treat the result as a new artifact with a new digest and a fresh evidence chain. “Build once” is therefore an identity rule, not a slogan.

Use canaries and health signals to limit blast radius

A canary release sends a small, controlled share of production traffic to the proposed version before wider rollout. It cannot make a bad release harmless, but it limits initial exposure and creates a decision point based on real Workboard behavior.

Start with internal or low-risk tenants where policy permits, then increase traffic in predefined stages such as 1%, 5%, 25%, and 100%. Compare the proposed version and baseline over a minimum observation window. Use service-level signals—request success, latency, saturation, login completion, authorization denials, cross-tenant denial probes, todo-write success, and queue backlog—rather than merely checking that containers are alive. Guard against low sample sizes and segment by tenant and endpoint so an aggregate can hide a severe minority failure.

Write thresholds before deployment: for example, halt on any tenant-isolation probe failure, a sustained error-rate increase beyond the allowed budget, or a material latency regression. Automation should stop expansion and route traffic back to the known version. A human may investigate or approve a revised plan, but must not reinterpret an explicit security breach as healthy.

Design rollback before release

Rollback restores a previously accepted artifact and configuration after a release fails. It is credible only when designed and rehearsed before production, because application bytes can roll back quickly while data shape, messages, caches, and external side effects may not.

Keep the previous Workboard digest deployable and its evidence accessible. Make database changes backward-compatible across the whole canary and rollback window: add nullable columns or new tables first, deploy code that can read old and new shapes, backfill separately, and remove old fields only after the previous application version can no longer return. Use version-tolerant events and APIs, and ensure background workers from both versions can coexist. A destructive migration tied to the new image turns an application rollback into a data-recovery exercise.

The rollback checklist covers more than image selection:

ConcernPre-release proof
ApplicationPrevious digest starts with current runtime configuration
DatabaseOld and new versions read and write the transitional schema
Queues and jobsMessages are versioned; duplicate work is idempotent
Configuration and policyCompatible prior version is retained and auditable
User-visible writesIrreversible side effects have compensation or manual recovery
DecisionTrigger, authority, command, validation probe, and communication owner are named

After routing back, verify recovery with user journeys and tenant-denial probes, not just infrastructure health. Preserve the failed digest, telemetry, decision record, and timestamps for investigation. Rollback reduces time exposed; it does not erase data already disclosed or actions already completed.

Further reading

These primary sources define the supply-chain and secure-development claims used here. Versions and status are explicit because a living project page is not itself a frozen standard.

  • SLSA specification v1.2 — Version 1.2, status Approved; use its distinct Build and Source tracks, and its artifact-verification guidance. Accessed 2026-07-26.
  • NIST SP 800-218, Secure Software Development Framework — SSDF Version 1.1, final, published February 2022; practices for reducing and responding to software vulnerabilities. Accessed 2026-07-26.
  • NIST SP 800-218A — final SSDF Community Profile, published July 2024; AI-specific additions used alongside SP 800-218. Accessed 2026-07-26.
  • OWASP Software Component Verification Standard — SCVS v1.0, released June 2020; vendor-neutral component and supply-chain control baseline. Accessed 2026-07-26.
  • OpenSSF Scorecard — living official project guidance for automated open-source project risk signals; a score informs review rather than proving a dependency safe. Accessed 2026-07-26.

Key takeaways

This delivery system preserves the connection between one Workboard artifact and the evidence used to admit it. The controls establish bounded, reviewable confidence; none makes a zero-defect promise.

  • Separate environments and identify immutable artifacts by digest, not mutable tag.
  • Keep secrets out of source, agent context, build output, images, provenance, and logs; inject narrow secrets only at runtime.
  • Treat dependency controls, SBOM inventory, vulnerability and license review, signatures, provenance predicates, and correctness gates as different evidence.
  • State SLSA v1.2 Build and Source claims precisely; hermeticity is an additional Workboard goal, not an automatic Build Level 3 requirement.
  • Fail closed when evidence is missing or stale, and constrain exceptions by digest, owner, compensating control, and expiry.
  • Promote the identical digest, use measured canary stages, and preserve schema compatibility for rollback.

Checklist

This checklist turns the release design into evidence an operator can inspect before approving Workboard. Complete it for the proposed digest, not merely for a source branch.

  • [ ] The release manifest pins the artifact digest, source revision, builder identity, evidence locations, and configuration version.
  • [ ] Secret scans cover source, agent output, build logs, and the finished image; production secrets enter only at runtime.
  • [ ] Frozen dependency installation uses the committed lockfile, approved registries, constrained install scripts, and pinned tools and base images.
  • [ ] The SBOM covers resolved transitive dependencies; vulnerability and license decisions have explicit policy results.
  • [ ] The artifact signer is allowed, and the verified provenance predicate names this digest, builder, source revision, build type, and parameters.
  • [ ] Independent correctness and tenant-denial gates passed against the proposed artifact.
  • [ ] Policy fails closed; every exception is narrowly scoped, independently approved, visible, owned, and unexpired.
  • [ ] Development, staging, and production promote the same digest without rebuilding.
  • [ ] Canary stages, observation windows, automatic halt conditions, and baseline comparisons are declared before rollout.
  • [ ] The previous digest remains deployable, and application, schema, message, configuration, and side-effect rollback paths were rehearsed.