Network Trust, Infrastructure, and Delivery

Enterprise problem: A correctly entitled user can still be exposed when an untrusted workload steals a token, dials an arbitrary endpoint, reads a broad secret, or deploys an unreviewed agent version, making application authorization alone insufficient.

Whole-course context: The incoming evidence is a team-scoped application deployment with independent knowledge, connector, and app-role bindings; today turns those logical grants into a constrained production path.

Today’s slice: We derive workload identity, strict mutual TLS, destination policy, secret brokerage, infrastructure ownership, and immutable canary delivery across the provider control plane and hosted runtime without granting the generated app infrastructure authority.

End-of-day evidence: A release receipt and paired connection probes prove the reviewed digest reached a canary, the runtime can call only its approved gateway with two keys, and a stolen user token or wrong workload is denied.

Still unsolved: Run-time revocation propagation, complete decision lineage, adversarial attack suites, rollback exercises, and production operations remain deferred.

Infrastructure can be “green” while the wrong workload or artifact is running, so customer outcomes must include cryptographic identity and release provenance. The use cases cover both the private connection and the path that delivers its policy.

Use case ID	Actor	Customer job	Success outcome	Denial or recovery evidence
D10-UC-01	Enterprise analyst running a generated application	Use an approved private Slack or warehouse connector without exposing credentials to the generated app or agent runtime	The hosted runtime reaches the exact connector adapter through the gateway using workload identity plus delegated authority, and the downstream operation succeeds	The same run token from an untrusted workload and the trusted workload without a valid run token both fail; an approved positive call remains healthy
D10-UC-02	Platform release owner	Promote a reviewed immutable agent and policy bundle through canary to production with reproducible infrastructure ownership	The exact artifact, manifest, policy, schema, infrastructure revision, and approvals appear in one release receipt and the canary probes pass	A changed tool schema or prohibited capability quarantines the release; rollback restores the prior digest without mutating generated-app data

Infrastructure ownership follows security domains. The cloud platform team owns clusters, networks, keys, cloud identity, secret manager, databases, and private endpoints. Identity and security owns enterprise federation, membership, authorization, credential brokerage, token issuance, and the workload trust domain. The AI platform owns agent control plane, runtime, tool gateway, connector registry, protocol adapters, versioning, and supervision. Customer application teams own definitions, deployment requests, capability allowlists, resource constraints, tests, and named approvers.

A firewall rule or successful build is not customer proof because neither connects the initiating actor to the deployed policy. These stories make the two-key connection and immutable delivery chain observable from the product. SPIFFE (Secure Production Identity Framework for Everyone) gives workloads a standard identity format, while mTLS (mutual Transport Layer Security) makes both network peers authenticate with certificates.

Story ID	Use case IDs	User story	Observable acceptance conditions
D10-US-01	D10-UC-01	As a Finance analyst, I want my generated application to reach only its approved private connector through an authenticated runtime, so that token theft or server-side request forgery cannot redirect my authority	The allowed probe records the user/team/deployment token, SPIFFE workload principal, exact destination, mTLS peer, argument digest, downstream result, and trace; wrong workload, missing token, and unapproved destination probes deny independently
D10-US-02	D10-UC-02	As a platform release owner, I want deployment generated from reviewed code and policy to promote by immutable digest, so that I can prove what runs and recover without hidden drift	Schema, workflow, manifest, connector compatibility, policy tests, prompt-injection evaluation, cycle detection, and approvals pass before version creation; canary health and security probes attach to the receipt; rollback selects the prior digest and records a new auditable transition

The “two-key rule” requires both a valid hosted workload identity and a valid delegated run token. A stolen run token presented by an untrusted pod fails workload authentication; a legitimate runtime with no current user/team/deployment grant fails business authorization. Neither key inherits the other.

A private endpoint does not prove that the right workload or release used it, so the flow starts at a visible application or release action and ends with correlated evidence. The flow therefore checks both the delegated run authority and the previously defined SPIFFE/mTLS workload channel.

Flow ID	Use case IDs	Path	Trigger	Numbered steps	Terminal evidence
D10-FLOW-01	D10-UC-01	Happy	Analyst selects “Search approved workspace” in the generated application	1. The generated app validates its tenant session and submits an opaque provider deployment reference.<br>2. The provider gateway resolves current user, organization, active team, deployment, capability, and exact tool arguments.<br>3. The token service issues a short-lived delegated run token for the tool gateway audience.<br>4. The hosted runtime authenticates as spiffe://platform.internal/ns/agents/sa/runtime over strict mTLS.<br>5. The egress proxy permits only tool-gateway:443; the gateway requires both identities and evaluates the intent.<br>6. A single-use capability token binds connector, tool, resource, arguments, decision, and expiry.<br>7. The connector adapter validates it, then the credential broker resolves the external secret and calls the provider.	Evidence records actor, connector resource, search scope, current-membership and approved-deployment preconditions, expected allowed call, observed downstream result, production-canary environment, timestamp, workload principal, peer certificate, argument digest, decision ID, and trace ID
D10-FLOW-02	D10-UC-01	Denied	Security probe replays the run token from an untrusted workload and then probes an unapproved host	1. The gateway rejects the caller because its SPIFFE principal is absent from the workload allowlist.<br>2. No capability token or credential lookup occurs.<br>3. A trusted runtime then attempts arbitrary.example:443; its egress proxy denies the destination.<br>4. The trusted runtime performs the approved read as an unaffected positive control.	Actor and workload, attempted resources, scopes, preconditions, expected independent denies, observed mTLS/egress reasons, positive-control result, environment, timestamp, policy revision, and immutable probe-run ID
D10-FLOW-03	D10-UC-02	Happy	Release owner approves “Promote reviewed version to canary”	1. CI validates agent schema, workflow, capability manifest, prohibited tools, connector compatibility, data classification, policy tests, prompt-injection cases, and agent-invocation cycles.<br>2. Build service emits an immutable artifact and software bill of materials.<br>3. Policy and tool schema digests are attached to the version.<br>4. Team administrator approves deployment bindings and risk policy.<br>5. Infrastructure reconciler applies versioned network, workload, and secret-reference configuration.<br>6. Canary runs positive and negative probes.<br>7. Promotion atomically selects the same digest for production.	Receipt records release actor, artifact resource and production scope, approved-check preconditions, expected digest equality, observed canary and production revisions, environment, timestamp, source commit, artifact digest, policy digest, and immutable release ID
D10-FLOW-04	D10-UC-02	Recovery	Canary detects a changed connector tool schema after review	1. Discovery hashes differ from the reviewed schema digest.<br>2. Connector registry quarantines the tool and blocks promotion.<br>3. Release controller keeps production on its prior immutable digest.<br>4. Owner selects rollback for the canary.<br>5. Reconciler restores the prior deployment and network policy.<br>6. Positive and negative probes prove service and containment.	Evidence records actor, changed schema resource, deployment scope, prior-review precondition, expected quarantine, observed no production promotion and restored canary, environment, timestamp, old/new hashes, rollback transition, and immutable incident ID

The capability token is audience-bound, connector-instance-bound, tool-bound, resource-bound, argument-digest-bound, short-lived, optionally single-use, and linked to the authorization decision. It is not the downstream OAuth refresh token. The runtime never receives secret-manager permission; only the broker or narrowly scoped gateway identity can decrypt or exchange credentials.

When build, runtime, network, and secret concerns share one administrator or store, compromise and deployment drift have the same blast radius. The system splits reviewable desired state from hosted enforcement while letting every downstream layer veto.

Use case ID	Entry point	Responsible services	Authoritative store	Failure evidence
D10-UC-01	Generated app connector action	App session service; provider identity and authorization; token issuer; hosted runtime; workload certificate service; Envoy/Istio ingress and egress; tool gateway; connector adapter; credential broker	Generated-app session store owns app login only; provider policy store owns human/team/tool grant; workload registry and deployment state own runtime identity/routes; vault owns external credentials; downstream provider owns resource ACLs	Invalid app mapping, token audience error, unknown SPIFFE principal, mTLS failure, egress deny, capability replay, broker-not-called assertion, or downstream deny correlated by trace
D10-UC-02	Provider release approval and promotion screen	CI policy gates; artifact builder; schema registry; deployment controller; infrastructure reconciler; canary evaluator; rollback controller	Artifact and provenance store owned by build service; release state and approvals owned by control plane; cluster desired/observed state owned by runtime platform; generated app retains its own domain database unchanged	Failed gate, schema quarantine, digest mismatch, reconciliation drift, canary probe failure, blocked promotion, or rollback receipt

An enforceable repository separates applications, policies, schemas, platform components, infrastructure modules, environment overlays, and authorization tests. Infrastructure as code makes each change reviewable, but ownership remains split: runtime workload identities cannot grant human roles, network policy cannot approve a tool argument, and a generated-app administrator cannot alter provider releases or cloud identities.

A deployment custom resource binds organization, target team, immutable agent/version digest, invocation surfaces, capability names, team connector references, exact enabled tools, resource constraints, and risk policy. Secret-delivery resources contain references to vault entries, not refresh tokens in agent manifests or ordinary generated-app configuration.

Reference enforcement manifests

Strict mesh policy rejects plaintext peer traffic, while a separate authorization policy admits only the runtime workload to the tool-intent route. The gateway still evaluates the delegated run token and exact intent; network admission is only the first key.

apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
  name: strict-mtls
  namespace: tools
spec:
  mtls:
    mode: STRICT
---
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: allow-agent-runtime
  namespace: tools
spec:
  selector:
    matchLabels:
      app: tool-gateway
  action: ALLOW
  rules:
    - from:
        - source:
            principals:
              - spiffe://platform.internal/ns/agents/sa/runtime
      to:
        - operation:
            methods: ["POST"]
            paths: ["/v1/tool-intents"]

The reviewed deployment binds capability names to one team’s governed connector and arguments, never to a credential:

apiVersion: ai.platform.example/v1
kind: AgentDeployment
metadata:
  name: campaign-launch-sales
spec:
  organizationId: org_acme
  targetTeamId: team_sales
  agent:
    id: agent_campaign_launch
    version: 17
    digest: sha256:AGENT_VERSION_DIGEST
  invocation:
    surfaces: [library, agent_to_agent, schedule]
  capabilities:
    - name: collaboration.messages
      binding:
        kind: ConnectorInstance
        name: sales-slack
      tools:
        - name: slack.search_messages
          approval: none
        - name: slack.post_message
          approval: per_call
      constraints:
        allowedChannelIds: [C_SALES, C_CAMPAIGN_OPS]
  risk:
    maximumDataClassification: confidential
    externalSideEffects: approval_required

The connector resource declares protocol, team access, credential mode, tool risk, and egress. A user OAuth grant and an admin-provisioned service grant remain separate bindings; neither inherits the other. The dedicated credential vault owns refresh tokens, while deployment manifests hold only an opaque connector instance and a secret reference.

apiVersion: ai.platform.example/v1
kind: ConnectorInstance
metadata:
  name: sales-slack
spec:
  organizationId: org_acme
  type: slack
  transport: mcp
  teamAccess: [team_sales]
  authentication:
    mode: user_oauth
    clientCredentialRef: sales-slack-oauth-client
  tools:
    - name: slack.search_messages
      enabled: true
      riskClass: read_internal
    - name: slack.post_message
      enabled: true
      riskClass: external_side_effect
    - name: slack.admin_invite
      enabled: false
  network:
    egressPolicy: slack-api-only

Cloud identity follows the same least-privilege split: the runtime gets logging and monitoring, not secret-manager access; the gateway or broker alone gets narrowly scoped decrypt/access roles. Environment modules create clusters, keys, private endpoints, trust identities, and databases, while application repositories supply only reviewed deployment requests and tests.

module "agent_runtime_identity" {
  source = "../../modules/workload-identity"
  kubernetes_namespace       = "agents"
  kubernetes_service_account = "runtime"
  cloud_service_account_name = "agent-runtime"
  permissions = ["roles/logging.logWriter", "roles/monitoring.metricWriter"]
}

module "tool_gateway_identity" {
  source = "../../modules/workload-identity"
  kubernetes_namespace       = "tools"
  kubernetes_service_account = "tool-gateway"
  cloud_service_account_name = "tool-gateway"
  permissions = ["roles/secretmanager.secretAccessor", "roles/cloudkms.cryptoKeyDecrypter", "roles/logging.logWriter"]
}

Release and network state becomes unsafe when mutable names are mistaken for immutable identity, so every desired-state record pins digests and every observed-state record reports what was actually enforced. Opaque cross-plane references correlate evidence but never authorize by themselves.

Generated-application database: Not created in this slice — network trust and delivery use durable provider release, hosted-runtime identity, policy, and evidence stores; an existing generated app supplies only its independently authorized session and retains its domain data unchanged through promotion or rollback.

Record or entity	Store and owner	Primary key	Foreign key or opaque reference	Tenant key	Material constraint	Lifecycle and deletion	Use case IDs
WorkloadIdentity	Runtime workload registry; identity/security platform	spiffe_id	Local FK trust_domain_id; opaque Kubernetes service-account reference	organization_id for dedicated tenant runtime or None — platform workload is isolated by namespace and deployment policy	Unique SPIFFE identity per namespace/service account; certificate is short-lived and cannot assert user authority	Provision from reviewed service account; rotate certificates; revoke identity on workload retirement; retain issuance metadata	D10-UC-01
EgressPolicyRevision	Runtime network-policy store; cloud platform	egress_policy_digest	Opaque deployment_id; local destination-set revision	organization_id plus team_id	Default deny; exact source workload, destination, port, transport, timeout, and limit; no wildcard public egress	Create through reviewed infrastructure change; supersede atomically; retain revision and probe evidence; delete after deployment retirement	D10-UC-01, D10-UC-02
CapabilityTokenEvidence	Authorization evidence store; token issuer	token_jti_hash	Opaque run, decision, connector, and workload references	organization_id plus active team_id	Audience, tool, resource, argument digest, expiry, and single-use constraints all match; bearer value never stored	Created at allow decision; expires in minutes; replay marker retained to audit policy; purge per security retention	D10-UC-01
ArtifactVersion	Artifact/provenance store; build service	artifact_digest	Opaque source_commit_sha; local software-bill-of-materials reference	organization_id plus project ID	Content-addressed and immutable; signatures, manifest, and schema digests must verify	Create after green gates; never mutate; retain while referenced or for audit; garbage-collect only after retention and no deployments	D10-UC-02
ReleaseTransition	Provider release store; deployment controller	release_transition_id	Local FK current/prior artifact_digest; opaque runtime revision	organization_id plus team_id	State machine permits reviewed → canary → promoted or rolled back; production digest must equal tested digest	Append each request, approval, block, promotion, and rollback; retain immutable audit history; tombstone customer display after retention	D10-UC-02
ToolSchemaReview	Connector registry; AI platform owner	tool_schema_digest	Local FK connector type/version; opaque release reference	organization_id when customer-specific or None — globally reviewed connector schema	Discovery hash must equal reviewed hash; changed semantics quarantine before use	Create on review; supersede rather than overwrite; retain old digests and quarantine outcomes	D10-UC-01, D10-UC-02
RuntimeReconciliation	Runtime state store; infrastructure reconciler	reconciliation_id	Opaque provider release transition and cluster revision	organization_id plus deployment ID	Desired artifact, policy, workload, secret reference, and network digests must equal observed state	Append on apply and drift scan; retry bounded failures; retain proof through release retention; delete operational detail after audit export	D10-UC-01, D10-UC-02

The operational invariant is a chain of equalities: reviewed artifact digest equals canary digest equals promoted production digest, while reviewed schema and policy digests equal those loaded by the gateway. Rollback is a new transition to a previously verified immutable artifact; it never rewrites history or restores the generated application’s customer data.