Secrets, Network, Admission, and Supply-Chain Guardrails

This course is built around the public Zheta Kubernetes Lab monorepo. The excerpt below is runnable source, not pseudocode.

Source: gitops/apps/forge/base/workloads.yaml

serviceAccountName: control-plane
      securityContext: { runAsNonRoot: true, fsGroup: 65532, fsGroupChangePolicy: OnRootMismatch, seccompProfile: { type: RuntimeDefault } }
      containers:
        - name: control-plane
          image: zheta-forge/control-plane:v1
          envFrom: [{ configMapRef: { name: forge-environment } }, { secretRef: { name: control-plane-secrets } }]
          ports: [{ name: http, containerPort: 8080 }]
          readinessProbe: { httpGet: { path: /healthz, port: http }, periodSeconds: 5 }
          livenessProbe: { httpGet: { path: /healthz, port: http }, periodSeconds: 10 }
          resources: { requests: { cpu: 50m, memory: 64Mi }, limits: { cpu: 500m, memory: 256Mi } }

Code to reality

Declared intent

Give one microservice a distinct workload identity, hardened runtime, health contract, and capacity bounds.

Interpreter

Kubernetes admission, scheduler, kubelet, and workload-identity integration interpret different fields.

Software effect

The Pod runs as the control-plane ServiceAccount and publishes readiness only after its health endpoint responds.

Hardware effect

The scheduler reserves requested CPU and memory; the runtime enforces the limit and Linux security context.

Observable evidence

Pod identity, security context, readiness, resource metrics, and a denied excess-authority probe prove the contract.

The source tables below remain the detailed contract. Begin with these customer paths:

• D27-UC-01
  • Person: Service owner
  • Job: Run Forge with environment-scoped secrets and only required network paths
  • Observable result: Workload identity retrieves approved secret material and services communicate through named policies
• D27-UC-02
  • Person: Security engineer
  • Job: Prevent privileged or untrusted workloads entering production
  • Observable result: Current pod-hardening dry-run passes and missing signature admission is a launch veto

Now turn each customer job into a story with a result that an engineer can check:

• D27-US-01
  • Story: As a service owner, I want least-privilege workload identity and network policy, so that a compromised service cannot inherit another service’s data…
  • Observable acceptance: Positive and denied tests name service account, secret ARN, destination, policy revision, result, environment, time, and trace
• D27-US-02
  • Story: As a security engineer, I want the launch gate to distinguish declared pod hardening from missing signature verification, so that an unimplemented…
  • Observable acceptance: Dry-run identifies manifest/policy result and gate records absent verifier, affected digest policy, actor, time, and audit ID

Finally trace each story through the system that owns its state and the evidence that proves the outcome:

• D27-FLOW-01
  • Trigger: Reviewer validates a digest-pinned candidate
  • Responsible systems: EKS workload identity, IAM, secret manager/controller, Kubernetes NetworkPolicy/CNI, Forge service
  • Authoritative state: AWS Secrets Manager for secret value; IAM/policy APIs for authority; Kubernetes API for bindings and network intent
  • Owned record: WorkloadGrant
  • Observable evidence: Manifest digest, dry-run result, service accounts, policy/identity refs, explicit live-proof gaps, environment, time, and run ID
  • Failure signal: Wrong secret scope, credential exchange denial, missing revision, blocked required flow, or unexpected allowed egress
• D27-FLOW-02
  • Trigger: Launch review searches for an unsigned-image denial control
  • Responsible systems: Kubernetes API admission, manifest reviewer, missing-signature-control gate
  • Authoritative state: Git policy revision, server dry-run result, immutable gap record
  • Owned record: AdmissionPolicyRevision
  • Observable evidence: Missing verifier finding, zero false denial claim, render result, policy revision, actor, environment, time, and audit ID
  • Failure signal: Gap incorrectly waived, unsupported field accepted, or production proceeds without signature verifier

Enterprise problem: A digest-pinned image can still expose secrets, call arbitrary networks, run with dangerous privileges, or originate from an untrusted build. Whole-course context: Promotion proves exact image identity only after real digests exist; today audits what the repository actually constrains. Today’s slice: The base declares non-root containers, dropped capabilities, read-only filesystems, resource limits, service accounts, and NetworkPolicy; Terraform declares Pod Identity and Secrets Manager scope. No signature-verifying admission controller is installed. End-of-day evidence: Server dry-run and authorization checks can prove declared controls, while unsigned-image denial remains explicitly unproven. Still unsolved: Launch is blocked until signature admission and live network/secret positive and negative tests exist.

Security controls matter only when they stop a realistic forbidden action and preserve an approved path. Four mechanisms form one defence but retain separate owners.

Mounting one shared credential into many services expands blast radius, so each service receives its own machine authority. Human delegated authority remains a separate optional grant.

The customer action remains running a generated application; guardrails are successful when they are mostly invisible on the happy path and decisive on forbidden paths.

Secrets, identity, networking, API admission, pod hardening, and image signing are complementary controls at different interfaces. The first five have declarations; signature verification is currently a required gap.

Secret values must not enter Git or evidence records, but secret revision and denial metadata remain auditable. Existing generated-app data remains independently authorized.

Generated-application database: Required in this slice — generated applications keep tenant data and authorization predicates; guardrails protect access without transferring data ownership.

Rendering catches missing declarations but cannot prove live admission or packets, so combine local render with server-side dry-run in the explicit production context. The repository declares Pod Identity associations in infra/modules/environment, pod hardening and Services in the base, NetworkPolicy in gitops/apps/forge/base/network-policy.yaml, and required release policy in the production overlay; it does not yet install a signature-verifying admission controller, so signature admission remains a launch gap rather than a passed control. AWS requires the Pod Identity Agent for Pod Identity associations on ordinary EKS clusters (EKS Pod Identity); the module installs that addon.

: "${AWS_PROFILE:?set the approved AWS profile}" "${AWS_REGION:?set the intended AWS region}" "${EXPECTED_ACCOUNT_ID:?set the intended 12-digit AWS account}"
actual_account="$(aws --profile "$AWS_PROFILE" --region "$AWS_REGION" sts get-caller-identity --query Account --output text)"
[ "$actual_account" = "$EXPECTED_ACCOUNT_ID" ] || { echo "AWS account mismatch" >&2; exit 1; }
kubectl kustomize gitops/apps/forge/overlays/prod >/tmp/forge-prod.yaml
aws --profile "$AWS_PROFILE" --region "$AWS_REGION" eks update-kubeconfig --name zheta-forge-prod --alias zheta-forge-prod
kubectl --context zheta-forge-prod apply --server-side --dry-run=server -f /tmp/forge-prod.yaml
kubectl --context zheta-forge-prod -n zheta-forge auth can-i get secret --as=system:serviceaccount:zheta-forge:generator
kubectl --context zheta-forge-prod -n zheta-forge get networkpolicy,serviceaccount

Provider membership may approve releases, hosted-runtime grants let pods reach secrets and connectors, and generated-app roles govern tenant rows. No token crosses those boundaries without an explicit audited mapping.

Decision rule: revoke a workload grant without deleting application users; revoke an application session without rotating deployment identity; retire workloads before secret and network teardown.

Production guardrails form a chain of independent controls.

• Machine authority precedes optional delegated user authority; neither inherits the other.
• Default-deny needs explicit positive paths and negative proof.
• Admission must verify immutable artifact and pod behavior before scheduling.

The runtime is guarded only when failure is closed and observable.

• [ ] Secret values never enter Git or evidence
• [ ] Service identities are environment and responsibility scoped
• [ ] Network policy allows required paths and denies an unapproved path
• [ ] Signature-verifying admission is installed and an unsigned image is denied; until then launch stays blocked