Isolate AWS Accounts, IAM, State, and Guardrails

Enterprise problem: If dev, staging, and production share an AWS account, Elastic Kubernetes Service (EKS) cluster, Terraform state, or broad administrator role, one mistaken variable or compromised runner can cross every environment and erase the evidence needed for recovery. Whole-course context: The incoming evidence is a Terraform change model with modules, locked state, physical effects, and guarded repair; today implements the maximum-isolation topology chosen for HelixWorks. Today’s slice: We assign dedicated AWS accounts, Virtual Private Clouds (VPCs), EKS clusters, state backends, short-lived Identity and Access Management (IAM) roles, secrets, observability, backups, deletion authority, and promotion gates to dev, staging, and production while local remains non-cloud. End-of-day evidence: An environment matrix, role-denial test, backend separation proof, cluster identities, artifact-digest promotion chain, customer probes, unaffected control, timestamps, run, and trace IDs demonstrate isolation. Still unsolved: The next slice will bootstrap and reconcile workloads through Argo CD, a GitOps controller, inside each isolated environment.

The customer outcome is a reviewable isolate aws accounts, iam, state, and guardrails change, not a collection of requirements. This day starts with the implementation boundary, then uses the command or manifest below to produce positive, denied, and recovery evidence.

AWS Organizations/IAM, Terraform state roles, EKS control planes, and audit trail; account-local compute/network; per-account state bucket/lock table and CloudTrail/S3 evidence; cache: not involved.

This map names the implementation boundary for this day. The service or controller changes only the state it owns; runtime and audit evidence let the operator distinguish a declared change from an effective one.

Name every environment owner

Isolation becomes actionable when each paid or sensitive resource has one accountable owner and deletion order. Local uses Kind and local secrets only; every cloud environment has its own account and cluster.

Policy review cannot prove the deployed trust relationship, so use a bounded dev identity to request explicitly forbidden production access. These commands must run in an approved test harness and are read-only or expected to fail.

: "${AWS_REGION:?set the intended AWS region}" "${EXPECTED_DEV_ACCOUNT_ID:?}" "${EXPECTED_PROD_ACCOUNT_ID:?}"
dev_account="$(aws --profile "${DEV_PROFILE:?set the approved dev profile}" --region "$AWS_REGION" sts get-caller-identity --query Account --output text)"
[ "$dev_account" = "$EXPECTED_DEV_ACCOUNT_ID" ] || { echo "dev account mismatch" >&2; exit 1; }
aws --profile "$DEV_PROFILE" --region "$AWS_REGION" sts assume-role --role-arn "${PROD_TERRAFORM_ROLE_ARN:?}" --role-session-name denied-test
aws --profile "$DEV_PROFILE" --region "$AWS_REGION" s3api head-object --bucket "${PROD_STATE_BUCKET:?}" --key "${PROD_STATE_KEY:?use the approved backend.hcl key}"
prod_account="$(aws --profile "${PROD_READ_PROFILE:?}" --region "$AWS_REGION" sts get-caller-identity --query Account --output text)"
[ "$prod_account" = "$EXPECTED_PROD_ACCOUNT_ID" ] || { echo "prod account mismatch" >&2; exit 1; }
aws --profile "$PROD_READ_PROFILE" --region "$AWS_REGION" eks describe-cluster --name helixworks-forge-prod --query 'cluster.arn'
test -n "${FORGE_URL:?set FORGE_URL to a reachable production control-plane URL}"
FORGE_URL="$FORGE_URL" FORGE_TOKEN="${FORGE_TOKEN:?set the explicit bearer token}" ./scripts/smoke-product.sh

The exact environment root is infra/stacks/prod, and the module derives cluster name helixworks-forge-prod; backend profiles and keys remain operator-supplied through backend.hcl, so the snippet requires them explicitly. The smoke script defaults to localhost only for local Compose; cloud proof must set an explicit reachable FORGE_URL.

The two dev-to-production calls must return access denied; the approved production read-only identity and customer probe are positive controls. Never grant temporary production access merely to make the test convenient.

Maximum isolation uses separate accounts, clusters, state, identities, and approvals while reusing modules and immutable artifacts.

• Namespaces do not isolate AWS IAM, Terraform state, cluster control plane, or account blast radius.
• Promotion moves a digest and evidence; it never rebuilds application bytes.
• A real access-denied test plus positive control proves deployed guardrails.

An AWS environment is isolated only when both intended access and denied cross-access are observable.

• [ ] Dev, staging, and production have distinct accounts, VPCs, EKS clusters, state keys, and roles
• [ ] No environment runner holds long-lived cloud or kubeconfig credentials
• [ ] Promotion preserves one signed artifact digest through independent gates
• [ ] Cross-account denials and approved production positive controls are recorded