21

Repeatable VPCs and Private Connectivity

Build the same isolated network contract in three AWS accounts before any Kubernetes workload exists.

System map · Day 21

Whole-system design

Five stable layers. Today's work is expanded and linked; the rest stays in context.

Product and authority

Covered — People and product entry points · Identity and policy · HelixWorks control plane · Generated application plane

Delivery and desired state

Covered — Git desired state · CI and immutable artifacts · Argo CD reconciliation

Cloud and orchestration

Terraform and AWS APIs

Source-backed today

Reuses one network module while each environment root and state remain separate.

Accounts, VPC, DNS, and private paths

Source-backed today

Builds private subnets, routes, endpoints, DNS, and NAT egress; this is not a fully no-internet cluster.

Kubernetes or EKS control plane

Source-backed today

Places cluster endpoints and workload network interfaces inside the declared private substrate.

Compute and traffic

Covered — Worker compute · Platform service workloads · Generated app workloadsAhead — Ambient mesh data plane

Storage and evidence

Covered — Infrastructure state · Cluster desired and live state · Product data and artifacts · Evidence and observability

The enterprise problem and today’s slice

Enterprise problem: A generated application can pass local tests yet leak data or become unreachable in AWS when routes, endpoints, and egress rules differ by environment. Whole-course context: The incoming evidence separates infrastructure state from workload state; today creates the network foundation consumed by every later EKS cluster. Today’s slice: We inspect the real reusable infra/modules/environment VPC declaration and instantiate it independently in dev, staging, and production AWS accounts. End-of-day evidence: Three reviewed plans identify account, VPC, public and private subnets, NAT routes, AWS service and optional enterprise connector endpoints, denied ingress expectation, environment, timestamp, and immutable run ID; VPC Flow Logs are explicitly a remaining gap because the current module does not declare them. Still unsolved: No AWS apply, EKS runtime, workload, public route, or Flow Log proof exists until owner-approved cloud execution.

Customer outcome and implementation focus

The customer outcome is a reviewable repeatable vpcs and private connectivity change, not a collection of requirements. This day starts with the implementation boundary, then uses the command or manifest below to produce positive, denied, and recovery evidence.

Components in focus

Terraform VPC module, Route 53 private DNS, endpoint/service endpoint, security groups/NACLs, and workload clients; ENI/NAT/PrivateLink network compute; Terraform state and VPC Flow Logs/S3 evidence; cache: not involved.

This map names the implementation boundary for this day. The service or controller changes only the state it owns; runtime and audit evidence let the operator distinguish a declared change from an effective one.

Declare private VPC connectivity as an enforceable route

Declare and inspect the network contract

An environment can silently target the wrong account, so validate caller identity before planning. Run this labelled snippet from the public helixworks-kubernetes-lab monorepo; its stable inputs are infra/modules/environment and infra/stacks/dev (with sibling staging and prod stacks).

: "${AWS_PROFILE:?set the approved AWS profile}" "${AWS_REGION:?set the intended AWS region}" "${EXPECTED_ACCOUNT_ID:?set the intended 12-digit AWS account}"
actual_account="$(aws --profile "$AWS_PROFILE" --region "$AWS_REGION" sts get-caller-identity --query Account --output text)"
[ "$actual_account" = "$EXPECTED_ACCOUNT_ID" ] || { echo "AWS account mismatch" >&2; exit 1; }
aws --profile "$AWS_PROFILE" --region "$AWS_REGION" sts get-caller-identity                         # Prove which account the shell can change.
terraform -chdir=infra/stacks/dev init -reconfigure -backend-config=backend.hcl
terraform -chdir=infra/stacks/dev plan -out=plan.bin -var-file=environment.tfvars
terraform -chdir=infra/stacks/dev show -json plan.bin > plan.json

backend.hcl and environment.tfvars are intentionally operator-supplied and uncommitted; the committed examples do not grant AWS authority. The module’s current network is private EKS placement with NAT egress and selected endpoints, not a fully private cluster with no internet egress.

Three authorization planes and lifecycle rule

Network reachability does not grant product authority, or a private packet could become an administrator session. The provider plane owns organizations and release policy; the hosted-runtime plane owns VPC, EKS network, workload identity, and deployment; each generated application owns its tenants, sessions, roles, and domain rows.

Decision rule: promote the same module revision, never credentials or state; retire application traffic and data dependencies before destroying a cluster, and destroy the cluster before its VPC.

Key takeaways

Private networking is a tested product boundary, not a collection of subnets.

  • Reuse module code while isolating accounts, CIDRs, roles, and state.
  • Prove both an approved path and a denied path.
  • A local Kind network shares one laptop and does not prove AWS failure-domain isolation.

Checklist

The network foundation is ready only when all three stacks are reviewable and independent.

  • [ ] Dev, staging, and production target distinct AWS accounts and state keys
  • [ ] Private subnets have no accidental public ingress path
  • [ ] Endpoint success and ingress denial evidence share one run window
  • [ ] Destruction order protects clusters and managed data