64

Terraform: Plan, State, and Real Infrastructure

Follow one Terraform change from HCL through state and provider APIs to GKE nodes backed by real CPU, RAM, disks, and networks.

The enterprise problem and today’s slice

Enterprise problem: Ziba needs more schedulable memory, but applying an unreviewed Terraform edit can replace a node pool, race another operator, expose secrets through state, or leave paid resources partially changed while the blog remains unavailable. Whole-course context: The IaC discipline established versioned intent, controlled execution, and evidence; today examines Terraform as one stateful implementation of that discipline. Today’s slice: We connect HashiCorp Configuration Language (HCL), providers, refresh, plan, state, apply, and Google Cloud reality for a bounded GKE node-pool change. End-of-day evidence: A reviewer can explain every proposed create/update/replace/delete action, identify its state binding and physical effect, and prove the served Ziba revision after apply. Still unsolved: Kubernetes application reconciliation, advanced module design, state migration, disaster recovery, and production spend approval remain separate work.

Customer use cases

Terraform output is useful only when it protects the customer outcome, so these use cases bind resource mapping and concurrency to Ziba’s availability. One covers change; the other covers drift and failure recovery.

Use case IDActorCustomer jobSuccess outcomeDenial or recovery evidence
D64-UC-01Ziba infrastructure engineerAdd GKE node capacity through an approved saved planNew nodes register with expected allocatable memory, pods schedule, and Ziba serves the approved digestReplacement or destruction is rejected during review; production remains unchanged
D64-UC-02Ziba operations leadRecover safely from drift, lock contention, or partial applyState binding, provider reality, and configuration are reconciled without duplicate resourcesConcurrent writer is blocked; failed action and unaffected Ziba probe are retained

Actor-centred user stories

The words “plan succeeded” do not prove what will run, so actors need acceptance conditions across Terraform and Kubernetes. These stories make identity, inputs, resource IDs, and service evidence explicit.

Story IDUse case IDsUser storyObservable acceptance conditions
D64-US-01D64-UC-01As a Ziba infrastructure engineer, I want approval bound to a saved Terraform plan, so that apply performs exactly the reviewed GKE changesCommit, plan checksum, create/update/replace/delete summary, approver, state serial, provider IDs, node allocatable memory, image digest, HTTP result, environment, time, and run ID are captured
D64-US-02D64-UC-02As a Ziba operations lead, I want exclusive remote state and explicit drift recovery, so that two writers cannot corrupt ownership or create duplicate node poolsSecond writer receives lock denial, imported or reverted object has one binding, partial failure is recorded, and unaffected Ziba remains reachable

End-to-end product flows

Terraform crosses configuration, state, and remote APIs, so skipping any comparison makes the proposed change misleading. The flows use a saved plan and a recovery decision rather than assuming rerunning apply is always safe.

Flow IDUse case IDsPathTriggerNumbered stepsTerminal evidence
D64-FLOW-01D64-UC-01HappyEngineer proposes a larger Ziba node-pool capacity1. Initialise pinned providers/backend.
2. Validate HCL.
3. Lock and read state.
4. Refresh from Google APIs.
5. Build and save the plan.
6. Approve its checksum.
7. Apply that saved plan.
8. Observe nodes, pods, and Ziba.
Actor, resource, scope, precondition, planned/observed effects, state serial, environment, timestamp, commit, plan checksum, apply ID, and request trace
D64-FLOW-02D64-UC-02RecoveryProvider call fails after some resources changed or drift appears1. Preserve logs and lock.
2. Read state and Google reality.
3. Identify completed and failed actions.
4. Decide import, configuration adoption, revert, or retry.
5. Generate a fresh plan.
6. Deny an unapproved writer.
7. Apply approved recovery and probe Ziba.
Failed API action, resource ID, state before/after, denied writer, recovery decision, fresh plan/apply IDs, and positive-control request

System design derived from the flows

Terraform state is not the cloud and not merely a cache: it binds resource addresses in configuration to remote object identities and records metadata needed to plan changes. HashiCorp documents that Terraform refreshes resource information before operations and expects a one-to-one mapping between a configured resource instance and a remote object (Terraform state, purpose of state).

Use case IDEntry pointResponsible servicesAuthoritative storeFailure evidence
D64-UC-01Reviewed HCL and saved planTerraform CLI/runner, backend, Google provider, GKE and Compute Engine APIs, Kubernetes observersGit for configuration; backend for Terraform bindings/metadata; Google APIs for live resources; Kubernetes API for node/pod stateValidation error, stale plan, lock failure, replacement not approved, provider error, node absent, pod Pending, or failed HTTP probe
D64-UC-02Failed run or drift alertBackend lock/state, Terraform refresh/plan, provider import/read/update, audit logs, incident evidence storeRemote backend plus provider APIs and immutable execution evidenceConflicting lock, missing/duplicate binding, partial apply, manual drift, state persistence failure, or unresolved customer regression

Data model and ownership

Treating the state file as harmless generated output risks credential exposure and concurrent corruption. HashiCorp recommends a secure remote backend for collaboration and warns against storage without appropriate access control and locking because state can contain sensitive values (state storage guidance).

Generated-application database: Not created in this slice — Terraform backend records, provider resources, Kubernetes observations, and release evidence are sufficient; Ziba’s domain data is not owned by infrastructure state.

Record or entityStore and ownerPrimary keyForeign key or opaque referenceTenant keyMaterial constraintLifecycle and deletionUse case IDs
TerraformConfigurationGit, owned by infrastructure engineeringCommit SHA plus module pathProvider/module version checksumsEnvironment/project IDReviewed immutable revision; no plaintext secrets; resource addresses stable across refactorsVersioned, reviewed, retired through explicit removal/move blocks and applyD64-UC-01, D64-UC-02
TerraformStateSnapshotEncrypted remote backend, owned by platform operationsWorkspace/key plus serialRemote provider IDs bound to Terraform resource addressesEnvironment/project IDOne remote object binds to one resource instance; writes require current lock/serialNew snapshot after successful operations; versions retained/recovered by backend policy; never hand-edited
SavedExecutionPlanCI artifact store, owned by release engineeringPlan checksumConfiguration commit, prior state serial, provider schema/versionEnvironment/project IDApproval applies only to exact immutable inputs; secret access restrictedCreated per proposal, invalidated by changed inputs, deleted after retention windowD64-UC-01
ApplyAndRuntimeProofCI evidence store, Google audit logs, Kubernetes observationsApply run IDOpaque plan checksum, cloud resource IDs, node UIDs, and request traceEnvironment/project IDObserved effects and customer probe must correspond to approved plan and image digestAppended per run, retained immutably, expired under audit policyD64-UC-01, D64-UC-02

Expand plan and apply one block at a time

The shallow model says Terraform reads code and creates infrastructure, but it actually compares three views before issuing operations. Expand the middle box into configuration, prior bindings, refreshed remote attributes, a dependency graph, and an execution result.

Terraform’s language is declarative: resource blocks describe intended infrastructure rather than an ordered procedure (Terraform language). Apply can create, update, destroy, or replace resources and then update state so configuration, remote objects, and state correspond (resource lifecycle overview).

HCL for Ziba’s GKE node pool

A minimal example can still cause expensive resources and disruption, so treat this HCL as a reviewed lab shape rather than a paste-to-production recipe. It creates a GKE cluster without the default node pool, then a separately managed node pool whose VMs supply pod CPU and memory.

terraform {
  required_version = "~> 1.11"
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 6.0"
    }
  }
}

provider "google" {
  project = var.project_id
  region  = "europe-west2"
}

resource "google_container_cluster" "ziba" {
  name                     = "ziba-prod"
  location                 = "europe-west2"
  remove_default_node_pool = true
  initial_node_count       = 1
  networking_mode          = "VPC_NATIVE"
  deletion_protection      = true
}

resource "google_container_node_pool" "general" {
  name     = "general"
  cluster  = google_container_cluster.ziba.id
  location = google_container_cluster.ziba.location

  autoscaling {
    min_node_count = 1
    max_node_count = 4
  }

  node_config {
    machine_type = "e2-standard-4"
    disk_type    = "pd-balanced"
    disk_size_gb = 100
  }
}

Do not hard-code credentials. Supply short-lived workload identity to the runner, restrict the backend separately from infrastructure APIs, and review provider/module upgrades because they can change schemas and plans.

Code versus real effects

HCL names logical resources, while providers translate them into API operations whose software and hardware consequences vary. Review the proposed action, especially replacement, rather than inferring safety from a small text edit.

HCL or commandTerraform/control-plane effectSoftware effectHardware effectZiba deployment outcome
terraform initConfigures backend and installs pinned providers/modulesDownloads executable provider codeAllocates no cloud hardwareMakes planning possible; does not deploy Ziba
terraform plan -out=ziba.tfplanLocks/reads state, refreshes remote objects, serializes proposed actionsProvider reads Google APIsNormally no hardware mutationReveals likely blast radius; does not prove availability
Change machine_typeOften proposes node VM replacement through node-pool updateOld kubelet leaves; pods drain and restart on new nodesReleases old VM allocation and requests different vCPU/DRAM-backed VMsMay fix Pending capacity but can disrupt under-replicated workloads
Increase max_node_countUpdates autoscaler ceilingAutoscaler may later create nodes for unschedulable podsNo VM is added until scaling demand triggers itDoes not fix a container’s unchanged memory limit
terraform apply ziba.tfplanExecutes exact saved actions and advances stateControllers and node agents converge around changed resourcesCreates/changes/deletes paid VMs, disks, IPs, and routesCapacity changes only become useful after node registration and pod scheduling
terraform destroyProposes and executes deletions for managed resourcesStops cluster/node softwareReleases infrastructure, subject to deletion protection/dependenciesCan make Ziba unavailable and destroy infrastructure stateful dependencies

Operate the workflow and recover safely

Rerunning a failed apply without inspection can hide partial success or collide with another writer. Use a bounded sequence, preserve the lock unless you prove it is stale, and never edit state JSON directly.

terraform fmt -check
terraform init -lockfile=readonly
terraform validate
terraform plan -out=ziba.tfplan
terraform show ziba.tfplan
sha256sum ziba.tfplan
terraform apply ziba.tfplan
terraform output -json
kubectl get nodes -o wide
kubectl get pods -n blog -l app=ziba -o wide
curl --fail --show-error "${ZIBA_URL}/healthz"

If reality contains a legitimate object absent from state, inspect and import it under one reviewed resource address, then generate a fresh plan. If configuration and state deliberately change addresses, use supported refactoring operations. terraform state rm forgets an object but does not delete the remote object, so it can create unmanaged paid infrastructure if used casually.

Failure modes and decision rules

A Terraform failure is not automatically solved by force-unlocking or applying again. Identify whether configuration, binding, remote API execution, persistence, or runtime verification failed.

EvidenceLikely boundarySafe next decision
Backend lock heldConcurrent or abandoned writerIdentify owner/run; wait, or force-unlock only after proving no writer exists
Plan unexpectedly replaces cluster/node poolImmutable argument or provider behaviorStop and review disruption, dependencies, and alternative migration
Apply partially failsSome provider actions completedPreserve logs, refresh, compare state with provider reality, then create a fresh recovery plan
Resource exists but is absent from stateOut-of-band creation or lost bindingDecide import, deletion, or adoption; never bind one object twice
Apply succeeds but Ziba failsInfrastructure API success did not prove application outcomeInspect nodes, scheduling, readiness, Service path, and image revision

Decision rule: approve the graph of actual proposed operations, not the visual size of the HCL diff. Protect state as sensitive operational data, serialize writers, and prove both provider convergence and the customer request.

Key takeaways

Terraform implements IaC through declarative configuration, provider plugins, execution plans, and persistent state bindings. None of these alone is the real infrastructure.

  • Configuration says what Terraform should manage; state maps addresses to remote object identities.
  • Plan compares configuration, state, and refreshed provider reality; apply performs the proposed API operations.
  • A VM resource ultimately allocates physical compute, memory, storage, and networking capacity.
  • Saved-plan approval, remote locking, state protection, and runtime probes form one safety chain.
  • Terraform can create GKE foundations; Argo CD will later complement it by reconciling Kubernetes application manifests.

Checklist

A Terraform run is complete only when resource ownership and the customer outcome agree. Retain this evidence with the exact configuration revision.

  • [ ] Pinned Terraform, provider, and module versions with a committed dependency lock file
  • [ ] Encrypted remote backend, restricted access, versioning, and supported locking/concurrency control
  • [ ] Fresh saved plan reviewed for create/update/replace/delete and bound to approval
  • [ ] Short-lived least-privilege provider identity and separate backend authority
  • [ ] State serial, provider IDs, node allocatable values, pod placement, image digest, and Ziba HTTP result captured
  • [ ] Recovery procedure covers partial apply, import, refactor, stale lock, and state backup restoration