Terraform: Plan, State, and Real Infrastructure
Follow one Terraform change from HCL through state and provider APIs to GKE nodes backed by real CPU, RAM, disks, and networks.
The enterprise problem and today’s slice
Enterprise problem: Ziba needs more schedulable memory, but applying an unreviewed Terraform edit can replace a node pool, race another operator, expose secrets through state, or leave paid resources partially changed while the blog remains unavailable. Whole-course context: The IaC discipline established versioned intent, controlled execution, and evidence; today examines Terraform as one stateful implementation of that discipline. Today’s slice: We connect HashiCorp Configuration Language (HCL), providers, refresh, plan, state, apply, and Google Cloud reality for a bounded GKE node-pool change. End-of-day evidence: A reviewer can explain every proposed create/update/replace/delete action, identify its state binding and physical effect, and prove the served Ziba revision after apply. Still unsolved: Kubernetes application reconciliation, advanced module design, state migration, disaster recovery, and production spend approval remain separate work.
Customer use cases
Terraform output is useful only when it protects the customer outcome, so these use cases bind resource mapping and concurrency to Ziba’s availability. One covers change; the other covers drift and failure recovery.
| Use case ID | Actor | Customer job | Success outcome | Denial or recovery evidence |
|---|---|---|---|---|
| D64-UC-01 | Ziba infrastructure engineer | Add GKE node capacity through an approved saved plan | New nodes register with expected allocatable memory, pods schedule, and Ziba serves the approved digest | Replacement or destruction is rejected during review; production remains unchanged |
| D64-UC-02 | Ziba operations lead | Recover safely from drift, lock contention, or partial apply | State binding, provider reality, and configuration are reconciled without duplicate resources | Concurrent writer is blocked; failed action and unaffected Ziba probe are retained |
Actor-centred user stories
The words “plan succeeded” do not prove what will run, so actors need acceptance conditions across Terraform and Kubernetes. These stories make identity, inputs, resource IDs, and service evidence explicit.
| Story ID | Use case IDs | User story | Observable acceptance conditions |
|---|---|---|---|
| D64-US-01 | D64-UC-01 | As a Ziba infrastructure engineer, I want approval bound to a saved Terraform plan, so that apply performs exactly the reviewed GKE changes | Commit, plan checksum, create/update/replace/delete summary, approver, state serial, provider IDs, node allocatable memory, image digest, HTTP result, environment, time, and run ID are captured |
| D64-US-02 | D64-UC-02 | As a Ziba operations lead, I want exclusive remote state and explicit drift recovery, so that two writers cannot corrupt ownership or create duplicate node pools | Second writer receives lock denial, imported or reverted object has one binding, partial failure is recorded, and unaffected Ziba remains reachable |
End-to-end product flows
Terraform crosses configuration, state, and remote APIs, so skipping any comparison makes the proposed change misleading. The flows use a saved plan and a recovery decision rather than assuming rerunning apply is always safe.
| Flow ID | Use case IDs | Path | Trigger | Numbered steps | Terminal evidence |
|---|---|---|---|---|---|
| D64-FLOW-01 | D64-UC-01 | Happy | Engineer proposes a larger Ziba node-pool capacity | 1. Initialise pinned providers/backend. 2. Validate HCL. 3. Lock and read state. 4. Refresh from Google APIs. 5. Build and save the plan. 6. Approve its checksum. 7. Apply that saved plan. 8. Observe nodes, pods, and Ziba. | Actor, resource, scope, precondition, planned/observed effects, state serial, environment, timestamp, commit, plan checksum, apply ID, and request trace |
| D64-FLOW-02 | D64-UC-02 | Recovery | Provider call fails after some resources changed or drift appears | 1. Preserve logs and lock. 2. Read state and Google reality. 3. Identify completed and failed actions. 4. Decide import, configuration adoption, revert, or retry. 5. Generate a fresh plan. 6. Deny an unapproved writer. 7. Apply approved recovery and probe Ziba. | Failed API action, resource ID, state before/after, denied writer, recovery decision, fresh plan/apply IDs, and positive-control request |
System design derived from the flows
Terraform state is not the cloud and not merely a cache: it binds resource addresses in configuration to remote object identities and records metadata needed to plan changes. HashiCorp documents that Terraform refreshes resource information before operations and expects a one-to-one mapping between a configured resource instance and a remote object (Terraform state, purpose of state).
| Use case ID | Entry point | Responsible services | Authoritative store | Failure evidence |
|---|---|---|---|---|
| D64-UC-01 | Reviewed HCL and saved plan | Terraform CLI/runner, backend, Google provider, GKE and Compute Engine APIs, Kubernetes observers | Git for configuration; backend for Terraform bindings/metadata; Google APIs for live resources; Kubernetes API for node/pod state | Validation error, stale plan, lock failure, replacement not approved, provider error, node absent, pod Pending, or failed HTTP probe |
| D64-UC-02 | Failed run or drift alert | Backend lock/state, Terraform refresh/plan, provider import/read/update, audit logs, incident evidence store | Remote backend plus provider APIs and immutable execution evidence | Conflicting lock, missing/duplicate binding, partial apply, manual drift, state persistence failure, or unresolved customer regression |
Data model and ownership
Treating the state file as harmless generated output risks credential exposure and concurrent corruption. HashiCorp recommends a secure remote backend for collaboration and warns against storage without appropriate access control and locking because state can contain sensitive values (state storage guidance).
Generated-application database: Not created in this slice — Terraform backend records, provider resources, Kubernetes observations, and release evidence are sufficient; Ziba’s domain data is not owned by infrastructure state.
| Record or entity | Store and owner | Primary key | Foreign key or opaque reference | Tenant key | Material constraint | Lifecycle and deletion | Use case IDs |
|---|---|---|---|---|---|---|---|
| TerraformConfiguration | Git, owned by infrastructure engineering | Commit SHA plus module path | Provider/module version checksums | Environment/project ID | Reviewed immutable revision; no plaintext secrets; resource addresses stable across refactors | Versioned, reviewed, retired through explicit removal/move blocks and apply | D64-UC-01, D64-UC-02 |
| TerraformStateSnapshot | Encrypted remote backend, owned by platform operations | Workspace/key plus serial | Remote provider IDs bound to Terraform resource addresses | Environment/project ID | One remote object binds to one resource instance; writes require current lock/serial | New snapshot after successful operations; versions retained/recovered by backend policy; never hand-edited | |
| SavedExecutionPlan | CI artifact store, owned by release engineering | Plan checksum | Configuration commit, prior state serial, provider schema/version | Environment/project ID | Approval applies only to exact immutable inputs; secret access restricted | Created per proposal, invalidated by changed inputs, deleted after retention window | D64-UC-01 |
| ApplyAndRuntimeProof | CI evidence store, Google audit logs, Kubernetes observations | Apply run ID | Opaque plan checksum, cloud resource IDs, node UIDs, and request trace | Environment/project ID | Observed effects and customer probe must correspond to approved plan and image digest | Appended per run, retained immutably, expired under audit policy | D64-UC-01, D64-UC-02 |
Expand plan and apply one block at a time
The shallow model says Terraform reads code and creates infrastructure, but it actually compares three views before issuing operations. Expand the middle box into configuration, prior bindings, refreshed remote attributes, a dependency graph, and an execution result.
Terraform’s language is declarative: resource blocks describe intended infrastructure rather than an ordered procedure (Terraform language). Apply can create, update, destroy, or replace resources and then update state so configuration, remote objects, and state correspond (resource lifecycle overview).
HCL for Ziba’s GKE node pool
A minimal example can still cause expensive resources and disruption, so treat this HCL as a reviewed lab shape rather than a paste-to-production recipe. It creates a GKE cluster without the default node pool, then a separately managed node pool whose VMs supply pod CPU and memory.
terraform {
required_version = "~> 1.11"
required_providers {
google = {
source = "hashicorp/google"
version = "~> 6.0"
}
}
}
provider "google" {
project = var.project_id
region = "europe-west2"
}
resource "google_container_cluster" "ziba" {
name = "ziba-prod"
location = "europe-west2"
remove_default_node_pool = true
initial_node_count = 1
networking_mode = "VPC_NATIVE"
deletion_protection = true
}
resource "google_container_node_pool" "general" {
name = "general"
cluster = google_container_cluster.ziba.id
location = google_container_cluster.ziba.location
autoscaling {
min_node_count = 1
max_node_count = 4
}
node_config {
machine_type = "e2-standard-4"
disk_type = "pd-balanced"
disk_size_gb = 100
}
}
Do not hard-code credentials. Supply short-lived workload identity to the runner, restrict the backend separately from infrastructure APIs, and review provider/module upgrades because they can change schemas and plans.
Code versus real effects
HCL names logical resources, while providers translate them into API operations whose software and hardware consequences vary. Review the proposed action, especially replacement, rather than inferring safety from a small text edit.
| HCL or command | Terraform/control-plane effect | Software effect | Hardware effect | Ziba deployment outcome |
|---|---|---|---|---|
terraform init | Configures backend and installs pinned providers/modules | Downloads executable provider code | Allocates no cloud hardware | Makes planning possible; does not deploy Ziba |
terraform plan -out=ziba.tfplan | Locks/reads state, refreshes remote objects, serializes proposed actions | Provider reads Google APIs | Normally no hardware mutation | Reveals likely blast radius; does not prove availability |
Change machine_type | Often proposes node VM replacement through node-pool update | Old kubelet leaves; pods drain and restart on new nodes | Releases old VM allocation and requests different vCPU/DRAM-backed VMs | May fix Pending capacity but can disrupt under-replicated workloads |
Increase max_node_count | Updates autoscaler ceiling | Autoscaler may later create nodes for unschedulable pods | No VM is added until scaling demand triggers it | Does not fix a container’s unchanged memory limit |
terraform apply ziba.tfplan | Executes exact saved actions and advances state | Controllers and node agents converge around changed resources | Creates/changes/deletes paid VMs, disks, IPs, and routes | Capacity changes only become useful after node registration and pod scheduling |
terraform destroy | Proposes and executes deletions for managed resources | Stops cluster/node software | Releases infrastructure, subject to deletion protection/dependencies | Can make Ziba unavailable and destroy infrastructure stateful dependencies |
Operate the workflow and recover safely
Rerunning a failed apply without inspection can hide partial success or collide with another writer. Use a bounded sequence, preserve the lock unless you prove it is stale, and never edit state JSON directly.
terraform fmt -check
terraform init -lockfile=readonly
terraform validate
terraform plan -out=ziba.tfplan
terraform show ziba.tfplan
sha256sum ziba.tfplan
terraform apply ziba.tfplan
terraform output -json
kubectl get nodes -o wide
kubectl get pods -n blog -l app=ziba -o wide
curl --fail --show-error "${ZIBA_URL}/healthz"
If reality contains a legitimate object absent from state, inspect and import it under one reviewed resource address, then generate a fresh plan. If configuration and state deliberately change addresses, use supported refactoring operations. terraform state rm forgets an object but does not delete the remote object, so it can create unmanaged paid infrastructure if used casually.
Failure modes and decision rules
A Terraform failure is not automatically solved by force-unlocking or applying again. Identify whether configuration, binding, remote API execution, persistence, or runtime verification failed.
| Evidence | Likely boundary | Safe next decision |
|---|---|---|
| Backend lock held | Concurrent or abandoned writer | Identify owner/run; wait, or force-unlock only after proving no writer exists |
| Plan unexpectedly replaces cluster/node pool | Immutable argument or provider behavior | Stop and review disruption, dependencies, and alternative migration |
| Apply partially fails | Some provider actions completed | Preserve logs, refresh, compare state with provider reality, then create a fresh recovery plan |
| Resource exists but is absent from state | Out-of-band creation or lost binding | Decide import, deletion, or adoption; never bind one object twice |
| Apply succeeds but Ziba fails | Infrastructure API success did not prove application outcome | Inspect nodes, scheduling, readiness, Service path, and image revision |
Decision rule: approve the graph of actual proposed operations, not the visual size of the HCL diff. Protect state as sensitive operational data, serialize writers, and prove both provider convergence and the customer request.
Key takeaways
Terraform implements IaC through declarative configuration, provider plugins, execution plans, and persistent state bindings. None of these alone is the real infrastructure.
- Configuration says what Terraform should manage; state maps addresses to remote object identities.
- Plan compares configuration, state, and refreshed provider reality; apply performs the proposed API operations.
- A VM resource ultimately allocates physical compute, memory, storage, and networking capacity.
- Saved-plan approval, remote locking, state protection, and runtime probes form one safety chain.
- Terraform can create GKE foundations; Argo CD will later complement it by reconciling Kubernetes application manifests.
Checklist
A Terraform run is complete only when resource ownership and the customer outcome agree. Retain this evidence with the exact configuration revision.
- [ ] Pinned Terraform, provider, and module versions with a committed dependency lock file
- [ ] Encrypted remote backend, restricted access, versioning, and supported locking/concurrency control
- [ ] Fresh saved plan reviewed for create/update/replace/delete and bound to approval
- [ ] Short-lived least-privilege provider identity and separate backend authority
- [ ] State serial, provider IDs, node allocatable values, pod placement, image digest, and Ziba HTTP result captured
- [ ] Recovery procedure covers partial apply, import, refactor, stale lock, and state backup restoration