03

Drawing the Boundary: The Hard Type-Split with Terraform

The whole model rests on one line: which cloud resources the in-cluster controller may own, and which stay with Terraform. Draw that line loosely and the two systems fight over the same objects; draw it as an unambiguous, enforceable test and they coexist cleanly forever. This day turns "foundation vs app-scoped" into a single-axis rule, works the genuinely hard case (a grant that crosses the line), and shows how to keep the boundary honest as the platform grows.

Why the boundary is load-bearing

Terraform and an in-cluster controller both reconcile continuously. If a single cloud object is ever owned by both, each will revert the other's changes in an endless loop — a reconcile war with no winner. So the boundary is not documentation; it is the invariant that keeps the system stable. The safe configuration is two disjoint domains, no cloud object owned by both.

Rule of thumb: the type-split is not a style preference, it is the precondition that prevents two reconcilers from co-managing one object — design it as an invariant you can enforce, not a convention you hope people follow.

The test: one axis, lifecycle plus sharing

The classification collapses to a single question about lifecycle and sharing. Everything else is noise.

Owned by one app, created and destroyed with it, referenced only by it → app-scoped (controller). Shared across apps, part of the network/security/account perimeter, or longer-lived than any single app → foundation (Terraform).

ResourceOwnerWhy
An app's own bucket / service account / IAM role / cacheControllerSingle-app lifecycle
VPC, subnets, project / accountTerraformPerimeter, shared
Shared KMS key, org / base IAM, DNS zoneTerraformMulti-tenant, security
A data store used by many appsTerraformShared lifecycle

The test is deliberately blunt so it produces the same answer for everyone. When a resource is genuinely ambiguous, there is a default: treat it as foundation and raise it for an explicit decision. The self-service layer must never silently claim something that might belong to the perimeter. Rule of thumb: encode the boundary as one testable axis with a safe default (foundation) — a rule that needs a judgement call every time is a rule that will drift.

The hard case: cross-boundary grants

The messy case is not classification — it is a grant that crosses the line. An app-scoped identity (controller-owned) frequently needs access to a foundation resource: permission to decrypt with a shared KMS key, or to read a shared bucket. The dangerous shortcut is to let the controller write a policy attachment onto the foundation object. Now two systems manage one object again, and the boundary has leaked.

The convention that keeps the split clean: the controller owns the app-side principal; Terraform owns any grant on a foundation resource. The app requests access through its claim; the foundation plane issues the grant on its own object. The controller never writes policy onto something Terraform owns.

Rule of thumb: when access must cross the boundary, split the two halves — the requesting principal belongs to the self-service plane, the grant on the shared resource belongs to the foundation plane — so that no single cloud object ever has two owners.

Greenfield makes the pilot safe

There is a reason to start app-scoped and greenfield: if you never migrate an existing Terraform-managed resource, there are no shared objects at all during the pilot, so boundary leakage cannot physically happen yet. New app-scoped resources are created fresh under the controller; everything Terraform holds today stays exactly where it is. Migration of existing resources is explicitly out of scope — you don't move a service's current role out of Terraform state into the controller, you mint a new per-service identity and bind the workload to it.

DecisionChoiceEffect
Existing TF-managed resourcesLeave in TerraformNo state surgery, no dual ownership
New app-scoped resourcesController (greenfield)Born under the new model, clean lifecycle
Foundation (VPC, DNS, base IAM, shared keys)Terraform, untouchedPerimeter unchanged

Rule of thumb: greenfield-first is a safety property, not just a scoping convenience — with nothing migrated, the worst failure mode of the boundary (two owners on one object) is impossible during the riskiest, earliest phase.

Keeping the boundary honest over time

A clean split on day one erodes unless something enforces it, because every tempting shortcut involves the self-service layer reaching across the line. The defence is layered, each layer independent so a gap in one is caught by the next.

LayerEnforces
Claim schema validation (CEL on the XRD)Requested capabilities ⊆ allowlist; owner required; obvious escalations rejected
Composition templateOnly known capabilities expand; provider/account pinned, not claim-chosen
Admission policy (Kyverno)Reject raw resources that bypass the claim path; block perimeter-owned kinds
Cloud-side permission boundaryA hard ceiling the minted identities can never exceed, whatever the claim says

Eventually, "which side owns this?" should be answerable from an ownership registry rather than tribal memory — a machine-checkable list that the pre-production hardening turns from convention into a gate. The registry is the definition the type-split test compiles into. Rule of thumb: a boundary defended by one control is one bug from leaking; layer independent checks (schema, template, admission, cloud ceiling) so that no single failure lets the self-service plane claim or grant across the line.

Key takeaways

  • Two continuously-reconciling systems make disjoint object ownership the only stable configuration; the type-split is that invariant.
  • The classification is one axis — lifecycle plus sharing: single-app-and-dies-with-it → controller; shared/perimeter/long-lived → Terraform; ambiguous → default to foundation and raise it.
  • The hard case is the cross-boundary grant: the controller owns the app-side principal, Terraform issues the grant on its own foundation object — never the reverse.
  • Greenfield-first makes boundary leakage impossible during the pilot because nothing is migrated and no object has two owners.
  • Keep the boundary honest with layered enforcement (schema, composition, admission, cloud permission boundary) and, eventually, a machine-checkable ownership registry.

Checklist

  • [ ] I can explain why overlapping ownership causes a reconcile war and why disjoint ownership is the only stable state.
  • [ ] I can apply the lifecycle-plus-sharing test to a new resource and say what to do when the answer is ambiguous.
  • [ ] I can describe a cross-boundary grant and split it correctly: app-side principal to the controller, grant-on-foundation to Terraform.
  • [ ] I can explain why greenfield-first removes the worst boundary failure mode during the pilot.
  • [ ] I can list the layered enforcement controls and explain why one control is not enough to keep the boundary honest.