Choosing the Control Plane: Crossplane vs KCC vs ACK

Teams need one portable claim API while platform engineers retain cloud-specific implementation detail. The decision is not “which controller is best”; it is whether the product boundary needs composition across AWS and GCP or can expose provider-native resources directly.

All candidates run as Kubernetes controllers, but their API surface and composition layer differ. etcd stores desired and observed Kubernetes state; cloud APIs remain authoritative for cloud resources; no application database or cache is introduced.

The same delivery chain exists regardless of controller; the implementation difference is where cloud variation is hidden. A portable claim adds a composition compute layer, not a second source of truth.

spec:
  parameters: { serviceAccount: exporter, access: [object-read] }
  compositionSelector:
    matchLabels: { platform.example.com/cloud: aws }

Declared intent: choose the AWS implementation without changing the consumer API. Interpreter: Crossplane composition selection and provider controller. Software effect: AWS-specific managed-resource objects are emitted to etcd. Hardware effect: controllers consume cluster CPU/RAM; the provider creates IAM records in its control-plane storage. Evidence: composition revision, managed-resource Ready condition, and CloudTrail agree.

It is tempting to choose the "most production-ready" tool, but every one of these is GA and widely run. A generic "production-ready" badge launders over the fit questions that actually decide the outcome: how each behaves under your GitOps setup, your cloud spread, and your ownership boundary. So the useful comparison re-tests each claim against the target stack — two clouds (a GKE side and an EKS side), a GitOps app-of-apps, and a hard split where no cloud object is owned by both Terraform and the controller.

Rule of thumb: never let "X is production-ready" end a tool decision — re-test the specific claims that matter (deletion, drift, identity, GitOps health, topology) against your own stack, because generic maturity and fit-for-you are different questions.

This is the highest-stakes dimension, because all three tools delete the live cloud resource by default when the custom resource is deleted. Under a GitOps app-of-apps, an accidental prune, a mis-scoped Application, or a stray kubectl delete can cascade into real cloud deletion. What differs is where you can set a safe default.

All three run a reconcile loop that continuously corrects drift. That is a feature — until a Terraform-owned object and a controller-owned object are ever the same object, at which point the two systems enter a reconcile war, each reverting the other forever. This is why the hard type-split from Day 01 is not a nice-to-have but a precondition: keep the domains disjoint and the fight cannot start. (Under a greenfield pilot with no migration, it structurally cannot occur — there are no shared objects yet.)

Rule of thumb: overlapping ownership between an IaC tool and an in-cluster controller is not a bug to fix later, it is a state to make impossible by construction — disjoint object ownership is the only stable configuration.

Here the tools diverge structurally. Crossplane runs one control plane spanning both clouds (with per-cloud provider packages underneath). KCC is one controller for all of GCP — but GKE-only. ACK is one controller per AWS service: IAM and object storage are separate deployments, charts, and identities, so the AWS side is "1 + N", not one.

The controller holds the credential that mints cloud IAM, so the controller's own identity is the real blast radius — more than any resource it manages. KCC's namespaced mode is strongest here (a per-namespace identity, scopable with IAM Conditions). ACK scopes per service controller, but its IAM controller is the dangerous one — without a permission boundary it can mint roles more privileged than itself. Crossplane is weakest by default (one identity per provider), so scoping leans on multiple provider configs plus an AWS permission boundary.

Rule of thumb: evaluate the minting identity before the minted resources — a self-service IAM system is only as safe as the ceiling on the controller that issues the grants, and that ceiling (permission boundary, IAM Conditions, allowlist) is mandatory, not optional.

None of the three sync cleanly out of the box; each needs custom health and diff handling or it reports false-green on cloud failures. Crossplane's managed resources may need custom health wiring (and, depending on version, a tracking-method change). KCC and ACK both write state back into the spec, creating a permanent diff that needs per-type ignoreDifferences. All three benefit from custom health logic so a failed cloud operation shows red, not green.

Rule of thumb: assume any in-cluster provisioner is silently healthy until proven otherwise — budget for custom health and diff suppression as part of adoption, and test the failure path (does a broken cloud op actually show unhealthy?) before you trust the dashboards.

Now the six dimensions resolve into a choice, and it hinges on one premise. If you are not building a unified, portable claim API, a hard type-split gains little from a single control plane, and per-cloud tools fit better — KCC's namespaced identity is the cleanest app-scoped fit on GCP, and cloud-isolated controllers keep blast radius per-cloud. But if you are building Compositions — one platform-authored claim surface that app teams use identically on both clouds — then Crossplane's abstraction layer is the product, and neither KCC nor ACK can offer a single claim API across both clouds.

A tool-first selection can produce two cloud-specific APIs and several controller surfaces even when teams need one portable request, increasing operating work and exposing cloud details to every service team. The target state begins with the product requirement—a single reviewed claim—and selects Crossplane because its Composition layer can preserve that interface across both clouds.

The comparison does not claim that the provider permissions become identical: the right side standardises the team's request while each Composition still renders the correct cloud-native grant. If a unified claim is no longer required, the decision can legitimately return to the per-cloud path on the left.

• All three tools are GA; the decision is about fit to your stack and goal, not maturity — re-test each production-ready claim against your clouds, GitOps, and ownership boundary.
• Deletion defaults to destroy on every tool; differences are only in where a safe retain default can live. Pair with Prune=false + finalizers regardless.
• Continuous drift correction makes a disjoint type-split with Terraform a precondition — overlap causes an unwinnable reconcile war.
• Topology differs structurally: Crossplane is one control plane (concentrated blast radius); KCC/ACK are per-cloud and, for ACK, per-service (1+N surfaces).
• The controller's own identity is the real blast radius; a permission boundary / allowlist on the minting identity is mandatory.
• The decision is premise-dependent: build a unified cross-cloud claim API → Crossplane; don't → per-cloud KCC+ACK fit better. Here the goal is the unified API, so Crossplane v2.

• [ ] I can explain why "production-ready" is the wrong axis and name the six dimensions that actually decide the tool.
• [ ] I can state the default deletion behaviour of all three tools and the mitigation that does not depend on the tool default.
• [ ] I can explain why continuous drift correction makes a disjoint Terraform boundary a precondition, not a nicety.
• [ ] I can contrast one-control-plane vs 1+N topology and say what each does to blast radius and operational load.
• [ ] I can state the premise (unified claim API or not) that flips the decision, and justify choosing Crossplane under that premise.