Kustomize

This course is built around the public HelixWorks Kubernetes Lab monorepo. The excerpt below is runnable source, not pseudocode.

Source: gitops/apps/helixworks/overlays/dev/kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - ../../base
replicas:
  - name: helixworks
    count: 3

Code to reality

Declared intent

Reuse one workload base while changing only the development replica policy.

Interpreter

Kustomize loads the base and applies the declarative replica transformer before submission.

Software effect

The rendered development Deployment requests three replicas without copying the base manifest.

Hardware effect

Kubernetes may schedule one additional container, increasing node CPU and memory consumption.

Observable evidence

kubectl kustomize shows replicas: 3 and kubectl rollout status proves all three became ready.

You want one set of manifests reused across dev/staging/prod, changing only replicas, image tags, or a namespace per environment. Helm solves this with templates; Kustomize solves it without templates — you keep plain, valid YAML and layer environment-specific patches on top of a shared base.

Solution: Keep canonical manifests in a base, then add thin overlays per environment that patch only what differs, and build/apply with kubectl -k.

An architect keeps one base building blueprint and, rather than redrawing it for every location, tapes small change notes onto it per site, taller ceilings here, a different door there. The base drawing is never altered; at each site the notes are merged onto a fresh copy to produce the final drawing handed to the builders. Kustomize does exactly this: a base holds the canonical manifests, thin overlays patch only what differs per environment with no templating, and kubectl apply -k merges them into final manifests sent to the apiserver.

Both tools customize manifests per environment, but they take opposite approaches. This table contrasts where each one fits:

Same app (an nginx Deployment), same four edits, done each way. Helm parameterizes a template and feeds it values; Kustomize patches plain, valid YAML in an overlay. Read each pair top-to-bottom: Helm first, Kustomize second.

1. Set replicas per environment

# HELM — templates/deployment.yaml + values-prod.yaml
spec:
  replicas: {{ .Values.replicaCount }}   # template
---
replicaCount: 5                          # values-prod.yaml

# KUSTOMIZE — overlays/prod/kustomization.yaml
replicas:
  - name: web
    count: 5

2. Override the image tag

# HELM — templates/deployment.yaml + values-prod.yaml
image: "{{ .Values.image.repo }}:{{ .Values.image.tag }}"   # template
---
image: { repo: nginx, tag: "1.27.2" }                       # values-prod.yaml

# KUSTOMIZE — overlays/prod/kustomization.yaml
images:
  - name: nginx
    newTag: "1.27.2"

3. Add resource limits

# HELM — templates/deployment.yaml (guarded block) + values-prod.yaml
        resources:
          {{- toYaml .Values.resources | nindent 10 }}   # template
---
resources:                                               # values-prod.yaml
  limits: { cpu: "500m", memory: 256Mi }

# KUSTOMIZE — overlays/prod/resources-patch.yaml (strategic merge)
apiVersion: apps/v1
kind: Deployment
metadata: { name: web }
spec:
  template:
    spec:
      containers:
        - name: web
          resources:
            limits: { cpu: "500m", memory: 256Mi }

4. Inject a config value

# HELM — templates/configmap.yaml + values-prod.yaml
data:
  LOG_LEVEL: {{ .Values.logLevel | quote }}   # template
---
logLevel: info                                # values-prod.yaml

# KUSTOMIZE — overlays/prod/kustomization.yaml
configMapGenerator:
  - name: app-config
    literals:
      - LOG_LEVEL=info        # gets a content-hash suffix -> pods roll on change

Apply + lifecycle — the biggest practical difference:

# HELM: tracked releases, real upgrade/rollback history
helm upgrade --install web ./chart -f values-prod.yaml
helm rollback web 1          # revert to revision 1
helm uninstall web

# KUSTOMIZE: no release object — just render and apply
kubectl apply -k overlays/prod
kubectl delete -k overlays/prod
# "rollback" = re-apply the previous Git commit (or use Argo CD); Kustomize itself keeps no history

Same outcome on the cluster; different philosophy: Helm hides YAML behind a templating language and owns a release lifecycle, Kustomize leaves YAML intact and owns nothing but the merge.

Kustomize projects follow a base-plus-overlays tree, one folder per environment. A typical layout looks like this:

The base holds the canonical, environment-agnostic manifests that every overlay reuses. Its kustomization.yaml lists the resources and any shared labels:

# base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - deployment.yaml
  - service.yaml
commonLabels:
  app: web

# base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 1
  selector:
    matchLabels: { app: web }
  template:
    metadata:
      labels: { app: web }
    spec:
      containers:
        - name: web
          image: nginx:1.27

Overlays reference the base and apply built-in transformers like namespace, namePrefix, images, and replicas — no patch file needed for these.

# overlays/prod/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: prod
namePrefix: prod-
resources:
  - ../../base
images:
  - name: nginx
    newTag: "1.27.2"
replicas:
  - name: web
    count: 5
patches:
  - path: replicas-patch.yaml          # strategic merge patch
configMapGenerator:
  - name: app-config
    literals:
      - LOG_LEVEL=info

A strategic merge patch is a partial manifest matched by name/kind; a JSON 6902 patch is a precise operation list against a path.

# overlays/prod/replicas-patch.yaml  (strategic merge)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  template:
    spec:
      containers:
        - name: web
          resources:
            limits: { cpu: "500m", memory: 256Mi }

# inline JSON 6902 patch
patches:
  - target:
      kind: Deployment
      name: web
    patch: |-
      - op: replace
        path: /spec/replicas
        value: 8
      - op: add
        path: /spec/template/spec/containers/0/env
        value:
          - name: TIER
            value: prod

Once a base and overlay exist, these commands render the merged result or apply it straight to the cluster:

# render the final manifests (no apply) — inspect before shipping
kubectl kustomize overlays/prod

# build AND apply in one step
kubectl apply -k overlays/prod
kubectl delete -k overlays/prod

# the standalone CLI works too
kustomize build overlays/prod | kubectl apply -f -

This shows exactly what kubectl apply -k does behind the scenes, in order. It reads the overlay, pulls in the base it points to, then layers the edits on top, the built-in transformers (like namespace and image tag), then any patches, then generators that build ConfigMaps, before emitting one set of merged manifests and sending them to the api-server. Follow the diagram top to bottom.

Graph legend — each node is a real stage of kubectl apply -k:

A complete tree: a shared base plus a dev overlay (1 replica, dev image) and a prod overlay (5 replicas, pinned image, resource limits via a strategic-merge patch), built and applied with kubectl apply -k.

1. Lay out the files:

2. The base (base/):

# base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - deployment.yaml
  - service.yaml
commonLabels:
  app: web

# base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 1
  selector:
    matchLabels: { app: web }
  template:
    metadata:
      labels: { app: web }
    spec:
      containers:
        - name: web
          image: nginx
          ports: [{ containerPort: 80 }]

# base/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: web
spec:
  selector: { app: web }
  ports: [{ port: 80, targetPort: 80 }]

3. The dev overlay: its own namespace, an unstable image tag, 1 replica:

# overlays/dev/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: dev
namePrefix: dev-
resources:
  - ../../base
images:
  - name: nginx
    newTag: "1.27-alpine"
replicas:
  - name: web
    count: 1

4. The prod overlay: pinned image, 5 replicas, plus a strategic-merge patch adding resource limits:

# overlays/prod/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: prod
namePrefix: prod-
resources:
  - ../../base
images:
  - name: nginx
    newTag: "1.27.2"
replicas:
  - name: web
    count: 5
patches:
  - path: resources-patch.yaml

# overlays/prod/resources-patch.yaml  (strategic merge)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  template:
    spec:
      containers:
        - name: web
          resources:
            limits: { cpu: "500m", memory: 256Mi }

5. Render each overlay and confirm only the intended fields differ:

kubectl kustomize overlays/dev | grep -E "name:|namespace:|replicas:|image:"
# expected: name: dev-web, namespace: dev, replicas: 1, image: nginx:1.27-alpine

kubectl kustomize overlays/prod | grep -E "name:|namespace:|replicas:|image:|cpu:"
# expected: name: prod-web, namespace: prod, replicas: 5, image: nginx:1.27.2, cpu: "500m"

6. Apply prod and verify the live objects:

kubectl create namespace prod
kubectl apply -k overlays/prod
kubectl -n prod get deploy prod-web -o jsonpath='{.spec.replicas}{"\n"}'   # 5
kubectl -n prod get deploy prod-web -o jsonpath='{..limits.cpu}{"\n"}'      # 500m

7. Change one field in the base (image only) and re-apply both overlays to prove reuse:

# bump base image once; both overlays inherit it unless they override the tag
sed -i 's/image: nginx/image: nginx/' base/deployment.yaml
kubectl apply -k overlays/dev
kubectl apply -k overlays/prod
# dev still pins 1.27-alpine, prod still pins 1.27.2 -> overlay tags win

Graph legend — each node is a real file or command from the worked tree above:

These are the mistakes that trip people up most with Kustomize:

   - edited base for one env       -> defeats the point; patch in the overlay instead
   - generator name changed        -> ConfigMap/Secret names get a content hash suffix
   - JSON6902 path wrong           -> /spec/replicas not /spec/Replicas; arrays are 0-indexed
   - patch didn't match            -> name/kind in the patch must match the target resource
   - wrong apiVersion in kust file -> use kustomize.config.k8s.io/v1beta1
   - expecting rollback            -> Kustomize has none; it's just apply (use Helm/Argo for that)

• Kustomize is template-free and built into kubectl (apply -k).
• A base holds canonical manifests; overlays patch per environment.
• Built-in transformers (namespace, namePrefix, images, replicas) cover common edits.
• Strategic merge patches by field; JSON 6902 patches by precise path/op.
• Generators add a content hash suffix so pods roll on config change.
• kubectl kustomize previews the merged YAML before you apply it.

• [ ] Built a base with resources and commonLabels
• [ ] Wrote a prod overlay referencing the base with namespace/images/replicas
• [ ] Added a strategic merge patch and a JSON 6902 patch
• [ ] Used a configMapGenerator and saw the hashed name
• [ ] Rendered with kubectl kustomize and applied with kubectl apply -k