Ingress Controllers and Ingress Resources

Exposing every Service as its own LoadBalancer is expensive (one cloud LB each) and gives you no shared routing, TLS, or host/path rules. Ingress lets one entry point route external HTTP(S) traffic to many Services by hostname and path, with TLS termination — using a single load balancer.

Solution: Install an ingress controller and define Ingress rules to route external HTTP(S) by host/path to many Services through one load balancer, terminating TLS.

Every truck rolling into the port hits one main gate, where a customs officer reads the truck's destination address and waves it down the road to the correct pier. One gate serves the whole port, so you do not build a separate entrance per pier. A Kubernetes Ingress controller is that gate and officer combined, the Ingress host and path rule is the address the officer routes by, and each Service is the pier the traffic is finally sent to.

The same cluster entities appear in every day's notes; the diagram below shows where this day's topic fits.

The Ingress resource is just declarative intent — a record in etcd. The Ingress controller is a running program with two halves: a control plane that watches the Ingress/Service/Secret APIs, and a data plane (the actual proxy) it reconfigures on every change.

   Ingress RESOURCE    = the rules (host/path -> Service). Just YAML; does nothing alone.
   Ingress CONTROLLER  = controller process (watches API) + data-plane proxy (moves bytes)
                         nginx, Kong, Envoy/Contour, Traefik, HAProxy, or a cloud controller.

An Ingress resource with no controller installed does nothing. You must install a controller first, and tie the resource to it with ingressClassName.

Graph legend — the two halves of an ingress-nginx controller pod:

This walks the full journey of one external HTTPS request, from the public DNS name down to a pod, so you can see where each piece (DNS, the cloud load balancer, the controller's watch loop, TLS, and routing) does its part. The numbered steps below trace that path in order.

1. DNS: shop.boutique.example.com resolves to the public IP/hostname of the controller's LoadBalancer (or a NodePort + external LB).
2. L4 entry: the cloud LB forwards TCP 80/443 to the controller pods.
3. Watch loop: the controller's informer has already turned every Ingress, Service, EndpointSlice, and TLS Secret into live proxy configuration.
4. TLS termination: the proxy presents the cert from the referenced Secret and decrypts the request (unless passthrough is configured).
5. Routing decision: it matches Host header + request path against the compiled rules to pick a backend.
6. Upstream selection: most modern controllers route directly to pod IPs from EndpointSlices (bypassing kube-proxy), load-balancing across ready pods.
7. Response: the proxy streams the response back, optionally re-encrypting.

Graph legend — a request reaching the boutique frontend through ingress-nginx:

A bare Ingress resource does nothing until a controller is running, so the first step is to install one. This applies the ingress-nginx controller and then checks that its pods, its public Service, and its IngressClass came up.

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.11.0/deploy/static/provider/cloud/deploy.yaml
kubectl -n ingress-nginx get pods,svc      # the controller + its LoadBalancer Service
kubectl get ingressclass                   # the "nginx" class this controller owns

The controller's own Service (type LoadBalancer or NodePort) is the single public entry point; everything else routes through it.

This is the routing rule itself: it ties the nginx controller to a hostname, terminates TLS with a Secret, and sends two URL paths to two different backend Services. The backends are real Online Boutique microservices (cartservice, frontend) — Google's widely-used demo app — so the example mirrors a real storefront instead of placeholders.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: boutique
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  tls:
    - hosts: ["shop.boutique.example.com"]
      secretName: boutique-tls       # a TLS Secret (cert + key)
  rules:
    - host: shop.boutique.example.com
      http:
        paths:
          - path: /cart
            pathType: Prefix
            backend:
              service:
                name: cartservice     # Online Boutique cart microservice
                port: { number: 7070 }
          - path: /
            pathType: Prefix
            backend:
              service:
                name: frontend        # Online Boutique storefront
                port: { number: 80 }

The diagram below traces a single HTTPS request as it travels from the client through the ingress controller (where TLS is decrypted and the host/path rule is matched) to the right Service and finally a pod.

One LB plus one ingress controller fans out to many Services by host and path.

Every controller fulfills the same Ingress spec but wraps a different data-plane proxy and a different config model (annotations vs CRDs vs Gateway API).

• ingress-nginx: the reference, CNCF-maintained NGINX controller. Templates nginx.conf and reloads on change; behavior tuned via nginx.ingress.* annotations. Note: distinct from F5's commercial NGINX Ingress Controller.
• Kong Ingress Controller: data plane is Kong Gateway (historically NGINX+Lua/OpenResty). Adds a rich plugin system (auth, rate-limit, transforms) via CRDs like KongPlugin, plus Gateway API support.
• Contour: a control plane that programs Envoy over xDS. Uses the HTTPProxy CRD for advanced routing, plus Gateway API.
• Emissary-ingress (formerly Ambassador): also Envoy-based, configured by Mapping CRDs; API-gateway oriented.
• Traefik v3: a Go-native proxy with dynamic configuration (no config reload). Uses IngressRoute CRDs and "middlewares"; strong Gateway API and Let's Encrypt integration.
• HAProxy Ingress / HAProxy Kubernetes Ingress Controller: HAProxy data plane known for high performance and fine-grained L4/L7 control.
• Cloud controllers — AWS Load Balancer Controller (provisions an ALB) and GKE Ingress (provisions a Google Cloud HTTP(S) LB): there is no in-cluster proxy; routing runs on the cloud's managed L7 load balancer.

Comparison table

Graph legend — each branch is how a real controller turns rules into proxy config:

Goal: route api.example.com/v1 to the api Service and app.example.com to the web Service, both over HTTPS.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: store
spec:
  ingressClassName: nginx
  tls:
    - hosts: ["api.example.com", "app.example.com"]
      secretName: store-tls
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /v1
            pathType: Prefix
            backend: { service: { name: api, port: { number: 8080 } } }
    - host: app.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend: { service: { name: web, port: { number: 80 } } }

How each realizes it:

• NGINX: renders two server { server_name ... } blocks, each with a location and an ssl_certificate from store-tls; reloads nginx.conf.
• Kong: creates Kong Routes/Services from the Ingress; the SNI cert is loaded into Kong; plugins can be attached per-route via KongPlugin.
• Contour (Envoy): produces Envoy VirtualHosts with domains and routes plus FilterChains for SNI, pushed to Envoy over xDS — equivalent CRD:

apiVersion: projectcontour.io/v1
kind: HTTPProxy
metadata:
  name: api
spec:
  virtualhost:
    fqdn: api.example.com
    tls: { secretName: store-tls }
  routes:
    - conditions: [{ prefix: /v1 }]
      services: [{ name: api, port: 8080 }]

Goal: send 10% of traffic to app-v2, keep 90% on app-v1, and cap abusive clients. Each controller expresses this differently.

NGINX (two Ingress objects; the canary one carries weight + a limit):

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: app-canary
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "10"
    nginx.ingress.kubernetes.io/limit-rps: "20"
spec:
  ingressClassName: nginx
  rules:
    - host: app.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend: { service: { name: app-v2, port: { number: 80 } } }

Kong (rate-limit as a plugin attached to the Ingress; weights via upstream targets):

apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: rate-limit
plugin: rate-limiting
config:
  minute: 100
  policy: local
---
# annotate the Ingress: konghq.com/plugins: rate-limit

Traefik v3 (weighted services + a RateLimit middleware via IngressRoute):

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: ratelimit
spec:
  rateLimit: { average: 100, burst: 50 }
---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: app
spec:
  routes:
    - match: Host(`app.example.com`)
      kind: Rule
      middlewares: [{ name: ratelimit }]
      services:
        - { name: app-v1, port: 80, weight: 90 }
        - { name: app-v2, port: 80, weight: 10 }

Graph legend — a canary split with rate limiting at the proxy:

The basic Ingress spec only covers host/path routing, so every controller adds its own way to express the advanced features (canary releases, rate limits, mTLS). Those ways fall into three styles — annotations, custom resources (CRDs), and the newer Gateway API — and this section contrasts them.

• Annotations (NGINX, HAProxy, cloud): quick, but untyped strings, vendor specific, and easy to misconfigure.
• CRDs (Kong, Contour HTTPProxy, Emissary Mapping, Traefik IngressRoute): typed, validated, expressive — but non-portable across controllers.
• Gateway API (gateway.networking.k8s.io): the successor to Ingress — a standard set of CRDs (GatewayClass, Gateway, HTTPRoute, GRPCRoute) with role separation (infra owns Gateway, app teams own routes) and built-in traffic splitting, header matching, and cross-namespace references. Most controllers above now implement it, converging the ecosystem.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: app
spec:
  parentRefs: [{ name: prod-gateway }]
  hostnames: ["app.example.com"]
  rules:
    - matches: [{ path: { type: PathPrefix, value: / } }]
      backendRefs:
        - { name: app-v1, port: 80, weight: 90 }
        - { name: app-v2, port: 80, weight: 10 }   # portable canary, no annotations

pathType decides how strictly a request URL must match a rule's path before it is sent to that backend. These are the matching modes you will reach for most often.

   pathType: Prefix   /api matches /api, /api/v1, /api/x   (most common)
   pathType: Exact    /api matches ONLY /api
   host-based         shop.example.com vs admin.example.com -> different Services
   default backend    requests matching no rule -> a fallback Service (e.g. 404 page)

Put the cert + key in a kubernetes.io/tls Secret and reference it in spec.tls. The controller terminates HTTPS and forwards plain HTTP to the Service (see Day 20 for re-encrypt/passthrough options). For mTLS, NGINX uses auth-tls-* annotations, while Envoy-based and Traefik controllers expose client cert validation in their CRDs.

kubectl create secret tls shop-tls --cert=shop.crt --key=shop.key

These are the failures people hit most with Ingress, each paired with what usually causes it so you can jump straight to the fix.

   - Ingress created but 404/no route   -> no controller installed, or wrong ingressClassName
   - TLS not working                    -> Secret missing/wrong type, or host mismatch
   - paths leak to wrong app            -> rewrite-target / pathType misconfigured
   - works in-cluster, not externally   -> the controller Service has no external IP
   - annotation ignored                 -> wrong prefix for the controller (nginx vs traefik vs kong)
   - cert-manager for real certs        -> automate Let's Encrypt instead of manual Secrets

• Ingress resource = routing rules; Ingress controller = control plane + data-plane proxy that enforce them.
• A resource without a controller does nothing — install one and set ingressClassName.
• Controllers differ mainly by data plane (NGINX, Envoy, Traefik, HAProxy, Kong) and config model (annotations vs CRDs vs Gateway API).
• Route by host + path; terminate TLS with a tls Secret; one LB for many apps.
• Advanced traffic management (canary, rate limit, mTLS) is vendor-specific today; Gateway API is converging it into a portable standard.
• Use cert-manager to automate certificates in production.

• [ ] Explained resource vs controller (control plane + data plane) and why both are needed
• [ ] Installed an ingress controller and found its external Service + IngressClass
• [ ] Wrote an Ingress with host + multiple path rules and TLS
• [ ] Compared at least three controllers by data plane and config model
• [ ] Expressed a canary + rate-limit on two different controllers
• [ ] Diagnosed a 404 (missing controller / wrong ingressClassName)