The API Gateway Pattern

As a system grows from one application into many small services, every client — web app, mobile app, third-party integration — suddenly faces a crowd of backend services, each with its own address, its own auth handling, its own quirks. An API gateway is a single server that sits in front of all those services and acts as the one front door clients talk to. Clients call the gateway; the gateway forwards each call to the right service and returns the response.

The problem it solves is twofold. First, without a gateway every client must know the address of every service and be updated whenever services split, move, or multiply — a brittle coupling. Second, concerns that every service needs — checking authentication, enforcing rate limits, terminating encryption, emitting logs — would otherwise be re-implemented in each service, inconsistently. The gateway gives clients one stable address and pulls those shared concerns into one place. The next sections are the list of jobs it takes on.

A cross-cutting concern is a responsibility that nearly every service shares but that is not any single service's business logic — authentication, rate limiting, logging, and so on. The gateway's core value is hosting these concerns once, at the edge, so the services behind it can focus purely on their domain.

Think of the gateway as a checkpoint every request passes through. By the time a request reaches a service, it has already been decrypted, its caller has been authenticated, its rate has been checked, and it has been logged — so the service receives a clean, trusted, already-vetted request. Move a concern into the gateway and you delete it from a dozen services; leave it in the services and you maintain a dozen copies. The following sections walk each responsibility in turn.

The gateway's most basic job is routing: deciding which backend service should handle each incoming request and forwarding it there. Routing is usually driven by the request path or host — /users/* goes to the user service, /orders/* to the order service — so clients address logical paths, not physical service locations.

Routing decouples clients from topology. When the order service is split into an order service and a fulfillment service, only the gateway's routing rules change; clients keep calling /orders/* exactly as before. Services can be moved, scaled, or renamed behind the gateway without any client ever noticing.

Protocol translation is a close cousin: the gateway can speak one protocol to clients and a different one to services. A common shape is accepting REST/JSON over HTTP/1.1 from browsers at the front while calling internal services over gRPC (the gRPC day) at the back — the gateway translates between them.

This lets you keep a clean, browser-friendly public API while enjoying a fast binary protocol internally — the exact split the gRPC day recommended, made real by the gateway.

Three security-related jobs concentrate naturally at the gateway because doing them once at the edge is both safer and simpler than scattering them.

TLS termination means the gateway is where the encrypted HTTPS connection ends: it holds the TLS certificate, decrypts incoming traffic, and passes plain (or re-encrypted) requests to services on the trusted internal network. Certificates live in one place instead of on every service, and services skip the cost of decryption.

Authentication (authn) answers "who is this caller?" The gateway verifies the credential on each request — validating a JWT's signature, checking an API key, confirming a session — once, at the door. It then passes the verified identity (say, a user id in a trusted header) to the services, which no longer each need to re-validate raw tokens.

Authorization (authz) answers "is this caller allowed to do this?" The gateway can enforce coarse-grained rules — this API key may only reach /public/*, that role may not call admin routes — while services keep the fine-grained, domain-specific permission checks.

The division of labor to remember: the gateway handles decryption, identity verification, and coarse route-level permissions; each service still owns the fine-grained rules that only it understands (can this user edit this document?).

Two more concerns belong at the edge because the gateway is the one place that sees all traffic.

Rate limiting caps how many requests a client may make in a time window — for example 1000 requests per minute per API key. Enforced at the gateway, it protects every service behind it at once from a misbehaving or abusive client, and returns a fast 429 Too Many Requests when the limit is exceeded instead of letting the flood reach the services. Because all of a client's traffic passes through the one gateway, it is the natural counting point.

Observability is the umbrella for logging, metrics, and tracing. Since every request crosses the gateway, it is the ideal spot to emit a consistent access log, count requests and error rates, measure latency, and stamp each request with a trace id that follows it through every downstream service. This gives you one uniform view of all traffic without instrumenting each service identically.

Sometimes one client screen needs data from several services, and making the client call each one is slow and chatty — especially over a mobile network. Request aggregation (also called a composition or scatter-gather) lets the gateway make several backend calls on the client's behalf and combine the results into one response.

The client makes a single request; the gateway fans out to the user, orders, and recommendations services in parallel, waits for them, and stitches the results into one payload. This cuts client round trips from three to one and hides the internal service layout. Note the overlap with GraphQL (the GraphQL day), which offers aggregation as a query language — a GraphQL server is often deployed as the aggregating gateway. The trade-off: aggregation moves latency and failure handling into the gateway (what if one of the three calls fails or is slow?), so it must decide whether to return partial data or fail the whole call.

These four terms overlap and get confused, so it is worth drawing the lines. They sit at different layers and answer different questions; a real system often uses several together.

The distinctions in plain terms: a load balancer spreads requests across identical copies of one service to share load and route around dead instances — it does not care what the request means. A reverse proxy is the general category of "a server that forwards client requests to backends" (nginx in its classic role); an API gateway is a specialized, application-aware reverse proxy that adds auth, rate limiting, and aggregation. An API gateway governs north-south traffic — clients coming into the system from outside. A service mesh (the Envoy/Kong day) governs east-west traffic — services calling each other inside the system — handling retries, mTLS, and routing between them. Rule of thumb: gateway at the edge for incoming client traffic, mesh inside for service-to-service traffic, load balancing happening within both.

A single gateway serving every kind of client tends to bloat, because a web app, an iOS app, and a smart TV each want differently-shaped responses. The backend-for-frontend (BFF) pattern responds by giving each frontend its own dedicated gateway, tailored to that client's needs.

Each BFF aggregates and shapes data specifically for its client — the mobile BFF returns lean payloads suited to a small screen and slow network, the web BFF returns richer data — while all of them sit on top of the same shared services. The benefit is that each frontend team owns its own gateway and can move fast without a shared gateway becoming a bottleneck or a lowest-common-denominator compromise. The cost is more gateways to build and operate, and some duplicated logic across them. Rule of thumb: reach for BFFs when different clients genuinely need different response shapes and are owned by different teams; a single gateway is simpler when clients are similar.

Concentrating so much at one front door creates one obvious danger: if the gateway goes down, everything behind it is unreachable, even though the services themselves are healthy. The gateway is a single point of failure (SPOF) and a potential bottleneck — every request pays its overhead, so a slow gateway slows the whole system.

The mitigations are the standard tools for any critical component:

• Run several gateway instances behind a load balancer, so no single instance is a SPOF and traffic spreads across them.
• Scale it horizontally and keep it stateless (push session state elsewhere) so you can add instances freely under load.
• Keep it thin. Every feature you cram into the gateway adds latency to every request and another thing that can break. Resist turning the gateway into a place for business logic.
• Fail gracefully. Time out slow backends, return cached or partial responses where sensible, and shed load rather than collapsing.

The honest tension: the gateway's whole value is centralizing shared concerns, but centralization is exactly what makes it a SPOF and bottleneck. You accept that trade and pay for it with redundancy and discipline — multiple thin, stateless, well-monitored instances — rather than avoiding the pattern.

• An API gateway is the single front door in front of many services: clients get one stable address, and shared concerns move off the services into one place.
• Cross-cutting concerns — routing, auth, rate limiting, TLS termination, logging/metrics/tracing — belong at the gateway so services stay focused on business logic.
• Routing decouples clients from service topology; protocol translation lets the gateway speak REST/JSON to clients and gRPC to services.
• The gateway does TLS termination, authenticates the caller once, and enforces coarse route-level authorization, while services keep fine-grained domain permissions.
• Rate limiting and observability belong at the gateway because it is the one place that sees all traffic.
• Request aggregation lets the gateway fan out to several services and return one combined response, cutting client round trips (GraphQL is one way to do this).
• A gateway (north-south, per-API concerns) differs from a load balancer (spread across replicas), a reverse proxy (general forwarding), and a service mesh (east-west, service-to-service).
• The BFF pattern gives each frontend its own tailored gateway; the cost of any gateway is that it is a SPOF and bottleneck, mitigated with multiple thin, stateless, redundant instances.

• [ ] I can explain the two problems an API gateway solves: one stable entry point, and shared concerns pulled off the services.
• [ ] I can list the gateway's core responsibilities (routing, auth/authz, rate limiting, TLS termination, aggregation, observability, protocol translation).
• [ ] I can explain how routing decouples clients from service topology.
• [ ] I understand the auth division of labor: gateway does authn + coarse authz, services do fine-grained authz.
• [ ] I can describe request aggregation and its latency/failure trade-off.
• [ ] I can distinguish an API gateway from a load balancer, a reverse proxy, and a service mesh.
• [ ] I can explain the BFF pattern and when it is worth the extra gateways.
• [ ] I understand why the gateway is a SPOF/bottleneck and how to mitigate it (redundant, thin, stateless instances).