The Components in One Map

Enterprise problem: An operator who cannot say which component owns each telemetry transition will route data through overlapping tools, hide failure boundaries, and lengthen incidents for customers.

Whole-course context: The incoming evidence is a four-signal design with collection, storage, query, rules, and notification responsibilities already introduced; this day turns those parts into one reviewable component contract.

Today’s slice: We map application instrumentation, exporters, collectors, specialized backends, Grafana, rulers, Alertmanager, and object storage without treating any product name as the customer outcome.

End-of-day evidence: A component matrix and two traced journeys show the producer, consumer, owner, protocol, and terminal evidence at every boundary.

Still unsolved: Deployment scale, signal economics, schema governance, and privacy policy remain deliberately outside today’s component map.

A component map is useful only when it explains how raw system behavior becomes evidence an operator can act on. Without that chain, a box inventory hides duplicate ownership and gaps between tools.

Thesis: The smallest complete component map is producer → evidence path → owned action; product names matter only after every transition has one accountable owner. Why this matters: during an incident, the team must identify the failing handoff without guessing which adjacent tool was supposed to do the work.

The boundary here includes evidence creation, collection, storage, query, recognition, notification, and action. It excludes deployment sizing and signal economics, which require the ownership map but do not change its smallest shape.

Expand the middle box in dependency order: first create domain evidence, then protect and route it, then preserve its signal-specific shape, and only then correlate it for a decision. Each added noun refines the same producer-to-action path.

When similar tools appear adjacent, teams can accidentally assign the same job twice and lose the actual failure boundary. The decisive question is what each component consumes, how it transforms that input, and what durable interface it exposes.

The general rule is to follow the evidence transition that failed, not to search every component independently. A simple example is a metric query that succeeds while its exemplar trace lookup fails: the metric store is healthy, so the missing correlation reference or trace boundary is the first suspect.

In a recurring checkout incident, the on-call starts with an elevated error-rate metric, opens an exemplar in Tempo, follows the trace context into Loki logs, and identifies the checkout service owner. The observed evidence should include the metric query, trace ID, related log query, component-map revision, tenant, and investigation run ID. If the trace is absent, the operator checks whether the application emitted context, whether the collector accepted and routed spans, and whether Tempo stored them—in that dependency order. A successful metric query remains the positive control while an invalid trace ID proves the failure path is distinguishable from an authorization or query outage.

This is reusable beyond checkout: preserve one healthy transition, locate the first missing output, and inspect only the producer-consumer contract that should have created it.

The main failure mode is an arrow labelled only “connects to”; it hides protocol, authentication, retry, tenancy, buffering, evidence, and ownership. The central trade-off is map simplicity versus diagnostic precision: too few boundaries conceal responsibility, while too many implementation boxes make the map stale and hard to operate.

represent a boundary when it changes accountable ownership, protocol, durable state, tenant enforcement, or observable failure semantics; otherwise keep it inside the owning component and document it at the lower-level runbook.

Use this boundary protocol for every represented edge:

1. Name the producer, consumer, protocol, direction, and accountable owner.
2. State whether the edge is push, pull, or query, and where buffering occurs.
3. Record tenant identity and authentication independently at collection and query boundaries.
4. Inject one invalid credential and one unavailable consumer; observe rejection, retry, drop, or backpressure.
5. Preserve the positive control: another authorized tenant and healthy edge still work.
6. Falsify the map if any observed hop, durable store, or operator action lacks a represented edge.

Use a product-level map when several teams need a shared incident contract. Avoid using it as a deployment manifest: replica counts and internal workers belong in topology evidence unless they create a distinct failure domain or owner.

A component map earns its maintenance cost only when it changes an operational decision and the result is measured. Apply Observe → Interpret → Decide → Act → Measure to one bounded edge at a time.

Observe a failed checkout trace link while the metric control succeeds. Interpret that the break lies between context creation, span routing, and trace storage. Decide which single owned boundary to probe first. Act by restoring the last passing collector route or instrumentation revision, not by changing every backend. Measure the effect by repeating the same trace lookup and requiring both the valid trace to appear and an invalid trace ID to remain a clean negative result within the agreed recovery window. If either probe fails, the action did not repair the component contract.

A memorable product list is not an architecture; the useful map follows evidence from behavior to accountable action.

• Applications and exporters create or translate evidence; collectors protect and route it.
• Prometheus, Mimir, Loki, Tempo, and Pyroscope retain and query different signal shapes.
• Grafana correlates signals, rulers detect conditions, and Alertmanager controls notification delivery.
• Every edge needs an owner, protocol, tenancy rule, and falsifiable failure observation.

An incomplete review creates false confidence, so complete these checks before accepting the component map.

• [ ] Every producer and consumer has one accountable owner.
• [ ] Every arrow names protocol, direction, tenant identity, and failure behavior.
• [ ] Application-owned business semantics are not delegated to infrastructure.
• [ ] The metric-to-trace-to-log journey has positive and negative evidence.
• [ ] Object storage retention and deletion remain owned by each backend.

Product responsibilities change, so these current claims were checked against official primary documentation.

• Grafana Alloy: how collection, processing, and backend routing work
• OpenTelemetry Collector processors
• Prometheus security model and component endpoint assumptions