The Telemetry Schema Is an API

Enterprise problem: An unreviewed field rename can silently break dashboards, alerts, service-level objectives, automation, cost allocation, and security investigations even though the application deployment appears healthy.

Whole-course context: The incoming cost ledger identifies expensive dimensions and critical evidence; this day turns shared telemetry names, units, types, semantics, and deprecations into a versioned consumer contract.

Today’s slice: We govern service identity plus metric, log, and trace conventions, then prove compatible rollout and rejection paths.

End-of-day evidence: A schema revision, compatibility report, dual-read migration, consumer inventory, and rollback probe demonstrate that producers and consumers evolve safely.

Still unsolved: Data minimization, endpoint access, tenant isolation, and retention enforcement remain the next security-focused slice.

Schema governance matters only when it protects customer decisions, so these cases connect producer changes to live operational consumers.

Use case ID	Actor	Customer job	Success outcome	Denial or recovery evidence
D20-UC-01	Application owner	Publish a new service version without breaking telemetry consumers	Old and new producers remain queryable through a bounded migration window and all registered consumers pass	An incompatible field, type, unit, or label is rejected with named affected consumers
D20-UC-02	Observability platform steward	Deprecate a telemetry field safely	Usage reaches zero, replacement evidence is complete, and removal occurs after the published deadline	A remaining consumer blocks removal and rollback restores the prior schema revision

“Use consistent fields” cannot define done, so each actor needs machine-checkable acceptance and an observed failure path.

Story ID	Use case IDs	User story	Observable acceptance conditions
D20-US-01	D20-UC-01	As an application owner, I want schema compatibility checked before release, so that a healthy deployment cannot silently blind operations	Contract tests cover required fields, units, types, cardinality policy, and representative dashboard, alert, and trace queries
D20-US-02	D20-UC-02	As a platform steward, I want deprecation based on consumer evidence, so that fields disappear only after dependants migrate	Registry shows owners and last-seen use; removal is denied while one fixture still queries the old field

A schema document without producer and consumer gates drifts immediately. These flows begin at release actions and end with compatibility evidence.

Flow ID	Use case IDs	Path	Trigger	Numbered steps	Terminal evidence
D20-FLOW-01	D20-UC-01	Happy	Application owner proposes a telemetry schema revision	1. Registry validates names, types, units, required attributes, and forbidden dimensions.<br>2. Fixtures emit old and new versions.<br>3. Consumer contract queries run.<br>4. Dual-read or transform compatibility is checked.<br>5. Revision is published.	Actor, producer version, schema URL or revision, consumer versions, expected and observed queries, environment, timestamp, fixture hash, and run ID
D20-FLOW-02	D20-UC-02	Denied	Steward requests removal of a deprecated field	1. Registry checks deadline and consumer inventory.<br>2. One active consumer is detected.<br>3. Removal is denied.<br>4. Replacement field remains queryable as a positive control.<br>5. Owner and migration deadline are notified.	Field, remaining consumer, denial reason, replacement-query result, schema revision, timestamp, and immutable decision ID

If compatibility logic sits only in a collector, missing business meaning cannot be recovered after ingestion. The producer owns domain semantics, while the platform owns conventions, transforms, and consumer evidence.

Use case ID	Entry point	Responsible services	Authoritative store	Failure evidence
D20-UC-01	Release telemetry-contract check	Application instrumentation, schema registry, fixture emitter, collector compatibility layer, consumer test runner	Versioned schema registry owned by observability governance	Missing required attribute, changed type/unit, forbidden label, broken consumer query, or incompatible transform
D20-UC-02	Deprecation and removal action	Consumer inventory, usage scanner, schema registry, notification service	Deprecation registry owned by observability governance	Active consumer, deadline breach, missing replacement, or failed rollback query

An overwritten schema cannot explain an old incident or safely roll back, so revisions and consumer dependencies must be immutable and tenant-scoped. This slice creates no generated application database.

Generated-application database: Not created in this slice — telemetry schema and compatibility evidence are observability control-plane state.

Record or entity	Store and owner	Primary key	Foreign key or opaque reference	Tenant key	Material constraint	Lifecycle and deletion	Use case IDs
TelemetrySchemaRevision	Schema registry, observability governance owner	schema_revision_id	None — immutable root revision in a schema family	organization_id	Published revisions are immutable; field name, type, unit, meaning, allowed dimensions, and compatibility mode are required	Draft, publish, deprecate, retain permanently for interpretation, never mutate in place	D20-UC-01, D20-UC-02
ConsumerContract	Consumer registry, owning product team	consumer_contract_id	schema_revision_id local FK or schema-family opaque reference	organization_id	Owner, query fixtures, required fields, and last passing run are mandatory	Register, revise, deactivate after usage reaches zero, retain audit metadata	D20-UC-01, D20-UC-02
CompatibilityRun	Evidence store, release engineering owner	run_id	Opaque producer build, schema revision, and consumer fixture references	organization_id	Input fixture hash and expected/observed results are immutable	Retain through deprecation window plus audit period; expire bulky payloads separately	D20-UC-01, D20-UC-02

Shared identity breaks correlation when each signal uses a different service or environment name. Define canonical resource identity once and state signal-specific constraints separately.

Contract area	Required decisions
Shared identity	service.name, service.namespace, service.version, deployment.environment.name, region, cluster, owner, and tenant semantics
Metrics	Stable name and meaning, base unit, type, required and forbidden labels, histogram boundaries, owner, and deprecation policy
Logs	Timestamp, severity, event name, service identity, trace reference, error type, schema version, structured fields, and sensitive-data classification
Traces	Span naming, resource attributes, parent/context propagation, status/error semantics, business attributes, sampling, and sensitive-attribute classification

The schema is an API because dashboards, alerts, capacity models, service-level objective calculations, incident automation, cost allocation, deployment analysis, and security tools depend on it. OpenTelemetry’s stable schema specification exists precisely because source and consumer assumptions otherwise change together; a schema URL can identify an immutable version and describe supported transformations. It does not replace an organization’s complete field and value contract.

Infrastructure can capture mechanics such as HTTP status or CPU time. Only application code can state that a checkout failed, a policy was rejected, a model response was unsafe, a tenant boundary was violated, or a payment was duplicated. A collector may validate, redact, enrich, or perform a documented lossless transform; it must not invent that missing meaning.

Happy-path fixtures miss silent consumer breakage, so test old/new producers against real query shapes and a deliberately incompatible change.

1. Inventory dashboards, alerts, SLO rules, cost jobs, security queries, and automation with owners.
2. Freeze old and new producer fixtures plus consumer query fixtures.
3. Validate identity, field presence, type, unit, histogram boundaries, and cardinality policy.
4. Run consumers against old, dual-emitted or transformed, and new telemetry.
5. Attempt a breaking rename without migration; require the gate to deny it and name every affected consumer.
6. Prove an unaffected field and replacement query still work.
7. Remove the old field only after usage is zero for the published window; falsify readiness if any live consumer or unexplained query remains.

Telemetry is production-facing interface state even when customers never see its field names directly.

• Shared identity makes metrics, logs, traces, profiles, deployments, and owners correlatable.
• Producers own business meaning; collectors cannot invent semantics.
• Published revisions are immutable and deprecations are evidence-driven.
• Compatibility requires consumer fixtures, negative tests, a migration window, and rollback.

A schema change is ready only when both producers and consumers can prove their expectations.

• [ ] Every field has meaning, type, unit, owner, and cardinality or sensitivity policy.
• [ ] Canonical service identity is consistent across signals.
• [ ] Consumer inventory includes dashboards, alerts, SLOs, cost, security, and automation.
• [ ] A deliberately breaking fixture is denied with affected owners.
• [ ] Deprecation has replacement, deadline, zero-use evidence, and rollback.

Schema mechanics and naming conventions evolve, so current statements were checked against official specifications and documentation.

• OpenTelemetry telemetry schemas specification
• Prometheus metric and label naming
• OpenTelemetry configuration types and resource schema URL