R5.4 · One trace context across the composed path

Theme	R5 · Technology stack
Domain	Infrastructure, operations
Status	Proposed
Needs client validation	Yes
Reinforces	R2.2 — the instrumentation a derived budget has to be measured on · R4.1
Anchored in	The ADS's observability row · assessment findings C-01 and F-04, conditions 4 and 7 · the call path in the API landscape · the stripped x-federated-id header in the experience map
Open questions	Raises 35; informs 14
Deliverables	D2.1 · D3.1

Concern

Three observability products are named. No identifier is named that survives the fan-out.

The ADS's observability row is a product list. New Relic, PromStack and ADX, for "metrics, logs, errors". No correlation identifier, no propagation requirement, and no statement of which of the three answers a question about a single request.

The request it would have to follow crosses six boundaries. The API landscape draws the path — Akamai, F5 GTM, Azure Application Gateway, APIM, then microservices, databases and mainframe — and the dashboard adds the shell, the BFF and the Integration API's fan-out in front of thirteen hard dependencies (finding C-01), owned by six lines of business plus Enterprise Technology Services.

The estate has already shown that headers do not survive those hops. The experience map records the PROD path stripping x-federated-id. A header that silently disappears between DEV and PROD is exactly what happens to a trace header that is nobody's stated requirement, and this one had a purpose.

What it costs to leave unspecified

R2.2 asks for an availability and latency budget derived from the dependency model rather than inherited. A budget that cannot be attributed is a number, not a control: when the homepage misses its 800 ms commitment at P99, the useful question is which of the fan-out legs spent it, and each leg belongs to a different team. The same gap disarms R4.1 — a gateway can only shed load or enforce a quota for a caller it can identify across hops — and finding F-04, which records no incident command for a cross-BU sev-1, describes an on-call rotation that would have to reconstruct a composed request from three tools by timestamp.

Recommendation

Adopt W3C Trace Context, propagated by OpenTelemetry instrumentation, as a requirement on every hop of the dashboard critical path — and name one system of record for a composed request.

traceparent propagates; it is not terminated. Every hop, including APIM and the business-unit Data APIs, forwards the incoming context and annotates its own span rather than starting a fresh trace. A gateway terminating the trace and originating a new one is the common default, and it is the specific failure that hides the fan-out: each leg is individually observable and the composition is not.

One query surface answers "show me this page load, end to end." Three products is not the problem; three products with no agreed home for that question is. Name which one owns it. The other two keep the jobs they have.

The trace identifier carries no customer data. R4.2's argument applies directly: a correlation ID is a join key, not a subject. No CIAM ID, policy number or member number in span attributes or log lines — the trace is the one artifact that is deliberately readable by every team on the path, which is precisely why it is the wrong place for an identifier the aggregate's classification would restrict.

The component this adds is a collector. An OpenTelemetry collector on the path is the new piece, and it is the cheapest part of the recommendation. The expensive part is the propagation requirement landing in six business units' API contracts, which is why it belongs in the target state document rather than in a runbook.

What it changes

It makes R2.2's budget measurable. A derived per-dependency budget and the per-dependency deadline budgets in R2.1 are both allocations, and an allocation nobody can measure against is not enforced. This is the instrumentation that closes the assessment's condition 4 in practice rather than on paper.

It gives the shared runtime's on-call something to open. Condition 7 asks for named accountable owners; open question 14 carries it. An incident commander for a cross-BU sev-1 needs one trace, not three consoles and a clock.

It puts a propagation clause into the fan-out contract. The business-unit Data APIs are the hops most likely to drop it, and they are also the ones whose latency the budget most needs attributed. That makes it a D2.1 integration requirement, not an observability preference.

What we are not recommending

Replacing New Relic, PromStack or ADX. The recommendation is a propagated standard and one named owner for the composed view. Which products stay is not an architectural question, and all three already have jobs.

A sampling strategy, retention period or span naming convention. Implementation design is out of scope under SOW Schedule "A" §3.0.

Instrumenting the estate. The domain is the dashboard critical path — the same first-domain argument R4.3 makes for mediation, for the same reason: it is the path with the composed failure mode, and it is small enough to finish.

Dependencies and open questions

• Open question 35 — which of the three observability products is the system of record for a composed request, and whether trace context survives APIM today. The second half is answerable this week by someone with access, and the answer sets how much of this is new work.
• Open question 14 — the shared runtime's owner. Tracing is one of the things that owner would operate.
• The stripped x-federated-id header. It is a different header with a different purpose, but it is the same class of defect, and whoever explains it also explains what else the PROD path removes.