KubernetesXC · Distributed TracingTracing
Tracing + metrics + logs — the three signals integrated
What you'll learn
- Integrate the three signals
- Use the trace IDs for correlation
- Configure the OTel Collector as the central pipeline
- Plan the production architecture
Prerequisites
Verified against Kubernetes 1.34.x · kubeadm 1.34.x · kubectl 1.34.x · etcd 3.6.x · CoreDNS 1.11.x · containerd 1.7.x / 2.x · 2026-08-16
The three signals (traces, metrics, logs) are the observability pillars. The trace IDs are the correlation across signals. The OTel Collector is the central pipeline. The Grafana is the integration point. This lesson walks the integration, the correlation, the pipeline, and the production patterns.
The three signals
The three signals:
flowchart LR
A[Application] --> B[Traces]
A --> C[Metrics]
A --> D[Logs]
B --> E[Jaeger/Tempo]
C --> F[Prometheus]
D --> G[Loki]
E --> H[Grafana]
F --> H
G --> H
The three signals are the observability pillars.
The correlation
The correlation:
{
"timestamp": "2026-08-16T10:00:00.000Z",
"level": "INFO",
"message": "Request processed",
"trace_id": "0af7651916cd43dd8448eb211c80319c",
"span_id": "b7ad6b7169203331"
}
The trace IDs in the logs enable the correlation.
The OTel Collector as the central pipeline
The OTel Collector:
flowchart LR
A[Application] --> B[OTel SDK]
B --> C[OTel Collector]
C --> D[Jaeger]
C --> E[Prometheus]
C --> F[Loki]
The OTel Collector is the central pipeline.
The Grafana integration
The Grafana integration:
flowchart LR
A[Grafana] --> B[Prometheus datasource]
A --> C[Loki datasource]
A --> D[Jaeger/Tempo datasource]
B --> E[Metrics]
C --> F[Logs]
D --> G[Traces]
E --> H[Dashboard]
F --> H
G --> H
The Grafana is the integration point.
The cross-signal navigation
The cross-signal navigation:
flowchart LR
A[Grafana dashboard] --> B[Prometheus metric]
B --> C[See logs]
C --> D[Loki logs]
D --> E[See trace]
E --> F[Jaeger trace]
F --> G[See metrics]
G --> B
The navigation is the cross-signal.
The OTel Collector exporters
The exporters:
exporters:
prometheus:
endpoint: 0.0.0.0:8889
loki:
endpoint: http://loki:3100/loki/api/v1/push
jaeger:
endpoint: jaeger:14250
tls:
insecure: true
The exporters send the signals to the backends.
The trace-based metrics
The trace-based metrics:
processors:
spanmetrics:
metrics_exporter: prometheus
latency_histogram_buckets: [2ms, 10ms, 50ms, 100ms, 500ms, 1s, 5s]
dimensions:
- name: http.method
- name: http.status_code
The spanmetrics processor generates metrics from the traces.
The correlation in logs
The correlation in logs:
import (
"go.opentelemetry.io/otel/trace"
)
log.Info("Request processed",
"trace_id", trace.SpanContextFromContext(ctx).TraceID().String(),
"span_id", trace.SpanContextFromContext(ctx).SpanID().String(),
)
The trace IDs are added to the logs.
The SLO-driven observability
The SLO-driven observability:
flowchart LR
A[Service] --> B[Metrics + SLO]
A --> C[Traces for errors]
A --> D[Logs for context]
B --> E[Alert]
C --> E
D --> E
E --> F[Runbook]
The SLOs are the integration point.
The production patterns
The production patterns:
flowchart LR
A[Application] --> B[OTel SDK]
B --> C[OTel Collector]
C --> D[Prometheus]
C --> E[Loki]
C --> F[Jaeger]
D --> G[Grafana]
E --> G
F --> G
G --> H[Dashboard]
G --> I[Alert]
The production pattern is the integration.
The cross-course references
The Observability course covers the integration in detail.
- The OpenTelemetry course covers the SDK.
- The Prometheus course covers the metrics.
- The Loki course covers the logs.
- The Jaeger course covers the traces.
Quiz
Knowledge check · 4 questions
Q1. What is the role of the trace IDs in the three signals?
Q2. The OTel Collector is the central pipeline for the three signals.
Q3. Walk the integrated observability for a workload.
Workload: HTTP API with 3 services. The team is configuring the integrated observability.
Q4. What is the spanmetrics processor in the OTel Collector?
Passing score: 75%. Answers are checked in this browser.
Production discipline
- Integrate the three signals. Traces, metrics, logs.
- Use the trace IDs for correlation. The integration.
- Use the OTel Collector as the central pipeline. The integration.
- Configure the cross-signal navigation. The Grafana.
- Use the spanmetrics processor. The trace-based metrics.
- Document the integration. The SDK, the backends, the navigation.
The three signals integrated is the production observability. Operating it well is via the OTel Collector, with the trace IDs for correlation, and the Grafana for the integration.