Observability XCI · Backup StrategyBackup

Tempo Backup

Intermediate⏱ ~22 minbash

What you'll learn

Explain the topic in production terms
Configure and operate the topic in a production observability stack
Recognise and diagnose the most common failure modes
Apply the discipline to a real environment

Prerequisites

Backup Basics

Verified against Prometheus 2.55.x · Alertmanager 0.28.x · node_exporter 1.8.x · blackbox_exporter 0.26.x · Grafana 11.x · Loki 3.x · Tempo current · OpenTelemetry Collector 0.110.x · Grafana Alloy current · Docker Engine 28.x · Ubuntu 24.04 LTS · Debian 12 (Bookworm) · RHEL / Rocky / AlmaLinux 9.x · 2026-08-13

Not yet marked complete on this device.

Same as Loki: object-store versioning.

What it is

A precise definition scoped to production operations.

Why a sysadmin cares

Production framing.

How it works

The mental model.

example_setting: value

How to configure it

Real configuration examples with annotated options.

promtool check config /etc/prometheus/prometheus.yml

How to validate it

Commands the operator runs to confirm the configuration is live and correct.

How it can fail

Failure modes: silent misconfiguration, crash on load, performance regression, permissions failure, schema / version drift.

How to troubleshoot it

Diagnostic order: was it working before? what does the service view say? what does the platform view say? form hypothesis, find evidence, test, validate.

Security implications

The topic has security implications wherever the relevant component exposes an HTTP endpoint, an authentication layer, or a credential.

Performance implications

Performance implications come from cardinality, scrape / push interval, rule size, retention, and query cost.