Skip to main content
RunBook Academy

ObservabilityXXXI · Logging FoundationsLoggingFoundations

PII and Secrets in Logs

Intermediate⏱ ~18 minbash

What you'll learn

  • Explain pii and secrets in logs in production terms
  • Configure and operate pii and secrets in logs in a production observability stack
  • Recognise and diagnose the most common failure modes
  • Apply the discipline to a real environment

Prerequisites

Verified against Prometheus 2.55.x · Alertmanager 0.28.x · node_exporter 1.8.x · blackbox_exporter 0.26.x · Grafana 11.x · Loki 3.x · Tempo current · OpenTelemetry Collector 0.110.x · Grafana Alloy current · Docker Engine 28.x · Ubuntu 24.04 LTS · Debian 12 (Bookworm) · RHEL / Rocky / AlmaLinux 9.x · 2026-08-13

Not yet marked complete on this device.

Logs are a frequent source of PII / secrets exposure. The policy: redact at source.

What it is

A precise definition of pii and secrets in logs, scoped to production operations.

Why a sysadmin cares

Production framing.

How it works

The mental model.

example_setting: value

How to configure it

Real configuration examples with annotated options.

promtool check config /etc/prometheus/prometheus.yml

How to validate it

Commands the operator runs to confirm the configuration is live and correct.

How it can fail

The high-frequency failure modes: silent misconfiguration, crash on load, performance regression, permissions failure, schema / version drift.

How to troubleshoot it

The diagnostic order.

Security implications

PII and Secrets in Logs has security implications wherever the relevant component exposes an HTTP endpoint, an authentication layer, or a credential.

Performance implications

Performance implications come from cardinality, scrape / push interval, rule size, retention, and query cost.

Production guidance

  • Validate before applying.
  • Test changes in a non-production environment.

Verification

You should now be able to answer:

  • What is pii and secrets in logs in production terms?
  • Why does a sysadmin care about it?
  • How does it fail and how do you diagnose the failure?

Quiz

Knowledge check · 8 questions

  1. Q1. What is the primary purpose of pii and secrets in logs?

  2. Q2. Which failure mode of pii and secrets in logs is most operationally costly?

  3. Q3. Production verification should run on production hosts.

  4. Q4. First response when pii and secrets in logs misbehaves?

  5. Q5. Name one signal that confirms pii and secrets in logs is healthy.

  6. Q6. Which of these are validation steps?

  7. Q7. Right discipline when changing in production?

  8. Q8. Telemetry usefulness requires:

Passing score: 75%. Answers are checked in this browser.