Audit log backends — log file and webhook

The audit log backends are the destinations for the audit events. The log file, the webhook, the dynamic backends are the options. The volume management is the rotation, the archival. This lesson walks the backends, the volume management, and the production patterns.

The log file backend

The log file backend:

apiVersion: v1
kind: Pod
metadata:
  name: kube-apiserver
spec:
  containers:
  - name: kube-apiserver
    command:
    - kube-apiserver
    - --audit-policy-file=/etc/kubernetes/audit-policy.yaml
    - --audit-log-path=/var/log/kubernetes/audit.log
    - --audit-log-maxage=30
    - --audit-log-maxbackup=10
    - --audit-log-maxsize=100
    volumeMounts:
    - name: audit-policy
      mountPath: /etc/kubernetes/audit-policy.yaml
    - name: audit-log
      mountPath: /var/log/kubernetes
  volumes:
  - name: audit-policy
    hostPath:
      path: /etc/kubernetes/audit-policy.yaml
  - name: audit-log
    hostPath:
      path: /var/log/kubernetes

The log file backend writes the audit events to a file.

The webhook backend

The webhook backend:

apiVersion: v1
kind: Pod
metadata:
  name: kube-apiserver
spec:
  containers:
  - name: kube-apiserver
    command:
    - kube-apiserver
    - --audit-policy-file=/etc/kubernetes/audit-policy.yaml
    - --audit-webhook-config-file=/etc/kubernetes/audit-webhook-config.yaml
    volumeMounts:
    - name: audit-policy
      mountPath: /etc/kubernetes/audit-policy.yaml
    - name: audit-webhook-config
      mountPath: /etc/kubernetes/audit-webhook-config.yaml
  volumes:
  - name: audit-policy
    hostPath:
      path: /etc/kubernetes/audit-policy.yaml
  - name: audit-webhook-config
    hostPath:
      path: /etc/kubernetes/audit-webhook-config.yaml

The webhook backend sends the audit events to an HTTP endpoint.

The webhook config

The webhook config:

apiVersion: v1
kind: Config
clusters:
- name: audit-webhook
  cluster:
    server: https://audit-collector.example.com/audit
    certificate-authority: /etc/kubernetes/ca.crt
contexts:
- name: default
  context:
    cluster: audit-webhook
    user: kube-apiserver
current-context: default
users:
- name: kube-apiserver
  user:
    client-certificate: /etc/kubernetes/apiserver.crt
    client-key: /etc/kubernetes/apiserver.key

The webhook config is the client config.

The dynamic backend

The dynamic backend:

apiVersion: v1
kind: Pod
metadata:
  name: kube-apiserver
spec:
  containers:
  - name: kube-apiserver
    command:
    - kube-apiserver
    - --audit-policy-file=/etc/kubernetes/audit-policy.yaml
    - --audit-dynamic-configuration

The dynamic backend allows the operator to enable the backends at runtime.

The volume management

The volume management:

# Log rotation
maxsize: 100M (max size per file)
maxbackup: 10 (number of backups)
maxage: 30 (number of days)

# The audit log is rotated when the size is exceeded.
# The audit log is archived when the age is exceeded.

The volume management is the rotation.

The audit shipping

The audit shipping:

# Tail the audit log
tail -f /var/log/kubernetes/audit.log

# Ship to Loki
promtail -config.file=audit-promtail.yaml

# Ship to SIEM
filebeat -e -c audit-filebeat.yaml

The audit shipping is the integration.

The audit volume isolation

The audit volume isolation:

volumes:
- name: audit-log
  hostPath:
    path: /var/log/kubernetes
    type: DirectoryOrCreate

The audit volume is isolated from the host’s other volumes.

The production patterns

The production patterns:

flowchart LR
    A[Audit event] --> B[Log file]
    A --> C[Webhook]
    B --> D[Local rotation]
    C --> E[SIEM]
    D --> F[Long-term storage]
    E --> F

The pattern is the production discipline.

The cross-course references

• The API server course covers the audit configuration.
• The Security course covers the audit policies.
• The SIEM course covers the integration.

Quiz

Production discipline

• Configure the log file backend. The default.
• Configure the rotation. The maxsize, maxbackup, maxage.
• Configure the webhook backend. The SIEM integration.
• Ship the audit log. The long-term storage.
• Document the backends. The configuration.
• Test the audit log. Verify the events.

The audit log backends are the security trail. Operating it well is the log file + webhook, the rotation, and the production patterns.