Kubernetes events — the 1-hour TTL and the event-recorder

Kubernetes events are the cluster’s change log. The 1-hour default TTL is the retention. The event-recorder is the API for emitting events. The rate limiting is the throttle. This lesson walks the events, the TTL, the recorder, the rate limiting, and the production patterns.

What is a Kubernetes event

The Kubernetes event:

kubectl get events

LAST SEEN   TYPE      REASON              OBJECT                             MESSAGE
30s         Normal    Scheduled           pod/nginx-1-abc                    Successfully assigned to worker-1
30s         Normal    Pulling             pod/nginx-1-abc                    Pulling image nginx:1.25
30s         Normal    Pulled              pod/nginx-1-abc                    Successfully pulled image nginx:1.25
30s         Normal    Created             pod/nginx-1-abc                    Created container
30s         Normal    Started             pod/nginx-1-abc                    Started container

The event is associated with a Kubernetes object.

The event fields

The event fields:

apiVersion: v1
kind: Event
metadata:
  name: nginx-1-abc.1a2b3c4d
  namespace: default
  uid: 1a2b3c4d-5e6f-7g8h-9i0j-k1l2m3n4o5p6
involvedObject:
  kind: Pod
  name: nginx-1-abc
  namespace: default
reason: Scheduled
message: Successfully assigned to worker-1
source:
  component: default-scheduler
type: Normal
firstTimestamp: 2026-08-16T10:00:00Z
lastTimestamp: 2026-08-16T10:00:00Z
count: 1

The fields describe the event.

The event types

The event types:

The events are categorized by the type.

The 1-hour TTL

The 1-hour TTL:

apiVersion: v1
kind: ConfigMap
metadata:
  name: event-config
data:
  apiVersion: eventratelimit.admission.k8s.io/v1alpha1
  kind: Configuration
  limits:
  - type: Server
    qps: 100
    burst: 200

Wait, the TTL is different from the rate limit. The TTL is the time the events are kept in etcd:

Default TTL: 1 hour

The events are kept in etcd for 1 hour by default.

The event recorder

The event recorder:

import (
    "sigs.k8s.io/controller-runtime/pkg/recorder"
)

type Reconciler struct {
    client.Client
    Recorder record.EventRecorder
}

func (r *Reconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
    pod := &corev1.Pod{}
    r.Get(ctx, req.NamespacedName, pod)
    
    r.Recorder.Event(pod, corev1.EventTypeNormal, "Scheduled", "Pod is scheduled")
    r.Recorder.Event(pod, corev1.EventTypeWarning, "Failed", "Failed to schedule")
    
    return ctrl.Result{}, nil
}

The recorder is the API for emitting events.

The event rate limiting

The event rate limiting:

Default: 10 events per second per object

The rate limiting is to prevent event floods.

import (
    "k8s.io/client-go/util/flowcontrol"
)

rateLimiter := flowcontrol.NewTokenBucketRateLimiter(10, 20)

The rate limiter is the throttle.

The kubectl get events

The kubectl get events:

kubectl get events -A
kubectl get events --field-selector type=Warning
kubectl get events --sort-by='.lastTimestamp'
kubectl get events -n default --watch

The kubectl get events is the interactive inspection.

The events API

The events API:

kubectl get --raw=/api/v1/events

The events API exposes the events in JSON.

The event use cases

The event use cases:

flowchart LR
    A[Pod] --> B[Scheduled]
    A --> C[Pulled]
    A --> D[Started]
    A --> E[Failed]
    E --> F[Alert]
    F --> G[Operator]
    G --> H[Investigate]

The events are the input for the diagnosis.

The cross-course references

• The Observability course covers the events in detail.
• The etcd course covers the storage.
• The Kubelet course covers the runtime events.

Quiz

Production discipline

• Use the events API. The source of truth.
• Use the event recorder. The API for emitting events.
• Configure the rate limit. The eventratelimit controller.
• Use event-exporter. The long-term storage.
• Monitor the warning events. The detection.
• Document the events. The investigation.

The Kubernetes events are the cluster’s change log. Operating it well is the events API, the recorder, the rate limit, and the long-term storage.