spec and status — the discipline of declared intent vs observed reality

The spec-vs-status discipline is the contract that makes Kubernetes a declarative system: operators write spec, controllers write status, and the spec-vs-status gap is what reconciliation closes. This lesson covers the discipline in depth and the operational patterns that arise from it.

The contract

flowchart LR
    O[Operator] -->|writes| S[spec<br/>intent]
    C[Controller] -->|writes| T[status<br/>observed]
    R[Reconciler] -->|closes gap between| S
    R -->|closes gap between| T

Every Kubernetes object has a spec and status (with a few exceptions where spec is empty). The contract is:

• Operators (humans, CI, GitOps) write spec
• Controllers write status
• The API server enforces schema and concurrency
• The reconciler closes the gap between them

When spec and status agree, the system is converged. When they disagree, reconciliation is in progress.

spec is intent

The operator declares what should be true. The controller makes it true.

spec:
  replicas: 5
  selector:
    matchLabels:
      app: web
  template:
    spec:
      containers:
      - name: nginx
        image: nginx:1.27.1

The controller reads this and acts. It does not consult other fields to decide; the spec is the source of truth.

Production principle: operators should only edit spec. Direct edits to status are temporary and will be overwritten by the controller on the next reconcile.

status is the system’s report

The controller writes what is currently true:

status:
  replicas: 5
  readyReplicas: 5
  availableReplicas: 5
  observedGeneration: 3
  conditions:
  - type: Progressing
    status: "True"
    reason: NewReplicaSetAvailable
    lastUpdateTime: "2026-08-15T12:01:01Z"
  - type: Available
    status: "True"
    lastTransitionTime: "2026-08-15T12:01:01Z"

The status carries:

• Counters — replicas, ready, available
• Conditions — the controller’s assessment
• Timestamps — when the state was last reported

The operator reads status to verify convergence, identify problems, and decide on next steps.

The spec-vs-status gap

sequenceDiagram
    autonumber
    participant API as API server
    participant Ctrl as Controller
    participant W as World

    Note over API: spec.replicas = 5
    Note over API: status.replicas = 3 (gap=2)
    Ctrl->>API: GET Deployment
    Ctrl->>W: count owned Pods (3)
    Ctrl->>API: create 2 more Pods
    W-->>Ctrl: Pods created
    Ctrl->>API: PATCH status.replicas: 5
    Note over API: spec.replicas == status.replicas (converged)

The gap is what the controller closes. Different controllers have different gaps:

• Deployment: spec.replicas vs status.replicas
• Node: spec.unschedulable vs status.conditions[Ready]
• Pod: spec.containers vs status.containerStatuses
• PersistentVolume: spec.claimRef vs status.phase

Who owns which field

This is critical for server-side apply and managed fields:

• Operator owns spec, metadata.labels, metadata.annotations
• Controller owns status, metadata.finalizers (typically)
• API server owns metadata.uid, metadata.resourceVersion, , metadata.managedFields, metadata.creationTimestamp`

A common operational mistake is editing status manually. The API server accepts the write (with the right RBAC), but the controller overwrites it on the next reconcile. The manual edit has no lasting effect.

managedFields and server-side apply

Server-side apply (SSA) is the canonical way to update objects in production. SSA tracks field ownership:

metadata:
  managedFields:
  - manager: kubectl
    operation: Apply
    apiVersion: apps/v1
    fields:
      f:spec:
        f:replicas: 5
        f:template: ...
  - manager: deployment-controller
    operation: Update
    apiVersion: apps/v1
    fields:
      f:status:
        f:replicas: 5
        f:readyReplicas: 5

When two actors both use SSA, their fields are kept separate. The Deployment controller can update status while the operator’s spec fields are preserved.

kubectl apply --server-side -f manifest.yaml

Without SSA, the operator’s kubectl apply could overwrite the controller’s status update (lost write). SSA preserves field ownership.

The discipline:

• Operators use SSA for declarative management
• Controllers write fields they own (status, finalizers)
• The API server prevents lost writes by tracking field ownership

How to inspect managedFields

kubectl get pod web-abc -o json | jq '.metadata.managedFields'

[
  {
    "manager": "kubectl",
    "operation": "Apply",
    "apiVersion": "v1",
    "fields": {
      "f:metadata": {
        "f:labels": {
          "f:app": {}
        }
      },
      "f:spec": {
        "f:containers": {
          "k:{\"name\":\"web\"}": {
            ".": {},
            "f:image": {},
            "f:name": {},
            "f:resources": {}
          }
        }
      }
    }
  },
  {
    "manager": "kubelet",
    "operation": "Update",
    "apiVersion": "v1",
    "fields": {
      "f:status": {
        "f:conditions": {},
        "f:containerStatuses": {},
        "f:phase": {}
      }
    }
  }
]

manager: kubectl indicates the operator’s manifest. manager: kubelet indicates the kubelet’s status updates. They own different fields; SSA merges them safely.

Conflict resolution in SSA

When two managers apply the same field:

# kubectl's apply sets spec.replicas = 5
# HPA's apply sets spec.replicas = 8
# Conflict on spec.replicas

The API server rejects the second apply if both managers have the field. Resolution options:

• --force-conflicts — take the field from the new apply
• --server-side --validate=false — skip validation
• Configure one manager to skip the conflicting field

In production, the HPA is typically allowed to manage spec.replicas. The operator’s manifest should set spec.replicas to a “minimum” or a “max” that HPA respects, or use a different field (e.g., spec.minReplicas via a custom controller).

The apiVersion is also a contract

The API version implies a schema. When the schema changes, the contract changes:

# apps/v1beta1 (deprecated)
spec:
  replicas: 5
  template:
    metadata:
      annotations:
        pod.beta.kubernetes.io/...

# apps/v1
spec:
  replicas: 5
  template:
    metadata:
      annotations:
        ...

Different versions may have different field names, different required fields, and different defaults. Production manifests should pin the apiVersion and update it deliberately, not auto-upgrade.

Convergence is necessary but not sufficient

A system can be technically converged and unhealthy:

flowchart LR
    S[spec: replicas=5] --> C{Converged?}
    C -->|status.replicas == 5| OK[Technically converged]
    OK --> H{Healthy?}
    H -->|probes pass| HLTH[Healthy]
    H -->|probes fail| UN[Converged but unhealthy]

A Deployment at 5/5 replicas is converged. If those Pods are crashlooping (liveness restarts), the Deployment is converged but unhealthy.

Production observability must cover both:

• Convergence: spec.replicas == status.replicas
• Health: probes pass; application KPIs are met

Common operational mistakes

Mistake 1: writing status manually

kubectl edit pod web-abc
# edit status.phase: Running
# (reverted by kubelet in seconds)

Fix: don’t edit status manually; fix the underlying cause.

Mistake 2: kubectl apply overwriting controller fields

kubectl apply -f deployment.yaml  # without --server-side
# spec is updated, but status fields may be overwritten

Fix: use kubectl apply --server-side.

Mistake 3: trusting status as the source of truth

kubectl get deployment web -o jsonpath='{.status.replicas}'
# 5
# "Spec says 5; status says 5; converged" — but the Pods may
# be unhealthy

Fix: validate health with probes and external metrics, not just status.

Cross-course references

• The Linux course part VI-Linux-Processes covers process state transitions; spec vs status is the cluster’s analogue.
• The Observability course part CIX-Observability-InvestigationWorkflows covers investigation methodology for converged-but-unhealthy clusters.
• The Docker course part XXX-Docker-Lifecycle covers the container lifecycle that drives much of the status reporting.
• The Linux course part XXIV-Linux-Time covers chrony — timestamps in status depend on clock accuracy.

Quiz

Production discipline

• Operators write spec; controllers write status. Never confuse the two.
• Use kubectl apply --server-side for declarative management; client-side apply can overwrite controller fields.
• Inspect managedFields when investigating “who changed this?” questions.
• Treat status.conditions as the controller’s assessment; conditions stuck False are the controller telling you it cannot converge.
• Pair convergence (spec == status) with health (probes, KPIs) — convergence alone is not health.