Cost and operational trade-offs — total cost of ownership

The cost and operational trade-offs between managed and self-managed Kubernetes are subtle. The apparent cost (control plane fee vs infrastructure) is rarely the total cost. This lesson walks TCO components, the hidden costs, team capacity, and the operational discipline.

The TCO components

flowchart LR
    A[Total cost of ownership] --> B[Direct costs]
    A --> C[Operational costs]
    A --> D[Hidden costs]
    B --> B1[Control plane fee or infrastructure]
    B --> B2[Worker node cost]
    B --> B3["Networking, storage"]
    C --> C1[SRE team time]
    C --> C2[On-call burden]
    C --> C3[Upgrade windows]
    D --> D1[Lock-in]
    D --> D2[Migration cost]
    D --> D3[Opportunity cost]

The TCO components:

• Direct costs. Control plane fee (managed) or infrastructure cost (self-managed); worker node cost; networking and storage.
• Operational costs. SRE team time for control plane management; on-call burden for control plane incidents; upgrade windows requiring downtime.
• Hidden costs. Vendor lock-in; future migration cost; opportunity cost (engineering time that could be spent on features).

The cost comparison

flowchart LR
    A[Managed TCO] --> A1["Direct: control plane + workers"]
    A --> A2["Operational: low (provider handles most)"]
    A --> A3["Hidden: lock-in, migration cost"]
    B[Self-managed TCO] --> B1["Direct: 3 control-plane + workers"]
    B --> B2["Operational: high (operator handles all)"]
    B --> B3["Hidden: low (cloud-agnostic)"]

The cost comparison:

Managed Kubernetes TCO:

• Direct: control plane fee ($0.10/hour for EKS/GKE, free for AKS/OKE) + worker nodes.
• Operational: low (provider handles control plane upgrades, etcd, patching).
• Hidden: lock-in (provider-specific CNI, CSI, IAM).
• Migration cost: high if lock-in is realised.

Self-managed Kubernetes TCO:

• Direct: 3 control-plane nodes (HA) + worker nodes.
• Operational: high (operator handles everything).
• Hidden: low (cloud-agnostic, portable).
• Migration cost: lower (manifests and add-ons are portable).

Team capacity and operational maturity

flowchart LR
    A[Team capacity] --> B{SRE team size}
    B -->|<3 SREs| C["Managed: insufficient capacity for self-managed"]
    B -->|3-10 SREs| D["Hybrid: managed for most, self-managed for special cases"]
    B -->|10+ SREs| E["Self-managed: capacity and expertise available"]

The decision framework:

• Small team (<3 SREs). Managed is the only realistic choice; self-managed is over-provisioned for the team’s capacity.
• Medium team (3-10 SREs). Hybrid is appropriate; managed for most clusters, self-managed for special cases (compliance, on-prem).
• Large team (10+ SREs). Self-managed is feasible; the team has the capacity and expertise.

Operational maturity also matters:

• Low maturity. Managed reduces the risk of misconfiguration.
• High maturity. Self-managed gives full control.

The TCO calculation

flowchart LR
    A[Managed TCO] --> A1[Control plane fee per cluster × cluster count]
    A --> A2[+ worker node cost × node count × hour]
    A --> A3["+ lock-in risk cost (subjective)"]
    A1 --> B[Total managed TCO]
    A2 --> B
    A3 --> B
    C[Self-managed TCO] --> C1[Control plane node cost × 3 × hour]
    C --> C2[+ worker node cost × node count × hour]
    C --> C3[+ SRE time × hourly rate]
    C1 --> D[Total self-managed TCO]
    C2 --> D
    C3 --> D
    E["Decision: lower TCO wins"]
    B --> E
    D --> E

The TCO calculation:

1. Estimate the cluster count (now and projected).
2. Calculate the managed TCO: control plane fee + worker nodes + lock-in risk cost.
3. Calculate the self-managed TCO: control plane nodes + worker nodes + SRE time.
4. Compare.

The decision is not just about cost; it’s about team capacity, lock-in tolerance, and operational maturity.

The operational discipline

The TCO discipline:

1. Model the TCO before choosing. Direct cost alone is misleading; include operational and hidden costs.
2. Re-evaluate annually. Team capacity, costs, and lock-in tolerance change.
3. Document the choice and rationale. The decision and the drivers must be in the runbook.
4. Plan for migration. Even if managed is chosen, abstract cloud-specific dependencies to enable future migration.
5. Monitor TCO components. The control plane fee changes (EKS has changed pricing); the SRE time changes (new tools reduce it).

The operational failure modes

TCO modelling fails for predictable reasons:

• Apparent cost only. The team chooses managed based on the control plane fee alone, ignoring SRE time savings; or chooses self-managed based on infrastructure alone, ignoring SRE time cost.
• Hidden costs ignored. Lock-in, migration cost, opportunity cost are not modelled.
• Static analysis. The TCO is calculated once; team capacity and costs change over time.
• No fallback plan. The choice is made without a plan to migrate if the choice proves wrong.

Quiz

Production discipline

TCO modelling in production rests on five non-negotiable elements:

• Model TCO before choosing. Direct cost alone is misleading; include operational and hidden costs.
• Assess team capacity honestly. A small team with self-managed is a recipe for incidents.
• Re-evaluate annually. Costs, team capacity, and lock-in tolerance change.
• Document the choice and rationale. The decision and the drivers must be in the runbook.
• Plan for migration. Even if managed is chosen, abstract cloud-specific dependencies.

TCO modelling is the foundation of good architecture decisions. The discipline is to model before choosing, to re-evaluate regularly, and to plan for change.