Redundancy levels - N+1, N+2, 2N and 2N+1

linux-n-plus-1-capacity established one spare and the survivable ceiling that follows from it. Real designs also have to justify more than one spare, or explain why one is enough. This lesson gives the levels their precise meanings and the arithmetic that prices them.

The naming, precisely

The letter N in these names is the capacity the workload requires, not the number of machines you own. The suffix is the number of spare units on top of it:

2N is not “N+N spares scattered about”. It means two independent complete sets, either of which carries the full load alone: A and B power feeds, two full UPS strings, an active site and a standby site. 2N+1 adds one more unit so that a failure inside the surviving set is still absorbed, and it is where serious facility power designs land.

The ceiling for any number of spares

The single-failure rule generalises directly. With T total nodes each at utilisation u, losing s of them spreads T x u over the T - s survivors:

per-node load after s failures = T x u / (T - s)

Set that to 100% and solve for u:

survivable ceiling: u <= (T - s) / T

For one spare this is the familiar table. For two, the ceilings drop sharply at small T:

Read the s = 2 column as the price list for N+2. A three-node cluster that must survive two failures may not exceed one third utilisation, because the single survivor has to carry everything.

As before, the ceiling is the point at which the failure becomes an outage, not a target. Survivors pinned at exactly 100% do not deliver the latency they delivered at 70%, and load rarely redistributes evenly.

Why N+2 is cheap on a wide cluster and brutal on a narrow one

The same two-failure tolerance costs wildly different amounts depending on how many nodes you spread it across. Two designs, both N+2, both using 100-unit nodes:

3 nodes, ceiling 33.3%
  Capacity bought:   300 units
  Useful load:       100 units
  After 2 failures:  100 units on 1 node = 100%

6 nodes, ceiling 66.7%
  Capacity bought:   600 units
  Useful load:       400 units
  After 2 failures:  400 units on 4 nodes = 100% each

Twice the hardware carries four times the useful load at the same tolerance. The spare is a fixed number of nodes, so the wider the cluster, the smaller a fraction of it the spare represents. That is the capacity argument for wide clusters, and it is the same argument as the single-failure case, only sharper.

The corollary is uncomfortable and worth saying plainly: on three nodes, N+2 usually is not worth buying. Either accept N+1 and shorten the repair time, or widen the cluster to five or six nodes and get N+2 almost as a side effect.

Choosing the number of spares

The number of spares is a function of two things you can measure:

• How long a unit stays broken. A node that takes three days to repair - parts on order, a vendor SLA, a reinstallation - spends three days at N+0 if you only bought N+1. A node you can rebuild from configuration management in forty minutes barely dents the budget.
• How often units break. Independent failures within the same repair window are what N+2 exists for. If your fleet loses a node a month and repairs take three days, the probability of overlap is small; if repairs take three weeks, it is not.

State it as a sentence the design has to defend: “we are N+1 with a four-hour repair target, so our exposure is four hours at N+0 per node failure, roughly twice a year.” That is a sentence a business can accept or reject. “We are N+1” is not.

Maintenance is a planned failure

Patching, kernel upgrades and hardware swaps take a node out deliberately. During the window, an N+1 cluster is running at N and has no protection at all.

This is not a reason to buy N+2 everywhere. It is a reason to be honest about which of these you have chosen:

• Accept the exposure. Keep the window short, announce it, and do not schedule two of them at once. This is the normal and correct answer for most services.
• Buy the second spare. N+2 so that patching leaves N+1 intact. Justified where the maintenance cadence is high or the window cannot be short - a rolling kernel upgrade across a large cluster, for instance.
• Drain rather than remove. Move the load off the node before taking it down, so the cluster is at N+1 with redistributed load rather than at N. This works when the remaining nodes are below the ceiling for T - 1.

A spare that cannot take the load is not a spare

Capacity is necessary and not sufficient. A spare has to be able to actually run the work, and these are the ways it turns out it cannot:

• Wrong failure domain. The spare shares the rack, the switch, the power feed or the hypervisor with the unit it covers. The event takes both.
• Cold. The spare needs an hour of data replay or a manual restore before it can serve. That hour is in your MTTR whether the capacity plan mentions it or not.
• Not licensed, not configured, not permitted. The database licence covers two sockets. The firewall permits the two known IPs. The application has a per-node key. All of these are discovered during the failover.
• Blocked by a placement rule. A Pacemaker colocation or anti-affinity constraint forbids the resource from running where the capacity is. linux-pacemaker-constraints-colocation-order covers the mechanism.
• Never tested. A spare that has not carried production load is a hypothesis about a spare.

Knowledge check