Shared state and replicated state - when stateless is not enough

When a service cannot be stateless, the state must be managed. This lesson covers the patterns: shared storage, replication, and consensus.

Shared vs replicated state

Shared state: all nodes access the same storage. Examples: NFS, SAN, a database.

Replicated state: each node has its own copy; updates propagate. Examples: Galera cluster, Raft consensus.

Shared state is simpler but has a single point of failure (the shared storage). Replicated state is more resilient but more complex.

Patterns

Shared storage

All nodes mount the same filesystem (NFS, SAN). State is read and written by all nodes.

Pros:

• Simple.
• Consistent view.

Cons:

• Single point of failure (the shared storage).
• Network latency.
• Storage contention.

Use when:

• Data is small and not high-throughput.
• Consistency is critical.
• Shared storage is already available (SAN).

Active-passive replication

One node is the primary; replicas copy from it. Reads can go to replicas; writes go to the primary.

Pros:

• Better than shared storage for high availability.
• Replicas can serve reads (scale).

Cons:

• Primary is a single point of failure (with replication lag).
• Replica lag.

Use when:

• Read-heavy workload.
• Replica lag is acceptable.

Active-active replication

All nodes are writable. Updates are coordinated via consensus (Raft, Paxos).

Pros:

• No single point of failure.
• Scales for writes.

Cons:

• Complex (consensus is hard to implement correctly).
• Latency for writes (consensus rounds).

Use when:

• High availability for writes.
• Consensus is acceptable (low to moderate write rate).

Leader election

For active-passive, a leader election mechanism picks the primary. Common tools:

• Pacemaker + Corosync: full cluster manager.
• Keepalived: VRRP for VIP, plus scripts.
• etcd: distributed key-value store with leader election.
• ZooKeeper: similar to etcd.

The leader is the only node that accepts writes. On failure, a new leader is elected.

Consensus algorithms

For active-active, consensus is required:

• Paxos: classic, hard to understand, hard to implement.
• Raft: easier to understand, used by etcd, Consul.
• Zab: used by ZooKeeper.

These algorithms ensure all nodes agree on the same order of operations. They require a majority for each decision (quorum).

Choose the pattern

Knowledge check