Load balancing concepts - distributing traffic across backends

Load balancing distributes traffic across multiple backends to improve availability, scalability, and resource use. This lesson covers the concepts and the common patterns.

Layer 4 vs Layer 7

Layer 4 is fast but limited (cannot see the URL). Layer 7 is smarter but slower (parses the request).

Algorithms

• Round robin: each request goes to the next backend in turn. Simple, fair.
• Least connections: route to the backend with fewest active connections. Good for uneven request times.
• Source IP hash: same client IP always goes to the same backend. Provides session affinity.
• Weighted: backends with different capacities get different weights.
• Random: each request to a random backend. Surprisingly effective for many workloads.

Choose the algorithm based on:

• Session affinity needed: source hash.
• Even request times: round robin.
• Uneven times: least connections.
• Different backend capacities: weighted.

Health checks

The LB monitors each backend and removes failed ones from the pool:

• TCP: connection succeeds.
• HTTP: HTTP 200 from a /health endpoint.
• Custom: check process, files, dependencies.

Failed backends are removed from rotation automatically. The LB stops sending traffic to a backend that is unhealthy.

Session persistence

Some applications need a client to always reach the same backend (e.g. session state stored locally):

• Cookie-based: the LB sets a cookie that pins the client to a backend.
• IP-based: same source IP always to the same backend.
• Header-based: the application sets a header that the LB reads.

For stateless applications, no persistence is needed.

Common patterns

Web application

Internet → Load Balancer → Web 1
                          → Web 2
                          → Web 3

The LB distributes HTTP requests; backends are stateless or share session state.

Database with read replicas

App → Write to primary
    → Read from replicas (LB chooses)

The LB routes writes to the primary; reads to replicas.

Microservices

API gateway → Service A (multiple instances)
            → Service B (multiple instances)
            → Service C (multiple instances)

The gateway is the LB; each service is a backend pool.

Choose the right tool

Knowledge check