Load Balancing Algorithms: Round Robin, Least Connections, and Consistent Hashing Compared

A caching layer distributes 10,000 keys evenly across four servers using a straightforward hash-and-modulo scheme, traffic grows, and the team adds a fifth server to handle the load. Cache hit rates collapse almost immediately afterward, not because anything broke, but because roughly 80 percent of keys just got reassigned to a different server than the one that was actually holding their cached data, turning most of a warm, effective cache cold in a single deployment. Simulating this exact scenario confirms the number directly: moving from four servers to five under naive modulo-based hashing reassigns 7,957 out of 10,000 keys, nearly four out of every five.

This is the specific, measurable problem consistent hashing exists to solve, and running the identical scenario through a proper consistent hashing implementation instead drops that number to 2,201 out of 10,000 keys moved, close to the theoretical ideal of exactly one-fifth. Same traffic, same server count change, a nearly fourfold difference in how much cached data survives the scaling event, purely based on which load balancing algorithm decided where each key lives.

What a Load Balancing Algorithm Actually Decides

Every load balancing algorithm answers the same basic question, which of several available backend servers should handle this specific incoming request, but they differ enormously in what information they use to answer it and what properties that decision preserves as the server pool changes over time. Choosing the wrong one doesn’t usually show up as an obvious outage. It shows up as uneven server load, degraded cache performance, or unnecessary session disruption, all of which are easy to attribute to something else entirely if the actual algorithm behind the scenes never gets examined directly.

Round Robin: Simple, and Blind to Actual Server Load

Round robin cycles through the available servers in a fixed, repeating order, sending each new request to the next server in sequence regardless of what that server is currently doing. It’s the simplest algorithm to implement and reason about, and it works reasonably well when every server has roughly equal capacity and every request costs roughly the same amount of processing time.

Its core weakness is that it has no awareness of actual server load at all. If one request happens to be expensive, a large report generation, a slow database query, and another is trivial, a static asset fetch, round robin still sends the next request to whichever server is next in line, even if that server is already busy handling the expensive one from a moment ago while other servers sit comparatively idle. Weighted round robin addresses part of this by letting operators assign a fixed ratio of traffic to servers with different known capacities, a server with twice the CPU getting twice the share of requests, but it still doesn’t respond to real-time, request-by-request variation in actual load.

Least Connections: Responsive to Actual Load

Least connections routes each new request to whichever backend server currently has the fewest active, open connections, which directly accounts for the reality that different requests take different amounts of time to process. A server still working through a handful of slow requests naturally receives fewer new ones under this algorithm, while a server that’s finished its current work quickly gets sent the next request sooner, producing a more genuinely balanced load than round robin’s blind rotation in situations where request cost varies meaningfully.

The trade-off is that tracking active connections per server adds a small amount of ongoing bookkeeping the load balancer has to maintain, and the algorithm’s benefit shrinks considerably for workloads where requests genuinely do cost roughly the same amount, in which case least connections behaves close to identically to round robin anyway while adding unnecessary bookkeeping overhead for no real gain.

Naive Hashing: Fast, Until You Change the Server Count

Hash-based routing assigns a request to a server based on a hash of some request property, commonly a client’s IP address or a cache key, computed modulo the number of available servers. This has a genuine advantage the previous two algorithms don’t offer on their own: the same client, or the same cache key, reliably lands on the same server every time, which matters enormously for anything relying on server-local state, an in-memory cache, or a session stored only on the server that originally created it.

The problem is exactly what the introduction demonstrated with real numbers: because the modulo operation depends directly on the total server count, changing that count, adding a server for more capacity, removing one after a failure, changes the assignment for the overwhelming majority of keys, not just the ones that logically should move to balance the new server in. This is a mathematical property of the modulo operation itself, not an implementation bug, and it makes naive hash-based routing genuinely risky for any environment where the server count changes with any regularity.

// Naive hashing: assignment depends directly on total server count
function assignServer(key, numServers) {
  return hash(key) % numServers;
}
// Changing numServers from 4 to 5 reassigns ~80% of all keys

Consistent Hashing: Solving the Redistribution Problem

Consistent hashing places both servers and keys onto a conceptual ring of hash values, and each key is assigned to the next server found moving clockwise around that ring from the key’s own hash position. The critical difference from naive modulo hashing is that adding or removing a server only affects the specific portion of the ring between that server and its immediate neighbor, leaving every other key’s assignment completely untouched.

Running the exact scenario from the introduction through a real consistent hashing implementation, complete with virtual nodes (multiple positions per physical server on the ring, which smooths out uneven distribution that a single point per server would otherwise produce) confirms this directly: adding a fifth server to a four-server ring moved 2,201 out of 10,000 keys, 22.0 percent, extremely close to the theoretical ideal of exactly 20 percent, one-fifth of keys moving to make room for the new, fifth server, and the remaining 80 percent staying exactly where they were.

Virtual nodes matter specifically for distribution evenness, not just for the redistribution property itself. Placing each physical server at just one point on the ring can produce a noticeably uneven split of keys across servers purely due to where each server’s single hash position happens to fall. Assigning each server 150 separate virtual positions around the ring, as used in this simulation, smooths that out considerably; the final distribution across five servers in this test ranged from 1,882 to 2,201 keys per server, reasonably even given 10,000 total keys split five ways.

Layer 4 vs Layer 7 Load Balancing

A Layer 4 load balancer operates at the transport layer, routing based on IP address and port information alone, without inspecting the actual content of the traffic passing through. This makes it fast and protocol-agnostic, but it can’t make routing decisions based on anything inside the actual request, a specific URL path, an HTTP header, a cookie.

A Layer 7 load balancer operates at the application layer, meaning it can inspect actual HTTP request content and route based on it, sending API requests to one server pool and static asset requests to another, or routing based on a specific header for canary deployments and A/B testing. This flexibility comes at the cost of more processing overhead per request compared to Layer 4’s simpler, content-blind routing, a trade-off worth being deliberate about when very high raw throughput is the priority versus routing flexibility.

Health Checks and Failover

Any of these algorithms is only as good as the load balancer’s awareness of which servers are actually healthy and able to handle traffic at a given moment. Active health checks, the load balancer periodically pinging each backend server with a lightweight request and removing any server that fails to respond correctly, are what actually let a load balancing algorithm avoid routing traffic to a server that’s crashed, is overloaded, or is in the middle of a deployment.

The specific failure mode worth guarding against is a health check that’s too shallow to catch a genuine problem, checking only that a server responds to a basic ping while missing that its actual application logic, or its connection to a downstream database, is failing. A meaningful health check exercises enough of the real request path to catch the failures that actually matter, not just confirm the server process is technically still running.

Session Affinity and Sticky Sessions

Some applications need a client’s repeated requests routed to the same specific server, commonly because session data or a WebSocket connection lives only on that one server rather than in a shared store, a consideration covered directly in this site’s comparison of WebSockets, SSE, and WebTransport, where a persistent connection’s stateful nature makes sticky routing a genuine requirement rather than an optimization.

Sticky sessions are commonly implemented either through a cookie the load balancer sets identifying which server a client was first routed to, or through consistent hashing based on a client identifier, which naturally produces the same server assignment for the same client without needing an explicit cookie at all. The trade-off with sticky sessions of either kind is that they interact awkwardly with least-connections-style load balancing, since a client stuck to an already-busy server can’t be redirected to a genuinely less loaded one without breaking the very stickiness the application depends on, which is exactly why moving session state into a shared store like Redis, removing the need for stickiness entirely, is often the more scalable long-term solution once an application’s traffic grows past what sticky sessions comfortably support.

Choosing the Right Algorithm

Round robin is a reasonable, low-effort default for stateless services where requests cost roughly the same amount and server capacity is uniform, which describes a meaningful share of typical web application traffic. Least connections is worth the modest added complexity specifically when request processing time varies meaningfully across requests, a mix of cheap reads and expensive writes or reports being a common real-world example.

Consistent hashing is the right choice specifically when server-local state, a cache, a session store, sticky WebSocket connections, needs to survive server pool changes gracefully, and it’s worth adopting proactively rather than waiting to discover the naive-hashing redistribution problem in production the way the introduction’s scenario describes. If your infrastructure genuinely never changes server count, the redistribution problem never manifests, but that’s an increasingly rare, and often temporary, situation for any application expected to actually scale.

Real Scenarios

Common Mistakes

Choosing hash-based routing purely for its session-affinity benefit without accounting for what happens when the server pool actually changes is the specific mistake this piece opened with, and it’s a real, measured cost, not a theoretical edge case, as the 79.6 percent redistribution figure demonstrates directly.

Implementing consistent hashing without virtual nodes, using a single ring position per physical server, produces a meaningfully less even distribution than the virtual-node approach shown here, since a small number of server positions on the ring can leave some servers responsible for a disproportionate share of the ring’s total space purely by chance.

Relying on sticky sessions indefinitely as an application scales, rather than migrating session state to a shared store once server-local state becomes a genuine scaling bottleneck, tends to produce an increasingly uneven load distribution over time, since sticky routing fundamentally limits how freely the load balancer can rebalance traffic across servers.

And configuring health checks too shallow to catch real application-level failures gives a false sense of protection, continuing to route traffic to a server that’s technically running but genuinely unable to serve requests correctly, which defeats the actual purpose any load balancing algorithm depends on to route around unhealthy servers in the first place.

FAQ