Load Balancers, Explained With a Grocery Store

Picture a grocery store with one checkout lane. It works fine on a Tuesday morning. On the Saturday before a holiday, the line wraps around the bakery, and if that one cashier calls in sick, nobody buys anything at all.

The fix is obvious. Open more lanes. But now you need someone near the front waving people toward whichever lane is free. That person is a load balancer.

In web terms, instead of one server handling every request for your site, you run several identical copies of your app. The load balancer sits in front, takes each incoming request, and hands it to one of them. Visitors only ever see one address. They have no idea there are four servers behind it.

You get two things out of this, and the second one is honestly the bigger deal. First, capacity: four servers can handle more traffic than one. Second, resilience: the load balancer runs health checks, regularly poking each server to make sure it's answering properly. If one stops responding, it gets pulled out of rotation and traffic goes to the others. That also means you can take one server down on purpose, update it, and put it back, with no downtime for anyone.

How it picks a server varies. Round robin just goes in order: one, two, three, four, one, two. Least connections sends the next request to whichever server is currently the least busy, which is smarter when some requests take much longer than others. Some setups hash the visitor's IP so the same person keeps landing on the same server.

That last one hints at the catch. If your app keeps anything in local memory or on local disk (login sessions, uploaded files, a cache), it breaks the moment a visitor bounces to a different server. Suddenly they're logged out, or their upload has vanished. "Sticky sessions" paper over this by pinning a visitor to one server, but the cleaner fix is to move that state somewhere shared: sessions in a database or Redis, uploads in object storage. Then any server can handle any request, which is the whole point.

If you've used a reverse proxy, a lot of this will sound familiar, and that's because they overlap heavily. Nginx, HAProxy, Caddy and Traefik can all do load balancing. Cloud providers sell it as a managed service too. You'll also hear "layer 4" versus "layer 7." Layer 4 balancers just shuffle network connections around without looking inside them. Layer 7 balancers understand HTTP, so they can route by hostname or path, terminate HTTPS, and so on. For websites, you'll almost always be dealing with layer 7.

Do you need one? For a personal site or a small app on a single decent server, probably not. One well-configured machine handles a surprising amount of traffic, and a load balancer adds moving parts. It starts making sense when downtime actually costs you something, or when one server genuinely can't keep up. And remember the load balancer itself can become the single point of failure, which is why serious setups run more than one.

If you're curious, though, it's a great thing to try on a spare box. Spin up two copies of a small app in Docker, put Caddy or Nginx in front with both as upstreams, then stop one container and watch the site keep working. It's one of those things that clicks much faster once you've seen it.