Load Balancers: When You Actually Need One

A load balancer sits in front of several servers and distributes requests
between them. It solves two problems that are easy to conflate: having enough
capacity, and surviving the loss of one machine. Most people who think they need
one for the first reason actually want it for the second.

The two reasons

Capacity. One server can no longer handle the traffic, so
you add more and spread the load. Before reaching for this, check whether a
larger single server would do, because it is simpler in every respect. See
scaling up vs scaling out.

Availability. One server is a single point of failure. With
two behind a load balancer that health-checks both, losing one degrades capacity
instead of causing an outage. This is usually the more valuable reason, and it
applies even when a single server has ample capacity.

Being clear about which you are buying changes the configuration. For
availability, both servers must be able to carry the full load alone, otherwise
losing one simply moves the outage.

What you get beyond distribution

• Health checks, removing an unresponsive server from
  rotation automatically. This is the core of the availability benefit.
• TLS termination, handling certificates in one place rather
  than on every machine.
• A stable entry point, so servers can be replaced without
  touching DNS.
• Zero-downtime deployment, by removing one server at a time
  from rotation, updating it, and returning it.

That last one is frequently the first real benefit people notice, because it
changes deployment from an event into a routine.

What your application needs to support

Putting a load balancer in front of an application that assumed one server
exposes several assumptions:

• Sessions must not live in one server’s memory. Use a shared
  store, or make requests stateless. Sticky sessions are available on most load
  balancers and are a workaround rather than a fix, since losing a server still
  loses those sessions.
• Uploads must not be written to local disk. Move them to
  object storage, covered in
  object storage vs block
  storage.
• Scheduled jobs must run on exactly one machine.
• The real client IP address arrives in a forwarded header
  rather than in the connection, which affects logging and any rate limiting.

When you do not need one

Be honest about this, because a load balancer adds a component, a cost and a
thing that can be misconfigured.

You do not need one if a single appropriately sized server handles your
traffic comfortably and a short outage is acceptable. That describes a great
many internal tools, brochure sites, small SaaS products and personal projects.

You do not need one to serve static assets faster. A CDN does that better.

And you do not need one to enable HTTPS. A single server handles TLS fine.

A cheaper first step

If the motivation is availability but the cost or complexity is unwelcome,
there is a middle option: keep one server, but make it quickly rebuildable.
Infrastructure defined in code, automated configuration, tested backups and a
reserved IP address that can be moved to a replacement machine.

This gives recovery in minutes rather than seconds. For many systems that is
genuinely sufficient, and it is considerably simpler than running two of
everything.

If you do add one

1. Run at least two backends. One behind a load balancer has all the cost and
   none of the benefit.
2. Make the health check meaningful. Request a path that touches the database,
   not a static file that will keep returning success while the application is
   broken.
3. Place backends in different availability zones where possible, so a
   facility failure does not take both.
4. Terminate TLS at the load balancer and keep certificate management in one
   place.
5. Check that your application reads the forwarded client address, before your
   logs become useless.
6. Test by deliberately stopping one backend and confirming that traffic
   continues. An untested failover is an assumption.