Skip to content
All tools

Explainer

How a load balancer handles failure

Watch requests flow through a load balancer to a pool of instances. Crash one, or send a traffic spike, and see what the system does next.

Diagram: clients send requests to a load balancer, which routes them to application instances that read from a shared database.UsersLoad balancerhealth check every 2sAHealthyBHealthyCHealthyDSpareDatabase
Served
0
Failed
0

What's happening

  1. Three instances are serving traffic. The load balancer spreads requests round-robin.

What to notice

Failures happen between health checks. When an instance crashes, the load balancer keeps sending it traffic until the next health check fails. Those requests are lost. A shorter check interval means fewer failed requests, but more checking traffic and a higher risk of removing an instance that was only briefly slow.

A restarted instance has to earn its way back. It only gets traffic again after passing a health check, so users don't hit a server that is still starting up.

Autoscaling is not instant. During a spike the existing instances absorb the extra load while a new one boots. That boot time is why you keep some headroom, and why scaling in waits a while and drains connections before removing an instance.