Almost every system design candidate draws a load balancer. A box labeled "LB" sits between the clients and the app servers, an arrow fans out to three identical rectangles, and then we move on to the database.
The interviewer rarely stops you there. That's the problem. The box looks done, so you skip the layer underneath it, and the gap shows up two questions later when they ask "okay, one of those servers is hot, why?" and you have nothing.
This is the layer to actually understand. Not because it's hard, but because it's easy to fake until someone pushes.
What a load balancer is actually deciding
A load balancer takes incoming requests and picks a backend to send each one to. That's it. Every interesting question is about how it picks and what it knows when it picks.
Two things drive every answer: which layer it operates at, and which algorithm it uses to choose.
L4 vs L7, in plain terms
L4 (transport layer) balances on TCP/UDP. It sees IP addresses and ports. It does not open the request to read what's inside. It picks a backend, then shovels bytes back and forth. It's fast and cheap because it isn't parsing anything.
L7 (application layer) balances on the actual HTTP request. It can read the path, the headers, the cookies, the method. So it can send /api/video to one pool and /api/search to another, do TLS termination, retry a failed request, or pin a user to a server with a cookie.
The trade is straightforward. L4 is faster and dumber. L7 is slower and smarter. Most web systems you'll design in an interview want L7 at the edge, because routing by path and terminating TLS in one place is worth the cost. L4 shows up when you need raw throughput, or in front of the L7 layer itself.
If you remember one line: L4 routes connections, L7 routes requests. When the interviewer asks "how does it know to send image uploads to the upload service," you need L7, and you should say why.
The algorithms, and when each breaks
This is where follow-ups land. Naming "round robin" isn't enough. Know how each one fails.
Round robin sends request 1 to A, 2 to B, 3 to C, then back to A. Simple and fine when every request costs about the same and every server is identical. It breaks when requests aren't uniform. One slow request to A while B and C handle ten fast ones each, and A falls behind even though it got "its share."
Least connections sends the next request to whichever backend has the fewest open connections. This handles uneven request durations much better, which is why it's a good default for anything with long-lived or variable work. It's the answer when someone says "but some requests take way longer than others."
Weighted versions of either let you account for servers that aren't identical - a bigger box gets a higher weight and more traffic. Useful during a migration when old and new instance types coexist.
Hashing (consistent hashing in particular) routes by a key, usually the client IP or a user ID. Same key, same backend, every time. You reach for this when a server holds state for a user - a cache, a session, a websocket. The honest caveat: hashing fights even distribution. One heavy user can hammer one backend, and you've built a hotspot. That's often the exact "why is one server hot" answer the interviewer is fishing for.
There's no best algorithm. There's a best fit for the traffic shape, and naming the trade is what separates a real answer from a memorized one.
Health checks are the part people forget
A load balancer that keeps sending traffic to a dead server is worse than no load balancer. Health checks are how it knows who's alive.
The cheap version pings a port. The useful version hits a real endpoint - /healthz - that returns 200 only when the server can actually serve. The difference matters: a process can be up, accepting connections, and still broken because its database connection died. A good health check catches that.
When a check fails, the LB pulls that backend from rotation and keeps probing. When it recovers, it goes back in. Mention this unprompted and you've answered the failover question before it's asked.
One more thing worth saying out loud: the load balancer itself is a single point of failure. Real systems run more than one, usually behind DNS or a floating IP. You don't need to architect the whole HA setup in an interview, but noticing the LB can die is a point in your favor.
How to practice this
Next time you whiteboard a design, don't let the LB box stay a box. Say the layer (L7 at the edge), say the algorithm and why (least connections, because request times vary), and say what happens when a backend dies (health check pulls it). Three sentences.
Do that a few times in a mock and it stops being the layer you skip. It becomes thirty seconds where you sound like you've run something in production, which is the whole point of the question.
If the interviewer keeps pushing past those three sentences, the load balancer internals walkthrough covers what comes next: L4 vs L7 in depth, connection draining, and how Envoy, HAProxy and ALB differ.