Design: How Does LoadBalancer Work?

Similar Posts:

  • Typical Features/Considerations
Name Summary
Route traffic LoadBalancer Algorithms
L3/L4 LoadBalancer vs L7 LoadBalancer link
Redundant Load Balancers Load balancer active/passive
Health checks Load balancers should only forward traffic to “healthy” backend servers
Sticky session  
SSL termination  
Service throttling  
Predictive analytics Smart load balancers can help to determine traffic bottlenecks

Q: How LoadBalancer route requests?

Typical algorithms:

Name Summary
Round robin  
Weighted round robin Servers with higher weights get more connections
Least Connection  
Least Response Time  
Least Bandwidth  
IP Hash Caculate IP hashing, then decide a server
choice-of-2 Absolute best choice using incomplete data may not be optimal
  • choice-of-2: Instead of making the absolute best choice using incomplete data, with “power of two choices” you pick two queues at random and chose the better option of the two, avoiding the worse choice.

Link: NGINX and the “Power of Two Choices” Load-Balancing Algorithm

Commonly used algorithms for HAProxy(link):

  • roundrobin: Round Robin selects servers in turns. This is the default algorithm.
  • leastconn: Selects the server with the least number of connections–it is recommended for longer sessions. Servers in the same backend are also rotated in a round-robin fashion.
  • source: This selects which server to use based on a hash of the source IP i.e. your user’s IP address. This is one method to ensure that a user will connect to the same server.

Q: What are the pros/cons of different algorithms? And typical use cases of each?


Q: L3/L4 LoadBalancer vs L7 LoadBalancer

  • L3/L4 Load Balancer: traffic is routed by IP address and port. L3 is network layer (IP). L4 is session layer (TCP).
  • L7 Load Balancer: traffic is routed by what is inside the HTTP protocol. L7 is application layer (HTTP).

Q: What are sticky sessions?

Some applications require that a user continues to connect to the same backend server. This persistence is achieved through sticky sessions, using the appsession parameter in the backend that requires it.

Q: How the Readiness Probe is done for backend services?

Often, session tracking is accomplished via a consistent hashing function.

Q: How loadbalancer enable/enable one specific backend server, per health check?


Q: Elastic IP vs loadbalancer?


Q: For the failure of loadbalancer, would people add more than one redundant load balancer?


Q: Consistent hashing in load balancing?


Configuration example in HAProxy using consistent hashing: link

balance source
hash-type consistent

Share It, If You Like It.

Leave a Reply

Your email address will not be published. Required fields are marked *