Design: How Does LoadBalancer Work?
Similar Posts:
- Typical Features/Considerations
| Name | Summary |
|---|---|
| Route traffic | LoadBalancer Algorithms |
| L3/L4 LoadBalancer vs L7 LoadBalancer | link |
| Redundant Load Balancers | Load balancer active/passive |
| Health checks | Load balancers should only forward traffic to “healthy” backend servers |
| Sticky session | |
| SSL termination | |
| Service throttling | |
| Predictive analytics | Smart load balancers can help to determine traffic bottlenecks |
Q: How LoadBalancer route requests?
Typical algorithms:
| Name | Summary |
|---|---|
| Round robin | |
| Weighted round robin | Servers with higher weights get more connections |
| Least Connection | |
| Least Response Time | |
| Least Bandwidth | |
| IP Hash | Caculate IP hashing, then decide a server |
| choice-of-2 | Absolute best choice using incomplete data may not be optimal |
- choice-of-2: Instead of making the absolute best choice using incomplete data, with “power of two choices” you pick two queues at random and chose the better option of the two, avoiding the worse choice.
Link: NGINX and the “Power of Two Choices” Load-Balancing Algorithm
Commonly used algorithms for HAProxy(link):
- roundrobin: Round Robin selects servers in turns. This is the default algorithm.
- leastconn: Selects the server with the least number of connections–it is recommended for longer sessions. Servers in the same backend are also rotated in a round-robin fashion.
- source: This selects which server to use based on a hash of the source IP i.e. your user’s IP address. This is one method to ensure that a user will connect to the same server.
Q: What are the pros/cons of different algorithms? And typical use cases of each?
A: TODO
Q: L3/L4 LoadBalancer vs L7 LoadBalancer
- L3/L4 Load Balancer: traffic is routed by IP address and port. L3 is network layer (IP). L4 is session layer (TCP).
- L7 Load Balancer: traffic is routed by what is inside the HTTP protocol. L7 is application layer (HTTP).
Q: What are sticky sessions?
Some applications require that a user continues to connect to the same backend server. This persistence is achieved through sticky sessions, using the appsession parameter in the backend that requires it.
Q: How the Readiness Probe is done for backend services?
Often, session tracking is accomplished via a consistent hashing function.
Q: How loadbalancer enable/enable one specific backend server, per health check?
A: TODO
Q: Elastic IP vs loadbalancer?
A: TODO
Q: For the failure of loadbalancer, would people add more than one redundant load balancer?
A: TODO
Q: Consistent hashing in load balancing?
A:
Configuration example in HAProxy using consistent hashing: link
balance source hash-type consistent