System Design Basics - Load Balancing Algorithms
The Essential Guide to Load Balancing Strategies and Techniques Visualizing Load Balancer Algorithms. Examples and Best Practices when using Load Balancers.
Hello guys, load balancing algorithms or strategies are an important piece of knowledge for any senior developer, tech lead or software architect. It allows you to distribute your load properly to ensure that your system is not dying in load and performing properly.
If you are new here, I talk about system design and software architecture regularly and in the past, I have talked about common system design concepts and software architecture components like Rate Limiter, Database Scaling, API Gateway vs Load Balancer and Horizontal vs Vertical Scaling, Forward proxy vs reverse proxy, and Single Point Failure, and in this article we will talk about different load balancing algorithms which are very important for many distributed system architecture.
For this article, I have teamed up with Hayk and we'll dive into the fundamental concepts of load balancers and different load balancing algorithms, and provide practical insights to help you make informed decisions about your Software Architecture and System Design.
By the way, if you are preparing for System design interviews and want to learn System Design in depth then you can also checkout sites like ByteByteGo, Design Guru, Exponent, Educative, Codemia.io, and Udemy which have many great System design courses
Now, let’s start with the Load balancing algorithms and how to use them on different scenarios.
🎁35% OFF For Next 3 Days
But, before we start, I just want to remind about our Women’s Day Offer which is still going on, we're offering you a limited-time offer of 35% off a paid subscription.
Instead of paying 50$ / year, you pay 32.5$ / year (only 3$ / month)!
Here are the benefits you unlock with a paid subscription:
Get access to paid subscribers posts. 📗
Access to Full archive of more than 152+ posts 🏆
Many expense it with their team's learning budget
What are Load Balancers?
Load balancers distribute incoming network traffic across multiple servers to ensure no single server bears too much load. By spreading the requests efficiently, they increase the capacity and reliability of applications.
Here are some common strategies and algorithms used in load balancing:
1. Round Robin
Round Robin is the simplest form of load balancing, where each server in the pool gets a request in a sequential, rotating order. When the last server is reached, it loops back to the first.
This works well for servers with similar specifications and when the load is uniformly distributable.
2. Least Connections
Least Connections algorithm directs traffic to the server with the fewest active connections. This is particularly useful when there are sessions of variable lengths and demands.
It is ideal for longer tasks or when the server load is not evenly distributed.
3. Least Response Time
This responsiveness-focused algorithm chooses the server with the lowest response time and with the fewest active connections.
Effective when the goal is to provide the fastest response to requests.
4. IP Hash
IP Hash determines which server receives the request based on the hash of the client’s IP address. This ensures a client consistently connects to the same server.
Useful for session persistence in applications where it’s important that a client consistently connects to the same server.
5. Weighted Algorithms
There are also variants of the above methods that can be weighted. For example, in Weighted Round Robin or Weighted Least Connections, servers are assigned weights typically based on their capacity or performance metrics.
Weighted algorithms are effective when servers in the pool have different capabilities (e.g., CPU, RAM).
6. Geographical Algorithms
These are location-based algorithms that direct requests to the server geographically closest to the user or based on specific regional requirements.
Useful for global services where latency reduction and local regulatory compliance are priorities.
7. Consistent Hashing
Consistent hashing uses a hash function to distribute data across various nodes.
Imagine a hash space that forms a circle where the end wraps around to the beginning, often referred to as a “hash ring”. Both the nodes (servers) and the data (like keys of stored values) are hashed onto this ring.
This also ensures that the same client consistently connects to the same server.
Health Checks
An essential feature of load balancers is continuous health checking of servers to ensure traffic is only directed to servers that are online and responsive. If a server fails, the load balancer will stop sending traffic to it until it is back online.
Load Balancer Examples
Load balancers come in various forms, including hardware appliances, software solutions, and cloud-based services. Here are some examples:
Hardware Load Balancers
F5 BIG-IP: A widely used hardware load balancer known for its high performance and extensive feature set, offering local traffic management, global server load balancing, and application security.
Citrix ADC: Formerly known as NetScaler, it provides load balancing, content switching, and application acceleration.
Software Load Balancers
HAProxy: A popular open-source software load balancer and proxy server for TCP and HTTP-based applications. It’s known for its efficiency, reliability, and low memory footprint.
NGINX: Often used as a web server, NGINX also functions as a load balancer and reverse proxy for HTTP and other network protocols.
Cloud-Based Load Balancers
AWS Elastic Load Balancing (ELB): Offers several types of load balancers as part of the AWS cloud services, including the Application Load Balancer, Network Load Balancer, and Classic Load Balancer.
Microsoft Azure Load Balancer: Provides high availability and network performance to applications running in Azure. It supports inbound and outbound scenarios.
Google Cloud Load Balancing: A fully distributed, software-defined, managed service for all your traffic. It offers various types, including HTTP(S), TCP/SSL, and UDP load balancing.
Virtual Load Balancers
VMware NSX Advanced Load Balancer (Avi Networks): Offers a software-defined application delivery controller that can be deployed on-premises or in the cloud.
What happens when the load balancer goes down?
The load balancer is basically a single point of failure, and in case it goes down, all of the servers become unavailable for the clients.
To avoid or minimize the impact of a load balancer failure, several strategies are typically employed:
Redundancy
Implementing redundant load balancing by using more than one load balancer, often in pairs, is a common approach. If one fails, the other takes over, a method known as failover.
Health Checks and Monitoring
Continuous monitoring and health checks of the load balancer itself can ensure that any issues are detected early and can be addressed before causing significant disruption.
Auto-scaling and Self-Healing Systems
Some modern infrastructures are designed to automatically detect the failure of a load balancer and replace it with a new instance without manual intervention.
DNS Failover
In some configurations, DNS failover can reroute traffic away from an IP address that’s no longer accepting connections (like a failed load balancer) to a preconfigured standby IP.
Thanks for the read! For weekly insights on web development that you won’t want to miss, subscribe to my newsletter.
Want to level up your system design skills with Hayk‘s personal guidance? Join his community and gain access to weekly lessons, real-world examples, and hands-on guidance.
🎁35% OFF For Next 3 Days
But, before we go, I just want to remind about our Women’s Day Offer which is still going on, we're offering you a limited-time offer of 35% off a paid subscription.
Instead of paying 50$ / year, you pay 32.5$ / year (only 3$ / month)!
Here are the benefits you unlock with a paid subscription:
Get access to paid subscribers posts. 📗
Access to Full archive of more than 152+ posts 🏆
Many expense it with their team's learning budget. You can see a sample email for reimbursement here.
Other System Design and Software Architecture Articles you may like
 | A guest post by
|