• What is Load Balancing?

Question

Answer:
Load balancing is a technique for distributing incoming network
traffic across multiple backend servers or clusters to optimize
resource utilization, maximize throughput, minimize response time,
and avoid overloading any single resource. Using load balancing
helps ensure high availability and reliability of applications by
providing the flexibility to add or subtract backend servers as
needed.

Some common load balancing algorithms:

o Round Robin: Requests are distributed across backend

servers sequentially in a circular order. This is a simple
implementation that spreads requests evenly across servers.
o Weighted Round Robin: Servers are assigned a weight
(based on capacity), and requests are distributed
proportionally to server weight. This accounts for
heterogeneous hardware with varying processing capacities.
o Least Connections: Directs traffic to the server with the
fewest active connections. This approach is useful when
there are large variations in server processing speeds.
o IP Hash: A hash of the client IP address is calculated to direct
requests to the same backend server each time. Useful for
applications that require state to be maintained at the
backend.
Benefits of Load Balancing:
o Increased application availability and reliability by mitigating
against server failure.

o Improved resource utilization by preventing any single server

from being overloaded.

o Enhanced ability to handle increasing traffic and user load by

scaling horizontally.

o Reduced response times by balancing load across multiple

servers.
 Implementation:
Load balancers are implemented through hardware (dedicated
appliances) or software (load balancer processes).

A load balancer sits in front of the backend servers and acts as a

reverse proxy, receiving and forwarding client requests to the
appropriate server based on the balancing algorithm. Common
load balancing methods for web applications:

o Round Robin: Requests distributed sequentially to servers

o Session Affinity/Sticky Sessions: Requests from same client

session go to same backend server

o IP Hash: Client IP hashed and mapped to backend server

Overall, load balancing is a critical component of building reliable,

scalable applications by distributing work evenly across available
compute resources.

• 2.
What is Round-Robin Load Balancing?
Question

Answer:
Round-robin load balancing is one of the simplest algorithms for
distributing incoming requests evenly across multiple backend
servers.

With round-robin LB, the load balancer maintains an ordered list of

backend servers. It iterates through this list sequentially, forwarding
each new request to the next server. When it reaches the end of the
list, it loops back to the beginning.

This ensures requests are distributed evenly across all backend

servers in a circular fashion.

How Round-Robin LB Works

1. Load balancer maintains an ordered list of backend servers (S1, S2,
S3...)

2. LB receives a client request

3. Request forwarded to the next server in the list (S1)

4. LB receives another request

5. Forwarded to the next server (S2)

6. Process continues round-robin through the server list

7. After last server, loops back to first server

Benefits:
o Simple to implement

o Requests distributed evenly across backends

o Prevents one server from being overloaded

Drawbacks:
o Stateless - does not consider individual server load

o Traffic is distributed sequentially regardless of server

capacity

o Could lead to uneven loads if server capacities are different

Overall, round-robin LB is easy to implement and provides basic

load distribution. However, more advanced algorithms like
weighted round-robin or least connections are needed for
production systems.
 • 3.
What are the key advantages of Round-Robin Load Balancing
Question

Answer:

Advantages of Round-Robin Load Balancing

o Simple to implement - Round-robin does not require

complex algorithms, so it is easy to code and integrate into
systems.
o Evenly distributes requests - Each backend server receives
an equal number of requests over time, preventing one
server from being overloaded.
o Stateless - The load balancer does not have to maintain or
check any session state information when balancing
requests.
o Prevents request flooding - Requests are distributed across
backends sequentially, avoiding massive spikes in traffic to
any single server.
o Provides basic load distribution - For simple applications,
round-robin can provide adequate load balancing without
complex server monitoring.
o Cost effective - Does not require expensive hardware or
software to implement. A basic round-robin balancer can be
developed in-house.
o Easy to add/remove servers - No complicated reconfiguring
needed when adding or removing backend servers.
However, round-robin load balancing also has drawbacks like not
accounting for individual server load or capacities. More advanced
algorithms are needed for complex, production environments. But
for simple applications, round-robin provides a cost-effective way
to introduce basic load distribution.

Different Categories of Load Balancing

Generally, load balancers are grouped into three categories…
1. Layer 4 (L4) Load Balancer
In the OSI model layer 4 is the transport layer(TCP/SSL) where the routing
decisions are made. Layer 4 load balancer is also referred to as Network
Load Balancing and as the name suggests it leverages network layer
information to make the routing decision for the traffic. It can control millions
of requests per second and it handles all forms of TCP/UDP traffic. The
decision will be based on the TCP or UDP ports that packets use along with
their source and destination IP addresses. The L4 load balancer also performs
Network Address Translation (NAT) on the request packet but it doesn’t
inspect the actual contents of each packet. This category of load balancer
maximizes utilization and availability by distributing the traffic across IP
addresses, switches, and routers.
2. Layer 7 (L7) Load Balancer
Layer 7 load balancer is also referred to as Application Load
Balancer or HTTP(S) Load Balancer. It is one of the oldest forms of load
balancing. In the OSI model, Layer 7 is the application layer (HTTP/HTTPS)
where the routing decisions execute. Layer 7 adds content switching to load
balancing and it uses information such as HTTP header, cookies, uniform
resource identifier, SSL session ID, and HTML form data to decide the routing
request across the servers.
3. Global Server Load Balancing (GSLB)
Today a lot of applications are hosted in cloud data centers in multiple
geographic locations. This is the reason a lot of organizations are moving to
a different load balancer that can deliver applications with greater reliability
and lower latency to any device or location. With the significant change in the
capability of the load balancers, GSLB fulfills these expectations of IT
organizations. GSLB extends the capability of L4 and L7 servers in different
geographic locations and distributes a large amount of traffic across multiple
data centers efficiently. It also ensures a consistent experience for end-users
when they are navigating multiple applications and services in a digital
workspace.
Load Balancing Algorithms
We need a load-balancing algorithm to decide which request should be
redirected to which backend server. The different system uses different ways
to select the servers from the load balancer. Companies use varieties of load-
balancing algorithm techniques depending on the configuration. Some of the
common load-balancing algorithms are given below:
1. Round Robin
Requests are distributed across the servers in a sequential or rotational
manner. For example, the first request goes to the first server, the second one
goes to the second server, the third request goes to the third server and it
continues further for all the requests. It is easy to implement but it doesn’t
consider the load already on a server so there is a risk that one of the servers
receives a lot of requests and becomes overloaded.
2. Weighted Round Robin
It is much similar to the round-robin technique. The only difference is, that
each of the resources in a list is provided a weighted score. Depending on the
weighted score the request is distributed to these servers. So in this method,
some of the servers get a bigger share of the overall request.
3. Least Connection Method
In this method, the request will be directed to the server with the fewest
number of requests or active connections. To do this load balancer needs to
do some additional computing to identify the server with the least number of
connections. This may be a little bit costlier compared to the round-robin
method but the evaluation is based on the current load on the server. This
algorithm is most useful when there is a huge number of persistent
connections in the traffic unevenly distributed between the servers.
4. Least Response Time Method
This technique is more sophisticated than the Least connection method. In
this method, the request is forwarded to the server with the fewest active
connections and the least average response time. The response time taken
by the server represents the load on the server and the overall expected user
experience.
5. Source IP Hash
In this method, the request is sent to the server based on the client’s IP
address. The IP address of the client and the receiving compute instance are
computed with a cryptographic algorithm.
How to Use Load Balancing During System Design Interviews?
In your system design interview, you’ll be asked some sort of scalability
question where you’ll have to explain how load balancers help distribute the
traffic and how it ensures scalability and availability of services in your
application. The overall concept that you need to keep in mind from this
article is…
• A load balancer enables elastic scalability which improves the
performance and throughput of data. It allows you to keep many
 copies of data (redundancy) to ensure the availability of the
system. In case a server goes down or fails you’ll have the backup
to restore the services.
• Load balancers can be placed at any software layer.
• Many companies use both hardware and software to implement
load balancers, depending on the different scale points in their
system.
Where Are Load Balancers Typically Placed?
Below is the image where a load balancer can be placed…

• In between the client application/user and the server

• In between the server and the application/job servers
• In between the application servers and the cache servers
• In between the cache servers the database servers

What are the types of load balancing technology?

Load balancers are one of two types: hardware load balancer and software load balancer.

Hardware load balancers

A hardware-based load balancer is a hardware appliance that can securely process and redirect
gigabytes of traffic to hundreds of different servers. You can store it in your data centers and use
virtualization to create multiple digital or virtual load balancers that you can centrally manage.

Software load balancers

Software-based load balancers are applications that perform all load balancing functions. You
can install them on any server or access them as a fully managed third-party service.

Comparison of hardware balancers to software load balancers

Hardware load balancers require an initial investment, configuration, and ongoing maintenance.
You might also not use them to full capacity, especially if you purchase one only to handle peak-
time traffic spikes. If traffic volume increases suddenly beyond its current capacity, this will affect
users until you can purchase and set up another load balancer.

In contrast, software-based load balancers are much more flexible. They can scale up or down
easily and are more compatible with modern cloud computing environments. They also cost less
to set up, manage, and use over time.
What are the types of load balancing?
We can classify load balancing into three main categories depending on what the load balancer
checks in the client request to redirect the traffic.

Application load balancing

Complex modern applications have several server farms with multiple servers dedicated to a
single application function. Application load balancers look at the request content, such as HTTP
headers or SSL session IDs, to redirect traffic.

For example, an ecommerce application has a product directory, shopping cart, and checkout
functions. The application load balancer sends requests for browsing products to servers that
contain images and videos but do not need to maintain open connections. By comparison, it
sends shopping cart requests to servers that can maintain many client connections and save cart
data for a long time.

Network load balancing

Network load balancers examine IP addresses and other network information to redirect traffic
optimally. They track the source of the application traffic and can assign a static IP address to
several servers. Network load balancers use the static and dynamic load balancing algorithms
described earlier to balance server load.

Global server load balancing

Global server load balancing occurs across several geographically distributed servers. For
example, companies can have servers in multiple data centers, in different countries, and in
third-party cloud providers around the globe. In this case, local load balancers manage the
application load within a region or zone. They attempt to redirect traffic to a server destination
that is geographically closer to the client. They might redirect traffic to servers outside the client’s
geographic zone only in case of server failure.

DNS load balancing

In DNS load balancing, you configure your domain to route network requests across a pool of
resources on your domain. A domain can correspond to a website, a mail system, a print server,
or another service that is made accessible through the internet. DNS load balancing is helpful for
maintaining application availability and balancing network traffic across a globally distributed pool
of resources.

How does load balancing work?

Companies usually have their application running on multiple servers. Such a server
arrangement is called a server farm. User requests to the application first go to the load balancer.
The load balancer then routes each request to a single server in the server farm best suited to
handle the request.

Load balancing is like the work done by a manager in a restaurant. Consider a restaurant with
five waiters. If customers were allowed to choose their waiters, one or two waiters could be
overloaded with work while the others are idle. To avoid this scenario, the restaurant manager
assigns customers to the specific waiters who are best suited to serve them.
What are the benefits of load balancing?
Load balancing directs and controls internet traffic between the application servers and their
visitors or clients. As a result, it improves an application’s availability, scalability, security, and
performance.

Application availability

Server failure or maintenance can increase application downtime, making your application
unavailable to visitors. Load balancers increase the fault tolerance of your systems by
automatically detecting server problems and redirecting client traffic to available servers. You can
use load balancing to make these tasks easier:

• Run application server maintenance or upgrades without application downtime

• Provide automatic disaster recovery to backup sites
• Perform health checks and prevent issues that can cause downtime
Application scalability

You can use load balancers to direct network traffic intelligently among multiple servers. Your
applications can handle thousands of client requests because load balancing does the following:

• Prevents traffic bottlenecks at any one server

• Predicts application traffic so that you can add or remove different servers, if needed
• Adds redundancy to your system so that you can scale with confidence
Application security

Load balancers come with built-in security features to add another layer of security to your
internet applications. They are a useful tool to deal with distributed denial of service attacks, in
which attackers flood an application server with millions of concurrent requests that cause server
failure. Load balancers can also do the following:

• Monitor traffic and block malicious content

• Automatically redirect attack traffic to multiple backend servers to minimize impact
• Route traffic through a group of network firewalls for additional security
Application performance

Load balancers improve application performance by increasing response time and reducing
network latency. They perform several critical tasks such as the following:

• Distribute the load evenly between servers to improve application performance

• Redirect client requests to a geographically closer server to reduce latency

• Ensure the reliability and performance of physical and virtual computing resources
What are load balancing algorithms

Effective load balancers intelligently determine which device

within a given server farm is best able to process an incoming
data packet. Doing so requires algorithms programmed to
distribute loads in a specific way.

Algorithms vary widely, depending on whether a load is

distributed on the network or application layer. Algorithm
selection impacts the effectiveness of load distribution
mechanisms and, consequently, performance and business
continuity.

Here we will be discussing the pros and cons of several widely

used algorithms found in both network and application layer load
balancing solutions.

Network vs Application layer

Network layer and application layer algorithms differ in how

they’re able to analyze incoming traffic and the criteria they use
to distribute traffic loads.

Network layer Application

algorithms layer
algorithms

Distribution logic Statistical/randomized Data

driven

Suitable Symmetrical/even All types

infrastructure
types

Server load No Yes

visibility
 Session Yes Yes
persistence

Unpredictable No Yes
load handling

Network layer algorithms

The main challenge to network layer load balancers is a lack of

visibility into traffic flow, limited to information stored in network
packet headers. Routing decisions have to be based on only a few
factors—primarily source and destination IP data.

Network layer load balancers cannot assess the nature of

incoming requests, their expected load generation and available
server resources at a given time. A certain amount of
guesstimation is needed for them to make routing decisions.

Examples of network layer algorithms include:

• Round robin – A batch of servers are programmed to

handle load in a rotating sequential manner. The algorithm
assumes that each device is able to process the same
number of requests and isn’t able to account for active
connections.
• Weighted round robin – Servers are rated based on the
relative amount of requests each is able to process. Those
having higher capacities are sent more requests.
• Least connections – Requests are sent to the server
having the fewest number of active connections, assuming
all connections generate an equal amount of server load.
• Weighted least connections – Servers are rated based on
their processing capabilities. Load is distributed according to
both the relative capacity of the servers and the number of
active connections on each one.
• Source IP hash – Combines the source and destination IP
address in a request to generate a hash key, which is then
 designated to a specific server. This lets a dropped
connection be returned to the same server originally
handling it.

While these algorithms are adequate in predictable traffic

scenarios, they’re not as effective in dealing with
uneven/unexpected server loads.

Application layer algorithms

Application layer load balancers distribute requests based on

content of the requests being processed, including its HTTP/S
header and message in addition to session cookies. They can also
track responses as they travel back from the server, thereby
providing data on the load each server is processing at all times.

As opposed to the speculative nature of network load balancing,

application load balancing is data driven. This provides the
intelligent distribution of incoming requests.

The most notable application layer algorithm is least pending

requests (LPR). It monitors pending HTTP/S requests and
distributes them to the most available server. LPR can instantly
adjust to a sudden influx of new connections while continuously
monitoring the workload of all servers within a server farm.

LPR benefits include:

• Accurate load distribution – Unlike network layer

algorithms distributing requests according to preset rules,
LPR intelligently selects the server best suited to process an
incoming connection in real-time.
• Request specific distribution – LPR can acknowledge that
connection requests take different processing times and
distributes load accordingly. As a result, traffic isn’t routed
to busy servers.

Choosing the right load balancing algorithm

There are a number of considerations in selecting a load

balancing algorithm. Specifically, your choice needs to be able to
handle both predictable scenarios (e.g., typical traffic flows), and
unpredictable ones (e.g., a sudden request influx that creates a
heavy load).

Unpredictable scenarios, which network layer algorithms are

incapable of handling, are the most concerning to site operators.
Additionally, TTL related delays make network load balancing a
less than optimal solution.

Application layer algorithms, on the other hand, help to maintain

optimal website performance during both predictable and
unpredictable scenarios.

Load balancing is an essential technique used in cloud computing to

optimize resource utilization and ensure that no single resource is
overburdened with traffic. It is a process of distributing workloads across
multiple computing resources, such as servers, virtual machines, or
containers, to achieve better performance, availability, and scalability.
1. In cloud computing, load balancing can be implemented at various
levels, including the network layer, application layer, and database
layer. The most common load balancing techniques used in cloud
computing are:
2. Network Load Balancing: This technique is used to balance the
network traffic across multiple servers or instances. It is
implemented at the network layer and ensures that the incoming
traffic is distributed evenly across the available servers.
3. Application Load Balancing: This technique is used to balance the
workload across multiple instances of an application. It is
implemented at the application layer and ensures that each
instance receives an equal share of the incoming requests.
4. Database Load Balancing: This technique is used to balance the
workload across multiple database servers. It is implemented at
the database layer and ensures that the incoming queries are
distributed evenly across the available database servers.
Load balancing helps to improve the overall performance and reliability of
cloud-based applications by ensuring that resources are used efficiently and
that there is no single point of failure. It also helps to scale applications on
demand and provides high availability and fault tolerance to handle spikes
in traffic or server failures.
Sure, here are some advantages and disadvantages of load balancing in
cloud computing:

Advantages:

1. Improved Performance: Load balancing helps to distribute the

workload across multiple resources, which reduces the load on
each resource and improves the overall performance of the system.
2. High Availability: Load balancing ensures that there is no single
point of failure in the system, which provides high availability and
fault tolerance to handle server failures.
3. Scalability: Load balancing makes it easier to scale resources up or
down as needed, which helps to handle spikes in traffic or changes
in demand.
4. Efficient Resource Utilization: Load balancing ensures that
resources are used efficiently, which reduces wastage and helps to
optimize costs.

Disadvantages:

1. Complexity: Implementing load balancing in cloud computing can

be complex, especially when dealing with large-scale systems. It
requires careful planning and configuration to ensure that it works
effectively.
2. Cost: Implementing load balancing can add to the overall cost of
cloud computing, especially when using specialized hardware or
software.
3. Single Point of Failure: While load balancing helps to reduce the
risk of a single point of failure, it can also become a single point of
failure if not implemented correctly.
4. Security: Load balancing can introduce security risks if not
implemented correctly, such as allowing unauthorized access or
exposing sensitive data.
Overall, the benefits of load balancing in cloud computing outweigh the
disadvantages, as it helps to improve performance, availability, scalability,
and resource utilization. However, it is important to carefully plan and
implement load balancing to ensure that it works effectively and does not
introduce additional risks.
Cloud load balancing is defined as the method of splitting workloads and
computing properties in a cloud computing. It enables enterprise to manage
workload demands or application demands by distributing resources among
numerous computers, networks or servers. Cloud load balancing includes
holding the circulation of workload traffic and demands that exist over the
Internet. As the traffic on the internet growing rapidly, which is about 100%
annually of the present traffic. Hence, the workload on the server growing
so fast which leads to the overloading of servers mainly for popular web
server. There are two elementary solutions to overcome the problem of
overloading on the servers-
• First is a single-server solution in which the server is upgraded to
a higher performance server. However, the new server may also be
overloaded soon, demanding another upgrade. Moreover, the
upgrading process is arduous and expensive.
• Second is a multiple-server solution in which a scalable service
system on a cluster of servers is built. That’s why it is more cost
effective as well as more scalable to build a server cluster system
for network services.
Load balancing is beneficial with almost any type of service, like HTTP,
SMTP, DNS, FTP, and POP/IMAP. It also rises reliability through
redundancy. The balancing service is provided by a dedicated hardware
device or program. Cloud-based servers farms can attain more precise
scalability and availability using server load balancing. Load balancing
solutions can be categorized into two types –
1. Software-based load balancers: Software-based load balancers
run on standard hardware (desktop, PCs) and standard operating
systems.
2. Hardware-based load balancer: Hardware-based load balancers
are dedicated boxes which include Application Specific Integrated
Circuits (ASICs) adapted for a particular use. ASICs allows high
speed promoting of network traffic and are frequently used for
transport-level load balancing because hardware-based load
balancing is faster in comparison to software solution.
Major Examples of Load Balancers –
1. Direct Routing Requesting Dispatching Technique: This approach
of request dispatching is like to the one implemented in IBM’s Net
 Dispatcher. A real server and load balancer share the virtual IP
address. In this, load balancer takes an interface constructed with
the virtual IP address that accepts request packets and it directly
routes the packet to the selected servers.
2. Dispatcher-Based Load Balancing Cluster: A dispatcher does
smart load balancing by utilizing server availability, workload,
capability and other user-defined criteria to regulate where to send
a TCP/IP request. The dispatcher module of a load balancer can
split HTTP requests among various nodes in a cluster. The
dispatcher splits the load among many servers in a cluster so the
services of various nodes seem like a virtual service on an only IP
address; consumers interrelate as if it were a solo server, without
having an information about the back-end infrastructure.
3. Linux Virtual Load Balancer: It is an opensource enhanced load
balancing solution used to build extremely scalable and extremely
available network services such as HTTP, POP3, FTP, SMTP, media
and caching and Voice Over Internet Protocol (VoIP). It is simple
and powerful product made for load balancing and fail-over. The
load balancer itself is the primary entry point of server cluster
systems and can execute Internet Protocol Virtual Server (IPVS),
which implements transport-layer load balancing in the Linux
kernel also known as Layer-4 switching.