Lesson 2: Load Balancing in

Cloud Computing

By Dr. Gaddisa Olani ([email protected])

By Dr. Gaddisa Olani ([email protected])
Overview
• What is Load Balancing
• Why load balance
• What services should you load balance
• What are some common load balancing algorithms
• What are some open source load balancing technologies

By Dr. Gaddisa Olani ([email protected])

Definition
• Load balancing is the process of improving the performance of a
parallel and distributed system through a redistribution of load
among the processors.

By Dr. Gaddisa Olani ([email protected])

What does a load balancer do?
• Distributes client requests efficiently across multiple servers

• Ensures high availability and reliability by sending requests only to

servers that are online

• Provides the flexibility to add or remove servers as demand dictates

• Improves security by protecting against denial-of-service attacks

By Dr. Gaddisa Olani ([email protected])

Types of load balancers
• Software load-balancers

• Hardware load-balancers

• Switches with extended functionality

By Dr. Gaddisa Olani ([email protected])

Hardware Load Balancers

• These are hardware appliances built using accelerators such as

application-specific integrated circuits (ASICs) or field-programmable
gate arrays (FPGAs). This reduces the burden on the main CPU
processing requests. You can buy them from vendors such as F5 or
Citrix and deploy them on your premises. They’re strong performers,
but require hands-on management, and because they’re hardware
based, they can be difficult to scale, decreasing the flexibility of the
system.

By Dr. Gaddisa Olani ([email protected])

Software Load Balancers

• Software load balancers can be deployed and managed on your own

servers. There are both commercial and open source software load
balancers. Software load balancing is very flexible, and can be easily
deployed on commodity servers, co-exist with other services not
requiring dedicated servers, and offer easy instrumentation and
debugging capabilities.

By Dr. Gaddisa Olani ([email protected])

Load balancers can also be distinguished by the kind of requests
they handle.

• Following the OSI model, load

balancers are implemented in either
layer 4 or layer 7. A request in a
load balancer is TCP or UDP
connections for L4 load balancers,
or HTTP requests for L7 load
balancers. Choosing the right layer
of load balancers depends primarily
on your application’s use case.

By Dr. Gaddisa Olani ([email protected])

Open Source Load Balancers
• Open source software has become synonymous with high quality, reliability, flexibility, and
security. Load balancers are no exception to this. While the closed source load balancers
offered by AWS, GCP, and Azure are fast and reliable, they offer limited customization and
can cause lock-in.

• On the other hand, open source load balancers provide best-in-class performance and
multiple deployment options, including both cloud and on-prem support. In addition,
popular open source software projects have the advantage of large communities driving
their use.

• This is especially useful for crucial components such as load balancers, making it easy to
build a production-ready setup using established best practices and community knowledge.

By Dr. Gaddisa Olani ([email protected])

Open Source Load Balancers

By Dr. Gaddisa Olani ([email protected])

Load balancing: The big picture

By Dr. Gaddisa Olani ([email protected])

Load balancing: The big picture

By Dr. Gaddisa Olani ([email protected])

Why Load Balance
• Improve performance
• Improve redundancy
• More cost effective scaling
• Security / IPS / Choke Point

By Dr. Gaddisa Olani ([email protected])

 How SLB Devices Make Decisions
• The SLB device can make its load-balancing decisions based on several factors.
• Some of these factors can be obtained from the packet headers (i.e., IP address, port
numbers, etc.).
• Other factors are obtained by looking at the data beyond the network headers. Examples:
• HTTP Cookies
• HTTP URLs
• SSL Client certificate

• The decisions can be based strictly on flow counts or they can be based on
knowledge of application.
• For some protocols, like FTP, you have to have knowledge of protocol to correctly
load-balance (i.e., control and data connection must go to same physical server).

By Dr. Gaddisa Olani ([email protected])

Load-Balancing Algorithms
• Round robin
• weighted round robin
• least connections:
• weighted least connections

• There are other algorithms that look at or try to

predict server load in determining the load of the
real server.
By Dr. Gaddisa Olani ([email protected])
Round Robin Load Balancing
• In a nutshell, round robin network load balancing rotates connection
requests among web servers in the order that requests are received.

By Dr. Gaddisa Olani ([email protected])

Round Robin Load Balancing

By Dr. Gaddisa Olani ([email protected])

Round Robin Load Balancing
• In a nutshell, round robin network load balancing rotates connection
requests among web servers in the order that requests are received.

• For a simplified example, assume that an enterprise has a cluster of

three servers: Server A, Server B, and Server C.

The first request is sent to Server A.

The second request is sent to Server B.
The third request is sent to Server C.
By Dr. Gaddisa Olani ([email protected])
Issues with RRLB
• The biggest drawback of using the round robin algorithm in load
balancing is that the algorithm assumes that servers are similar
enough to handle equivalent loads.

• If certain servers have more CPU, RAM, or other specifications, the

algorithm has no way to distribute more requests to these servers.

• As a result, servers with less capacity may overload and fail more
quickly while capacity on other servers lie idle.

By Dr. Gaddisa Olani ([email protected])

Weighted Load Balancing
• The weighted round robin load balancing algorithm allows site
administrators to assign weights to each server based on criteria like
traffic-handling capacity. Servers with higher weights receive a higher
proportion of client requests.

• For a simplified example, assume that an enterprise has a cluster of

three servers:
• Server A can handle 15 requests per second, on average
• Server B can handle 10 requests per second, on average
• Server C can handle 5 requests per second, on average

By Dr. Gaddisa Olani ([email protected])

Weighted Load Balancing
• Let say, server A can take 3 requests per second, server B can take 2
requests per second on an average, and server C can take 1 requests
per second.

• So the load balancer will assign a weight to the server A=3,B=2,C=1.You

can see the diagram below

By Dr. Gaddisa Olani ([email protected])

Weighted Load Balancing

By Dr. Gaddisa Olani ([email protected])

 least connections Algorithms

 On server A the client's 1 and 2 requests

are not disconnected yet

 On server B client's 3 request are not

disconnected.

 But the client's 4,5 & 6 requests are

already disconnected.

If the underlying load balancing algorithm is Round Robin, if the new request to which server the request will be
forwarded?
Yes, To Server A. Now from here, we can see loads on server A to pile up and server A resources may be
exhausted quickly. By Dr. Gaddisa Olani ([email protected])
least connections Algorithms

Least connection load balancing is a dynamic load balancing

algorithm where client requests are distributed to the application
server with the least number of active connections at the time the
client request is received.

By Dr. Gaddisa Olani ([email protected])

least connections Algorithms

By Dr. Gaddisa Olani ([email protected])

least connections Algorithms

By Dr. Gaddisa Olani ([email protected])

weighted least connections

• weighted least connections: associate a weight / strength for each

server and distribute load across server farm based on the weights of
all servers in the farm.

By Dr. Gaddisa Olani ([email protected])

Source IP Hash load balancing

• Source IP Hash load balancing uses an algorithm that takes the source
and destination IP address of the client and server to generate a
unique hash key. This key is used to allocate the client to a particular
server. As the key can be regenerated if the session is broken, this
method of load balancing can ensure that the client is directed to the
same server that it was using previously. This is useful if it’s important
that a client should connect to a session that is still active after a
disconnection and reconnection.

By Dr. Gaddisa Olani ([email protected])

 Load balancing metrics
• Throughput: This metric is used to calculate the number of processes completed per unit
time.
• Response time: It measures the total time that the system takes to serve a submitted
task.
• Makespan: This metric is used to calculate the maximum completion time or the time
when the resources are allocated to a user.
• Scalability: It is the ability of an algorithm to perform uniform load balancing in the
system according to the requirements upon increasing the number of nodes. The
preferred algorithm is highly scalable.
• Fault tolerance: It determines the capability of the algorithm to perform load balancing
in the event of some failures in some nodes or links.
• Migration time: The amount of time required to transfer a task from an overloaded node
to an under-loaded one.
By Dr. Gaddisa Olani ([email protected])
Load balancing metrics cont.…
• Degree of imbalance: This metric measures the imbalance among VMs.

• Energy consumption: It calculates the amount of energy consumed by

all nodes. Load balancing helps to avoid overheating and therefore
reducing energy usage by balancing the load across all the nodes.

• Carbon emission: It calculates the amount of carbon produced by all

resources. Load balancing has a key role in minimizing this metric by
moving loads from underloaded nodes and shutting them down.

By Dr. Gaddisa Olani ([email protected])

SLB: Architectures
• Traditional
• SLB device sits between the Clients and the Servers being load-balanced.
• Distributed
• SLB device sits off to the side, and only receives the packets it needs to based
on flow setup and tear down.

By Dr. Gaddisa Olani ([email protected])

SLB: Traditional View with NAT
Server1

Client SLB Server2

Server3

By Dr. Gaddisa Olani ([email protected])

SLB: Traditional View without NAT
Server1

Client SLB Server2

Server3

By Dr. Gaddisa Olani ([email protected])

Virtual machine migration
• The movement of VMs from one resource to another, such as from
one physical host to another physical host, or data store to data store,
is known as VM migration.

• There are two types of VM migration: cold and live.

By Dr. Gaddisa Olani ([email protected])

Virtual machine migration
• The service-on-demand nature of cloud computing implies that when
there is a service request, the resources should be provided.

• Sometimes resources (often VMs) should be migrated from one

physical server to another, possibly on a far location.

• Designers of load-balancing algorithms have to consider two issues in

such cases:
• Time of migration that affects the performance and the probability of attacks
(security issue).
By Dr. Gaddisa Olani ([email protected])
Single point of failure
• Some of the load-balancing algorithms are centralized.

• In such cases, if the node executing the algorithm (controller) fails,

the whole system will crash because of that single point of failure.

The challenge here is to design distributed or decentralized

algorithms.

By Dr. Gaddisa Olani ([email protected])

Energy management
• Load-balancing algorithms should be designed to minimize the
amount of energy consumption.

By Dr. Gaddisa Olani ([email protected])