Latency in Cloud Computing

Latency:

Many people have likely heard the term latency being used before but what is latency
exactly? In terms of network latency, this can be defined by the time it takes for a request to
travel from the sender to the receiver and for the receiver to process that request. In other
words, the round trip time from the browser to the server. It is obviously desired for this time
to remain as close to 0 as possible, however, there can be a few things at play preventing your
website latency times to remain low.

Example:

Your video game might experience high latency which causes the game to freeze at
times and not deliver a live update of the other players. Fewer delays would mean that the
connection is experiencing lower latency. Network lag happens for a few basic reasons, namely
distance and congestion. In some cases, you might be able to fix internet lag by tweaking how
your device interacts with the network.

Latency vs Bandwidth:

 Bandwidth determines how narrow or wide a pipe is. The narrower it is, the
less data is able to be pushed through it at once and vice versa.
 Latency determines how fast the contents within a pipe can be transferred from
the client to the server and back.

Causes of Network Latency:

There are 4 main causes that can affect network latency times.

 Transmission mediums such as WAN or fiber optic cables all have limitations
and can affect latency simply due to their nature.
 Propagation is the amount of time it takes for a packet to travel from one source
to another.
 Routers take time to analyze the header information of a packet as well as, in
some cases, add additional information. Each hop a packet takes from router to
router increases the latency time.

1
  Storage Delays can occur when a packet is stored or accessed resulting in a
delay caused
by intermediate devices like switches and bridges.

Typical Internet Connection Latencies:

Besides the limits of physics, additional network latency is caused when traffic
is routed through servers and other backbone devices. The typical latency of an internet
connection also varies depending on its type.

The study Measuring Broadband America (posted in late 2018) reported these typical internet
connection latencies for common forms of U.S. broadband service:

 Fiber optic: 12-20 ms

 Cable internet: 15-34 ms
 DSL: 25-80 ms
 Satellite internet: 594-612 ms

Network Latencies in Data Centers:

If in the past it was common to think about latency on the scale of milliseconds, today
network elements within the data center are several orders of magnitude faster than that. If in
the past it took 640ns to transmit just 64B using 1GE technology, today it is possible to transmit
five 1518B packets at the same time. Sub-microsecond, high port count, top of rack switches
are common and there is little difference in latency between different 10GE and 100GE from
the same manufacturing generation.

To understand the scale and distribution of latency as experienced by a user in a data center,
we measure the round-trip time between multiple virtual machines rented from different cloud
operators. Our aim is to calibrate our latency exploration based upon current data center
measurements and further use these values as inputs to our controlled latency environment.
This permits us to evaluate the performance of an application as it is subject to varying latency
values.

2
VR-assist service:

Finding an apartment/house to rent can be time consuming process. Virtual reality tours
can help tenants to view a range of properties without ever needing to set foot in the actual
house.

Deploying VR-assist rental service on conventional cloud, all databases of in/out side house
viewing are stored at core cloud, and it will cause a network congestion if many tenants access
this service at the same time. Distributed cloud is a key solution to greatly mitigate network
congestion and enhance performance of this service by distributing geography based data to
edge cloud. Each edge cloud will handle a part of the data base from all houses located in same
area. Tenants in every area tend to look up apartments in their area instead of other areas. In
case of looking for apartments in another area the edge cloud will communicate with each other
to find matched data for tenants. We believe that this kind of application cloud be popular in
forthcoming years as distributed cloud is mature.

How to Reduce Latency:

Latency can be reduced using a few different techniques as described below. Reducing
the amount of server latency will help load your web resources faster, thus improving the
overall page load time for your visitors.

3
1. HTTP/2:
The use of the ever prevalent HTTP/2 is another great way to help minimize
latency. HTTP/2 helps reduce server latency by minimizing the number of
round trips from the sender to the receiver and with parallelized transfers.
KeyCDN proudly offers HTTP/2 support to customers across all of our edge
servers.

2. Fewer External HTTP requests:

Reducing the number of HTTP requests not only applies to images but also for
other external resources such as CSS or JS files. If you are referencing
information from a server other than your own, you are making an external
HTTP request which can greatly increase website latency based on the speed
and quality of the third party server.

3. Using a CDN:
Using a CDN helps bring resources closer to the user by caching them in
multiple locations around the world. Once those resources are cached, a user’s
request only needs to travel to the closest Point of Presence to retrieve that data
instead of going back to the origin server each time.

4. Using Prefetching Methods:

Prefetching web resources doesn’t necessarily reduce the amount of latency per
se however it improves your website’s perceived performance. With
prefetching implemented, latency intensive processes take place in the
background when the user is browser a particular page. Therefore, when they
click on a subsequent page, jobs such as DNS lookups have already taken place,
thus loading the page faster.

5. Browser Caching:
Another type of caching that can be used to reduce latency is browser caching.
Browsers will cache certain resources of a website locally in order to help
improve latency times and decrease the number of requests back to the server.

4
 Read more about browsing caching and the various directives that exist in
our cache-control article.

6. TRAFFIC LOAD:
Spikes in internet utilization during peak usage times of day often cause lag.
The nature of this lag varies by service provider and a person's geographic
location. Unfortunately, other than moving locations or changing internet
service, an individual user can't avoid this kind of lag.

7. LAG SWITCHES:

Some people who play online games install a device called a lag switch on their
local network. A lag switch is specially designed to intercept network signals
and introduce significant delays in the flow of data back to other gamers
connected to a live session. You can do little to solve this kind of lag problem
other than avoiding playing with those who use lag switches; fortunately, they're
relatively uncommon.

8. NO LOCAL CACHE:

One way to reduce latency is to utilize caching in your browser, which is a way
for the program to store recently used files so that you can access them again
locally the next time you request them.

How to Measure Network Latency:

Network latency can be tested using either Ping, Traceroute, or MTR. Each of these
tools is able to determine specific latency times, with MTR being the most detailed.

The use of MTR allows a user to generate a report that will list each hop in a network that was
required for a packet to travel from point A to point B. The report will include details such as
Loss%, Average latency, etc.

Essentially, latency is measured using one of two methods:

5
  Round Trip Time (RTT)
 Time to First Byte (TTFB)

The round trip time can be measured using the methods above and involves measuring the
amount of time it takes between when a client sends a request to the server and when it gets it
back. On the other hand, the TTFB measure the amount of time it takes between when a client
sends a request to a server and when it receives its first byte of data.

Tools Help Improve Network Latency:

Using tools to improve network latency is familiar to most network professionals, and
there are several different options with network latency measuring features.

A network performance monitoring tool is the most comprehensive kind of tool you can use,
as it normally includes features let you address latency and network performance. A tool
like Solar Winds Network Performance Monitor (NPM) also provides functions like network
latency testing, network mapping, problem troubleshooting, and general network baselining.

Types of Latency:

Latency occurs in many various environments including audio, networks, operations, etc. The
following describes two additional scenarios where latency is also prevalent

 Fiber Optic Latency:

Latency in the case of data transfer through fiber optic cables can’t be fully explained
without first discussing the speed of light and how it relates to latency.

Based on the speed of light alone (299,792,458 meters/second), there is a latency of 3.33
microseconds (0.000001 of a second) for every kilometer of path covered. Light travels slower
in a cable which means the latency of light traveling in a fiber optic cable is around 4.9
microseconds per kilometer.

Based on how far a packet must travel, the amount of latency can quickly add up. Cable
imperfections can also degrade the connection and increase the amount of latency incurred by
a fiber optic cable.

6
  Audio Latency:
This form of latency is the time difference between a sound being created and heard.
The speed of sound plays a role in this form of latency which can vary based on the environment
it travels through e.g. solids vs liquids. In technology, audio latency can occur from various
sources including analog to digital conversion, signal processing, hardware / software used,
etc.

References:

1. https://www.keycdn.com/support/what-is-latency
2. https://www.lifewire.com/lag-on-computer-networks-and-online-817370
3. https://sourcedaddy.com/windows-7/what-causes-latency-how-measure-it-and-how-
to-control-it.html
4. https://books.google.com.pk/books?id=73eQDwAAQBAJ&printsec=frontcover#v=on
epage&q&f=false
5. https://www.cloudflare.com/learning/performance/glossary/what-is-latency/
6. https://www.dnsstuff.com/network-latency
7. Minlan Yu, Lavanya Jose, and Rui Miao. “Software Defined Traffic Measurement with
Open Sketch”. In: Proceedings of the 10th USENIX Conference on Networked
Systems.

7
8