Distributed and Cloud Computing

K. Hwang, G. Fox and J. Dongarra

Chapter 3: Virtual Machines and Virtualization

of Clusters and datacenters
(suggested for use in 4 lectures in 200 minutes)

Prepared by Kai Hwang

University of Southern California
March 30, 2012

Copyright © 2012, Elsevier Inc. All rights reserved. 3-1

Virtualization for Datacenter Automation
to serve millions of clients, simultaneously

 Server Consolidation in Virtualized Datacenter

 Virtual Storage Provisioning and Deprovisioning

 Cloud Operating Systems for Virtual Datacenters

 Trust Management in virtualized Datacenters

Copyright © 2012, Elsevier Inc. All rights reserved. 3-2

Difference between Traditional
Computer and Virtual machines

(Courtesy of VMWare, 2008)

Copyright © 2012, Elsevier Inc. All rights reserved. 3-3

What is Virtualization ?

Copyright © 2012, Elsevier Inc. All rights reserved. 3-4

Copyright © 2012, Elsevier Inc. All rights reserved. 3-5
 Virtual Machine, Guest Operating System,
and VMM (Virtual Machine Monitor) :

The Virtualization layer is the middleware between the

underlying hardware and virtual machines represented
in the system, also known as virtual machine monitor
(VMM).
Copyright © 2012, Elsevier Inc. All rights reserved. 3-6
 (Courtesy of VMWare, 2008)

Copyright © 2012, Elsevier Inc. All rights reserved. 3-7

VMM : Virtual Machine Monitor

(Courtesy of Mendel Rosenblum, 2005)

Copyright © 2012, Elsevier Inc. All rights reserved. 3-8

Low-Level VMM Operations (1)

(Courtesy of Mendel Rosenblum, 2006)

Copyright © 2012, Elsevier Inc. All rights reserved. 3-9

Low-Level VMM Operations (2)

(Courtesy of Mendel Rosenblum, 2006)

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 10

Low-Level VMM Operations (3)

(Courtesy of Mendel Rosenblum, 2006)

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 11

Low-Level VMM Operations (4)

(Courtesy of Mendel Rosenblum, 2006)

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 12

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 13
Virtualization at ISA level:
Emulating a given ISA by the ISA of the host machine. For example,
MIPS binary code can run on an x-86-based host machine with the
help of ISA emulation. Typical systems: Bochs, Crusoe, Quemu,
BIRD, Dynamo
Advantage: It can run a large amount of legacy binary codes
written for various processors on any given new hardware host
machines; best application flexibility
Shortcoming & limitation: One source instruction may require tens
or hundreds of native target instructions to perform its function,
which is relatively slow. V-ISA requires adding a processor-specific
software translation layer in the complier.

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 14

Virtualization at Hardware Abstraction level:

Virtualization is performed right on top of the hardware. It generates

virtual hardware environments for VMs, and manages the underlying
hardware through virtualization. Typical systems: VMware, Virtual PC,
Denali, Xen

Advantage: has higher performance and good application isolation

Shortcoming & limitation: very expensive to implement (complexity)

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 15

Virtualization at Operating System level:
It is an abstraction layer between traditional OS and user
placations. This virtualization creates isolated containers on a single
physical server and the OS-instance to utilize the hardware and
software in datacenters. Typical systems: Jail / Virtual Environment /
Ensim's VPS / FVM

Advantage: have minimal starup/shutdown cost, low resource

requirement, and high scalability; synchronize VM and host state
changes.

Shortcoming & limitation: all VMs at the operating system level must
have the same kind of guest OS; poor application flexibility and
isolation.

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 16

Virtualization at OS Level

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 17

Virtualization for Linux and
Windows NT Platforms

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 18

Advantages of OS Extension for Virtualization

1. VMs at OS level has minimum startup/

shutdown costs

2. OS-level VM can easily synchronize with its

environment

Disadvantage of OS Extension for Virtualization

All VMs in the same OS container must have the same
or similar guest OS, which restrict application flexibility of
different VMs on the same physical machine.
Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 19
Library Support level:

It creates execution environments for running alien

programs on a platform rather than creating VM to run
the entire operating system. It is done by API call
interception and remapping. Typical systems: Wine,
WAB, LxRun , VisualMainWin

Advantage: It has very low implementation effort

Shortcoming & limitation: poor application flexibility

and isolation

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 20

Virtualization with Middleware/Library Support

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 21

User-Application level:

It virtualizes an application as a virtual machine. This

layer sits as an application program on top of an
operating system and exports an abstraction of a VM
that can run programs written and compiled to a
particular abstract machine definition. Typical systems:

JVM , NET CLI , Panot

Advantage: has the best application isolation

Shortcoming & limitation: low performance, low

application flexibility and high implementation
complexity.

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 22

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 23
 Full Virtualization vs. Para-Virtualization

Full virtualization does not need to modify guest OS, and

critical instructions are emulated by software through the use of
binary translation. On the other hand, para virtualization needs to
modify guest OS, and non-virtualizable instructions are replaced by
hypercalls that communicate directly with the hypervisor or VMM.
As of full virtualization, the advantage is no need to modify OS.
However, this approach of binary translation slows down the
performance a lot.
Para virtualization reduces the overhead, but the cost of
maintaining paravirtualized OS is high. The improvement depends
on the workload. VMware Workstation applies full virtualization,
which uses binary translation to automatically modify x86 software
on-the-fly to replace critical instructions. The para virtualization is
supported by Xen, Denali and VMware ESX.
Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 24
Full Virtualization

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 25

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 26
Feather- Weight VM (FVM)

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 27

Major VMM and Hypervisor Providers

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 28

The vCUBE for Virtualization of GPGPU

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 29

Hypervisor
A hypervisor is a hardware virtualization technique allowing
multiple operating systems, called guests to run on a host machine.
This is also called the Virtual Machine Monitor (VMM).
Type 1 hypervisor or the bare metal hypervisor sits on the
bare metal computer hardware like the CPU, memory, etc. All the guest
operating systems are a layer above the hypervisor. So the hypervisor
is the first layer over the hardware. The original CP/CMS hypervisor
developed by IBM was of this kind. Examples are Microsoft Hyper-V.
Type 2 or the hosted hypervisor do not run over the bare metal
hardware but they run over a host operating system. The hypervisor is
the second layer over the hardware. The guest operating systems run a
layer over the hypervisor and so they form the third layer. Examples
are FreeBSD. The operating system is usually unaware of the
virtualization
Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 30
Hypervisor and the XEN Architecture

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 31

The XEN Architecture (1)

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 32

The XEN Architecture (2)

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 33

The XEN Architecture (3)

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 34

Para-Virtualization

In para-virtualization, the guest operating system has to be

modified. Para-virtualization provides specially defined ‘hooks’ to do
some tasks in the host and guest operating systems, which would
otherwise have been done in a virtual environment, which is slower.
The VMM in a para-virtualized platform is simpler because the critical
tasks are now performed in the operating system rater than by the
VMM. Since the virtualization overhead decreases the performance
increases.

Some of the disadvantages are the compatibility and the portability is

reduces because of the modified operating system. Also the cost of
maintenance is high because of the deep OS modifications.
Some examples of Para-virtualization are KVM and the
XenWindowsGplPv project.

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 35

Para-
Virtualization
with
Compiler
Support.

The KVM builds

offers kernel-
based VM on
the Linux
platform, based
on para-
virtualization

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 36

Host-based Virtualization

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 37

Virtual Cores vs. Physical Processor Cores

Physical cores Virtual cores

The actual physical cores present in There can be more virtual cores
the processor. visible to a single OS than there are
physical cores.
More burden on the software to write Design of software becomes easier
applications which can run directly on as the hardware assists the software
the cores. in dynamic resource utilization.
Hardware provides no assistance to Hardware provides assistance to the
the software and is hence simpler. software and is hence more
complex.
Poor resource management. Better resource management.
The lowest level of system software The lowest level of system software
has to be modified. need not be modified.

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 38

 Binary
Translation
of Guest
OS
Requests
using a
VMM:

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 39

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 40
Hypercall Execution

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 41

VMWare ESX Server for Para-Virtualization

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 42

Virtualization Support at Intel

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 43

 (Courtesy of Mendel Rosenblum, 2006)

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 44

 (Courtesy of Mendel Rosenblum, 2006)

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 45

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 46
Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 47
Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 48
Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 49
Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 50
Copyright © 2012, Elsevier Inc. All rights reserved. (Courtesy of VMWare, 2008)
3 - 51
Conclusions on CPU, Memory
and I/O Virtualization :
 CPU virtualization demands hardware-assisted traps of
sensitive instructions by the VMM

 Memory virtualization demands special hardware support

(shadow page tables by VMWare or extended page table by
Intel) to help translate virtual address into physical address
and machine memory in two stages.

 I/O virtualization is the most difficult one to realize due to

the complexity if I/O service routines and the emulation
needed between the guest OS and host OS.

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 52

 Multi-Core Virtualization:
VCPU vs. traditional CPU

Figure 3.16 Four VCPUs are exposed the software, only three cores are
actually present. VCPUs V0, V1, and V3 have been transparently migrated,
while VCPU V2 has been transparently suspended. (Courtesy of Wells, et al.,
“Dynamic Heterogeneity and the Need for Multicore Virtualization”, ACM
SIGOPS Operating Systems Review, ACM Press, 2009 [68] )

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 53

 (Courtesy of Marty and Hill, 2007)

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 54

Virtual Clusters in Many Cores
Space Sharing of VMs -- Virtual Hierarchy

(Courtesy of Marty and Hill, 2007)

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 55

A Taxonomy of Virtual Machines

(Courtesy of Smith and Nail, 2005)

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 56

 (Courtesy of Smith and Nail, 2005)
Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 57
 (Courtesy of Smith and Nail, 2005)

Process VM :
An example that emulates
Guest IA32 Applications to run
on an Alpha Windows platform
Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 58
System Virtual Machine

(Courtesy of Smith and Nail, 2005)

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 59

Example of Virtual Machine Applications

Emulation Replication Composition

(Courtesy of Smith and Nail, 2005)

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 60

 Virtual Cluster Characteristics
 The virtual cluster nodes can be either physical or virtual machines. Multiple VMs

running with different OSs can be deployed on the same physical node.

 A VM runs with a guest OS, which is often different from the host OS, that manages
the resources in the physical machine, where the VM is implemented.

 The purpose of using VMs is to consolidate multiple functionalities on the same

server. This will greatly enhance the server utilization and application flexibility.

 VMs can be colonized (replicated) in multiple servers for the purpose of promoting
distributed parallelism, fault tolerance, and disaster recovery.

 The size (number of nodes) of a virtual cluster can grow or shrink dynamically,
similarly to the way an overlay network varies in size in a P2P network.

 The failure of any physical nodes may disable some VMs installed on the failing
nodes. But the failure of VMs will not pull down the host system.
Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 61
Virtual Clusters vs. Physical Clusters

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 62

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 63
Virtual Cluster Projects

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 64

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 65
Cluster-on-Demand (COD Project
at Duke University

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 66

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 67
VIOLIN Project at Purdue University

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 68

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 69
Live Migration of Virtual Machines

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 70

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 71
Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 72
 Cluster Partitioning
for VM-based Parallel Systems

(Courtesy of Smith and Nail, 2005)

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 73

System VM-based Partitioning

(Courtesy of Smith and Nail, 2005)

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 74

A Virtual Cluster on a Real Cluster

(Courtesy of Smith and Nail, 2005)

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 75
Virtualization with Different Host and Guest ISAs

(Courtesy of Smith and Nail, 2005)

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 76
Parallax for VM Storage Management

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 77

Cloud OS for Building Private Clouds

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 78

Eucalyptus : An Open-Source OS for
Setting Up and Managing Private Clouds

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 79

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 80
Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 81
Trusted Zones for VM Insulation
Insulate Anti-malware
Federate infrastructure from
Identity Malware, Trojans Cybercrime
identities with
federation and cybercriminalsintelligence
public clouds
Strong
APP APP
Tenant #2 authentication
OS OS

Virtual Infrastructure Insulate

Control and information
Virtual isolate VM in
from other Data loss
network the virtual prevention
tenants
security infrastructure APP APP
Tenant #1
OS OS

Virtual Infrastructure Insulate Encryption &

information key mgmt
Access Segregate and
from cloud
Mgmtcontrol user access
providers’
employees Tokenization
Cloud Provider
Security Info. &
Event Mgmt Physical Infrastructure GRC
Enable end to end view of security events
and compliance across infrastructures
(Courtesy of L. Nick, EMC 2008)
Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 82
Projected Growth of Virtualization
Market from 2006 to 2011 by IDC

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 83

 Reading Assignments :
1. K. Hwang, G. Fox and J. Dongarra, Distributed Systems
and Cloud Computing, Chapter 3, 2011
2. M. Rosenblum and T. Garfinkel, “Virtual Machine
Monitors: Current Technology and Future Trends”,
IEEE Computer Magazine, May 2005, pp.39-47.
3. VM Ware, Inc., “Virtualization Overview “,
White paper, http://www.vmware.com , 2006.
4. Virtual Machines by James Smith and Ravi Nair,
Morgan Kaufmann, an Elesevier imprint, 2005

Copyright © 2012, Elsevier Inc. All rights reserved. 3 - 84