CS 5450

Virtualization

Vitaly Shmatikov
Virtualization

slide 2
Physical Machine
uPhysical hardware
• Processors, memory,
chipset, I/O devices, etc.
• Resources often grossly
underutilized
uSoftware
• Tightly coupled to physical
hardware
• Single active OS instance
• OS controls hardware

slide 3
OS Limitations
uOSes provide a way of virtualizing hardware
resources among processes
uHelps isolate processes from one another, but does
not provide a virtual machine to a user who may
wish to run a different OS
uHaving hardware resources managed by a single
OS limits the flexibility of the system in terms of
available software, security, and failure isolation

slide 4
Virtual Machine
uSoftware abstraction
• Behaves like hardware
• Encapsulates all OS and
application state
uVirtualization layer
• Extra level of indirection
• Decouples hardware, OS
• Enforces isolation
• Multiplexes physical
hardware across VMs

slide 5
Types of Virtualization
uProcess virtualization
• Language-level Java, .NET, Smalltalk
• OS-level processes, Solaris Zones, BSD Jails, Virtuozzo
• Cross-ISA emulation Apple 68K-PPC-x86, Digital FX!32
uDevice virtualization
• Logical vs. physical VLAN, VPN, NPIV, LUN, RAID
uSystem virtualization
• “Hosted” VMware Workstation, Microsoft VPC, Parallels
• “Bare metal” VMware ESX, Xen, Microsoft Hyper-V

slide 6
Virtualization Properties
uIsolation of faults and performance
uEncapsulation of entire VM state
• Enables snapshots and cloning of VMs
uPortability
• Independent of physical hardware
• Enables migration of live, running VMs
uInterposition
• Transformations on instructions, memory, I/O
• Enables transparent resource overcommitment,
encryption, compression, replication …
slide 7
Benefits of Virtualization

A system VM provides a Virtualization helps

sandbox that isolates isolate the effects of a
one system failure to the VM where
environment from other the failure occurred
environments
Multiple
Failure
Secure
Isolation
Environments

Better
Mixed-OS
System
Environment
A single hardware Utilization A virtualized system
platform can support can be (dynamically or
multiple operating statically) re-configured
systems concurrently for changing needs

slide 8
What Is Virtualization?
Informally, a virtualized system (or subsystem) is a mapping of its interface, and
all resources visible through that interface, to the interface and resources of a
real system

Formally, virtualization involves the construction of an isomorphism that maps a

virtual guest system to a real host system (Popek and Goldberg 1974)

Function V maps the

e(Si)
guest state to the host state
Si Sj
Guest
For a sequence of operations, e,
that modifies a guest state, there V(Si) V(Sj)
is a corresponding e’ in the host
that performs an equivalent
e’(Si’)
modification
Si’ Sj’
Host
slide 9
Virtual Machine Monitor
[Popek and Goldberg 1974]

slide 10
 Abstraction
uThe key to managing complexity in computer
systems is their division into levels of abstraction
separated by well-defined interfaces

File File

Disk
• Files are an abstraction of a disk
• A level of abstraction provides a simplified interface to underlying resources
slide 11
Virtualization and Abstraction
uVirtualization uses abstraction but the level of
detail is often the same as in underlying system

Virtual Disks

File File

Disk

slide 12
 “Classic” Virtualization
uClassical VMM
• IBM mainframes:
IBM S/360, IBM VM/370
• Co-designed proprietary
hardware, OS, VMM
• “Trap and emulate” model
uApplications
From IBM VM/370 product announcement, ca. 1972 • Timeshare several
single-user OS instances
on expensive hardware
• Compatibility
slide 13
IBM VM/370
uRobert Jay Creasy (1939-2005)
• Project leader of the first full virtualization hypervisor:
IBM CP-40, a core component in the VM system
uFirst VM system: VM/370

slide 14
Virtualization Renaissance
uRecent proliferation of VMs
• Considered exotic mainframe technology in 1990s
• Now pervasive in datacenters and clouds
• Huge commercial success
uWhy?
• Introduction on commodity x86 hardware
• Ability to “do more with less” saves $$$
• Innovative new capabilities
• Extremely versatile technology

slide 15
Modern Virtualization Applications
uServer consolidation
• Convert underutilized servers to VMs
• Significant cost savings (equipment, space, power)
• Increasingly used for virtual desktops
uSimplified management
• Datacenter provisioning and monitoring
• Dynamic load balancing
uImproved availability
• Automatic restart, fault tolerance, disaster recovery
uTest and development
slide 16
A Mixed OS Environment

Linux Red
Solaris 10 XP Vista Mac
Hat

Virtual Machine Monitor

Hardware

slide 17
Multiprocessor Virtualization

P P P P P P P P P P P P

M I/O M I/O M I/O M I/O

Virtual Machine Monitor

Processor Processor Processor Processor

Cache Cache Cache Cache

Bus or Crossbar Switch

Memory
I/O

slide 18
Virtualizing Processor

slide 19
Trap and Emulate
uRun guest operating system deprivileged
uAll privileged instructions trap into VMM
uVMM emulates instructions against virtual state
• E.g., disable virtual interrupts, not physical interrupts
uResume direct execution from next guest
instruction

slide 20
Trap and Emulate

Guest OS + Applications

Unprivileged
Page Undef
Fault Instr
vIRQ

Privileged
MMU CPU I/O
Emulation Emulation Emulation

Virtual Machine Monitor

Pretend to OS it’s still running in privileged mode slide 21

Issues with Trap-and-Emulate
uNot all architectures support it
uTrap costs may be high
uVMM consumes a privilege level
• Need to virtualize the protection levels

slide 22
“Strictly Virtualizable”
A processor or mode of a processor is strictly
virtualizable if, when executed in a lesser privileged
mode:
uall instructions that access privileged state trap
uall instructions either trap or execute identically

slide 23
x86 is Not Strictly Virtualizable
uOn x86, popf instruction takes a word off the stack
and puts it into the flags register
uAt OS level, popf updates interrupt enable flag (IF)
uAt user level, updates to IF silently dropped
• To prevent user-level code from messing up the OS
uWhen VMM runs the OS at user level, all OS
modifications to IF are dropped and VMM doesn’t
know whether OS wants interrupts enabled or not
uSeveral other reasons why trap-and-emulate
wouldn’t work for x86
slide 24
Binary Translation
Goal: translate potentially dangerous and non-virtualizable
instruction sequences one-by-one into safe sequences
• Privileged instructions, control flow, memory accesses
uIf safe instruction, copy into translation cache
uIf simple dangerous instruction, “inline” translate into short
sequence of emulation code, copy into translation cache
• Example: modification of the Interrupt Enable flag
uIf another dangerous instructions, execute emulation code
in the monitor (“call-out”)
• Example: change to the page table base, branch ending basic block
uMonitor jumps to the start of the translated basic block with
the virtual registers in the hardware registers.
slide 25
Example of Binary Translation

Guest Code Translation Cache

vEPC mov ebx, eax mov ebx, eax start

cli mov [VIF], 0
and ebx, ~0xfff and ebx, ~0xfff
mov ebx, cr3 mov [CO_ARG], ebx
sti call HANDLE_CR3
ret mov [VIF], 1
test [INT_PEND], 1
jne
call HANDLE_INTS
jmp HANDLE_RET

slide 26
Issues with Binary Translation
uTranslation cache management
uPC synchronization on interrupts
uSelf-modifying code
• Notified on writes to translated guest code
uProtecting VMM from guest

slide 27
x86 Memory Management
uThe processor operates with virtual addresses
uPhysical memory operates with physical addresses
uHardware translation lookaside buffer (TLB) maps
virtual to physical page addresses
uTLB misses handled in hardware
• Hardware walks the page tables and inserts virtual to
physical mapping

slide 28
x86 Memory Management

slide 29
Virtualizing Memory
uOS assumes that it has full control over memory
• Management: assumes it owns it all
• Mapping: assumes it can map any virtual à physical
uHowever, VMM partitions memory among VMs
• VMM needs to assign hardware pages to VMs
• VMM needs to control mapping for isolation
– Cannot allow OS to map any virtual à hardware page
uHardware-managed TLBs make this difficult
• On TLB misses, hardware walks page tables in memory
• VMM needs to control access by OS to page tables
slide 30
Virtualized Memory

Virtual
Pages
Native

Virtual
Pages
Virtualized

slide 31
Shadow Page Tables
uThree abstractions of memory
• Machine: actual hardware memory (e.g. 2GB of DRAM)
• Physical: abstraction of hardware memory, OS managed
– E.g. VMM allocates 512 MB to a VM, the OS thinks the computer
has 512 MB of contiguous physical memory (underlying machine
memory may be discontiguous)
• Virtual: virtual address space
– Standard 2^32 address space
uIn each VM, OS creates and manages page tables
for its virtual address spaces without modification
• But these page tables are not used by the MMU
slide 32
Traditional Address Translation

Virtual Address Physical Address

TLB

1 4 2 5

Operating System’s
Page Fault Handler

Process
Page Table
2

slide 33
Using Shadow Page Tables

Virtual Address Machine Address

TLB
4
1 5 2 6

Shadow Guest
Page Table Page Table 3 PMap
2
A

slide 34
Using Nested Page Tables

Virtual Address Machine Address

TLB

3
1

Guest PhysMap
2 3
Page Table By VMM

slide 35
Virtualizing I/O
u Challenge: Lots of I/O devices... writing device drivers
for all of them in the VMM layer is not a feasible option
u Insight: Device drivers already written for popular
operating systems
u Solution: Present virtual I/O devices to guest VMs and
channel I/O requests to a trusted host VM (popular OS)

slide 36
I/O Virtualization Implementations
Emulated I/O Passthrough I/O
Hosted or Split Hypervisor Direct

Guest OS Guest OS Guest OS

Device Driver Device Driver Device Driver

Host OS/Dom0/
Parent Domain

Device Emulation Device Emulation Device Emulation

I/O Stack I/O Stack

Device
Device Driver Device Driver Manager

VMware Workstation, VMware Server, VMware ESX VMware ESX (FPT)

Xen, Microsoft Hyper-V, Virtual Server
slide 37
Full Virtualization
Example: VMware ESX

uFunctionally identical to
underlying physical hardware
uFunctionality is exposed to
the VMs
uAllows unmodified guest OS
to execute on the VMs
• Transparent to OS: VM looks
like the physical machine
• This might result in some
performance degradation

slide 38
Para-Virtualization
Example: Xen

uVirtual hardware
abstraction similar, but
not identical to the real
hardware
uGuest OS modified to
cooperate with the VMM
• Lower overhead leading to
better performance

slide 39
Type 1 vs. Type 2
uNative/Bare metal (Type 1)
• Higher performance
• ESX, Xen, HyperV

uHosted (Type 2)
• Easier to install
• Leverage host’s device drivers
• VMware Workstation, Parallels

slide 40