1

2
3
4
•
•
•
•

5
•
•

6
•Compute resource only:
–Move a VM, but not its storage, to another host.
–For a hot migration, vSphere vMotion is used to move the VM.
•Storage only:
–Move a VM's files or objects to a new datastore.
–For a hot migration, vSphere Storage vMotion is used to move the VM.
•Both compute resource and storage:
–Move a VM to another host and datastore.
–For a hot migration, vSphere vMotion and vSphere Storage vMotion are used to move
the VM.
•Cross vCenter Server export:
–Move the VM to a host and datastore managed by a different vCenter instance that is
not linked to the current SSO domain.

7
•
•
•

8
Host Configuration for vSphere vMotion

9
•
•

Host Configuration for vSphere vMotion

10
 https://vmware.bravais.com/s/FbzaDb6owpSMKyKc940F

1.A shadow VM is created on the destination host.

2.The VM’s memory state is copied over the vSphere vMotion network from the source host to
the target host through the vSphere vMotion network. Users continue to access the VM and,
potentially, update pages in memory. A list of modified pages in memory is kept in a memory
bitmap on the source host.
3.After the first pass of memory state copy completes, another pass of memory copy is
performed to copy any pages that changed during the last iteration. This iterative memory
copying continues until the number of changed pages is small enough to copy across within
500 milliseconds.
4.After most of the VM’s memory is copied from the source host to the target host, the VM is
quiesced. No additional activity occurs on the VM. In the quiesce period, vSphere vMotion
transfers the VM device state and memory bitmap to the destination host.
5.Immediately after the VM is quiesced on the source host, the VM is initialized and starts
running on the target host. A Gratuitous Address Resolution Protocol (GARP) request notifies
the subnet that VM A’s MAC address is now accessible through a different physical switch
port. The VM's files are unlocked by the source host and locked by the destination host.
6.Users access the VM on the target host instead of the source host.
7.The memory pages that the VM was using on the source host are marked as free.

https://blogs.vmware.com/vsphere/2019/07/the-vmotion-process-under-the-hood.html

11
 •

https://docs.vmware.com/en/VMware-vSphere/index.html

12
•
–
–

•
•

13
 •

–
–
–
•
–
–

https://docs.vmware.com/en/VMware-vSphere/index.html

14
•Change compute resource
•Change both compute resource and storage
•Cross vCenter Server export

15
16
•
•

17
18
•
•
•
•

19
20
•
•
•
•

21
22
23
 •
•
•
•

http://www.vmware.com/resources/compatibility

24
•Create an empty cluster with EVC mode configured and move hosts into the cluster.
•Configure EVC mode on an existing cluster.

http://kb.vmware.com/kb/1003212

25
•

•
•

26
•

•
•

27
28
•
•
•

•
•

29
30
31
•
•
•
•

32
33
•
•
•
•

34
•

•
•

.nvram

35
 https://vmware.bravais.com/s/FnHZwq043PJ8dV3ZRV7p

1.Initiate storage migration.

2.Use the VMkernel data mover or vSphere Storage APIs – Array Integration to copy data.
3.Start a new VM process.
4.Mirror I/O calls to file blocks that are already copied to the virtual disk on the destination
datastore.
5.Transition to the destination VM process to begin accessing the virtual disk copy.

36
•

37
•
•

.nvram

38
•
•

39
•
•

•
•

40
41
•
•
•
•

42
43
•

44
•

45
 •
•

•Both vCenter instances must be time-synchronized with each other for correct vCenter Single
Sign-On token verification.
•Both vCenter instances do not need to be in the same Enhanced Linked Mode group.
•For migration of compute resources only, both vCenter instances must be connected to the
shared storage on which the VM is located.

http://kb.vmware.com/kb/2106952

46
47
48
49
•
•
•

50
 •
•
•
•
•

•Default TCP/IP stack: Provides networking support for the management traffic between
vCenter and ESXi hosts and for system traffic such as vSphere vMotion, IP storage, and
vSphere Fault Tolerance.
•vSphere vMotion TCP/IP stack: Supports the traffic for hot migrations of VMs.
•Provisioning TCP/IP stack: Supports the traffic for VM cold migration, cloning, and snapshot
creation. You can use the provisioning TPC/IP stack to handle NFC traffic during long-distance
vSphere vMotion migration. VMkernel adapters configured with the provisioning TCP/IP stack
handle the traffic from cloning the virtual disks of the migrated VMs in long-distance vSphere
vMotion.
By using the provisioning TCP/IP stack, you can isolate the traffic from the cloning operations
on a separate gateway. After you configure a VMkernel adapter with the provisioning TCP/IP
stack, all adapters on the default TCP/IP stack are deactivated for the provisioning traffic.
•Custom TCP/IP stacks: You can create a custom TCP/IP stack on a host to forward
networking traffic through a custom application. Open an SSH connection to the host and run
the vSphere CLI command:
esxcli network ip netstack add -N="stack_name"

51
52
•
•
•

•
•

53
•
—
—
•
—
—
—

54
•

55
56
•
•
•
•

57
58
 •
•
•

•Memory state: The contents of the VM’s memory. The memory state is captured only if the
VM is powered on and if you select the Snapshot the virtual machine’s memory check box
(selected by default).
•Settings state: The VM settings.
•Disk state: The state of all the VM’s virtual disks.

59
 •
•

• #-delta.vmdk
• #-sesparse.vmdk
•
•
•
• Delta object

•VMFSsparse: VMFS5 uses the VMFSsparse format for virtual disks smaller than 2 TB.
VMFSsparse is implemented on top of VMFS. The VMFSsparse layer processes I/O
operations issued to a snapshot VM. Technically, VMFSsparse is a redo log that starts empty,
immediately after a VM snapshot is taken. The redo log expands to the size of its base VMDK,
when the entire VMDK is rewritten with new data after the VM snapshot. This redo log is a file
in the VMFS datastore. On snapshot creation, the base VMDK attached to the VM is changed
to the newly created sparse VMDK.
•SEsparse: SEsparse is a default format for all delta disks on the VMFS6 datastores. On
VMFS5, SEsparse is used for virtual disks of the size 2 TB and larger. SEsparse is a format
that is like VMFSsparse with some enhancements. This format is space efficient and supports
the space-reclamation technique. With space reclamation, blocks that the guest OS deletes
are marked. The system sends commands to the SEsparse layer in the hypervisor to unmap
those blocks. The unmapping helps to reclaim space allocated by SEsparse after the guest
operating system deletes the data.
•vsanSparse: vASAN Express Storage Architecture (ESA) snapshots are stored within the
same object.

60
 • -Snapshot#.vmsn
• -Snapshot#.vmem
• -00000#.vmdk
• -00000#-delta.vmdk
• -00000#-sesparse.vmdk
• .vmsd

•Snapshot delta file: This file contains the changes to the virtual disk’s data since the snapshot
was taken. When you take a snapshot of a VM, the state of each virtual disk is preserved. The
VM stops writing to its -flat.vmdk file. Writes are redirected to >-######-delta.vmdk
(or -######-sesparse.vmdk) instead (for which ###### is the next number in the
sequence). You can exclude one or more virtual disks from a snapshot by designating them as
independent disks. Configuring a virtual disk as independent is typically done when the virtual
disk is created, but this option can be changed whenever the VM is powered off.
•Disk descriptor file: -00000#.vmdk. This file is a small text file that contains information
about the snapshot.
•Configuration state file: -.vmsn. # is the next number in the sequence, starting with 1. This
file holds the active memory state of the VM at the point that the snapshot was taken,
including virtual hardware, power state, and hardware version.
•Memory state file: -.vmem. This file is created if the option to include memory state was
selected during the creation of the snapshot. It contains the entire contents of the VMs at the
time that the snapshot of the VM was taken.
•Snapshot active memory file: -.vmem. This file contains the contents of the VM memory if
the option to include memory is selected during the creation of the snapshot.
•The .vmsd file is the snapshot list file and is created at the time that the VM is created. It
maintains snapshot information for a VM so that it can create a snapshot list in the vSphere
Client. This information includes the name of the snapshot .vmsn file and the name of the
virtual disk file.
•The snapshot state file has a .vmsn extension and is used to store the state of a VM when a
snapshot is taken. A new .vmsn file is created for every snapshot that is created on a VM and
is deleted when the snapshot is deleted. The size of this file varies, based on the options
selected when the snapshot is created. For example, including the memory state of the VM in
the snapshot increases the size of the .vmsn file.

61
_.vmem

61
62
63
64
•Edit the snapshot: Edit the snapshot name and description.
•Delete the snapshot: Remove the snapshot from the Snapshot Manager, consolidate the
snapshot files to the parent snapshot disk, and merge with the VM base disk.
•Delete all snapshots: Commit all the intermediate snapshots before the current-state icon
(You are here) to the VM and remove all snapshots for that VM.
•Revert to a snapshot: Restore, or revert to, a particular snapshot. The snapshot that you
restore becomes the current snapshot.

65
 You are here

https://vmware.bravais.com/s/WhbcXR4sSwk2Vl7MeaXD

66
 -delta.vmdk

https://vmware.bravais.com/s/l0JYYQzMTv7pvxBqNcQp

67
 You are here level

https://vmware.bravais.com/s/NiQxPT3iycemQ8WYXKom

68
 https://vmware.bravais.com/s/L3ilQHlrywEhIgr5p7RP
You are here
You are here

69
•

• -delta.vmdk -sesparse.vmdk
• -flat.vmdk

70
71
http://kb.vmware.com/kb/1025279

72
73
•
•
•
•

74
75
•
•
•
•
•

76
•

77
•

• .vswp

•
vmx-*.vswp

.vswp vmx-*.vswp

78
•Page sharing: ESXi can use a proprietary technique to transparently share memory pages
between VMs, eliminating redundant copies of memory pages. Although pages are shared by
default within VMs, as of vSphere 6.0, pages are no longer shared by default among VMs.
•Ballooning: If the host memory begins to get low and the VM's memory use approaches its
memory target, ESXi uses ballooning to reduce that VM's memory demands. Using the
VMware-supplied vmmemctl module installed in the guest operating system as part of
VMware Tools, ESXi can cause the guest operating system to relinquish the memory pages it
considers least valuable. Ballooning provides performance closely matching that of a native
system under similar memory constraints. To use ballooning, the guest operating system must
be configured with sufficient swap space.
•Memory compression: If the VM's memory use approaches the level at which host-level
swapping is required, ESXi uses memory compression to reduce the number of memory
pages that it must swap out. Because the decompression latency is much smaller than the
swap-in latency, compressing memory pages has significantly less impact on performance
than swapping out those pages.
•Swap to host cache: Host swap cache is an optional memory reclamation technique that
uses local flash storage to cache a virtual machine’s memory pages. By using local flash
storage, the virtual machine avoids the latency associated with a storage network that might
be used if it swapped memory pages to the virtual swap (.vswp) file.
•Regular host-level swapping: When memory pressure is severe and the hypervisor must
swap memory pages to disk, the hypervisor swaps to a host swap cache rather than to a
.vswp file. When a host runs out of space on the host cache, a virtual machine’s cached
memory is migrated to a virtual machine’s regular .vswp file. Each host must have its own

79
host swap cache configured.

79
80
•

81
82
•
•
•
•
•

83
84
•
•
•

85
•

86
 •
•

https://docs.vmware.com/en/VMware-vSphere/index.html

87
•

•
•

88
 •

• .vswp

•Benefits: Assigning a limit is useful if you start with a few VMs and want to manage user
expectations. The performance deteriorates as you add more VMs. You can simulate having
fewer resources available by specifying a limit.
•Drawbacks: You might waste idle resources if you specify a limit. The system does not allow
VMs to use more resources than the limit, even when the system is underused and idle
resources are available. Specify the limit only if you have good reasons for doing so.

89
•

90
91
92
•VM D is powered on with 1,000 shares.
•Before VM D was powered on, a total of 5,000 shares were available, but VM D’s addition
increases the total shares to 6,000.
•The result is that the other VMs' shares decline in value. But VM A is still allocated one-sixth
of the resource because it owns one-sixth of the shares.

93
94
95
96
97
•
•
•

98
•
•

99