VCP6-DCV STUDY GUIDE

[UNOFFICIAL]

By Vladan SEGET
www.vladan.fr

Running out of Capacity Again?

Its Time to Hyperconverge

90% Capacity Savings Guaranteed.

HY

ER

MPL I VI TY

TE

SI

GUARAN

SimpliVity HyperGuarantee
The Industrys Most Complete Guarante

www.simplivity.com/vcp6

Contents
VCP6-DCV Objective 1.1 Configure and Administer Role-based Access Control .................................................................... 3
VCP6-DCV Objective 1.2 Secure ESXi, vCenter Server, and vSphere Virtual Machines .......................................................... 9
VCP6-DCV Objective 1.3 - Enable SSO and Active Directory Integration.............................................................................. 17
VCP6-DCV Objective 2.1 - Configure Advanced Policies/Features and Verify Network Virtualization Implementation ................. 26
VCP6-DCV Objective 2.2 - Configure Network I/O Control (NIOC) ...................................................................................... 41
VCP6-DCV Objective 2.3 Configure vSS and vDS Policies ............................................................................................... 45
VCP6-DCV Objective 3.1 - Manage vSphere Storage Virtualization ..................................................................................... 52
VCP6-DCV Objective 3.2 - Configure Software-defined Storage ......................................................................................... 65
VCP6-DCV Objective 3.3 - Configure vSphere Storage Multi-pathing and Failover ................................................................ 76
VCP6-DCV Objective 3.4 - Perform Advanced VMFS and NFS Configurations and Upgrades ................................................... 83
VCP6-DCV Objective 3.5 - Setup and Configure Storage I/O Control .................................................................................. 93
VCP6-DCV Objective 4.1 - Perform ESXi Host and Virtual Machine Upgrades ....................................................................... 96
VCP6-DCV Objective 4.2 - Perform vCenter Server Upgrade ............................................................................................ 100
VCP6-DCV Objective 5.1 - Configure Advanced/Multilevel Resource Pools ......................................................................... 108
VCP6-DCV Objective 6.1 - Configure and Administer a vSphere Backups/Restore/Replication Solution .................................. 116
VCP6-DCV Objective 7.1 - Troubleshoot vCenter Server, ESXi Hosts, and Virtual Machines ................................................. 132
VCP6-DCV Objective 7.2 - Troubleshoot vSphere Storage and Network Issues................................................................... 139
VCP6-DCV Objective 7.3 - Troubleshoot vSphere Upgrades ............................................................................................. 144
VCP6-DCV Objective 7.4 - Troubleshoot and Monitor vSphere Performance ....................................................................... 149
VCP6-DCV Objective 7.5 - Troubleshoot HA and DRS Configurations and Fault Tolerance .................................................... 156
VCP6-DCV Objective 8.1 - Deploy ESXi Hosts Using Autodeploy ....................................................................................... 166
VCP6-DCV Objective 8.2 - Customize Host Profile Settings .............................................................................................. 172
VCP6-DCV Objective 8.3 - Consolidate Physical Workloads using VMware Converter ........................................................... 177
VCP6-DCV Objective 9.1 - Configure Advanced vSphere HA Features ............................................................................... 181
VCP6-DCV Objective 9.2 - Configure Advanced vSphere DRS Features ............................................................................. 189
VCP6-DCV Objective 10.1 - Configure Advanced vSphere Virtual Machine Settings............................................................. 192
VCP6-DCV Objective 10.2 - Create and Manage Multi-Site Content Library ........................................................................ 200
VCP6-DCV Objective 10.3 - Configure and Maintain a vCloud Air Connection ..................................................................... 205

VCP6-DCV OBJECTIVE 1.1 CONFIGURE AND ADMINISTER ROLE -BASED ACCESS CONTROL
Today's VCP6-DCV goal is to talk about - VCP6-DCV Objective 1.1 - Configure and Administer Role-based Access
Control. VMware VCP exam is a gold standard of VMware certification exams. VCP exam is the most known VMware
exams, even if it's not the highest technical level.
But it's most recognized. By a future employer, by industry as a whole. We will cover VCP6-DCV exam certification
based on VMware latest VMware VCP6-DCV blueprint. Check VCP6-DCV page for all objectives.

VMware vSphere Knowledge

Identify common vCenter Server privileges and roles

Describe how permissions are applied and inherited in vCenter Server
View/Sort/Export user and group lists
Add/Modify/Remove permissions for users and groups on vCenter Server inventory objects
Create/Clone/Edit vCenter Server Roles
Determine the correct roles/privileges needed to integrate vCenter Server with other VMware products
Determine the appropriate set of privileges for common tasks in vCenter Server

IDENTIFY

COMMON V C ENTER

S ERVER

PRIVILEGES AND ROLES

There are roles and privileges. Role is a collection of privileges assigned to group or a user. There are certain number
of Out-of-the-box (predefined) roles when we look at the vSphere client > Roles. You can keep them, clone them,
delete or edit.

Four different types of permissions

Not only vCenter server, like the ones above, but also Local permissions for ESXi. The full list:

Global Permissions Global permissions are applied to a global root object that spans solutions. To assign
permissions via global root allows to propagate them to the other products relying on SSO (vCO, vROPS, vCD..)
vCenter Server Permissions Hierarchical model. Permission gives you a certain number of privileges. Similar
like in Microft's AD. You Select object > assign role to a group of users > to give them privileges on that object.
Group Membership in vSphere.local Groups The vsphere.local domain includes several predefined groups.
Assign users from AD (if you're using AD) to one of those groups to be able to perform the corresponding
actions.
For some services that are not managed by vCenter Server directly, privileges are determined by membership
to one of the vCenter Single Sign-On groups. For example, a user who is a member of the Administrator group
can manage vCenter Single Sign-On. A user who is a member of the CAAdmins group can manage the VMware
Certificate Authority, and a user who is in the LicenseService.Administrators group can manage licenses.

Note: to be able to find the AD groups it's necessary to add Identity sources via:
Home > Administration > Single Sign-ON > Configuration > Identity sources.

The user [email protected] can perform tasks that are associated with services included with the Platform Services
Controller.

ESXi Local Host Permissions If you are managing a standalone ESXi host that is not managed by a vCenter
Server system, you can assign one of the predefined roles to users.

DESCRIBE HOW PERMISSIONS ARE APPLIED AND INHERITED IN VC ENTER SERVER

The global permissions are assigned via web client only (SSO), via Home > Administration > Global permissions.
If you deselect the propagate to children the objects lying down the road won't be accessible by that particular
user/group. (It's like when you manage NTFS permissions on Windows servers and you uncheck the heritage check
box). Permissions are applicable directly and propagated to children by default.

If you click the "View Children" link, it'll show you the permission of all the children which permission will apply to (if
"Propagate to children is selected).

Inheritance of Multiple Permissions - If user is member of more than one group? Then combined privileges
within the roles apply. Example below showing user member of both groups.

Child permissions override Parent permissions - Permissions applied on a child object always override
permissions that are applied on a parent object. See examples P. 119 of vSphere Security Guide.

Ex. Role 1 can power on VMs and Role 2 can take snapshots.
Group A is granted Role 1 on VM folder and permissions propagate to child objects
Group B is granted Role 2 on VM B
User 1, who belongs to groups A and B, logs on. Because Role 2 is assigned at a lower point in the hierarchy than Role
1, it overrides Role 1 on VM B. User 1 can power on VM A, but not take snapshots. User 1 can take snapshots of VM B,
but not power it on.

User role overriding group role - if two permissions are defined on the same object.

Permissions are on the same object. One permission is granted to a group, the other to a user which at the same time
is member of the group. Role 1 can power VMs Group A is granted Role 1 on VM folder and at the same time User 1 is
granted No Access role on VM folder.
User 1, who belongs to group A, logs on. The No Access role granted to User 1 on VM Folder overrides the role assigned
to the group. User 1 has no access to VM Folder or VMs A and B.

V IEW /S ORT /E XPORT

USER AND GROUP LISTS

To check Global permissions you have to go and use Web client > Home > Administration > Global permissions.
You can be export to a CSV file or copy to the Clipboard selected or All items. You can also use CTRL+Click to copy to
the clipboard.

A DD /M ODIFY /R EMOVE

PERMISSIONS FOR USERS AND GROUPS ON V C ENTER

S ERVER

INVENTORY

OBJECTS

To modify/add permissions you must Select an object > Manage > Permissions.
Than you can use the delete, edit or Add icons there...

CREATE/CLONE/E DIT V CENTER S ERVER R OLES

To edit, create or clone vCenter roles it's necessary to use vSphere Web client > Administration > Roles OR Home >
Roles. Default roles are:

Administrator
Read-Only
No Access

To clone role click the icon...

vSphere Security Guide (p. 121).

DETERMINE THE CORRECT ROLES / PRIVILEGES
OTHER VM WARE PRODUCTS

NEEDED TO INTEGRATE V C ENTER

S ERVER

WITH

Global permissions are applied to a global root object that spans solutions, for example, both vCenter Server and
vCenter Orchestrator. Use global permissions to give a user or group privileges for all objects in all object hierarchies.
Global permissions are applied to a global root object that spans solutions, for example, both vCenter Server and
vCenter Orchestrator. Use global permissions to give a user or group privileges for all objects in all object hierarchies.
P. 122

DETERMINE

THE APPROPRIATE SET OF PRIVILEGES FOR COMMON TASKS IN V C ENTER

Common tasks Required Privileges - p.127

All privileges - p.229

Tools:

vSphere Installation and Setup Guide

vSphere Security Guide
Whats New in the VMware vSphere 6.0 Platform
vSphere Administration with the vSphere Client Guide
vSphere Client / vSphere Web Client

S ERVER

VCP6-DCV OBJECTIVE 1.2 SECURE ESX I, VCENTER SERVER, AND VSPHERE VIRTUAL
M ACHINES
This post covers VCP6-DCV Objective 1.2 - Secure ESXi, vCenter Server, and vSphere Virtual Machines. A very
interesting chapter indeed, where we cover all the "locks" which an admin can put in place to secure his/here
environment. And you don't have to be Linux expert as all this is done without much difficulty!
For whole exam coverage I created a dedicated VCP6-DCV page. Or if you're not preparing to pass a VCP6-DCV, you
might just want to look on some how-to, news, videos about vSphere 6 - check out my vSphere 6 page. If you find out
that I missed something, don't hesitate to comment.

Knowledge

Enable/Configure/Disable services in the ESXi firewall

Enable Lockdown Mode
Configure network security policies
Add an ESXi Host to a directory service
Apply permissions to ESXi Hosts using Host Profiles
Configure virtual machine security policies
Create/Manage vCenter Server Security Certificates

E NABLE/CONFIGURE/DISABLE

SERVICES IN THE

ESX I

FIREWALL

HOW TO ENABLE / DISABLE SERVICES IN THE ESX I FIREWALL - THE HARD WAY ( VIA CLI)

CHECK

WHIH SERVICES ARE ACTIVE

esxcli network firewall ruleset list

O PEN

FIREWALL PORT VIA

CLI:

esxcli network firewall ruleset set -e true -r httpClient

HOW TO ENABLE / DISABLE SERVICES IN THE ESX I FIREWALL - THE EASY WAY ( VIA VS PHERE CLIENT )
Note that you can do the same by selecting the host through vSphere client > configuration > security profile >
Firewall

Services can be Started, Stopped, or Restarted. Services can be configured to Start and stop with host, Start and stop
manually, or Start and stop with port usage.
ESXi Shell and SSH are disabled (Set to Start and stop manually) by default. ESXi Shell and SSH can be enabled/disabled
in the DCUI from the Troubleshooting Mode Options menu.

10

E NABLE LOCKDOWN M ODE

When you enable lockdown mode, you can't connect directly from the console. the host is accessible only through the
vSphere client directly or via vCenter server.

Lockdown Modes:

Disabled - Lockdown mode is disabled.

Normal - Lockdown mode is enabled. The host can only be accessed from vCenter or from the console (DCUI).
Strict - Lockdown mode is enabled. The DCUI service is stopped. The host can not be accessed from the console
(DCUI).

[TIP]: You can activate DCUI from within SSH session

Type this after login in with Putty or other SSH client.
dcui
There you see the DCUI screen

11

vSphere 6 introduced "Exception users" which are users with local accounts or Microsoft Active Directory accounts
with permissions defined locally on the host where these users have host access. You can define those exception
locally on the host, but its not recommended for normal user accounts, but rather for service accounts. You should
set permissions on these accounts to strict minimum and only whats required for the application to do its task and
with an account that needs only read-only permissions to the ESXi host.
This is basically the same principle of local server accounts on Windows member server, where you can create local
accounts, but as a best practice to give them only the permissions they need
Smart Card Authentication to DCUI There is new function, but apparently it is for U.S. federal customers only. It
allows DCUI login access using a Common Access Card (CAC) and Personal Identity Verification (PIV). In this case
the ESXi host must be part of Microsoft AD.

CONFIGURE

NETWORK SECURITY POLICIES

Network security policies are defined on two places:

vSwitch level
Portgroup level

Three different policies:

Promiscuous mode If set to Accept then it allows the guest OS to receive all traffic observed on the
connected vSwitch or PortGroup (the switch becames a HUB basically - with all the inconveniences, packet
colisions, performance degradation etc... ). By default it's Reject
MAC address changes A host is able to accepts requests to change the effective MAC address to a different
address than the initial MAC address. By default it's Accept
Forged transmits A host does not compare source and effective MAC addresses transmitted from a virtual
machine. By default it's Accept

Or via vSphere client (more convenient)

12

MAC address changes and Forged transmits if set to Reject, than it protects against MAC address spoofing. If changing
the settings at the Portgroup level there is an Override checkbox allowing you to set the policy on a portgroup rather
than on the vSwitch.

A DD

AN

ESX I H OST

TO A DIRECTORY SERVICE

Using Active Directory for user authentication simplifies the ESXi host configuration and reduces the risk for
configuration issues that could lead to unauthorized access. You can join or leave domain by selecting a host >
configuration > authentication services > properties. You can also join standalone ESXi hosts to AD. By using AD you
eliminate to manage locally users on ESXi hosts.

A special AD group named "ESX Admins" shall be manually created before host is joined to AD. Why?
Because like this All members of this group (ESX admins) are automatically assigned with the Administrator
role on the host when this host is joined to AD. If not the permissions has to be applied manually.

13

vSphere web client > Hosts and clusters > Select ESXi host > Manage > Settings > Authentication services.

A PPLY

PERMISSIONS TO

ESX I H OSTS

USING

H OST P ROFILES

Host profiles are very cool feature allowing to homogenize configuration across ESXi hosts and automate compliance.
In some cases, host profiles can be also useful when for example you need to reset esxi root password on a host.
Check vSphere Security guide (PDF) on p. 133, but basically this procedure apply:
1. Set up the reference host to specification and create a host profile.
2. Attach the profile to a host or cluster.
3. Apply the host profile of the reference host to other hosts or clusters.

If you haven't done yet, go to Home > Host profiles > Extract profile from host. Once you have that profile you can
apply it to a host...

Select the host profile > Click Actions > Edit Host Profile (or right click > edit settings)
Expand Security and Services
Select the Permission Rules folder > click the Plus Sign

14

Root password is encrypted within the host profile, however by joining hosts to AD via Host profiles leaves password
in plain text... -:(.
Configure virtual machine security policies
VMs are fragile. The same for Guest OS. Treat them accordingly ... -:). Seriously, you should patch to the latest release
for the OS patches, Antivirus patches and/or Malware patches.... That's a bare minimum to prevent system corruption.

Be organized - Use templates to deploy virtual machines

Minimize use of virtual machine console
15

Prevent virtual machines from taking over resources

Disable unnecessary functions inside virtual machines - usually Windows/Linux services can be stopped, to put
them on manual instead of automatic startup, etc..
Remove unnecessary hardware devices - floppy, printers, sound devices... All you don't need you can remove
to have lower overhead.
Disable unused display features
Disable unexposed features
Disable HGFS file transfers
Disable copy and past operations between guest operating system and remote console (by default is disabled
- on per host level, but you can add an advanced settings:)

isolation.tools.copy.disable
isolation.tools.paste.disable = true

true

Limiting exposure of sensitive data copied to the Clipboard

Restrict users from running commands within a virtual machine

1. Click Administration and select Roles > click create role > NO Guest Access > select all privileges
2. Deselect All Privileges >Virtual machine > Guest Operations to remove the Guest Operations set of privileges >
validate OK.

Prevent a virtual machine user or process from disconnecting devices

Modify guest operating system variable memory limit
Prevent guest operating system process from sending configuration messages to the host
Avoid using Independent Nonpersistent Disks - keep in mind non persistent disks are not affected by
snapshots. If you use snapshots. A redo log is created to capture all subsequent writes to that disk. However,
if the snapshot is deleted, or the virtual machine is powered off, the changes captured in that redo log are
discarded for that Independent Non-persistent VMDK.

CREATE/M ANAGE V C ENTER S ERVER S ECURITY CERTIFICATES

Certificates got easier with vSphere 6 as those can be viewed and renewed within vSphere Web client.

There are two operations modes:

Root CA - (by default)

Issuer CA possibility integrate Microsoft Certification authority. In this case youll create the CSR (request) >
Go to Microsoft Cert Server and get certificate.

To view certificates:

16

The VMware Certificate Authority (VMCA) provisions vCenter Server components and ESXi hosts with certificates that
use VMCA as the root certificate authority by default.
The vSphere Certificate Manager utility allows you to perform most certificate management tasks interactively from
the command line.
Example. On Windows you must go to this directory:
C:\Program Files\VMware\vCenter Server\vmcad\certificate-manager.bat
Link to Online documentation for using vSphere Certificate manager utility.
vSphere Certificate Manager prompts you for the task to perform, for certificate locations and other information as
needed, and then stops and starts services and replaces certificates for you.
vCenter Certificate Utilities:

vSphere Certificate Manager utility certificate replacement tasks from a command line utility.
Certificate management CLIs dir-cli, certool, and vecs-cli command line utilities.
o certool can Generate and manage certificates and keys. Part of VMCA.
o dir-cli is a able to create and update certificates in VMware Directory Service. Part of VMAFD.
o ves-cli can manage the contents of VMware Certificate Store instances. Part of VMAFD

vSphere Web Client certificate management view certificate information in the Web Client

Tools

vSphere Installation and Setup Guide

vSphere Security Guide
Whats New in the VMware vSphere 6.0 Platform
Security of the VMware vSphere Hypervisor
vSphere Administration with the vSphere Client Guide
VMware Hardened Virtual Appliance Operations Guide added to Tech Resource Directory
vSphere Client / vSphere Web Client

VCP6-DCV OBJECTIVE 1.3 - ENABLE SSO AND ACTIVE DIRECTORY INTEGRATION

In no particular order I'll start covering VCP6-DCV sections to help out folks learning towards VCP6-DCV VMware
certification exam. Due to VMware recertification policy the VCP exam has now an expiration date. You can renew by
passing delta exam while still holding current VCP or pass VCAP. The topic today - VCP6-DCV Objective 1.3 - Enable
SSO and Active Directory Integration.
For whole exam coverage I created a dedicated VCP6-DCV Wordpress page. If you just look on some how-to, news,
videos about vSphere 6 check out my vSphere 6 page. vSphere 6 grew up quite big compared to vSphere 5.5 release,
but simplified the deployment and management. vSphere Web client is more present and used in this release as the
legacy C# client does not allow to configure advanced configuration options and functions like SSO, FT, VSAN

You'll need certain knowledge that we'll try to cover today:

Configure/Manage Active Directory Authentication

Configure/Manage Platform Services Controller (PSC)
Configure/Manage VMware Certificate Authority (VMCA)
Enable/Disable Single Sign-On (SSO) Users
17

Identify available authentication methods with VMware vCenter

CONFIGURE/M ANAGE A CTIVE DIRECTORY A UTHENTICATION

Step 1: Connect to your vCenter server by entering the ip address you have entered during the deployment
process:
https://vCenter Server IP/vsphere-client
and by using the [email protected] as a user name and your password you have used during the
deployment.

Step 2: Click the Administration button on the left and

And then go to Single Sign-On > Configuration > Identity Sources > Click the "+" sign to add your AD as an identity
source. Normally it will populate your local AD automatically, so you just have to click the OK button...
18

You can also click the globe icon to make the AD as the default while you're there...
Screenshot showing the Identity source where we added our AD - lab.local

N EXT S TEP : P ERMISSIONS

You'll need to assign permissions to users which will administer the vSphere infrastructure. Usually it's domain admin,
but not always..... Also keep in mind where you assign those permissions. If it's at the Datacenter level, vCenter level
or at the cluster level... Usually you'll want to do it at the vCenter Level.
Go to Home > vCenter Inventory Lists > vCenter Servers > vCenter.lab.local (in my case) > Click the Manage Tab >
Permissions
There you click the "+" sign > Add button > make sure that you select the drop-down for your Microsoft Ad to make
appear the Domain admin user...

19

Click OK to validate. You can disconnect and connect as domain admin now... Note that in case your workstation is
part of Microsoft AD, you just have to check the box and no need to enter your domain user password... -:)

Some of you might wonder why there is this Single Sign-On. The vCenter Single Sign On is an authentication service
which allows the different vSphere software components present in the vCloud suite, to communicate between each
other via a secure token exchange mechanism.

CONFIGURE/M ANAGE P LATFORM S ERVICES CONTROLLER (PSC)

The Platform Services Controller (PSC) provides:

Single Sign-On (SSO)

Licensing
20

Certificate Authority (VMCA)

You can deploy it on at the same time or a part and you can deploy it as Windows based or Appliance based (VCSA). It's
important to know that PSO is completely transparent working with Windows or VCSA based vCenter!
PSC Deployment Options - A two different type installation are allowed:

Embedded (in the same VM)

External

The embedded PSC is meant to be used for standalone sites where vCenter server will be the only SSO integrated
solution. In this case a replication to another PSC is not necessary.
External PSC shall be deployed in anvironments where there is more then one SSO enabled solution (vCenter Server,
vRealize Automation, etc) OR where replication to another PSC (another site) is necessary.
Here is the screenshot from the installation process (VCSA) showing the different options and changing the options
also changes the different phases of the deployment (on the left).

PSC features:

Manages and generates SSL certificates for your vSphere environment.

Stores and replicates VMware License Keys
Stores and replicates permissions via the Global Permissions layer.
Manages the storage and replication of TAGS and CATEGORIES.
There is a Built-in automatic replication between different, logical SSO sites. (if any)
There is only one single default domain for the identity sources.

D EPLOYMENT O PTIONS:

21

Embedded Platform Service Controller

All services bundled with the Platform Services Controller are deployed on the same virtual machine or
physical server as vCenter Server.
External Platform Service Controller
The services bundled with the Platform Services Controller and vCenter Server are deployed on different
virtual machines or physical servers.

Recommended reads:
VMware vSphere Blog - vCenter Server 6 Deployment Topologies and High Availability.
VMware KB - Recommended topologies for vSphere 6.0.x (2108548).
Configure/Manage VMware Certificate Authority (VMCA)
When you first install vSphere, the default certificates are deployed with 10 years of life span. The VMCA generates
those self-signed certs during the installation process, and provisions each of the ESXi host with a signed certificate
by this root certificate authority. Earlier versions of vSphere with self-signed certificates are automatically replaced
by new self-signed certificates by VMCA.
There are different ESXi Certificate replacement modes:

Default - VMCA as cert authority where VMCA issues certs for your hosts.
Custom - you can override and do and issue certs manually via VMCA
Thumbprint mode - this way you keep certs from vSphere 5.5

To check this go to the View Support Information after logging to your ESXi host:

22

W HERE TO CHECK THE CERTIFICATES IN W EB CLIENT ?

Home -> System Configuration -> Nodes -> Node -> Manage -> Certificate Authority
Note: If you're not a member of SystemConfiguration.Administrators group than you might want to add yourself there.
If of course you're connecting as an domain administrator....

Back to where to check the certificates on vSphere Web Client:

Home > System Configuration > Nodes > Node > Manage > Certificate Authority

E NABLE/DISABLE S INGLE S IGN-O N (SSO) USERS

The VMware SSO uses different configuration policy which can be found via vSphere Web client only:

Administration > Single Sign-On > Configuration Policies

23

Password Policy
Lockout Policy
Token Policy

P ASSWORD P OLICY

You can configure the following parameters:

Description Password policy description. Required.

Maximum lifetime Maximum number of days that a password can exist before it has to be changed.
Restrict re-use Number of the users previous passwords that cannot be set again.
Maximum length Maximum number of characters that are allowed in the password.
Minimum length Minimum number of characters required in the password.
Character requirements Minimum number of different character types required in the password.
Identical adjacent characters Maximum number of identical adjacent characters allowed in the password.

To get to this screen You must click Administration > Single Sign-On > Configuration

By clicking the Edit button you are able to change values there

24

If you leave the default values and after 90 days you will want to log-in you might end up with messages saying that:

User Account is locked.

User Account is disabled.

Those SSO policies are pretty much the same as in vSphere 5.5, but with a difference that in vSphere 5.5 we also had
an administrator password expiry on the vCenter server appliance (VCSA). The VCSA 6.0 is pretty much locked out and
the GUI we use to manage VCSA accessible via the port 5480 is no longer available.
Lockout Policy
Specifies the condition under which a vCenter SSO account is locked when the user attempts to log in with incorrect
credentials. Five login attempts and three minutes between failures are set by default. This policy also specifies the
time that must elapse before the account is automatically unlocked.

Description Description of the lockout policy. Required.

Max. number of failed login attempts Maximum number of failed login attempts that are allowed before
the account is locked.
Time interval between failures (seconds) Time period in which failed login attempts must occur to trigger a
lockout.
Unlock time (seconds) Amount of time that the account remains locked. If you enter 0, the account must be
explicitly unlocked by an administrator.

To see the lockout policy parameters, click on the Policies tab and select Lockout Policy:

Token Policy - also interesting as for example the Clock tolerance shows time difference, in milliseconds, that vCenter
Single Sign-On tolerates between a client clock and the domain controller clock. If the time difference is greater than
the specified value, vCenter Single Sign-On declares the token invalid.

25

Other configuration options:

Maximum token renewal count Maximum number of times that a token can be renewed. After the
maximum number of renewal attempts, a new security token is required.
Maximum token delegation count Holder-of-key tokens can be delegated to services in the vSphere
environment. A service that uses a delegated token performs the service on behalf of the principal that
provided the token. A token request specifies a DelegateTo identity. The DelegateTo value can either be a
solution token or a reference to a solution token. This value specifies how many times a single holder-of-key
token can be delegated.
Maximum bearer token lifetime Bearer tokens provide authentication based only on possession of the
token. Bearer tokens are intended for short-term, single-operation use. A bearer token does not verify the
identity of the user or entity that is sending the request. This value specifies the lifetime value of a bearer
token before the token has to be reissued.
Maximum holder-of-key token lifetime Holder-of-key tokens provide authentication based on security
artifacts that are embedded in the token. Holder-of-key tokens can be used for delegation. A client can obtain
a holder-of-key token and delegate that token to another entity. The token contains the claims to identify the
originator and the delegate. In the vSphere environment, a vCenter Server obtains delegated tokens on a
users behalf and uses those tokens to perform operations. This value determines the lifetime of a holder-ofkey token before the token is marked invalid.

IDENTIFY

AVAILABLE AUTHENTICATION METHODS WITH

VM WARE V CENTER

We have already saw that at the beginning of the post. The possible identity sources can be found via web client >
Administration > Single Sign-On > Configuration > Identity Sources
And we can see that there are four of them:

AD integrated (preferred)
Active Directory LDAP
Open LDAP
Local OS

Yep, you can obviously use Local OS option only if you don't want to interconnect with your AD (for security reasons
or isolation purposes).
Check How-to, news, videos and tutorials at my vSphere 6 page too or check Free VMware tools page.
Tools to get the knowledge and further reading:

vSphere Installation and Setup Guide

vSphere Security Guide
Whats New in the VMware vSphere 6.0 Platform
VMware vCenter Server 6.0 Deployment Guide
Direct Console User Interface (DCUI)
vSphere Client / vSphere Web Client

VCP6-DCV OBJECTIVE 2.1 - CONFIGURE ADVANCED POLICIES/FEATURES AND VERIFY

NETWORK VIRTUALIZATION IMPLEMENTATION
Today's VCP6-DCV topic Objective 2.1: Configure Advanced Policies/Features and Verify Network Virtualization
Implementation is the core of virtualization networking. Together with 2 other chapters it covers all vSphere 6
networking.

26

You can follow the VCP6-DCV study guide built through my VCP6-DCV page. When finished, there will be a PDF version
which will get its proper formatting for better reading experience. We're more than half way through right now, and
the work continues. Let's kick on with this chapter!

vSphere Knowledge

Identify vSphere Distributed Switch (vDS) capabilities

Create/Delete a vSphere Distributed Switch
Add/Remove ESXi hosts from a vSphere Distributed Switch
Add/Configure/Remove dvPort groups
Add/Remove uplink adapters to dvUplink groups
Configure vSphere Distributed Switch general and dvPort group settings
Create/Configure/Remove virtual adapters
Migrate virtual machines to/from a vSphere Distributed Switch
Configure LACP on Uplink portgroups
Describe vDS Security Polices/Settings
Configure dvPort group blocking policies
Configure load balancing and failover policies
Configure VLAN/PVLAN settings
Configure traffic shaping policies
Enable TCP Segmentation Offload support for a virtual machine
Enable Jumbo Frames support on appropriate components
Determine appropriate VLAN configuration for a vSphere implementation

IDENTIFY V S PHERE DISTRIBUTED S WITCH ( V DS)

CAPABILITIES

VMware vSphere Distributed Switch (vDS) is in its version 6 and packed in more feature than in previous relase of VDS.
If you're upgrading you shall upgrade vDS to version 6.0 as well to benefit the latest features.
The vDS separates the data plane and management plane to separate them. The data plane resides on ESXi host, but
the management plane moves to vCenter server. The data plane is called host proxy switch.

NetFlow Support - Netflow is used for troubleshooting, it picks a configurable number of samples of network
traffic for monitoring..
PVLAN Support - PVLAN is able to get more from VLANs (which are limited in numbers) and you can use these
PVLANS to further segregate your traffic and increase security. (Note: Enterprise plus licensing required! Check
my detailed post on PVLANs here.
Ingress and egress traffic shaping - Inbound/outbound traffic shaping, which allows you throttle bandwidth
to the switch.
VM Port Blocking - can block VM ports in case of viruses or troubleshooting...
Load Based Teaming - LBT is an additional load balancing that works off the amount of traffic a queue is
sending
Central Management across cluster - vDS can create the config once and push it to all attached hosts...so you
don't have to go to each host one-by-one...
Per Port Policy Settings - It's possible to override policies at a port level which gives you more controll
Port State Monitoring - This feature allows each port to be monitored separately from other ports
LLDP - Allows supports for link layer discovery protocol
Network IO Control - possibility to set priority on port groups and reserve bandwidth for VMs connected to
this port group. Check the detailed chapter on NIOC here: Objective 2.2: Configure Network I/O Control (NIOC)
LACP Support - LACP (Link aggregation control protocol) ability to aggregate links together into a single link
(your physical switch must support it!)
Backup/Restore Network config - It's possible to backup/restore network config at the vDS level (Not new!
It's here since 5.1! - save and restore network config...)
Port Mirroring - Allows monitoring and can send all traffic from one port to another
27

Stats stays at the VM level - statistics move with the VM even after vMotion.

CREATE/DELETE

A V S PHERE

DISTRIBUTED S WITCH

Create a vSphere vDS - Networking Guide on p27. vSphere Web client > Networking > Rigt click datacenter >
Distributed switch > New Distributed switch

Put a name and then select the version...

Select how many uplinks, specify if you want to enable Network I/O control and rename the default port group (not
mandatory)...

28

A DD /R EMOVE ESX I

HOSTS FROM A VS PHERE

DISTRIBUTED S WITCH

You can add/remove ESXi hosts from vDS to manage their networking (or not) from a central location. The good thing
is that you can analyse impact before breaking a connectivity, so you're able to see the impact. The impact can be as
follows:

No Impact
Important impact
Critical Impact

Next...

A DD /CONFIGURE/R EMOVE

DV P ORT GROUPS

Right click on the vDS > New Distributed Port Group.

To remove a port group. Simple. Right click on the port group > delete...
29

A DD /R EMOVE

UPLINK ADAPTERS TO DV U PLINK GROUPS

Again, right click is your friend... -:)

If you want to add/remove (increase or decrease) number of uplinks you can do so by going to the properties of the
vDS.

Right click on the vDS > Edit settings

And on the next screen you can do that... Note that at the same time you can give a different names to your uplinks...

30

CONFIGURE V S PHERE DISTRIBUTED S WITCH

GENERAL AND DV P ORT GROUP SETTINGS

General properties of vDS can be reached via Right click on the vDS > Settings > Edit settings

Port binding properties (at the dvPortGroup level - Right click port group > Edit Settings)

Static binding - Assigns a port to a VM when the virtual machine is connected to the PortGroup.
Dynamic binding - it's kind of deprecated. For best performance use static binding
Ephemeral no binding

Port allocation:

31

Elastic - Increase or decreas on-the-fly..... 8 at the beginning (default). Increases by 8 when needed.
Fixed - There is 128 by default.

CREATE/CONFIGURE/R EMOVE

VIRTUAL ADAPTERS

VMkernel adapters can be add/removed at the Networking level

vSphere Web Client > Host and Clusters > Select Host > Manage > Networking > VMkernel adapters

Different VMkernel Services, like :

vMotion traffic
Provisioning traffic
Fault Tolerance (FT) traffic
Management traffic
vSphere Replication traffic
vSphere Replication NFC traffic
VSAN traffic

M IGRATE

VIRTUAL MACHINES TO/ FROM A VS PHERE

DISTRIBUTED SWITCH

Migrate VMs to vDS. Right click vDS > Migrate VM to another network

Make sure that you previously created a distributed port group with the same VLAN that the current VM is running...
(in my case the VMs run at VLAN 7)

32

Pick a VM...

Done!

CONFIGURE LACP

ON

UPLINK

PORTGROUPS

LACP can be found in the Networking guide on p.65.

vSphere Web Client > Networking > vDS > Manage > Settings > LACP
Create Link Aggregation Groups (LAG)

33

LAG Mode can be:

Passive - where the LAG ports respond to LACP packets they receive but do not initiate LACP negotiations.
Active - where LAG ports are in active mode and they initiate negotiations with LACP Port Channel.

LAG load balancing mode (LNB mode):

Source and destination IP address, TCP/UDP port and VLAN

Source and destination IP address and VLAN
Source and destination MAC address
Source and destination TCP/UDP port
Source port ID
VLAN

Note that you must configure the LNB hashing same way on both virtual and physical switch, at the LACP port channel
level.
Migrate Network Traffic to Link Aggregation Groups (LAG)

34

DESCRIBE V DS S ECURITY P OLICES /S ETTINGS

Note that those security policies exists also on standard switches.
There are 3 different network security policies:

Promiscuous mode Reject is by default. In case you set to Accept > the guest OS will receive all traffic
observed on the connected vSwitch or PortGroup.
MAC address changes Reject is by default. In case you set to Accept > then the host will accepts requests to
change the effective MAC address to a different address than the initial MAC address.
Forged transmits Reject is by default. In case you set to Accept > then the host does not compare source
and effective MAC addresses transmitted from a virtual machine.

35

Network security policies can be set on each vDS PortGroup.

Configure dvPort group blocking policies
Port blocking can be enabled on a port group to block all ports on the port group

or you can configure the vDS or uplink to be blocked at the vDS level...

vSphere Web Client > Networking > vDS > Manage > Ports

And then select the port > edit settings > Miscellaneous > Override check box > set Block port to yes.

36

CONFIGURE

LOAD BALANCING AND FAILOVER POLICIES

Load balancing algos can be found in the Networking Guide on p. 91.

vDS load balancing (LNB):

Route based on IP hash - The virtual switch selects uplinks for virtual machines based on the source and
destination IP address of each packet.
Route based on source MAC hash - The virtual switch selects an uplink for a virtual machine based on the
virtual machine MAC address. To calculate an uplink for a virtual machine, the virtual switch uses the virtual
machine MAC address and the number of uplinks in the NIC team.
Route based on originating virtual port - Each virtual machine running on an ESXi host has an associated
virtual port ID on the virtual switch. To calculate an uplink for a virtual machine, the virtual switch uses the
virtual machine port ID and the number of uplinks in the NIC team. After the virtual switch selects an uplink
for a virtual machine, it always forwards traffic through the same uplink for this virtual machine as long as the
machine runs on the same port. The virtual switch calculates uplinks for virtual machines only once, unless
uplinks are added or removed from the NIC team.
Use explicit failover order - No actual load balancing is available with this policy. The virtual switch always
uses the uplink that stands first in the list of Active adapters from the failover order and that passes failover
detection criteria. If no uplinks in the Active list are available, the virtual switch uses the uplinks from the
Standby list.
Route based on physical NIC load (Only available on vDS) - based on Route Based on Originating Virtual Port,
where the virtual switch checks the actual load of the uplinks and takes steps to reduce it on overloaded
uplinks. Available only for vSphere Distributed Switch. The distributed switch calculates uplinks for virtual
machines by taking their port ID and the number of uplinks in the NIC team. The distributed switch tests the
uplinks every 30 seconds, and if their load exceeds 75 percent of usage, the port ID of the virtual machine with
the highest I/O is moved to a different uplink.

Virtual switch failover order:

Active uplinks
Standby uplinks
Unused uplinks
37

CONFIGURE VLAN/PVLAN

SETTINGS

private VLANs allows further segmentation and creation of private groups inside each of the VLAN. By using private
VLANs (PVLANs) you splitting the broadcast domain into multiple isolated broadcast subdomains.
Private VLANs needs to be configured at the physical switch level (the switch must support PVLANs) and also on the
VMware vSphere distributed switch. (Enterprise Plus is required). Its more expensive and takes a bit more work to
setup.

THERE

ARE DIFFERENT TYPES OF

PVLAN S :

P RIMARY

Promiscuous Primary VLAN Imagine this VLAN as a kind of a router. All packets from the secondary VLANS
go through this VLAN. Packets which also goes downstream and so this type of VLAN is used to forward packets
downstream to all Secondary VLANs.

S ECONDARY

Isolated (Secondary) VMs can communicate with other devices on the Promiscuous VLAN but not with other
VMs on the Isolated VLAN.
Community (Secondary) VMs can communicate with other VMs on Promiscuous and also w those on the
same community VLAN.

The graphics shows it all

CONFIGURE

TRAFFIC SHAPING POLICIES

Networking Guide p.105

vDS supports both ingress and egress traffic shaping.
38

Traffic shaping policy is applied to each port in the port group. You can Enable or Disable the Ingress or egress traffic

Average bandwidth in kbits (Kb) per second - Establishes the number of bits per second to allow across a port,
averaged
over time. This number is the allowed average load.

Peak bandwidth in kbits (Kb) per second - Maximum number of bits per second to allow across a port when
it is sending or receiving a burst of traffic. This number limits the bandwidth that a port uses when it is using
its burst bonus.
Burst size in kbytes (KB) per second - Maximum number of bytes to allow in a burst. If set, a port might gain
a burst bonus if it does not use all its allocated bandwidth. When the port needs more bandwidth than
specified by the average bandwidth, it might be allowed to temporarily transmit data at a higher speed if a
burst bonus is available

E NABLE TCP S EGMENTATION O FFLOAD

SUPPORT FOR A VIRTUAL MACHINE

Use TCP Segmentation Offload (TSO) in VMkernel network adapters and virtual machines to improve the network
performance in workloads that have severe latency requirements.
When TSO is enabled, the network adapter divides larger data chunks into TCP segments instead of the CPU. The
VMkernel
and
the
guest
operating
system
can
use
more
CPU
cycles
to
run
applications.
By default, TSO is enabled in the VMkernel of the ESXi host , and in the VMXNET 2 and VMXNET 3 virtual machine
adapters

E NABLE JUMBO F RAMES

SUPPORT ON APPROPRIATE COMPONENTS

There are many places where you can enable Jumbo frames and you should enable jumbo frames end-to-end. If not
the performance will not increase, but rather the opposite. Jumbo Frames can be enabled on a vSwitch, vDS, and
VMkernel Adapter.

39

Jumbo frames maximum value = 9000.

DETERMINE

APPROPRIATE

VLAN

CONFIGURATION FOR A V S PHERE IMPLEMENTATION

There are three main places or three different ways to tag frames in vSphere.

External Switch Tagging (EST) - VLAN ID is set to None or 0 and it is the physical switch that does the VLAN
tagging.
Virtual Switch Tagging (VST) - VLAN set between 1 and 4094 and the virtual switch does the VLAN tagging.
Virtual Guest Tagging (VGT) - the tagging happens in the guest OS. VLAN set to 4095 (vSwitch) or VLAN
trunking on vDS.

The best to understand this is I guess this document from VMware called Best Practices for Virtual Networking and
from there I also "borrowed" this screenshot...

Networking is big chapter. If I missed something, just comment or email me your suggestion. Thanks...
vSphere documentation tools

vSphere Installation and Setup Guide

vSphere Networking Guide
Whats New in the VMware vSphere 6.0 Platform
Leveraging NIC Technology to Improve Network Performance in VMware vSphere
40

VDS Network Health Check

vSphere Client / vSphere Web Client

VCP6-DCV OBJECTIVE 2.2 - CONFIGURE NETWORK I/O CONTROL (NIOC)

VCP6-DCV Study time... In no particular order I start covering VCP6-DCV section of the VMware blueprint to help out
folks learning towards VCP6-DCV VMware certification exam. Due to VMware recertification policy the VCP exam has
now an expiration date. You can renew by passing delta exam while still holding current VCP or pass VCAP. If you're
new to virtualization and do not have any VMware certification exam, the VCP is the exam to have. Today's topic?
VCP6-DCV Objective 2.2 - Configure Network I/O Control (NIOC).
For whole exam coverage I created a dedicated VCP6-DCV page. If you just look on some how-to, news, videos about
vSphere 6 check out my vSphere 6 page. vSphere 6 grew up quite big compared to vSphere 5.5 release, but simplified
the deployment and management. "White boxing" got more complicated as drivers for unsupported hardware not
always works. vSphere Web client is more present and used in this release as the legacy C# client does not allow to
configure advanced configuration options and functions like SSO, FT, VSAN. Let's get started.

vSphere Knowledge

Identify Network I/O Control requirements

Identify Network I/O Control capabilities
Enable/Disable Network I/O Control
Monitor Network I/O Control

IDENTIFY N ETWORK I/O CONTROL

REQUIREMENTS

What is network I/O control? It's a mechanism which allows to prioritize certain data flows on distributed switch over
others. It allows to allocate more network bandwidth to business critical applications/VMs where those have to "fight"
for bandwidth. (similarly like SIOC for storage).

THE

REQUIREMENTS :

Licensing - Enterprise + license required because it uses vSphere Distributed Switch.

VDS Only - the Network I/O control can be enabled only on VDS
Network I/O control v3 possible only on VDS 6.0
SR-IOV is not available for virtual machines configured to use Network I/O Control version 3.

IDENTIFY N ETWORK I/O CONTROL

CAPABILITIES

When enabled NIOC divides the traffic into resource pools. Bandwidth reservations can be used to isolate network
resources for a class of traffic, for example in VSAN cluster you'd want to reserve part of the traffic only for VSAN
traffic no matter what happens to the other traffic.

E NABLE/DISABLE N ETWORK I/O C ONTROL

Where to enable? In vSphere 6 when creating new VDS it gets enabled by default.
vSphere Web Client > Networking > vDS > Manage > Resource Allocation > System traffic
Note: If you have previous version of vSphere and you upgraded, than you might see previous version of NIOC (version
2) and so there is not the menu "system traffic". Make sure that you upgrade your VDS to v 6.0.
41

So in our case we can see the menu system traffic... The traffic types are all set to 50 shares except the VM
traffic. No reservation or limits are set by default.

Management traffic - VM traffic

NFS traffic
Virtual SAN traffic
iSCSI
vMotion
vSphere Replication (VR)
Fault tolerance (FT)
vSphere Data protection (VDP) backup traffic

Shares and reservations at their default state. No limits or Reservations.

B ANDWIDTH A LLOCATION FOR V IRTUAL M ACHINE TRAFFIC

Version 3 of Network I/O Control lets you configure bandwidth requirements for individual virtual machines. You can
also use network resource pools where you can assign a bandwidth quota from the aggregated reservation for the
virtual machine traffic and then allocate bandwidth from the pool to individual virtual machines.
42

Individual VMs can be configured according to bandwidth requirements through VM options at the network level...

Shares - The relative priority, from 1 to 100, of the traffic through this VM network adapter against the capacity of the
physical
adapter
that
is
carrying
the
VM
traffic
to
the
network.
Reservation - The minimum bandwidth, in Mbps, that the VM network adapter must receive on the physical adapter.
Limit - The maximum bandwidth on the VM network adapter for traffic to other virtual machines on the same or on
another host.
Enable/Disable Network I/O Control - at the vDS level..

To enable bandwidth allocation for virtual machines by using Network I/O Control, configure the virtual machine
system traffic. The bandwidth reservation for virtual machine traffic is also used in admission control. When you power
on a virtual machine, admission control verifies that enough bandwidth is available.
43

Check the following requirements:

vSphere Distributed Switch is version 6.0.0 and later.

Network I/O Control on the switch is version 3.
Network I/O Control is enabled.

Network Resource Pools - You can create new network resource pools to reserve part of the aggregated bandwidth
for VMs system trafic on all the physical adapters connected to the VDS.
For example, if the virtual machine system traffic has 0.5 Gbps reserved on each 10 GbE uplink on a distributed switch
that has 10 uplinks, then the total aggregated bandwidth available for VM reservation on this switch is 5 Gbps. Each
network resource pool can reserve a quota of this 5 Gbps capacity.
Example from vSphere Networking Guide p.167

Create network resource pool: Distributed switch > Manage > Resource allocation > Network resource pools > Add
Once you create network resource pool you can add distributed port group so you an allocate bandwidth to the VMs
that are connected to that portgroup.
Monitor Network I/O Control
You can check and monitor Network I/O Control through vSphere web client. Networking > vDS > Manage > Resource
Allocation
Concerning the system traffic it's possible to have a look a those metrics and details:

44

Network I/O Control Status (state is Enabled/Disabled)

NIOC Version
Physical network adapters details
Available bandwidth capacity
Total bandwidth capacity
Maximum reservation allowed
Configured reservation
Minimum link speed

Documentation and Tools

vSphere Installation and Setup Guide

vSphere Networking Guide
Whats New in the VMware vSphere 6.0 Platform
Performance Evaluation of Network I/O Control in VMware vSphere 6
vSphere Client / vSphere Web Client

VCP6-DCV OBJECTIVE 2.3 CONFIGURE VSS AND VDS POLICIES

VCP6-DCV Study guide continues today by covering the VCP6-DCV Objective 2.3 - Configure vSS and vDS Policies.
vSphere networking is one of the tough parts to know and this part is where any IT admins have difficulties. This
chapter works hand in hand with the VCP6-DCV Objective 2.1 Configure Advanced Policies/Features and Verify
Network Virtualization Implementation.
You can also check the vSphere 6 page where you'll find many how-to, videos, and tutorials about vSphere 6. Let's get
back to our today's objective.

vSphere Knowledge

Identify common vSS and vDS policies

Describe vDS Security Polices/Settings
Configure dvPort group blocking policies
Configure load balancing and failover policies
Configure VLAN/PVLAN settings
Configure traffic shaping policies
Enable TCP Segmentation Offload support for a virtual machine
Enable Jumbo Frames support on appropriate components
Determine appropriate VLAN configuration for a vSphere implementation

IDENTIFY

COMMON V SS AND V DS POLICIES

Since vSphere 4 we have had vSphere distributed switches. But let's start with virtual standard switches first.
The virtual standard switches (vSS) can have following policies and settings:

Traffic shaping (outbound only)

VLANs (none, VLAN ID, All) - at the portgroup level config
MTU
Teaming and failover
Security

45

If you set VLAN policy to 4095 (All) it allows you to pass All VLANs, and the tagging is done at the Guest
OS level
vSphere distributed switches (vDS) policies and settings:

Traffic filtering and marking

MTU
VLANs (none, VLAN ID, VLAN trunking, PVLANs)
Monitoring (netflow)
Security
Traffic Shaping - inbound and outbount (ingress / egress)
LACP
Port mirroring
Health check for VLAN and MTU, teaming and failover - allows to check the status of the overall config.
And Teaming and failiover like on vSS swiches.

DESCRIBE V DS S ECURITY P OLICES /S ETTINGS

There are three network security policies on vDS. Those are promiscuous mode, MAC address changes and Forged
transmits.

Promiscuous Mode - Default settings are set to reject for both (VSS and VDS). If you change to accept then
the guest OS can receive all traffic which passes through the vSwitch or Portgroup.
MAC address change - The default setting is reject for VDS but accept on VSS. If set to allow then the host
accepts requests to change the effective MAC address to a different one than the original.
Forged transmits - The default settings is Reject for VDS but accept on VSS. The host do not compare source
and effective MAC addresses which are transmitted from a VM.

Each settings can be set to Accept or Reject and it can be done at the virtual switch level or at the port group level.
More granular ist's obviously at the port group level.

CONFIGURE

DV P ORT GROUP BLOCKING POLICIES

Ports can be blocked to prohibit them from sending or receiving data. Only available for distributed switches.
The port blocking policy is done at the portgroup level. vSphere web client > Networking > Right click a portgroup >
Edit settings.

Then you get the Miscelaneous option

46

You can also block individual distributed switch or uplink port. It can be done by selecting the VDS > Manage > Ports
> Select Port > Edit > check the box and select Yes.

CONFIGURE

LOAD BALANCING AND FAILOVER POLICIES

vSphere Networking Guide on p. 93

You can configure various load balancing algorithms on a virtual switch to determine how network traffic is
distributed between the physical NICs in a team.

Route Based on Originating Virtual Port - The virtual switch selects uplinks based on the virtual machine port
IDs on the vSphere Standard Switch or vSphere Distributed Switch.

47

Route Based on Source MAC Hash - The virtual switch selects an uplink for a virtual machine based on the
virtual machine MAC address. To calculate an uplink for a virtual machine, the virtual switch uses the virtual
machine MAC address and the number of uplinks in the NIC team.
Route Based on IP Hash - The virtual switch selects uplinks for virtual machines based on the source and
destination IP address of each packet
Route Based on Physical NIC Load - Route Based on Physical NIC Load is based on Route Based on Originating
Virtual Port, where the virtual switch checks the actual load of the uplinks and takes steps to reduce it on
overloaded uplinks.

And for VDS there is another one called Use Explicit Failover Order.

Use Explicit Failover Order - No actual load balancing is available with this policy. The virtual switch always
uses the uplink that stands first in the list of Active adapters from the failover order and that passes failover
detection criteria. If no uplinks in the Active list are available, the virtual switch uses the uplinks from the
Standby list.

N ETWORK F AILOVER D ETECTION OPTIONS :

Link Status only - check link availability. Is the adapter is physically up or down? Depending on the result it can
possibly detects physical switch failures.
Beacon Probing - Sends out and listens for beacon probes on all NICs in the team. Can be used together with
link status and get better results to determine if there is a link failure. Beacon probing should not be used with
IP hash load balancing policy or on vSwitches which has less than 3 uplinks. Unused NICs do not participate in
beacon probing. Active/active or active/standby only.

FAILOVER ORDER :
It can be specified at the vSwitch level or at the port group level, where you basically override the vSwitch level policy
(VSS). If there is a failover, then standby NIC became active in order that they're specified/listed. You must define if
during failback the physical adapter is returned to active state (and if it is!).

CONFIGURE VLAN/PVLAN

SETTINGS

3 types of VLAN:

None - no tags. Physical switch ports are configured as an access ports or VLAN is configured as native VLAN
on trunk port
VLAN - in this case, the VLAN ID Tag is done on the virtual switch level.
VLAN Trunking - VLANs are tagged at the guest OS level.
PVLAN - private VLANs

Note: Same for vSphere web client. Youll be doing it at the vDS level, so select and right click the vDS > Edit Settings
> Private VLAN tab. Once there you can add some PVLANs. Notice the Secondary Promiscuous was created
automatically when you created the Primary private VLAN.

48

So in my example above I created Primary Private VLAN 500 which automatically created secondary PVLAN 500. Then
I only could create an Isolated Secondary VLAN 501 and Community VLAN 502.
Now we have those PVLANs created and this gives us the possibility to use them for new or existing port
groups. Example below Im creating new port group with some name and after selecting the PVLAN, a new drop-down
menu appears which gives the option to choose an entry between the Isolated, or Community.

THERE

ARE DIFFERENT TYPES OF

PVLAN S :

P RIMARY

Promiscuous Primary VLAN Imagine this VLAN as a kind of a router. All packets from the secondary VLANS
go through this VLAN. Packets which also goes downstream and so this type of VLAN is used to forward packets
downstream to all Secondary VLANs.

S ECONDARY

Isolated (Secondary) VMs can communicate with other devices on the Promiscuous VLAN but not with other
VMs on the Isolated VLAN.
Community (Secondary) VMs can communicate with other VMs on Promiscuous and also w those on the
same community VLAN.

49

CONFIGURE

TRAFFIC SHAPING POLICIES

On vDS there are Ingress and Egress traffic shaping policies.

Average bandwidth in kbits (Kb) per second - Bits per second to allow across a port, averaged over time.
Peak bandwidth in kbits (Kb) per second - Maximum number of bits per second to allow across a port when it
is sending or receiving a burst of traffic.
Burst size in kbytes (KB) per second - Maximum number of bytes to allow in a burst.

At the port group level (both Web client or vSphere client). Home > Networking > right click the port group > traffic
shaping.

E NABLE TCP S EGMENTATION O FFLOAD

SUPPORT FOR A VIRTUAL MACHINE

(TSO)

TCP segmentation offload is used for reducing a CPU overhead of TCP/IP on fast networks. TSO breaks down large
groups of data sent over a network into smaller segments that pass through all the network elements between the
source and destination.
Only on enhanced vmxnet adapters. If you using just vmxnet you must replace the adapter by enhanced vmxnet
adapter.
From VMware KB Enabling TSO in a Windows virtual machine
To use TSO, enable it in three places: the VMkernel, the virtual machine, and the guest operating system.
1. TSO is enabled for the VMkernel by default. If it is disabled on your system, you can enable it in the VMware
Management Interface Advanced Settings page. Access this page by clicking the Options tab.
2. Enable TSO for the virtual machine by powering off the virtual machine and adding the following line to the
configuration file (.vmx):ethernetn.features = "0x2"
In this example, n is the number of the virtual Ethernet adapter.
How to check If a physical network adapter supports TSO?
If yes, then TSO is enabled by default.

50

Via CLI - Run this command

to see if TSO is supported on the physical network adapter on a host:
esxcli network nic tso get

lab output:

E NABLE JUMBO F RAMES

SUPPORT ON APPROPRIATE COMPONENTS

Jumbo frames (MTU 9000) shall be enabled end-to-end if not they will ont raise the network performance, but the
opposite will happens. By defaul the MTU is 1500. Jumbo Frames can be enabled on a vSwitch, vDS, and VMkernel
Adapter.

DETERMINE

APPROPRIATE

VLAN

CONFIGURATION FOR A V S PHERE IMPLEMENTATION

You should check further the vSphere Networking guide (p.131)

VLAN configuration in a vSphere environment provides certain benefits.

Integrates ESXi hosts into a pre-existing VLAN topology.

Isolates and secures network traffic.
Reduces congestion of network traffic

Tools

vSphere Installation and Setup Guide

vSphere Networking Guide
Leveraging NIC Technology to Improve Network Performance in VMware vSphere
vSphere Client / vSphere Web Client

51

VCP6-DCV OBJECTIVE 3.1 - M ANAGE VSPHERE STORAGE VIRTUALIZATION

VMware VCP certification exam for vSphere 6 is now available and you can register for the exam. We'll start to cover
VCP6-DCV sections to help out folks learning towards VCP6-DCV VMware certification exam. Todays topic is VCP6DCV Objective 3.1 - Manage vSphere Storage Virtualization. It's quite large chapter but it' broken into several
sections, always with screenshots. We will use vSphere Web Client only (I know not everyone's favorite, but new
features aren't exposed to the old C# client anymore...).
Due to VMware re-certification policy the VCP exam has now an expiration date. You can renew by passing delta exam
while still holding current VCP or pass VCAP. For whole exam coverage I created a dedicated VCP6-DCV page. Or if
youre not preparing to pass a VCP6-DCV, you might just want to look on some how-to, news, videos about vSphere 6
check out my vSphere 6 page.

vSphere Knowledge

Identify storage adapters and devices

Identify storage naming conventions
Identify hardware/dependent hardware/software iSCSI initiator requirements
Compare and contrast array thin provisioning and virtual disk thin provisioning
Describe zoning and LUN masking practices
Scan/Rescan storage
Configure FC/iSCSI LUNs as ESXi boot devices
Create an NFS share for use with vSphere
Enable/Configure/Disable vCenter Server storage filters
Configure/Edit hardware/dependent hardware initiators
Enable/Disable software iSCSI initiator
Configure/Edit software iSCSI initiator settings
Configure iSCSI port binding
Enable/Configure/Disable iSCSI CHAP
Determine use case for hardware/dependent hardware/software iSCSI initiator
Determine use case for and configure array thin provisioning

IDENTIFY

STORAGE ADAPTERS AND DEVICES

We will be heavily using one document - vSphere 6 Storage Guide PDF.

VMware vSphere 6 supports different classes of adapters: SCSI, iSCSI, RAID, Fibre Channel, Fibre Channel over
Ethernet (FCoE), and Ethernet. ESXi accesses adapters directly through device drivers in the VMkernel.
Note that you must enable certain adapters (like the software iSCSI), but this isn't new as it's been the case already in
previous release.
W HERE TO CHECK STORAG E ADAPTERS ?

Web Client > Hosts and clusters > host > manage > storage > storage adapters

52

You can also check storage devices there which shows basically all storage attached to the host...

IDENTIFY

STORAGE NAMING CONVENTIONS

When you select the device tab (as on the image above), you'll see that there is a storage device(s) that are accessible
to the host. Depending of the type of storage, ESXi host uses different algorithms and conventions to generate an
identifier for each storage device. There are 3 types of identifiers:

SCSI Inquire identifiers - the host query via SCSI INSUIRY command a storage device. The resulting data are
being used to generate a unique identifier in different formats (naa.number or t10.number OR eui.number).
This is because of the T10 standards.
Path-based identifiers - ex. mpx.vmhba1:C0:T1:L3 means in details - vmhbaAdapter is the name of the
storage adapter. Channel - Target - LUN. MPX path is generated in case the device does not provide a device
identifier itself. Note that the generated identifiers are not persistent across reboots and can change.
Legacy identifiers - In addition to the SCSI INQUIRY or mpx. identifiers, for each device, ESXi generates an
alternative legacy name. The identifier has the following format:

vml.number
The legacy identifier includes a series of digits that are unique to the device.
Check via CLI to see all the details:
esxcli storage core device list

53

Note that the display name can be changed - web client Select host > Manage > Storage > Storage Devices > select >
click rename icon.

54

There are also:

Fibre Channel targets which uses World Wide Names (WWN)

World Wide Port Names (WWPN)

World Wide Node Names (WWNN)

Check vSphere Storage Guide p.64 for iSCSI naming conventions

B ASICALLY SIMILAR TO THE W ORLDW IDE N AME (WWN) FOR FC DEVICES . ISCSI NAMES ARE FORMATTED IN TWO DIFFERENT
WAYS . T HE MOST COMMON IS THE IQN FORMAT .
iSCSI Qualified Name (IQN) Format

iqn.yyyy-mm.naming-authority:unique name,
where:

yyyy-mm is the year and month when the naming authority was established.
naming-authority is usually reverse syntax of the Internet domain name of the naming authority. For
example, the iscsi.vmware.com naming authority could have the iSCSI qualified name form of iqn.
1998-01.com.vmware.iscsi. The name indicates that the vmware.com domain name was registered in
January of 1998, and iscsi is a subdomain, maintained by vmware.com.
unique name is any name you want to use, for example, the name of your host. The naming authority
must make sure that any names assigned following the colon are unique, such as:
o iqn.1998-01.com.vmware.iscsi:name1
o iqn.1998-01.com.vmware.iscsi:name2
o iqn.1998-01.com.vmware.iscsi:name999

OR
ENTERPRISE U NIQUE I DENTIFIER (EUI) NAMING FORMAT

eui.16
hex
Example: eui.16hexdigits ie eui.0123456789ABCDEF
IDENTIFY

digits.

HARDWARE / DEPENDENT HARDWARE / SOFTWARE I SCSI INITIATOR REQUIREMENTS

Two types of iSCSI adapters.

Hardware based - add-On iSCSI cards (can do boot-on-lan). Those types of adapters are also capable of
offloading the iSCSI and network processing so the CPU activity is lower. Hardware adapters can be dependent
or independent. Compared to Dependent, the Indpendent adapters do not use VMkernel adapters for
connections to the storage.
Software based - activated after installation (cannot do boot-on-lan). Brings a very light overhead. Software
based iSCSI uses VMkernel adapter to connect to iSCSI storage over a storage network.

Dependent adapters can use CHAP, which is not the case of Independent adapters.

COMPARE

AND CONTRAST ARRAY THIN PROVISIONING AND VIRTUAL DISK THIN PROVISIONING

55

Virtual disk thin provisioning allows to allocate only small amount of disk space at the storage level, but the guest
OS sees as it had the whole space. The thin disk grows in size when adding more data, installing applications at the
VM level. So it's possible to over-allocate the datastore space, but it brings a risks so it's important to monitor actual
storage usage to avoid conditions when you run out of physical storage space.
Image says thousands words... p.254 of vSphere Storage Guide

Thick Lazy Zeroed - default thick format. Space is allocated at creation, but the physical device is not erased
during the creation proces, but zeroed-on-demand instead.
Thick Eager Zeroed - Used for FT protected VMs. Space is allocated at creation and zeroed immediately. The
Data remaining on the physical device is zeroed out when the virtual disk is created. Takes longer to create
Eager Zeroed Thick disks.
Thin provission - as on the image above. Starts small and at first, uses only as much datastore space as the
disk needs for its initial operations. If the thin disk needs more space later, it can grow to its maximum capacity
and occupy the entire datastore space provisioned to it. Thin disk can be inflated (thin > thick) via datastore
browser (right click vmdk > inflate).

Check the different VMDK disk provisioning options when creating new VM or adding an additional disk to existing VM

56

Thin-provissioned LUN
Array Thin Provisioning and VMFS Datastores on p. 257.
ESXi also supports thin-provisioned LUNs. When a LUN is thin-provisioned, the storage array reports the LUN's logical
size, which might be larger than the real physical capacity backing that LUN. A VMFS datastore that you deploy on the
thin-provisioned LUN can detect only the logical size of the LUN.
For example, if the array reports 2TB of storage while in reality the array provides only 1TB, the datastore considers
2TB to be the LUN's size. As the datastore grows, it cannot determine whether the actual amount of physical space is
still sufficient for its needs.
Via Storage API -Array integration (VAAI) you CAN be aware of underlying thing-provisioned LUNs. VAAI let the array
know about datastore space which has been freed when files are deleted or removed to allow the array to reclaim the
freed blocks.
Check thin provissioned devices via CLI:
esxcli storage core device list -d vml.xxxxxxxxxxxxxxxx

57

DESCRIBE

ZONING AND

LUN

MASKING PRACTICES

Zoning is used with FC SAN devices. Allow controlling the SAN topology by defining which HBAs can connect to which
targets. We say that we zone a LUN. Allows:

Protecting from access non desired devices the LUN and possibly corrupt data
Can be used for separation different environments (clusters)
Reduces number of targets and LUN presented to host
Controls and isolates paths in a fabric.

Best practice? Single-initiator-single target

58

LUN MASKING

esxcfg-scsidevs -m the -m
esxcfg-mpath -L | grep naa.5000144fd4b74168
esxcli storage core claimrule add -r 500 -t location -A vmhba35 -C 0 -T 1 -L 0 -P MASK_PATH
esxcli storage core claimrule load
esxcli storage core claiming reclaim -d naa.5000144fd4b74168
U NMASK A LUN

esxcli storage core claimrule remove -r 500

esxcli storage core claimrule load
esxcli storage core claiming unclaim -t location -A vmhba35 -C 0 -T 1 -L 0
esxcli storage core adapter rescan -A vmhba35
S CAN/R ESCAN STORAGE

Perform the manual rescan each time you make one of the following changes.

Zone a new disk array on a SAN.

Create new LUNs on a SAN.
Change the path masking on a host.
Reconnect a cable.
Change CHAP settings (iSCSI only).
Add or remove discovery or static addresses (iSCSI only).
Add a single host to the vCenter Server after you have edited or removed from the vCenter Server a datastore
shared by the vCenter Server hosts and the single host.

You can scan at the Host level or at the datacenter level (storage > select datacenter > right click > Storage > Rescan
storage.

Click host > manage > storage > storage adapters

59

Scan for New Storage Device Rescans HBAs for new storage devices
Scan for New VMFS Volumes Rescans known storage devices for VMFS volumes

CONFIGURE FC/ ISCSI LUN S AS ESX I BOOT DEVICES

Few requirements. As being said, only the hardware iSCSI can boot from LUN.
Boot from SAN is supported on FC, iSCSI, and FCoE.

1:1 ratio - Each host must have access to its own boot LUN only, not the boot LUNs of other hosts.
Bios Support - Enable the boot adapter in the host BIOS
HBA config - Enable and correctly configure the HBA, so it can access the boot LUN.

Docs:

Boot from FC SAN - vSphere Storage Guide on p. 49

Boot from iSCSI SAN - p.107.
Boot from Software FCoE - P.55

CREATE

AN

NFS

SHARE FOR USE WITH V S PHERE

An NFS client built into ESXi uses the Network File System (NFS) protocol over TCP/IP to access a designated NFS
volume that is located on a NAS server. The ESXi host can mount the volume and use it for its storage needs. vSphere
supports versions 3 and 4.1 of the NFS protocol.
How? By exporting NFS volume as NFS v3 or v4.1 (latest release). Different storage vendors have different methods of
enabling this functionality, but typically this is done on the NAS servers by using the no_root_squash option. If the
NAS server does not grant root access, you might still be able to mount the NFS datastore - but read only.
NFS uses VMkernel port so you need to configure one.
v3 and v4.1 compare:

60

E NABLE/CONFIGURE/DISABLE V CENTER S ERVER

STORAGE FILTERS

When you perform VMFS datastore management operations, vCenter Server uses default storage protection filters.
The filters help you to avoid storage corruption by retrieving only the storage devices that can be used for a particular
operation. Unsuitable devices are not displayed for selection. p. 167 of vSphere 6 storage guide.

Where?
Hosts and clusters > vCenter server > manage > settings > advanced settings

In the value box type False for appropriate key.

From the vSphere Storage Guide:

61

CONFIGURE /E DIT HARDWARE / DEPENDENT HARDWARE INITIATORS

W HERE ?

Host and Clusters > Host > Manage > Storage > Storage Adapters.
It's possible to rename the adapters from the default given name. It's possible to configure the dynamic and static
discovery for the initiators.

It's not so easy to find through Web client, as before we use to do it eyes closed through a vSphere client...

E NABLE/DISABLE

SOFTWARE I SCSI INITIATOR

CONFIGURE/E DIT

SOFTWARE I SCSI INITIATOR SETTINGS

As being said above, to configure and Edit Software iSCSI initiator settings, you can use Web client or C# client. Web
Client > Host and Clusters > Host > Manage > Storage > Storage Adapters
And there you can:

View/Attach/Detach Devices from the Host

Enable/Disable Paths
Enable/Disable the Adapter
Change iSCSI Name and Alias
Configure CHAP
62

Configure Dynamic Discovery and (or) Static Discovery

Add Network Port Bindings to the adapter
Configure iSCSI advanced options

CONFIGURE I SCSI

PORT BINDING

Port binding allows to configure multipathing when :

iSCSI ports of the array target must reside in the same broadcast domain and IP subnet as the VMkernel
adapters.
All VMkernel adapters used for iSCSI port binding must reside in the same broadcast domain and IP
subnet.
All VMkernel adapters used for iSCSI connectivity must reside in the same virtual switch.
Port binding does not support network routing.

Do not use port binding when any of the following conditions exist:

Array target iSCSI ports are in a different broadcast domain and IP subnet.
VMkernel adapters used for iSCSI connectivity exist in different broadcast domains, IP subnets, or use
different virtual switches.
Routing is required to reach the iSCSI array.

Note: The VMkernel adapters must be configured with single Active uplink. All the others as unused only (not
Active/standby). If not they are not listed...

E NABLE/CONFIGURE/DISABLE I SCSI CHAP

W HERE ?
Web Client > Host and Clusters > Host > Manage > Storage > Storage Adapters > Properties > Authentication (Edit
button).

63

p. 98 of vSphere 6 Storage Guide.

Challenge Handshake Authentication Protocol (CHAP), which verifies the legitimacy of initiators that access targets on
the network.
Unidirectional CHAP - target authenticates the initiator, but the initiator does not authenticate the target.
Bidirectional CHAP - an additional level of security enables the initiator to authenticate the target. VMware supports
this method for software and dependent hardware iSCSI adapters only.
CHAP METHODS :

None - CHAP authentication is not used.

Use unidirectional CHAP if required by target - Host prefers non-CHAP connection but can use CHAP if
required by target.
Use unidirectional CHAP unless prohibited by target - Host prefers CHAP, but can use non-CHAP if target does
not support CHAP.
Use unidirectional CHAP - Requires CHAP authentication.
Use bidirectional CHAP - Host and target support bidirectional CHAP.

CHAP does not encrypt, only authenticates the initiator and target.
Determine use case for hardware/dependent hardware/software iSCSI initiator
It's fairly simple, as we know that if we use the software iSCSI adapter we do not have to buy additional hardware and
we're still able to "hook" into iSCSI SAN.
The case for Dependent Hardware iSCSI Adapter which is dependant on the VMKernel adapter but offloads iSCSI
processing to the adapter, which accelerates the treatment and reduces CPU overhead.
On the other hand, the Independent Hardware iSCSI Adapter has its own networking, iSCSI configuration, and
management interfaces. So you must go through the BIOS and the device configuration in order to use it.

DETERMINE

USE CASE FOR AND CONFIGURE ARRAY THIN PROVISIONING

64

Some arrays do support thin provissioned LUNs while others do not. The benefit is to offer more capacity (visible) to
the ESXi host while consuming only what's needed at the datastore level. (attention however for over-subscribing, so
proper monitoring is needed). So at the datastore level it's possible to use thin provisioned virtual disk or on the array
using thin provisioned LUNs.

Tools

vSphere Installation and Setup Guide

vSphere Storage Guide
Best Practices for Running VMware vSphere on iSCSI
vSphere Client / vSphere Web Client

VCP6-DCV OBJECTIVE 3.2 - CONFIGURE SOFTWARE -DEFINED STORAGE

VCP6-DCV (datacenter virtualization) VMware certification exam was recently released and the registration will be
soon available. The term software-defined, you can love or hate this term, but Software-defined storage is here, and
this post covers VCP6-DCV Objective 3.2 - Configure Software-defined Storage. Hopefully it will help you to learn this
topic towards the exam...
For whole exam coverage I created a dedicated VCP6-DCV page. Or if you're not preparing to pass a VCP6-DCV, you
might just want to look on some how-to, news, videos about vSphere 6 - check out my vSphere 6 page.
If you find out that I missed something, don't hesitate to comment.

vSphere Knowledge Covered in this post:

Configure/Manage VMware Virtual SAN

Create/Modify VMware Virtual Volumes (VVOLs)
Configure Storage Policies
Enable/Disable Virtual SAN Fault Domains

CONFIGURE/M ANAGE VM WARE V IRTUAL SAN

VMware VSAN (traditional) needs some spinning media (SAS or SATA) and 1 SSD per host (SATA, SAS or PCIe).
VMware VSAN (All-Flash) needs some SATA/SAS for capacity tier and 1 SSD hight performance and endurance
for caching.
HBA which is on the VMware HCL (queue depth > 600)
All hardware must be part of HCL (or if you want easy way -> via VSAN ready nodes!)
HBA with RAID0 jor direct pass-through so ESXi can see the individual disks, not a raid volume.
SSD sizing - 10% of consumed capacity
1Gb Network (10GbE recommended)
1 VMkernel unterface configured (dedicated) for VSAN traffic
Multicast activated on the switch
IGMP Snooping and an IGMP Querier can be used to filter multicast traffic to a limited to specific port group.
Usefull if other non-Virtual SAN network devices exist on the same layer 2 network segment (VLAN).
IPv4 only on the switch
Minimum 3 hosts in the cluster (4 recommended) - maxi. 64 hosts (vSphere 6)

CREATE VM KERNEL INTERFACE WITH VSAN TRAFFIC ON

65

Host > Manage > Networking > VMkernel Adapters > Add

ENABLE VSAN AT THE CLUSTER LEVEL

Hosts and Clusters > Cluster > Manage > Settings > Virtual SAN > General

Add disk to storage:

Manual Requires manual claiming of any new disks.

Automatic All empty disks on cluster hosts will be automatically claimed by VSAN

CREATE DISK GROUPS

Hosts and Clusters > Cluster > Manage > Settings > Virtual SAN > Disk Management

66

CLAIM DISKS FOR VSAN

You can do several tasks when managing disk in VSAN cluster.

Claim Disks for VSAN

Create a new disk group (when adding more capacity).
Remove the disk group
Add a disk to the selected disk group
Place a host in maintenance mode

S O H OW TO M ARK LOCAL DISK AS SSD DISK ?

Connect to your vCenter > Go to Hosts and clusters > Select a Host > Select disk which you want to tag as SSD. You
can click to enlarge.

This brings a small warning window saying that you might deteriorate the performance of datastores and services that
use them, but if youre sure on what youre doing, then go ahead and validate on Yes button.

As a result, after few seconds (without even refreshing the clients page) the disk turns into a SSD disk Its magic, no?

It works also the other way around! SSD to HDD. Note that this works only in VSAN 6.0!
TAG D ISKS FOR CAPACITY OR CACHING
67

So lets demonstrate it in my lab. I use VMware Workstation for the job where I quickly created few ESXi VMs. I
configured the ESXi 6 host with 7 hard drives, where each virtual disk is destined to fill different function. Here are the
details:

40Gb is local disk where is installed ESXi

20 Gb drives are the ones which I need to tag as capacity
5 Gb drive is the caching tier

The view or our disks

To check the status of your disks as ESXi sees them you can use the vdq -q command
So in our case:

vdq -q
gives us this:

We can see that the mpx.vmhba1:C0:T6:L0 is our disk which we need to tag to be able to use is in our disk group.
(otherwise the disk wont appear to be used in VSAN as capacity tier).
We need to connect via SSH to our host. If you havent enabled yet, please enable SSH by going and selecting your
host > Manage > Security Profile > services > Edit
After you have identified the disk which you need to tag, just enter this command:
esxcli vsan storage tag add -d naa.XYZ -t capacityFlash

68

where naa.XYZ is your hard drive. In my example

esxcli vsan storage tag add -d mpx.vmhba1:C0:T5:L0 -t capacityFlash

After tagging all of the 20Gb disks we can create a disk group where those disks will appear as data disks below (You
can see that our mpx.vmhba1:C0:T6:L0 device can now be selected to be used data disk)

Note: You can not only tag but also untag!

Check this:
esxcli vsan storage tag remove -d naa.XYZ -t capacityFlash
the above command will simply remove the capacityFlash tag from the storage device.
How to check if SSD is participating as capacity tier or not?
So if you just want to check which tag does your storage has you can use this command:
vdq -q
See the output here

69

You should get this VSAN Troubleshooting Reference Manual which is great resource
VSAN AND M AINTENANCE M ODE
Maintenance mode for each ESXi participating in VSAN cluster has new options depending what you want to do with
the data located on the particular host (the object's locations are on the local storage of each host) So, Virtual SAN
host's when you want to put them in maintenance mode thay allows 3 options:

Ensure accessibility - Virtual SAN ensures that all virtual machines on this host will remain accessible if the
host is shut down or removed from the cluster.
Full data migration - Virtual SAN migrates all data that resides on this host.
No data migration - Virtual SAN will not migrate any data from this host. Some virtual machines might become
inaccessible if the host is shut down or removed from the cluster.

CREATE/M ODIFY VM WARE V IRTUAL V OLUMES (VVOLS)

70

VVOls are new in vSphere 6. By using a special set of APIs called vSphere APIs for Storage Awareness
(VASA), the storage system becomes aware of the virtual volumes and their associations with the relevant
virtual machines. Through VASA, vSphere and the underlying storage system establish a two-way out-ofband communication to perform data services and offload certain virtual machine operations to the storage
system. For example, such operations as snapshots, storage DRS and clones can be offloaded.

VVOLs are supported on SANs compatible with VAAI (vSphere APIs for Array Integration).
VVOLs supports vMotion, sVMotion, Snapshots, Linked-clones, vFRC, DRS
VVOLs supports backup products which uses VADP (vSphere APIs for Data Protection)
VVOLs supports FC, FCoE, iSCSI and NFS

Image courtesy VMware

71

VVOL S L IMITATIONS

VVOLs Does not works with standalone ESXi hosts (needs vCenter)
VVOLs do not support RDMs
VVOLs wih the virtual datastores are tighten to vCenter sor if used with Host profiles, than only within this
particular vCenter as the extracted host profile can be attached only to the hosts withing the same vCenter as
the reference host is located.
No IPv6 support
NFS v3 only (v4.1 isn't supported)
Multipathing only on SCSI-based endpoints, not on NFS-based protocol endpoint.

VVOLs vSphere Storage Guide p211.

Virtual volumes are encapsulations of virtual machine files, virtual disks, and their derivatives. Virtual volumes are not
preprovisioned, but created automatically when you perform virtual machine management operations. These
operations include a VM creation, cloning, and snapshotting. ESXi and vCenter Server associate one or more virtual
volumes to a virtual machine.

Storage Provider - A Virtual Volumes storage provider, also called a VASA provider, is a software component
that acts as dastorage awareness service for vSphere.

Storage Container - A storage container is a part of the logical storage fabric and is a logical unit of the
underlying hardware. The storage container logically groups virtual volumes based on management and
administrative needs.
Protocol Endpoints -ESXi hosts use a logical I/O proxy, called the protocol endpoint, to communicate with
virtual volumes and virtual disk files that virtual volumes encapsulate. ESXi uses protocol endpoints to establish
a data path on demand from virtual machines to their respective virtual volumes.
Virtual Datastores - A virtual datastore represents a storage container in vCenter Server and the vSphere Web
Client.

Steps to Enable VVOLs (p.218):

Step 1: Register Storage Providers for VVOLs

vCenter Inventory Lists > vCenter Servers > vCenter Server > Manage > Storage Providers

Step 2: Create a Virtual Datastore

72

vCenter Inventory Lists > Datastores

Step 3: Review and manage protocol endpoints

vCenter Inventory Lists > Hosts > Host > Manage > Storage > Protocol Endpoints

(optional) Change the path selection policy (psp) for protocol endpoint.

Manage > Storage > Protocol Endpoints > select the protocol endpoint you want to change and click Properties >
Under multipathing Policies click Edit Multipathing

CONFIGURE S TORAGE P OLICIES (VM

STORAGE POLICIES )

Virtual Machine Storage policies are covered vSphere Storage Guide on p. 225. Virtual machine storage policies are
essential to virtual machine provisioning. These policies help youdefine storage requirements for the virtual machine

73

and control which type of storage is provided for the virtual machine, how the virtual machine is placed within the
storage, and which data services are offered for the virtual machine. SP contains storage rule or collection of storage
rules.
define a storage policy, you specify storage requirements for applications that run on virtual machines. After you apply
this storage policy to a virtual machine, the virtual machine is placed in a specific datastore that can satisfy the storage
requirements.
In case of VSAN and VVOLs, the SP determines how the VM storage objects are handled and allocated within the
datastore to guarantee the SLA.

Rules based on storage-specific data service - VSAN and VVOLs uses VASA to surface the storage capability to
VMstorage policies's interface
Rules based on TAGs - by tagging a specific datastore. More than One tag can be applied per datastore

V IEW VM S AND DISKS IF THEY COMPLY WITH VM STORAGE POLICIES

VM Storage Policies > Click a particular Storage Policy > Monitor

74

E NABLE/DISABLE V IRTUAL SAN F AULT DOMAINS

VSAN fault domains allows to create an environment where the in case of failure 2 hosts for example, which are in the
same rack. Failure of all hosts within a single fault domain is treated as one failure. VSAN will not store more than
one replica in this group (domain).
VSAN Storage Guide p.22
Requirements: 2*n+1 fault domains in a cluster. In order to leverage fault domain you need at least 6 hosts (3 fault
domains). Using a three domains does not allow the use of certain evacuation modes, nor is Virtual SAN able to
reprotect data after a failure.
VMware recommends 4 Fault domains. (the same for vSAN clusters - 4 hosts in a VSAN cluster).
On the pic below you see my hosts are down, but VSAN still works and provide storage for my VM... (nested
environment).
Hosts and Clusters > Cluster > Manage > Settings > Virtual SAN > Fault Domains

If a host is not a member of a fault domain, Virtual SAN interprets it as a separate domain.

Tools

Administering VMware Virtual SAN

vSphere Storage Guide
What's New: VMware Virtual SAN 6.0
Whats New in the VMware vSphere 6.0 Platform
Virtual SAN 6.0 Performance: Scalability and Best Practices
vSphere Client / vSphere Web Client

vSphere how-to, news, videos on my Dedicated vSphere 6 page!

75

VCP6-DCV OBJECTIVE 3.3 - CONFIGURE VSPHERE STORAGE MULTI-PATHING AND

FAILOVER
Todays VCP6-DCV goal is to talk about VCP6-DCV Objective 3.3 - Configure vSphere Storage Multi-pathing and
Failover. VMware VCP exam is a gold standard of VMware certification exams. VMware vSphere 6 brings new
certification exam.
VCP exam is the most known VMware exams, even if its not the highest technical level. But its most recognized. By a
future employer, by industry as a whole. We will cover VCP6-DCV exam certification based on VMware latest VMware
VCP6-DCV blueprint. Check VCP6-DCV page for all objectives.

vSphere knowledge

Configure/Manage Storage Load Balancing

Identify available Storage Load Balancing options
Identify available Storage Multi-pathing Policies
Identify features of Pluggable Storage Architecture (PSA)
Configure Storage Policies
Enable/Disable Virtual SAN Fault Domains

CONFIGURE/M ANAGE S TORAGE LOAD B ALANCING

The goal of load balancing policy is to give equal "chance" to each storage processors and the host server paths by
distributing the IO requests equally. Using the load balancing methods allows to optimize Response time, IOPs or MBPs
for VMs performance.
To get started, if you're using block storage - check the Storage > Datastore > Manage > Settings > Connectivity and
Multipathing

76

IDENTIFY

AVAILABLE

S TORAGE LOAD B ALANCING

OPTIONS

You can manage multipathing using the vSphere Client, the esxcli command, or using the following commands. Use
the HostStorageSystem.multipathStateInfo property to access the HostMultipathStateInfo.
SAN storage systems require continual redesign and tuning to ensure that I/O is load balanced across all storage system
paths. To meet this requirement, distribute the paths to the LUNs among all the SPs toprovide optimal load balancing.
Multipathing allows you to have more than one physical path from the ESXi host to a LUN on a storage system.
Generally, a single path from a host to a LUN consists of an iSCSI adapter or NIC, switch ports, connecting cables, and
the storage controller port. If any component of the path fails, the host selects another available path for I/O. The
process of detecting a failed path and switching to another is called path failover.
Path information:

Active - Paths available for issuing I/O to a LUN. A single or multiple working paths currently used for
transferring data are marked as Active (I/O).
Standby - If active paths fail, the path can quickly become operational and can be used for I/O
Disabled - path disabled, no transfer possible.
Dead - impossible to connect to the disk via this path.

IDENTIFY

AVAILABLE

S TORAGE M ULTI - PATHING P OLICIES

You can select different path selection policy from the default ones, or if you have installed a third party product which
has added its own PSP:
Fixed - (VMW_PSP_FIXED) the host uses designated preferred path if configured. If not it uses first working path
discovered. Prefered path needs to be configured manually.

77

Most Recently Used - (VMW_PSP_MRU) The host selects the path that it used most recently. When the path
becomes unavailable, the host selects an alternative path. The host does not revert back to the original path
when that path becomes available again. There is no preferred path setting with the MRU policy. MRU is the
default policy for most active-passive arrays.

Round Robin (RR) - VMW_PSP_RR - The host uses an automatic path selection algorithm rotating through all
active
paths
when
connecting
to
active-passive
arrays,
or
through
all
available paths when connecting to active-active arrays. RR is the default for a number of arrays and can be
used with both active-active and active-passive arrays to implement load balancing across paths for different
LUNs.

78

IDENTIFY

FEATURES OF

P LUGGABLE S TORAGE A RCHITECTURE (PSA)

VMware NMP - default multipathing module (Native Multipathing Plugin). Nmp plays a role when associating
the set of physical paths with particular storage device or LUN, but delegates the details to SATP plugin. On
the other hand the choice of path used when IO comes is is handled by PSP (Path Selection Plugin)
VMware SATP - Storage Array Type Plugins runs hand in hand with NMP and are responsible for array based
operations. ESXi has SATP for every supported SAN, It also provides default SATPs that support non-specific
active-active and ALUA storage arrays, and the local SATP for direct-attached devices.
VMware PSPs - Path Selection Plugins are sub plugins of VMware NMP and they choose a physical path for IO
requests.

The multipathing modules perform the following operations:

Manage physical path claiming and unclaiming.

Manage creation, registration, and deregistration of logical devices.
Associate physical paths with logical devices.
Support path failure detection and remediation.
Process I/O requests to logical devices:
o Select an optimal physical path for the request.
o Depending on a storage device, perform specific actions necessary to handle path failures and I/O
command retries.
Support management tasks, such as reset of logical devices.

CONFIGURE S TORAGE P OLICIES

A storage policy can include multiple rule sets. Storage-Specific Data Service rules and Tag based rules can be combined
in the same storage policy. VM Storage Policies, where?
Home > VM Storage Policies
Guide: vSphere Storage Guide on p. 225
Storage rules based on:
Rules based on storage-specific data service VSAN and VVOLs uses VASA to surface the storage capability to
VMstorage policiess interface. To supply information about underlying storage to vCenter Server, Virtual SAN
and Virtual Volumes use storage providers, also called VASA providers. Storage information and datastore
characteristics appear in the VM Storage Policies interface of the vSphere Web Client as data services offered by
the specific datastore type.

79

Rules based on TAGs by tagging a specific datastore. More than One tag can be applied per datastore.

First you must tag a datastore

Then you go back to a VM storage policy > Add new policy icon > put some meaningful name > click Add tag-based
rule > choose your rule from the category drop down menu > click Next > choose a compatible datastore

80

Check compliance via VM storage Policies > Storage policy > monitor

If you want to change from default storage policy to newly created one, you must first change it at the VM level and
then check back at VM storage Policies > Storage policy > monitor

81

E NABLE/DISABLE V IRTUAL SAN F AULT DOMAINS

VMware fault domains in VSAN environment allows to spread the replicas over different locations (different racks) in
order to "not to put all eggs in the same basket" - literarly. Let's say you have 4 hosts per rack and you want to achieve
a redundancy in case of failure multiple components within single rack. VSAN considers each fault domain as single
host.
Virtual SAN Fault Domains ensures replicas of VM data is spread across the defined failure domains. Fault domains
provide the ability to tolerate:

Rack failures
Storage controller
Network failures
Power failure

Image courtesy of VMware

Where to manage VSAN fault domains?

Hosts and Clusters > Cluster > Manage > Settings > Virtual SAN > Fault Domains

82

If a host is not a member of a fault domain, Virtual SAN interprets it as a separate domain.
VMware recommends to configure minimum 3 or more fault domains in the VSAN cluster, and also you should assing
the same number of hosts per fault domain. It's not necessary however assign all hosts to fault domains.
Note: If a host is moved to another cluster, VSAN hosts retain their fault domain assignements.
Tools:

vSphere Installation and Setup Guide

vSphere Storage Guide
Multipathing Configuration for Software iSCSI Using Port Binding
vSphere Client / vSphere Web Client

VCP6-DCV OBJECTIVE 3.4 - PERFORM ADVANCED VMFS AND NFS CONFIGURATIONS AND
UPGRADES
This post covers VCP6-DCV Objective 3.4 - Perform Advanced VMFS and NFS Configurations and Upgrades. Important
storage chapter where you'll learn the inside out about VMFS, datastores, management or enable/disable vStorage
API for array integration.
For whole exam coverage I created a dedicated VCP6-DCV page which follows the exam's blueprint. If you just want
to look on some how-to, news, videos about vSphere 6 check out my vSphere 6 page. If you find out that I missed
something in this post, dont hesitate to comment.

83

VMware vSphere Knowledge

Identify VMFS and NFS Datastore properties

Identify VMFS5 capabilities
Create/Rename/Delete/Unmount a VMFS Datastore
Mount/Unmount an NFS Datastore
Extend/Expand VMFS Datastores
Place a VMFS Datastore in Maintenance Mode
Identify available Raw Device Mapping (RDM) solutions
Select the Preferred Path for a VMFS Datastore
Enable/Disable vStorage API for Array Integration (VAAI)
Disable a path to a VMFS Datastore
Determine use case for multiple VMFS/NFS Datastores

IDENTIFY VMFS

AND

NFS DATASTORE

PROPERTIES

What's Datastore? - it's kind of a logical container which stores VMDKs of your VMs. VMFS is a clustered file system
which allows multiple hosts access files on shared datastore.
VMFS uses locking mechanism (ATS or ATS + SCSI) which prevents multiple hosts from concurrently writing to the
metadata and ensure that there is no data corruption. Check Page 149 for vSphere Storage guide for more on the ATS
or ATS+SCSI locking mechanism.
NFS - Network file system, can be mounted by ESXi host (which uses NFS client). NFS datastores supports vMotion or
SvMotion, HA, DRS, FT or host profiles (note that NFS 4.1 do not supports FT). NFS v3 and NFS v4.1 are supported with
vSphere 6.0.
VMDKs are provisionned as "Thin" by default on the NFS datastore.

IDENTIFY VMFS5

CAPABILITIES

Larger than 2TB storage devices for each VMFS5 extent.

Support of virtual machines with large capacity virtual disks, or disks greater than 2TB.
Increased resource limits such as file descriptors.
Standard 1MB file system block size with support of 2TB virtual disks.
Greater than 2TB disk size for RDMs
Support of small files of 1KB.Ability to open any file located on a VMFS5 datastore in a shared mode by a
maximum of 64 hosts.
Can reclaim physical storage space on thin provisioned storage devices.

Upgrades from previous version of VMFS:

VMFS datastores can be upgraded without disrupting hosts or virtual machines.

If creating new VMFS datastore there is choice to create VMFS 3 or VMFS 5 version of datastore
New VMFS datastores are created with the GPT format.
VMFS datastore which has been upgraded will continue to use the MBR format until it is expanded beyond
2TB. If that's the case then the MGS format is converted to GPT.
Maximum VMFS datastores per host - 256 VMFS datastores
Host needs to run ESXi 5.0 or higher
No way back (VMFS 5 to VMFS 3) the upgrade process cannot downgrade back to VMFS v3.

84

CREATE/R ENAME/D ELETE/UNMOUNT

VMFS DATASTORE

Create Datastore - vSphere Web Client > Hosts and Clusters > Select Host > Actions > Storage > New Datastore

And you have a nice assistant which you follow...

The datastore can be created also via vSphere C# client.

To rename datastore > Home > Storage > Right click datastore > Rename

85

As you can see you can also unmount or delete datastore via the same right click.

Make sure that:

There are NO VMs on that datastore you want to unmount.

If HA configured, make sure that the datastore is not used for HA heartbeats
Check that the datastore is not managed by Storage DRS
Verify also that Storage IO control (SIOC) is disabled on the datastore

M OUNT /UNMOUNT

AN

NFS DATASTORE

Create NFS mount. Similar way as above Right click datacenter > Storage > Add Storage.

86

You can use NFS 3 or NFS 4.1 (note the limitations of NFS 4.1 for FT or SIOC). Enter the Name, Folder, and Server (IP or
FQDN)
To Mount/unmout NFS datastore...

And then choose the host(s) to which you want this datastore to mount...

E XTEND /E XPAND VMFS DATASTORES

It's possible to expand existing datastore by using extent OR by growing an expandable datastore to fill the available
capacity.

87

and then you just select the device..

You can also Add a new extent. Which means that datastore can span over up to 32 extents and appear as a single
volume.... But in reality, not many VMware admins likes to use extents....

P LACE

VMFS DATASTORE

IN

M AINTENANCE M ODE

Maintenance mode for datastore is available if the datastore takes part in Storage DRS cluster. (SDRS). Regular
datastore cannot be placed in maintenance mode. So if you want to activate SDRS you must first create SDRS cluster
by Right click Datacenter > Storage > New Datastore Cluster.
then only you can put the datastore in maintenance mode...

88

IDENTIFY

AVAILABLE

R AW DEVICE M APPING (RDM)

SOLUTIONS

vSphere storage guide p. 203. RDM allows a VM directly access a LUN. Think of an RDM as a symbolic link from a VMFS
volume to a raw LUN.

An RDM is a mapping file in a separate VMFS volume that acts as a proxy for a raw physical storage device. The RDM
allows a virtual machine to directly access and use the storage device. The RDM contains metadata for managing and
redirecting disk access to the physical device.
When to use RDM?

When SAN snapshot or other layered applications run in the virtual machine. The RDM better enables scalable
backup offloading systems by using features inherent to the SAN.
In any MSCS clustering scenario that spans physical hosts virtual-to-virtual clusters as well as physical-tovirtual clusters. In this case, cluster data and quorum disks should be configured as RDMs rather than as virtual
disks on a shared VMFS.

If RDM is used in physical compatibility mode - no snapshoting of VMs... Virtual machine snapshots are available for
RDMs with virtual compatibility mode.
Physical Compatibility Mode - VMkernel passes all SCSI commands to the device, with one exception: the REPORT
LUNs command is virtualized so that the VMkernel can isolate the LUN to the owning virtual machine. If not, all physical
characteristics of the underlying hardware are exposed. It does allows the guest operating system to access the
hardware directly. VM with physical compatibility RDM has limits like that you cannot clone such a VM or turn it into
a template. Also sVMotion or cold migration is not possible.
Virtual Compatibility Mode - VMkernel sends only READ and WRITE to the mapped device. The mapped device
appears to the guest operating system exactly the same as a virtual disk file in a VMFS volume. The real hardware
characteristics are hidden. If you are using a raw disk in virtual mode, you can realize the benefits of VMFS such as
advanced file locking for data protection and snapshots for streamlining development processes. Virtual mode is also
more portable across storage hardware than physical mode, presenting the same behavior as a virtual disk file.
(VMDK). You can use snapshots, clones, templates When an RDM disk in virtual compatibility mode is cloned or a
template is created out of it, the contents of the LUN are copied into a .vmdk virtual disk file.
Other limitations:

89

You cannot map to a disk partition. RDMs require the mapped device to be a whole LUN.
VFRC - Flash Read Cache does not support RDMs in physical compatibility (virtual compatibility is compatible).
If you use vMotion to migrate virtual machines with RDMs, make sure to maintain consistent LUN IDs for RDMs
across all participating ESXi hosts

S ELECT

THE

P REFERRED P ATH

FOR A

VMFS DATASTORE

For each storage device, the ESXi host sets the path selection policy based on the claim rules. The different path policies
we treated in our earlier chapter here - Configure vSphere Storage Multi-pathing and Failover.
Now if you want just to select preferred path, you can do so. Ifyou want the host to use a particular preferred path,
specify it manually.
Fixed is the default policy for most active-active storage devices
Fixed (VMW_PSP_FIXED) the host uses designated preferred path if configured. If not it uses first working path
discovered. Preffered path needs to be configured manually.

E NABLE/DISABLE V S TORAGE API

FOR

A RRAY INTEGRATION (VAAI)

You need to have hardware that supports the offloading storage operations like:

Cloning VMs
Storage vMotion migrations
Deploying VMs from templates
VMFS locking and metadata operations
Provisioning thick disks
Enabling FT protected VMs

HOW TO DISABLE ? OR ENABLE ?

Enable = 1
Disable = 0
vSphere Web Client > Manage tab > Settings > System, click Advanced System Settings > Change the value for any of
the options to 0 (disabled):

VMFS3.HardwareAcceleratedLocking
DataMover.HardwareAcceleratedMove
DataMover.HardwareAcceleratedInit
90

you can check the status of the hardware via CLI (via esxcli storage core device vaai status get)

or on the NAS devices with (esxcli storage nfs list).

Via vSphere web client you can also see if a datastore has hardware acceleration support...

91

DISABLE

A PATH TO A

VMFS DATASTORE

It's possible to temporarily disable storage path, for example for maintenance reasons. Check Storage Paths in the
vSphere Storage Guide on p 192.

One can disable the path from through the web client from the datastore view OR storage device OR adapter view.

DETERMINE

USE CASE FOR MULTIPLE

VMFS/NFS DATASTORES

Usually the choice for multiple VMFS/NFS datastores are based on performance, capacity and data protection.
Separate spindles having different RAID groups to help provide better performance. Than you can have multiple
VMs, executing applications which are I/O intensive. If you make a choice with single big datastore, than you might
have performance issues...
Separate RAID groups. for certain applications, such as SQL server you may want to configure a different RAID
configuration of the disks that the logs sit on and that the actual databases sit on.
Redundancy You might want to replicate VMs to another host/cluster. You may want the replicated VMs to be
stored on different disks than the production VMs. In case you have failure on production disk system, you most
likely still be running the secondary disk system just fine.
Load balancing - you can balance performance/capacity across multiple datastores.
Tiered Storage Arrays comes often with Tier 1, Tier 2, Tier 3 and so you can place your VMs according to
performance levels...
Tools

vSphere Installation and Setup Guide

vSphere Storage Guide

VMware vSphere Storage APIs Array Integration (VAAI)
92

VCP6-DCV OBJECTIVE 3.5 - SETUP AND CONFIGURE STORAGE I/O CONTROL

This post will cover VCP6-DCV Objective 3.5 - Setup and Configure Storage I/O Control. Storage I/O is one of the
features that are overlooked. But Storage I/O can "heal" part of your storage performance problems by setting a
priority at the VM level (VMDK). You know the "noisy neighbor story"....
When you enable Storage I/O Control on a datastore, ESXi host starts to monitor the device latency that hosts observe
when communicating with that datastore. When device latency exceeds a threshold, the datastore is considered to be
congested and each VM that accesses that datastore is allocated I/O resources in proportion to their shares. (by
default all VMs are set to Normal (1000) You set shares per VMDK. You can adjust the number for each based on need.
Default is 1000.
I started to cover this VCP6-DCV exam blueprint since few weeks and It seems that for VCP6 there is more material to
study and more topics to master than for previous version of VCP as the technology has evolving with each release of
vSphere. But this never mind, we like technology, we like virtualization and we like VMware. Let's kick some tires.. -:)
For whole exam coverage I created a dedicated VCP6-DCV page.
VMware vSphere Knowledge

Enable/Disable Storage I/O Control

Configure/Manage Storage I/O Control
Monitor Storage I/O Control

E NABLE/DISABLE S TORAGE I/O CONTROL

Before we jump in I'd like to explain how storage I/O control helps to prioritize certain VMs over the others. I think it's
best to check out this image from VMware which shows basically that after activating the SIOC on shared datastore
and setting up the shares (at the VMDK level) on the VMs properties, the SIOC is able to prioritize those VMs over the
others....

Quote from VMware:

93

Storage I/O Control operates as a datastore-wide disk scheduler. Once Storage I/O Control has been enabled for a
specific datastore, it will monitor that datastore, summing up the disk shares for each of the VMDK files on it. Storage
I/O Control will then calculate the I/O slot entitlement per ESXi host based on the percentage of shares virtual machines
running on that host have relative to the total shares for all hosts accessing that datastore.

Few limitations and requirements:

NFS v4.1 isn't supported (it is for NFS v3).

Storage I/O Control does not support datastores with multiple extents.
SAN with auto-tiering has to be certified for SIOC.
Datastores that are Storage I/O Control-enabled must be managed by a single vCenter Server system.
Must be disabled before removing a datastore.
Raw Device Mapping (RDM) is not supported. (it is on iSCSI NFS and FC).

Storage I/O Requirements at the Online vSphere 6 documentation center. (here)

Activate at the datastore level via vSphere client or vSphere Web client.
Configure/Manage Storage I/O Control
Configuring Storage I/O Control is a two-step process
1. Enable Storage I/O Control for the datastore
In the vSphere Client > select a datastore > Configuration tab > Properties > Storage I/O Control, select the Enabled
check box.

The advanced settings - Threshold - default value there. Check if the value is 30ms.
2. Set the number of storage I/O shares and upper limit of I/O operations per second (IOPS) allowed for each virtual
machine. Those settings at the VMDK level so you could possibly prioritize disk where you important production DB
sits!
Set the threshold. More the VM is important, greater the number...... You can use the drop down or the custom and
enter your value...

94

In case you're getting error on activating SIOC this can be due 2 reasons:

Not having proper licensing - Enterprise Plus is required. Storage I/O Control (SIOC) requires Enterprise Plus
licensing. Without this license, the option to enable SIOC is grayed out
Check that the host is installed with ESXi 4.1 or higher.

M ONITOR S TORAGE I/O CONTROL

There is a Performance TAB to monitor Storage I/O. How Storage I/O Control handles the I/O workloads of the virtual
machines accessing a datastore based on their shares.
Datastore performance charts allow monitoring:

Average latency and aggregated IOPS on the datastore.

Latency among hosts n Queue depth among hosts.
Read/write IOPS among hosts.
Read/write latency among virtual machine disks n Read/write IOPS among virtual machine disks.

95

W HERE ?

vSphere Web client > Datastore > Monitor tab > Performance tab > View drop-down menu > select
Performance.

Tools

Administering VMware Virtual SAN

vSphere Storage Guide
vSphere Resource Management Guide
vSphere Client / vSphere Web Client

Links: VCP6-DCV page.

VCP6-DCV OBJECTIVE 4.1 - PERFORM ESXI HOST AND VIRTUAL M ACHINE UPGRADES
We will In no particular order start to cover VCP6-DCV sections to help out folks learning towards VCP6-DCV VMware
certification exam. Due to VMware recertification policy the VCP exam has now an expiration date. You can renew by
passing delta exam while still holding current VCP or pass VCAP. Today's topic of VCP6-DCV Objective 4.1 - Perform
ESXi Host and Virtual Machine Upgrades.

96

For whole exam coverage I created a dedicated VCP6-DCV page. Or if you're not preparing to pass a VCP6-DCV, you
might just want to look on some how-to, news, videos about vSphere 6 - check out my vSphere 6 page.
We'll cover the topic today present on the VMware VCP6-DCV blueprint:

Identify upgrade requirements for ESXi hosts

Upgrade a vSphere Distributed Switch
Upgrade VMware Tools
Upgrade Virtual Machine hardware
Upgrade an ESXi Host using vCenter Update Manager
Stage multiple ESXi Host upgrades
Determine whether an in-place upgrade is appropriate in a given upgrade scenario

IDENTIFY

UPGRADE REQUIREMENTS FOR

ESX I

HOSTS

Even if this post we will now talk about the host requirement, the vCenter server shall be upgraded first.... Then you
should definitely check the vSphere 6 Upgrade PDF from VMware which has all the details.
ESXi 6 support booting via UEFI or BIOS, but if you plan to use autodeploy, then you might privilege BiOS as UEFI isn't
supported. Changing from BIOS to UEFI after install isn't supported.

CPU with 2 cores at least

VMware Supported Hardware via VMware HCL http://www.vmware.com/go/hcl
NX/XD bit enabled in the BIOS
To support 64-bit virtual machines, support for hardware virtualization (Intel VT-x or AMD RVI) must be
enabled on x64 CPUs. Note: for very old CPUs to check if they support x64 see this post - VMware Guest 64
Check Free Utility To check if CPU can run 64 Bit Workflows
Minimum of 4 GB of physical RAM, (if planning VSAN then at least 6Gb of RAM is required) with 8 GB of physical
RAM recommended.
For Serial ATA (SATA), a disk connected through supported SAS controllers or supported on-boardSATA
controllers.
SCSI disk or a local, non-network, RAID LUN with unpartitioned space for the virtual machines.
At least 1 GbE Nic
Minimum 1 GB boot device. But even if 1GB USB or SD device suffices for a minimal installation, you should
use a 4GB or larger device where the extra space will be used for an expanded coredump partition on the
USB/SD device.

UPGRADE

A V S PHERE

DISTRIBUTED S WITCH

The upgrade from 5.x to 6.0 is not reversible. There are two requirements:
1. You have upgraded your vCenter to vCenter 6.0
2. You have upgraded your hosts to ESXi 6.0 (check different methods of upgrading ESXi - via CLI or ISO, VUM, Online
VMware repository)
W HERE ?
vSphere Web client > Networking > Right-click the distributed switch and select > Upgrade > Upgrade Distributed
Switch
It's non disruptive operation, so no downtime.

97

Check the the vSphere Networking Guide (page 28) for more.

UPGRADE VM WARE TOOLS

VMware tools shall always have the latest version, but they depends on which vSphere Hardware Version (VHV) you
run your VMs. You can very well have the virtual machine compatibility set for version 5.0 (vmx-09) because of some
reasons and not the vmx-11 (vSphere 6). But the VMware tools will run being updated to the latest version for this VM
compatibility.

vSphere Virtual Machine Administration Guide on page 22

UPGRADE V IRTUAL M ACHINE

HARDWARE

98

Where?
Edit VM's Settings via vSphere web client.

Note that once you upgrade the Virtual machine hardware, there is no easy way back. There is three ways to
downgrade virtual machine hardware version (supported by VMware).
Upgrade an ESXi Host using vCenter Update Manager
Note that only hosts running ESXi 5.0, ESXi 5.1, or ESXi 5.5 are directly upgradable to the ESXi 6.0. If you're still on 4.1
then you must first upgrade to 5.0. vCenter server 6 and vSphere Update Manager 6 (VUM) must be used for the
upgrade. Details - vSphere Upgrade Guide (p. 135).
1. If you didnt downloaded the ESXi 6.0 installation ISO, youll need to do so. Download Link.
2. Youll need to install/configure VMware Update Manager follow this guide.
3. Connect via vSphere client > select your host (or cluster) and go to the Update Manager TAB > Admin View > ESXi
Images > Import ESXi Image

99

4. Follow with the assistant and create a new baseline (we have named it ESXi 6.0) > Change to Compliance View
and Attach this new baseline > Scan > Remediate > Watch and wait till the server apply the upgrade and reboots the
server.
Stage multiple ESXi Host upgrades
The same principle, but you selecting the host candidates for the upgrade at the cluster level (not at the host level).
In case youre applying the upgrade to a whole cluster you have other options, like deactivate DPN. But basically
whats happening is that host after host is patched and rebooted where the VMs residing on those hosts are
"vMotioned" elsewhere before the patches are applied.
Hosts that are part of VSAN cluster might need more time to evacuate VMs out as the local storage holding the
VMDKs must shift some of those VMDKs elsewhere in order to be able to put the host into maintenance mode and
launch the upgrade. 1 host at a time.

DETERMINE

WHETHER AN IN- PLACE UPGRADE IS APPROPRIATE IN A GIVEN UPGRADE SCENARIO

Upgrade using vSphere Update Manger (VUM)

Interactive Upgrade from an ESXi image on a CD/DVD or USB flash drive
Scripted Upgrade
Using Auto Deploy and reboot it with a new image profile.
Upgrade the ESXi via CLI (using SSH and Putty utility) by usigng esxcli software vib update -d - check this
post.

Tools and Resources:

vSphere Upgrade Guide

vSphere Virtual Machine Administration Guide
vSphere Client / vSphere Web Client

VCP6-DCV OBJECTIVE 4.2 - PERFORM VCENTER SERVER UPGRADE

In no particular we started to cover VCP6-DCV sections to help out folks learning towards VCP6-DCV VMware
certification exam. Due to VMware re-certification policy the VCP exam has now an expiration date. You can renew
by passing delta exam while still holding current VCP or pass VCAP. Today's topic VCP6-DCV Objective 4.2 - Perform
vCenter Server Upgrade.

100

For whole exam coverage I created a dedicated VCP6-DCV page. Or if you're not preparing to pass a VCP6-DCV, you
might just want to look on some how-to, news, videos about vSphere 6 - check out my vSphere 6 page.
vSphere Knowledge covered in today's objective:

Identify steps required to upgrade a vSphere implementation

Identify upgrade requirements for vCenter
Upgrade vCenter Server Appliance (VCA)
Identify the methods of upgrading vCenter
Identify/troubleshoot vCenter upgrade errors

IDENTIFY

STEPS REQUIRED TO UPGRADE A V S PHERE IMPLEMENTATION

VMware recommends few steps before going straight to the upgrade. You should take few precautions, like backing
up vCenter DB (or the whole VM). Also, depending of the vCenter installation (its size, if there are several sites, etc..)
it's necessary to start the upgrade process by:

Read the release notes (what is and what is not supported)

Verify that your system meets vSphere hardware and software requirements.
Check the Update sequence for vSphere 6.0 and its compatible VMware products (2109760)
Best Practices KB - Upgrading to vCenter Server 6.0 best practices (2109772)
Verify compatibility of your Backup/replication/monitoring products (if ok Upgrade them Before you run the
vCenter upgrade). THIS is my take on it. Because If you find yourself in trouble with vCenter upgrade process,
you can always revert to the backup of your vCenter VM that you have done just before you started the
upgrade process. With the latest release of the backup product...
Check the VMware Product Interoperability Matrix in case you're using other VMware solutions (vCD, SRM,
....)
Upgrade vCenter Server
Upgrade vSphere Update Manager
Upgrade ESXi hosts
Upgrade VR, SRM...
Apply vSphere 6 licensing
Upgrade virtual hardware of your VMs and VM tools

IDENTIFY

UPGRADE REQUIREMENTS FOR V C ENTER

vCenter Server requires a 64-bit operating system, and the 64-bit system DSN is required for vCenter Serverto connect
to the external database.

OS support:

2008 SP2 with latest patches and upgrades

2012R2

Internal or external DB

For environments with up to 20 hosts and 200 virtual machines, you can use the bundled PostgreSQL database.
External DB support Oracle, Microsoft SQL. Check Interoperability Matrix!

101

For Windows - synchronize clocks on all machines running the vCenter Server 5.x services (if distributed).
the vSphere Upgrade Guide (p. 30).
If your vCenter Server service is running in a user account other than the Local System account, check that the account
account in which the vCenter Server service is running is:

Member of the Administrators group

Log on as a service
Act as part of the operating system (if the user is a domain user)
Verify that the LOCAL SERVICE account has read permission on the folder in which vCenter Server is installed
and on the HKLM registry.
Check that the connection between the virtual machine or physical server and the domain controller is working

L OGON AS A SERVICE [TIP ] - WHERE TO CHECK

The steps:

Click Start, point to Control Panel, point to Administrative Tools, and then double-click Local Security Policy.
In the console tree, double-click Local Policies, and then click User Rights Assignment.
In the details pane, double-click Log on as a service.

102

Click Add User or Group, and then add the appropriate account to the list of accounts that possess the Log
on as a service right.

vCenter Requirements - Storage

vCenter Requirements - Hardware

Video upgrade 5.5 to 6.0: https://youtu.be/IRsa8a_YApk

UPGRADE V C ENTER S ERVER A PPLIANCE (VCA)

vCenter Server Appliance 5.1U3 and vCenter Server Appliance 5.5 can be upgraded to vCenter Server
Appliance 6. (Not 5.1U2).
VMware vCenter Server Appliance can be deployed only on hosts that are running ESXi version 5.0 or later.
If an external vCenter SSO is used, check out the upgrade process here.
The vCenter Server Appliance PostgreSQL database supports up to 1000 hosts and 10,000 virtual machines.
An Oracle 11g database or an Oracle 12c database are the only external databases supported by the vCenter
Server Appliance.

If you're plan using VUM:

vSphere Update Manager also requires a supported database. Use separate databases for vCenter Server and vSphere
Update Manager.
Upgrade from VCSA 5.5 to VCSA 6.0 is not in place upgrade but rather side-by-side upgrade. We setup a new VCSA 6.0
appliance which will pull all configuration of the current environment from the old VCSA 5.5 appliance (including
historical/performance data).

103

CHECK THIS BEFORE STARTING THE UPGRADE :

Do a backup or create a snapshot of your existing VCSA.

You should check that the vCenter Server SSL certificate for the curent environment is valid and without missconfiguration. There is a VMware KB 2057223.
Verify that the clocks of all machines on the vSphere network are synced. Synchronizing Clocks on the vSphere
Network.
Verify that the ESXi host on which you deploy the vCenter Server Appliance is not in lockdown or maintenance
mode.
In case you are on external database (SQL for example), make sure that you back it up.
The upgrade outline can be found at the vSphere 6 documentation page

Here is what Ive done to upgrade to the latest vSphere 6.0 vCenter (VCSA). After downloading the VCSA 6.0 iso
image from VMware (The latest one is the VMware-VCSA-all-6.0.0-2562643.iso version), there is just very few steps
to do:
1. Mount the ISO and go to the vcsa folder to install VMware Client Integration plugin.

2. Once done, double-click the vcsa-setup.html file located at the root of the DVD

3. This bring the famous window offering you to do a clean install or an upgrade. You might have seen it in my
detailed post about here.

104

Youll get a nag telling you basically that youll have to be on VCSA 5.1 U3 or VCSA 5.5 in order to upgrade to VCSA
6.0 Thats the only options. If youre on other version, you must first upgrade to those two supported ones

VMware has a new KB article on the simple upgrade too.

vCenter Applicance

Appliance Version
vCenter Server IP or FQDN
vCenter Administrator Username
vCenter Administrator Password
vCenter HTTPS Port
Appliance Root password (when using https://vc-address:5480

Source ESXi Host

ESXi host IP or FQDN

ESXi host username
ESXi host password

Check ALL the steps for upgrade VCA in my Detailed Step-by-Step post here - How to Upgrade from VCSA 5.5 to 6.0
Lab Time. Note that I run into a problem with default certificate (solved) during the upgrade.
In case you're doing CLEAN install you might want to check scripted install guide of vCenter server appliance here.
105

IDENTIFY

THE METHODS OF UPGRADING V C ENTER

Embedded Deployment Model The Platform Service Controller (PSC) and the vCenter Server are installed on
the same machine.
External Deployment Model PSC is installed on a separate machine from the vCenter Server.

vCenter 5.5 and earlier deployed using Simple Install option will be upgraded to vCenter Server with embedded
Platform Services Controller.
If vCenter Single Sign-On was on a different machine than vCenter Server, the upgrade will be an external
deployment model.
If vCenter Single Sign-On was on the same node as vCenter Server, the upgrade will product an embedded
deployment model.
Upgrade external SSO servers to Platform Service Controllers, then upgrade vCenter Servers.
Check also this:

List of recommended topologies for VMware vSphere 6.0.x (2108548)

vCenter Server Example Upgrade Paths in the vSphere 6.0 Documentation Center.

Upgrade including an AutoDeploy Server (4) - the upgrade process upgrades it when upgrading the associated
vCenter Server instance. Auto Deploy server included with an earlier version of the product cannot be used in
conjunction with vCenter Server 6.0. If the Auto Deploy server is running on a remote system, it is upgraded and
migrated to the same system as vCenter Server during the upgrade process. Settings are migrated to the new
location. ESXi hosts must be reconfigured to point to the new Auto Deploy location.

106

Upgrading with Remote Web Client Server (5) - it is upgraded along with the vCenter Server instance to which it is
registered and migrated to the same location as the vCenter Server instance.

IDENTIFY / TROUBLESHOOT V CENTER

UPGRADE ERRORS

Windows Based - Logs collection Via:

Installation wizard - browse the generated .zip file on the desktop

Manually - navigate to

%PROGRAMDATA%VMwareCISlogs directory, usually C:ProgramDataVMwareCISlogs

OR to Temp directory
%TEMP% directory, usually C:UsersusernameAppDataLocalTemp
Which files? vminst.log, pkgmgr.log, pkgmgr-comp-msi.log, and vim-vcs-msi.log
For vCenter server appliance

via DCUI (Alt+F1)

pi shell
to access the Bash shell

vc-support.sh
This generates a .tgz archive in /var/tmp
Export it with

scp /var/tmp/vc-etco-vm-vlan11-dhcp-63-151.eng.vmware.com-2014-02-28--21.11.tgz [email protected]:/tmp

And Determine which firstboot script failed.

cat /var/log/firstboot/firstbootStatus.json
VMware Resources:

vSphere Installation and Setup Guide

vSphere Upgrade Guide
VMware vCenter Server 6.0 Deployment Guide
Command-Line Installation and Upgrade of VMware vCenter Server 6.0 for Windows
Command-Line Installation and Upgrade of VMware vCenter Server Appliance 6.0
vSphere Client / vSphere Web Client

107

VCP6-DCV OBJECTIVE 5.1 - CONFIGURE ADVANCED/M ULTILEVEL RESOURCE POOLS

Today's VCP6-DCV topic will touch resource pools. Resource pools aren't folders, remember? Hey, resource pools are
cool when used sparingly, not with 3 levels of inception... VCP6-DCV exam blueprint has this chapter about resource
pools and it's important to know the insight out - VCP6-DCV Objective 5.1 - Configure Advanced/Multilevel Resource
Pools.
The whole exam details, and all topics from the blueprint can be found on the VCP6-DCV page. So in In today's topic
we will learn about resource pools, but also there is a chapter about vFlash architecture. As you know vFRC caching
has been here since vSphere 5.5 and it allows read-only caching mechanism to accelerate applications and VMs.

vSphere Knowledge

Describe the Resource Pool hierarchy

Define the Expandable Reservation parameter
Describe vFlash architecture
Create/Remove a Resource Pool
Configure Resource Pool attributes
Add/Remove virtual machines from a Resource Pool
Create/Delete vFlash Resource Pool
Assign vFlash resources to VMDKs
Determine Resource Pool requirements for a given vSphere implementation
Evaluate appropriate shares, reservations and limits for a Resource Pool based on virtual machine workloads

DESCRIBE

THE

R ESOURCE P OOL

HIERARCHY

vSphere resource management p. 51. vSphere Resource pools can be grouped into hierarchies and used to
hierarchically partition available CPU and memory resources. Resource pools always start at the root level. Each
standalone host and DRS cluster has (invisible) root resource pool. You have to enable DRS first in order to create a
resource pool.
Note: DRS is available in vSphere Enterprise and Enterprise Plus editions.
Resource Pools should be used when you would need to limit or to guarantee resources to VMs. By having resource
pool you don't have to gurantee the resources to VMs individually, but only at the pool level.

Child resource pool - It's possible to create child resource pools under the root resource pool or under any other usercreated resource pool. Each child resource pool owns some of the parent's resources. Inside of each child resource
pool it's possible create another resource pool. (Russian dolly like).
Resource pool can contain:

108

Child resource pools

VMs
Both

Siblings - Resource pools and VMs at the same level are called siblings.
Creating multiple RP allows you to aggregate computing capacity from the underlying hosts within the DRS cluster.
You then can set resources for each resource pool instead on individual VMs. For each resource pool you specify
reservation, limit, shares and you can also specify if the reservation shall be expandable.

DEFINE

THE

E XPANDABLE R ESERVATION

PARAMETER

Expandable Reservation parameter is a value allowing the resource pool resources became available to child
resource pools and virtual machines.

If a VMs workload increases and its resource pool cannot allocate more resources because there arent any available,
the resource pool will asks its parent resource pool to borrow resources. Resource pools that have VMs and workloads
varies, then you should possibly enable expandable reservations.
When the check box is selected (default), expandable reservations are considered during admission control.
If you power on a virtual machine in this resource pool, and the combined reservations of the virtual machines are
larger than the reservation of the resource pool, the resource pool can use resources from its parent or ancestors.

DESCRIBE V F LASH

ARCHITECTURE

New version of VMware vSphere has introduced VMware vFlash Read cache, which enables you to use local SSD
devices pooled together forming a pool of storage tier. The vFlash is integrated with vMotion, HA and DRS. The
solution, the vFlash caching software, is tightly integrated into the hypevizor (placed into the data path), as an API,
which is also available for third party caching modules. vFlash is a service within vSphere.
109

Flash Pooling as a resource pool:

vFlash will appear as a new type of resource pool

No consumption when VM is powered Off
vMotion and DRS can be used
The allocation of resources is based per virtual object (VM, Host)

The Flash Resource management uses:

Reservations, limits
Uses per VMDK or per VM allocation (the config is at the VM level).
Enforces admission control
vFlash is a broker and manager for the entities which consumes the resources

V F LASH

R ELEASE 1.0

SUPPORTS

W RITE THROUGH CACHE (R EAD

ONLY )

The first release supports write through mode, which is read only. The write back mode will be available in future
releases. Its important to understand the the publicly available APIs gives opportunity to other storage companies to
integrate their flash caching solution.

W HAT S

NEEDED FOR V FRC?

A configured of hosts with each one with at least one SSD or PCIe SSD
vSphere 5.5 (vCenter 5.5 and ESXi 5.5)

W HERE

TO START WITH V FRC?

At the cluster level. You have the choice. You can right-click the cluster >All vCenter actions > Add virtual flash
resource capacity.

110

On the next screen you select available SSD from each ESXi host and click OK.

CREATE/R EMOVE

R ESOURCE P OOL

To be able to create Resource pool you must enable DRS. You can use both vSphere C# client or vSphere Web Client.
(Web client)
Select Hosts and clusters > Manage > vSphere DRS > Edit > Check the Turn ON.

111

Easiest way to create resource pool is perhaps the Right click at the cluster > New resource pool...

To delete, simple too. Right click the Resource Pool > Delete

CONFIGURE R ESOURCE P OOL

ATTRIBUTES

Navigate to the Host and Clusters view (View > Inventory > Hosts and Clusters)
Right-click on the resource pool you want to edit and select Edit Settings
Change the name if desired
Change the CPU Shares, Reservation, Expandable Reservation and Limit if desired
Change the Memory Shares, Reservation, Expandable Reservation and Limit if desired

CPU R ESOURCES
Shares - Specify shares for this resource pool with respect to the parents total resources. The amounts of shares you
allocate to a resource pool are relative to the shares of any sibling (virtual machine or resource pool) and relative to
its parents total resources. Sibling resource pools share resources according to their relative share values bounded
by the reservation and limit.
Different types of shares - Low (1), Normal (2), or High (4) which specify share values in a ratio. Or you can select
Custom to give each RP a specific number of shares, which expresses a proportional weight.

112

Reservation - Specify a guaranteed CPU or memory allocation for this resource pool. Defaults to 0. A nonzero
reservation is subtracted from the unreserved resources of the parent (host or resource pool). The resources are
considered reserved, regardless of whether virtual machines are associated with the resource pool.
Limit - Upper limit for this resource pools CPU allocation. Select Unlimited to specify no upper limit.
Memory Resources
Shares - Memory shares for this resource pool with respect to the parents total. Sibling resource pools share
resources according to their relative share values bounded by the reservation and limit. Select Low (1), Normal (2),
or High (4), which specify share values in a ratio.
Select Custom to give each virtual machine a specific number of shares, which expresses a proportional weight.
Reservation - Guaranteed memory allocation for this resource pool.
Limit - Upper limit for this resource pools memory allocation. If you give RP limit 32Gb RAM it will never receive
more RAM even if the host/cluster is able to allocate more. Select Unlimited to specify no upper limit.

A DD /R EMOVE

VIRTUAL MACHINES FROM A

R ESOURCE P OOL

No difficulties here. It's possible to use both clients. Drag and drop... -:)

113

Or when creating new VM, during the wizard creation you're asked whether you want to place the VM into specific
resource pool...
If the resource pool does not have enough resources to guarantee the virtual machine reservation(s) then the move
into the resource pool will fail (for powered-on virtual machine).
Create/Delete vFlash Resource Pool
To delete RP, similar as creation. Do a right-click on the RP > delete.
Drag-and-drop the virtual machine into another resource pool. You can also drag it into the root of the DRS cluster
which will move it into the root resource pool.

A SSIGN V F LASH

RESOURCES TO

VMDK S

Once you've added a capacity to the cluster by providing some flash resources from each of the hosts present in the
cluster, you can now add those ressources to individual VMs (or respectively the VMDKs).You can check the Flash
Read cache resource availability through the Summary Tab.

114

You have to go and do it at the VM level (vFRC operates per VMDK). Select an individual VM and click edit settings >
Virtual hardware tab >next to the Virtual Flash Read Cache > click Advanced.

Now can select the amount of GB (Mb) that will be reserved for that particular virtual hard drive. Depending of your
workloads, because vFRC has variable block size capability (4kb 1M). So the best selection will depends on your
application, which IO size or your application which runs in your VM. (you can used for example vISCSI stats to find
out) Then you carve up those information to match the block size of the vFlash to give the best possible performance.
Not every node in the vFlash cluster needs to have SSD installed, but if thats the case, the particular host wont be
able to provide any vFlash resources.

DETERMINE R ESOURCE P OOL

REQUIREMENTS FOR A GIVEN V S PHERE IMPLEMENTATION

It depends is a good answer... Before determining the requirements you'll need to determine the workloads that will
be running in the environment and also priorities within the whole infrastructure. RP are here to help to segment the
resources by organization, by workload or other business requirements.
Once you have defined the workloads, you can start dividing up the resources pools the way it is able to meet the
requirements of the workloads running on the DRS cluster.
You should check whether the RP need to reach out to the parent RP to provide more resources -> configure
expandable reservations.
Check if you need reservations or limits. Do not use per-VM reservations as it's like if you would use per file NTFS
permissions... [Administrative Overhead]. If you're using reservation then use it at the resource pool level.

115

E VALUATE

APPROPRIATE SHARES , RESERVATIONS AND LIMITS FOR A

RESOURCE P OOL

BASED ON

VIRTUAL MACHINE WORKLOADS

Know your workload first, then only you'll be able to define shares, reservations and (or) limits. We have talked
about CPU shares, reservations, limits and Memory shares, reservations, limits in the chapter above. All the
resources available within the cluster can be managed and distributed by Resource pools depending on how they're
configured, but this determines the requirements. Note that limits is a resource limit and so it's not the same as if
you were used shares which depends on other resources and their availability.
Tools and links:

vSphere Resource Management Guide

vSphere Virtual Machine Administration Guide
Whats New in VMware vSphere Flash Read Cache
vSphere Client / vSphere Web Client

VCP6-DCV OBJECTIVE 6.1 - CONFIGURE AND ADMINISTER A VSPHERE

BACKUPS /RESTORE /REPLICATION SOLUTION
VMware vSphere comes with free vSphere Data protection (VDP) product. There is no more VDP and VDP advanced
as the VDP inherited all advanced features of VDP Advanced. So VDP is Advanced by Default. This post will cover
VCP6-DCV Objective 6.1 - Configure and Administer a vSphere Backups/Restore/Replication Solution.
It seems that for VCP6 there is more material to study and more topics to master. For whole exam coverage I created
a dedicated VCP6-DCV page. Or if you're not preparing to pass a VCP6-DCV, you might just want to look on some
how-to, news, videos about vSphere 6 - check out my vSphere 6 page.
VMware vSphere Knowledge:

Identify snapshot requirements

Identify VMware Data Protection requirements
Explain VMware Data Protection sizing Guidelines
Identify VMware Data Protection version offerings
Describe vSphere Replication architecture
Create/Delete/Consolidate virtual machine snapshots
Install and Configure VMware Data Protection
Create a backup job with VMware Data Protection
Install/Configure/Upgrade vSphere Replication
Configure VMware Certificate Authority (VMCA) integration with vSphere Replication
Configure Replication for Single/Multiple VMs
Identify vSphere Replication compression methods
Recover a VM using vSphere Replication
Perform a failback operation using vSphere Replication
Determine appropriate backup solution for a given vSphere implementation

IDENTIFY

SNAPSHOT REQUIREMENTS

As you know vSphere FT VMs can now be protected (backed up) via backup solutions using snapshots. No manual
snapshots for those VMs however as the snapshots are managed through an API calls only.
VDP utilizes the Changed Block Tracking (CBT) greatly reducing the backup time of a VMs and so you can process
much more VMs during your backup window than without using CBT. Note that CBT is also leveraged during restores

116

where the if restored to the original location, VDP can determine the missing blocks in the destination and only
restore those. Not all the blocks.
VDP leverages deduplication technology based on Avamar's code. Full VM recovery, File level recovery - both
supported in VDP. vSphere data protection (VDP) and vSphere replication (VR) both uses snapshots on regular basis
to protect VMs (or to replicate them). In the case of VR the RPO is as low as 15 min.

IDENTIFY VM WARE DATA P ROTECTION

REQUIREMENTS

Image level backups - vSphere Data Protection creates imagelevel backups, which are integrated with the vStorage
API for Data Protection, a feature set within vSphere to offload the backup processing overhead from the virtual
machine to the VDP Appliance. The VDP Appliance communicates with the vCenter Server to make a snapshot of a
virtual machines .vmdk files. Deduplication takes place within the appliance by using a patented variablelength
deduplication technology.
Guest-level backup - VDP supports guestlevel backups for Microsoft SQL Servers, Exchange Servers, and Share Point
Servers. With
guestlevel backups, client agents (VMware VDP for SQL Server Client, VMware VDP for Exchange Server
Client, or VMware VDP for SharePoint Server Client) are installed on the SQL Server, Exchange Server, or
SharePoint Server in the same manner that backup agents are typically installed on physical servers
VDP can not only protect VMs but also physical systems! - Microsoft Exchange, SQL Server, SharePoint when
backed up by VDP, the agents which needs to get installed on those servers in order to protect them efficiently, are
leveraged for granular restores. It does not have to be VMs to allow application level recovery.

E XPLAIN VMWARE DATA P ROTECTION

SIZING

G UIDELINES

vSphere web client is necessary for deployment and administration of the VDP, which can be deployed on practically
any storage (VMFS, NFS or VSAN). vSphere Data Protection 6.0 Administration Guide p.20
CAPACITY R EQUIREMENTS :

Up to 20 VDP appliances per vCenter server

Each appliance can protect up to 400 VMs
8TB of deduplicated backups

S IZING DEPENDS ON FOLLOWING FACTORS :

Types of data being backed up (files, DB, OS files)

Data change rate
Size of protected VMs and their numbers
Retention period (daily, weekly, monthly or yearly)
Deployment availability of VDP as 0.5TB, 1Tb, 2Tb, 4Tb, 6Tb, 8Tb (if deployed small size, can be increased
later).

vSphere data protection 6.0 administration guide p. 21

117

S OFTWARE R EQUIREMENTS :

Minium requirements is vCenter 5.1 to install VDP 6, but 5.5 or higher is recommended.
VDP 6 supports vCSA and Windows based vCenters
vSphere Web client where browsers needs Flash player 11.3 or above version installed.
NOTE: VDP do not support of backup of vCenter server appliance (VCSA) itself.
VMs to be protected must be on virtual hardware version 7 or higher (CBT) and VMware tools installed.
VDP repository usually fills rapidly for the first few weeks. This is because nearly every client that is backed
up contains unique data. But then VDP deduplication allows to save space when other similar clients have
been backed up, or the same clients have been backed up at least once.

U NSUPPORTED VM S DISKS :

Independent
RDM Independent - virtual compatibility mode
RDM w. physical compatibility mode

IDENTIFY VM WARE DATA P ROTECTION

VERSION OFFERINGS

I guess this is a bit erroneous topic as the VDP is by default now a VDP Advanced. However I think it's worth to know
that in the past there were two versions: VDP and VDP advanced. VDP allows:

Disk level Granularity allows backup/restore individual VMDK (virtual disks).

Restore directly with ESXi (if vCenter is not available) by going to https://<ip_of_vdp>/vdp-configure you can
access to Emergency restore tab where you can trigger restores.
Detachable/remountable data partitions for DR scenarios of VDP
Replication to the cloud off site backups
Time-of-day scheduling schedule backup to be triggered exactly when you want
Removal of the blackout window

VDP has also:

Application-level replication
Ability to expand current datastore
Backup to a Data Domain system
Ability to restore to a granular level on Microsoft Servers and automatic backup verification.
VDP also supports guest-level backups and restores of Microsoft SQL Servers, Exchange Servers, and Share
Point Servers, providing for application consistent backups of these servers.

A migration tool is included with VDP 5.1.10 and later releases. This tool handles migration of data and restore
points. Backup jobs cannot be migrated.

DESCRIBE V S PHERE R EPLICATION

ARCHITECTURE

118

vSphere replication is separate product included in vSphere. It allows to configure replication of VMs from source
site to target site. It uses snapshots (points-in-time) to transfer delta informations to the other side.
Types of replication:

Within single site - from one cluster to another

From multiple source sites - to shared remote site
From source site to target site

vCenter server (Windows) or VCSA can be used. Possibility to deploy additional VR servers to enhance.
VMware VSAN is supported as target (destination) datastore.
A RCHITECTURE :

The vSphere replication appliance contains the following:

vCenter Plugin for vSphere web client.

An embedded database storing replication config and management information.
vSphere Replication management server - configures vSphere replication server, enables, manages, monitors
replication and also authenticate users and check their permissions for VR operations.
vSphere Replication Server - provide the core of VR infra.

Below example of architecture with single vCenter server and single site (possible also multi-site to shared location
or two sites in between).

From the network perspective it's necessary to setup vmkernel adapter per ESXi host which is used as a replication
source, for isolation of the replication traffic.

CREATE/DELETE/CONSOLIDATE

VIRTUAL MACHINE SNAPSHOTS

To create a VM snapshot. Two ways possible (vSphere client or vSphere web client). Select VM > take snapshot of
this virtual machine.
Delete snapshot - via snapshot manager > delete

119

Consolidate VM snapshots - if any VM that shows that needs to consolidate, just select and right click that
particular VM and choose Consolidate.
Right click > Shapshot > Consolidate

INSTALL

AND

C ONFIGURE VM WARE DATA P ROTECTION

VDP is VSA based (Linux). The deployment as an OVF is fast and convenient.

Requirements:

NTP - All vSphere hosts and the vCenter Server must have NTP configured properly. The VDP Appliance gets
the correct time through vSphere and must not
be configured with NTP.
DNS - create DNS forward and reverse record and check that you have vCenter server responding via
nslookup.

Deploy the OVF file via vSphere Web client to a VMFS5 datastore (to avoid block size limitations).
After the deployment and start up of the VM go to the IP address precised on the console.
https://ip_of_vdp:8543/vdp-configure
Login:
pass: changeme

root

Follow the assistant, you should have the info pre-filled when you click the next button...
120

continue with the wizard. Test your connection to vCenter to avoid issues...

Create storage. Here you can (but don't have to) check the box "store with appliance" in case you have enough space
on the shared storage datastore you have chosen.

121

Continue with the assistant until the end. After the setup finished the appliance will reboot...

122

It takes up to 15 min to fully setup after the reboot... -:) You'll have to log off and log in back again through vSphere
web client to see this new plugin to appear..

CREATE

A BACKUP JOB WITH

VM WARE DATA P ROTECTION

To create a first backup job, just click through the new icon on the dashboard in vSphere web client.

Then start an assistant...

123

continue..

Choose a VM(s)...

124

Backup schedule...

Specify retention policy.... Note that this can be changed later. (Think of sizing).

Give the job some meaningful name...

And off you go.

125

Just created first backup job. If you go and click the Configuration TAB, then down there you can configure the Backup
window configuration... If not the default backup starts at 8PM...

INSTALL/C ONFIGURE/UPGRADE V S PHERE R EPLICATION

vSphere Replication is distributed as ISO. Mount the ISO to access the OVF file to be deployed.
Requirements:

Source and target site must have vSphere web client and the client integration plugin is installed as well
Select the vCenter Server instance on which you are deploying vSphere Replication, click Manage > Settings >
Advanced Settings, and verify that the VirtualCenter.FQDN
value is set to a fully-qualified domain name or a literal address

Network ports - For a list of all the ports that must be open for vSphere Replication, see
http://kb.vmware.com/kb/2087769
Bandwidth - vSphere Replication transfers blocks based on the RPO schedule. If you set an RPO of one hour,
vSphere Replication transfers any block that has changed in that hour to meet that RPO. vSphere Replication
only transfers the block once in its current state at the moment that vSphere Replication creates the bundle
of blocks for transfer. vSphere Replication only registers that the block has changed within the RPO period,
not how many times it changed

V S PHERE

R EPLICATION DEPLOYMEN T

vSphere Replication 6.0 administration guide p. 31 Select cluster and then Actions > deploy OVF template > local file
> browse... and so on...
If you don't want to relay on the DHCP you can use fixed IP.... Select a network from the list of available networks, set
the IP protocol and IP allocation, and click Next. vSphere Replication supports both DHCP and static IP addresses. You
can also change network settings by using the virtual appliance management interface (VAMI) after installation.

126

And then

Once done. Log off and log back again to see the VR plugin

127

CONFIGURE VM WARE CERTIFICATE A UTHORITY (VMCA)

R EPLICATION

INTEGRATION WITH V SPHERE

You can change the SSL certificate, for example if your company's security policy requires that you use trust by validity
and thumbprint or a certificate signed by a certification authority. You change the certificate by using the virtual
appliance management interface (VAMI) of the vSphere Replication appliance. For information about the SSL
certificates that vSphere Replication uses, see vSphere Replication Certificate Verification, on page 45 and
Requirements When Using a Public Key Certificate with vSphere Replication, on page 46.

128

CONFIGURE R EPLICATION

FOR

S INGLE/M ULTIPLE VM S

Before this, make sure that you have the permissions.

Step 1: Select VM(s) > Right click > All vSphere Replication Actions > configure Replication
Now if you haven't restarted the vCenter service, you see this (1), because after restart you should see this (2). Also,
you'll get some error on the permissions if you don't restart, and so you won't be able to configure the replication
for your VMs. That "from the field" experience ...

Step 2: Replicate to a vCenter server (or service provider) > select target site > target location...

129

And enable compression...

Step 3: You can change the RPO settings and enable the Point in time instances on this screen...

IDENTIFY V S PHERE R EPLICATION

COMPRESSION METHODS

vSphere Replication 6.0 administration guide p. 16. The compression settings depends on the version of VR and
version of ESXi at the destination. But basically if source or destination has earlier than ESXi 6.0 and VR earlier than
6.0 the compression is not used.
But what's interesting is the fact that if compression is enabled. Quick quote:
However, if the target ESXi host is earlier than 6.0,vSphere Replication prevents vMotion from moving replication
source VMs to that host because it does notsupport data compression. This prevents DRS from performing
automated vMotion operations to hosts thatdo not support compression. Therefore, if you need to move a
replication source VM to an ESXi host earlier than 6.0, before you perform the vMotion operation, you must
reconfigure the replication to disable data compression.

R ECOVER

VM

USING V S PHERE

R EPLICATION

vSphere Replication 6.0 administration guide p. 77. With Sphere Replication, you can recover virtual machines that
were successfully replicated at the target site. You can recover one virtual machine at a time.

130

Web client > vSphere replication > Home tab > Monitor > Incoming replication

From there you have two options:

1. Recover with recent changes - Performs a full synchronization of the virtual machine from the source site to
the target site before recovering the virtual machine. Selecting this option avoids data loss, but it is only
available if the data of the source virtual machine is accessible. You can only select this option if the virtual
machine is powered off.
2. Recover with latest available data - Recovers the virtual machine by using the data from the most recent
replication on the target site, without performing synchronization. Selecting this option results in the loss of
any data that has changed since the most recent replication. Select this option if the source virtual machine is
inaccessible or if its disks are corrupted.

You continue and select folder where you want to recover the VM...

P ERFORM

A FAILBACK OPERATION USING V S PHERE

R EPLICATION

vSphere Replication 6.0 administration guide p. 79. Failback is manual, it means that after performing a successful
recovery on the target vCenter Server site, you can perform failback. You log in to the target site and manually
configure a new replication in the reverse direction, from the target site to the source site. The disks on the source
site are used as replication seeds, so that vSphere Replication only synchronizes the changes made to the disk files
on the target site.
Before you configure a reverse replication, you must unregister the virtual machine from the inventory on the source
site.

DETERMINE

APPROPRIATE BACKUP SOLUTION FOR A GIVEN V S PHERE IMPLEMENTATION

131

Depending on your needs it's necessary to size accordingly your backup solution. You must take into account the daily
delta changes within your all environment and see if the product you want to use as a backup solution is suitable. How
it scale? What's the limitations?
You must also take into account the possible conflicts with other vSphere products you may be using (vSphere
replication, SRM, vCD....). If you're planning to use VDP, than you should certainly check vSphere compatibility matrix.
Tools:

VMware vSphere Data Protection 6.0

vSphere Data Protection Administration Guide
VMware vSphere Data Protection Evaluation Guide
Whats New in the VMware vSphere 6.0 Platform
VMware vSphere Replication Administration
VDR Data Migration Tool
VDP Configure Utility
vSphere Client / vSphere Web Client

VCP6-DCV OBJECTIVE 7.1 - TROUBLESHOOT VCENTER SERVER , ESXI HOSTS , AND VIRTUAL
M ACHINES
In today's Objective we'll discuss VCP6-DCV Objective 7.1 - Troubleshoot vCenter Server, ESXi Hosts, and Virtual
Machines. You can check the whole VCP6-DCV Study Guide page for all topics there. You can also check the vSphere
6 page where youll find many how-to, videos, and tutorials about vSphere 6.
Another troubleshooting chapter today. After we cracked the troubleshooting of vSphere upgrades, in another
troubleshooting chapter we hit the storage and network issues, today we'll hit the Toubleshooting of vCenter, ESXi
and VMs.
When something goes wrong with vCenter, only things that rely on vCenter does suffer. Things like HA, DRS or FT
continues to work, but you can't manually vMotion a VM if you don't have an access to vCenter. It can be that one of
the vCenter services went down or something like that. Today well' have a look at those different things which can
happened.

vSphere Knowledge

Identify general ESXi host troubleshooting Guidelines

Identify general vCenter troubleshooting Guidelines
Troubleshoot Platform Services Controller (PSC) issues
Troubleshoot common installation issues
Monitor ESXi system health
Locate and analyze vCenter and ESXi logs
Export diagnostic information
Identify common Command Line Interface (CLI) commands
Troubleshoot common virtual machine issues
Troubleshoot virtual machine resource contention issues
Identify Fault Tolerant network latency issues
Troubleshoot VMware Tools installation issues
Identify/Troubleshoot virtual machines various states (e.g. orphaned, unknown, etc.)
Identify virtual machine constraints
Identify the root cause of a storage issue based on troubleshooting information
Identify common virtual machine boot disk errors
132

Identify and detect common knowledge base article solutions

IDENTIFY

GENERAL

ESX I

HOST TROUBLESHOOTING

G UIDELINES

When starting troubleshooting, you should first:

Identify symptoms - WTF? .... is going on?

Define problem space - software? Hardware? What is causing the problem? What's excluded?
Test solutions - Once knwing the symptoms and problem space, you can test solutions, one by one until
problem resolved.

check vSphere 6 troubleshooting guide p.7 and onward...

IDENTIFY

GENERAL V C ENTER TROUBLESHOOTING

G UIDELINES

Few good troubleshooting scenarios is in the vSphere 6 troubleshooting guide p.33

You'll find problems (and their resolution) like those one below:

vCenter Server Upgrade Fails When Unable to Stop Tomcat Service

Microsoft SQL Database Set to Unsupported Compatibility Mode Causes
vCenter Server Installation or Upgrade to Fail
Error When You Change vCenter Server Appliance Host Name
vCenter Server System Does Not Appear in vSphere Web Client Inventory
Unable to Start the Virtual Machine Console
Unable to View the Alarm Definitions Tab of a Data Center
vCenter Server Cannot Connect to the Database
vCenter Server Cannot Connect to Managed Hosts

TROUBLESHOOT P LATFORM S ERVICES CONTROLLER (PSC)

ISSUES

PSC logs location and names:

cis-license - VMware Licensing Service

SSO - VMware Secure Token Service
VMCA - VMware Certificate Service
vmdird - VMware Directory Service

For Platform Services Controller node deployments, additional runtime logs are located at
C:\ProgramData\VMware\CIS\runtime\VMwareSTSService\logs
including logs for these services:

VMware Secure Token Service

VMware Identity Management Service

TROUBLESHOOT

COMMON INSTALLATION ISSUES

Recursive panic might occur when using ESXi Dump Collector - PSOD. Check release notes.

vSphere installation guide p.245

133

V C ENTER SERVER ON

W INDOWS

Collect Installation Logs by Using the Installation Wizard - You can use the Setup Interrupted page of the
installation
wizard
to
browse
to
the
generated
.zip
file
of
the
vCenter Server for Windows installation log files. If the installation fails, the Setup Interrupted page appears
with the log collection check boxes selected by default.

The installation files are collected in a .zip file on your desktop, for example, VMware-VCS-logs-time-of-installationattempt.zip
You can then unzip the log file located on your desktop and start checking what's wrong.
Manual retrieve of logs:
C:\ProgramData\VMware\vCenterServer\logs
C:\Users\username\AppData\Local\Temp
The files in the %TEMP% directory include vminst.log, pkgmgr.log, pkgmgr-comp-msi.log, and vim-vcs-msi.log
V C ENTER

A PPLIANCE
The full path to the log files is displayed in the vCenter Server Appliance deployment wizard.
1. Log in to the Windows host machine on which you want to download the bundle.
2. Open a Web browser and enter the URL to the support bundle displayed in the DCUI.
https://appliance-fully-qualified-domain-name:443/appliance/support-bundle
3. Enter the user name and password of the root user.
4. Click Enter > The support bundle is downloaded as .tgz file on your Windows machine.
5. (Optional) To determine which firstboot script failed, examine the firstbootStatus.json file.
If you ran the vc-support.sh script in the vCenter Server Appliance Bash shell, to examine the firstbootStatus.json file,
run
cat /var/log/firstboot/firstbootStatus.json

Attempt to Install a Platform Services Controller After a Prior Installation Failure

Collect Installation Logs by Using the Installation Wizard.

M ONITOR ESX I

SYSTEM HEALTH

Hardware Monitoring on ESXi - The Common Information Model (CIM) is used on ESXi instead of installing the
hardware agents in the Service Console. The different CIM providers are available for different hardware installed in
the server (HBA, Network cards, Raid Controllers etc). [source...]
If connected through vCenter:

134

OR, If connected directly to the ESXi host:

LOCATE

AND ANALYZE V C ENTER AND

ESX I

LOGS

VMware KB - Location of log files for VMware products (1021806)

Export diagnostic information
Create a Log Bundle (via Web client)
Locate/Analyze VMware Log Bundles
To collect ESX/ESXi and vCenter Server diagnostic data:
1. Start the vSphere Web Client and log in to the vCenter Server system.
2. Under Inventory Lists, select vCenter Servers.
3. Click the vCenter Server that contains the ESX/ESXi hosts from which you want to export logs.
135

4. Click the Monitor tab and click System Logs.

5. Click Export System Logs.

1.
2.
3.
4.
5.

Select the ESX/ESXi hosts from which you want to export logs.
Select the Include vCenter Server and vSphere Web Client logs option. This step is optional.
Click Next.
Select the system logs that are to be exported.
Select Gather performance data to include performance data information in the log files.Note: You can update
the duration and interval time between which you want to collect the data.
6. Click Next.
7. Click Generate Log Bundle. The Download Log Bundles dialog appears when the Generating Diagnostic Bundle
task completes.

1. Click Download Log Bundle to save it to your local computer.Note: The host or vCenter Server generates .zip
bundles containing the log files. The Recent Tasks panel shows the Generate diagnostic bundles task in
progress.

TO

EXPORT THE EVENTS LOG :

1.
2.
3.
4.

Select an inventory object.

Click the Monitor tab, and click Events.
Click the Export icon.
In the Export Events window, specify what types of event information you want to export.
136

5. Click Generate CSV Report, and click Save.

Same covered in VCP6-DCV Objective 7.3 Troubleshoot vSphere Upgrades.

IDENTIFY

COMMON

C OMMAND LINE I NTERFACE (CLI)

COMMANDS

Cli commands. Depending what you want to do, which part of the infrastructure you targetting:

vmkping - simple ping via vmkernel interface (ex. How-to troubleshoot iSCSI connection to your SAN )
vmkfstools - works with VMFS volumes, VMDKs ... (ex Recreate a missing VMDK header file )
esxcli network <namespace> - ( ex. How to create custom ESXi Firewall rule )
esxcli storage <namespace>- ( ex. How to tag disk as SSD VMware esxi 5.x and 6.0 )
esxtop - performance monitoring - (ex. How-to check Queue Depth Of Storage Adapter or Storage Device )

TROUBLESHOOT COMMON VIRTUAL MACHINE ISSUES

TROUBLESHOOT VIRTUAL MACHINE RESOURCE CONTENTION
IDENTIFY F AULT TOLERANT NETWORK LATENCY ISSUES

ISSUES

For FT you'll need 10GbE pipe. That's a fact. vSphere 6 Features - New Config Maximums, Long Distance vMotion and
FT for 4vCPUs.

TROUBLESHOOT VM WARE TOOLS

INSTALLATION ISSUES

VMware KB Article 1003908 Troubleshooting a Failed VMware Tools Installation in a Guest Operating
System.
How to remove VMware Tools manually if uninstall or upgrade finish with error
Manual Download of VMware Tools from VMware Website

IDENTIFY /TROUBLESHOOT
ETC .)

VIRTUAL MACHINES VARIOUS STATES

( E. G .

ORPHANED , UNKNOWN,

A virtual machine is deleted outside of vCenter Server - A user can delete a virtual machine through the VMware
Management Interface while vCenter Server is down, through the vSphere Client directly connected to an ESX/ESXi
host, or by deleting the virtual machine's configuration file through the service console. These virtual machines can
be removed from the vCenter Server by right-clicking the virtual machine and selecting delete

Virtual machines appear as invalid or orphaned in vCenter Server (1003742)

137

IDENTIFY

VIRTUAL MACHINE CONSTRAINTS

VMware KB Article 1008360 Troubleshooting Virtual Machine Performance Issues

Troubleshooting a virtual machine that has stopped responding: VMM and Guest CPU usage comparison
(1017926)
VMware KB Article 2001003 Troubleshooting ESX/ESXi Virtual Machine Performance Issues

IDENTIFY

THE ROOT CAUSE OF A STORAGE ISSUE BASED ON TROUBLESHOOTING INFORMATION

Often the root cause is storage. We all know that spinning media are slowly replaced by SSDs, but they still have
some years to come. Storage contention happens when the demand of hosts for IOs exceeds the the storage and
hba(s). The contention can happens at the VM level, HBA level or at the arrray level.
ESXTOP:
davg average response time for a command which are sent to the device.
kavg average response time a command is in the vmkernel
gavg response time as it appears to the VM. (davg + kavg).
CMD/s number of IOps sent or received from the device or the VM

IDENTIFY

IDENTIFY

COMMON VIRTUAL MACHINE BOOT DISK ERRORS

kb.vmware.com/kb/1006296 - Cannot boot or start a virtual machine converted by VMware vCenter

Converter 4.x/5.x (1006296)
Identifying critical Guest OS failures within virtual machines
AND DETECT COMMON KNOWLEDGE BASE ARTICLE SOLUTIONS

KB 2000988 Troubleshooting vSphere Auto Deploy

KB 653 Collecting Diagnostic Information for VMware ESX/ESXi
KB 1008360 Troubleshooting Virtual Machine Performance Issues
KB 2001003 Troubleshooting ESX/ESXi Virtual Machine Performance Issues
KB 1003908 Troubleshooting a Failed VMware Tools Installation in a Guest Operating System
KB 1003999 Identifying Critical Guest OS Failures Within Virtual Machines.

Tools used for this Objective

vSphere Installation and Setup Guide

vSphere Troubleshooting Guide
vSphere Virtual Machine Administration Guide
vSphere Server and Host Management Guide
vSphere Monitoring and Performance Guide
vSphere Security Guide
vSphere Client / vSphere Web Client

138

VCP6-DCV OBJECTIVE 7.2 - TROUBLESHOOT VSPHERE STORAGE AND NETWORK ISSUES

Today's topic of VCP6-DCV Study Guide is touching troubleshooting. In case something goes wrong and you loose
connectivity to your application, you must probably troubleshoot the underlying VM first, the network second, but
also a storage. When storage is under a pressure then your whole infrastructure just slows down and you might
experience disconnections at the VM/application level. VCP6-DCV Objective 7.2 - Troubleshoot vSphere Storage
and Network Issues is today's lesson.
You can also check vSphere 6 page where you'll find how-to's, news, videos concerning vSphere 6.x. Last but not
least, my Free Tools page where are the post popular tools for VMware and Microsoft. Daily updates of the blog are
taking time, but we do it in the goal to provide a guide which is helpful for the community and folks learning towards
VCP6-DCV certification exam. If you find one of those posts useful for your preparation, just share.. -:).

vSphere Knowledge

Verify network configuration

Verify storage configuration
Troubleshoot common storage issues
Troubleshoot common network issues
Verify a given virtual machine is configured with the correct network resources
Troubleshoot virtual switch and port group configuration issues
Troubleshoot physical network adapter configuration issues
Troubleshoot VMFS metadata consistency
Identify Storage I/O constraints
Monitor/Troubleshoot Storage Distributed Resource Scheduler (SDRS) issues

V ERIFY

NETWORK CONFIGURATION

Start from one end. Either from the host level > physical switch > uplinks > switches > port groups > VMs

Check the vNIC status - connected/disconnected

Check the networking config inside Guest OS - yes it might also be one of the issues. Bad network config of
the networking inside of a VM.
Verify physical switch config
Check the vSwitch or vDS config
ESXi host network (uplinks)

Guest OS config
Check for disabled/inactive adapters or other unused hardware (if Guest OS has been P2V)
In Windows VM do this:
Click on Start > Run > devmgmt.msc > click + next to network adapters > check if it's not disabled or not present
You can also check the network config like IP address, Netmask, default gateway and DNS servers. Make sure that
those informations are correct.

If a VM was P2V - check if there are no "ghosted adapters". To check that:

On your VM go to Start > RUN > CMD > Enter > Type

set devmgr_show_nonpresent_devices=1
139

While still in the command prompt window type:

devmgmt.msc
and then open Device Manager and click on the Menu go to View > Show Hidden Devices (like on the pic).

Then you should see which devices are marked like ghosted devices.They are grayed out. Those devices you can safely
remove from the device manager.

Check IP stack - It happened to me several times that the IP stack of a VM was corrupted. The VM has had
intermittent networking connectivity, everything seems to be ok but isn't. You can clear the local cache by
entering this:

ipconfig /renew
For Linux:

dhclient
dhclient eth0
V ERIFY

-r

STORAGE CONFIGURATION

Check the documentation of vSphere storage, the basic concepts, iSCSI etc.
I've done few posts in configuring iSCSI and vSphere (not particulary related to vSphere 6 but those are step-by-steps:

How to configure FreeNAS 8 for iSCSI and connect to ESX(i)

How to configure ESXi 5 for iSCSI connection to Drobo
Configuring iSCSI port binding with multiple NICs in one vSwitch for VMware ESXi 5.x and 6.0.x

Also check this VMware KB for Teaming and Failover Policy section in the vSphere Networking guide.

140

TROUBLESHOOT

COMMON STORAGE ISSUES

Storage Issues - Check that the virtual machine has no underlying issues with storage or it is not experiencing
resource contention, as this might result in networking issues with the virtual machine. You can do this by logging
into ESX/ESXi or Virtual Center/vCenter Server using the VI/vSphere Client and logging into the virtual machine
console.
Good doc - Troubleshooting Storage guide (p.55 - p.70) which talks about:

Resolving SAN Storage Display Problems - page 56

Resolving SAN Performance Problems on page 57
Virtual Machines with RDMs Need to Ignore SCSI INQUIRY Cache on page 62
Software iSCSI Adapter Is Enabled When Not Needed on page 62
Failure to Mount NFS Datastores on page 63
VMkernel Log Files Contain SCSI Sense Codes on page 63
Troubleshooting Storage Adapters on page 64
Checking Metadata Consistency with VOMA on page 64
Troubleshooting Flash Devices on page 66
Troubleshooting Virtual SAN on page 69
Troubleshooting Virtual Volumes on page 70

TROUBLESHOOT

COMMON NETWORK ISSUES

Again, networking can be tricky to troubleshoot. But choosing one end to start with should help. Another tip is perhaps
to check load balancing policies when more than 1 nic is connected to a VM.
Verify that the virtual machine is configured with two vNICs to eliminate a NIC or a physical configuration issue. To
isolate a possible issue:

If the load balancing policy is set to Default Virtual Port ID at the vSwitch or vDS level:
o Leave one vNIC connected with one uplink on the vSwitch or vDS, then try different vNIC and pNIC
combinations until you determine which virtual machine is losing connectivity.
If the load balancing policy is set to IP Hash:
a. Ensure the physical switch ports are configured as port-channel. For more information on verifying
the configuration on the physical switch, see Sample configuration of EtherChannel / Link aggregation
with ESX/ESXi and Cisco/HP switches (1004048).
b. Shut down all but one of the physical ports the NICs are connected to, and toggle this between all the
ports by keeping only one port connected at a time. Take note of the port/NIC combination where the
virtual machines lose network connectivity.
Load balancing and failover policies - configure VM with 2 vNICs to eliminate physical NIC problems. Check
esxtop using the n option (for networking) to see which pNIC the virtual machine is using. Try shutting down
the ports on the physical switch one at at time to determine where the virtual machine is losing network
connectivity.
Check the vNIC's connection - check the status of the vNIC, (connected/disconnected) at the VM level AND
also the NIC inside of the Guest OS (activated/deactivated).

Check more in this KB: Troubleshooting virtual machine network connection issues (1003893)

V ERIFY

A GIVEN VIRTUAL MACHINE IS CONFIGURED WITH THE CORRECT NETWORK RESOURCES

I've invoked few areas already above. All or most of the possible problems can be found in this KB - KB 1003893

TROUBLESHOOT

VIRTUAL SWITCH AND PORT GROUP CONFIGURATION ISSUES

141

Same name for port groups - Make sure that the Port Group name(s) associated with the virtual machine's
network adapter(s) exists in your vSwitch or Virtual Distributed Switch and is spelled correctly. Usually if this
isn't done right on per-port group then you have connectivity problems

VLANs - check VLANS on each standard switch

TROUBLESHOOT

PHYSICAL NETWORK ADAPTER CONFIGURATION ISSUES

Physical switch config is usually simple if "trunking" ports are used. Perhaps some of the issues might be if vNICs are
not set to automatic (default) but fixed network speed, which do not match the speed of the physical switch... I
doubt it...
If beacon probing is used, make sure that you have more than 2 pNICs in the team....
VMware KBs:

1005577 - What is beacon probing?

1004048 - Sample configuration of EtherChannel / Link Aggregation Control Protocol (LACP) with ESXi/ESX and
Cisco/HP switches (1004048)
1001938 - Host requirements for link aggregation for ESXi and ESX

TROUBLESHOOT VMFS

METADATA CONSISTENCY

There is a VMware KB which explains what to do if:

You have problems accessing certain files on a VMFS datastore.

You cannot modify or erase files on a VMFS datastore.
Attempting to read files on a VMFS datastore may fail with the error:

invalid argument
You can run file system metadata check by using VOMA.
Check it out - Using vSphere On-disk Metadata Analyzer (VOMA) to check VMFS metadata consistency (2036767)
Quote:

To perform a VOMA check on a VMFS datastore and send the results to a specific log file, the command
syntax is:
voma -m vmfs -d /vmfs/devices/disks/naa.00000000000000000000000000:1 -s /tmp/analysis.txt
where naa.00000000000000000000000000:1 is replaced with the LUN NAA ID and partition to be checked. Note the
":1" at the end. This is the partition number containing the datastore and must be specified. See note below. As an
advisory, if you run voma more than once, add the NAA ID and a time stamp to the output log file name. EG: -s
/tmp/naa.00000000000000000000000000:1_analysis_<<hhmm>>.txt
Note: VOMA must be run against the partition and not the device.

IDENTIFY S TORAGE I/O

CONSTRAINTS

Again, Good KB article to check - VMware KB 1008205.

Per LUN basis - To monitor storage performance on a per-LUN basis:

Start esxtop > Press u to switch to disk view (LUN mode).

142

Press f to modify the fields that are displayed.

Press b, c, f, and h to toggle the fields and press Enter.
Press s and then 2 to alter the update time to every 2 seconds and press Enter.

Per HBA - To monitor storage performance on a per-HBA basis:

Start esxtop by typing esxtop > Press d to switch to disk view (HBA mode).
To view the entire Device name, press SHIFT + L and enter 36 in Change the name field size.
Press f to modify the fields that are displayed.
Press b, c, d, e, h, and j to toggle the fields and press Enter.
Press s and then 2 to alter the update time to every 2 seconds and press Enter.

Then the metrics to check out:

GAVG, DAVG, KAVG - latency stats.

You should check this community thread from which I quote the main part because I think that it's a very good work
done by the community:
Latency values are reported for all IOs, read IOs and all write IOs. All values are averages over the measurement
interval.
All IOs: KAVG/cmd, DAVG/cmd, GAVG/cmd, QAVG/cmd
Read IOs: KAVG/rd, DAVG/rd, GAVG/rd, QAVG/rd
Write IOs: KAVG/wr, DAVG/wr, GAVG/wr, QAVG/wr
GAVG - This is the round-trip latency that the guest sees for all IO requests sent to the virtual storage device. GAVG
should be close to the R metric in the figure.
Q: What is the relationship between GAVG, KAVG and DAVG?
A: GAVG = KAVG + DAVG
KAVG - These counters track the latencies due to the ESX Kernel's command.
The KAVG value should be very small in comparison to the DAVG value and should be close to zero. When there is a
lot of queuing in ESX, KAVG can be as high, or even higher than DAVG. If this happens, please check the queue
statistics, which will be discussed next.
DAVG - This is the latency seen at the device driver level. It includes the roundtrip time between the HBA and the
storage.
DAVG is a good indicator of performance of the backend storage. If IO latencies are suspected to be causing
performance problems, DAVG should be examined. Compare IO latencies with corresponding data from the storage
array. If they are close, check the array for misconfiguration or faults. If not, compare DAVG with corresponding data
from points in between the array and the ESX Server, e.g., FC switches. If this intermediate data also matches DAVG
values, it is likely that the storage is under-configured for the application. Adding disk spindles or changing the RAID
level may help in such cases.
QAVG - The average queue latency. QAVG is part of KAVG.

143

M ONITOR /TROUBLESHOOT S TORAGE DISTRIBUTED R ESOURCE S CHEDULER (SDRS)

ISSUES

Even when Storage DRS is enabled for a datastore cluster, it might be disabled on some virtual disks in the datastore
cluster.
Check the vSphere, ESXi and vCenter server troubleshooting guide p.47 and p.52.
Scenarios like the one below are invoked there:
Storage DRS generates an alarm to indicate that it cannot operate on the datastore.
Problem - Storage DRS generates an event and an alarm and Storage DRS cannot operate.
Cause - The following scenarios can cause vCenter Server to disable Storage DRS for a datastore.

The datastore is shared across multiple data centers - Storage DRS is not supported on datastores that are
shared
across
multiple
data
centers.
This
configuration can occur when a host in one data center mounts a datastore in another data center, or
when a host using the datastore is moved to a different data center. When a datastore is shared across
multiple data centers, Storage DRS I/O load balancing is disabled for the entire datastore cluster.
However, Storage DRS space balancing remains active for all datastores in the datastore cluster that are
not shared across data centers.
The datastore is connected to an unsupported host - Storage DRS is not supported on ESX/ESXi 4.1 and earlier
hosts.
The datastore is connected to a host that is not running Storage I/O Control. The datastore must be visible
in
only
one
data
center.
Move
the
hosts
to
the
same
data
center
or
unmount the datastore from hosts that reside in other data centers.
Ensure that all hosts associated with the datastore cluster are ESXi 5.0 or later.
Ensure that all hosts associated with the datastore cluster have Storage I/O Control enabled.

Tools

vSphere Networking Guide

vSphere Storage Guide
vSphere Troubleshooting Guide
vSphere Server and Host Management Guide
vSphere Client / vSphere Web Client

VCP6-DCV OBJECTIVE 7.3 - TROUBLESHOOT VSPHERE UPGRADES

In today's Objective we'll discuss VCP6-DCV Objective 7.3 - Troubleshoot vSphere Upgrades. You can check the
whole VCP6-DCV Study Guide page for all topics there. You can also check the vSphere 6 page where youll find many
how-to, videos, and tutorials about vSphere 6.
VCP6-DCV exam validates you have the skills required to successfully install, deploy, scale and manage VMware
vSphere 6. If someone asks you to activate trivial logging you must know how to do it and where.... And this also is
part of today's Objective for the VCP6 exam. Note that Trivia logging (Extended verbose) - Displays information,
error, warning, verbose, and trivia log entries....
vSphere Knowledge:

Identify vCenter Server and vCenter Server Appliance Upgrade Issues

144

Create a Log Bundle

Locate/Analyze VMware Log Bundles
Identify Alternative Methods to Upgrade ESXi Hosts in Event of Failure
Configure vCenter Logging Options

Tools:

VMware Documentations and KB

IDENTIFY V C ENTER S ERVER

AND V C ENTER

S ERVER A PPLIANCE UPGRADE ISSUES

First thing to do is to check logs:

Check logs for vCenter server or ESXi- Collecting logs for ESXi and vCenter via Web Client - VMware KB Article
2032892.... or VMware KB Article 1011641 for vCenter.
Create a log bundle.
Collect logs via vSphere Client - VMware KB Article 653

Blog posts from the lab, which gives you step-by-step to follow...

ESXi 5.5 upgrade to 6.0 via VMware Online Repository Plus few other CLI commands
ESXi Offline Bundle Download To Upgrade ESXi Free (Internet connection is necessary) [Guide]
Patch ESXi 5.5 to ESXi 6.0 Lab Time (via vSphere Upgrade bundle OR via ISO) [Guide]
Upgrade ESXi with VMware Update Manager (VUM) [Guide] Needs to install VUM first.
How to Upgrade from VCSA 5.5 to 6.0 Lab Time [Guide]

CREATE A LOG B UNDLE ( VIA W EB CLIENT )

LOCATE/A NALYZE VM WARE LOG B UNDLES
To collect ESX/ESXi and vCenter Server diagnostic data:
1.
2.
3.
4.
5.

Start the vSphere Web Client and log in to the vCenter Server system.
Under Inventory Lists, select vCenter Servers.
Click the vCenter Server that contains the ESX/ESXi hosts from which you want to export logs.
Click the Monitor tab and click System Logs.
Click Export System Logs.

1.
2.
3.
4.
5.

Select the ESX/ESXi hosts from which you want to export logs.
Select the Include vCenter Server and vSphere Web Client logs option. This step is optional.
Click Next.
Select the system logs that are to be exported.
Select Gather performance data to include performance data information in the log files.Note: You can
update the duration and interval time between which you want to collect the data.
6. Click Next.
7. Click Generate Log Bundle. The Download Log Bundles dialog appears when the Generating Diagnostic Bundle
task completes.
145

1. Click Download Log Bundle to save it to your local computer.Note: The host or vCenter Server
generates .zip bundles containing the log files. The Recent Tasks panel shows the Generate diagnostic bundles
task in progress.

TO

EXPORT THE EVENTS LOG :

1.
2.
3.
4.
5.

Select an inventory object.

Click the Monitor tab, and click Events.
Click the Export icon.
In the Export Events window, specify what types of event information you want to export.
Click Generate CSV Report, and click Save.

TO RUN A VM- SUPPORT IN A CONSOLE SESSION

Open an SSH session via putty for examle and run the following command:

vm-support
As a result..

146

A compressed bundle of logs is produced and stored in a file with a .tgz extension in one of these locations:

/var/tmp/
/var/log/

The current working directory

To export the log bundle to a shared vmfs datastore, use this command: vm-support
/vmfs/volumes/DATASTORE_NAME

-f

-w

More VMware KBs...

Using vm-support command line tool (VMware KB 1010705, Collecting Diagnostic Information Using the vmsupport Command in VMware ESX/ESXi)
How-to obtain vCenter Server Log Bundles (VMware KB 1011641, Collecting Diagnostic Information for
VMware vCenter Server)
By Using PowerCLI (VMware KB 1027932, Collecting Diagnostic Information for VMware vCenter Server and
ESX/ESXi Using the vSphere PowerCLI)
How-to obtain vCenter Server and ESXi Log Bundles (VMware KB 653, Collecting Diagnostic Information for
Vmware ESX/ESXi Using the vSphere Client)

IDENTIFY A LTERNATIVE M ETHODS

TO

UPGRADE ESX I H OSTS

IN

E VENT

OF

F AILURE

There is quite a few methods to upgrade ESXi.

Via VUM - vsphere update manager. I've done the step-by-step in the lab.
Via Scripted upgrade - not my prefered. Check the steps here in the VMware documentation.
vSphere Auto Deploy - via autodeploy you can provision a host with new image profile which would contain
the ESXi upgrade to 6.0. It would be necessary to use Image builder. You can check VCP6-DCV Autodeoploy
Objective here.
ESXCLI - well know for free ESXi. And easy to do.
Interactive Upgrade - And old fashion method, but easy. By booting the CD. You'll need to burn a CD first with
the ISO image. Step-by-step here.

CONFIGURE V CENTER L OGGING O PTIONS

Not often used but it's on the blueprint! You might need to change the logging settings when implementing a
monitoring solution too...
vSphere web client > vCenter Inventory Lists > vCenter servers, click vCenter > Manage TAB > Settings > General >
Edit > Logging Settings

147

The options are:

N OTES

None (Disable logging) - Turns off logging

Error (Errors only) - Displays only error log entries
Warning (Errors and warnings) - Displays warning and error log entries
Info (Normal logging) - Displays information, error, and warning log entries
Verbose (Verbose) - Displays information, error, warning, and verbose log entries
Trivia (Extended verbose) - Displays information, error, warning, verbose, and trivia log entries
AND REMARKS ...

Info about SQL SQL 2012 Enterprise SP1 and SQL 2008 Standard R2 SP1 are supported as upgrade option...

vSphere 6 page on ESX Virtualization - how to, videos, step-by-steps

Enabling trivia logging in VMware vCenter Server - VMware KB1001584
Important Information before upgrading to vSphere 6 (KB 2110293)
Upgrading to vCenter Server 6.0 best practices (KB 2109772)
List of recommended topologies for vSphere 6.0.x (KB 2108548)
Update sequence for vSphere 6.0 and its compatible VMware products (KB 2109760)
Methods for upgrading to VMware ESXi 6.0 (KB 2109711)
vSphere 6.0 is here! KBs you need to know about (link)
Release Notes here here is a third-party interpretation (part 1 and part 2) of the release notes that is a long
read but has some good points.
Whats new here
For those home lab guys like myself who sometimes cut corners you will need to check this out to learn more
about dropped supported hardware.
You can find out the status of your backup software here, or at the vendor too of course. I use Veeam and I
know that it will break with this upgrade and I will need to wait a bit for it to work!

148

VCP6-DCV OBJECTIVE 7.4 - TROUBLESHOOT AND MONITOR VSPHERE PERFORMANCE

In today's Objective we'll discuss VCP6-DCV Objective 7.4 - Troubleshoot and Monitor vSphere Performance. You
can check the whole VCP6-DCV Study Guide page for all topics there.
You can also check the vSphere 6 page where youll find many how-to, videos, and tutorials about vSphere 6.
Performance is a key to everything. When your application is slow, you must pinpoint many values to find out what's
going on at your virtual infrastructure.
If it's the underlying VM which is experiencing problems (wrong sizing of CPU, Memory, Disk...) or is it the underlying
storage system, network or physical CPU of the host. Quite complex to find out what's going on.
vSphere Knowledge

Describe how Tasks and Events are viewed in vCenter Server

Identify critical performance metrics
Explain common memory metrics
Explain common CPU metrics
Explain common network metrics
Explain common storage metrics
Identify CPU/Memory contention issues
Identify Host Power Management Policy
Monitor performance through esxtop
Troubleshoot Enhanced vMotion Compatibility (EVC) issues
Troubleshoot virtual machine performance via vRealize Operations
Compare and contrast Overview and Advanced Charts

DESCRIBE

HOW

TASKS

AND

E VENTS

ARE VIEWED IN V C ENTER

S ERVER

TASKS
You can view tasks that are associated with a single object or all objects in the vSphere Client inventory. The Tasks &
Events tab lists completed tasks and tasks that are currently running. By default, the tasks list for an object also
includes tasks performed on its child objects. You can filter the list by removing tasks performed on child objects and
by using keywords to search for tasks.

Select Host, VM, Datastore or network TAB > Below, Select object on the left > Monitor TAB > Tasks.
You can also select cluster, datacenter or vCenter object to see the tasks...

149

EVENTS
The same for events. Example showing the events at the cluster level. Again, you can choose another object like
host, datastore, VM....

IDENTIFY

CRITICAL PERFORMANCE METRICS

Performance metrics are organized into logical groups based on the object or object device. Statistics for one or
more metrics can be displayed in a chart through vSphere client or web client. Most important and common metrics
are CPU, memory, storage and network.

E XPLAIN

COMMON MEMORY METRICS

Memory overhead - this metrics shows how much memory is necessary for the ESXi to be able to run a VM
workload.
Active guest memory - is amount of memory that VMkernel thinks that it has been used by VM actively.
Host memory (consumed) - amount of memory allocated to a VM
Host memory (overhead) - is amount consumed for the virtualization overhead to run this particular VM.
Avg Memory Usage in KB - similar to Average CPU Usage, this should be reported at both Host and Guest levels. It
can give you an indication in terms of who is using the most memory but high usage does not necessarily indicate a
bottleneck. If memory usage is high, check the values for Memory Ballooning/Swapping.
Balloon (KB) - MCTL - Host cannot meet its memory requirements, so there is a memory pressure on the host. The
Balloon driver is installed via VMware Tools onto Windows and Linux guests and its job is to force the operating
system, of lightly used guests, to page out unused memory back to ESX so it can grand more memory to other VMs.
Swap Used KB - if you see values being reported at the Host for Swap, this indicates that memory demands cannot
be satisfied and processes are swapped out to the vSwp file. This is going bad as swapping is the last resort for the
hypervisor to manage the memory at some point... Consider vMotioning some VMs out of this host or plan to add
more physical RAM....
Consumed - Consumed memory is the amount of Memory Granted on a Host to its guests minus the amount of
Memory Shared across them. Memory can be over-allocated, unlike CPU, by sharing common memory pages such as
Operating System pages. This metric displays how much Host Physical Memory is actually being used (or consumed)
and includes usage values for the Service Console and VMkernel.
Active - this metric reports the amount of physical memory recently used by the guests on the Host and is displayed
as Guest Memory Usage in vCenter at Guest level.
From vSphere Monitoring and Performance guide p. 136

150

SWR/s (MB) - Rate at which the ESXi host swaps in memory from disk for the resource pool or virtual machine.
SWW/s (MB) - Rate at which the ESXi host swaps resource pool or virtual machine memory to disk.
SWCUR (MB) - Current swap usage by this resource pool or virtual machine.
SWTGT (MB) - Target where the ESXi host expects the swap usage by the resource pool or virtual machine
to be
MCTL? - Check if the memory balloon driver is installed or not. N means no, Y means yes.
MCTLSZ (MB) - Amount of physical memory reclaimed from the resource pool by way of ballooning.
MCTLTGT (MB) - Amount of physical memory the ESXi system attempts to reclaim from the resource pool or
virtual machine by way of ballooning.
MCTLMAX (MB) - Maximum amount of physical memory the ESXi system can reclaim from the resource pool
or virtual machine by way of ballooning. This maximum depends on the guest operating system type.

E XPLAIN

COMMON

CPU

METRICS

vSphere Monitoring and Performance guide p. 131

%USED - Percentage of physical CPU core cycles used by the resource pool, virtual machine, or world. %USED
might depend on the frequency with which the CPU core is running. When running with lower CPU core
frequency, %USED can be smaller than %RUN. On CPUs which support turbo mode, CPU frequency can also
be higher than the nominal (rated) frequency, and %USED can be larger than %RUN.
%USED = %RUN +
%SYS - %OVRLP
%RDY - Percentage of time the resource pool, virtual machine, or world was ready to run, but was not provided
CPU resources on which to execute. 100% = %RUN + %RDY + %CSTP + %WAIT
%CSTP - Percentage of time a resource pool spends in a ready, co-deschedule state. NOTE You might see this
statistic displayed, but it is intended for VMware use only.
100% = %RUN + %RDY + %CSTP + %WAIT

%SYS - Percentage of time spent in the ESXi VMkernel on behalf of the resource pool, virtual machine, or world
to process interrupts and to perform other system activities. This time is part of the time used to calculate
%USED. %USED = %RUN + %SYS - %OVRLP

%WAIT - Percentage of time the resource pool, virtual machine, or world spent in the blocked or busy wait
state. This percentage includes the percentage of time the resource pool, virtual machine, or world was
idle. 100% = %RUN + %RDY + %CSTP + %WAIT

E XPLAIN

COMMON NETWORK METRICS

vSphere Monitoring and Performance guide p 141.

MbTX/s -MegaBits transmitted per second.

MbRX/s -MegaBits received per second.

Dropped packed metrics:

%DRPTX - Percentage of transmit packets dropped

%DRPRX - Percentage of receive packets dropped.

E XPLAIN

COMMON STORAGE METRICS

Latency, latency, latency...

GAVG (Guest Average Latency) total latency as seen from vSphere

151

KAVG (Kernel Average Latency) time an I/O request spent waiting inside the vSphere storage stack.
QAVG (Queue Average latency) time spent waiting in a queue inside the vSphere Storage Stack.
DAVG (Device Average Latency) latency coming from the physical hardware, HBA and Storage device.

IDENTIFY CPU/M EMORY CONTENTION ISSUES

IDENTIFY H OST P OWER M ANAGEMENT P OLICY

High Performance - This power policy maximizes performance, using no power management features. It keeps
CPUs in the highest P-state at all times. It uses only the top two C-states (running and halted), not any of the
deep states (for example, C3 and C6 on the latest Intel processors).
Balanced - This power policy is designed to reduce host power consumption while having little or no impact
on performance. The balanced policy uses an algorithm that exploits the processors P-states. Balanced is the
default power policy for ESXi.
Low Power - This power policy is designed to more aggressively reduce host power consumption, through the
use of deep C-states, at the risk of reduced performance.
Custom - This power policy starts out the same as balanced, but it allows individual parameters to be modified.
If the host hardware does not allow the operating system to manage power, only the Not Supported policy is
available. (On some systems, only the High Performance policy is available.)

152

M ONITOR

PERFORMANCE THROUGH ESXTOP

Check this community thread ESXTOP. It's excellent!

TROUBLESHOOT E NHANCED V M OTION C OMPATIBILITY (EVC)

ISSUES

From this VMware KB - EVC and CPU Compatibility FAQ you can learn that:
EVC is short for Enhanced vMotion Compatibility. EVC allows you to migrate virtual machines between different
generations of CPUs. with EVC you can mix older and newer server generations in the same cluster and be able to
migrate virtual machines with vMotion between these hosts. This makes adding new hardware into your existing
infrastructure easier and helps extend the value of your existing hosts.

ESXi 6.0 supports these EVC modes:

AMD Opteron Generation 1 (Rev. E)

AMD Opteron Generation 2 (Rev. F)
AMD Opteron Generation 3 (Greyhound)
AMD Opteron Generation 3 (no 3Dnow!) (Greyhound)
AMD Opteron Generation 4 (Bulldozer)
AMD Opteron "Piledriver" Generation
Intel "Merom" Generation (Intel Xeon Core 2)
Intel "Penryn" Generation (Intel Xeon 45nm Core2)
Intel "Nehalem" Generation (Intel Xeon Core i7)
Intel "Westmere" Generation (Intel Xeon 32nm Core i7)
Intel "Sandy Bridge" Generation
Intel "Ivy Bridge" Generation
Intel "Haswell" Generation

TROUBLESHOOT

VIRTUAL MACHINE PERFORMANCE VIA V R EALIZE

O PERATIONS

vROPs is a separate vSphere product and needs really deep understanding on what's going on. I think that there
should be a separate chapter on the blueprint if required for the exam....
The architecture has changed as well (there is no more UI VM and Analytics VM like int he vCOPS 5.8). The appliance
works in cluster, and from within the dashboard youll be able to deploy/add an additional appliance (node) to the
system to scale out. The solution is highly resilient, by using Gemfire to spread the data across at least 2 nodes. Two
slices has the copy of the data (at least). If there is a failure of one of the slices, then another slice takes over.

153

V R EALIZE

O PERATION M ANAGEMENT S UITE 6.0 N EW

AND IMPROVED FEATURES

Increased Scale of a single deployment

Cluster shared data and UI
Resiliency (application RAID!)
Smart alerts with problem-definitions
Customizable Dashboards and Reports (drag and drop to create new)
Advanced capacity modeling via possibility to save capacity project and do a what-if analyse.
Public APIs released to partners in order to work on additional extensibilities.

Newly the product will feature a management pack integration (add-ons) which will be delivered by VMware and
partners for specific storage devices. There is 40-50 management packs available on the VMware Solution Exchange
and those management packs can be installed inside the vRealize Management Operation. (vROPS)
From the overview dashboard you can see which problems arise or will arise (in the Risk alerts section). By clicking
the link you can drill down to see the problem.

COMPARE

AND CONTRAST

O VERVIEW

AND

A DVANCED CHARTS

OVERVIEW CHARTS
vSphere Performance guide p14. Display multiple data sets in one panel to easily evaluate different resource
statistics, display thumbnail charts for child objects, and display charts for a parent and a child object. Advanced
charts display more information than overview charts, are configurable, and can be printed or exported.

154

Overview chart from my lab. Select Host > Monitor TAB > Performance > Drop down chose between Home or
Virtual Machines.

A DVANCED CHARS
Use advanced charts, or create your own custom charts, to see more performance data. Advanced charts can be useful
when you are aware of a problem but need more statistical data to pinpoint the source of the trouble.

Slect Host > Monitor TAB > Performance > Click Advanced

Advanced charts include the following features:

More information. Hover over a data point in a chart and details about that specific data point are displayed.
Customizable charts. Change chart settings. Save custom settings to create your own charts.
Export to spreadsheet.
Save to image file or spreadsheet

155

Tools
o
o
o
o
o

vSphere Resource Management Guide

vSphere Troubleshooting Guide
vSphere Monitoring and Performance Guide
vCenter Operations Manager Getting Started Guide (vSphere UI)
vSphere Client / vSphere Web Client

VCP6-DCV OBJECTIVE 7.5 - TROUBLESHOOT HA AND DRS CONFIGURATIONS AND FAULT

TOLERANCE
Today's VCP6 topic is following: VCP6-DCV Objective 7.5 - Troubleshoot HA and DRS Configurations and Fault
Tolerance. A large topic, which is difficult to fit into single post.
The VCP6-DCV certification exam validates that you have the skills required to successfully install, deploy, scale and
manage VMware vSphere 6 environments.
Check the VCP6-DCV Study Guide [Unofficial] page on my blog for all topics required to pass the exam. Stay tuned
for the PDF version .... Check also other How-to articles, videos, and news concerning vSphere 6 - dedicated vSphere
6 page.
vSphere Knowledge

Identify HA/DRS and vMotion requirements

Verify vMotion/Storage vMotion configuration
Verify HA network configuration
Verify HA/DRS cluster configuration
Troubleshoot HA capacity issues
Troubleshoot HA redundancy issues
Interpret the DRS Resource Distribution Graph and Target/Current Host Load Deviation
Troubleshoot DRS load imbalance issues
Troubleshoot vMotion/Storage vMotion migration issues
Interpret vMotion Resource Maps
Identify the root cause of a DRS/HA cluster or migration issue based on troubleshooting information
Verify Fault Tolerance configuration
Identify Fault Tolerance requirements

IDENTIFY HA/DRS

AND VM OTION REQUIREMENTS

vSphere HA is very easy to set up and manage and is the simplest high-availability solution available for
protecting virtual workloads.
HA R EQUIREMENTS :

Redundant Management Network - Verify that you are using redundant management network connections
for vSphere HA. For information about setting up network redundancy, see Best Practices for Networking.
Proper Licensing - vSphere Essentials Plus and higher licensing. Essentials (only) won't do the job...
Minimum 2 hosts in a cluster - HA needs 2 hosts to be able to initiate failover.
Static IP config - Host which participate in HA/DRS clusters has to be configured with static IP address.
156

Shared Storage - VMs must run on shared storage

Access All hosts to VM neworks and datastores - All Hosts shall be able to reach the VM's networks and
datastores.
VMware tools on VMs - All VMs has to have VMware tools in stalled in order to be able to activate VM
Monitoring
Configure Two Shared Datastores at least - to have redundancy for vSphere HA datastore hearbeating.
ipv6 and ipv4 are supported - vSphere HA supports both IPv4 and IPv6. See Other vSphere HA
Interoperability Issues, on page 31 for considerations when using IPv6.
Enable APD Timeout - If you want to use VM Component Protection, hosts must have the All Paths Down
(APD) Timeout feature enabled.
Wants VMCP with HA? - To use VM Component Protection, clusters must contain ESXi 6.0 hosts or later.

DRS R EQUIREMENTS :
vCenter server resource management p.63

Shared storage - SAN/NAS, VSAN... any supported shared storage.

Configure all managed hosts to use shared VMFS volumes. Place the disks of all virtual machines on VMFS
volumes that are accessible by source and destination hosts. Make sure that the VMFS volume is sufficiently
large to store all virtual disks for your virtual machines and also make sure that all VMFS volumes on source
and destination hosts use volume names, and all virtual machines use those volume names for specifying the
virtual disks.
CPU Requirements - use EVC to help you out with different hardware in your cluster.

V M OTION

R EQUIREMENTS :

Gigabit ethernet for vMotion is a bare minimum - make sure you comply with that
No RDM or MSCS support -Microsoft Cluster service (MSCS) isn't supported.

157

VMs with CDROM Unattached - Cannot vMotion a VM that is backed by a device that isn't accessible to the
target host. I.E. A CDROM connected to local storage on a host. You must disconnect these devices first. USB
is supported as long as the device is enabled for vMotion
For VMs with USB - must enable all USB devices that are connected to the virtual machine from a host for
vMotion. If one or more devices are not enabled for vMotion, migration
will fail.
TCP port 8000 - incoming and outgoing firewall port for ESXi hosts, this is a required port for vMotion.

V ERIFY V M OTION/S TORAGE V M OTION

CONFIGURATION

Check the vmkernel network interfaces for the correct network config.
Make sure that the EVC in the cluster is configured (if needed) and tested prior enabling DRS.
Make sure that all hosts within cluster can reach the shared storage and no VMs are left on local storage
somewhere....

V ERIFY HA

NETWORK CONFIGURATION

Check this section at the vSphere Availability Guide p.29 and p.39

When you change the networking configuration on the ESXi hosts themselves, for example, adding port
groups, or removing vSwitches, suspend Host Monitoring. After you have made the networking configuration
changes, you must reconfigure vSphere HA on all hosts in the cluster, which causes the network information
to be reinspected. Then re-enable Host Monitoring.

On ESXi hosts in the cluster, vSphere HA communications, by default, travel over VMkernel networks. With an ESXi
host, if you wish to use a network other than the one vCenter Server uses to communicate with the host for vSphere
HA, you must explicitly enable the Management traffic check-box.
Der, Die, Das! Isolation Address
das.isolationaddress
By default, the network isolation address is the default gateway for the host. Only one default gateway is specified,
regardless of how many management networks have been defined. You should use the das.isolationaddress[...]
advanced option to add isolation addresses for additional networks.
This address is pinged only when heartbeats are not received from any other host in the cluster. If not specified, the
default gateway of the management network is used. This default gateway has to be a reliable address that is
available, so that the host can determine if it is isolated from the network. You can specify multiple isolation
addresses (up to 10) for the cluster:

158

das.isolationaddressX, where X = 0-9.

Typically you should specify one per management network. Specifying too many addresses makes isolation detection
take too long.
Check p.37 for all advanced options.

V ERIFY HA/DRS

CLUSTER CONFIGURATION

You can check the cluster summary through vSphere client or vSphere web client.

vSphere client...

TROUBLESHOOT HA

CAPACITY ISSUES

As you know the 3 possible HA admission config policies you must know are:

159

Host Failures Cluster Tolerates - With the Host Failures Cluster Tolerates admission control policy, VMware
HA ensures that a specified number of hosts can fail and sufficient resources remain in the cluster to fail over
all the virtual machines from those hosts
Percentage of Cluster Resources - You can configure VMware HA to perform admission control by reserving a
specific percentage of cluster resources for recovery from host failures. With the Percentage of Cluster
Resources Reserved admission control policy, VMware HA ensures that a specified percentage of aggregate
cluster resources is reserved for failover.
Specify a Failover Host - when a host fails, VMware HA attempts to restart its virtual machines on a specified
failover host. If this is not possible, for example the failover host itself has failed or it has insufficient resources,
then VMware HA attempts to restart those virtual machines on other hosts in the cluster.

The three HA admission configuration policies...

What can go wrong? Hosts disconnected, unconfigured (right click > reconfigure for HA). Also when (if) setting "specify
failover host" policy, than you might end up with some VMs non restarted if several hosts fails, as you did not set
enough hosts for failover. I usually use "percentage of cluster resources" or "host failures cluster tolerates" policies.
If your cluster contains any virtual machines that have much larger reservations than the others, they will distort slot
size calculation. To avoid this, you can specify an upper bound for the CPU or memory component of the slot size by
using the das.slotcpuinmhz or das.slotmeminmb advanced attributes, respectively.
Slot size is comprised of two components, CPU and memory.

vSphere HA calculates the CPU component by obtaining the CPU reservation of each powered-on virtual
machine and selecting the largest value. If you have not specified a CPU reservation for a virtual machine, it is
assigned a default value of 32MHz. You can change this value by using the das.vmcpuminmhz advanced
attribute.)
vSphere HA calculates the memory component by obtaining the memory reservation, plus memory overhead,
of each powered-on virtual machine and selecting the largest value. There is no default value for the memory
reservation.

If large VMs present in the cluster than you might want to use "percentage of cluster resources" admission policy as
you won't need to deal with slot sizes.

TROUBLESHOOT HA

REDUNDANCY ISSUES

160

NIC teaming is the answer. Redundancy, redundancy.... Use 2 or more pNICs in a team to provide failover possibility.
If possible use separate physical switches to provide redundancy.

INTERPRET THE DRS R ESOURCE DISTRIBUTION G RAPH

DEVIATION

AND

TARGET /CURRENT H OST LOAD

Even if VMware is pushing the web client, I feel that the C# client shows more details when flying over with a mouse
on a chart to display the memory utilization of a host within cluster, you can actually see an individual VM, how
such a VM consumes memory on that particular host...
You can access the charts (in vSphere client) from the summary tab when selecting your cluster on the left hand side
first. Click the "View resource distribution chart" link, as on the image below....

161

This is not the case of vSphere Web client....

The DRS Resource Distribution chart displays CPU or Memory metrics for each of the hosts in the cluster. YOu can
switch from percentage to mebabytes (for memory) resp from percentage to megaherty (for CPU).
DRS cluster is load balanced when each of its hosts level of consumed resources is equivalent to the others. When
they arent, the cluster is considered to be imbalanced and VMs must be relocated to restore the balance.

TROUBLESHOOT DRS

LOAD IMBALANCE ISSUES

Imbalanced load issues can happens if:

Host is in maintenance mode

VM-host affinity/anti-affinity rules being used
VM-VM affinity rules being used

A cluster might become unbalanced because of uneven resource demands from virtual machines and unequal
capacities of hosts.

The migration threshold is too high - A higher threshold makes the cluster a more likely candidate for load
imbalance.
Affinity/Anti-Affinity Rules - VM/VM or VM/Host DRS rules prevent virtual machines from being moved.
Disabled DRS - DRS is disabled for some VMs...
A device is mounted to one or more virtual machines preventing DRS from moving the virtual machine in order
to balance the load.
Virtual machines are not compatible with the hosts to which DRS would move them. That is, at least one of
the hosts in the cluster is incompatible for the virtual machines that would be migrated. For example, if host
A's CPU is not vMotion-compatible with host B's CPU, then host A becomes incompatible for powered-on
virtual machines running on host B.
It would be more detrimental for the virtual machine's performance to move it than for it to run where it is
currently located. This may occur when loads are unstable or the migration cost is high compared to the
benefit gained from moving the virtual machine.
Unconfigured/disabled vMotion - vMotion is not enabled or set up for the hosts in the cluster.

162

TROUBLESHOOT V M OTION/S TORAGE V M OTION

MIGRATION ISSUES

First, check requirements for vMotion/sVMotion.

VMware tools status - Make sure that VMtools installaiton is not "stuck" in a VM...as during installation of
VMware tools it's not possible to do a VMotion of such a VM due to hearbeats.
Source destination datastores are available - make sure that this apply...
Licensing - sVMotion requires vSphere "standard"licensing...
If RDM is used in physical compatibility mode - no sVMotion or snapshoting of VMs... Virtual machine
snapshots are available for RDMs with virtual compatibility mode only.Physical Compatibility Mode VMkernel passes all SCSI commands to the device, with one exception: the REPORT LUNs command is
virtualized so that the VMkernel can isolate the LUN to the owning virtual machine. If not, all physical
characteristics of the underlying hardware are exposed. It does allows the guest operating system to access
the hardware directly. VM with physical compatibility RDM has limits like that you cannot clone such a VM or
turn it into a template. Also sVMotion or cold migration is not possible.

A quick quote from VMware blog post, which is new (note that sVMotion do not work with such a disks):
In vSphere 6.0, you can configure two or more VMs running Windows Server Failover Clustering (or MSCS for preWindows 2012 OSes), using common, shared virtual disks (RDM) among them AND still be able to successfully
vMotion any of the clustered nodes without inducing failure in WSFC or the clustered application. What's the bigdeal about that? Well, it is the first time VMware has ever officially supported such configuration without any thirdparty solution, formal exception, or a number of caveats. Simply put, this is now an official, out-of-the-box feature
that does not have any exception or special requirements other than the following:

The VMs must be in "Hardware 11" compatibility mode - which means that you are either creating and
running the VMs on ESXi 6.0 hosts, or you have converted your old template to Hardware 11 and deployed it
on ESXi 6.0
The disks must be connected to virtual SCSI controllers that have been configured for "Physical" SCSI Bus
Sharing mode
And the disk type *MUST* be of the "Raw Device Mapping" type.

INTERPRET V M OTION R ESOURCE M APS

A vCenter map is a visual representation of your vCenter Server topology. Maps show the relationships between the
virtual and physical resources available to vCenter Server.
Maps are available only when the vSphere Client is connected to a vCenter Server system.
The maps can help you determine such things as which clusters or hosts are most densely populated, which
networks are most critical, and which storage devices are being utilized. vCenter Server provides the
following map views.

Virtual Machine Resources - Displays virtual machine-centric relationships.

Host Resources - Displays host-centric relationships.
Datastore Resources - Displays datastore-centric relationships.
vMotion Resources - Displays hosts available for vMotion migration.

163

You can configure the maximum requested topology entities (helps for large environments) via vSphere client by
going to the Client Menu > Edit > Client settings > Maps TAB

IDENTIFY

THE ROOT CAUSE OF A

DRS/HA

CLUSTER OR MIGRATION ISSUE BASED ON

TROUBLESHOOTING INFORMATION

V ERIFY F AULT TOLERANCE

CONFIGURATION

vSphere 6 has introduced New FT with up to 4vCPU support. However if virtual machine has only a single vCPU,
however, you can use legacy FT instead, for backward compatibility. But, unless technically necessary, use of legacy
FT is not recommended.
To use legacy Fault Tolerance, you must configure an advanced option for the virtual machine. After you complete
this configuration, the legacy FT VM is different in some ways from other fault tolerant VMs.
Difference between Legacy FT (used in previous releases of vSphere) and FT (v6).

164

If you want/need to use legacy FT, check the requirements.

IDENTIFY F AULT TOLERANCE

REQUIREMENTS

Licensing - The number of vCPUs supported by a single fault tolerant VM is limited by the level of licensing that you
have purchased for vSphere. Fault Tolerance is supported as follows:

vSphere Standard and Enterprise. Allows up to 2 vCPUs

vSphere Enterprise Plus. Allows up to 4 vCPUs

10 GbE Network - hard requirement for FT v6!

CPU Requirements - CPUs that are used in host machines for fault tolerant VMs must be compatible with vSphere
vMotion or improved with Enhanced vMotion Compatibility. Also, CPUs that support Hardware MMU virtualization
(Intel EPT or AMD RVI) are required. The following CPUs are supported.

Intel Sandy Bridge or later. Avoton is not supported.

AMD Bulldozer or later.

P OSSIBLE E NFORCING

AT THE HOST LEVEL

Advanced settings:

das.maxftvmsperhost
The maximum number of fault tolerant VMs allowed on a host in the cluster. Both Primary VMs and
Secondary VMs count toward this limit. The default value is 4.
das.maxftvcpusperhost

165

The maximum number of vCPUs aggregated across all fault tolerant VMs on a host. vCPUs from both Primary VMs
and Secondary VMs count toward this limit. The default value is 8.
Tools

vSphere Resource Management Guide

vSphere Monitoring and Performance Guide
vSphere Installation and Setup Guide
vSphere Troubleshooting Guide
vSphere Availability Guide
vSphere Client / vSphere Web Client

VCP6-DCV OBJECTIVE 8.1 - DEPLOY ESX I HOSTS USING AUTODEPLOY

In today's topic we will take a look at Autodeploy - VCP6-DCV Objective 8.1 - Deploy ESXi Hosts Using Autodeploy,
which is quite large topic. Autodeploy allows provision dozens (or hundreds) physical hosts with ESXi images. It's
possible to manage large deployments where hosts are booted via network from a central Auto-deploy server. In
conjunction with host profiles it's possible to attach hosts to clusters and push a different configurations depending
on parameters like hardware vendor.
Check the VCP6-DCV Study Guide or other How-to articles, videos, and news concerning vSphere 6 at the dedicated
vSphere 6 page.

IDENTIFY ESX I A UTODEPLOY

REQUIREMENTS

There are some requirements but at the same time that there are also limitations, so make sure that any of those
limits does not actually puts a brake on your project. Before you can start to use vSphere Auto Deploy, you must
prepare your environment. You start with server setup and hardware preparation. You must register the Auto
Deploy software with the vCenter Server system that you plan to use for managing the hosts you provision, and also
install the VMware PowerCLI on a management station (or Windows based vCenter server).

Hardware requirements for ESXi 6.0 - check here.

ESXi hardware must be set to use BIOS (EFI isn't supported)
Require Ports opened between vCenter server and ESXi hosts - check here
If VLANs used, then check that works properly.
Minimum Storage - 2Gb of storage for storing ESXi images, where each of those images requires about 350Mb.
So depending of how many profiles you will use by taking that number into consideration.
Autodeploy server must use IPv4 - The PXE boot infra does not support IPv6.
Install ESXi Dump colletor - this will allow

CONFIGURE A UTODEPLOY
You must first enable the service. Go to vSphere Web Client > System Configuration > Services > Select Autodeploy >
Actions > Edit Startup Type

166

This will prompt you for the service settings:

And then make sure that you start the service!

On the vCenter Server Appliance, the Auto Deploy service by default is set to Manual (on Windows it's Disabled). If
you want the Auto Deploy service to start automatically upon OS startup, select Automatic.
CONFIGURE TFTP:
In a vSphere Web Client > Inventory list > select the vCenter Server > Manage tab > Settings > Auto Deploy.

167

Then click the Download TFTP Boot Zip to download the TFTP configuration file and unzip the file to the directory in
which your TFTP server stores files.
Install TFTP server (I usually use the Free TFTP server from Solarwinds). The installer creates a default directory
which can be changed. I changed mine to c:tftp to keep it simple. You can configure the option by going to File >
Configure menu. While there, make sure that you start the service. (Note: you can also go to Windows services to
make the TFTP service start automatically during the boot as by default it has manual start only).

Thats it for TFTP server. There is nothing else to play with and we can move on.
DHCP S ERVER O PTIONS
Next Ill show you the options you need to configure on your DHCP server. There are just two options which needs to
be configured at the scope level. When you click on the Autodeploy icon in vSphere client, youll end up on this page
where you can see some strange name of file. But this exact name will be needed for setting up options in our DHCP
server! Its the undionly.kpxe.vmw-hardwired.
So next step is to click and download the TFTP boot zip files to the c:tftp directory that we created and set up on our
TFTP server. Unzip the file into the same directory You should have a view like this:

168

Once done, we can copy this name of the file (undionly.kpxe.vmw-hardwired) as an option 67 in our DHCP server. In
my case I have Windows DHCP server which sits on my domain controller.

Now you should configure each of your ESXi host's BIOS to boot from network.

E XPLAIN P OWER CLI

CMDLETS FOR

A UTODEPLOY

Auto Deploy uses a PXE boot infrastructure in together with vSphere host profiles to provision and customize host(s).
No state is stored on the host(s) itself. But rather, the Auto Deploy server manages state information for each host.
Autodeploy server has the informations about the location of image profile and host profiles and this information is
specified in the rules that map machines to image profiles and host profile. Whe host boots up from the first time it's
vCenter server who creates a host objects and stores the information in the vCenter DB.
The whole architecture:

169

A UTO DEPLOY CMDLETS

There are many more auto deploy cmdlets than the ones Im using in this post, so here is the full list for reference:
Command

Description

Get-DeployCommand

Gives you a list of Auto Deploy cmdlets.

New-DeployRule

Creates a new rule with the specified items and patterns

Set-DeployRule

Updates an existing rule with the specified items and patterns. Rules that belong to a
working ruleset can not be updated.

Get-DeployRule

Retrieves rules as specified by an administrator.

Copy-DeployRule

Clones and updates an existing rule.

Add-DeployRule

Adds one or more rules to the working and active ruleset(s). The NoActivate parameter
can be specified to add a rule only to the working ruleset.

Remove-DeployRule

Removes one or more rules from the working and active rule set. The rule(s) can be
deleted by using the -Delete parameter.

Set-DeployRuleSet

Explicitly sets the list of rules in the working rule set.

Get-DeployRuleSet

Retrieves the current working rule set or active rule set.

Switch-ActiveDeployRuleSet Activates a rule set so that any new requests are evaluated through the rule set.

170

Get-VMHostMatchingValues

Retrieves rules matching a pattern. For example, all rules that apply to hosts can be
retrieved.

TestDeployRulesetCompliance

Checks whether items associated with a specified host are in compliance with an active
rule set.

RepairDeployRuleSetCompliance

Updates the image profile, host profile and location for each host in the vCenter Server
inventory based on the results of Test-DeployRulesetCompliance.

Apply-EsxImageProfile

Associates the specified image profile with the specified host.

Get-VMHostImageProfile

Retrieves the image profile in use by a specified host.

Repair-DeployImageCache

Command can be used if the image cache is accidentally deleted.

Get-VMHostAttributes

Returns attributes for a host that are used when the Auto Deploy server evaluates the
rules.

Stateless caching - Autodeploy does not store ESXi configuration or state on the host disk by default. Rather an
image profile defines the image that the host is provisioned with, and other host attributes are managed through
host profiles. A host that uses Auto Deploy for stateless caching has to have an access to Autodeploy server and
vCenter server. That's why the vCenter server has to be UP in order to be able to provission those hosts (SPOF???).

Stateful installs - In this case it is possible to provision a host with Auto Deploy and set up the host to store the
image to disk. On subsequent boots, the host boots from disk

DEPLOY /M ANAGE

MULTIPLE

ESX I

HOSTS USING

A UTODEPLOY

1. Install PowerCLI
2. Use the PowerCLI cmdlets to define rule which assigns an image profile and host profile (optional) to the host.
3. Configure reference host and create a host profile where you'll keep what's common for all hosts (storage,
networking and other). Write a rule that assigns not only the already tested image profile but also the host profile to
the target host.
4. If you need manual information to be entered, you can specify user input in the customization of the host within
the vSphere web client.
Getting Help with PowerCLI commandlets

Basic help: Get-Help cmdlet_name

Detailed help: Get-Help cmdlet_name -Detailed

I have done a blog post series covering host profiles, autodeploy...when learning towards VCAP exam. You
can use it as a guide for preparation for the VCP exam as most things hasn't changed...

VCAP Diary VMware vSphere ESXi Image Builder

VMware vSphere AutoDeploy Run some PowerCLI and youre the Boss
VMware vSphere AutoDeploy Install and configure
VCAP Diary VMware vSphere Host Profiles

171

VMware vSphere Host Profiles options and troubleshooting

VMware documentation and Tools

vSphere Installation and Setup Guide

vSphere Client / vSphere Web Client
Direct Console User Interface (DCUI)

Some more links:

About Reprovisioning Hosts.

Test and repair rule compliance

VCP6-DCV OBJECTIVE 8.2 - CUSTOMIZE HOST PROFILE SETTINGS

In today's Objective we'll discuss VCP6-DCV Objective 8.2 - Customize Host Profile Settings. Host profiles are feature
which is present in the vSphere enterprise plus licensing, and allows the to "uniformize" and/or push configuration
changes to all hosts in the cluster. Host profiles are necessary when using VMware vSphere Autodeploy, which takes
advantage of host profile after the stateless image is loaded in memory, to apply a configuration through that host
profile. Whats needed is also an Autodeploy installed and configured, together with DHCP options enabled for
Autodeploy to work with. But we'll look at this Objective another time.
Check the VCP6-DCV Study Guide page or other How-to articles, videos, and news concerning vSphere 6 at the
dedicated vSphere 6 page.
vSphere Knowledge

Create/Edit/Remove a Host Profile from an ESXi host

Import/Export a Host Profile
Attach/Apply a Host Profile to an ESXi host or cluster
Perform compliance scanning and remediation of an ESXi host using Host Profiles

CREATE/E DIT /R EMOVE

H OST P ROFILE

FROM AN

ESX I

HOST

Create host profile by extracting a reference host's config.

vSphere web client > Host profiles > Click the Plus sign > Select Host > Enter Name for the host profile > Next >
Finish

172

TO DELETE HOST PROFILE :

Select the host profile to delete > Actions > delete

TO E DIT H OST PROFILE :

Select the Host profile > Actions > Edit settings > Next > Edit Host profile > When done, click Finish.

Host Profiles can be also used to validate the configuration of a host by checking compliance of a host or cluster against
the Host Profile that is associated with that host or cluster.
173

IMPORT /E XPORT

H OST P ROFILE

It's possible to export host profile as a *.vpf file (VMware Profile Format) ... As you can see the administrator's
password aren't exported for security reasons.

You will be prompted to re-enter the values for the password after the profile is imported and the password is applied
to a host.
HOW TO EXPORT ?
vSphere Web Client > Host Profiles > Select Profile > Actions > Export Host Profile

A TTACH /A PPLY

H OST P ROFILE

TO AN

ESX I

HOST OR CLUSTER

That the second step after creating a host profile from reference host. You need to attach the host or cluster to the
Host Profile.
Web Client > Select Host profile > Actions > Attach/detach Hosts and Clusters

And then on this screen you can select single host or whole cluster...

174

You can update or change the user input parameters for the Host Profiles policies by customizing the host.

P ERFORM

COMPLIANCE SCANNING AND REMEDIATION OF AN

ESX I

HOST USING

H OST P ROFILES

vSphere host profiles PDF p. 12

You can confirm the compliance of a host or cluster to its attached Host Profile and determine which, if any,
configuration parameters on a host are different from those specified in the Host Profile.
HOW TO PERFORM COMPLIANCE SCANNING ?
After attaching the host/cluster to a profile you can check the compliance....

Select the host profile > click the check the compliance icon (or go to Actions > Check Host Profile compliance) .

To see more detail on compliance failures, select a Host Profile from the Objects tab for which the last compliance
check produced one or more failures. In order to see specific detail on which parameters differ between the host
that failed compliance and the Host Profile, click on the Monitor tab and select the Compliance view. Then, expand
the object hierarchy and select the failing host. The differing parameters are displayed in the Compliance window,
below the hierarchy.
R EMEDIATE A HOST
In the event of a compliance failure, use the Remediate function to apply the Host Profile settings onto the host. This
action changes all Host Profile managed parameters to the values contained in the Host Profile attached to the host.
Navigate to the Host profile > Select Monitor Tab > Click Compliance > Right click the host > Host profiles > Remediate

175

vSphere Documentation and Tools

vSphere Installation and Setup Guide

vSphere Host Profiles Guide
vSphere Client / vSphere Web Client

So another VCP6-DCV topic done. Host profiles with autodeploy are advanced enterprise features/topics which some
of you might not need every day or will never implement, especially Autodeploy as IMHO it introduces SPOF (single
point of failure) - dependent on vCenter server.
But it's just my own opinion and it's also possible to mitigate such a risk with protecting vCenter server FT. But that's
another story...

176

VCP6-DCV OBJECTIVE 8.3 - CONSOLIDATE PHYSICAL WORKLOADS USING VM WARE

CONVERTER
VCP6-DCV blueprint covers P2V chapter too. This post will cover VCP6-DCV Objective 8.3 - Consolidate Physical
Workloads using VMware Converter. VMware converter was (and still is) very popular free tool for P2V or V2V
conversions. This was the first tool I actually started to work with when I first started with datacenter virtualization.
Converting physical systems to VMs is kind of fascinating
Compared to VCP 5 it seems that for VCP6 there is more material to study and more topics to master. For whole
exam coverage I created a dedicated VCP6-DCV page. Or if youre not preparing to pass a VCP6-DCV, you might just
want to look on some how-to, news, videos about vSphere 6 check out my vSphere 6 page.
VMware Knowledge

Identify VMware Converter requirements

Convert Physical Workloads using VMware Converter
Modify server resources during conversion
Interpret and correct errors during conversion

IDENTIFY VM WARE CONVERTER

REQUIREMENTS

VMware vCenter Converter Standalone User's Guide p.17

S YSTEM R EQUIREMENTS :

Windows - Windows XP Professional (32-bit and 64-bit) SP3 and higher, 2003 srv (x32 and x64) and up to 2012
(not 2012R2 - but I think it'll get updated).
Linux - RHEL 3.x - 6.x, SUSE 9.x - 11.x, Ubuntu 10.04 LTS - 13.04 .... both x32 and 64bit versions.

177

S UPPORTED F IRMWARE I NTERFACES :

The converter standalone supports BIOS and UEFI sources and the firmware intereface is preserved (cannot convert
BIOS to UEFI). For UEFI the supported destination types are Workstation 8.0 and later or ESXi 5.0 and later or vCenter
5.0 and later.
Supported Sources:
P OWERED O N:

Remote Windows (Linux) physical machines

Local Windows physical machines
Windows VM running on Hyper-V Server
Powered On VMware VMs
Powered On Hyper-V 2012 VMs
Powered On VMs running KVM, XEN

VMWARE V C ENTER VMS :

vCenter server 4.0, 4.1, 5.0, 5.1 and 5.5

ESX 4.0 and 4.1
ESXi 4.1, 5.0, 5.1 and 5.5

VMWARE VIRTUAL M ACHINES :

VMware Workstation 7.x, 8.x, 9.x, and 10.x

VMware Fusion 3.x, 4.x, 5.x, and 6.x
VMware Player 3.x, 4.x, 5.x, and 6.x

H YPER -V S ERVER VMS

Windows Server 2003 (x86 and x64), SP1 and SP2

Windows Server 2003 (x86 and x64) R2 SP1 and SP2
Windows Server 2008 (x86 and x64) SP2
Windows Server 2008 (x64) R2 and R2 SP1
Windows 7 (except Home editions)
Windows Vista SP1 and SP2 (except Home editions)
Windows XP Professional SP2, SP3, and x64 SP2

THIRD P ARTY VM S OR SYSTEM IMAGE

Acronis, Norton Ghost, ....

S UPPORTED D ESTINATION TYPES :

VMware vCenter VMs - (ESX 4.0 and 4.1), ESXi 4.1, ESXi 4.0, 4.1, 5.0, 5.1, and 5.5, vCenter Server 4.0, 4.1, 5.0,
5.1, and 5.5
VMware Hosted VMs - VMware Workstation 7.x, 8.x, 9.x, and 10.x, VMware Fusion 3.x, 4.x, 5.x, and 6.x,
VMware Player 3.x, 4x, 5.x, and 6.x

Unsupported Sources Disk type - RAID, GPT/MBR hybrid disks.

Supported destination types - VMware vCenter Converter Standalone User's Guide p.22
178

TCP/IP AND UDP P ORT R EQUIREMENTS FOR CONVERSION

VMware vCenter Converter Standalone User's Guide p.25
P2V - Depending on where you connecting.

Converter server to standalone VM or physical system - TCP - 445, 139, 9089;UDP - 137, 138
Converter to vCenter server - TCP 443
Converter Server to ESXi - TCP 902
Powered on Source machine to ESXi - TCP 443, 902
Linux VM uses additionally port 22 (SSH)

V2V - TCP 443, 445, 139; UDP 137, 138

CONVERT P HYSICAL W ORKLOADS

USING

VM WARE CONVERTER

Before launching conversion, make sure to disable Windows firewall (or allow File and Printer Sharing). Turn off
simple sharing.
The steps to convert a physical system can be resumed like this (but this is only one of the ways that's possible.
Other ways client-server are possible as well):
1. Install VMware converter on the Window/Linux server and click Convert Machine > Powered On machine > This
local machine
2. Select Destination type > choose VMware infrastructure VM > enter vCenter credentials > Put some meaningful
name for your VM

3. Choose Cluster or host > Datastore > Virtual Machine Version > Click Next
4. Click the Advanced Link > chose the disk type of your choice (thick or thin). If you do not copy all disks and
maintain layout the volume-based cloning is used. (at the block level).

179

You can also modify other resources which the VM do not need ... like delete some unwanted NICs, Windows
services, or adjust the number of vCPUs and Memory...
By default, Converter Standalone optimizes the disk partitions alignment. Optimizing the partitionalignment improves
the performance of the destination virtual machine. (it's basically says that the process will align the VM to the LUN).
So leave the box checked...

M ODIFY

SERVER RESOURCES DURING CONVERSION

Number of concurrent tasks - It's possible to modify the number of concurrent tasks by going to Administration >
Maximum concurrent tasks. (1 to 12 concurrent tasks) But the 12 is by default and if your Converter server lacks
resources you might want to lower down a bit of number of tasks taking place at the same time.
Number of data connections per task - if you converting systems with multiple disks and volumes, it's possible to
decrease the conversion time by cloning multiple disks and volumes simultaneously. Each data transfer uses a
separate TCP connection. Check Administration > Data connections per Task.
It's possible to synchronize changes after the first conversion has finished. It's because the source machine continues
to generate data. So the delta changes can be synced and the source VM powered down...

180

INTERPRET

AND CORRECT ERRORS DURING CONVERSION

Check the following KB articles:

TIPS

Troubleshooting when vCenter Converter fails to complete a conversion of a physical or virtual machine.
Testing port connectivity with Telnet (1003487)
Best practices for using and troubleshooting VMware Converter (1004588)
Troubleshooting a virtual machine converted with VMware Converter that fails to boot with the error: STOP
0x0000007B INACCESSIBLE_BOOT_DEVICE (1006295)
Required VMware vCenter Converter 4.x/5.x ports (1010056)
Collecting diagnostic information for VMware Converter (1010633)
TCP and UDP Ports required to access VMware vCenter Server, VMware ESXi and ESX hosts, and other network
components (1012382)
VMware vCenter Converter is unable to see the disks when converting Windows operating systems (1016992)
vCenter Standalone Converter errors when an ESXi 5.x host is selected as a destination: The access to the host
resource settings is restricted. Use the management server as a destination (2012310)
AND

TRICKS

FROM

ESX V IRTUALIZATION

AND

V LADAN... -:)

How-to disable SSL in VMware vCenter Converter Standalone to speed up P2V conversions
How-to Reduce VMDK size: VMware Converter
How to use VMware Converter to Synchronize changes when P2V (or V2V)
VMware Converter Best Practices

VMware Tools and Guides

vSphere Installation and Setup Guide

VMware vCenter Converter Standalone Guide
vSphere Client / vSphere Web Client
VMware vCenter Converter Standalone Client

VCP6-DCV OBJECTIVE 9.1 - CONFIGURE ADVANCED VSPHERE HA FEATURES

VMware VCP6-DCV certification exam is kind of holy grail as it's an exam you can't fake. You have to know your
stuff. Many folks also need to re-certify after expiring their VCP 4 or VCP 5. For current VCP5-DCV holders it's also
possible to pass the VCP6-DCV delta exam, which has 45 questions only. Today's topc? VCP6-DCV Objective 9.1 Configure Advanced vSphere HA Features.
Those study blog posts are covering topics and objectives from the blueprint from VCP 6 page and are here to help
out with studying towards the VMware Certification Exam VCP6-DCV (Datacenter Virtualization). This exam validates
you have the skills required to successfully install, deploy, scale and manage VMware vSphere 6.
vSphere Knowledge

Explain Advanced vSphere HA settings

Enable/Disable Advanced vSphere HA settings
Explain how vSphere HA interprets heartbeats
Interpret and correct errors during conversion
Identify virtual machine override priorities
Identify Virtual Machine Component Protection (VMCP) settings
181

E XPLAIN A DVANCED V S PHERE HA

SETTINGS

vSphere HA Advanced Options do not need to be changed in most environments.The HA advanced settings are
applied at the cluster level.
There is a very good VMware knowledge base article at http://kb.vmware.com/kb/2033250, which is based on
vSphere 5.x but still relevant for vSphere 6.
From vSphere 6.0 documentation center:

das.isolationaddress[...] - Sets the address to ping to determine if a host is isolated from the network. This
address is pinged only when heartbeats are not received from any other host in the cluster. If not specified,
the default gateway of the management network is used. This default gateway has to be a reliable address
that is available, so that the host can determine if it is isolated from the network. You can specify multiple
isolation addresses (up to 10) for the cluster: das.isolationaddressX, where X = 0-9. Typically you should specify
one per management network. Specifying too many addresses makes isolation detection take too long.
das.usedefaultisolationaddress - By default, vSphere HA uses the default gateway of the console network as
an isolation address. This option specifies whether or not this default is used (true|false).
das.isolationshutdowntimeout - The period of time the system waits for a virtual machine to shut down
before powering it off. This only applies if the host's isolation response is Shut down VM. Default value is 300
seconds.
das.slotmeminmb - Defines the maximum bound on the memory slot size. If this option is used, the slot size
is the smaller of this value or the maximum memory reservation plus memory overhead of any powered-on
virtual machine in the cluster.
das.slotcpuinmhz - Defines the maximum bound on the CPU slot size. If this option is used, the slot size is the
smaller of this value or the maximum CPU reservation of any powered-on virtual machine in the cluster.
das.vmmemoryminmb - Defines the default memory resource value assigned to a virtual machine if its
memory reservation is not specified or zero. This is used for the Host Failures Cluster Tolerates admission
control policy. If no value is specified, the default is 0 MB.
das.vmcpuminmhz - Defines the default CPU resource value assigned to a virtual machine if its CPU
reservation is not specified or zero. This is used for the Host Failures Cluster Tolerates admission control policy.
If no value is specified, the default is 32MHz.
das.iostatsinterval - Changes the default I/O stats interval for VM Monitoring sensitivity. The default is 120
(seconds). Can be set to any value greater than, or equal to 0. Setting to 0 disables the check. Note: Values of
less than 50 are not recommended since smaller values can result in vSphere HA unexpectedly resetting a
virtual machine.
das.ignoreinsufficienthbdatastore - Disables configuration issues created if the host does not have sufficient
heartbeat datastores for vSphere HA. Default value is false.
das.heartbeatdsperhost - Changes the number of heartbeat datastores required. Valid values can range from
2-5 and the default is 2.
fdm.isolationpolicydelaysec - The number of seconds system waits before executing the isolation policy once
it is determined that a host is isolated. The minimum value is 30. If set to a value less than 30, the delay will
be 30 seconds.
das.respectvmvmantiaffinityrules - Determines if vSphere HA enforces VM-VM anti-affinity rules. Default
value is "false", whereby the rules are not enforced. Can also be set to "true" and rules are enforced (even if
vSphere DRS is not enabled). In this case, vSphere HA does not fail over a virtual machine if doing so violates
a rule, but it issues an event reporting there are insufficient resources to perform the failover.
das.maxresets - The maximum number of reset attempts made by VMCP. If a reset operation on a virtual
machine affected by an APD situation fails, VMCP retries the reset this many times before giving up
das.maxterminates - The maximum number of retries made by VMCP for virtual machine termination.
das.terminateretryintervalsec - If VMCP fails to terminate a virtual machine, this is the number of seconds the
system waits before it retries a terminate attempt

182

das.config.fdm.reportfailoverfailevent - When set to 1, enables generation of a detailed per-VM event when

an attempt by vSphere HA to restart a virtual machine is unsuccessful. Default value is 0. In versions earlier
than vSphere 6.0, this event is generated by default.
vpxd.das.completemetadataupdateintervalsec - The period of time (seconds) after a VM-Host affinity rule is
set during which vSphere HA can restart a VM in a DRS-disabled cluster, overriding the rule. Default value is
300 seconds.
das.config.fdm.memreservationmb - By default vSphere HA agents run with a configured memory limit of 250
MB. A host might not allow this reservation if it runs out of reservable capacity. You can use this advanced
option to lower the memory limit to avoid this issue. Only integers greater than 100, which is the minimum
value, can be specified. Conversely, to prevent problems during master agent elections in a large cluster
(containing 6,000 to 8,000 VMs) you should raise this limit to 325 MB.

Note : Once one of the options is changed, for all hosts in the cluster you must run the Reconfigure HA task.
Also, when a new host is added to the cluster or an existing host is rebooted, this task should be performed
on those hosts in order to update this memory setting.
E NABLE/DISABLE A DVANCED V S PHERE HA

SETTINGS

If you change the value of any of the following advanced options, you must disable and then re-enable vSphere HA
before your changes take effect. You can use both clients (Windows C# client or vSphere Web client). You
enable/disable always at the cluster level

Using the vSphere Web Client

1.
2.
3.
4.
5.
6.
7.
8.

Log in to VMware vSphere Web Client.

Click Home > vCenter > Clusters.
Under Object click on the cluster you want to modify.
Click Manage.
Click vSphere HA.
Click Edit.
Click Advanced Options.
Click Add and enter in Option and Value fields as appropriate (see below).

1. Deselect Turn ON vSphere HA.

2. Click OK.
183

3. Wait for HA to unconfigure, click Edit and check Turn ON vSphere HA.
4. Click OK and wait for the cluster to reconfigure.

To get back to the defaults:

remove fdm.cfg file on each hosts in the cluster OR reset the values to defaults on each host in the cluster.

E XPLAIN

HOW VS PHERE

HA

INTERPRETS HEARTBEATS

When configuring VMware High Availability (HA) cluster, you have the possibility to check as a secondary
communication channel a datastore (or several ones), during the configuration wizard. VMware Datastore
Hearbeating provides an additional option for determining if host is in failed state or not.
In case the Master cannot communicate with a slave (dont receives the heartbeat), but the heartbeat datastore
answers, the server is still working. So if thats the case, the host is partitioned from the network, or isolated. The
Datastore heartbeat function helps greatly to determine the difference between host which failed and host that has
just been isolated from others.

THE P URPOSE OF THE . VS PHERE -HA FOLDER

This folder resides on shared datastore which is used as a secondary communication channel in HA architecture. This
folder has several files inside, and everyone of them has different rle (I don't think that's the required topic of the
exam, but it's interesting to know in case you browse your shared datastore and see the folder inside):

host-xxx-hb files those files are for the heartbeat datastore. The heartbeat mechanism uses the part of the
VMFS volume for regular updates. Each host in cluster has its own file like this in the .vSphere-HA folder.
protected list file when you open this file, youll see a list of VMs protected by a HA. The master host uses
this file for storing the inventory and the state of each VM.
host-xxx-poweron files this files roles is to track the running VMs for each host of the cluster. The file is read
by the master host which will know if a slave host is isolated from the network. Slave hosts uses this poweron
file to tell the master host hey, Im isolated. The content of this file reveals that there can be two states:
zero or one. Zero = not isolated and One = isolated. If the slave host is isolated, master host informs vCenter.
184

The .vSphere HA folder is created only on datastores that are used for the datastore heartbeating. You shouldnt
delete or modify those files. The space used is minimum, depending on the VMFS version used and number of hosts
that uses this datastore for heartbeating. It can be maximum about 3 Gb for on VMFS 3 and 2Mb on VMFS 5
(maximm and typical usage). The overhead isnt big either.

Limitations of Datastore hearbeating:

No VSAN support

INTERPRET

AND CORRECT ERRORS DURING CONVERSION

This chapter is concerning VMware converter. It's been recently update to version 6.

TIPS

Troubleshooting when vCenter Converter fails to complete a conversion of a physical or virtual machine.
Testing port connectivity with Telnet (1003487)
Best practices for using and troubleshooting VMware Converter (1004588)
Troubleshooting a virtual machine converted with VMware Converter that fails to boot with the error: STOP
0x0000007B INACCESSIBLE_BOOT_DEVICE (1006295)
Required VMware vCenter Converter 4.x/5.x ports (1010056)
Collecting diagnostic information for VMware Converter (1010633)
TCP and UDP Ports required to access VMware vCenter Server, VMware ESXi and ESX hosts, and other network
components (1012382)
VMware vCenter Converter is unable to see the disks when converting Windows operating systems (1016992)
vCenter Standalone Converter errors when an ESXi 5.x host is selected as a destination: The access to the host
resource settings is restricted. Use the management server as a destination (2012310)
AND

TRICKS

FROM

ESX V IRTUALIZATION

AND

V LADAN -:)

How-to disable SSL in VMware vCenter Converter Standalone to speed up P2V conversions
How-to Reduce VMDK size: VMware Converter
How to use VMware Converter to Synchronize changes when P2V (or V2V)
VMware Converter Best Practices
185

IDENTIFY

VIRTUAL MACHINE OVERRIDE PRIORITIES

You can customize settings for each VM in the cluster for VM restart priority, VMCP (see bellow), Host isolation
response or VM monitoring.
W HERE ?
In the vSphere Web Client, browse to the vSphere HA cluster > Manage tab > Settings > Under Settings, select VM
Overrides and click Add > Click the + button to select virtual machines to which to apply the overrides > OK.

If applied on the per-VM level, the settings now have more priority than the cluster settings and so they are different
on every other VMs. At the same time you can apply DRS rules there (you can see on the image above I have some
VMs which are not balanced automatically by DRS when Fully automated DRS is configured.

IDENTIFY V IRTUAL M ACHINE COMPONENT P ROTECTION (VMCP)

SETTINGS

HA was further enhanced with a function related to shared storage and its called VM Component Protection
(VMCP).
When VMCP is enabled, vSphere can detect datastore accessibility failures, APD (All paths down) or PDL
(Permannent device lost), and then recover affected virtual machines by restarting them on other host in the cluster
which is not affected by this datastore failure. VMCP allows the admin to determine the response that vSphere HA
will make. It can be simple alarm only or it can be the VM restart on other host. The latter one is perhaps what were
looking for. Lets HA handle this for us.

Limitations:

VMCP does not support vSphere Fault Tolerance. If VMCP is enabled for a cluster using Fault Tolerance, the
affected FT virtual machines will automatically receive overrides that disable VMCP.
No VSAN support (if VMDKs are located on VSAN then they're not protected by VMCP).
No VVOLs support (same here)
No RDM support (same here)

HOW TO ENABLE ?
At the cluster level. vSphere Client Select Hosts and clusters > Manage > vSphere HA > Edit > Protect against
Storage Connectivity Loss.
You must configure it on two places

186

1. Check the box Protect against Storage Connectivity Loss

2. Expand the Failure conditions and VM response

The second condition allows to specify what happens. There you have to specify 3 options:
By default it does not restart the VM on another host so its important to do it.
There youll see to options which you need to configure:
1. Response for Datastore with Permanent Device Lost (PDL)
2. Response for Datastore with All Path down (APD) with this one you have two choses. To be more
conservative or more aggressive. Basically it means to wait longer (or shorter) time in case the problem is
resolved. As I mentioned at the beginning of my post, APD can be resolved (can be temporary outage) but PDL
cant.
3. Response for APD recovery after APD timeout change it to reset VMs as by default its disabled.

187

All paths down (APD) - vSphere will restart the VM after user-configured timeout only if there is enough capacity.
Action? Restart on a healthy host. Reset a VM if APD clears after APD timeout.
Permannent device lost (PDL) - vSphere suppose that the device wont show up back again and is lost due to
hardware failure.
Action? Terminate VM immediately and restart on a healthy host.
If the Host Monitoring or VM Restart Priority settings are disabled, VMCP cannot perform virtual machine restarts.
The VMCP settings has to be changed from their default values as by default the Response for APD recovery after
APD is disabled.
You can check settings at the cluster level, but also via the VMs properties at the VM level by selecting the VM
through vSphere Web client.

188

Those fine-grain options allows to react on unpredictable APD and PDL signals when using shared storage within
your environment and give you significant insurance in case of connectivity problems to your shared storage.

LINKS

AND

TOOLS

vSphere Installation and Setup Guide

vSphere Availability Guide
Whats New in the VMware vSphere 6.0 Platform
vSphere Administration with the vSphere Client Guide
vSphere Client / vSphere Web Client

VCP6-DCV OBJECTIVE 9.2 - CONFIGURE ADVANCED VSPHERE DRS FEATURES

VMware VCP6-DCV certification exam might seems tough exam you can't fake. True, you have to know your stuff.
But we like technology, we like VMware and so we like this exam. Many folks need to re-certify after expiring their
VCP 4 or VCP 5. For new people who learning towards this exam I'm currently working on each one of the Objectives
from the VMware VCP6-DCV blueprint. For current VCP5-DCV holders it's also possible to pass the VCP6-DCV delta
exam, which has 45 questions only.
Those study blog posts are covering topics and objectives from the blueprint from VCP 6 page and are here to help
out with studying towards the VMware Certification Exam VCP6-DCV (Datacenter Virtualization). This exam validates
you have the skills required to successfully install, deploy, scale and manage VMware vSphere 6.
vSphere Knowledge

Identify Distributed Resource Scheduler (DRS) affinity rules

Enable/Disable Distributed Resource Scheduler (DRS) affinity rules
Identify Distributed Resource Scheduler (DRS) Automation levels
Configure Distributed Resource Scheduler (DRS)Automation levels

IDENTIFY DISTRIBUTED R ESOURCE S CHEDULER (DRS)

AFFINITY RULES

The affinity rules controls the initial placement of VMs in DRS enabled clusters. From vSphere 6.0 documentation...
Two Types:

VM-Host (Between a group of virtual machines and a group of hosts) - An affinity rule specifies that the
members of a selected virtual machine DRS group can or must run on the members of a specific host DRS
group. An anti-affinity rule specifies that the members of a selected virtual machine DRS group cannot run on
the members of a specific host DRS group.
VM-VM (Between individual virtual machines) - A rule specifying affinity causes DRS to try to keep the
specified virtual machines together on the same host, for example, for performance reasons. With an antiaffinity rule, DRS tries to keep the specified virtual machines apart, for example, so that when a problem occurs
with one host, you do not lose both virtual machines.

Requirements:

Licenisng. You have to be able activate vSphere HA and DRS cluster.

Shared Storage - you need obviously shared storage to be able to activate HA, DRS, vMotion (yes vMotion as
well).
189

VM-HOST AFFINITY RULE

specifies an affinity relationship between a group of virtual machines and a group of hosts. There are 'required' rules
(designated by "must") and 'preferential' rules (designated by "should".)
A VM-Host affinity rule includes the following components:

One virtual machine DRS group.

One host DRS group.

VM-VM AFFINITY RULE

Whether VMs should run on the same host or be kept on separate hosts.
With an anti-affinity rule, DRS tries to keep the specified virtual machines apart. You could use such a rule if you
want to guarantee that certain virtual machines are always on different physical hosts. In that case, if a problem
occurs with one host, not all virtual machines would be placed at risk.

E NABLE/DISABLE DISTRIBUTED R ESOURCE S CHEDULER (DRS)

AFFINITY RULES

Where? In the vSphere Web Client > Host and clusters > Manage TAB > VM/Host Rules > Add > Give your rule a
name
From the Type menu, select Virtual Machines to Hosts. Select the virtual machine DRS group and the host DRS group
to which the rule applies.

If you select the Keep virtual machines together (third option in the image above), and so be able to use this rule you
must first create VM/host Groups.... (option above close to the step 2 on the left hand side in the picture)

Must run on hosts in group - Virtual machines in VM Group 1 must run on hosts in Host Group A.
Should run on hosts in group - Virtual machines in VM Group 1 should, but are not required, to run on hosts
in Host Group A.
Must not run on hosts in group - Virtual machines in VM Group 1 must never run on host in Host Group A.
190

Should not run on hosts in group - Virtual machines in VM Group 1 should not, but might, run on hosts in Host
Group A.

Create Affinity Rule...

Create Anti-Affinity Rule...

IDENTIFY DISTRIBUTED R ESOURCE S CHEDULER (DRS) A UTOMATION

LEVELS

TIP: When DRS is disabled, the clusters resource pool hierarchy and affinity rules are not reestablished when DRS is
turned back on. So if you disable DRS, the resource pools are removed from the cluster.
To avoid losing the resource pools, instead of disabling DRS, you should suspend it by changing the DRS automation
level to manual (and disabling any virtual machine overrides). This prevents automatic DRS actions, but preserves
the resource pool hierarchy.
There you can check the drop down menu and try to check the:

191

FT VMs can benefit from DRS (EVC must be enabled) to be initially placed at best. If FT VMs are on cluster with EVC
disabled, then the FT VMs are given the DRS automation levels of "disabled".
A FFINITY RULES AND FT VM S
VM-VM affinity rule is applying to the primary VM only.
Host-VM affinity rule applies to both primary and secondary VM.

CONFIGURE DISTRIBUTED R ESOURCE S CHEDULER (DRS)A UTOMATION

LEVELS

Where? Select Hosts and clusters > Manage > settings > vSphere DRS > Edit
Then from the drop down menu choose the automation level you need.

Tools

vSphere Installation and Setup Guide

vSphere Administration with the vSphere Client Guide
Whats New in the VMware vSphere 6.0 Platform
vSphere Resource Management
vSphere Client / vSphere Web Client

VCP6-DCV OBJECTIVE 10.1 - CONFIGURE ADVANCED VSPHERE VIRTUAL M ACHINE

SETTINGS
My VCP6-DCV Study Guide on my blog is getting crowdy with more and more objectives. Today's topic is Objective
10.1 - Configure Advanced vSphere Virtual Machine Settings. We'll be looking into some advanced options (with
some tweaks) which are not only needed to pass the VCP6 exam, but are useful in real life.
There is many tips and tricks I have published in the past for vSphere 5.x and vSphere 6. You can check the How-to
articles, config/troubleshooting videos on vSphere 5.5/vSphere 6.x on those two Wordpress pages.
But today's topic needs some more deep info concerning the VMs configuration parameters including settings like
disabling VMs acceleration.
192

vSphere Knowledge

Identify available virtual machine configuration settings

Interpret virtual machine configuration files (.vmx) settings
Identify virtual machine DirectPath I/O feature
Enable/Disable Advanced virtual machine settings

IDENTIFY

AVAILABLE VIRTUAL MACHINE CONFIGURATION SETTINGS

The configuration settings of a VM can be accessed through vSphere client and vSphere web client. We'll focus
however on the settings through vSphere web client as this is the main client going forward even if it's still flash
based and here and there the performance aren't optimal. We shall see HTML5 based client in the next update of
VMware vSphere.
So start vSphere web client and edit a single VM by going to Select VM > Edit settings > VM Options

General Options - Virtual machine name and location of the virtual machine configuration file and virtual machine
working location. View or change the type and version of the guest operating system.

VMware Remote Console Options - Locking behavior and settings for simultaneous connections.

193

VMware Tools - Power Controls behavior, VMware Tools scripts, automatic upgrades, and time synchronization
between the guest and host.

Power Management - Virtual machine Suspend behavior and wake on LAN.

Boot Options - You can set the boot delay and other cool stuff here. Virtual machine boot options. Add a delay
before booting, force entry into the BIOS or EFI setup screen, or set reboot options.

194

Advanced Advanced virtual machine option:

Settings - Specify acceleration and logging settings.

Debugging and statistic - Specify the level of debugging information that is being collected.
Swap file location - Specify the swap file location.
Configuration Parameters - View, modify, or add configuration parameters.
Latency Sensitivity - Set a value for latency sensitivity.

Fibre Channel NPIV Virtual node and port World Wide Names (WWNs).

195

INTERPRET

VIRTUAL MACHINE CONFIGURATION FILES

(.VMX )

SETTINGS

The VMX settings can be changed through the VMs Options > Advanced configuration > Edit configuration

Usually the VMX file is in the same folder as the VM, but it can happen that the VMx files are stored elsewhere. To
check where are the files located you can see it in general options where the path to the location of the virtual

196

machine configuration file shows. The path to the virtual machine working location appears in the VM Working
Location text box.
1. The location of the VMX file
2. The location of the working location (VMDK,

VMs files:

IDENTIFY

VIRTUAL MACHINE

DIRECT P ATH I/O

FEATURE

VMdirect Path I/O - what's that? When enabled, the VM can access physical PCI functions with an I/O memory
management unit (MMU). vSphere DirectPath I/O allows a guest operating system on a virtual machine to directly
access physical PCI and PCIe devices connected to a host. Each virtual machine can be connected to up to six PCI
devices. PCI devices connected to a host can be marked as available for passthrough from the Hardware Advanced
Settings in the configuration tab for the host.
L IMITATIONS (QUITE A FEW...):

No snapshot support - Snapshots are not supported with PCI vSphere Direct Path I/O devices
No Hot Add - Hot adding and removing of virtual devices
No Suspend and resume
No Record and replay
No FT - No Fault tolerance
No HA - No High availability support either...
DRS? - A kind of. DRS is limited to static..... The VM can be inside of DRS cluster, but cannot be vMotionned...

W HERE TO ENABLE ?
Edit Settings > On the Hardware tab, click Select > select PCI Device and click Add > Select the passthrough device to
connect to the virtual machine from the drop-down list > click Next.

197

D IRECT P ATH I/O VS SR-IOV

SR-IOV offers performance benefits and tradeoffs similar to those of DirectPath I/O. DirectPath I/O and SR-IOV have
similar functionality but you use them to accomplish different things.
SR-IOV is beneficial in workloads with very high packet rates or very low latency requirements. Like DirectPath I/O,
SR-IOV is not compatible with certain core virtualization features, such as vMotion. SR-IOV does, however, allow for
a single physical device to be shared amongst multiple guests.
With DirectPath I/O you can map only one physical function to one virtual machine. SR-IOV lets you share a single
physical device, allowing multiple virtual machines to connect directly to the physical function.
ENABLE SR-IOV ON A H OST P HYSICAL A DAPTER
You must first enable it on the host level. In the vSphere Web Client, Select the host > Manage tab > Networking
and select Physical adapters > Select the physical adapter > Edit > Select Enabled from the Status drop-down menu
> OK > Restart the host.

once enabled at the host level, then it's accessible to the VM as a physical device... The VM must be turned off
before starting to add the device.
TO A SSIGN V IRTUAL F UNCTION AS SR-IOV P ASSTHROUGH A DAPTER TO A V IRTUAL M ACHINE
198

VM settings > Add new device > Network > from the Adapter type drop-down menu, select SR-IOV passthrough.
Than expand the memory section, select reserve all guest memory (All locked) and click OK. I/O memory
management unit (IOMMU) must reach all virtual machine memory so that the passthrough device can access the
memory by using direct memory access (DMA).

E NABLE/DISABLE A DVANCED

VIRTUAL MACHINE SETTINGS

Well here we could list how to enable/disable different parameters, but I think it's pretty obvious as I added a
screenshot for each of those values. Keep in mind that you're modifying config of individual VMs so to keep track of
those changens on per-individual VM might be quite tedious, but it might be worthy the effort when seeking to gain
a performance or troubleshoot an issue (activate logging).
One of the features that we haven't discussed is the Change swap file location. As you know, when a VM is powered
On, the ESXi host creates vmkrnel swap file which allows to back up the VMs RAM content. The default swap file
(vmname.vswp) location is at the same location as the other VMs files.

Default - Use the settings of the cluster or host containing the VM

VMs Directory - store the swap files in the same directory as the VM
Datastore specified by host - you can store the swap files in the datastore specified by the host to be used for
swap files. Note that using a datastore that is not visible to both hosts during vMotion might affect the
performance of the vMotion operation for the VM(s).

CHANGE A S WAP FILE LOCATION H OW - TO ?

vSphere web client Select VM > Edit settings > VM Options > Advanced

Tools and documentation for this topic

vSphere Installation and Setup Guide

vSphere Administration with the vSphere Client Guide
vSphere Virtual Machine Administration Guide
vSphere Client / vSphere Web Client

199

VCP6-DCV OBJECTIVE 10.2 - CREATE AND MANAGE M ULTI-SITE CONTENT LIBRARY

VCP6-DCV Study Guide is here to help you study towards VCP6-DCV (or delta) exam. Today's topic is new in vSphere
6. Feature called vSphere Content Library was not present in vSphere 5.5 and made its apparition in vSphere 6 during
its release. VCP6-DCV Objective 10.2 - Create and Manage Multi-Site Content Library is todays lesson. vSphere
content library centrally manages virtual machine templates, ISO images, and scripts, and it performs the content
delivery of associated data from the published catalog to the subscribed catalog at other sites.
You can also check vSphere 6 page where youll find how-tos, news, videos concerning vSphere 6.x. Last but not
least, my Free Tools page where are the post popular tools for VMware and Microsoft. Daily updates of the blog are
taking time, but we do it in the goal to provide a guide which is helpful for the community and folks learning towards
VCP6-DCV certification exam. If you find one of those posts useful for your preparation, just share.. -:).
Before we start I'd like to point a screenshot showing the ISO management... (must select the Other types button..)

OK, let's get started. vSphere Knowledge

Configure Content Library to work across sites

Configure Content Library authentication
Set/Configure Content Library roles
Add/Remove Content Libraries

CONFIGURE CONTENT LIBRARY

TO WORK ACROSS SITES

Content Library lets you store and manage content from a central location. Admins can organize content logically
into several libraries. Each individual librarys storage can be individually configured and managed. Admins can
populate each library using several methods:

Clone existing templates in folders into Content Library (migrate your existing templates into Content Library
with ease)
Clone a VM as a template into Content Library
Import from a web server
Synchronize content from a vCloud Director catalog
Upload contents from file system

Content library can be shared across multiple vCenter server systems.

A VM template, vApp template or another type file is considered as a library item. Each item can contain several files
(ex. OVF has several files .ovf, .vmdk, .mf, ...) however vSphere client shows only the .ovf through the content library.
200

What's the different types of content libraries?

Local Libraries - Local library stores items in single vCenter environment. When you publish to the local library, other
users from external vCenter servers can subcribe to this library. And to protect the access you can configure
password authentication.
Subscribed Libraries - When you subscribe to published library, then you create a subscribed library, which can be
created at the same vCenter server as the original content library or in another vCenter server system.
Pull the content - there is two different ways that you can pull the content out of vSphere content library:
1. Either you can download all the content of the published library after you create the subscribed library
2. You can download only metadata for the items in the subscribed library so you save space.
Permission Requirements

User needs those permissions on the vCenter Server instance where you want to create the library:

Content library
Create local library or Content library
Create subscribed library

Note that Global permission must be assigned to the user....

Content libraries are not direct children of a vCenter Server system from an inventory perspective. The direct
parent for content libraries is the global root. This means that if you set a permission at a vCenter Server level and
propagate it to the children objects, the permission applies to data centers, folders, clusters, hosts, virtual machines,
and so on, but does not apply to the content libraries that you see and operate with in this vCenter Server instance.
To assign a permission on a content library, an Administrator must grant the permission to the user as a global
permission. Global permissions support assigning privileges across solutions from a global root object.

201

See the diagram from VMware vSphere 6.0 Documentation...

CONFIGURE CONTENT LIBRARY

AUTHENTICATION

To enable authentication, select the library > Actions > Edit settings > Check the "Enable user authentication for
access to this library".

202

S ET/C ONFIGURE CONTENT LIBRARY

ROLES

Content Library Administrator

Content Library Administrator role is a predefined role that gives a user privileges to monitor and manage a library
and its contents.
A user who has this role can perform the following tasks:

Create, edit, and delete local or subscribed libraries.

Synchronize a subscribed library and synchronize items in a subscribed library.
View the item types supported by the library.
Configure the global settings for the library.
Import items to a library.
Export library items.

You can clone this role or use this role as is and assign this role to the user that shall manage the content library.

A DD /R EMOVE CONTENT LIBRARIES

To Add a Content Library: (to create)
vCenter Inventory Lists > Content Libraries > Click the Objects tab > Click the Create a New Library icon
Give it some meaningful name..

Click next to follow the assistant and choose one of the options...

Then again continue with the next button and choose a storage...
203

Hit next and finish.

To Delete a Content library:

vSphere Web Client > vCenter Inventory Lists > Content Libraries > Select library from the list > Actions > Delete >
Confirm

Synchronize Library Items:

Web Client > vCenter Inventory Lists > Content Libraries > Select a subscribed library from the list, and click the
Related Objects tab. > Synchronize the item you want to use.
On the Templates tab, right-click a VM or a vApp template, and select Synchronize Item > On the Other Types tab,
right-click an item, and select Synchronize Item.

204

After synchronization completes, the item content and metadata are downloaded to the backing storage of the
subscribed library, and in the Related Objects tab the value for the item in the Stored Content Locally column
changes to Yes.
Tools

vSphere Installation and Setup Guide

vSphere Administration with the vSphere Client Guide
Whats New in the VMware vSphere 6.0 Platform
vSphere Virtual Machine Administration Guide
vSphere Client / vSphere Web Client

VCP6-DCV OBJECTIVE 10.3 - CONFIGURE AND MAINTAIN A VCLOUD AIR CONNECTION

Last chapter in the big VCP6-DCV series today where we'll learn about vCloud Air and connection through vCenter:
VCP6-DCV Objective 10.3 - Configure and Maintain a vCloud Air Connection is the title of the objective.
You will learn details on the requirements to setup vCloud Air connection, configuration of vCenter server
connection to vCloud Air.
The whole VCP6-DCV Study Guide page. Register for the VCP6-DCV exam here. In addition, you might want to visit
our Free Tools page or vSphere 6 page for latest updates and news concerning vSphere 6 or free tools for IT
administrators.
vSphere Knowledge

Identify vCenter Server and vCloud Air Connection requirements

Configure vCenter Server connection to vCloud Air
Identify connection types
Configure replicated objects in vCloud Air Disaster Recovery service

IDENTIFY V C ENTER S ERVER

AND V C LOUD

A IR CONNECTION

Setting up the vCloud Air DR service is done through VMware web site.
205

REQUIREMENTS

Requirements:

vSphere 5.5 or later (6.0 recommended)

My VMware Account
Firewall Ports - 10000 to 10010 of ESXi hosts are open for outgoing traffic . The required ports are open
automatically when you install a VIB on each supported ESXi host in the environment where the vSphere
Replication appliance is deployed

Compatible products - vSphere replication appliance 6.0, ESXi 5.0, 5.1.x, 5.5.x or 6.0, vCenter 6.0, vSphere
Web client 6.0
Roles, permissions to the cloud - usually assigned through vCloud Air UI after successfully installing vSphere
replication.
Check that you have VR up and running in your environment
Verify that the Disaster Recovery to Cloud service is enabled in the target cloud organization
Configure connection to the cloud organization.

CONFIGURE V CENTER S ERVER

CONNECTION TO V C LOUD

A IR

vSphere replication to the cloud p.12

When you create a connection to the cloud, the vCloud Tunneling Agent in the vSphere Replication appliance creates
a tunnel to secure the transfer of replication data to your cloud Organization.
When a tunnel is created, the vCloud Tunneling Agent opens a port on the vSphere Replication appliance. ESXi hosts
connect to that port to send replication data to a cloud organization. The port is picked randomly from a
configurable range. The default port range is 10000-10010 TCP.
In vSphere Replication, you must establish a connection to your cloud provider before you configure replications to
cloud. The vSphere Replication UI requires you to enter the cloud provider address and the cloud organization name.
Click VR icon in the vSphere web client > On the Home TAB click the Manage button.

206

The Manage tab should be preselected > click Target Sites > and then click the Connect to a Cloud Provider icon.

A pop-up windows shows up where you'll be able to enter the connection details. The information that you need is
included in the subscription email that you receive from VMware vCloud Air.

On the Connection settings page, type the address of your cloud provider, the organization name, and credentials to
authenticate with the cloud. By default, vSphere Replication uses these credentials to establish a user session to the
cloud and for system monitoring purposes. To enable system monitoring, these credentials will be stored in the
vSphere Replication appliance, unless you select to use another user account for system monitoring.
(Optional) If you do not want to store the credentials that you used for authentication, select the Use a different
account for system monitoring check box, and type the credentials to be used for system monitoring. These
credentials are encrypted and stored in the vSphere Replication database.
Click Next > The Connect to a Cloud Provider wizard displays a list of virtual data centers to which you can connect.
If a virtual data center is already connected to the vCenter Server, that data center does not appear in the list. From
the list of virtual data centers, select a target for the connection and click Next > Finish

207

You'll need the Cloud provider address and Organization name. You can find those information when you Connect
to your vCloud Air portal > The Replication tab.

IDENTIFY

CONNECTION TYPES

There is two types of credentials when you create a connection to the target virtual data center (VDC):

Connection credentials - used for authentication within the cloud organization. The priviledges are managed
by cloud provider. Few rights are required: ManageRight, ViewRight, View Organization Networks,
View Organizations, View organization VDC, View Organization VDC. Credentials to the cloud are
needed for each target site, once per user session. When the authenticated user session to a target site
expires, users are prompted to input their credentials again
System monitoring credentials - used for system runtime, so the source and destination sites can
communicate together. Those credentials are stored in the VR appliance on the source site. The user name
must have VR role with few priviledges: ManageRight, ViewRight, View organization Networks, View
Organizations, View Organization VDCs

CONFIGURE

REPLICATED OBJECTS IN V C LOUD

A IR DISASTER R ECOVERY

SERVICE

The installation and deployment of VR has been detailed in the Objective covering vSphere Data protection - VCP6DCV Objective 6.1 Configure and Administer a vSphere Backups/Restore/Replication Solution. See the details of the
deployment there.
vCloud air DR user's guide p. 19
You can configure replicate single VM or multiple VMs at a time. The same way as configuring replication between
hosts in your On Premise environment.
You will be able to set a recovery point objective (RPO) to determine the maximum data loss that you can tolerate.
For example, an RPO of 1 hour seeks to ensure that a virtual machine loses the data for no more than 1 hour during
the recovery. vSphere Replication guarantees crash consistency amongst all the disks that belong to a virtual
machine. (VSS checkbox)
NOTE: By default, when you configure a virtual machine for replication to cloud, its NICs and MAC addresses are
copied automatically to the target site as part of the provisioning of the placeholder virtual machine. If the test
network is not isolated from the production network and these networks have common routing, a test recovery of a
replicated virtual machine might result in duplicate MAC addresses in your virtual data center.
You can check p.16 of the vSphere Replication to the Cloud document for details how to disable that.
When you configure replication by using vSphere Replication at your source site, the Disaster Recovery service
creates placeholder virtual machines in vCloud Air which represent the virtual machines at your source site.

208

The placeholders are VM for which you are testing recovery, and virtual machines recovered to the cloud. A
placeholder virtual machine appears in the VM's tab after the initial full synchronization of replication data from the
source site successfully completes.
Use the Virtual Machines tab to test recovery and recover the virtual machines to the cloud in the event your source
site is unavailable. The status of each placeholder determines what actions are available for that virtual machine
represented. After you test a recovery or recover a virtual machine to the cloud, the Disaster Recovery service
replaces the placeholder with a test or production virtual machine respectively.
You can enable multiple point in time recovery snapshots.

If you enable multiple point in time (MPIT) setting, you can use previous replication points for better control on
failover. It allows you to:

Set up to 24 previous restore points

Choose your restore point
Restore up to 24 days previous replication points (dependent on your RPO setting)

209

Tools

vSphere Installation and Setup Guide

vSphere Administration with the vSphere Client Guide
vSphere Networking Guide
VMware vCloud Air Disaster Recovery Users Guide
vSphere Client / vSphere Web Client

210