Operational Best Practices For NSX in Vmware: Environments
Operational Best Practices For NSX in Vmware: Environments
#NET5790
Agenda
Virtual
Network
Operations
1. Decouple
2. Reproduce
3. Automate
Physical
Hardware
independence
Physical
No change to network
from end host perspective
Cloud
Operations
Operational benefits
of virtualization
Agenda
RSPAN/ERSPA
N
Performance
Statistics
Syslog
Physical
Virtual
Capability
VM level
visibility
Network
Snapshot
CLI
UI
API
Physical
?
?
Virtual
All NSX components such as the NSX Controller, NSX vSwitch and
NSX Edge provide detailed network visibility and data
DC Networks
Virtualization Administrators
Network Administrators
Security Administrators
Service Providers
Agenda
10
Consumption
NSX Manager
Management
vCenter Server
Plane
NSX Edge
Logical Router
NSX Controller
User World Agent
Control
Plane
NSX vSwitch
VXLAN
Distributed
Firewall
Logical Router
ESXi
NSX Edge
Services
Gateway
VDS
Data
Plane
11
NSX vSwitch
Distributed network edge
Line Rate performance
NSX Edge
VM form factor
Data Plane for North South
traffic
Routing and Advanced
services
Configures Controller
Cluster via a REST API
and hosts via a
message bus
Generates certificates to
secure control plane
communications
12
Logical Router
Logical Router
VXLAN
VXLAN
Controller
VXLAN Directory
Service
MAC table
ARP table
Logical Router
VTEP table
VXLAN
NSX Controllers are clustered for scale out and high availability
Network information is sliced across nodes in a Controller Cluster
Enables dependency on multicast routing/PIM in the physical network to
be removed
Components UW agent
UW agent is a TCP (SSL) client that communicates with the Controller using
the control plane protocol
Controller
Controller
Controller
User World
Agent
Client
Client
Client
Kernel
Modules
ESXi Host
14
NSX
MGR
VXLAN
LR
NSX vSwitch
NSX Edge
Logical
Router
(vSphere VIBs)
VDS
VXLAN
vSphere
Logical Router
Firewall
VMkernel Modules
VXLAN
Distributed Routing
Distributed Firewall
Switch Security
Message Bus
15
NSX Edge
Services
Gateway
updates to Controller
L3-L7 Services:
NAT, DHCP, LB, VPN, Interface based
FW
Dynamic Routing
VM form factor
High Availability
Agenda
16
pktcap-uw
New with vSphere 5.5
Enhanced tool that provides a framework for packet capture and tracing at the
Uplink, vSwitch, vmknic, vnic and port level at any stage in a packets lifecycle
17
pktcap-uw
Extensive range of filters
such as source/destination
mac, IP, Protocol, VLAN,
VXLAN, ports etc.
18
19
NetFlow / IPFIX
NetFlow collector
address and port
20
21
22
NSX Manager
NSX
Controller
Cluster
UWA
UWA
UWA
VTEP
UWA
VTEP
UWA
VTEP
UWA
VTEP
VTEP
VTEP
vSphere Cluster A
vSphere Cluster B
24
VM
IP1
VM
MAC1
IP
2
MAC2
VNI
VNI
5001
5001
VXLAN 5001
10.20.10.10
11
vSphere Host
10.20.10.12
11
vSphere Host
VNI
VNI
VTEPIPIP
VTEP
VNI
VNI
VTEPIP
IP
VTEP
5001
5001
10.20.10.10
10.20.10.10
10.20.10.11
5001
5001
10.20.10.11
10.20.10.10
10.20.10.11
VTEPIPIP
VTEP
10.20.10.10
10.20.10.10
10.20.10.11
vSphere Host
Controller
10
Send VNI,VTEP
Mapping to
Controller
10
Management
Network
25
7
3
26
27
28
NSX for vSphere provides a new workflow for preparation and configuration
Supports multiple VTEPs per host
29
At Logical Switch level on the Monitoring Tab you can use the
unicast or broadcast test to verify the connectivity between VTEPs
30
Unicast Mode
All replication occurs using
unicast
Hybrid Mode
Local replication offloaded to
physical network, while remote
replication occurs via unicast
Multicast Mode
Requires IGMP for a Layer 2
topology and Multicast Routing
for L3 topology
# esxcli network vswitch dvs vmware vxlan network mac list vds-name=Compute_VDS -vxlan-id=5001
# esxcli network vswitch dvs vmware vxlan network arp list --vds-name Compute_VDS --vxlan-id=5001
# esxcli network vswitch dvs vmware vxlan network port list --vds-name Compute_VDS --vxlan-id=5001
# esxcli network vswitch dvs vmware vxlan network stats list --vds-name Compute_VDS --vxlan-id=5001
32
33
34
35
Logging commands
show log {follow|reverse}
show flowtable
Debug/troubleshooting commands
traceroute <ip_address or dns_name>
ping <ip address> or ping interface addr <alternate_src_ip> <ip_address>
debug packet display interface <vNic_0-9> <EXPRESSION>
37
38
39
40
4) ESXi Hosts
41
42
NSX Manager exposes web service API over HTTPS (TCP 443)
43
Agenda
44
45
Questions
47
Backup Slides
48
Logical Topology
VM to VM connectivity
L2
Controller Cluster
Inventory/Fault Mgmt
vSwitch
ESXi
vSwitch
vSwitch
vSwitch
vSwitch
ESXi
ESXi
ESXi
NSX Edge
WAN/Inter
net
49
Upgrade Management
VM to VM connectivity (Logical)
IPFIX (VDS)
NSX Edge Flow Monitoring
Upgrade Management
API visibility
External Tools
50
REST API
vSphere API
NSX Manager
vCenter Server
REST API
Control Plane
Data Plane
Control Plane
Protocol
CP Protocol
NSX Controller
NSX Edge
Logical
Router
NSX vSwitch
51
Message Bus
vSphere API
UWA
NSX Edge
Services
Gateway
Extensible.
App ID in message common header.
52
1 Generation
NSX Manager
SSL
NSX Manager DB
REST API
Message Bus
UW
Agent
OVF
Deployment
UW
Agent
VTEP
UW
Agent
VTEP
UW
Agent
VTEP
Controller Cluster
vSphere Cluster A
53
SSL
SSL
VTEP
UW
Agent
VTEP
UW
Agent
VTEP
vSphere Cluster B
VM
IP1
VM
MAC1
IP
2
MAC2
VXLAN 5001
10.20.10.11
10.20.10.12
vSphere Host
vSphere Host
vSphere Host
VNI
VM MAC
VNI
VM MAC
5001
MAC1
5001
MAC2
VNI
VNI
VMMAC
MAC
VM
VTEP
VTEP
5001
5001
MAC1
MAC1
10.20.10.10
10.20.10.1
0
5001
MAC2
10.20.10.1
1
Controller
Send VNI,VM
MAC Mapping
and VTEP IP to
Controller
7
6
3
Management
Network
54
VM
IP1
VM
MAC1
IP
2
MAC2
VXLAN 5001
10.20.10.11
10.20.10.12
vSphere Host
vSphere Host
vSphere Host
VNI
VM
IP
VM
MAC
VNI
VM
IP
VM
MAC
5001
IP1
MAC1
5001
IP2
MAC2
VNI
VNI
VMIPIP
VM
VMMAC
MAC
VM
5001
5001
5001
IP1
IP1
IP2
MAC1
MAC1
MAC2
Controller
Send VM MAC,
IP Mapping and
VNI to Controller
7
6
3
Management
Network
55
VM
L2
Payload
IP1
VM
IP2
MAC2
MAC1
VNI
VM
IP
VM
MAC
VTEP
5001
IP1
MAC1
10.20.10.10
5001
IP2
MAC2
10.20.10.11
VXLAN 5001
10.20.10.10
10.20.10.11
10.20.10.12
vSphere Host
vSphere Host
vSphere Host
VNI
VNI
VM
VM
IP
IP
IP1
IP1
VM
VM
MAC
MAC
MAC
1MAC1
5001
5001
10.20.10.1
010.20.10.10
5001
IP2
MAC2
10.20.10.11
VTEP
VTEP
Controller
ARP Report for
VM IP2, MAC2
sent to VTEP
10.20.10.10
Management
Network
56
L2
VM
Payload
IP1
VM
MAC2
10
MAC1
IP2
VXLAN 5001
VNI
V
M
IP
VM
MAC
VTEP
5001
IP1
MAC1
10.20.10.10
5001
IP2
MAC2
10.20.10.11
10.20.10.11
10.20.10.12
vSphere Host
vSphere Host
vSphere Host
VNI
VM
IP
VM
MAC
VTEP
VNI
VNI
VM
VM
IP
IP
VM
VM
MAC
MAC
VTEP
VTEP
500
1
IP1
MAC1
10.20.10.10
500
1500
1
IP2
IP2
MAC2
MAC1
10.20.10.11
10.20.10.11
500
1
IP2
MAC2
10.20.10.11
500
1
IP1
MAC2
10.20.10.10
9
VXLAN Transport
Network
DA: 10.20.10.11
SA: 10.20.10.10
DA: MAC1
5001
L2 IP UDP VXLAN L2
57
SA: MAC2
Payload
Controller
Download Edge Gateway Tech Support Logs using the Web Client
58
= GET
URI
HEADERS
= Authorization
59
= POST
URI
= https://<NSX Manager
Hostname>/api/2.0/vdn/scopes/vdnscope-1/virtualwires
HEADERS
= Authorization, Content-Type
HTTP Body =
<virtualWireCreateSpec>
<name>Test-Logical-Switch-01</name>
<description>Created via REST API</description>
<tenantId>virtual wire tenant</tenantId>
<multicastProxy>true</multicastProxy>
<disableMulticast>true</disableMulticast>
</virtualWireCreateSpec>
60
The Response Body provides the virtualwire-id, which can be used for additional
operations (eg, attaching to a Logical Router LIF or for Distributed Firewall rules)
61
Preparation
Login to ESXi host (destination)
Add Logical Switch
Connect VMs to Logical Switch
Data Collection
Start data collection on destination host, output to a share (that is also accessible on
analyzer)
62
References
63
HOL:
HOL-SDC-1303
VMware NSX Network Virtualization Platform
64
THANK YOU
NET5790
Operational Best Practices for NSX in VMware
Environments
Ray Budavari, VMware
Thomas Kraus, VMware
#NET5790