Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release

6.x
First Published: 2014-01-04
Last Modified: 2016-03-29

Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: 408 526-4000
800 553-NETS (6387)
Fax: 408 527-0883
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS,
INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.

THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH
THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY,
CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain version
of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.

NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS" WITH ALL FAULTS.
CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT
LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS
HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network
topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional
and coincidental.

This product includes cryptographic software written by Eric Young ([email protected]).

This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit. (http://www.openssl.org/)

This product includes software written by Tim Hudson ([email protected]).

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: http://
www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership
relationship between Cisco and any other company. (1110R)

© 2014 - 2015 Cisco Systems, Inc. All rights reserved.

 CONTENTS

Preface Preface v
Audience v
Document Conventions v
Related Documentation for Cisco Nexus 9000 Series Switches vi
Documentation Feedback vi
Obtaining Documentation and Submitting a Service Request vii

CHAPTER 1 New and Changed Information 1

New and Changed Information 1

CHAPTER 2 Overview 3
VXLAN Overview 3
VXLAN Encapsulation and Packet Format 4
VXLAN Tunnel Endpoint 4
VXLAN Packet Forwarding Flow 4
Cisco Nexus 9000 as Hardware-Based VXLAN Gateway 4
vPC Consistency Check for vPC VTEPs 5

CHAPTER 3 Configuring VXLAN 7

Information About VXLAN 7
Guidelines and Limitations for VXLAN 7
Considerations for VXLAN Deployment 8
VPC Considerations for VXLAN Deployment 9
Network Considerations for VXLAN Deployments 11
Considerations for the Transport Network 12
Configuring VXLAN 13
Enabling VXLANs 13

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
iii
 Contents

Mapping VLAN to VXLAN VNI 13

Creating and Configuring an NVE Interface and Associate VNIs 14
Disabling VXLANs 15
Verifying the VXLAN Configuration 15
Example of VXLAN Bridging Configuration 17

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
iv
 Preface
This preface includes the following sections:

• Audience, page v
• Document Conventions, page v
• Related Documentation for Cisco Nexus 9000 Series Switches, page vi
• Documentation Feedback, page vi
• Obtaining Documentation and Submitting a Service Request, page vii

Audience
This publication is for network administrators who install, configure, and maintain Cisco Nexus switches.

Document Conventions
Command descriptions use the following conventions:

Convention Description
bold Bold text indicates the commands and keywords that you enter literally
as shown.

Italic Italic text indicates arguments for which the user supplies the values.

[x] Square brackets enclose an optional element (keyword or argument).

[x | y] Square brackets enclosing keywords or arguments separated by a vertical

bar indicate an optional choice.

{x | y} Braces enclosing keywords or arguments separated by a vertical bar

indicate a required choice.

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
v
 Preface
Related Documentation for Cisco Nexus 9000 Series Switches

Convention Description
[x {y | z}] Nested set of square brackets or braces indicate optional or required
choices within optional or required elements. Braces and a vertical bar
within square brackets indicate a required choice within an optional
element.

variable Indicates a variable for which you supply values, in context where italics
cannot be used.

string A nonquoted set of characters. Do not use quotation marks around the
string or the string will include the quotation marks.

Examples use the following conventions:

Convention Description
screen font Terminal sessions and information the switch displays are in screen font.

boldface screen font Information you must enter is in boldface screen font.

italic screen font Arguments for which you supply values are in italic screen font.

<> Nonprinting characters, such as passwords, are in angle brackets.

[] Default responses to system prompts are in square brackets.

!, # An exclamation point (!) or a pound sign (#) at the beginning of a line

of code indicates a comment line.

Related Documentation for Cisco Nexus 9000 Series Switches

The entire Cisco Nexus 9000 Series switch documentation set is available at the following URL:
http://www.cisco.com/en/US/products/ps13386/tsd_products_support_series_home.html

Documentation Feedback
To provide technical feedback on this document, or to report an error or omission, please send your comments
to [email protected]. We appreciate your feedback.

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
vi
 Preface
Obtaining Documentation and Submitting a Service Request

Obtaining Documentation and Submitting a Service Request

For information on obtaining documentation, using the Cisco Bug Search Tool (BST), submitting a service
request, and gathering additional information, see What's New in Cisco Product Documentation at: http://
www.cisco.com/c/en/us/td/docs/general/whatsnew/whatsnew.html.
Subscribe to What's New in Cisco Product Documentation, which lists all new and revised Cisco technical
documentation as an RSS feed and delivers content directly to your desktop using a reader application. The
RSS feeds are a free service.

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
vii
 Preface
Obtaining Documentation and Submitting a Service Request

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
viii
 CHAPTER 1
New and Changed Information
This chapter provides release-specific information for each new and changed feature in the Cisco Nexus
9000 Series NX-OS VXLAN Configuration Guide.

• New and Changed Information, page 1

New and Changed Information

This table summarizes the new and changed features for the Cisco Nexus 9000 Series NX-OS VXLAN
Configuration Guide, Release 6.x and where they are documented.

Table 1: New and Changed Features

Feature Description Changed Where Documented

in
Release
VXLAN vPC consistency Enables configuration 6.1(2)I3(4) vPC Consistency Check for
check support compatibility for two switches vPC VTEPs
configured as a vPC pair.

SVI uplinks support Added support for SVI uplinks. 6.1(2)I3(1) Configuring VXLAN, on page
7
Enables VxLAN encap over
SVI uplinks to spine.

Non-default VRF support Added support for VRF. 6.1(2)I3(1) Configuring VXLAN, on page
7
Enables VxLAN forwarding
over uplinks in non-default
VRFs.

anycast RP support Added support for anycast RP. 6.1(2)I3(1) Configuring VXLAN, on page
7
Enables the use of anycast RP
on spine for underlay multicast
load-balancing and redundancy.

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
1
 New and Changed Information
New and Changed Information

Feature Description Changed Where Documented

in
Release
per NVE peer statistics Added support to display per 6.1(2)I2(2a) Verifying the VXLAN
NVE peer statistics. Configuration, on page 15

per VNI statistics Added support to display per 6.1(2)I2(2a) Verifying the VXLAN
VNI statistics. Configuration, on page 15

VXLAN Initial VXLAN support. 6.1(2)I2(1) This document.

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
2
 CHAPTER 2
Overview
This chapter contains the following sections:

• VXLAN Overview, page 3

VXLAN Overview
Cisco Nexus 9000 switches are designed for hardware-based VXLAN function. It provides Layer 2 connectivity
extension across the Layer 3 boundary and integrates between VXLAN and non-VXLAN infrastructures.
This can enable virtualized and multitenant data center designs over a shared common physical infrastructure.
VXLAN provides a way to extend Layer 2 networks across Layer 3 infrastructure using MAC-in-UDP
encapsulation and tunneling. VXLAN enables flexible workload placements using the Layer 2 extension. It
can also be an approach to building a multitenant data center by decoupling tenant Layer 2 segments from
the shared transport network.
When deployed as a VXLAN gateway, Cisco Nexus 9000 switches can connect VXLAN and classic VLAN
segments to create a common forwarding domain so that tenant devices can reside in both environments.
VXLAN has the following benefits:
• Flexible placement of multitenant segments throughout the data center.
It provides a way to extend Layer 2 segments over the underlying shared network infrastructure so that
tenant workloads can be placed across physical pods in the data center.
• Higher scalability to address more Layer 2 segments.
VXLAN uses a 24-bit segment ID, the VXLAN network identifier (VNID). This allows a maximum of
16 million VXLAN segments to coexist in the same administrative domain. (In comparison, traditional
VLANs use a 12-bit segment ID that can support a maximum of 4096 VLANs.)
• Utilization of available network paths in the underlying infrastructure.
VXLAN packets are transferred through the underlying network based on its Layer 3 header. It uses
equal-cost multipath (ECMP) routing and link aggregation protocols to use all available paths.

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
3
 Overview
VXLAN Encapsulation and Packet Format

VXLAN Encapsulation and Packet Format

VXLAN is a Layer 2 overlay scheme over a Layer 3 network. It uses MAC Address-in-User Datagram Protocol
(MAC-in-UDP) encapsulation to provide a means to extend Layer 2 segments across the data center network.
VXLAN is a solution to support a flexible, large-scale multitenant environment over a shared common physical
infrastructure. The transport protocol over the physical data center network is IP plus UDP.
VXLAN defines a MAC-in-UDP encapsulation scheme where the original Layer 2 frame has a VXLAN
header added and is then placed in a UDP-IP packet. With this MAC-in-UDP encapsulation, VXLAN tunnels
Layer 2 network over Layer 3 network.
VXLAN uses an 8-byte VXLAN header that consists of a 24-bit VNID and a few reserved bits. The VXLAN
header together with the original Ethernet frame goes in the UDP payload. The 24-bit VNID is used to identify
Layer 2 segments and to maintain Layer 2 isolation between the segments. With all 24 bits in VNID, VXLAN
can support 16 million LAN segments.

VXLAN Tunnel Endpoint

VXLAN uses VXLAN tunnel endpoint (VTEP) devices to map tenants’ end devices to VXLAN segments
and to perform VXLAN encapsulation and de-encapsulation. Each VTEP function has two interfaces: One
is a switch interface on the local LAN segment to support local endpoint communication through bridging,
and the other is an IP interface to the transport IP network.
The IP interface has a unique IP address that identifies the VTEP device on the transport IP network known
as the infrastructure VLAN. The VTEP device uses this IP address to encapsulate Ethernet frames and transmits
the encapsulated packets to the transport network through the IP interface. A VTEP device also discovers the
remote VTEPs for its VXLAN segments and learns remote MAC Address-to-VTEP mappings through its IP
interface.
The VXLAN segments are independent of the underlying network topology; conversely, the underlying IP
network between VTEPs is independent of the VXLAN overlay. It routes the encapsulated packets based on
the outer IP address header, which has the initiating VTEP as the source IP address and the terminating VTEP
as the destination IP address.

VXLAN Packet Forwarding Flow

VXLAN uses stateless tunnels between VTEPs to transmit traffic of the overlay Layer 2 network through the
Layer 3 transport network.

Cisco Nexus 9000 as Hardware-Based VXLAN Gateway

VXLAN is a new technology for virtual data center overlays and is being adopted in data center networks
more and more, especially for virtual networking in the hypervisor for virtual machine-to-virtual machine
communication. However, data centers are likely to contain devices that are not capable of supporting VXLAN,
such as legacy hypervisors, physical servers, and network services appliances, such as physical firewalls and
load balancers, and storage devices, etc. Those devices need to continue to reside on classic VLAN segments.
It is not uncommon that virtual machines in a VXLAN segment need to access services provided by devices
in a classic VLAN segment. This type of VXLAN-to-VLAN connectivity is enabled by using a VXLAN
gateway.

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
4
 Overview
vPC Consistency Check for vPC VTEPs

A VXLAN gateway is a VTEP device that combines a VXLAN segment and a classic VLAN segment into
one common Layer 2 domain.
A Cisco Nexus 9000 Series Switch can function as a hardware-based VXLAN gateway. It seamlessly connects
VXLAN and VLAN segments as one forwarding domain across the Layer 3 boundary without sacrificing
forwarding performance. The Cisco Nexus 9000 Series eliminates the need for an additional physical or virtual
device to be the gateway. The hardware-based encapsulation and de-encapsulation provides line-rate
performance for all frame sizes.

vPC Consistency Check for vPC VTEPs

The vPC consistency check is a mechanism used by the two switches configured as a vPC pair to exchange
and verify their configuration compatibility. Consistency checks are performed to ensure that NVE
configurations and VN-Segment configurations are identical across vPC peers. This check is essential for the
correct operation of vPC functions.

Parameter vPC Check Type Description

VLAN-VNI mapping Type-1-nongraceful Brings down the affected VLANs on vPC ports on both sides.

VTEP-Member-VNI Type-1-nongraceful Member VNIs must be the same on both nodes. VNIs that are
not common bring down the corresponding VLANs on vPC ports
on both sides.

VTEP-emulated IP Type-1-nongraceful If an emulated IP address is not the same on both nodes, all
gateway vPC ports on one side (secondary) are brought down.
Alternatively, one side of all vPC ports is brought down.
The VTEP source loopback on the vPC secondary is also brought
down if the emulated IP address is not the same on both sides.

NVE Oper State Type-1-nongraceful The NVE needs to be in the oper UP state on both sides for the
vPC consistency check.
If both VTEPs are not in the OPER_UP state, the secondary leg
is brought down along with the VTEP source loopback on the
vPC secondary.

VLAN-to-VXLAN VN-segment mapping is a type-1 consistency check parameter. The two VTEP switches
are required to have identical mappings. VLANs that have mismatched VN-segment mappings will be
suspended. When the graceful consistency check is disabled and problematic VLANs arise, the primary vPC
switch and the secondary vPC switch will suspend the VLANs.
The following situations are detected as inconsistencies:
• One switch has a VLAN mapped to a VN-segment (VXLAN VNI), and the other switch does not have
a mapping for the same VLAN.
• The two switches have a VLAN mapped to different VN-segments.

The following is an example of displaying vPC information:

sys06-tor3# sh vpc consistency-parameters global

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
5
 Overview
vPC Consistency Check for vPC VTEPs

Legend:
Type 1 : vPC will be suspended in case of mismatch

Name Type Local Value Peer Value

------------- ---- ---------------------- -----------------------
Vlan to Vn-segment Map 1 1024 Relevant Map(s) 1024 Relevant Map(s)
STP Mode 1 MST MST
STP Disabled 1 None None
STP MST Region Name 1 "" ""
STP MST Region Revision 1 0 0
STP MST Region Instance to 1
VLAN Mapping
STP Loopguard 1 Disabled Disabled
STP Bridge Assurance 1 Enabled Enabled
STP Port Type, Edge 1 Normal, Disabled, Normal, Disabled,
BPDUFilter, Edge BPDUGuard Disabled Disabled
STP MST Simulate PVST 1 Enabled Enabled
Nve Oper State, Secondary 1 Up, 4.4.4.4 Up, 4.4.4.4
IP
Nve Vni Configuration 1 10002-11025 10002-11025
Allowed VLANs - 1-1025 1-1025
Local suspended VLANs - - -

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
6
 CHAPTER 3
Configuring VXLAN
This chapter contains the following sections:

• Information About VXLAN, page 7

• Configuring VXLAN, page 13
• Verifying the VXLAN Configuration, page 15
• Example of VXLAN Bridging Configuration, page 17

Information About VXLAN

Guidelines and Limitations for VXLAN
VXLAN has the following guidelines and limitations:
• Bind NVE to a loopback address that is separate from other loopback addresses that are required by
Layer 3 protocols. A best practice is to use a dedicated loopback address for VXLAN. This best practice
should be applied not only for the VPC VXLAN deployment, but for all VXLAN deployments.
• show commands with the internal keyword are not supported.
• FEX ports do not support IGMP snooping on VXLAN VLANs.
• Beginning with Cisco NX-OS Release 7.0(3)I4(2), VXLAN is supported for the Cisco Nexus
93108TC-EX and 93180YC-EX switches and for Cisco Nexus 9500 Series switches with the X9732C-EX
line card.
• DHCP snooping (Dynamic Host Configuration Protocol snooping) is not supported on VXLAN VLANs.
• The device cannot be an IP transient device and a VXLAN terminating device for the same VXLAN or
VXLANs that share the same transport multicast group.
• SPAN TX for VXLAN encapsulated traffic is not supported for the Layer 3 uplink interface.
• RACLs are not supported on Layer 3 uplinks for VXLAN traffic. Egress VACLs support is not available
for de-capsulated packets in the network to access direction on the inner payload.
As a best practice, use PACLs/VACLs for the access to the network direction.

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
7
 Configuring VXLAN
Guidelines and Limitations for VXLAN

• QoS classification is not supported for VXLAN traffic in the network to access direction on the Layer
3 uplink interface.
• The QoS buffer-boost feature is not applicable for VXLAN traffic.
• Only one NVE (Network Virtualization Edge) interface on a switch.
• SNMP is not supported on the NVE interface.
• VXLAN SVI uplinks are not supported over underlying Layer 2 VPC ports.
• A VXLAN SVI uplink VLAN cannot be a member of the peer-link.
• VTEP does not support Layer 3 subinterface uplinks. In addition, non-VXLAN subinterface VLANs
cannot be shared with VXLAN VLANs.
• For 6.1(2)I3(4) and earlier, VXLAN does not support consistency checks.
• Point to multipoint Layer 3 and SVI uplinks are not supported. Since both uplink types can only be
enabled point-to-point, they cannot span across more than two switches.
• A FEX host interface port is not supported for a VLAN that is extended with VXLAN.

Considerations for VXLAN Deployment

• A loopback address is required when using the source-interface config command. The loopback address
represents the local VTEP IP.
• To establish IP multicast routing in the core, IP multicast configuration, PIM configuration, and RP
configuration is required.
• VTEP to VTEP unicast reachability can be configured through any IGP protocol.
•
• When configuring BGP-EVPN on Cisco Nexus 9300-EX switches and Cisco Nexus 9500 switches with
N9K-X9732C-EX line cards, use the system routing template-vxlan-scale command. Performing this
step requires a reload of the switch. This command is not applicable on Cisco Nexus 9200 switches,
Cisco Nexus 9300 switches, and Cisco Nexus 9500 switches with N9K-X9564PX, N9K-X9564TX, and
N9K-X9536PQ line cards.
• As a best practice when changing the IP address of a VTEP device, shut the NVE interface before
changing the IP address.
• Configuring an Rendezvous Point (RP) on a leaf node is not supported. As a best practice, the RP for
the multicast group should be configured only on the spine layer. Use the anycast RP for RP load
balancing and redundancy.
The following is an example of an anycast RP configuration on spines:
ip pim rp-address 1.1.1.10 group-list 224.0.0.0/4
ip pim anycast-rp 1.1.1.10 1.1.1.1
ip pim anycast-rp 1.1.1.10 1.1.1.2

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
8
Configuring VXLAN
Guidelines and Limitations for VXLAN

Note • 1.1.1.10 is the anycast RP IP address that is configured on all RPs participating in
the anycast RP set.
• 1.1.1.1 is the local RP IP.
• 1.1.1.2 is the peer RP IP.

VPC Considerations for VXLAN Deployment

• Bind NVE to a loopback address that is separate from other loopback addresses that are required by
Layer 3 protocols. A best practice is to use a dedicated loopback address for VXLAN.
• On VPC VXLAN, it is recommended to increase the delay restore interface-vlan timer under the VPC
configuration, if the number of SVIs are scaled up. For example, if there are 1000 VNIs with 1000 SVIs,
it is recommended to increase the delay restore interface-vlan timer to 45 Seconds.
• The loopback address used by NVE needs to be configured to have a primary IP address and a secondary
IP address.
The secondary IP address is used for all VxLAN traffic that includes multicast and unicast encapsulated
traffic.
• VPC peers must have identical configurations.
◦Consistent VLAN to VN-segment mapping.
◦Consistent NVE1 binding to the same loopback interface
◦Using the same secondary IP address.
◦Using different primary IP addresses.

◦Consistent VNI to group mapping.

• For multicast, the VPC node that receives the (S, G) join from the RP (rendezvous point) becomes the
DF (designated forwarder). On the DF node, encap routes are installed for multicast.
Decap routes are installed based on the election of a decapper from between the VPC primary node and
the VPC secondary node. The winner of the decap election is the node with the least cost to the RP.
However, if the cost to the RP is the same for both nodes, the VPC primary node is elected.
The winner of the decap election has the decap mroute installed. The other node does not have a decap
route installed.
• On a VPC device, BUM traffic (broadcast, unknown-unicast, and multicast traffic) from hosts is replicated
on the peer-link. A copy is made of every native packet and each native packet is sent across the peer-link
to service orphan-ports connected to the peer VPC switch.
To prevent traffic loops in VXLAN networks, native packets ingressing the peer-link cannot be sent to
an uplink. However, if the peer switch is the encapper, the copied packet traverses the peer-link and is
sent to the uplink.

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
9
 Configuring VXLAN
Guidelines and Limitations for VXLAN

Note Each copied packet is sent on a special internal VLAN (VLAN 4041).

• When peer-link is shut, the loopback interface used by NVE on the VPC secondary is brought down
and the status is Admin Shut. This is done so that the route to the loopback is withdrawn on the upstream
and that the upstream can divert all traffic to the VPC primary.

Note Orphans connected to the VPC secondary will experience loss of traffic for the period
that the peer-link is shut. This is similar to Layer 2 orphans in a VPC secondary of a
traditional VPC setup.

• When peer-link is no-shut, the NVE loopback address is brought up again and the route is advertised
upstream, attracting traffic.
• For VPC, the loopback interface has 2 IP addresses: the primary IP address and the secondary IP address.
The primary IP address is unique and is used by Layer 3 protocols.
The secondary IP address on loopback is necessary because the interface NVE uses it for the VTEP IP
address. The secondary IP address must be same on both vPC peers.
• The VPC peer-gateway feature must be enabled on both peers.
As a best practice, use peer-switch, peer gateway, ip arp sync, ipv6 nd sync configurations for improved
convergence in VPC topologies.
In addition, increase the STP hello timer to 4 seconds to avoid unnecessary TCN generations when VPC
role changes occur.
The following is an example (best practice) of a VPC configuration:

switch# sh ru vpc

version 6.1(2)I3(1)
feature vpc
vpc domain 2
peer-switch
peer-keepalive destination 172.29.206.65 source 172.29.206.64
peer-gateway
ipv6 nd synchronize
ip arp synchronize

• On a VPC pair, shutting down NVE or NVE loopback on one of the VPC nodes is not a supported
configuration. This means that traffic failover on one-side NVE shut or one-side loopback shut is not
supported.
• When the NVE or loopback is shut in VPC configurations:
◦If the NVE or loopback is shut only on the primary VPC switch, the global VxLAN VPC consistency
checker fails. Then the NVE, loopback, and VPCs are taken down on the secondary VPC switch.
◦If the NVE or loopback is shut only on the secondary VPC switch, the global VXLAN VPC
consistency checker fails. Then the NVE, loopback, and secondary VPC are brought down on the
secondary. Traffic continues to flow through the primary VPC switch.

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
10
Configuring VXLAN
Guidelines and Limitations for VXLAN

As a best practice, you should keep both the NVE and loopback up on both the primary and secondary
VPC switches.
• Redundant anycast RPs configured in the network for multicast load-balancing and RP redundancy are
supported on VPC VTEP topologies.
• Enabling vpc peer-gateway configuration is mandatory. For peer-gateway functionality, at least one SVI
is required to be enabled across peer-link and also configured with PIM. This provides a backup path
in the case when VTEP loses complete connectivity to the spine. Remote peer reachability is re-routed
over peer-link in this case.
The following is an example of SVI with PIM enabled:

swithch# sh ru int vlan 2

interface Vlan2
description special_svi_over_peer-link
no shutdown
ip address 30.2.1.1/30
ip pim sparse-mode

// example config for backup SVI:

interface Vlan2000
description backup_svi_over_peer-link //change “special” into “backup”
no shutdown
no ip redirects
ip address 20.20.20.1/24
no ipv6 redirects
ip router ospf 1 area 0.0.0.0
ip pim sparse-mode
ip igmp static-oif route-map match-mcast-groups

route-map match-mcast-groups permit 1

match ip multicast group 225.1.1.1/32

Note In BUD node topologies, the backup SVI needs to be added as a static OIF for each
underlay multicast group.

Note The SVI must be configured on both VPC peers and requires PIM to be enabled.

• As a best practice when changing the secondary IP address of an anycast VPC VTEP, the NVE interfaces
on both the VPC primary and the VPC secondary should be shut before the IP changes are made.
• DHCP relay is supported when the DHCP server is reachable through a default VRF. However, DHCP
relay is not supported when the DHCP client and DHCP server are in the same non-default VRF.

Network Considerations for VXLAN Deployments

• MTU Size in the Transport Network
Due to the MAC-to-UDP encapsulation, VXLAN introduces 50-byte overhead to the original frames.
Therefore, the maximum transmission unit (MTU) in the transport network needs to be increased by 50
bytes. If the overlays use a 1500-byte MTU, the transport network needs to be configured to accommodate

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
11
 Configuring VXLAN
Guidelines and Limitations for VXLAN

1550-byte packets at a minimum. Jumbo-frame support in the transport network is required if the overlay
applications tend to use larger frame sizes than 1500 bytes.
• ECMP and LACP Hashing Algorithms in the Transport Network
As described in a previous section, Cisco Nexus 9000 Series Switches introduce a level of entropy in
the source UDP port for ECMP and LACP hashing in the transport network. As a way to augment this
implementation, the transport network uses an ECMP or LACP hashing algorithm that takes the UDP
source port as an input for hashing, which achieves the best load-sharing results for VXLAN encapsulated
traffic.
• Multicast Group Scaling
The VXLAN implementation on Cisco Nexus 9000 Series Switches uses multicast tunnels for broadcast,
unknown unicast, and multicast traffic forwarding. Ideally, one VXLAN segment mapping to one IP
multicast group is the way to provide the optimal multicast forwarding. It is possible, however, to have
multiple VXLAN segments share a single IP multicast group in the core network. VXLAN can support
up to 16 million logical Layer 2 segments, using the 24-bit VNID field in the header. With one-to-one
mapping between VXLAN segments and IP multicast groups, an increase in the number of VXLAN
segments causes a parallel increase in the required multicast address space and the amount of forwarding
states on the core network devices. At some point, multicast scalability in the transport network can
become a concern. In this case, mapping multiple VXLAN segments to a single multicast group can
help conserve multicast control plane resources on the core devices and achieve the desired VXLAN
scalability. However, this mapping comes at the cost of suboptimal multicast forwarding. Packets
forwarded to the multicast group for one tenant are now sent to the VTEPs of other tenants that are
sharing the same multicast group. This causes inefficient utilization of multicast data plane resources.
Therefore, this solution is a trade-off between control plane scalability and data plane efficiency.
Despite the suboptimal multicast replication and forwarding, having multiple-tenant VXLAN networks
to share a multicast group does not bring any implications to the Layer 2 isolation between the tenant
networks. After receiving an encapsulated packet from the multicast group, a VTEP checks and validates
the VNID in the VXLAN header of the packet. The VTEP discards the packet if the VNID is unknown
to it. Only when the VNID matches one of the VTEP’s local VXLAN VNIDs, does it forward the packet
to that VXLAN segment. Other tenant networks will not receive the packet. Thus, the segregation
between VXLAN segments is not compromised.

Considerations for the Transport Network

The following are considerations for the configuration of the transport network:
• On the VTEP device:
◦Enable and configure IP multicast.
◦Create and configure a loopback interface with a /32 IP address.
(For vPC VTEPs, you must configure primary and secondary /32 IP addresses.)
◦Enable IP multicast on the loopback interface.
◦Advertise the loopback interface /32 addresses through the routing protocol (static route) that runs
in the transport network.
◦Enable IP multicast on the uplink outgoing physical interface.

• Throughout the transport network:

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
12
 Configuring VXLAN
Configuring VXLAN

◦Enable and configure IP multicast.

Configuring VXLAN
Enabling VXLANs
SUMMARY STEPS

1. configure terminal
2. [no] feature nv overlay
3. [no] feature vn-segment-vlan-based
4. (Optional) copy running-config startup-config

DETAILED STEPS

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.

Step 2 [no] feature nv overlay Enables the VXLAN feature.

Step 3 [no] feature vn-segment-vlan-based Configures the global mode for all VXLAN bridge domains.

Step 4 copy running-config startup-config (Optional)

Saves the change persistently through reboots and restarts by
copying the running configuration to the startup configuration.

Mapping VLAN to VXLAN VNI

SUMMARY STEPS

1. configure terminal
2. vlan vlan-id
3. vn-segment vnid
4. exit

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
13
 Configuring VXLAN
Creating and Configuring an NVE Interface and Associate VNIs

DETAILED STEPS

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.

Step 2 vlan vlan-id Specifies VLAN.

Step 3 vn-segment vnid Specifies VXLAN VNID (Virtual Network Identifier)

Step 4 exit Exit configuration mode.

Creating and Configuring an NVE Interface and Associate VNIs

An NVE interface is the overlay interface that terminates VXLAN tunnels.
You can create and configure an NVE (overlay) interface with the following:

SUMMARY STEPS

1. configure terminal
2. interface nve x
3. source-interface src-if
4. member vni vni
5. mcast-group start-address [end-address]

DETAILED STEPS

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.

Step 2 interface nve x Creates a VXLAN overlay interface that terminates VXLAN tunnels.
Note Only 1 NVE interface is allowed on the
switch.
Step 3 source-interface src-if The source interface must be a loopback interface that is configured on the
switch with a valid /32 IP address. This /32 IP address must be known by the
transient devices in the transport network and the remote VTEPs. This is
accomplished by advertising it through a dynamic routing protocol in the
transport network.

Step 4 member vni vni Associate VXLAN VNIs (Virtual Network Identifiers) with the NVE interface.

Step 5 mcast-group start-address Assign a multicast group to the VNIs.

[end-address] Note used only for BUM
traffic

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
14
 Configuring VXLAN
Disabling VXLANs

Command or Action Purpose

Disabling VXLANs
SUMMARY STEPS

1. configure terminal
2. no feature vn-segment-vlan-based
3. no feature nv overlay
4. (Optional) copy running-config startup-config

DETAILED STEPS

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.

Step 2 no feature vn-segment-vlan-based Disables the global mode for all VXLAN bridge domains

Step 3 no feature nv overlay Disables the VXLAN feature.

Step 4 copy running-config startup-config (Optional)

Saves the change persistently through reboots and restarts by
copying the running configuration to the startup configuration.

Verifying the VXLAN Configuration

To display the VXLAN configuration information, enter one of the following commands:

Table 2: Display VXLAN configuration information (Release 6.1(2)I3(4) and earlier)

Command Purpose
show logging level nve Displays logging level.

show tech-support nve Displays related NVE tech-support information.

show run interface nve x Displays NVE overlay interface configuration.

show nve interface Displays NVE overlay interface status.

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
15
 Configuring VXLAN
Verifying the VXLAN Configuration

Command Purpose
show nve peers Displays NVE peer status.

show nve peers peer_IP_address interface Displays per NVE peer statistics.
interface_ID counters

clear nve peers peer_IP_address interface Clears per NVE peer statistics.
interface_ID counters

clear nve peer-ip peer-ip-address Clears stale NVE peers.

Stale NVE peers are peers that do not have MAC
addresses learnt behind them.

show nve vni Displays VXLAN VNI status.

show nve vni vni_number counters Displays per VNI statistics.

clear nve vni vni_number counters Clears per VNI statistics.

show nve vxlan-params Displays VXLAN parameters, such as VXLAN

destination or UDP port.

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
16
 Configuring VXLAN
Example of VXLAN Bridging Configuration

Example of VXLAN Bridging Configuration

• An example of a loopback interface configuration and routing protocol configuration:

Figure 1: VXLAN topology for VTEP

◦Nexus 9000 VTEP-1 configuration:

switch-vtep-1(config)# feature ospf
switch-vtep-1(config)# feature pim
switch-vtep-1(config)# router ospf 1
switch-vtep-1(config-router)# router-id 100.100.100.1
switch-vtep-1(config)# ip pim rp-address 10.1.1.1 group-list 224.0.0.0/4
switch-vtep-1(config)# interface loopback0
switch-vtep-1(config-if)# ip address 100.100.100.1/32
switch-vtep-1(config-if)# ip router ospf 1 area 0.0.0.0
switch-vtep-1(config-if)# ip pim sparse-mode
switch-vtep-1(config)# interface e2/1
switch-vtep-1(config-if)# ip address 20.1.1.1/30
switch-vtep-1(config-if)# ip router ospf 1 area 0.0.0.0
switch-vtep-1(config-if)# ip pim sparse-mode
switch-vtep-1(config)# feature nv overlay
switch-vtep-1(config)# feature vn-segment-vlan-based
switch-vtep-1(config)# interface e1/1
switch-vtep-1(config-if)# switchport
switch-vtep-1(config-if)# switchport access vlan 10
switch-vtep-1(config-if)# no shutdown
switch-vtep-1(config)# interface nve1
switch-vtep-1(config-if)# no shutdown
switch-vtep-1(config-if)# source-interface loopback0

switch-vtep-1(config-if)# member vni 10000 mcast-group 230.1.1.1

switch-vtep-1(config)# vlan 10
switch-vtep-1(config-vlan)# vn-segment 10000
switch-vtep-1(config-vlan)# exit

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
17
 Configuring VXLAN
Example of VXLAN Bridging Configuration

◦Nexus 9000 VTEP-2 configuration:

switch-vtep-2(config)# feature ospf
switch-vtep-2(config)# feature pim
switch-vtep-2(config)# router ospf 1
switch-vtep-2(config-router)# router-id 100.100.100.2
switch-vtep-2(config)# ip pim rp-address 10.1.1.1 group-list 224.0.0.0/4
switch-vtep-2(config)# interface loopback0
switch-vtep-2(config-if)# ip address 100.100.100.2/32
switch-vtep-2(config-if)# ip router ospf 1 area 0.0.0.0
switch-vtep-2(config-if)# ip pim sparse-mode
switch-vtep-2(config)# interface e2/1
switch-vtep-2(config-if)# ip address 30.1.1.1/30
switch-vtep-2(config-if)# ip router ospf 1 area 0.0.0.0
switch-vtep-2(config-if)# ip pim sparse-mode
switch-vtep-2(config)# feature nv overlay
switch-vtep-2(config)# feature vn-segment-vlan-based
switch-vtep-2(config)# interface e1/1
switch-vtep-2(config-if)# switchport
switch-vtep-2(config-if)# switchport access vlan 10
switch-vtep-2(config-if)# no shutdown
switch-vtep-2(config)# interface nve1
switch-vtep-2(config-if)# no shutdown
switch-vtep-2(config-if)# source-interface loopback0

switch-vtep-2(config-if)# member vni 10000 mcast-group 230.1.1.1

switch-vtep-2(config)# vlan 10
switch-vtep-2(config-vlan)# vn-segment 10000
switch-vtep-2(config-vlan)# exit

• An example of an ingress replication topology:

Figure 2: Ingress Replication topology

◦Nexus 9000 VTEP-1 configuration:

switch-vtep-1(config)# feature ospf

switch-vtep-1(config)# router ospf 1
switch-vtep-1(config-router)# router-id 200.200.8.8
switch-vtep-1(config)# interface loopback0
switch-vtep-1(config-if)# ip address 200.200.8.8/32
switch-vtep-1(config-if)# ip router ospf 1 area 0.0.0.0
switch-vtep-1(config)# interface e2/1

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
18
Configuring VXLAN
Example of VXLAN Bridging Configuration

switch-vtep-1(config-if)# ip address 20.1.1.1/30

switch-vtep-1(config-if)# ip router ospf 1 area 0.0.0.0
switch-vtep-1(config-if)# ip pim sparse-mode
switch-vtep-1(config)# feature nv overlay
switch-vtep-1(config)# feature vn-segment-vlan-based
switch-vtep-1(config)# interface e1/1
switch-vtep-1(config-if)# switchport
switch-vtep-1(config-if)# switch port mode trunk
switch-vtep-1(config-if)# switch port allowed vlan 11-12
switch-vtep-1(config-if)# no shutdown
switch-vtep-1(config)# vlan 11
switch-vtep-1(config-vlan)# vn-segment 10011
switch-vtep-1(config)# vlan 12
switch-vtep-1(config-vlan)# vn-segment 10012
switch-vtep-1(config)# interface nve1
switch-vtep-1(config-if)# no shutdown
switch-vtep-1(config-if)# source-interface loopback0
switch-vtep-1(config-if)# member vni 10011
switch-vtep-1(config-if)# ingress-replication protocol static
switch-vtep-1(config-if)# peer_ip 200.200.9.9
switch-vtep-1(config-if)# member vni 10012
switch-vtep-1(config-if)# ingress-replication protocol static
switch-vtep-1(config-if)# peer_ip 200.200.9.9
switch-vtep-1(config-vlan)# exit

switch-vtep-1# show nve vni ingress-replication

Interface VNI show nve vni ingress-replication
Interface VNI Replication List Up Time
--------- -------- ----------------- -------

nve1 10011 200.200.9.9 07:39:51

nve1 10012 200.200.9.9 07:39:40

◦Nexus 9000 VTEP-2 configuration:

switch-vtep-2(config)# feature ospf

switch-vtep-2(config)# router ospf 1
switch-vtep-2(config-router)# router-id 200.200.9.9
switch-vtep-2(config)# interface loopback0
switch-vtep-2(config-if)# ip address 200.200.9.9/32
switch-vtep-2(config-if)# ip router ospf 1 area 0.0.0.0
switch-vtep-2(config)# interface e2/1
switch-vtep-2(config-if)# ip address 30.1.1.1/30
switch-vtep-2(config-if)# ip router ospf 1 area 0.0.0.0
switch-vtep-2(config-if)# ip pim sparse-mode
switch-vtep-2(config)# feature nv overlay
switch-vtep-2(config)# feature vn-segment-vlan-based
switch-vtep-2(config)# interface e1/1
switch-vtep-2(config-if)# switchport
switch-vtep-2(config-if)# switch port mode trunk
switch-vtep-2(config-if)# switch port allowed vlan 11-12
switch-vtep-2(config-if)# no shutdown
switch-vtep-2(config)# vlan 11
switch-vtep-2(config-vlan)# vn-segment 10011
switch-vtep-2(config)# vlan 12
switch-vtep-2(config-vlan)# vn-segment 10012
switch-vtep-2(config)# interface nve1
switch-vtep-2(config-if)# no shutdown
switch-vtep-2(config-if)# source-interface loopback0
switch-vtep-2(config-if)# member vni 10011
switch-vtep-2(config-if)# ingress-replication protocol static
switch-vtep-2(config-if)# peer_ip 200.200.8.8

switch-vtep-2(config-if)# member vni 10012

switch-vtep-2(config-if)# ingress-replication protocol static
switch-vtep-2(config-if)# peer_ip 200.200.8.8

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
19
 Configuring VXLAN
Example of VXLAN Bridging Configuration

switch-vtep-2(config-vlan)# exit

switch-vtep-2# show nve vni ingress-replication

Interface VNI Replication List Up Time
--------- -------- ----------------- -------

nve1 10011 200.200.8.8 07:42:23

200.200.10.10 07:42:23

nve1 10012 200.200.8.8 07:42:23

• For a vPC VTEP configuration, the loopback address requires a secondary IP.
An example of a vPC VTEP configuration:

Figure 3: VXLAN topology for vPC VTEP

◦Nexus 9000 VTEP-1 configuration:

switch-vtep-1(config)# feature nv overlay
switch-vtep-1(config)# feature vn-segment-vlan-based
switch-vtep-1(config)# feature ospf
switch-vtep-1(config)# feature pim
switch-vtep-1(config)# router ospf 1
switch-vtep-1(config-router)# router-id 200.200.200.1
switch-vtep-1(config)# ip pim rp-address 10.1.1.1 group-list 224.0.0.0/4
switch-vtep-1(config)# interface loopback0
switch-vtep-1(config-if)# ip address 200.200.200.1/32
switch-vtep-1(config-if)# ip address 100.100.100.1/32 secondary
switch-vtep-1(config-if)# ip router ospf 1 area 0.0.0.0
switch-vtep-1(config-if)# ip pim sparse-mode
switch-vtep-1(config)# interface e2/1
switch-vtep-1(config-if)# ip address 20.1.1.1/30

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
20
Configuring VXLAN
Example of VXLAN Bridging Configuration

switch-vtep-1(config-if)# ip router ospf 1 area 0.0.0.0

switch-vtep-1(config-if)# ip pim sparse-mode
switch-vtep-1(config)# interface port-channel 10
switch-vtep-1(config-if)# vpc 10
switch-vtep-1(config-if)# switchport
switch-vtep-1(config-if)# switchport mode access
switch-vtep-1(config-if)# switchport access vlan 10
switch-vtep-1(config-if)# no shutdown
switch-vtep-1(config)# interface e1/1
switch-vtep-1(config-if)# channel-group 10 mode active
switch-vtep-1(config-if)# no shutdown
switch-vtep-1(config)# interface nve1
switch-vtep-1(config-if)# no shutdown
switch-vtep-1(config-if)# source-interface loopback0

switch-vtep-1(config-if)# member vni 10000 mcast-group 230.1.1.1

switch-vtep-1(config)# vlan 10
switch-vtep-1(config-vlan)# vn-segment 10000
switch-vtep-1(config-vlan)# exit

◦Nexus 9000 VTEP-2 configuration:

switch-vtep-2(config)# feature nv overlay
switch-vtep-2(config)# feature vn-segment-vlan-based
switch-vtep-2(config)# feature ospf
switch-vtep-2(config)# feature pim
switch-vtep-2(config)# router ospf 1
switch-vtep-2(config-router)# router-id 200.200.200.2
switch-vtep-2(config)# ip pim rp-address 10.1.1.1 group-list 224.0.0.0/4
switch-vtep-2(config)# interface loopback0
switch-vtep-2(config-if)# ip address 200.200.200.2/32
switch-vtep-2(config-if)# ip address 100.100.100.1/32 secondary
switch-vtep-2(config-if)# ip router ospf 1 area 0.0.0.0
switch-vtep-2(config-if)# ip pim sparse-mode
switch-vtep-2(config)# interface e2/1
switch-vtep-2(config-if)# ip address 20.1.1.5/30
switch-vtep-2(config-if)# ip router ospf 1 area 0.0.0.0
switch-vtep-2(config-if)# ip pim sparse-mode
switch-vtep-2(config)# interface port-channel 10
switch-vtep-2(config-if)# vpc 10
switch-vtep-2(config-if)# switchport
switch-vtep-2(config-if)# switchport mode access
switch-vtep-2(config-if)# switchport access vlan 10
switch-vtep-2(config-if)# no shutdown
switch-vtep-2(config)# interface e1/1
switch-vtep-2(config-if)# channel-group 10 mode active
switch-vtep-2(config-if)# no shutdown
switch-vtep-2(config)# interface nve1
switch-vtep-2(config-if)# no shutdown
switch-vtep-2(config-if)# source-interface loopback0

switch-vtep-2(config-if)# member vni 10000 mcast-group 230.1.1.1

switch-vtep-2(config)# vlan 10
switch-vtep-2(config-vlan)# vn-segment 10000
switch-vtep-2(config-vlan)# exit

◦Nexus 9000 VTEP-3 configuration:

switch-vtep-3(config)# feature nv overlay
switch-vtep-3(config)# feature vn-segment-vlan-based
switch-vtep-3(config)# feature ospf
switch-vtep-3(config)# feature pim
switch-vtep-3(config)# router ospf 1
switch-vtep-3(config-router)# router-id 100.100.100.2
switch-vtep-3(config)# ip pim rp-address 10.1.1.1 group-list 224.0.0.0/4
switch-vtep-3(config)# interface loopback0
switch-vtep-3(config-if)# ip address 100.100.100.2/32
switch-vtep-3(config-if)# ip router ospf 1 area 0.0.0.0
switch-vtep-3(config-if)# ip pim sparse-mode
switch-vtep-3(config)# interface e2/1
switch-vtep-3(config-if)# ip address 30.1.1.1/30

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
21
 Configuring VXLAN
Example of VXLAN Bridging Configuration

switch-vtep-3(config-if)# ip router ospf 1 area 0.0.0.0

switch-vtep-3(config-if)# ip pim sparse-mode

switch-vtep-3(config)# interface e1/1

switch-vtep-3(config-if)# switchport
switch-vtep-3(config-if)# switchport access vlan 10
switch-vtep-3(config-if)# no shutdown
switch-vtep-3(config)# interface nve1
switch-vtep-3(config-if)# no shutdown
switch-vtep-3(config-if)# source-interface loopback0

switch-vtep-3(config-if)# member vni 10000 mcast-group 230.1.1.1

switch-vtep-3(config)# vlan 10
switch-vtep-3(config-vlan)# vn-segment 10000
switch-vtep-3(config-vlan)# exit

Note The secondary IP is used by the emulated VTEP for VXLAN.

Note Ensure that all configurations are identical between the VPC primary and VPC secondary.

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 6.x
22