Cisco Nexus 7000 Series LAB

NATALE RUELLO ROBERT STARMER Technical Marketing - Data Center Business Unit

2009 Cisco Systems, Inc. All rights reserved

1 of 44

Nexus 7000
The Cisco Nexus 7000 Series is a modular data center class series of switching systems designed for highly scalable end-to-end 10 Gigabit Ethernet networks. The Cisco Nexus 7000 Series is purpose built for the data center and has many unique features and capabilities designed specifically for the most mission critical place in the network, the Data Center.

Cisco NX-OS, a state-of-the-art operating system, powers the Cisco Nexus 7000 Platform. Cisco NX-OS is a data center-class operating system built with modularity, resiliency, and serviceability at its foundation. Drawing on its Cisco IOS and Cisco SAN-OS heritage, Cisco NX-OS helps ensure continuous availability and sets the standard for mission-critical data center environments.

Lab Objectives
This instructor-led hands-on lab will introduce the participants to the NX-OS, the operating system powering the Nexus family switches. The participants will be exposed to the configuration of some of the new features present in NX-OS. The lab will also focus on some of the aspects that differentiate NX-OS from the classical IOS.

2009 Cisco Systems, Inc. All rights reserved

2 of 44

Lab Procedure
The Lab consists of 10 PODs. Each single POD represents a typical but simplified 3-tier Data Center design. The core consist of a Catalyst 6500, the aggregation layer consists of two Nexus 7000, while an ESX server and a Nexus 5000 compose the access layer. In more details, the aggregation layer (on which all the configuration for this lab is performed) is formed by two N7K-C7010 with one N7K-M148GT-12 and one N7K-M132XP-12 card each. These two systems run a pre-release version of NX-OS 4.1(3).

A group of two students is assigned to each Pod. Each student will be able to configure his own Nexus 7000 aggregation device. During the Lab procedure the students will go through the following steps: System Verification Management VRF Concept and Basic Connectivity CLI Tips Role Based Access Control (RBAC) Configuration Rollback Links Up and Spanning Tree Protocol HSRP Virtual Port Channel (vPC) vPC Failure Scenario OSPF Stateful Process Restart Wireshark Virtual Device Contexts (VDCs)

2009 Cisco Systems, Inc. All rights reserved

3 of 44

Lab Topology and Access

The diagrams below represent the logical lab setup for the odd and the even pods.

Figure 1 Topology for the odd Pods (1,3,5,7,9)

2/2 1/13

6K

C6K-1
2/1 1/13 Po 10 2/1-2

N7K-1
Po 10

N7K-2 Student 2

Student 1

2/1-2

Aggregation
2/10 2/9 2/10 2/9

Access VMware ESX

1/1

1/2

N5K

Figure 2 Topology for the even Pods (2,4,6,8,10)

6K
2/2

C6K-1
2/1

N7K-1 Student 1

1/25 Po 10 2/17-18 2/25 Po 10 2/17-18

1/25

N7K-2 Student 2

2/26

2/26

Aggregation

2/25

VMware ESX

Access

1/1

1/2

N5K

2009 Cisco Systems, Inc. All rights reserved

4 of 44

The real lab looks quite different We are actually using the Virtual Device Context feature.

N7K-Aggr

N7K-Aggr

Pod 1

Pod 2

With your teammate decide which of the two Nexus 7000 aggregation devices you will be working on. Each POD will be used by a group of two students that will work within the PODs Virtual Device Context. All access to your POD devices is via the ESX VMware server that is available via Microsoft Remote Desktop Client access. Remote desktop access is defined in the Table 1. In order to connect to the Nexus device: 1. Open the Microsoft Remote Desktop Client on your workstation and point your machine to the Pods VM instance as shown in Table1.

Table 1 POD Access Details

POD Information POD1 POD2 POD3 POD4 POD5 POD6 POD7 POD8 POD9 VM Instance 128.107.222.196 128.107.222.197 128.107.222.198 128.107.222.199 128.107.222.200 128.107.222.201 128.107.222.202 128.107.222.203 128.107.222.204 Login/Password Student1/NXospod1-S1 Student2/NXospod1-S2 Student1/NXospod2-S1 Student2/NXospod2-S2 Student1/NXospod3-S1 Student2/NXospod3-S2 Student1/NXospod4-S1 Student2/NXospod4-S2 Student1/NXospod5-S1 Student2/NXospod5-S2 Student1/NXospod6-S1 Student2/NXospod6-S2 Student1/NXospod7-S1 Student2/NXospod7-S2 Student1/NXospod8-S1 Student2/NXospod8-S2 Student1/Nxospod9-S1 Student2/Nxospod9-S2

2009 Cisco Systems, Inc. All rights reserved

5 of 44

POD10

128.107.222.205

Student1/Nxospod10-S1 Student2/Nxospod10-S2

2. For your convenience you will find the puTTY connections on the Desktop. Double click on the connection of the Nexus 7000 you decided to use (make sure your teammate will work on the other device). Click YES on the warning message.

Step 1 System Verification

PLEASE NOTE: the interfaces referred in most of the output shown in these steps refer to Pod1. If you are on a different Pod please refer to Figure 1 and Figure 2 on page 4 for the correspondent interfaces for your Pod. During the entire duration of this lab we will be just logging into the management interface via ssh. However it is good to keep in mind that the Nexus 7000 requires console access to perform the initial configuration of the system. After performing the initial configuration, the system can be completely managed from the management interface. Lets start by checking the system and its configuration.

N7K-1-pod1-S1# show module Mod --1 2 5 6 Mod --1 2 5 6 Mod --1 2 5 6 Mod --1 2 5 6 Ports ----48 32 0 0 Module-Type -------------------------------10/100/1000 Mbps Ethernet Module 10 Gbps Ethernet Module Supervisor module-1X Supervisor module-1X Hw -----1.0 1.3 1.0 1.0 Serial-Num ---------JAB122101LL JAB1220009J JAB12250199 JAB1225018U Model -----------------N7K-M148GT-11 N7K-M132XP-12 N7K-SUP1 N7K-SUP1 Status -----------ok ok ha-standby active *

Sw -------------4.1(2.7) 4.1(2.7) 4.1(2.7) 4.1(2.7)

MAC-Address(es) -------------------------------------00-1b-54-c1-d1-08 to 00-1b-54-c1-d1-3c 00-1b-54-c1-9a-40 to 00-1b-54-c1-9a-64 00-22-55-77-5e-e0 to 00-22-55-77-5e-e8 00-22-55-77-5e-50 to 00-22-55-77-5e-58 Online Diag Status -----------------Pass Pass Pass Pass

Xbar Ports --- -----

Module-Type Model Status -------------------------------- ------------------ ------------

2009 Cisco Systems, Inc. All rights reserved

6 of 44

1 2 3 Xbar --1 2 3 Xbar --1 2 3

0 0 0

Fabric Module 1 Fabric Module 1 Fabric Module 1 Hw -----1.0 1.0 1.0

N7K-C7010-FAB-1 N7K-C7010-FAB-1 N7K-C7010-FAB-1

ok ok ok

Sw -------------NA NA NA

MAC-Address(es) -------------------------------------NA NA NA

Serial-Num ---------JAB122300ZH JAB122400QQ JAB122400QK

* this terminal session N7K-1-pod1-S1#

Lets check now the software the system is running.

N7K-1-pod1-S1# show version Cisco Nexus Operating System (NX-OS) Software TAC support: http://www.cisco.com/tac Copyright (c) 2002-2008, Cisco Systems, Inc. All rights reserved. The copyrights to certain works contained in this software are owned by other third parties and used and distributed under license. Certain components of this software are licensed under the GNU General Public License (GPL) version 2.0 or the GNU Lesser General Public License (LGPL) Version 2.1. A copy of each such license is available at http://www.opensource.org/licenses/gpl-2.0.php and http://www.opensource.org/licenses/lgpl-2.1.php Software BIOS: version 3.17.0 loader: version N/A NX-OS Version kickstart: version 4.1(3) system: version 4.1(3) BIOS compile time: 03/23/08 kickstart image file is: bootflash:/n7000-s1-kickstart.4.1.3.bin.S7 kickstart compile time: 1/15/2009 12:00:00 [12/20/2008 11:18:14] system image file is: bootflash:/n7000-s1-dk9.4.1.3.bin.S7 system compile time: 1/15/2009 12:00:00 [12/20/2008 12:53:33] Hardware cisco Nexus7000 C7010 (10 Slot) Chassis ("Supervisor module-1X") Intel(R) Xeon(R) CPU with 4129620 kB of memory. Processor Board ID JAB123501Z7 Device name: N7K-1 bootflash: 2000880 kB slot0: 0 kB (expansion flash) Storage Devices

Images Location

CPU

Kernel uptime is 0 day(s), 23 hour(s), 39 minute(s), 59 second(s) Last reset at 185087 usecs after Tue Dec 2 04:59:22 2008

2009 Cisco Systems, Inc. All rights reserved

7 of 44

Reason: Reset Requested by CLI command reload System version: 4.1(1.66) Service: plugin Core Plugin, Ethernet Plugin N7K-1-pod1-S1# Active Plug-in

NX-OS is composed by two images: a kickstart image that contains the Linux Kernel and a system image that contain most of the NX-OS software components. They both show up in the configuration.
Note: Note:

In future release we will be adding other plug-ins, like the Storage plug-in for FCoE

Lets now take a look at the running configuration.

N7K-1-pod1-S1# show running-config version 4.1(3) <omitted config> vrf context management vlan 1-4 interface Ethernet2/1 interface Ethernet2/2 <omitted interface config> interface Ethernet2/16 interface mgmt0 ip address 192.168.100.20/24

These are the interfaces available to your Pod (Virtual Device Context)

Management Interface Config

This is the configuration of the first Pod. As explained earlier each Pod runs within a Virtual Device Context (VDC). By using the VDC feature, we can segment the physical Nexus 7000 in multiple logical switches each of which runs in a separate memory space and has visibility only of the hardware resources that it owns, providing total isolation between the VDCs.
Note:

The show running-config has been improved. One of the improvements consists in the ability to not only look at the running-config but to also at the defaults values, which do not show up in the base config. The keyword to be used is all.
N7K-1-pod1-S1# show running-config all | begin mgmt0 interface mgmt0

2009 Cisco Systems, Inc. All rights reserved

8 of 44

cdp enable description speed auto duplex auto no shutdown ip address 192.168.100.20/24 ip redirects ip port-unreachable ip arp gratuitous update ip arp gratuitous request line vty session-limit 32 no exec-timeout line console no exec-timeout terminal length 24 terminal width 80 cfs distribute no cfs eth distribute cfs ipv4 mcast-address 239.255.70.83 cfs ipv6 mcast-address ff15::efff:4653 no cfs ipv4 distribute no cfs ipv6 distribute ip source-route ip igmp event-history mtrace size small ip igmp event-history igmp-internal size small ip igmp event-history vrf size small ip igmp event-history events size medium ip igmp event-history debugs size medium <omitted output>

Step 2 Management VRF and Basic Connectivity

The management interface is, by default, part of the management VRF. The management interface mgmt0 is the only interface allowed to be part of this VRF. The philosophy beyond Management VRF is to provide total isolation for the management traffic from the rest of the traffic flowing through the box by confining the former to its own forwarding table.

In this step we will: - Verify that only the mgmt0 interface is part of the management VRF - Verify that no other interface can be part of the management VRF - Verify that the default gateway is reachable only using the management VRF

2009 Cisco Systems, Inc. All rights reserved

9 of 44

N7K-1-pod1-S1# show vrf VRF-Name default management VRF-ID 1 2 State Up Up Reason ---

N7K-1-pod1-S1# show vrf interface Interface mgmt0 Ethernet1/1 Ethernet1/2 Ethernet1/3 Ethernet1/4 Ethernet1/5 <omitted output> N7K-1-pod1-S1# show vrf management interface Interface mgmt0 VRF-Name management VRF-ID 2 VRF-Name management default default default default default VRF-ID 2 1 1 1 1 1

The management VRF interface is part of the default configuration and the management interface mgmt0 is the only interface that can be made member of this VRF. Lets verify it.
Note: N7K-1-pod1-S1# conf t N7K-1-pod1-S1(config)# interface ethernet 2/1 N7K-1-pod1-S1(config-if)# vrf member management % VRF management is reserved only for mgmt0 N7K-1-pod1-S1(config-if)# show int mgmt0 mgmt0 is up Hardware: GigabitEthernet, address: 0022.5577.5e50 (bia 0022.5577.5e50) Internet Address is 192.168.100.20/24 MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA full-duplex, 1000 Mb/s Auto-Negotiation is turned on 1 minute input rate 1264 bits/sec, 1 packets/sec 1 minute output rate 1136 bits/sec, 0 packets/sec Rx 743 input packets 679 unicast packets 60 multicast packets 4 broadcast packets 70900 bytes FastEthernet? GigabitEthernet?... no, just ethernet interfaces

2009 Cisco Systems, Inc. All rights reserved

10 of 44

Tx 567 output packets 542 unicast packets 23 multicast packets 2 broadcast packets 66407 bytes

Try to reach the out-of-bound management networks default gateway with a ping.
N7K-1-pod1-S1(config-if)# ping 192.168.100.250 PING 192.168.100.250 (192.168.100.250): 56 data ping: sendto 192.168.100.250 64 chars, No route Request 0 timed out ping: sendto 192.168.100.250 64 chars, No route Request 1 timed out ping: sendto 192.168.100.250 64 chars, No route Request 2 timed out ping: sendto 192.168.100.250 64 chars, No route Request 3 timed out ping: sendto 192.168.100.250 64 chars, No route Request 4 timed out bytes to host to host to host to host to host

--- 192.168.100.250 ping statistics --5 packets transmitted, 0 packets received, 100.00% packet loss N7K-1-pod1-S1(config-if)#

The ping fails because we are trying to reach a system on the out-of-band management network without specifying the correct VRF.
Note:

N7K-1-pod1-S1# ping 192.168.100.250 vrf management

Linux-like output

PING 192.168.100.250 (192.168.100.250): 56 data bytes Request 0 timed out 64 bytes from 192.168.100.250: icmp_seq=1 ttl=254 time=0.887 64 bytes from 192.168.100.250: icmp_seq=2 ttl=254 time=0.816 64 bytes from 192.168.100.250: icmp_seq=3 ttl=254 time=0.943 64 bytes from 192.168.100.250: icmp_seq=4 ttl=254 time=0.848 --- 192.168.100.250 ping statistics --5 packets transmitted, 4 packets received, 20.00% packet loss round-trip min/avg/max = 0.816/0.873/0.943 ms N7K-1-pod1-S1#

ms ms ms ms

2009 Cisco Systems, Inc. All rights reserved

11 of 44

Step 3 CLI Familiarization

NX-OS CLI is very IOS-like. As you may have already noticed, when configuring the system, NX-OS gives the user a very IOS look and feel sensation. However there are differences, which we consider improvements. One of the main differences consists in NX-OS implementing a hierarchy independent CLI. Every command can in fact be issued from anywhere in the configuration. In this step we will: - Verify the CLI hierarchy independence by issuing a ping from different places in the chain - Verify the CLI piping functionality

N7K-1-pod1-S1# conf t N7K-1-pod1-S1(config)# ping ? *** No matches in current mode, matching in (exec) mode *** <CR> A.B.C.D or Hostname IP address of remote system WORD Enter Hostname multicast Multicast ping N7K-1-pod1-S1(config)# ping 192.168.100.250 vrf management

CLI Hierarchy Independent

PING 192.168.100.250 (192.168.100.250): 56 data bytes 64 bytes from 192.168.100.250: icmp_seq=0 ttl=254 time=0.874 ms 64 bytes from 192.168.100.250: icmp_seq=1 ttl=254 time=0.733 ms <omitted output> --- 192.168.100.250 ping statistics --4 packets transmitted, 4 packets received, 0.00% packet loss round-trip min/avg/max = 0.733/0.787/0.874 ms

N7K-1-pod1-S1(config)# int e2/1 N7K-1-pod1-S1(config-if)# ping ? *** No matches in current mode, matching in (exec) mode *** <CR> A.B.C.D or Hostname WORD multicast IP address of remote system Enter Hostname Multicast ping CLI Hierarchy Independent

N7K-1-pod1-S1(config-if)# ping 192.168.100.250 vrf management PING 192.168.100.250 (192.168.100.250): 56 data bytes 64 bytes from 192.168.100.250: icmp_seq=0 ttl=254 time=0.943 ms <omitted output> N7K-1-pod1-S1(config-if)#

2009 Cisco Systems, Inc. All rights reserved

12 of 44

Note: Note:

You can use the up-arrow and get the command history from the exec mode Any command can be issued from anywhere within the configuration

The output piping has also been improved and its now very similar to the one on Linux machines.
N7K-1-pod1-S1# show running-config | ? cut egrep grep head last less no-more sed sort tr uniq vsh wc begin count end exclude include Print selected parts of lines. Egrep - print lines matching a pattern Grep - print lines matching a pattern Display first lines Improved CLI Piping Display last lines Filter for paging Turn-off pagination for command output Stream Editor Stream Sorter Translate, squeeze, and/or delete characters Discard all but one of successive identical lines The shell than understands cli command Count words, lines, characters Begin with the line that matches Count number of lines End with the line that matches Exclude lines that match Include lines that match

N7K-1-pod1-S1# sh running-config | grep ? WORD count ignore-case invert-match line-exp line-number next prev word-exp Search for the expression Print a total count of matching lines only Ignore case difference when comparing strings Print only lines that contain no matches for <expr> Print only lines where the match is a whole line Print each match preceded by its line number Print <num> lines of context after every matching line Print <num> lines of context before every matching line Print only lines where the match is a complete word

N7K-1-pod1-S1# sh running-config | grep --v grep (GNU grep) 2.5.1 Copyright 1988, 1992-1999, 2000, 2001 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

The following command will grab the instance of a line with mgmt0 and print the following 3 lines after that match.
N7K-1-pod1-S1# sh running-config | grep next 3 mgmt0 interface mgmt0 ip address 192.168.100.20/24

2009 Cisco Systems, Inc. All rights reserved

13 of 44

N7K-1-pod1-S1# conf t N7K-1-pod1-S1(config)# int mgmt 0 N7K-1-pod1-S1(config-if)# [TAB] cdp exit no description ip pop end ipv6 push

shutdown vrf where

The [TAB] does not only complete the command, but also it shows the available keywords.
Note: N7K-1-pod1-S1(config-if)# ? cdp description end exit ip ipv6 no pop push shutdown vrf where Configure CDP interface parameters Enter description of maximum 80 characters Go to exec mode Exit from command interpreter Configure IP features Configure IPv6 features Negate a command or set its defaults Pop mode from stack or restore from name Push current mode to stack or save it under name Enable/disable an interface Configure VRF parameters Shows the cli context you are in

If you want to know the CLI context you are in use the where command.

N7K-1-pod1-S1(config-if)# where conf; interface mgmt0 admin@N7K-1-pod1-S1%default

Step 4 Role Based Access Control (RBAC)

RBAC stands for Role Based Access Control. Upon login, every user gets assigned a role that defines the privileges of the user that gained access to system. NX-OS, through the RABC feature, provides a very flexible and powerful framework to create ad hoc roles for any type of user. The roles are groups of rules that permit or deny a set of operations on NX-OS components. In this step we will: - Display the default roles - Display the features and the feature-groups that can be used as part of the role - Create a new role and apply the role to a newly created user - Display the newly created role - Test the role

2009 Cisco Systems, Inc. All rights reserved

14 of 44

NX-OS implements 4 default roles for the default VDC. Since the students are logged into a non-default VDC, only the two VDC default roles will be visible. For completeness the CLI output below shows all of them but on the students Pods only the last two (in bold here) will be visible.
N7K-1-pod1-S1# show role Not visible on your Pod

role: network-admin description: Predefined network admin role has access to all commands on the switch attribute: global ------------------------------------------------------------------Rule Perm Type Scope Entity ------------------------------------------------------------------1 permit read-write Not visible on your Pod role: network-operator description: Predefined network operator role has access to all read commands on the switch attribute: global ------------------------------------------------------------------Rule Perm Type Scope Entity ------------------------------------------------------------------1 permit read Super-user within the Pod role: vdc-admin description: Predefined vdc admin role has access to all commands within a VDC instance attribute: local ------------------------------------------------------------------Rule Perm Type Scope Entity ------------------------------------------------------------------1 permit read-write Only show commands for the vdc-operator role: vdc-operator description: Predefined vdc operator role has access to all read commands within a VDC instance attribute: local ------------------------------------------------------------------Rule Perm Type Scope Entity ------------------------------------------------------------------1 permit read N7K-1-pod1-S1#

Step 4a. Feature and Feature-groups. All users when they login are associated to a particular role. It can be one of the default pre-configured roles or a user-made role. A role is a set of rules that define what operations the user can perform on individual CLI commands, features and feature-groups basis. Feature-groups are essentially groups of related features, such as the L3 feature group (defined by default). You can group features in featuregroups and assign read/read-write permission to the whole group of features. To see the set of features and the feature groups available to be defined as part of a role, issue the following commands.

2009 Cisco Systems, Inc. All rights reserved

15 of 44

N7K-1-pod1-S1# show role feature feature: feature: feature: feature: feature: <omitted aaa access-list arp callhome cdp output>

N7K-1-pod1-S1# sh role feature-group feature group: L3 feature: router-bgp feature: router-eigrp feature: router-isis feature: router-ospf feature: router-rip N7K-1-pod1-S1#

Step 4b. Create a new role. Creating a role is very easy. We will create a new role that is allowed to issue all the show commands, to check basic connectivity using ping and to configure just the Cisco Discovery Protocol: cdp. After creating the role we will define a new user and associate the role to the newly created user.
N7K-1-pod1-S1# config t Very granular access control up to N7K-1-pod1-S1(config)# role name nxos the single CLI command. Ability to N7K-1-pod1-S1(config-role)# ? deny access to interfaces description Add a description for the role end Go to exec mode exit Exit from command interpreter interface Configure the interface policy for this role no Negate a command or set its defaults pop Pop mode from stack or restore from name push Push current mode to stack or save it under name rule Enter the rule number vlan Configure the vlan policy for this role vrf Configure the vrf policy for this role where Shows the cli context you are in N7K-1-pod1-S1(config-role)# N7K-1-pod1-S1(config-role)# N7K-1-pod1-S1(config-role)# N7K-1-pod1-S1(config-role)# rule rule rule rule 1 2 3 4 permit permit permit permit read Note the * matches all read-write feature cdp command ping * command conf t ; interface * Space before and after ; Note: Note:

The rules are applied in descending order.

A role can also specify what resources in terms of Interfaces, VLANs and VRFs the user is entitled to access. Lets exercise the interface restriction.

N7K-1-pod1-S1(config-role)# interface ? policy Configure the interface policy for this role N7K-1-pod1-S1(config-role)# interface policy deny N7K-1-pod1-S1(config-role-interface)# permit interface ethernet 2/1 ------

2009 Cisco Systems, Inc. All rights reserved

16 of 44

Note:

Lets verify the role and create a user to who attach the role.

N7K-1-pod1-S1# show role name nxos role: test description: new role vlan policy: permit (default) interface policy: deny permitted interface Ethernet2/1 vrf policy: permit (default) ------------------------------------------------------------------Rule Perm Type Scope Entity ------------------------------------------------------------------4 permit command conf t ; interface * 3 permit command ping * 2 permit read-write feature cdp 1 permit read

Step4c. Attach the role. Create a new user and attach the role. After that, please log out and login as the rbac user and test the RBAC configuration.
N7K-1-pod1-S1# conf t N7K-1-pod1-S1(config)#username rbac password rbac role nxos N7K-1-pod1-S1(config)#end N7K-1-pod1-S1# exit

Step4d. Using the puTTY, ssh as rbac into the management interface show running int mgmt 0 will tell you the IP address of the your management interface
Password: rbac Cisco Data Center Operating System (NX-OS) Software TAC support: http://www.cisco.com/tac <omitted output> N7K-1-pod1-S1# ? clear Reset functions configure Enter configuration mode debug Debugging functions debug Debugging function end Go to exec mode exit Exit from command interpreter ping Test network reachability show Show running system information

Most of the commands are missing. Lets check the commands this user has been allowed to use.
Note:

N7K-1-pod1-S1# ping 192.168.100.250 vrf management PING 192.168.100.250 (192.168.100.250): 56 data bytes 64 bytes from 192.168.100.250: icmp_seq=0 ttl=127 time=1.387 64 bytes from 192.168.100.250: icmp_seq=1 ttl=127 time=0.935 64 bytes from 192.168.100.250: icmp_seq=2 ttl=127 time=0.899 64 bytes from 192.168.100.250: icmp_seq=3 ttl=127 time=0.927 ms ms ms ms

2009 Cisco Systems, Inc. All rights reserved

17 of 44

64 bytes from 192.168.100.250: icmp_seq=4 ttl=127 time=0.897 ms --- 192.168.100.250 ping statistics --5 packets transmitted, 5 packets received, 0.00% packet loss round-trip min/avg/max = 0.897/1.008/1.387 ms N7K-1-pod1-S1# N7K-1-pod1-S1# debug ? cdp Configure CDP debugging Note:

Only the CDP debug is actually available.

N7K-1-pod1-S1# conf t N7K-1-pod1-S1(config)# ? cdp CDP Configuration parameters end Exit configuration mode exit Exit from command interpreter interface Configure Interfaces Note:

Only the cdp commands are available.

N7K-1-pod1-S1(config)# cdp ? advertise enable format holdtime timer Highest CDP version supported on the switch Enable/disable CDP on all interfaces Device ID format for CDP CDP hold time advertised (in seconds) CDP refresh time interval (in seconds)

N7K-1-pod1-S1(config)#

Note:

Lets try to access an interface for which we dont have the permission. -------

N7K-1-pod1-S1(config)# interface ethernet 2/2 % Interface permission denied N7K-1-pod1-S1(config)# interface ethernet 2/1 N7K-1-pod1-S1(config-if)# no shut N7K-1-pod1-S1(config-if)#

The step is completed you can now close the puTTY terminal you were just using.

2009 Cisco Systems, Inc. All rights reserved

18 of 44

Step 5

Configuration Rollback

NX-OS fully supports Configuration Rollback. This functionality allows you to revert to a previous configuration state, effectively rolling back configuration changes. Lets verify its functionality within NX-OS. In this step we will: - Create a checkpoint for the current configuration - Modify the configuration for an interface - Rollback the configuration - Verify the interface configuration

N7K-1-pod1-S1# checkpoint ? <CR> WORD Checkpoint name (Max Size 75) file Create configuration rollback checkpoint to file N7K-1-pod1-S1# checkpoint nxos Note: Processing the Request... Please Wait ........Done N7K-1-pod1-S1# N7K-1-pod1-S1# show checkpoint summary Checkpoint Summary --------------------------------------------------------------------------1) nxos: Created by admin Created at Wed, 01:04:48 31 Dec 2008 Size is 7,021 bytes

Lets now modify the configuration of an interface.

Odd Pods use interface e2/31 N7K-1-pod1-S1# conf t N7K-1-pod1-S1(config)# interface e2/15 N7K-1-pod1-S1(config-if)# ip address 1.1.1.1/24 N7K-1-pod1-S1(config-if)# no ip redirects N7K-1-pod1-S1(config-if)# ip proxy-arp N7K-1-pod1-S1(config-if)# no shutdown N7K-1-pod1-S1(config-if)# end N7K-1-pod1-S1# sh running-config int e2/15 version 4.1(3) interface Ethernet2/15 ip address 1.1.1.1/24 no ip redirects ip proxy-arp no shutdown N7K-1-pod1-S1#

Finally the slash notation

2009 Cisco Systems, Inc. All rights reserved

19 of 44

Lets check the difference between the current configuration and the checkpoint we created before.
N7K-1-pod1-S1# show diff rollback-patch checkpoint nxos ? checkpoint running-config startup-config Use checkpoint as destination configuration Use running configuration as destination Use startup configuration as destination

N7K-1-pod1-S1# show diff rollback-patch checkpoint nxos running-config Processing the Request... Please Wait !! ! interface Ethernet2/15 ip address 1.1.1.1/24 no ip redirects ip proxy-arp no shutdown N7K-1-pod1-S1#

Lets now rollback the configuration

N7K-1-pod1-S1# rollback running-config checkpoint nxos atomic Processing the Request... Please Wait Generating the Rollbackpatch... Please Executing the patch... Please Wait `conf t` `interface Ethernet2/15` `shutdown` `no ip proxy-arp` `ip redirects` `no ip address 1.1.1.1/24` Wait

During the rollback process the CLI commands are undone and shown to the user

N7K-1-pod1-S1# sh running-config int e2/15 version 4.1(3) interface Ethernet2/15

Step 6 Links up with Spanning Tree

It is time to bring up the interfaces and configure the Spanning Tree Protocol. Rapid Spanning Tree Protocol (RSTP) is standardized in IEEE 802.1w. Cisco's implementation of RSTP in both NX-OS and IOS provides a separate spanning tree instance for each active VLAN, which permits greater flexibility of Layer 2 topologies in conjunction with IEEE 802.1Q trunking. This implementation is also referred to as Rapid Per-VLAN Spanning Tree (RapidPVST). Rapid-PVST is the default spanning tree mode for NX-OS, so it does not need to be explicitly enabled.

2009 Cisco Systems, Inc. All rights reserved

20 of 44

Best practices dictate controlling the placement of the spanning tree root switch in the network for each VLAN to ensure that it does not inadvertently end up by the election process on a small switch in the access layer that creates a sub-optimal topology or may be more prone to failure. We will bring up few port-channels so we first need to enable the service for the LACP protocol.
LACP is a conditional service

N7K-1-pod1-S1 (config)# feature lacp

Note: NX-OS is a fully modular operating system; most software modules dont run unless the correspondent service is enabled. We refer to these features that need to be specifically enabled as conditional services. Once the service is enabled, the CLI becomes visible and the feature can be used and configured.

N7K-1-pod1-S1(config)# spanning-tree vlan 1-4 priority <..> N7K-1-pod1-S1(config)# int po 10 N7K-1-pod1-S1(config-if)# switchport N7K-1-pod1-S1(config-if)# switchport mode trunk N7K-1-pod1-S1(config-if)# switchport trunk allowed vlan 1-4 N7K-1-pod1-S1(config-if)# N7K-1-pod1-S1(config-if)# N7K-1-pod1-S1(config-if)# N7K-1-pod1-S1(config-if)#

4096 for N7K1 (Student 1) 8192 for N7K2 (Student 2)

spanning-tree port type network ---spanning-tree guard loop description link to the other Nexus7000 no shutdown

N7K-1-pod1-S1(config-if)# int e2/1-2 N7K-1-pod1-S1(config-if-range)# switchport N7K-1-pod1-S1(config-if-range)# switchport mode trunk N7K-1-pod1-S1(config-if-range)# switchport trunk allowed vlan 1-4 N7K-1-pod1-S1(config-if-range)# no shutdown N7K-1-pod1-S1(config-if-range)# channel-group 10 mode active

Check the status of the port-channel

N7K-1-pod1-S1(config-if-range)# show port-channel summary Flags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed S - Switched R - Routed U - Up (port-channel) --------------------------------------------------------------------------Group PortType Protocol Member Ports Channel --------------------------------------------------------------------------10 Po10(SU) Eth LACP Eth2/1(P) Eth2/2(P)

Bring up the interfaces facing the Access Layer

N7K-1-pod1-S1(config-if-range)# int e2/9-10

2009 Cisco Systems, Inc. All rights reserved

21 of 44

N7K-1-pod1-S1(config-if-range)# N7K-1-pod1-S1(config-if-range)# N7K-1-pod1-S1(config-if-range)# N7K-1-pod1-S1(config-if-range)#

switchport switchport mode trunk switchport trunk allowed vlan 1-4 no shutdown

Check the spanning-tree from both the Nexus 7000 and the Nexus 5000 (for the latter use the puTTY link on the Desktop).
N7K-1-pod1-S1(config-if-range)# show spanning-tree vlan 3 VLAN0003 Spanning tree enabled protocol rstp Root ID Priority 4099 Address 0022.5579.d2c2 This bridge is the root Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Bridge ID Priority Address Hello Time Role ---Desg Desg Desg Sts --FWD FWD FWD 4099 (priority 4096 sys-id-ext 3) 0022.5579.d2c2 2 sec Max Age 20 sec Forward Delay 15 sec Cost --------1 2 2 Prio.Nbr -------128.4105 128.265 128.266 Type ----------------------------Network P2p P2p P2p

Interface ---------------Po10 Eth2/9 Eth2/10

N5K-1# show spanning-tree vlan 3 VLAN0003 Spanning tree enabled protocol rstp Root ID Priority 4099 Address 0022.5579.d2c2 Cost 2 Port 129 (Ethernet1/1) Hello Time 2 sec Max Age 20 sec Bridge ID Priority Address Hello Time Role ---Root Altn Sts --FWD BLK

The link between the N5K and the N7K-2 is blocked as expected

Forward Delay 15 sec

32771 (priority 32768 sys-id-ext 3) 000d.eca4.0081 2 sec Max Age 20 sec Forward Delay 15 sec Cost --------2 2 Prio.Nbr -------128.129 128.130 Type ----------------------------P2p P2p

Interface ---------------Eth1/1 Eth1/2

Step 7 HSRP
To provide redundancy for the IP default gateway services, several protocols exist, which are commonly referred to together as First Hop Redundancy Protocols (FHRPs). Cisco NX-OS supports implementations of multiple FHRPs: Hot Standby Router Protocol (HSRP), Gateway Load Balancing Protocol (GLBP), and Virtual Router Redundancy Protocol (VRRP). You will configure HSRP in this step.

2009 Cisco Systems, Inc. All rights reserved

22 of 44

Lets create an SVI for VLAN 2 and VLAN 3 and configure HSRP:
N7K-1-pod1-S1(config)# feature interface-vlan N7K-1-pod1-S1(config)# feature hsrp Note: Both the SVI service and the service for the HSRP protocol are conditional. Their code does not run unless the feature is explicitly enabled with the feature command.

N7K-1-pod1-S1(config)# int vlan 2 N7K-1-pod1-S1(config-if)# ip address 192.168.202.<Student #>/24 N7K-1-pod1-S1(config-if)# no shutdown N7K-1-pod1-S1(config-if)# N7K-1-pod1-S1(config-if)# hsrp 1 N7K-1-pod1-S1(config-if-hsrp)# preempt delay minimum 180 N7K-1-pod1-S1(config-if-hsrp)# priority <...> 40 for N7K1 (Student N7K-1-pod1-S1(config-if-hsrp)# timers 1 3 1) N7K-1-pod1-S1(config-if-hsrp)# ip 192.168.202.3 20 for N7K2 (Student N7K-1-pod1-S1(config-if-hsrp)# int vlan 3 N7K-1-pod1-S1(config-if)# ip address 192.168.203.<Student #>/24 N7K-1-pod1-S1(config-if)# no shutdown N7K-1-pod1-S1(config-if)# N7K-1-pod1-S1(config-if)# hsrp 1 N7K-1-pod1-S1(config-if-hsrp)# preempt delay minimum 180 40 for N7K1 (Student 1) N7K-1-pod1-S1(config-if-hsrp)# priority <...> 20 for N7K2 (Student 2) N7K-1-pod1-S1(config-if-hsrp)# timers 1 3 N7K-1-pod1-S1(config-if-hsrp)# ip 192.168.203.3 N7K-1-pod1-S1# show hsrp brief P indicates configured to preempt. | Interface Grp Prio P State Active addr Standby addr Vlan2 1 40 P Active local 192.168.202.2 Vlan3 1 40 P Active local 192.168.203.2

Group addr 192.168.202.3 192.168.203.3

2009 Cisco Systems, Inc. All rights reserved

23 of 44

Step 8 Moving the Topology from STP-based to vPC-based

The virtual Port Channel (vPC) functionality provides the following benefits: Allows a single device to use a port channel across two upstream devices Eliminates Spanning Tree Protocol (STP) blocked ports Provides a loop-free topology Uses all available uplink bandwidth Provides fast convergence if either the link or a device fails Provides link-level resiliency Assures high availability

The topology will change as follow:

N7K-1 N7K-2 N7K-1 vPC fault tolerant link N7K-2

vPC Member Current STP Topology

vPC peer link

vPC Member vPC Topology

N5K

N5K

The terminology used for vPCs is as follows: vPC The combined port channel between the vPC peer devices and the downstream device. vPC peer device One of a pair of devices that are connected with the special port channel known as the vPC peer link. vPC peer link The link used to synchronize states between the vPC peer devices. Both ends must be on 10-Gigabit Ethernet interfaces. vPC domain This domain is formed by the two vPC peer link devices. It is also a configuration mode for configuring some of the vPC peer link parameters. vPC fault-tolerant link The fault-tolerant link is a Layer3 link between the vPC peer devices used to ensure that both devices are up. The fault-tolerant link sends configurable, periodic keepalive messages between devices connected by the vPC peer link on an out-of-band link. vPC member port Interfaces that belong to the vPCs.

2009 Cisco Systems, Inc. All rights reserved

24 of 44

vPC will be available in the NX-OS 4.1(3) software in the Q1CY09. During this step you will: - Enable the vPC - Create the vPC domain - Configure the peer-link port channel, and place it in vpc peer-link mode - Configure the access layer facing port channels, and place them in vPC mode
N7K-1-pod1-S1#conf t N7K-1-pod1-S1(config)# feature vpc

Next well enable the vPC domain. This domain ID is used to differentiate multiple vPC tiers, allowing for an L2 unique Link Aggregation ID for LACP based configuration. We will also configure the role so that the primary vPC device is the same device which is also the STP root and the HSRP primary device. This is the recommended configuration
1000 for Student 1 2000 for Student 2

N7K-1-pod1-S1(config)# vpc domain 1 N7K-1-pod1-S1(config-vpc-domain)# role priority <...>

First thing to setup is the fault-tolerant link connection. For the fault-tolerant link we recommend a separate port, preferably 1GigE, between the vPC peer devices (it does NOT need to be a direct link). This port should belong to a separate VRF. Another alternative is to use the Out-of-Band management network through the Supervisors management interface and this is what well do in this lab.
N7K-1-pod1-S1(config-vpc-domain)# peer-keepalive dest 192.168.100.<...> source 192.168.100.<...> vrf management Enter your partners mgmt0 IP address

Enter your mgmt0 IP address

Lets check the status of the fault-tolerant link (peer-keepalive).

N7K-1-pod1-S1(config-vpc-domain)# show vpc peer-keepalive vPC keep-alive status : --Destination : --Send status : --Receive status : --Last update from peer : N7K-9-pod9-S1(config-vpc-domain)# peer is alive 192.168.100.21 Success Success (0 ) seconds, (40

) msec

2009 Cisco Systems, Inc. All rights reserved

25 of 44

Now that the base vPC domain is configured, we can configure the peer-link, and then we can validate that the base vPC infrastructure is running (assuming your Partner has done the same configuration steps on the other Nexus7000 in your Pod).

N7K-1-pod1-S1(config-int)# int port-channel 10 N7K-1-pod1-S1(config-int)# vpc peer-link N7K-1-pod1-S1(config-int)# show vpc brief Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 1 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status: success vPC role : primary vPC Peer-link status --------------------------------------------------------------------id Port Status Active vlans ---------- -------------------------------------------------1 Po10 up 1-4

The STP status hasnt changed on the Nexus 5000.

N5K# show spanning-tree vlan 3 VLAN0003 Spanning tree enabled protocol rstp Root ID Priority 4099 Address 001b.54c2.b1c2 Cost 2 Port 129 (Ethernet1/1) Hello Time 2 sec Max Age 20 sec Bridge ID Priority Address Hello Time Role ---Root Altn Sts --FWD BLK

Forward Delay 15 sec

32771 (priority 32768 sys-id-ext 3) 000d.eca4.0481 2 sec Max Age 20 sec Forward Delay 15 sec Cost --------2 2 Prio.Nbr -------128.129 128.130 Type ---------------------------P2p P2p

Interface ---------------Eth1/1 Eth1/2

Now that the peer-link is running and the vPC is up, we can add in the access facing vPC links.
N7K-1-pod1-S1(config)# int N7K-1-pod1-S1(config-int)# N7K-1-pod1-S1(config-int)# N7K-1-pod1-S1(config-int)# N7K-1-pod1-S1(config-int)# po 20 switchport switchport mode trunk switchport trunk allowed vlan 1-4 no sh

2009 Cisco Systems, Inc. All rights reserved

26 of 44

N7K-1-pod1-S1(config-int)# vpc 20

Lets now add the port facing the Access Layer (Nexus 5000) to the port-channel.
N7K-1-pod1-S1(config-int)# int e2/9 N7K-1-pod1-S1(config-int)# channel-group 20 mode active

Lets check the vPC status.

N7K-1-pod1-S1(config-if)# show vpc brief Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 1 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status: success vPC role : primary vPC Peer-link status --------------------------------------------------------------------id Port Status Active vlans ---------- -------------------------------------------------1 Po10 up 1-4 vPC status ---------------------------------------------------------------------id Port Status Consistency Reason Active vlans ---------- ----------- -------------------------- -----------20 Po20 down* failed Consistency Check Not Performed

The vPC status is down because we havent configured the port-channel on the Nexus5000 yet; in fact the port is in individual state from a LACP prospective.

N7K-1-pod1-S1(config-if)# sh port-channel summary Flags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed S - Switched R - Routed U - Up (port-channel) ------------------------------------------------------------------------Group PortType Protocol Member Ports Channel -------------------------------------------------------------------------10 Po10(SU) Eth LACP Eth2/1(P) Eth2/2(D) 20 Po20(SD) Eth LACP Eth2/9(I)

If your teammate has reached this point as well, one of you can go on the N5K and configure the port-channel.

2009 Cisco Systems, Inc. All rights reserved

27 of 44

N5K(config-if)# int e1/1-2 N5K(config-if-range)# channel-group 20 mode active

Lets check the STP and the port-channel status.

N5K(config-if-range)# show spanning-tree vlan 3 VLAN0003 Spanning tree enabled protocol rstp Root ID Priority 4099 Address 001b.54c2.b1c2 Cost 1 Port 4115 (port-channel20) Hello Time 2 sec Max Age 20 sec Bridge ID Priority Address Hello Time

Forward Delay 15 sec

32771 (priority 32768 sys-id-ext 3) 000d.eca4.0481 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type ---------------- ---- --- --------- -------- ---------------------------Po20 Root FWD 1 128.4115 P2p N5K(config-if-range)# show port-channel summary Flags: D I H R down U - up in port-channel Individual S - suspended Hot-standby (LACP only) Module-removed

-------------------------------------------------------------------------Group PortType Protocol Member Ports Channel -------------------------------------------------------------------------20 Po20(U) Eth LACP Eth1/1(U) Eth1/2(U)

The Nexus 5000 has now a port-channel connected to two different upstream devices. Lets check the status of the vPC and the STP on the Nexus7000.
N7K-1-pod1-S1(config-if)# sh vpc brief Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 1 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status: success vPC role : primary vPC Peer-link status --------------------------------------------------------------------id Port Status Active vlans ---------- -------------------------------------------------1 Po10 up 1-4 vPC status

2009 Cisco Systems, Inc. All rights reserved

28 of 44

---------------------------------------------------------------------id Port Status Consistency Reason Active vlans ---------- ----------- -------------------------- -----------20 Po20 up success success 1-4 N7K-1-pod1-S1(config-if)# show spanning-tree vlan 3 VLAN0003 Spanning tree enabled protocol rstp Root ID Priority 4099 Address 001b.54c2.b1c2 This bridge is the root Hello Time 2 sec Max Age 20 sec Bridge ID Priority Address Hello Time Role ---Desg Desg Desg Sts --FWD FWD FWD

Forward Delay 15 sec

4099 (priority 4096 sys-id-ext 3) 001b.54c2.b1c2 2 sec Max Age 20 sec Forward Delay 15 sec Cost --------1 1 2 Prio.Nbr -------128.4105 128.4115 128.266 Type ---------------------------(vPC peer-link) Network P2p (vPC) P2p P2p

Interface ---------------Po10 Po20 Eth2/10

The vPC topology is now up and running!

Step 9 vPC Failure Scenario

One of the advantages of the vPC approach to loop management is that failure recovery on a link or of an entire switch relies on port-channel failover rather than on STP re-learning the entire network. With port-channel failover, recovery is often sub-second. This alone is a key reason why vPC provides an efficient scaling mechanism relative to STP managed Layer 2 topologies. In this step we will bring down the vPC peer-link. In the unlikely case that both ports and line cards in the peer-link fail (being that two ports on two different line cards are the recommended minimum for the peer-link) the vPC software will look to the fault-tolerant link (the keep-alive link) to determine if the failure is a link level failure (perhaps a UDLD failure of some nature), or if in fact the remote peer has failed entirely. In the case that the remote peer is still alive (peer-keepalive messages are still being received), to avoid loops the vPC secondary switch will disable its vPC member ports and any Layer 3 interfaces attached to a vPC associated VLAN.

2009 Cisco Systems, Inc. All rights reserved

29 of 44

We will bring down the peer-link interfaces on the vPC primary device and observe what happens on the vPC secondary and on the Nexus 5000 devices.
N7K-1-pod1-S1# conf t N7K-1-pod1-S1(config)# int e2/1-2 N7K-1-pod1-S1(config-if-range)# shutdown

On the Nexus 5000 we can see how the port-channel port got suspended.
N5K(config-if-range)# show port-channel summary Flags: D I H R down U - up in port-channel Individual S - suspended Hot-standby (LACP only) Module-removed

-------------------------------------------------------------------------Group PortType Protocol Member Ports Channel -------------------------------------------------------------------------20 Po20(U) Eth LACP Eth1/1(U) Eth1/2(D) 30 Po30(D) Eth NONE --

While on the vPC secondary you should see the following:

%VPC-2-VPC_SUSP_ALL_VPC: Peer-link going down, suspending all vPCs on secondary

N7K-2-pod1-S2(config-if)# show int vlan 2 Vlan2 is down, line protocol is down Hardware is EtherSVI, address is 001b.54c2.af42 Internet Address is 192.168.202.2/24 MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec, <omitted output>

N7K-2-pod1-S2(config-if)# show port-channel summary Flags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed S - Switched R - Routed U - Up (port-channel) ------------------------------------------------------------------------Group PortType Protocol Member Ports Channel ------------------------------------------------------------------------10 Po10(SD) Eth LACP Eth2/1(D) Eth2/2(D) 20 Po20(SD) Eth LACP Eth2/9(D)

2009 Cisco Systems, Inc. All rights reserved

30 of 44

Now we can bring the peer-link interfaces on the vPC primary back up and check again the Nexus5000 first.
N7K-1-pod1-S1(config)# int e2/1-2 N7K-1-pod1-S1(config-if-range)# no shutdown

After few seconds you should see, the link back up:
N5K(config-if-range)# show port-channel summary Flags: D - down U - up in port-channel I - Individual S - suspended H - Hot-standby (LACP only) R - Module-removed ------------------------------------------------------------------------Group PortType Protocol Member Ports Channel ------------------------------------------------------------------------20 Po20(U) Eth LACP Eth1/1(U) Eth1/2(U)

Also on the vPC secondary the SVIs are back up:

N7K-2-pod1-S2(config-if)# show int vlan 2 Vlan2 is up, line protocol is up <omitted output>

Step 10 OSPF Configuration

OSPF is fully implemented in NX-OS as part of the Enterprise License (however you can use the feature leveraging the grace-period mode for 120 days). In this step we will configure OSPFv2 and we will see how the configuration is interface centric vs. the network centric IOS based OSPF configuration. These are the steps for this exercise: - Turn the OSPFv2 service on - Configure the Loopback interfaces - Instantiate an OSPF process - Verify OSPF configuration by issuing few show command

N7K-1-pod1-S1(config)# interface loopback0 N7K-1-pod1-S1(config-if)# ip address 10.1.255.<Student #>/24 N7K-1-pod1-S1(config-if)# feature ospf N7K-1-pod1-S1(config)# router ospf 1 N7K-1-pod1-S1(config-router)# log-adjacency-changes

2009 Cisco Systems, Inc. All rights reserved

31 of 44

N7K-1-pod1-S1(config-router)# auto-cost reference-bandwidth 1000000 Note: As you may have noticed the network x.x.x.x area y configuration lines are not present. This is a big different from IOS. OSPF, as well as other IGP protocols, are interface centric, as we will see with the next few commands.

Lets now configure the interfaces.

Odd Pods interface e1/25

N7K-1-pod1-S1(config)# int e1/13 N7K-1-pod1-S1(config-if)# description link to the Cat6k N7K-1-pod1-S1(config-if)# ip address 192.168.<Student #>.1/30 N7K-1-pod1-S1(config-if)# ip ospf hello-interval 2 N7K-1-pod1-S1(config-if)# ip ospf dead-interval 6 N7K-1-pod1-S1(config-if)# ip ospf network point-to-point N7K-1-pod1-S1(config-if)# ip router ospf 1 area 0 N7K-1-pod1-S1(config-if)# no shutdown N7K-1-pod1-S1(config-if)#

In the NX-OS the OSPF configuration is interface centric. The membership to an OSPF area is specified at the interface configuration level. This approach is more intuitive and manageable.
Note:

Now we can check the OSPF configuration we have been working on.
N7K-1-pod1-S1# sh running-config ? <CR> > aaa all am arp bgp <snip> l3vm license msdp netflow ospf ospfv3 pim pim6 <snip> Redirect it to a file Display aaa configuration Current operating configuration with defaults Display am information Display arp information Display bgp information Display l3vm information Display licensing configuration Display msdp information Show NetFlow configuration Display ospf information Display ospfv3 information Display pim information Display pim6 information

N7K-1-pod1-S1# sh running-config ospf version 4.1(3) feature ospf router ospf 1 auto-cost reference-bandwidth 1000000 interface Ethernet1/13

2009 Cisco Systems, Inc. All rights reserved

32 of 44

ip ip ip ip

ospf dead-interval 6 ospf hello-interval 2 ospf network point-to-point router ospf 1 area 0.0.0.0

Lets check now the complete OSPF configuration with its default values.
N7K-1-pod1-S1# sh running-config ospf all version 4.1(3) feature ospf snmp-server enable traps ospf rate-limit 10 7 snmp-server enable traps ospf 1 rate-limit 10 7 router ospf 1 graceful-restart graceful-restart grace-period 60 timers lsa-arrival 1000 distance 110 maximum-paths 8 auto-cost reference-bandwidth 1000000 ip ospf event-history size small ip ospf event-history cli size small ip ospf event-history redistribution size small ip ospf event-history spf size small ip ospf event-history lsa size small ip ospf event-history flooding size small ip ospf event-history ha size small ip ospf event-history event size small ip ospf event-history adjacency size small interface Ethernet1/13 ip ospf dead-interval 6 ip ospf hello-interval 2 ip ospf network point-to-point ip ospf priority 1 ip ospf retransmit-interval 5 ip ospf transmit-delay 1 ip router ospf 1 area 0.0.0.0

N7K-1-pod1-S1# sh ip ospf neighbors OSPF Process ID 1 VRF default Total number of neighbors: 1 Neighbor ID Pri State 192.168.100.5 1 FULL/ N7K-1-pod1-S1#

Up Time Address 00:08:58 192.168.201.2

Interface Eth1/13

2009 Cisco Systems, Inc. All rights reserved

33 of 44

Step 11 State-full Process Restart

NX-OS is a modern operating system. NX-OS continuously checks the health of each software module making sure that if a process crashes or hangs the right action is taken to allow service continuity and availability. NX-OS has been designed around the concept of zero service destruction. All Layer2 protocols (STP, CDP, LACP etc) and OSPF support the State-full Process Restart leveraging our PSS (Persistent Storage Service) architecture. With this exercise we will see how the system recovers from an OSPF crash in a seamless way. You will see how the connected Cat6K wont even realize that the process crashed and restarted. These are the steps for this exercise: - Display the OSPF process ID - Kill the OSPF process - Verify that the OSPF process has been restarted with a new process ID - Check the Cat6K screen Using the puTTY icon on the Desktop connect to the 6K so that you have both terminals open one on the Nexus 7000 and one on the Catalyst 6500.
Note:

Only one student can log into the Catalyst 6500 at a given time.

Just to show that the OSPF adjacency goes down as expected, shutdown the link on the N7K
N7K-1-pod1-S1(config)# int e1/13 N7K-1-pod1-S1(config-if)# shutdown

As you can see on the 6K terminal the link and the OSPF adjacency went down. Now bring the interface back up on the Nexus 7000.
N7K-1-pod1-S1(config-if)# no shutdown The interface is now up and the OSPF adjacency is back up. Now lets kill OSPF. N7K-1-pod1-S1# show process | inc ospf 1959 S NR NR NR NR 778f727b 1 0 0 0 0 ospf ospfv3 ospf ospfv3 ospf

Notice the PID on the left (you will need it in the killing process) and the number of restarts (bold, blu and underlined fonts).

2009 Cisco Systems, Inc. All rights reserved

34 of 44

The x in the following CLI commands is the number of your Pod, i.e 1 for Pod1, 2 for Pod2.

N7K-1-pod1-S1# N7K-1-pod1-S1# copy bootflash:proc.res bootflash:proc<x>.res N7K-1-pod1-S1# N7K-1-pod1-S1# N7K-1-pod1-S1# load bootflash:procx.res load_isanimg: entry load_isanimg: uri_info:0x809ba90 load_isanimg: type:0x8 Loading plugin version 4.0(2) ############################################################### Warning: debug-plugin is for engineering internal use only! For security reason, plugin image has been deleted. ############################################################### Successfully loaded service restart debug-plugin!!! Commands Available: help kill <pid> exit Enter Commands: kill <ospf pid> killing 2008 May 12 21:22:35 N7K-C1-1-pod1 %SYSMGR-2-SERVICE_CRASHED: Service "__inst_001__ospf" (PID 19700) hasn't caught signal 9 (no core). exit N7K-1-pod1-S1# sh process | inc ospf 16066 S 778f727b 2 NR 0 NR 0 NR 0 NR 0 NR 0 NR 0 NR 0 ospf ospfv3 ospf ospfv3 ospf ospfv3 ospf ospfv3

Notice how the OSPF process has now a new process ID and how, looking at the Cat6K terminal, the neighbor didnt even realized that our OSPF process was killed and restarted.

Step 12 Wireshark
Wireshark used to be known as Ethereal. Wireshark is the world's foremost network protocol analyzer and is the de facto (and often de jure) standard across many industries and educational institutions. NX-OS offers an integrated packet capture tool for packets directed to the control plane. This packet analyzer is built on top of Wireshark and it is called Ethanalyzer.

2009 Cisco Systems, Inc. All rights reserved

35 of 44

The primary function of this protocol analyzer is to be able to capture and analyze control packets, but it can also be leveraged to look at data traffic in its acl-log mode. When analyzing data traffic, such traffic will reach the Supervisor after being rate limited in hardware. During this step we will capture regular control traffic, and then we will set up an ACL just to show the procedure for capturing data-plane traffic, we wont actually capture data traffic during this lab. Ethanalyzer can be used only from the default-VDC. To start access the default-VDC by opening the Device Access folder located in the My Documents folder and double click on the N7K# default ssh connection, where # is 1 for Student1 and 2 for Student 2.
N7K-1# ethanalyzer local interface ? inband Inband/Outband interface mgmt Management interface N7K-1# ethanalyzer local <CR> > >> brief capture-filter decode-internal display-filter limit-captured-frames limit-frame-size write | interface inband ? Redirect it to a file Redirect it to a file in append mode Display only protocol summary Filter on ethanalyzer capture Include internal system header decoding Display filter on frames captured Maximum number of frames to be captured (default is 100) Capture only a subset of a frame Filename to save capture to Pipe command output to filter

The brief option will show one-liner info.

N7K-1# ethanalyzer local interface inband brief capture-filter "udp" limitcaptured-frames 10 Capturing on eth0 10 packets captured 2009-01-08 07:09:45.84 2009-01-08 07:09:45.87 2009-01-08 07:09:45.89 2009-01-08 07:09:45.89 2009-01-08 07:09:46.89 2009-01-08 07:09:46.89 2009-01-08 07:09:46.89 2009-01-08 07:09:46.90 2009-01-08 07:09:47.90 2009-01-08 07:09:47.90 N7K-1#

192.168.203.2 192.168.202.2 192.168.202.1 192.168.203.1 192.168.203.2 192.168.202.2 192.168.202.1 192.168.203.1 192.168.202.1 192.168.203.1

-> -> -> -> -> -> -> -> -> ->

224.0.0.2 224.0.0.2 224.0.0.2 224.0.0.2 224.0.0.2 224.0.0.2 224.0.0.2 224.0.0.2 224.0.0.2 224.0.0.2

HSRP HSRP HSRP HSRP HSRP HSRP HSRP HSRP HSRP HSRP

Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello

(state (state (state (state (state (state (state (state (state (state

Standby) Standby) Active) Active) Standby) Standby) Active) Active) Active) Active)

2009 Cisco Systems, Inc. All rights reserved

36 of 44

To see the entire packet remove the brief keyword. N7K-1# ethanalyzer local interface inband capture-filter "udp" limitcaptured-frames 1 | no-more Capturing on eth0 1 packets captured Frame 1 (62 bytes on wire, 62 bytes captured) Arrival Time: Nov 19, 2008 01:06:08.834050000 [Time delta from previous captured frame: 1227056768.834050000 seconds] [Time delta from previous displayed frame: 1227056768.834050000 seconds] [Time since reference or first frame: 1227056768.834050000 seconds] Frame Number: 1 Frame Length: 62 bytes Capture Length: 62 bytes [Frame is marked: False] [Protocols in frame: eth:ip:udp:hsrp] Ethernet II, Src: 00:22:55:79:be:42 (00:22:55:79:be:42), Dst: 01:00:5e:00:00:02 (01:00:5e:0 0:00:02) Destination: 01:00:5e:00:00:02 (01:00:5e:00:00:02) Address: 01:00:5e:00:00:02 (01:00:5e:00:00:02) .... ...1 .... .... .... .... = IG bit: Group address (multicast/broadcast) .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default) Source: 00:22:55:79:be:42 (00:22:55:79:be:42) Address: 00:22:55:79:be:42 (00:22:55:79:be:42) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default) Type: IP (0x0800) <IP Header Omitted> Cisco Hot Standby Router Protocol Version: 0 Op Code: Hello (0) State: Standby (8) Hellotime: Non-Default (1) Holdtime: Non-Default (3) Priority: 20 Group: 1 Reserved: 0 Authentication Data: Default (cisco) Virtual IP Address: 192.168.202.3 (192.168.202.3) N7K-1#

Lets capture and store the file on the bootflash, so we can copy it over and look at it on our Windows machine.
N7K-1# ethanalyzer local interface inband limit-captured-frames 30 write bootflash:capture Capturing on eth0 30 N7K-1# copy bootflash:capture sftp://Student#@192.168.100.250/HOME/Desktop vrf management

2009 Cisco Systems, Inc. All rights reserved

37 of 44

Connecting to 192.168.100.250... [email protected]'s password: < Your Remote Desktop Password > sftp> put /bootflash/capture /HOME/Desktop Uploading /bootflash/capture to /HOME/Desktop/capture /bootflash/capture 100% 822 0.8KB/s 00:00 sftp> exit

Now the capture is on your Desktop, launch Wireshark using the icon and load the file.

The following portion of the Wireshark step is optional... if you are running out of time jump to Step 13 Virtual Device Context!!!
Ethanalyzer can capture data traffic as well, so that network administrators can have an embedded and easy to use tool for on the fly capture. Ethanalyzer gives network administrators more visibility into applications behavior with few simple steps: 1. Identify the application characteristics 2. Create ad hoc ACL to match (and permit) the application flow between two servers 3. Use the log keyword to punt copies of matching packets to supervisor CPU 4. The original traffic gets forwarded with no impact 5. The copies sent to CPU are subjected to hardware rate limiter (100 pps by default) 6. These copies can be captured by our Ethanalyzer (Wireshark a.k.a Ethereal) 7. Ethanalyzer can output to screen or dump to file on flash which can be copied to PC for GUI analysis Lets suppose to have an application using TCP port 5600 between the server 1.1.1.24 and the client 1.1.1.16. Lets now create the ad hoc ACL and lets apply it to the interface. We wont actually capture traffic in this example and you do NOT need to run this part of the config:
N7K-1(config)# ip access-list etha N7K-1(config-acl)# statistics per-entry N7K-1(config-acl)# permit tcp host 1.1.1.24 host 1.1.1.16 eq 5600 log N7K-1(config-acl)# show ip access-lists etha IP access list etha statistics per-entry 10 permit tcp 1.1.1.24/32 1.1.1.16/32 eq 5600 log N7K-1(config)# int e1/1 N7K-1(config-if)# ip access-group etha in N7K-2-pod1(config-if)# end

We can now capture selectively these packets and save the capture to the usb1 (so we could use our laptop with the nice wireshark graphical interface):
N7K-1# ethanalyzer loc interf inband capture-filter "tcp port 5600" write bootflash:cap_acl_log

2009 Cisco Systems, Inc. All rights reserved

38 of 44

Step 13 Virtual Device Contexts

NX-OS introduces support for the Virtual Device Contexts (VDCs), which allow the Nexus7000 to be virtualized at the device level. Each configured VDC presents itself as a unique device to connected users within the framework of that physical switch. The VDC runs as a separate logical entity within the switch, maintaining its own unique set of running software processes, having its own configuration, and being managed by a separate administrator. This lab has used the VDC concept to allow multiple PODs to work on a single switch. These are the steps for this exercise: - Delete the VDC you were working on. - Create a new VDC and allocate resources to it. - switchto the newly created VDC and perform the initial configuration script For this last step keep using the putty terminal you were using for the previous step. You need to be in the default-VDC
N7K-1# show vdc vdc_id -----1 2 3 vdc_name -------N7K-1 pod1-S1 pod2-S1 state ----active active active mac ---------00:22:55:79:c4:41 00:22:55:79:c4:42 00:22:55:79:c4:43

You will now delete the Pod (that is VDC) you were working on.
N7K-1# conf t N7K-1(config)# no vdc pod< y >-S< x > is 1 for Student1, 2 for Student2 where y is your Pod number and x

Deleting this vdc will remove its config. Continue deleting this vdc? yes Note: Deleting VDC, one moment please ...

[no]

N7K-1(config)# 2009 Jan 8 07:43:34 N7K-1 %VDC_MGR-2-VDC_OFFLINE: vdc 2 is now offline

Now create a new VDC and allocate the following interfaces

POD# POD{ODD} POD{EVEN}

Interfaces e1/1-24, e2/1-16 e1/25-48, e2/17-32

2009 Cisco Systems, Inc. All rights reserved

39 of 44

N7K-1(config)# vdc pod< y >-S< x > 1 for Student1, 2 for Student2 Note:

where y is your Pod number and x is

Creating VDC, one moment please ...

2009 Jan 8 07:44:17 N7K-1 %VDC_MGR-2-VDC_LIC_WARN: Service using grace period will be shutdown in 30 day(s) 2009 Jan 8 07:44:34 N7K-9 %VDC_MGR-2-VDC_ONLINE: vdc 2 has come online

N7K-1(config-vdc)# ? allocate end exit ha-policy limit-resource no pop push template where Assign interfaces to vdc Go to exec mode Exit from command interpreter Change HA policy for this VDC Resource configuration Negate a command or set its defaults Pop mode from stack or restore from name Push current mode to stack or save it under name Change the template for this vdc Shows the cli context you are in

N7K-1(config-vdc)# allocate interface ethernet <check the table above> Moving ports will cause all config associated to them in source vdc to be removed. Are you sure you want to move the ports? [yes] yes

Should a control plane failure occur, the administrator has a set of options that can be configured on a per-VDC basis defining what action will be taken regarding that VDC. There are three actions that can be configured: restart, bringdown, and reset. The restart option will delete the VDC and then re-create it with the running configuration. This configured action will occur regardless of whether there are dual supervisors or a single supervisor present in the chassis. The bringdown option will simply delete the VDC. The reset option will issue a reset for the active supervisor when there is only a single supervisor in the chassis. If dual supervisors are present, the reset option will force a supervisor switchover. The default VDC always has a high-availability option of reset assigned to it. Subsequent VDCs created will have a default value of bringdown assigned to them. This value can be changed under configuration control.
N7K-1(config-vdc)# N7K-1(config-vdc)# m4route-mem m6route-mem monitor-session ha-policy single-sup restart dual-sup restart limit-resource ? Set ipv4 route memory limits Set ipv6 route memory limits Monitor local session

2009 Cisco Systems, Inc. All rights reserved

40 of 44

port-channel u4route-mem u6route-mem vlan vrf

Set Set Set Set Set

port-channel limits ipv4 route memory limits ipv6 route memory limits VLAN limits vrf resource limits

N7K-1(config-vdc)# limit-resource vrf minimum 16 maximum 20

N7K-1(config-vdc)# show vdc pod< y >-S< x > vdc id: 2 vdc name: pod1-S1 vdc state: active vdc mac address: 00:1b:54:c2:29:42 vdc ha policy: RESTART vdc dual-sup ha policy: RESTART vdc create time: Thu Aug 7 10:15:46 2008 vdc restart count: 0

detail

N7K-1(config-vdc)# show vdc pod< y >-S< x > membership vdc_id: 2 vdc_name: student1 Ethernet1/1 Ethernet1/5 Ethernet1/7 Ethernet1/10 Ethernet1/13 Ethernet1/16 N7K-1(config-vdc)# exit interfaces: Ethernet1/2 Ethernet1/5 Ethernet1/8 Ethernet1/11 Ethernet1/14 Ethernet1/3 Ethernet1/6 Ethernet1/9 Ethernet1/12 Ethernet1/15

Its now time to switchto the newly created VDC. You will go through the initial script configuration, which is similar to the one you would go through on a first time-booted Nexus7000.
N7K-1# switchto vdc pod< y >-S< x > ---- System Admin Account Setup ---Do you want to enforce secure password standard (yes/no): no Enter the password for "admin": Test Confirm the password for "admin": Test ---- Basic System Configuration Dialog VDC: 2 ---This setup utility will guide you through the basic configuration of the system. Setup configures only enough connectivity for management of the system. Please register Cisco Nexus7000 Family devices promptly with your supplier. Failure to register may affect response times for initial service calls. DC3 devices must be registered to receive entitled support services. Press Enter at anytime to skip a dialog. Use ctrl-c at anytime

2009 Cisco Systems, Inc. All rights reserved

41 of 44

to skip the remaining dialogs. Would you like to enter the basic configuration dialog (yes/no): yes Create another login account (yes/no) [n]: Configure read-only SNMP community string (yes/no) [n]: Configure read-write SNMP community string (yes/no) [n]: Enter the switch name : pod< y >-S< x > Continue with Out-of-band (mgmt0) management configuration? (yes/no) [y]: Mgmt0 IPv4 address : 192.168.100.<...> Mgmt0 IPv4 netmask : 255.255.255.0 Configure the default gateway? (yes/no) [y]: IPv4 address of the default gateway : 192.168.100.1 Configure advanced IP options? (yes/no) [n]: Enable the telnet service? (yes/no) [y]: Enable the ssh service? (yes/no) [n]: Configure the ntp server? (yes/no) [n]: Configure default interface layer (L3/L2) [L3]: Configure default switchport interface state (shut/noshut) [shut]: Configure default switchport trunk mode (on/off/auto) [on]: The following configuration will be applied: switchname pod1nxos interface mgmt0 ip address 192.168.100.20 255.255.255.0 no shutdown vrf context management ip route 0.0.0.0/0 192.168.100.1 exit telnet server enable no ssh server enable no system default switchport system default switchport shutdown Would you like to edit the configuration? (yes/no) [n]: Use this configuration and save it? (yes/no) [y]: y
20 for Odd Pods Student1 22 for Odd Pods Student2 21 for Even Pods Student1 23 for Even Pods Student2

Cisco Nexus Operating System (NX-OS) Software TAC support: http://www.cisco.com/tac

2009 Cisco Systems, Inc. All rights reserved

42 of 44

Copyright (c) 2002-2008, Cisco Systems, Inc. All rights reserved. The copyrights to certain works contained in this software are owned by other third parties and used and distributed under license. Certain components of this software are licensed under the GNU General Public License (GPL) version 2.0 or the GNU Lesser General Public License (LGPL) Version 2.1. A copy of each such license is available at http://www.opensource.org/licenses/gpl-2.0.php and http://www.opensource.org/licenses/lgpl-2.1.php

N7K-1-pod1# N7K-1<x>nxos# sh running-config version 4.0(3) username admin password 5 $1$XpvaHAKS$OhTkzciBdKkE4FOM0epik/ role vdc-admin telnet server enable ssh key rsa 1024 force no ssh server enable snmp-server user admin vdc-admin auth md5 0x77306315bd719b5d121cdeb6f0a9d697 priv 0x77306315bd719b5d121cdeb6f0a9d697 localizedkey vrf context management ip route 0.0.0.0/0 192.168.100.1 switchname pod1nxos <omitting interface config> interface mgmt0 ip address 192.168.100.20/26 N7K-1-pod1# ping 192.168.100.250 vrf management PING 192.168.100.250 (192.168.100.250): 56 data bytes 64 bytes from 192.168.100.250: icmp_seq=0 ttl=255 time=0.927 64 bytes from 192.168.100.250: icmp_seq=1 ttl=255 time=0.452 64 bytes from 192.168.100.250: icmp_seq=2 ttl=255 time=0.504 64 bytes from 192.168.100.250: icmp_seq=3 ttl=255 time=0.692 64 bytes from 192.168.100.250: icmp_seq=4 ttl=255 time=0.596 --- 192.168.100.250 ping statistics --5 packets transmitted, 5 packets received, 0.00% packet loss round-trip min/avg/max = 0.452/0.634/0.927 ms

ms ms ms ms ms

Congratulations!!! The lab is now complete!

Please LOG OFF from the Windows Machines (Click Start on the bottom left corner and Log Off right above), do NOT just close the Windows Remote Desktop window.

2009 Cisco Systems, Inc. All rights reserved

43 of 44

Recommended Reading

Cisco Nexus 7000 Series Switches: www.cisco.com/en/US/products/ps9402/index.html Cisco NX-OS Feature Navigator: www.cisco.com/go/nxosnav Cisco NX-OS Home Page: www.cisco.com/go/nxos

Complete Your Online Session Evaluation

Cisco values your input. Give us your feedback! We read and carefully consider your scores and comments, and incorporate them into the content program year after year Go to the Internet stations located throughout the Convention Center to complete your session evaluations Thank you!

2009 Cisco Systems, Inc. All rights reserved

44 of 44