Design Guide

Arista Networks Macro-Segmentation

Service™ (MSS™) Design & Deployment Guide

arista.com
 Design Guide

Table of contents

Introduction 4
Solution Overview 5
Use Cases 6
Securing East-West Traffic 6
Monitoring and Securing management traffic 7
Key Benefits 7
MSS Operations 8
Step 1: CloudVision as single point of control 8
Step 2: Firewall rules are implemented by the security team 8
Step 3: CloudVision applies an intercept to steer interesting traffic 9
Step 4: Data plane traffic steering with Macro Segmentation Service 9
Reference Design 10
Logical Topology 10
Firewall Policies 11
Physical Topology 11
Requirements for Arista MSS Integration with Palo Alto Firewall 12
Terminology 15 12
Configuration 13
Step 1: Deploy Arista CloudVision 13
Step 2: Enable the VXLAN Control Service on CVX 13
Step 3: Configure Access switch and Service switch ports 13
Access switch configuration 13
Service Switch port configuration 14
Step 4: Enable DirectFlow on access switches 19 16
Step 5: Enable routing on TOR switches 20 17
Step 6: Configure the MSS service on CVX 18

arista.com
 Design Guide

Table of contents

Step 7: Firewall Configuration 20

Firewall Interfaces and Zones 20
Firewall vWires 21
Firewall Policies 21
Caveats 22
Troubleshooting 23
Ensure the MSS service is enabled 23
Ensure the dynamic device group is enabled 23
Policy is not fetched from the firewall correctly 23
IP-MAC binding not learned by CVX 24
Incorrect Service VNI and Port-VLAN membership 25
Required DirectFlow rules are missing 27
Configuration 30
cvx01 30
cvx02 31
cvx03 32
intercept-1 33
intercept-2 35
service-1 38
service-2 41

arista.com
 Design Guide

Introduction
“I am convinced there are only two types of companies: those that have been hacked and those that will be.” Robert S. Mueller III,
Former FBI Director

Contemporary trends in data center architectures such as mobile applications and IoT present many new possible entry points for
security breaches that are not protected by legacy security infrastructure. Installing a firewall at the Internet edge no longer meets
the security control requirements for many companies. The complexity of providing secure access, protecting critical data and
end-user privacy, and ensuring business continuity in these hyper-dynamic, high-performance compute and data management
infrastructures is leading to a demand for a new approach in network and data security. Arista Macro-Segmentation Service
addresses a growing gap in security deployment models wherein embedded security in the virtualization hypervisor addresses
inter-VM communication and physical firewalls address at-depth protection for north-south traffic leaving the data center.

arista.com 4
 Design Guide

Solution Overview
Arista Macro-Segmentation Service (MSS) provides a software-driven dynamic and scalable network service to insert security
devices into the path of traffic with complete flexibility on placement of service devices and workloads. It is specifically aimed at
physical-to-physical (P-to-P) and physical to virtual (P-to-V) workloads.

Further, it enhances the paradigm of software defined service insertion driving operational efficiencies to eliminate many of the
factors causing unpreparedness in IT and security operations to deal key requirements, such as compliance, threat detection, etc.

What makes MSS unique is that it places the control of policy enforcement directly in the hands of security administrators. This is
accomplished using standards based forwarding with no proprietary frame formats and without placing limitations on where the
devices must exist within the network.

This design and deployment guide outlines a joint Arista Networks and Palo Alto Networks’ solution for modern data center
network security models. Arista Macro-Segmentation Service components include a leaf-spine switch fabric, Arista CloudVision and
a Palo Alto Networks firewall attached to service leaf switches. The goal of such a design is to allow for consistency in application
deployment, scale, manageability and easier scalability of the network and service layers.

Arista CloudVision provides a single point of integration to the network and ties in automation and orchestration capabilities with
a network wide view by aggregating the entire network state. Arista MSS is a service in CloudVision that provides the point of
integration between individual firewalls or firewall manager and the Arista network fabric.

When enabled, Arista MSS communicates with firewall, using the API, and requests security policies of interest. Upon receiving the
policy, CloudVision MSS instantiates intercepts on the required leaf switches to steer traffic from interesting hosts to the firewall for
further inspection.

One of the key requirements today, for deployment with Palo Alto Networks firewalls, is that the firewall must be configured in
virtual wire (vwire) mode. When in vwire mode, the firewall operates in as a bump in the wire and does not route or switch traffic.
This reduces the overhead of network protocol processing on the firewall. Once the intercepted and steered traffic has been
inspected by the firewall, it is sent to the final destination from the service leaf switch.

1
For more information about Palo Alto Networks vwire, please see https://www.paloaltonetworks.com/documentation/71/
pan-os/pan-os/networking/virtual-wire-deployments

arista.com 5
 Design Guide

Use Cases
The use cases below discuss two widely encountered security challenges in the modern datacenter. The first use case deals with
securing east-west traffic between physical-to-physical (P-to-P) and physical to virtual (P-to-V) servers. The second use case deals
with securing management interfaces

Securing East-West Traffic

A three (3) tier application is perhaps the most common security challenge that a network operator encounters. Prior to Micro-
Segmentation or Arista’s Macro-Segmentation, security was accomplished through a “firewall sandwich” approach where firewalls
are placed inline between security zones. A “firewall sandwich” represents significant architectural challenges and can impact both
scalability and performance.

Legacy Approach
Figure 1. Traditional approach to securing traffic in data center

Using Arista MSS, this restriction of firewall placement is removed. Firewalls can now be attached to a service leaf switch in the
network fabric and still protect hosts without regard to their physical location.

arista.com 6
 Design Guide

Figure 2: Modern Approach to securing east-west traffic

Monitoring and Securing management traffic

The modern data center also has an out-of-band network that caters to managing the application, storage, virtualization, network,
analytics and other layers. With virtualization, the hypervisor management also needs to be secured. Should an attacker gain access
to a hypervisor management interface, they could either hop to another device on the network or compromise the local virtual
machines, or even the storage that is often times shared between other hosts.

Arista’s MSS can be used to protect management interfaces. Only explicitly allowed hosts would be allowed access, such as a jump
host or administrator end user computing instances.

Key Benefits

Arista’s Macro-Segmentation Service (MSS) offers the following key benefits:

• Insert security between any physical and virtual workloads in data center

• Automatic and seamlessly orchestrated service insertion - eliminating manual steering of traffic, per workload or tenant

• Security policies follows the host and application throughout the network

• No proprietary frame formats, tagging, or encapsulation

• One point of control – e.g. the security policy manager for physical firewalls

• No server reconfiguration or per application overhead

arista.com 7
 Design Guide

MSS Operations
The section below walks through MSS from a high level.

Step 1: CloudVision as single point of control

Arista’s Macro-Segmentation service is enabled in CloudVision and Arista switches are configured to stream their state in real-time to
it.

Figure 3: Arista CloudVision providing single point of control

This allows CloudVision to build the database of hosts, network switches and service appliances such as firewalls attached to the
network, identifying physical ports, IP addresses, etc.

CloudVision is also configured to communicate and sync policies from the service appliance. As a first integration with Palo Alto, it
can be either a Palo Alto Networks Firewall or Panorama instance.

Step 2: Firewall rules are implemented by the security team

Security policies are created by the security team with the ‘Arista_MSS’ tag name. The figure below shows a snapshot of the Palo Alto
firewall user interface with tagged policies.

Figure 4: Snapshot of Palo Alto Networks Firewall policy rules

CloudVision will send a request to the firewall or firewall manager to provide the details of the security policies and will determine
where traffic needs to be intercepted.

There is continuous polling between the two entities, to ensure any state change is updated in near real-time.

arista.com 8
 Design Guide

Step 3: CloudVision applies an intercept to steer interesting traffic

Once a firewall policy has been created with the configured tag(s) that affects a host that CloudVision is aware of through NetDB
state, CloudVision matches the host’s physical switch port against its database. It then pushes intercept rules to the leaf switches
where the source is located as well as the service leafs where the firewall is attached.

Step 4: Data plane traffic steering with Macro Segmentation Service

Once the intercept has been applied to the leaf switch, the leaf switch begins to send intercepted traffic to the service leaf.

After the traffic ingresses the service leaf where the firewall is attached, it is forwarded completely unmodified to the firewall.

The firewall then applies the required actions based on the configured policy, such as inspection, log, allow or deny.

Finally, the service leaf sends the inspected traffic to its final destination, or the destination of the firewall policy.

arista.com 9
 Design Guide

Reference Design
The reference design below is an example of a typical MSS deployment with a 3-tiered application. The goal of this design is to limit
access between hosts in the Database zone (db1), Application zone (app1), and Web zone (web1).

Figure 5: Logical placement of firewalls between different tiers in modern datacenter with virtualization

Table 1: Different zones, vWires, interfaces and VLANs

Name Ingress Zone Egress Zone vWire VLAN PAN vWire Ingress PAN vWire Egress
Subinterface Subinterface
Database untrust-db trust-db db 100 Et1/1.100 Et1/2.100
Application untrust-app trust-app app 200 Et1/1.200 Et1/2.200
Web untrust-web trust-web web 300 Et1/1.300 Et1/2.300

Logical Topology

From a logical point of view, each server can be considered “on-a-stick,” meaning that each server interface is in its respective trusted
zone. This is contrary to classic firewall policies that have the source zone as an untrusted zone, and the destination zone as the next
application tier. An example of a classic policy is one that allows traffic in on port 80 in the web zone directly to the application zone.
MSS steers traffic towards the firewall and requires the traffic to egress the other side of the firewall on the untrusted side of the
same vWire.

Note that you do not need to configure a zone per server interface. For example, if you had multiple web servers they could all share
the same zones.

arista.com 10
 Design Guide

Once a policy with the proper tag is created, MSS steers ingress traffic towards
the untrusted side of the vWire. The firewall then actions against the traffic as
specified in the policy, and the traffic egresses out the trusted vWire.

Return traffic from the firewall is then reinserted into the original VLAN/VNI on the
service switch and bridged to where it needs to go.

Firewall Policies

End users access the web server through port TCP/443. Traffic flows through the
active firewall to the web server from the untrust-web security zone to the trust-
web security zone. In cases where the intercepted host does not initiate a session,
a return rule may be required for the firewall to allow traffic through. This rule
should not be tagged if the rule allowing traffic in the other direction is tagged.

The web server accesses the application server through port TCP/80 after
traversing the active firewall from the the untrust-app to the trust-app zone.

From there, the application server accesses the database through port TCP/1433
in the untrust-db zone to the trust-db zone.

Note that in the current implementation of MSS, each Layer 2 domain (web/app/
db) needs to be mapped into a distinct Palo Alto vWire, each with its own trusted
and untrusted sides. Arista MSS will steer traffic into the untrusted side of the
firewall, and then the Palo Alto will egress that traffic into the trusted zone.

In a traditional architecture, the firewall would either need to be physically placed

in “front” of every host, or function as a next hop for the entire subnet. With Arista
MSS, the firewall can be anywhere in the network and only protects specific hosts
Figure 6: Logical view of firewall placement with MSS
that have a configured security policy.

Physical Topology

The below graphic shows the physical topology of the reference design:

Figure 7: Physical topology of a reference design for MSS

arista.com 11
 Design Guide

Requirements for Arista MSS Integration with Palo Alto Firewall

• Arista CloudVision running EOS release 4.20.2.1F or later

• Arista switches must:

›› Be connected to the hosts to intercept traffic from and firewall devices

›› Be connected to and monitored by CloudVision

›› support MSS

›› Run EOS release 4.20.2.1F or later

• The network must be a VXLAN enabled fabric with CVX running VXLAN Control Service (VCS)

• Palo Alto Networks firewall(s) running PAN-OS version 7.1.x or greater with ports in Virtual Wire (vwire) mode

• Link Layer Discovery Protocol (LLDP) enabled on firewall interfaces attached to Arista TOR switches. Specifically, transmit-
receive, with port description and system description enabled. Enabling system name and system capabilities is recommended
for troubleshooting. Alternatively, you can manually configure firewall interface mappings.

Terminology

This reference design uses the following terminology:

• Intercept Switch/VTEP: Top of Rack switch and VXLAN tunnel endpoint connected to host from which traffic is intercepted. In
this design, intercept-1 and intercept-2

• Service Switch/VTEP: Top of Rack switch and VXLAN tunnel endpoint connected to firewall. In this design, service-1 and
service-2

• Service VNI: VXLAN tunnel created to redirect intercepted traffic to the firewall

• Intercept Interface: The interface at the Top of Rack switch that receives the packet from the host being intercepted. In this
design, Et3 of intercept-1 is called the intercept interface of traffic originated by web-1

• Egress/Near Interface: The interface on the service VTEP that forwards the intercepted traffic to the firewall.

• Ingress/Far Interface: The interface on the service VTEP that receives the intercepted traffic back from the firewall.

• VXLAN: Virtual eXtensible LAN - a standards based method of encapsulating Layer 2 traffic across a Layer 3 fabric

• CVX: Arista CloudVision eXchange (CVX) is a part of CloudVision and is a virtualized instance of the same Extensible Operating
System (EOS) that runs on physical switches. It functions as a point of integration between Palo Alto Networks firewalls or
Panorama and the Arista network in order to steer interesting traffic to the firewall.

arista.com 12
 Design Guide

Configuration

The steps below outline how to configure Arista MSS.

Step 1: Deploy Arista CloudVision

The first step is to deploy Arista CloudVision and configure the Arista TOR switches to connect to it. A CVX cluster of 3 instances with
hostnames of cvx01, cvx02, and cvx03 have been configured for this design.

Please see the CVX configuration guide for more information.

NOTE: It is a best practice to deploy CVX in a Highly Available (HA) cluster of at least three (3) instances.

Step 2: Enable the VXLAN Control Service on CVX

Once the three (3) Arista CVX instances have been deployed and the ToR switches have been configured to be managed by them,
the VXLAN Control Service (VCS) must be enabled on every CVX instance.

The VXLAN control service allows hardware VXLAN Tunnel End Points (VTEPs) to share state with each other in order to establish
VXLAN tunnels without the need for a multicast control plane.

CVX instance cvx01

cvx01(config-cvx)#service vxlan

cvx01(config-cvx-vxlan)#no shutdown

CVX instance cvx02

cvx02(config-cvx)#service vxlan

cvx02(config-cvx-vxlan)#no shutdown

CVX instance cvx03

cvx02(config-cvx)#service vxlan

cvx02(config-cvx-vxlan)#no shutdown

Step 3: Configure Access switch and Service switch ports

This step involves configuration of the switch ports connected to the hosts, whose traffic needs to be steered to firewalls and ‘service
switch’ which is connected to the firewalls.

Access switch configuration

The switch ports connected to the hosts, whose traffic needs to be intercepted, need to be configured as 802.1q trunks with the
VLAN that is mapped to the VNI requiring interception. Unique VLAN IDs are configured for each tier of the application.

2
https://www.arista.com/en/cg-cv-20171/cv-chapter-2-cloudvision-exchange-cvx

arista.com 13
 Design Guide

Access Switch (intercept-1)

intercept-1# configure
intercept-1(config)# interface et1
intercept-1(config-et1)# description db1
intercept-1(config-et1)# switchport mode trunk
intercept-1(config-et1)# switchport trunk allowed vlan 100
intercept-1(config)# interface et2
intercept-1(config-et2)# description app1
intercept-1(config-et2)# switchport mode trunk
intercept-1(config-et2)# switchport trunk allowed vlan 200
intercept-1(config)# interface et3
intercept-1(config-et3)# description web1
intercept-1(config-et3)# switchport mode trunk
intercept-1(config-et3)# switchport trunk allowed vlan 300

Access Switch (intercept-2)

intercept-2# configure
intercept-2(config)# interface et1
intercept-2(config-et1)# description db1
intercept-2(config-et1)# switchport mode trunk
intercept-2(config-et1)# switchport trunk allowed vlan 100
intercept-2(config)# interface et2
intercept-2(config-et2)# description app1
intercept-2(config-et2)# switchport mode trunk
intercept-2(config-et2)# switchport trunk allowed vlan 200
intercept-2(config)# interface et3
intercept-2(config-et3)# description web1
intercept-2(config-et3)# switchport mode trunk
intercept-2(config-et3)# switchport trunk allowed vlan 300
Note: LLDP is enabled by default on all Arista switches.

NOTE: For untagged traffic, configure a native VLAN on the port using the “switchport trunk native vlan” command.

Service Switch port configuration

A service switch is defined as the switch connecting to the firewalls. Switch ports connected to the firewalls are configured as trunk
ports, with allowed VLANs set to “none”. As MSS builds intercept rules based on configured firewall policies, CVX will dynamically
configure VLANs as required on these ports.

arista.com 14
 Design Guide

Service Switch (service-1)

service-1(config)# interface Port-Channel 101

service-1(config-if-Po101)# switchport trunk allowed vlan none
service-1(config-if-Po101)# switchport mode trunk
service-1(config-if-Po101)# description fw-ha-node-1
service-1(config-if-Po101)# spanning-tree bpdufilter enable
service-1(config-if-Po101)# spanning-tree portfast
service-1(config-if-Po101)# mlag 101
service-1(config)# interface Port-Channel 102
service-1(config-if-Po102)# switchport trunk allowed vlan none
service-1(config-if-Po102)# switchport mode trunk
service-1(config-if-Po102)# description fw-ha-node-2
service-1(config-if-Po102)# spanning-tree bpdufilter enable
service-1(config-if-Po102)# spanning-tree portfast
service-1(config-if-Po102)# mlag 102
service-1(config-et1)# switchport trunk allowed vlan none
service-1(config-et1)# switchport mode trunk
service-1(config-et1)# description fw-ha-node-1
service-1(config-et1)# spanning-tree bpdufilter enable
service-1(config-et1)# spanning-tree portfast
service-1(config-et1)# channel-group 101 mode on
service-1(config-et2)# switchport trunk allowed vlan none
service-1(config-et2)# switchport mode trunk
service-1(config-et2)# description fw-ha-node-2
service-1(config-et2)# spanning-tree bpdufilter enable
service-1(config-et2)# spanning-tree portfast
service-1(config-et2)# channel-group 102 mode on

Service Switch (service-2)

service-2(config)# interface Port-Channel 101

service-2(config-if-Po101)# switchport trunk allowed vlan none
service-2(config-if-Po101)# switchport mode trunk
service-2(config-if-Po101)# description fw-ha-node-1
service-2(config-if-Po101)# spanning-tree bpdufilter enable
service-2(config-if-Po101)# spanning-tree portfast
service-2(config-if-Po101)# mlag 101
service-2(config)# interface Port-Channel 102
service-2(config-if-Po102)# switchport trunk allowed vlan none
service-2(config-if-Po102)# switchport mode trunk
service-2(config-if-Po102)# description fw-ha-node-2
service-2(config-if-Po102)# spanning-tree bpdufilter enable
service-2(config-if-Po102)# spanning-tree portfast

arista.com 15
 Design Guide

service-2(config-if-Po102)# mlag 102

service-2(config-et1)# switchport trunk allowed vlan none
service-2(config-et1)# switchport mode trunk
service-2(config-et1)# description fw-ha-node-1
service-2(config-et1)# spanning-tree bpdufilter enable
service-2(config-et1)# spanning-tree portfast
service-2(config-et1)# channel-group 101 mode on
service-2(config-et2)# switchport trunk allowed vlan none
service-2(config-et2)# switchport mode trunk
service-2(config-et2)# description fw-ha-node-2
service-2(config-et2)# spanning-tree bpdufilter enable
service-2(config-et2)# spanning-tree portfast
service-2(config-et2)# channel-group 102 mode on

NOTE: Dynamically mapped VLANs are not shown in the switchport configuration. They can be viewed by issuing the “show vlan”
command on the switch once a policy is applied.
VXLAN to VNI mapping also needs to be configured. Both switches are configured identically:

interface Vxlan1
vxlan source-interface Loopback0
vxlan controller-client
vxlan udp-port 4789
vxlan vlan 100 vni 10100
vxlan vlan 200 vni 10200
vxlan vlan 300 vni 10300

For more information about how to configure VXLAN, please see the configuration guide.

Step 4: Enable DirectFlow on access switches

Arista MSS uses DirectFlow to steer interesting traffic from the intercepted host the firewall, and back. Directflow must be enabled
on every intercept switch as well as both service switches.

Switch service-1

service-1# configure
service-1(config)# directflow
service-1(config-directflow)# no shutdown

Switch service-2

service-2# configure
service-2(config)# directflow
service-2(config-directflow)# no shutdown

3
https://www.arista.com/assets/data/pdf/user-manual/um-eos/Chapters/VXLAN.pdf

arista.com 16
 Design Guide

Switch intercept-1

intercept-1# configure
intercept-1(config)# directflow
intercept-1(config-directflow)# no shutdown

Switch intercept-2

intercept-2# configure
intercept-2(config)# directflow
intercept-2(config-directflow)# no shutdown

Step 5: Enable routing on TOR switches

CVX uses Address Resolution Protocol (ARP) to determine where intercept hosts are physically located in the network.
It is recommended that VXLAN routing be configured on every ToR and service switch that will be intercepting traffic to ensure that
CVX is aware of every host ARP entry. The configuration below shows the routing configuration for each tier of the application, but
not the entire VXLAN configuration.
For more information on how to configure VXLAN and VXLAN routing, please see the VXLAN section in the EOS Configuration guide.
Switch intercept-1

intercept-1# configure
intercept-1(config)# ip routing
intercept-1(config)# interface vlan100
intercept-1(config-if-Vl100)# ip address virtual 10.10.10.254/24
intercept-1(config)# interface vlan200
intercept-1(config-if-Vl200)# ip address virtual 172.16.20.254/24
intercept-1(config)# interface vlan300
intercept-1(config-if-Vl300)# ip address virtual 192.168.30.254/24

Switch intercept-2

intercept-2# configure
intercept-2(config)# ip routing
intercept-2(config)# interface vlan100
intercept-2(config-if-Vl100)# ip address virtual 10.10.10.254/24
intercept-2(config)# interface vlan200
intercept-2(config-if-Vl200)# ip address virtual 172.16.20.254/24
intercept-2(config)# interface vlan300
intercept-2(config-if-Vl300)# ip address virtual 192.168.30.254/24

4
https://www.arista.com/en/um-eos-4182f/eos-vxlan

arista.com 17
 Design Guide

Before continuing onto step 6, please make sure to verify that VXLAN - both layer 2 bridging and layer 3 routing - is working prior to
enabling MSS. Some troubleshooting steps can be found in the troubleshooting section.

Step 6: Configure the MSS service on CVX

The next step enables the Arista MSS service on CVX. The reference design includes three (3) CVX instances in a cluster, and the
configuration must the same for every instance. In this step both the active and standby Palo Alto firewalls are configured. If
Panorama is used, only Panorama needs to be configured.

In this reference design, the primary Palo Alto firewall has a DNS name of “fw-ha-node-1”, and the standby firewall has a DNS
name of “fw-ha-node-2”. The username and password are both “admin”.

Command Description

service mss Enables the MSS service on CVX

vni range 30000-40000 A dynamic range of VNI’s that will be allocated for VXLAN encapsulated traffic to the
firewall
dynamic device-set panfw1 Creates a set of devices, typically a pair of firewalls with the name ‘panfw1‘
tag Arista_MSS Specifies the tag(s) that MSS looks for when reading security policy from the firewall or
firewall manager. Defaults to ‘Arista_MSS’ - and will not be displayed in the running
configuration. More than one tag can be specified
type palo-alto firewall Sets the firewall type
state active Allows you set the device set as active or disabled
device fw-ha-node-1 Defines a device. Note this is the hostname or IP address that MSS will use to
communicate with it
username admin password 0 admin Sets the username and password to access the device. Once entered the password is
encrypted

CVX instance cvx01

cvx01# configure
cvx01(config)# cvx
cvx01(config-cvx)# no shutdown
cvx01(config-cvx)# service mss
cvx01(config-mss)# no shutdown
cvx01(config-mss)# vni range 30000-40000
cvx01(config-mss)# dynamic device-set panfw1
cvx01(config-mss-panfw1)# tag Arista_MSS
cvx01(config-mss-panfw1)# type palo-alto firewall
cvx01(config-mss-panfw1)# state active
cvx01(config-mss-panfw1)# device fw-ha-node-1
cvx01(config-mss-panfw1-fw-ha-node-1)# username admin password 0 admin
cvx01(config-mss-panfw1-fw-ha-node-1)# device fw-ha-node-2
cvx01(config-mss-panfw1-fw-ha-node-2)# username admin password 0 admin

arista.com 18
 Design Guide

CVX instance cvx02

cvx02# configure
cvx02(config)# cvx
cvx02(config-cvx)# no shutdown
cvx02(config-cvx)# service mss
cvx02(config-mss)# no shutdown
cvx02(config-mss)# vni range 30000-40000
cvx02(config-mss)# dynamic device-set panfw1
cvx02(config-mss-panfw1)# tag Arista_MSS
cvx02(config-mss-panfw1)# type palo-alto firewall
cvx02(config-mss-panfw1)# state active
cvx02(config-mss-panfw1)# device fw-ha-node-1
cvx02(config-mss-panfw1-fw-ha-node-1)# username admin password 0 admin
cvx02(config-mss-panfw1-fw-ha-node-1)# device fw-ha-node-2
cvx02(config-mss-panfw1-fw-ha-node-2)# username admin password 0 admin

CVX instance cvx03

cvx03# configure
cvx03(config)# cvx
cvx03(config-cvx)# no shutdown
cvx03(config-cvx)# service mss
cvx03(config-mss)# no shutdown
cvx03(config-mss)# vni range 30000-40000
cvx03(config-mss)# dynamic device-set panfw1
cvx03(config-mss-panfw1)# tag Arista_MSS
cvx03(config-mss-panfw1)# type palo-alto firewall
cvx03(config-mss-panfw1)# state active
cvx03(config-mss-panfw1)# device fw-ha-node-1
cvx03(config-mss-panfw1-fw-ha-node-1)# username admin password 0 admin
cvx03(config-mss-panfw1-fw-ha-node-1)# device fw-ha-node-2
cvx03(config-mss-panfw1-fw-ha-node-2)# username admin password 0 admin

arista.com 19
 Design Guide

Step 7: Firewall Configuration

The following firewall configuration is used for the reference design.
Firewall Interfaces and Zones
Interfaces have been configured in aggregation groups:
admin@PA850-1> show interface all
<snip>
aggregation groups: 2
ae1 members:
ethernet1/1
ae2 members:
ethernet1/2

total configured logical interfaces: 13

Name zone forwarding tag

------- ----------- -------------- ------
ae1 vwire:ae2 0
ae1.100 web-trust vwire:ae2.100 100
ae1.200 app-trust vwire:ae2.200 200
ae1.300 db-trust vwire:ae2.300 300
ae2 vwire:ae1 0
ae2.100 web-untrust vwire:ae1.100 100
ae2.200 app-untrust vwire:ae1.200 200
ae2.300 db-untrust vwire:ae1.300 300

NOTE: LLDP (transmit and receive) must be enabled on firewall interfaces attached to the service switches. For instructions on
enabling LLDP on Palo Alto Firewall interfaces, please refer to the Palo Alto Networks documentation. To minimize reconvergence
times on network changes consider reducing LLDP transmit interval and holdtime multiples on the firewall.
Alternatively, the “device interface map” command can be used on the CVX to manually map a device to Arista switch interfaces.
In the case of multiple devices, add add a mapping entry for each. Each entry represents a link between a physical interface of the
firewall to a physical interface of the service VTEP. Note that even for an MLAG interface, every member link needs to be enumerated
with its own map command. The MAC addresses used below are the system MAC address of switches in an MLAG pair.
dynamic device-set fw1
device dc-firewall-1
map device-interface Ethernet1/1 switch 28:99:3a:8a:5d:e7 interface Ethernet1
map device-interface Ethernet1/3 switch 28:99:3a:8a:ba:53 interface Ethernet2
map device-interface Ethernet1/4 switch 28:99:3a:8a:ba:53 interface Ethernet1
map device-interface Ethernet1/2 switch 28:99:3a:8a:5d:e7 interface Ethernet2

5
For Palo Alto Networks PAN-OS documentation, please see https://www.paloaltonetworks.com/documentation/71/pan-os

arista.com 20
 Design Guide

Firewall vWires

Palo Alto vWires have been configured with the above interfaces:
admin@PA850-1> show virtual-wire all
total virtual-wire shown : 4
flags : m - multicast firewalling
p - link state pass-through
s - vlan sub-interface
i - ip+vlan sub-interface
t - tenant sub-interface

name interface1 interface2 flags allowed-tags

----------------------------------------------------------------------------
default_vwire ae1 ae2 mp
untrust-web ae1.100 ae2.100 ms 100
web-app ae1.200 ae2.200 ms 200
app-db ae1.300 ae2.300 ms 300

Firewall Policies

For this reference design, 3 policies are created in addition to the default implicit deny policy for interzone traffic. Note that the
implicit deny will ensure that interzone traffic will not be allowed unless a policy explicitly allows for it.

The first policy “web” is from the untrust web zone to the trust web zone, that allows HTTPS (web-browsing) traffic from anywhere to
the web server web1.

The second policy “app” is from the untrust app zone to the trust app zone that allows HTTP (web-browsing) traffic between the web
server web1 and the application server app1.

The third policy “db” is from the untrust db zone to the trust db zone that allows database traffic on port TCP/1433 (mssql-db)
between the app server app1 and the database server db1.

In order to build a resilient design capable of tolerating the complete loss of an edge rack, it is also recommended to deploy two sets
of four edge gateways in two separate edge racks. The below table describes the necessary configuration with ECMP edge.

Figure 8: Snapshot of Palo Alto Networks Firewall policy rules

To create a rule that Arista MSS will use to intercept and redirect traffic, add a firewall policy with the default “Arista_MSS” tag as
shown in the example above. MSS intercepts all traffic from endpoints identified in policies that match the tag value(s) configured in
CVX. The firewall will apply all rules - tagged or untagged - to all traffic it sees.
Note that return traffic does not require an MSS tag to function, but it still needs to be specified for the other side of the vWire.

arista.com 21
 Design Guide

Caveats
This section outlines some design guidelines for deploying Arista Macro-Segmentation Service.

Note that this section does not remove the requirement to review the release notes for known issues and platforms supported.

Firewall:

• All policies configured on the firewall must not have any whitespace character in their names, for example “PCI Zone” should be
“PCI_Zone”
• A maximum of 255 intercept hosts are supported per VLAN
• A firewall policy with ‘any’ source and ‘any’ destination zone cannot be tagged to be used with MSS, including the default Palo
Alto any/any policy. An alternate method is to tag a single policy that defines traffic from/ to a specific host or subnet that needs
to be intercepted and steered to the firewall.
General:

• VXLAN routing with MSS is only supported with Direct (Asymmetric) Routing.
• VXLAN routing with MSS is only supported with ‘ip address virtual’ configuration.
• The current implementation divides the original layer-2 domain into two subsets and places the FW in between the two
subsets. This adds a restriction where policies between multiple hosts can exist on either side of the FW. Consider the scenario
where hosts A, B and C communicate with each other as shown below:

Figure 9: Traffic flow between hosts in same VLAN, which needs to be inspected

• The current implementation mandates logical placement of certain hosts behind the firewall, as there are is only two sides of
a firewall vWire. This means, the traffic steering, for inspection, can be achieved for traffic between host A and host B as well as
between host A and host C, but not between host B and host C, as B and C would be considered to be on the same side of the
firewall, as shown below:

Figure 10: Traffic flow between hosts in same vlan, with Firewall enabled in vWire mode

• As with any system, certain scale factors need to be taken into consideration, while implementing intercepts. These factors
consist of, but aren’t limited to: number of intercepted hosts connected to the same switch, switch features used, number of
unique VLANs etc. The best practice is to tag a single rule on the firewall, per end-point that needs to be protected, even though
the FW can have a complex set of policies for the end-point. This allows any traffic to the endpoint to be steered to the firewall
for inspection, and scaling the intercept table on the switches as well.

arista.com 22
 Design Guide

Troubleshooting
This section contains a series of troubleshooting steps when traffic from a particular host are not being correctly intercepted by MSS.

Ensure the MSS service is enabled

The MSS service should be enabled on every CVX instance. To verify, run the following commands on CVX:

cvx# show service mss status

State: Enabled
Service VNIs: 30000-40000

The state should be “Enabled”, and reflect the configured VNI range. If it is not enabled, please reference Step 6: Configure the MSS
service on CVX

Ensure the dynamic device group is enabled

cvx# show service mss dynamic

Accessing external device(s), this may take a few seconds...
-----------------------------------------------------------------
Policy Source Device Set Service Device State
-------------------- -------------------- -------------------- --------------------
palo-alto-firewall panfw pan100 active

The state should be “active”. If the state is not active, please reference Step 6: Configure the MSS service on CVX

Policy is not fetched from the firewall correctly

The following command will list all the policies retrieved from the firewall by Arista MSS:

cvx# show service mss dynamic device-set <device_set_name> device <device_name> policies

• If no policies are seen by Arista MSS, check if CVX is able to communicate with the firewall or firewall manager using the
following command:
cvx# show service mss dynamic status
Service Device Policy Monitoring Status:
Device: pan100
State: Active
Device set name: panfw
Policy source type: paloAltoFirewall
Number of policies processed: 3
Last seen at time: 2017 Jun 22, 17:57:36

NOTE: The number of policies processed is an aggregate number and should increment every time MSS polls Palo Alto for new
policies.
• If only corresponding policy is missing, check the policy tag configured on the firewall policy. See Step 7: Firewall Configuration
for more information.

arista.com 23
 Design Guide

IP-MAC binding not learned by CVX

Check the status of the policy to ensure that CVX has the necessary information to redirect traffic:

cvx# show service mss policy name <policy_name>

Source Device Policy Config Status
-------------------- -------------------- -------------------- ---------- ------------------
paloAltoFirewall pan100 policy1 Enabled Initialized

• If the policy status is not “Initialized,” check the ARP table information received by CVX.

cvx# show service vxlan arp received

Received ARP Table
-------------------------------------------------------------------------
Switch VNI IP Address MAC Address Changes
------------------- -------- --------------- ------------------ ---------
00-00-91-02-00-00 1000 10.10.10.102 00:00:01:02:00:00 0
00-1c-73-00-e2-16 2000 10.10.20.103 00:00:01:03:00:00 0

• If the IP Address of the host is not seen in the CVX ARP table, ICMP ping a host which is not on the same subnet (or VNI) as the
intercept host and verify ARP table information again. If ARP information for the host is learned by the CVX after the ping, check
the status of the policy and ensure it’s “Initialized”.

• If the situation still persists, run the following commands on the intercept VTEP. If the host MAC address is learned on the VXLAN
interface (Vxlan1), this indicates that there is a Layer 2 (L2) loop in the network. Resolve the loop and verify the policy status
again.

intercept-switch# show arp

Address Age (min) Hardware Addr Interface
10.10.100.1 N/A 0000.0101.0000 Vlan100, Vxlan1

intercept-switch# show mac address-table

Mac Address Table
------------------------------------------------------------------

Vlan Mac Address Type Ports Moves Last Move

---- ----------- ---- ----- ----- ---------
100 0000.0101.0000 DYNAMIC Vx1 186 0:23:02 ago

arista.com 24
 Design Guide

Incorrect Service VNI and Port-VLAN membership

An incorrect service VNI and port-VLAN membership can cause host traffic of an initialized policy to not be correctly intercepted.

To troubleshoot, first look at which VNI is used to tunnel the traffic to the Service VTEP. This information can be obtained by running
the following command and looking at the value of the service VNI:
cvx#show service mss internal policy advertised
-------------------------------------------------------------------
Switch: 00-1c-73-7e-21-5d
Policies:
--------------------------------------------------------------
Name: panFW:10.92.64.99_N:Et1_7e215d_F:Et2_7e215d_V:100
Original Vni: 11000
Service Vni: 20000
Flows:
---------------------------------------------------------
Flow Name: InsideVtepEgress
Match:
Source Mac:
Destination Mac:
Vni: 20000
Action:
Rewrite Vni: 11000
Intercept Point: Egress
Service Interfaces:
---------------------------------------------------------
Flow Name: InsideVtepIngress-152.0.10.10
Match:
Source Mac: 00:50:56:b5:eb:d8
Destination Mac:
Vni: 11000
Action:
Rewrite Vni: 20000
Intercept Point: Ingress
Service Interfaces:
---------------------------------------------------------
Flow Name: ServiceVtepEgress
Match:
Source Mac:
Destination Mac:
Vni: 20000
Action:
Rewrite Vni: 11000
Intercept Point: Egress
Service Interfaces: Port-Channel10
---------------------------------------------------------
Flow Name: ServiceVtepIngress
Match:
Source Mac:
Destination Mac:
Vni: 11000

Action:
Rewrite Vni: 20000
Intercept Point: Ingress
Service Interfaces: Port-Channel10

arista.com 25
 Design Guide

NOTE: The policy name show above is the internal PAN policy name and cannot be modified at this time.

On the intercept and service VTEPs, check the VLAN to VNI mapping and VTEP floodlist for the service VNI:

switch# show interface Vxlan1

Vxlan1 is up, line protocol is up (connected)
Hardware is Vxlan
Source interface is Loopback0 and is active with 10.10.102.1
Replication/Flood Mode is headend with Flood List Source: VCS
Remote MAC learning via VCS
Static VLAN to VNI mapping is
[100, 1000]
Dynamic VLAN to VNI mapping for ‘mss’ is
[1008, 5000]
Note: All Dynamic VLANs used by VCS are internal VLANs.
Use ‘show vxlan vni’ for details.
Static VRF to VNI mapping is not configured
Headend replication flood vtep list is:
100 10.10.102.3 10.10.102.2 10.10.102.1
1008 10.10.102.3 10.10.102.1

In this example, “Vxlan1” is the name of the VXLAN interface. 5000 is the service VNI and VLAN 1008 is mapped to it. Note that the
VLAN is local to the VTEP. Service and intercept VTEPs can have different VLAN to VNI mappings for the same service VNI.

Next, check the port VLAN membership on the intercept VTEP. On the intercept VTEP, the intercept interface should be present in
the dynamically configured service VLAN (denoted by a *). In this example, interface Et1 is the host facing interface.

intercept-switch# show vlan 1008

VLAN Name Status Ports
----- -------------------------------- --------- -----------------
1008* VLAN1008 active Et1, Vx1

On the service VTEP, the firewall egress interface should be a member of the service VLAN and the intercept interface needs to be a
member of the original VLAN. In this example, Et10 is the egress interface that sends the packet towards the firewall and interface
Et20 is the ingress interface receiving the packet back from the firewall.

service-switch# show vlan

VLAN Name Status Ports
----- -------------------------------- --------- --------------
100 VLAN100 active Et20, Vx1
1009* VLAN1009 active Et10, Vx1

arista.com 26
 Design Guide

Required DirectFlow rules are missing

The VTEP needs a few DirectFlow rules to facilitate the packet flow. The command “show directflow detail” helps determine if rules
are missing.
The DirectFlow rules seen at the intercept VTEP can be divided into two broad categories, ingress rules and egress rules. Ingress rules
can again be divided into two types, host specific ingress rules and service VLAN specific rules.
There are at most two host specific ingress rules that forward the packet to service VLAN. One rule transfers any traffic from the host
coming with a VLAN tag from the original VLAN to the service VLAN. In addition to the original VLAN, this rule also matches host
MAC address and the switch intercept interface. This rule is always present for every intercepted host learned on a particular VTEP.
Another class of ingress rule can be observed for all host traffic that can be received by a VTEP without a VLAN tag. As MSS supports
only trunk intercept interfaces, this type of traffic is seen only on the original VLAN that matches the native VLAN of the intercept
interface. The name of this type of rule ends with “untagged”. This rule also matches host MAC address, original VLAN, and intercept
interface. Note that if no intercept host is learned in the native VLAN or native VLAN is not configured, the intercept VTEP does not
have this rule.
The service VLAN specific ingress rule allows ARP traffic on the service VLAN to be forwarded towards the firewall. The name of this
rule ends with ARP and mentions only the service VLAN in its match criteria.
On the egress side, there is a set of rules on every intercept interface. This rule matches the service VLAN and translate the packet
back to the original VLAN before sending it out on the intercept interface. Note that when multiple hosts on the same original VLAN
are learned on the same intercept interface, only one rule is sufficient to support the necessary egress translation; the rule does not
have any match criteria on the host MAC address.
intercept-switch# show directflow detail
Flow panFW:pan100_N:Et3_8aba53+Et3_8a5de7_F:Et4_8aba53+Et4_8a5de7_V:1502_InsideVtepIngress-
150.2.1.211-from-Po122-to-any: (Flow programmed)
persistent: False
priority: 0
priorityGroupType: default
tableType: vfp
hard timeout: 0
idle timeout: 0
match:
ingress interface:
Po122
source Ethernet address: 00:0c:29:7b:ba:af/ff:ff:ff:ff:ff:ff
VLAN ID: 1502
actions:
set VLAN ID to: 1012
forward normally
source: mss
matched: 0 packets, 0 bytes

Flow panFW:pan100_InsideVtepIngress-150.2.1.211-from-Po122-to-any-untagged: (Flow programmed)

persistent: False
priority: 0
priorityGroupType: default
tableType: vfp

arista.com 27
 Design Guide

hard timeout: 0
idle timeout: 0
match:
ingress interface:
Po122
source Ethernet address: 00:0c:29:7b:ba:af/ff:ff:ff:ff:ff:ff
Vlan Untagged
actions:
set VLAN ID to: 1012
forward normally
source: mss
matched: 0 packets, 0 bytes

Flow panFW:pan100-arp: (Flow programmed)

persistent: False
priority: 0
priorityGroupType: default
tableType: ifp
hard timeout: 0
idle timeout: 0
match:
VLAN ID: 1012
actions:
forward normally
source: mss
matched: 0 packets, 0 bytes

Flow panFW:pan100_InsideVtepEgress-from-any-to-Po122: (Flow programmed)

persistent: False
priority: 0
priorityGroupType: default
tableType: egress vlan translate
hard timeout: 0
idle timeout: 0
match:
VLAN ID: 1012
actions:
set VLAN ID to: 1502
output interfaces:
Po122
drop outer tag
source: mss
matched: 0 packets, 0 bytes

arista.com 28
 Design Guide

The DirectFlow rules on the Service VTEP are more generic and do not match any host MAC address, however they can also be
divided into ingress and egress rules.

On the ingress side, there is a DirectFlow rule that matches any packet received at the Near Interface on original VLAN and translates
it to service VLAN. Like the intercept VTEP, the service VTEP also has a rule that forwards ARP traffic on the service VLAN. Traffic
egressing from the service VTEP towards the firewall hits an egress rule and the packet is translated from the service VLAN to the
original VLAN.

service-switch# show directflow detail

Flow panFW:pan10_ServiceVtepIngress-from-Et12/4-to-any: (Flow programmed)
persistent: False
priority: 0
priorityGroupType: default
tableType: vfp
hard timeout: 0
idle timeout: 0
match:
ingress interface:
Et12/4
VLAN ID: 100
actions:
set VLAN ID to: 1008
forward normally
source: mss
matched: 35 packets, 4038 bytes

Flow panFW:pan100-arp: (Flow programmed)

persistent: False
priority: 0
priorityGroupType: default
tableType: ifp
hard timeout: 0
idle timeout: 0
match:
VLAN ID: 1008
actions:
forward normally
source: mss
matched: 0 packets, 0 bytes

Flow panFW:pan10_ServiceVtepEgress-from-any-to-Et12/4: (Flow programmed)

persistent: False
priority: 0
priorityGroupType: default
tableType: egress vlan translate
hard timeout: 0

arista.com 29
 Design Guide

idle timeout: 0
match:
VLAN ID: 1008
actions:
set VLAN ID to: 100
output interfaces:
Et12/4
source: mss
matched: 0 packets, 0 bytes

Note that the actual flow names reported in this command can vary and may contain several components to uniquely identify the
source of the flow.

Configuration
Below are the full configurations for each device, including CVX. Both intercept and service switches are Arista 7050SX-72Q models.
Please note that configuration may vary depending on device. Please see the Arista configuration guide for more details.

cvx01
cvx
no shutdown
!
service mss
no shutdown
vni range 30000-40000
!
dynamic device-set panfw
device fw-ha-node-1
username admin password 0 admin
!
device fw-ha-node-2
username admin password 0 admin
state active
type palo-alto firewall
!
service vxlan
no shutdown
!
transceiver qsfp default-mode 4x10G
!
hostname cvx01
!
ntp server 10.1.1.1
!
spanning-tree mode mstp
!
username admin role network-admin secret sha512 <removed>
!
clock timezone US/Pacific
!
interface Management1
ip address 10.10.1.11/24
!
ip route 0.0.0.0/0 10.10.1.1

arista.com 30
 Design Guide

!
no ip routing
!
management cvx
no shutdown
!
end

cvx02
cvx
no shutdown
!
service mss
no shutdown
vni range 30000-40000
!
dynamic device-set panfw
device fw-ha-node-1
username admin password 0 admin
!
device fw-ha-node-2
username admin password 0 admin
state active
type palo-alto firewall
!
service vxlan
no shutdown
!
transceiver qsfp default-mode 4x10G
!
hostname cvx02
!
ntp server 10.1.1.1
!
spanning-tree mode mstp
!
username admin role network-admin secret sha512 <removed>
!
clock timezone US/Pacific
!
interface Management1
ip address 10.10.1.12/24

!
ip route 0.0.0.0/0 10.10.1.1
!
no ip routing
!
management cvx
no shutdown
!
end

arista.com 31
 Design Guide

cvx03

cvx
no shutdown
!
service mss
no shutdown
vni range 30000-40000
!
dynamic device-set panfw
device fw-ha-node-1
username admin password 0 admin
!
device fw-ha-node-2
username admin password 0 admin
state active
type palo-alto firewall
!
service vxlan
no shutdown
!
transceiver qsfp default-mode 4x10G
!
hostname cvx03
!
ntp server 10.1.1.1
!
spanning-tree mode mstp
!
username admin role network-admin secret sha512 <removed>
!
clock timezone US/Pacific
!
interface Management1
ip address 10.10.1.13/24
!
ip route 0.0.0.0/0 10.10.1.1
!
no ip routing
!
management cvx
no shutdown
!
end

arista.com 32
 Design Guide

intercept-1

transceiver qsfp default-mode 4x10G

!
hostname intercept-1
ip name-server vrf mgmt 10.0.0.10
ip name-server vrf mgmt 10.0.0.11
!
spanning-tree mode mstp
!
aaa authorization exec default local
aaa authorization commands all default local
!
username admin privilege 15 role network-admin secret sha512 <removed>
!
vlan 100,200,300
!
vrf definition mgmt
rd 1:1
!
<clipped>
!
interface Ethernet10
description Web Server
switchport trunk allowed vlan 100
switchport mode trunk
!
<clipped>
!
interface Ethernet16
description App Server
switchport trunk allowed vlan 200
switchport mode trunk
!
<clipped>
!
interface Ethernet49/1
description Spine-1
speed forced 40gfull
no switchport
ip address 10.0.1.4/31
!
interface Ethernet49/2
shutdown
!
interface Ethernet49/3
shutdown
!
interface Ethernet49/4
shutdown
!

arista.com 33
 Design Guide

interface Ethernet50/1
description Spine-2
speed forced 40gfull
no switchport
ip address 10.0.2.4/31
!
interface Ethernet50/2
shutdown
!
interface Ethernet50/3
shutdown
!
interface Ethernet50/4
shutdown
!
<clipped>
!
interface Loopback0
ip address 192.168.1.1/32
!
interface Loopback1
ip address 192.168.2.1/32
ip address 192.168.2.50/32 secondary
!
interface Management1
vrf forwarding mgmt
ip address 10.0.0.202/24
!
interface Vlan100
ip address virtual 10.0.10.254/24
!
interface Vlan200
ip address virtual 10.0.20.254/24
!
interface Vlan300
ip address virtual 10.0.30.254/24
!
interface Vxlan1
vxlan source-interface Loopback1
vxlan controller-client
vxlan udp-port 4789
vxlan vlan 100 vni 11000
vxlan vlan 200 vni 12000
vxlan vlan 300 vni 13000
!
directflow
no shutdown
!
ip virtual-router mac-address 00:aa:aa:aa:aa:aa
!
ip route vrf mgmt 0.0.0.0/0 10.0.0.1
!

arista.com 34
 Design Guide

ip routing
ip routing vrf mgmt
!
router bgp 65532.20300
bgp asn notation asdot
router-id 192.168.1.1
maximum-paths 4 ecmp 4
neighbor eBGP-Spine peer-group
neighbor eBGP-Spine remote-as 65532.20100
neighbor eBGP-Spine send-community
neighbor eBGP-Spine maximum-routes 12000
neighbor 10.0.1.5 peer-group eBGP-Spine
neighbor 10.0.2.5 peer-group eBGP-Spine
network 192.168.1.1/32
network 192.168.2.1/32
network 192.168.2.50/32
!
management api http-commands
protocol http
no shutdown
!
vrf mgmt
no shutdown
!
management cvx
no shutdown
server host 10.0.0.153
server host 10.0.0.154
server host 10.0.0.155
source-interface Management1
vrf mgmt
!
end

intercept-2

transceiver qsfp default-mode 4x10G

!
hostname intercept-2
ip name-server vrf default 10.0.0.10
ip name-server vrf default 10.0.0.11
!
spanning-tree mode mstp
!
aaa authorization exec default local
aaa authorization commands all default local
!
username admin privilege 15 role network-admin secret sha512 <removed>
!

arista.com 35
 Design Guide

vlan 100,200,300
!
vrf definition mgmt
rd 1:1
!
<clipped>
!
interface Ethernet10
description DB Server
switchport trunk allowed vlan 300
switchport mode trunk
!
<clipped>
!
interface Ethernet49/1
description Spine-1
speed forced 40gfull
no switchport
ip address 10.0.1.6/31
!
interface Ethernet49/2
shutdown
!
interface Ethernet49/3
shutdown
!
interface Ethernet49/4
shutdown
!
interface Ethernet50/1
description Spine-2
speed forced 40gfull
no switchport
ip address 10.0.2.6/31
!
interface Ethernet50/2
shutdown
!
interface Ethernet50/3
shutdown
!
interface Ethernet50/4
shutdown
!
interface Loopback0
ip address 192.168.1.2/32
!
interface Loopback1
ip address 192.168.2.2/32
ip address 192.168.2.50/32 secondary
!

arista.com 36
 Design Guide

interface Management1
vrf forwarding mgmt
ip address 10.0.0.203/24
!
interface Vlan100
ip address virtual 10.0.10.254/24
!
interface Vlan200
ip address virtual 10.0.20.254/24
!
interface Vlan300
ip address virtual 10.0.30.254/24
!
interface Vxlan1
vxlan source-interface Loopback1
vxlan controller-client
vxlan udp-port 4789
vxlan vlan 100 vni 11000
vxlan vlan 200 vni 12000
vxlan vlan 300 vni 13000
!
directflow
no shutdown
!
ip virtual-router mac-address 00:aa:aa:aa:aa:aa
!
ip route vrf mgmt 0.0.0.0/0 10.0.0.1
!
ip routing
ip routing vrf mgmt
!
router bgp 65532.20400
bgp asn notation asdot
router-id 192.168.1.2
maximum-paths 4 ecmp 4
neighbor eBGP-Spine peer-group
neighbor eBGP-Spine remote-as 65532.20100
neighbor eBGP-Spine send-community
neighbor eBGP-Spine maximum-routes 12000
neighbor 10.0.1.7 peer-group eBGP-Spine
neighbor 10.0.2.7 peer-group eBGP-Spine
network 192.168.1.2/32
network 192.168.2.2/32
network 192.168.2.50/32
!
management api http-commands
protocol http
no shutdown
!
vrf mgmt
no shutdown
!

arista.com 37
 Design Guide

management cvx
no shutdown
server host 10.0.0.153
server host 10.0.0.154
server host 10.0.0.155
source-interface Management1
vrf mgmt
!
end

service-1

service interface unconnected expose

!
transceiver qsfp default-mode 4x10G
!
hostname service-1
ip name-server vrf default 10.0.0.10
ip name-server vrf default 10.0.0.10
!
spanning-tree mode mstp
!
aaa authorization exec default local
aaa authorization commands all default local
!
no aaa root
!
username admin privilege 15 role network-admin secret sha512 <removed>
!
vlan 100,200,300
!
vlan 4094
trunk group mlagpeer
!
vrf definition mgmt
rd 1:1
!
interface Port-Channel1
description MLAG Peer Link
switchport mode trunk
switchport trunk group mlagpeer
!
interface Port-Channel101
description fw-ha-node-1
switchport trunk allowed vlan none
switchport mode trunk
spanning-tree bpdufilter enable
spanning-tree mode portfast
mlag 101
!

arista.com 38
 Design Guide

interface Port-Channel102
description fw-ha-node-2
switchport trunk allowed vlan none
switchport mode trunk
spanning-tree bpdufilter enable
spanning-tree mode portfast
mlag 102
!
interface Recirc-Channel627
no switchport
switchport recirculation features vxlan
!
interface Ethernet1
description fw-ha-node-1
channel-group 101 mode on
!
interface Ethernet2
description fw-ha-node-2
channel-group 102 mode on
!
interface Ethernet3
description MLAG
channel-group 1 mode active
!
interface Ethernet4
description MLAG
channel-group 1 mode active
!
<clipped>
!
interface Ethernet49/1
description Spine-1
speed forced 40gfull
no switchport
ip address 10.0.1.8/31
!
interface Ethernet50/1
description Spine-2
speed forced 40gfull
no switchport
ip address 10.0.2.8/31
!
<clipped>
!
interface Loopback0
ip address 192.168.1.3/32
!
interface Loopback1
ip address 192.168.2.3/32
ip address 192.168.2.50/32 secondary
!

arista.com 39
 Design Guide

interface Management1
vrf forwarding mgmt
ip address 10.0.0.208/24
!
interface UnconnectedEthernet1
traffic-loopback source system device mac
channel-group recirculation 627
!
<clipped>
!
interface Vlan100
ip address virtual 10.0.10.254/24
!
interface Vlan200
ip address virtual 10.0.20.254/24
!
interface Vlan300
ip address virtual 10.0.30.254/24
!
interface Vlan4094
ip address 10.0.4.81/24
!
interface Vxlan1
vxlan source-interface Loopback1
vxlan controller-client
vxlan udp-port 4789
vxlan vlan 100 vni 11000
vxlan vlan 200 vni 12000
vxlan vlan 300 vni 13000
!
directflow
no shutdown
!
ip virtual-router mac-address 00:aa:aa:aa:aa:aa
!
ip route vrf mgmt 0.0.0.0/0 10.0.0.1
!
ip routing
ip routing vrf mgmt
!
mlag configuration
domain-id mlag
local-interface Vlan4094
peer-address 10.0.4.82
primary-priority 70
peer-link Port-Channel1
!
router bgp 65532.20500
bgp asn notation asdot
router-id 192.168.1.3
maximum-paths 4 ecmp 4
neighbor eBGP-Spine peer-group
neighbor eBGP-Spine remote-as 65532.20100

arista.com 40
 Design Guide

neighbor eBGP-Spine send-community

neighbor eBGP-Spine maximum-routes 12000
neighbor 10.0.1.9 peer-group eBGP-Spine
neighbor 10.0.2.9 peer-group eBGP-Spine
network 192.168.1.3/32
network 192.168.2.3/32
network 192.168.2.50/32
!
management api http-commands
protocol http
no shutdown
!
vrf mgmt
no shutdown
!
management cvx
no shutdown
server host 10.0.0.153
server host 10.0.0.154
server host 10.0.0.155
source-interface Management1
vrf mgmt
!
end

service-2
service interface unconnected expose
!
transceiver qsfp default-mode 4x10G
!
hostname service-2
ip name-server vrf default 10.0.0.10
ip name-server vrf default 10.0.0.10
!
spanning-tree mode mstp
!
aaa authorization exec default local
aaa authorization commands all default local
!
no aaa root
!
username admin privilege 15 role network-admin secret sha512 <removed>
!
vlan 100,200,300
!
vlan 4094
trunk group mlagpeer
!
vrf definition mgmt
rd 1:1
!

arista.com 41
 Design Guide

interface Port-Channel1
description MLAG Peer Link
switchport mode trunk
switchport trunk group mlagpeer
!
interface Port-Channel101
description fw-ha-node-1
switchport trunk allowed vlan none
switchport mode trunk
spanning-tree bpdufilter enable
spanning-tree mode portfast
mlag 101
!
interface Port-Channel102
description fw-ha-node-2
switchport trunk allowed vlan none
switchport mode trunk
spanning-tree bpdufilter enable
spanning-tree mode portfast
mlag 102
!
interface Recirc-Channel627
no switchport
switchport recirculation features vxlan
!
interface Ethernet1
description fw-ha-node-1
channel-group 101 mode on
!
interface Ethernet2
description fw-ha-node-2
channel-group 102 mode on
!
interface Ethernet3
description MLAG
channel-group 1 mode active
!
interface Ethernet4
description MLAG
channel-group 1 mode active
!
<clipped>
!
interface Ethernet49/1
description Spine-1
speed forced 40gfull
no switchport
ip address 10.0.1.10/31
!
interface Ethernet50/1
description Spine-2
speed forced 40gfull
no switchport
ip address 10.0.2.10/31
!

arista.com 42
 Design Guide

<clipped>
!
interface Loopback0
ip address 192.168.1.4/32
!
interface Loopback1
ip address 192.168.2.4/32
ip address 192.168.2.50/32 secondary
!
interface Management1
vrf forwarding mgmt
ip address 10.0.0.209/24
!
interface UnconnectedEthernet1
traffic-loopback source system device mac
channel-group recirculation 627
!
<clipped>
!
interface Vlan100
ip address virtual 10.0.10.254/24
!
interface Vlan200
ip address virtual 10.0.20.254/24
!
interface Vlan300
ip address virtual 10.0.30.254/24
!
interface Vlan4094
ip address 10.0.4.82/24
!
interface Vxlan1
vxlan source-interface Loopback1
vxlan controller-client
vxlan udp-port 4789
vxlan vlan 100 vni 11000
vxlan vlan 200 vni 12000
vxlan vlan 300 vni 13000
!
directflow
no shutdown
!
ip virtual-router mac-address 00:aa:aa:aa:aa:aa
!
ip route vrf mgmt 0.0.0.0/0 10.0.0.1
!
ip routing
ip routing vrf mgmt
!
mlag configuration
domain-id mlag
local-interface Vlan4094
peer-address 10.0.4.81
primary-priority 70
peer-link Port-Channel1
!

arista.com 43
 Design Guide

router bgp 65532.20600

bgp asn notation asdot
router-id 192.168.1.4
maximum-paths 4 ecmp 4
neighbor eBGP-Spine peer-group
neighbor eBGP-Spine remote-as 65532.20100
neighbor eBGP-Spine send-community
neighbor eBGP-Spine maximum-routes 12000
neighbor 10.0.1.9 peer-group eBGP-Spine
neighbor 10.0.2.9 peer-group eBGP-Spine
network 192.168.1.3/32
network 192.168.2.3/32
network 192.168.2.50/32
!
management api http-commands
protocol http
no shutdown
!
vrf mgmt
no shutdown
!
management cvx
no shutdown
server host 10.0.0.153
server host 10.0.0.154
server host 10.0.0.155
source-interface Management1
vrf mgmt
!
end

arista.com 44
 Design Guide

Santa Clara—Corporate Headquarters Ireland—International Headquarters India—R&D Office

3130 Atlantic Avenue Global Tech Park, Tower A & B, 11th Floor

5453 Great America Parkway,
Westpark Business Campus Marathahalli Outer Ring Road
Santa Clara, CA 95054 Shannon, Co. Clare Devarabeesanahalli Village, Varthur Hobli
Ireland Bangalore, India 560103
Phone: +1-408-547-5500
Vancouver—R&D Office Singapore—APAC Administrative Office
Fax: +1-408-538-8920
9200 Glenlyon Pkwy, Unit 300 9 Temasek Boulevard

Email: [email protected] Burnaby, British Columbia #29-01, Suntec Tower Two
Canada V5J 5J8 Singapore 038989
San Francisco—R&D and Sales Office 1390 Nashua—R&D Office
Market Street, Suite 800 10 Tara Boulevard
San Francisco, CA 94102 Nashua, NH 03062

Copyright © 2016 Arista Networks, Inc. All rights reserved. CloudVision, and EOS are registered trademarks and Arista Networks
is a trademark of Arista Networks, Inc. All other company names are trademarks of their respective holders. Information in this
document is subject to change without notice. Certain features may not yet be available. Arista Networks, Inc. assumes no
responsibility for any errors that may appear in this document. Jan 26, 2018 07-0011-01

arista.com 45