Demystifying EVPN in the data center: Part 1 in 2 episode series
9 likes2,463 views
The document discusses Ethernet VPN (EVPN) technology and its application in modern data center architectures, emphasizing its advantages over traditional VLAN-based architectures. It highlights the use cases for replacing legacy systems with EVPN, the operational benefits, and the technical specifications including BGP deployment models and MAC address handling. The material concludes with practical configuration examples for implementing EVPN, focusing on simplification to reduce potential errors.
Demystifying EVPN in the data center: Part 1 in 2 episode series
- 1. 1 Oct 12, 2017 Dinesh G Dutt | Cumulus Networks Part 1: Technology, Use Cases, Bridging Operationalizing EVPN in the DC
- 2. 2Cumulus Networks What is EVPN ? Why should you care ? Use cases and requirements BGP models for EVPN in FRR EVPN for bridging Configuring EVPN in FRR Agenda
- 3. 3Cumulus Networks What is EVPN • Ethernet VPN i.e. another form of L2 VPN ▪ Different from VPLS • Original EVPN RFC: RFC 7432 ▪ BGP MPLS-based Ethernet VPN ▪ Requirements defined in RFC 7209
- 4. 4Cumulus Networks Primary Goals of EVPN • Overcome the limitations of VPLS ▪ Support for multihoming and redundancy ▪ No data plane learning => no flooding ▪ Multicast optimization ▪ Allows supporting multiple encapsulation types (signaled via control protocol) ▪ Less configuration
- 5. 5 Wait! This all sounds like service provider stuff. Why should I care ?
- 6. 6Cumulus Networks The Story In the Data Center So Far SPINE LEAF • CLOS is the new network architecture • IP-based fabrics is in, VLAN/L2-based fabrics is out • Scale out wins over scale in • Fixed form factor boxes largely win over modular chassis solutions • Cloud-native apps rule!
- 7. 7Cumulus Networks Except... • Many enterprise DC still have plenty of legacy applications, designed with old world network assumptions ▪ See Ivan Pepelnjak’s blog post for these assumptions: http://blog.ipspace.net/2017/10/solving-problem-in-right-place. html ▪ Solutions such as VM Mobility are still steeped in the assumptions of an L2 segment, even though IP address can be maintained without requiring L2
- 8. 8Cumulus Networks VxLAN To The Rescue • VxLAN has become quite popular as the model for running L2 over a pure L3 network ▪ Primarily introduced as a multi-tenant, private cloud story • Original script was for a controller-based play • But controller-based play has had a limited run
- 9. 9 EVPN in the DC: BGP VxLAN-based Ethernet VPN
- 10. 10Cumulus Networks Meet The New EVPN • A new set of IETF drafts defining the adaptation of EVPN in the data center • Base draft is: draft-ietf-bess-evpn-overlay-08 ▪ A Network Virtualization Overlay Solution Using EVPN ▪ VNI (virtual network identifier) replaces VPN in terminology • Replaces MPLS-based fabrics with IP-based fabrics: ▪ VxLAN, NVGRE, and MPLS over GRE • Controller-less VxLAN
- 11. 11Cumulus Networks EVPN in the DC: Summary • Supports extending L2 segments over an IP fabric • Supports routing between L2 segments • L3 multicast in the overlay is a work in progress • BGP is the control plane • Multi-vendor support • Mainstream introduction of VxLAN routing in merchant silicon
- 12. 12 Use Cases & Requirements
- 13. 13Cumulus Networks Three Primary Use Cases • Replace VLAN-based access-agg-core enterprise architecture with EVPN-CLOS based architecture • Multi-tenant hosting • Data Center Interconnect (DCI)
- 14. 14Cumulus Networks Replacing L2 Core With L3 Core in Traditional Enterprises • Don’t require > 4K VLANs ▪ Typically tens to hundreds, maybe a couple of thousand • No other orchestrator usually available ▪ Orchestrating across compute and network • Routing between L2 VNIs mandatory • L3 multicast between L2 VNIs maybe required
- 15. 15Cumulus Networks Multi-Tenant DC a.k.a Private Cloud • Require > 4K VNIs in the fabric • Routing across VNIs in well-defined points in the network only ▪ Routing will be VRF-aware • Orchestrator maybe present to simplify deployment ▪ Example: Openstack • L3 multicast across tenants not common
- 16. 16Cumulus Networks Datacenter Interconnect (DCI) • Stretch L2 segment across DC • Support for isolating control plane chatter across DCs • Support for some form of aggregation/summary of MACs to scale out • Optimize replication to avoid replicating from local VTEP to every remote VTEP • Support multi-homing and redundancy of border routers • Translating VNIs
- 17. 17Cumulus Networks Why Focus on Use Cases ? • Modern DC network are built on the KISS principle ▪ Keep it simple stupid • Immutable infrastructure is the growing mantra ▪ Network doesn’t change dynamically in tune with app • EVPN has the potential to re-introduce all the complexity of old networks back into the modern DC network • Focusing on use cases and deployment models can put a check on complexity ▪ More as we go through the webinar
- 19. 19Cumulus Networks What’s iBGP Got To Do With It ? • eBGP is the deployment model in the modern DC • EVPN is typically deployed as an iBGP model with peering between VTEPs ▪ Holdover from SP world ▪ Assumes a different IGP protocol to setup fabric connectivity ▪ Spines become iBGP route reflectors(RR) to avoid iBGP full mesh SPINE LEAF
- 20. 20Cumulus Networks Simplify BGP Deployment Model • Make EVPN BGP peering work over eBGP • Leaves peer with spines as usual • Spines transport EVPN AFI/SAFI without pushing state into the data plane (similar to iBGP RR) • Modification: For EVPN AFI/SAFI, don’t automatically do next-hop-self SPINE LEAF
- 21. 21 EVPN Bridging
- 22. 22Cumulus Networks BGP and EVPN Basics • EVPN uses l2vpn AFI and evpn SAFI • Multiple different pieces of information to exchange: ▪ MAC and MAC/IP along with associated VNI and remote VTEP (VxLAN Tunnel Endpoint) binding ▪ List of VNIs each VTEP is interested in ▪ Route prefixes (subnet routes) ▪ Multicast routes ▪ etc. • Encoding these different types of information is done by defining route types ▪ There are ~12 route types defined today
- 23. 23Cumulus Networks Basic Bridging in EVPN • Forward packets based on MAC address lookup ▪ Learn where destination MAC is ▪ Learn the source-MAC to port binding • Handle BUM (broadcast, unknown unicast, multicast) ▪ Send BUM traffic only where desired • Optimize L2 multicast ▪ Send multicast packets where there are interested listeners Exchanged via BGP (Type 2 Routes) Traditional Learning Exchanged via BGP (Type 3 Routes) IGMP/MLD Proxy to BGP Type 6 Route Ingress Replication or L3 Multicast
- 24. 24Cumulus Networks Type 3 Routes Illustrated A X B C Y Z 10.1.1.410.1.1.310.1.1.210.1.1.1 10.1.1.5 10.1.1.6 L1 L2 L3 L4 S2S1 W When EVPN family is activated, L1 sends Type 3 route advt to its BGP peers indicating its interested in Brown and Blue VNIs S1 and S2 send this information to L2, L3 and L4 L2, L3 and L4 learn of L1’s VNI list • Similarly L2, L3 and L4 send their own Type 3 routes • At the end, each VTEP has a list of other VTEPs and the list of VNIs they’re interested in
- 25. 25Cumulus Networks Illustrating Unknown Unicast Data Plane A X B C Y Z 10.1.1.410.1.1.310.1.1.210.1.1.1 10.1.1.5 10.1.1.6 L1 L2 L3 L4 S2S1 X sends packet to Z L1 associates X’s MAC/VNI with ingress port. Since Z is unknown, does ingress replication to L3, L4 L3, L4 decapsulate packet and flood it out all known brown VNI ports since they don’t know Z’s location as well • Ingress replication is done only to L3/L4 which have brown VNI • Different switching chips support doing ECMP post ingress replication; static, predefined spreading of traffic is done where chip doesn’t support • No egress VTEP learns off of VxLAN packets (implicitly disabled with EVPN) W
- 26. 26Cumulus Networks Illustrating The Control Plane A X B C Y Z 10.1.1.410.1.1.310.1.1.210.1.1.1 10.1.1.5 10.1.1.6 L1 L2 L3 L4 S2S1 X sends packet to Z L1 learns X’s ingress port, sends Type 2 route with Mac of X, VNI, VTEP of X, to its BGP peers, S1 and S2 Spines sends the received Type 2 route to its peers, L2-L4. Nothing is installed on the spine itself L3 & L4 install a MAC table entry with Mac of X pointing to VTEP of L1. L2 merely stores this info in the BGP VNI RIB since it has no brown VNI W
- 27. 27 Handling BUMs
- 28. 28Cumulus Networks Three Choices For Handling BUMs Head end or ingress replication L3 multicast i.e. underlay uses multicast Drop unknown unicast and unknown multicast silently
- 29. 29Cumulus Networks Ingress Replication • Keeps the underlay simple ▪ No need to setup/debug L3 multicast • The default model on Cumulus Linux • The most popular when I speak to customers (potential or otherwise) ▪ Maybe biased info, since Cumulus only supports this today
- 30. 30Cumulus Networks L3 Multicast in Underlay • Map each VNI’s traffic to a L3 multicast group • Ideal is that each VNI is mapped to a separate L3 multicast group • Control and data plane efficiency limit ideal goals • More complex configuration due to additional configuration: ▪ Configuring PIM ▪ Mapping VNI to L3 multicast group ▪ Additional checking if VNI received in group is of interest • Only benefit is ability to handle lots of BUM traffic (or even L2 multicast)
- 31. 31Cumulus Networks Drop BUM Traffic • Many network admins consider BUM traffic as a potential DDOS attack vector • A key primary goal of EVPN was to eliminate BUM via control plane support • Useful mostly if used in conjunction with ARP suppression • Primary drawbacks: ▪ Inability to handle silent servers (speak only when spoken to). Do these even exist anymore ? ▪ Slower convergence due to control plane distributing information rather than learning via data plane
- 33. 33Cumulus Networks Dual-Attached Hosts Deployment Model • The two switches a dual-attached host connects to behave no differently w.r.t. BGP for EVPN than regular BGP ▪ Each of the two switches has its own ASN • MLAG typically used to provide a single logical bonded interface to the host • Peer-link/MLAG is sometimes debated ▪ Alternate proposal is to use the L3 core and BGP to exchange relevant information between the switches ▪ Type 1 and type 4 route types defined for this purpose ▪ Not commonly deployed or popular ▪ Maybe of interest for data center interconnect switches
- 34. 34Cumulus Networks VxLAN Configuration for Dual-Attached Hosts • Many switching ASICs do not support multiple VTEP IP addresses associated with a MAC/VNI in the MAC table • So both switches attached that a dual-attached host connects to MUST use an anycast IP address as the VTEP IP address ▪ Ensure that this anycast VTEP IP is advertised in BGP underlay
- 35. 35 ARP Suppression in EVPN
- 36. 36Cumulus Networks ARP Suppression • Eliminate or reduce ARP broadcasts by providing local ARP proxy ▪ Not a traditional L3 ARP Proxy, just a L2 ARP local response • Announce MAC/IP binding along with MAC/VNI to VTEP association ▪ This is also a Type-2 route • Can be enabled on a per-VNI basis
- 37. 37Cumulus Networks ARP Suppression: Vendor Notes • ARP Suppression can be enabled on Cumulus, independent of the VTEP being the gateway for that VNI • Some of the other vendors enable this feature only if VTEP is also the gateway for that VNI • Cumulus supports only ARP suppression today, ND support coming soon
- 38. 38 Modifications to Linux Kernel for EVPN Support
- 39. 39Cumulus Networks Three Primary Modifications to Support EVPN • The Linux kernel had three primary modifications: ▪ Support for ARP suppression ▪ Adding a flag to indicate a MAC table entry was learnt via an external source ▪ Adding a flag to indicate an IP/IPv6 neighbor entry was learnt via an external source • The first has been upstreamed and accepted into mainstream Linux kernel • The two flags are being upstreamed
- 41. 41Cumulus Networks Configuration Steps (Cumulus Linux/FRR specific) • Configure VxLAN VNI ▪ Map the VLAN the VNI maps to • Configure BGP ▪ eBGP ▪ advertise IPv4 unicast underlay, announce loopback and VTEP IP address at a minimum ▪ Activate l2vpn/evpn AFI/SAFI and advertise all VNI
- 42. 42Cumulus Networks Configure VxLAN VNI (for VNI 33) net add interface lo ip address 27.0.0.21/32 net add vxlan vx-33 vxlan id 33 net add vxlan vx-33 vxlan local-tunnelip 27.0.0.21 net add interface vx-33 bridge access 1000
- 43. 43Cumulus Networks Configure BGP for EVPN (for leaf and spine) LEAF CONFIG router bgp 65456 bgp router-id 27.0.0.21 neighbor fabric peer-group neighbor fabric remote-as external neighbor uplink-1 interface peer-group fabric neighbor uplink-2 interface peer-group fabric address-family ipv4 unicast neighbor fabric activate redistribute connected address-family l2vpn evpn neighbor fabric activate advertise-all-vni SPINE CONFIG router bgp 65535 bgp router-id 27.0.0.21 neighbor fabric peer-group neighbor fabric remote-as external neighbor swp1 interface peer-group fabric neighbor swp2 interface peer-group fabric address-family ipv4 unicast neighbor fabric activate redistribute connected address-family l2vpn evpn neighbor fabric activate
- 44. 44 Wait! What ? Thats the entire BGP Config ?
- 45. 45Cumulus Networks Cisco BGP Config (In Comparison, just a leaf) router bgp 200 router-id 40.1.1.1 neighbor 10.1.1.1 remote-as 100 update-source loopback0 ebgp-multihop 3 allowas-in send-community extended address-family l2vpn evpn allowas-in send-community extended neighbor 20.1.1.1 remote-as 100 update-source loopback0 ebgp-multihop 3 allowas-in send-community extended address-family l2vpn evpn allowas-in send-community extended vrf vxlan-900001 advertise l2vpn evpn evpn vni 2001001 l2 rd auto route-target import auto route-target export auto vni 2001002 l2 rd auto route-target import auto route-target export auto
- 46. 46Cumulus Networks Cisco BGP Config (contd.) vrf context vxlan-900001 vni 900001 rd auto address-family ipv4 unicast route-target import 65535:101 evpn route-target export 65535:101 evpn route-target import 65535:101 route-target export 65535:101 address-family ipv6 unicast route-target import 65535:101 evpn route-target export 65535:101 evpn route-target import 65535:101 evpn route-target export 65535:101 evpn
- 47. 47Cumulus Networks FRR’s Simplified Configuration • Assume sane defaults • Simplify the common case • Take out all the stuff that’s inconsequential • Those who want all the knobs and warts still have it GOAL: Simplify configuration to reduce human error
- 48. 48Cumulus Networks Summary • EVPN is a standards-based technology that allows enterprise networks to run traditional applications over a L3 core • EVPN uses VxLAN as its base data plane encapsulation • EVPN uses BGP as the control plane • FRR/Cumulus Linux use sane defaults to simplify the EVPN configuration and operations
- 49. 49Cumulus Networks Next Webinar Operationalizing EVPN in the DC: Part 2 Routing with EVPN & Putting It All Together Nov 2, 10 AM PDT
- 50. 50 Thank you! Visit us at cumulusnetworks.com or follow us @cumulusnetworks or slack.cumulusnetworks.com © 2017 Cumulus Networks. Cumulus Networks, the Cumulus Networks Logo, and Cumulus Linux are trademarks or registered trademarks of Cumulus Networks, Inc. or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. The registered trademark Linux® is used pursuant to a sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide basis.