PLNOG15: Is there something less complicated than connecting two LAN networks? VLAN and VXLAN integration - Emil Gągała
2 likes•256 views
The document discusses connecting virtual and physical infrastructure through virtual networking technologies like VXLAN. It describes how VXLAN uses encapsulation to transport layer 2 frames over a layer 3 network and provides logical network isolation through a VXLAN Network Identifier. It also summarizes options for bridging between virtual and physical networks through x86-based or hardware VTEPs and how they can be used to integrate non-virtualized workloads. OVSDB integration is also described as a way for hardware VTEPs to connect to logical networks without requiring multicast in the underlay.
PLNOG15: Is there something less complicated than connecting two LAN networks? VLAN and VXLAN integration - Emil Gągała
- 1. © 2014 VMware Inc. All rights reserved. Integrating Physical Infrastructure and Virtual Workloads How to connect VLAN & VXLAN ? Emil Gągała Network & Security Architect PLNOG 28.09.2015, Kraków
- 2. Site 3 Site 1 Site2 Site 1 Site2 A1 B1 A3 A2 B2 Ethernet over … • ATM LAN Emulation • MPLS VPLS, EVPN • IP VXLAN, EVPN, OVSDB
- 3. Ethernet characteristics • Multipoint to Multipoint connectivity • No control plane – only data plane MAC learning • BUM traffic - some form of multicast support required • Segmentation - VLAN • No built-in loop prevention mechanism • No native multi-homing capabilities • Scalability – scope of broadcast domain and number of MACs • Lack of Virtual Machines awareness
- 4. Physical DC Fabric Trends • From 2- or 3-tier to IP spine/leaf fabrics • Density & bandwidth jump • ECMP for layer 3 (and layer 2) • Reduce network oversubscription • Wire & configure once • Uniform configurations 4 WAN/Internet L3 L2 L3 L2 POD A POD B WAN/Internet L3 L2 L2 Ethernet L3 IP
- 5. VM5 Virtual Overlay Networking VM1 Overlay Network VM2 Logical Switch 5001 VM3 Physical Underlay IP Network VM4 Logical Switch 5002 Controller Management Cluster Control Plane Programming Data Plane Tunneling Subnet Red 172.16.10.0/24 Subnet Green 172.16.20.0/24 192.168.150.51 192.168.150.52 192.168.250.51 5 Transport Subnet A 192.168.150.0/24 Transport Subnet B 192.168.250.0/24
- 6. Traditional VLAN network VM1 VM2 Server 1 VM3 VM4 VM5 Server 2 VM6 VM7 VM8 Server 3 VM9 Physical switch Virtual networks: 1 2 3 VLANs
- 7. OVERLAY Networking VM1 VM2 Server 1 VM3 VM4 VM5 Server 2 VM6 VM7 VM8 Server 3 VM9 No VM network state Virtual networks: S3 VM9 Payload Transport network:
- 8. Virtual Extensible LAN (VXLAN) Overlay • VXLAN is an industry standard IP overlay technology - RFC 7348 : Used to tunnel Layer 2 traffic over an IP infrastructure 8 L2 frame • Why an IP encapsulation? – leverage VXLAN in order to decouple its data plane from the physical network: basic IP connectivity is enough to run SDDC • Why an additional VXLAN header? – VXLAN Network Identifier (VNI) VTEP: Virtual Tunnel End Point src IP: VTEP1, dst IP:VTEP2 UDP/VXLAN L2 frame VTEP2VTEP1 L2 frame
- 9. What is VXLAN ? (Overview) Ethernet in IP overlay network – Entire L2 frame encapsulated in UDP – 50+ bytes of overhead 24 bit VXLAN Network Identifier – 16 M logical networks VXLAN can cross Layer 3 network boundaries Technology approved by IETF as standard – RFC 7348 „Virtual eXtensible Local Area Network (VXLAN): A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks” – With Arista, Brocade, Cisco, Cumulus, Dell, HP, Juniper, Vmware Overlay between hosts and Gateways – VMs do NOT see VXLAN ID VTEP (VXLAN Tunnel End Point) – Interface which serves as the endpoint for encapsulation/de-encapsulation of VXLAN traffic VTEP acts like a learning bridge – Missing information relies on ARP generated from the host or Flood – Floods ports when encountering an unknown MAC – Flooding may happen when communicating with physical workloads – Flooding limited to a VXLAN segment Once destination MAC is known, communication is direct
- 10. VXLAN OVERLAY Networking VM1 VM2 Server 1 VM3 VM4 VM5 Server 2 VM6 VM7 VM8 Server 3 VM9 No VM network state IP Fabric Virtual networks: S3 VM9 Payload Transport network:
- 11. Software Defined Data Center VM1 VM2 Server 1 VM3 VM4 VM5 Server 2 VM6 VM7 VM8 Server 3 VM9 Virtual networks: S3 VM9 Payload Transport network: Controller
- 12. Software Defined Data Center VM1 VM2 Server 1 VM3 VM4 VM5 Server 2 VM6 VM7 VM8 Server 3 VM9 Virtual networks: Transport network: Controller BMS Server 4
- 13. Web-Tier App-Tier DB-Tier VMs Connect to Virtual Networks Virtual Networks Connect to non-virtualized Workloads Physical-Virtual Bridging and Routing Requirement: Communication between VMs connected to Logical Networks (VXLAN) and workloads (virtualized or not) deployed on traditional VLANs Different options depending on the connectivity needs: L2 (Bridging) SW L2 Bridges HW VTEP L2 Gateway L3 (Routing) L2/L3 services useful for supporting migration scenarios
- 14. Overlay to VLAN Bridging
- 15. Overlay to VLAN Gateway Functionality • The Overlay to VLAN gateway allows communication between virtual and physical world Physical Network VLAN backed network VM NSX: Virtual Network, VXLAN tunnels VLANVXLAN L2 payload VXLAN VLAN gateway Physical Workload
- 16. • L2 as well as L3 • Virtual to virtual, physical to virtual • Temporary, bandwidth and redundancy not critical Use Cases: Migration 16 VM VM Physical Workload Virtualized Workload (VLAN backed) Physical to Virtual Virtual to Virtual VXLAN VLAN
- 17. • Typically necessary for integrating a non-virtualized appliance • A gateway takes care of the on ramp/off ramp Use Cases: Integration of non-Virtualized Workloads 17 VM Physical Services / Workload VXLAN VLAN
- 18. Physical Workload Integration Physical Workloads VXLAN VLAN x86-based bridge Highest density but requires specific hardware Leverages any x86 server Physical Workloads VXLAN VLAN HW VTEP Use-case: Integrate non-virtualized workloads seamlessly with virtual networks
- 19. x86-based Overlay to VLAN solution
- 20. P-V Bridging Scale-out Multiple Bridge instances (VXLAN/VLAN) Pair Single bridging instance (VXLAN/VLAN pair) per Logical Switch Bandwidth limited by single bridging instance Bridged VLAN extends to reach physical devices in multiple racks VXLAN VLAN VLAN extended (!) SW VTEP Physical Servers (VLAN 10) VXLAN 5001 Scale-out model with multiple bridging instances active for separate VXLAN/VLAN pairs May allow to reduce the spanning of VLANs to a single rack if physical servers in a VLAN are contained in that rack L3-only (VXLAN) network VXLAN 5000 Physical Servers (VLAN 10) Physical Servers (VLAN 20) Bridging Instance 1 (VXLAN 5000 to VLAN 10) Bridging Instance 2 (VXLAN 5001 to VLAN 20) 20
- 21. Benefits of x86-based On/off-ramp Scale-up • x86 performance curve • x86 optimized processing (DPDK) • Encapsulation offloads • Encryption offloads Scale-out • Expand on scale-out • Active-active services Flexibility & Operations • Rich set of stateful services • Multi-tier logical routing • Advanced monitoring • Choice of form-factor • Scale as you grow 21VMware Confidential VLAN 10 VLAN 20 VLAN 30
- 22. HW VTEPs
- 23. HW VTEPs Motivation Integrate non-virtualized workloads seamlessly with virtual networks Servers with legacy or hard-to-virtualize applications Physical servers relying on specific hardware not supported by HVs Physical network & security appliances such as routers, load balancers, firewalls, IPS, WAN acceleration, etc. The following are potential use cases for HW L2 VTEPs: Low latency traffic Very large volumes of physical servers (>10G of bandwidth required for P-to-V communication) Support of physical hosts connected in different racks remove the need to extend VLAN connectivity across the racks 23
- 24. VM VXLAN Distributed Bridging Options with Hardware VTEPs One bridge instance • Bandwidth limited by single instance • VLAN connectivity extended to reach the physical devices. Hardware VTEPs deployed where physical workloads or services exist: • Bandwidth and physical ports scale-out • VLANs for Physical workloads only local to a rack 24 VXLAN VLAN VM Single Instance x86 L2 Gateway Multiple Instances 3rd party HW Gateway Non-virtualized devices (part of the same L2 segment)
- 25. L2 Extension with Hardware VTEPs Two options: 1. Hardware VTEP with IP multicast in the underlay 2. OVSDB Integration 25 VM VM L2 Network Non-virtualized devices LS A LS B VLAN A VLAN B VXLAN VLAN Hardware VTEP ToR Switch
- 26. VXLAN in Multicast Mode
- 27. VXLAN in Multicast Mode IGMP to Join VXLANs Assigned Multicast Groups Web VM Web VM DB VM DB VM IGMP Report to Multicast Group 239.1.1.1 VTEP VTEP VTEP L3 Core with multicast IGMP Report to Multicast Group 239.1.1.1 IGMP Report to Multicast Group 239.2.2.2 IGMP Report to Multicast Group 239.2.2.2 Mapping is required between a VXLAN ID and a Multicast Group VXLAN 5000 VXLAN 6000
- 28. VTEP 1 1.1.1.1 VTEP 2 2.2.2.2 VTEP 3 3.3.3.3 VXLAN Data Flow Example VM1 Communicating with VM2 in a VXLAN VM 1 VM 2 VM 3 MAC1 MAC2 Multicast Multicast Multicast ARP Request ARP Request ARP Request VM Source MAC Remote VTEP IP VM1:MAC1 1.1.1.1 MAC Table: VTEP 2 and VTEP 3 (Data Plane Learning)L3 Core with multicast VXLAN 5000
- 29. VTEP 1 1.1.1.1 VTEP 2 2.2.2.2 VTEP 3 3.3.3.3 VXLAN Data Flow Example (2) VM1 Communicating with VM2 in a VXLAN VM 1 VM 2 VM 3 MAC1 MAC2 VM Source MAC Remote VTEP IP VM1:MAC1 1.1.1.1 MAC Table: VTEP 2L3 Core with multicast ARP Response Unicast VXLAN 5000
- 30. VTEP 1 1.1.1.1 VTEP 2 2.2.2.2 VTEP 3 3.3.3.3 VXLAN Data Flow Example (3) VM1 Communicating with VM2 in a VXLAN VM 1 VM 2 VM 3 MAC1 MAC2 VM Source MAC Remote Host VXLAN IP VM1:MAC1 1.1.1.1 MAC Table: VTEP 2L3 Core with multicast ARP Response VM Source MAC Remote Host VXLAN IP VM2:MAC2 2.2.2.2 MAC Table: VTEP 1 (Data Plane Learning) VXLAN 5000
- 31. VTEP 1 1.1.1.1 VTEP 2 2.2.2.2 VTEP 3 3.3.3.3 VXLAN Data Flow Example (4) VM1 Communicating with VM2 in a VXLAN VM 1 VM 2 VM 3 MAC1 MAC2 VM Source MAC Remote Host VXLAN IP VM1:MAC1 1.1.1.1 MAC Table: VTEP 2L3 Core with multicast VM Source MAC Remote Host VXLAN IP VM2:MAC2 2.2.2.2 MAC Table: VTEP 1 Unicast VXLAN 5000
- 33. • Hardware VTEP enabled physical appliance • Attach any physical services appliance • Extensible (schema-based) • Integration not dependent on Multicast Overview • High density of physical ports to connect physical workloads • Broad ecosystem of partners • Compatible with HA M-LAG solution Benefits OVSDB Integration: Hardware VTEPs 33 Provide connectivity to physical workloads and services VM1 VM2 LS – VNI 5001 VLAN 100
- 34. What is OVSDB ?
- 35. What is OVSDB ? Open vSwitch Data Base (OVSDB) is a management protocol Helps attaching interfaces, gathering statistics, configuring features NOT related with and does NOT require Openflow RFC 7047 for „The Open vSwitch Database Management Protocol” 3rd Party GW OVSDB Server Operational State Forwarding State IP Fabric Service Nodes VM VMVM VM VM VM Controller Cluster CMP
- 36. • The controller exposes a northbound API - physical ports can be attached to logical switches. • Virtual ports of VMs are attached to build logical networks that span the physical and virtual worlds • The information exchanged by the control plane allows setting up the data plane, i.e. VXLAN tunnels between VTEPs • Switch terminates VXLAN tunnels OVSDB Integration: Hardware VTEPs VNI VTEP VM MAC/IP port
- 37. OVS hardware_vtep database schema Table Purpose Global Top-level configuration. Manager OVSDB management connection. Physical_Switch A physical switch. Physical_Port A port within a physical switch. Logical_Binding_Stats Statistics for a VLAN on a physical port bound to a logical network. Logical_Switch A layer−2 domain. Ucast_Macs_Local Unicast MACs (local) Ucast_Macs_Remote Unicast MACs (remote) Mcast_Macs_Local Multicast MACs (local) Mcast_Macs_Remote Multicast MACs (remote) Logical_Router A logical L3 router. Physical_Locator_Set Physical_Locator_Set configuration. Physical_Locator Physical_Locator configuration. CONFIDENTIAL 37
- 38. Summary
- 39. Integrating Physical Infrastructure and Virtual Workloads Conclusion x86-based Scale-up & scale-out model Pay as you grow Rich set of stateful services Growing set of routing features Software development cycle Versatile topology options HW VTEP Highest density & performance Broad partner choice • Limited stateful services (cost) Broad set of routing features • Longer innovation cycle VMware Confidential 39
- 40. Thank You