Enabling new protocol processing with DPDK using Dynamic Device Personalization

Brian Johnson – Solution Architect, Intel Corp
v1803.20

Ethernet Networking Division 2
Legal Disclaimer
INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL® PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS
IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL’S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY
EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL® PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE,
MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT. INTEL PRODUCTS ARE NOT INTENDED FOR USE IN MEDICAL, LIFE SAVING, OR LIFE
SUSTAINING APPLICATIONS.
Intel may make changes to specifications and product descriptions at any time, without notice.
All products, dates, and figures specified are preliminary based on current expectations, and are subject to change without notice.
Intel processors, chipsets, and desktop boards may contain design defects or errors known as errata, which may cause the product to deviate from published specifications. Current characterized errata are
available on request.
Alta, Red Rock Canyon, Seacliff Trail and other code names featured are used internally within Intel to identify products that are in development and not yet publicly announced for release. Customers,
licensees and other third parties are not authorized by Intel to use code names in advertising, promotion or marketing of any product or services and any such use of Intel's internal code names is at the
sole risk of the user
Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark* and MobileMark*, are measured using
specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests
to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products.
Intel processor numbers are not a measure of performance. Processor numbers differentiate features within each processor family, not across different processor families. Go to:
http://www.intel.com/products/processor_number
Intel product plans in this presentation do not constitute Intel plan of record product roadmaps. Please contact your Intel representative to obtain Intel's current plan of record product roadmaps.
Intel, Xeon the Intel logo are trademarks of Intel Corporation in the United States and other countries.
*Other names and brands may be claimed as the property of others.
Copyright ©2018 Intel Corporation.
Intel does not control or audit the design or implementation of third party benchmark data or Web sites referenced in this document. Intel encourages all of its customers to visit the referenced Web sites
or others where similar performance benchmark data are reported and confirm whether the referenced benchmark data are accurate and reflect performance of systems available for purchase.

Agenda
Problem statement
Intel® Ethernet 700 Series capabilities level set
What is a Dynamic Device Personalization Profile?
Personalization programming tool chain
Examples and Demo
Using the Data Plane Development Kit and Dynamic Device Personalization
Click here to access stream

Network Landscape
Microwave
Mobile
Backhaul
BRAS
Wireless Core GiLAN
IMS
DSLAM
CDN
IPFE
GTP
QinQ,
GRE,
MPLS Martini
VXLAN
Load distribution
(stateless hashing)
on any packet’s fields
QinQ
SCTP
NSH
NSH
NSHoUDP
IPSec – ESP stateless hashing
ARP
Load balancing
(stateless hashing)
on GTP TEID,
IPSec ESP SPI
VXLAN-gpe
Geneve
CAPWAP
MPLS
IPv4, IPv6, TCP, UDP, SCTP, VXLAN, NVGRE, GRE
VXLAN + PPPoE
VXLAN + GTP
Available Limited functionality
Software
Classification/
Distribution
Bottleneck
Classification/
Distribution in
HW Removes
Bottleneck
Load Distribution Software
Runs on CPU Core
Intel® Ethernet
700 Series
Worker
1
Worker
2
Worker
3
Worker
4
From 40GbE
Traffic Generator
MPLSoGREStandard TCP
Worker
0
Load Distribution Core
Load Distribution
Runs on XL710 Personalization Profile
Worker
1
Worker
2
Worker
3
Worker
4
Load Distribution using
MPLSoGRE DDP
Worker
0
Intel® Ethernet
700 Series
From 40GbE
Traffic Generator

Intel® Ethernet 700 Series Capabilities Level Set
Packets classification: supports up to 64 packet classifier types (PCTYPEs)
o “TCP in IPv4” and “UDP in IPv6” are different packet classifier types
filters (hash, flow director, tunnel filter) configured per PCTYPE
Reported packet types: up to 192 packet types as 8bit value on RX descriptor
o “MAC, IPv4, TCP, PAY4” and “MAC, IPv6, TCP, PAY4” packets will have different PTYPE
value on RX descriptor
Packets steering/mirroring: tunnel (cloud) filters for VXLAN, GRE, GENEVE
o ADq (Application Device Queues) to filter TCP/UDP traffic
Hash offload calculation (up to 48 bytes input set)
o Different queue regions can be assigned to different PCTYPEs
Exact flow matching (up to 8k rules) – Intel® Ethernet Flow Director

Intel® Ethernet 700 Series Rx Programmable Pipeline
Parser
Packet Context
Classifier
Switch
Classifier
Rx Post Filters
Packet Type
Resolver
3-StageFlowTables
FlowTables
...
Packet Type
Packet Type
Packet
Modifier
(RDPU)
Analyzer
(Packet Digest & Metadata)
uCode
FlexKeyperPacketType
Q
Stats
VSI
Actions
FV (Field Vecor)
QVSI
Q
Rx Desc
RDPU Rcp
Actions
Rx DescRDPU Rcp
Recipies
Packet
Buffer
Ports
Network
Interfaces
Rx
Descriptor
Packet
Data
PCIeEthernet
Drop
Drop
Metering/
Stats
Stats
Metadata
Filter/Selector
FlexN-TupleKeyBuild
FLUs
...
FLUs
...
FLUs
...
...
...
Stats Stats Stats
...
TC / QoS
FD
RSS
QH
...
FCoE
TC / QoS
Field
Extractor
Stats
Programming
by firmware
or by DDP1 2

What is Dynamic Device Personalization (DDP)?1
Loadable Packet Classifications (Parser) personalization
for any well-defined traffic profiles for VNFs or
appliances with a specific position in the network
 Enable new protocols:
– GTP tunnel support, GTP-C and GTP-U as a separate flow types
– IP protocols as new flow types, for example L2TPv3, ESP/AH for IPSec
– Legacy protocols: PPPoE, PPPoL2TPv2
 Modify existing protocols:
– VXLAN with IP payload, NSHoUDP
– New PTYPEs on RX descriptor: for example, IPv4 TCP in one/two
MPLS label(s)
 Make Smart NIC act Stupid:
– Disable GRE classification
– Disable IPv4 classification
 Debug:
– Read configuration/statistics registers in one admin command call
Configurable tunnel filters for traffic steering
(Classifier Switch)
 Steering packets to a VM on QinQ
 Steering packets to a VM on Tunnel ID
(VXLAN, NVGRE, MPLS, GTP, NSH*)
 User definable steering/mirroring rules
1Does not include Intel® Ethernet Connection X722

Ethernet Networking Division
DPDK API or
Linux driver*
Dynamic Device Personalization Programming Tool Chain
Profile Editor
No GTP
Support
1. Take current Parser NVM configuration
2. Use Profile Editor to modify existing entries/create new entries for
Analyzer/Parser to address customer’s request
3. Create binary package file to be applied to FVL by customer
4. Download profile to FVL to change Parser configuration: new
PCTYPEs, PTYPEs, Protocols, extracted fields.
Profile is not persistent, on reset the original configuration is restores from
the NVM. SW can restore original configuration without reset as profiles
support “roll-back” transaction.
1 2 4
5
Loader
Loader:
DPDK 17.05 to 17.11
Linux: in planning
GTP
Support
gtp.i40e file
pppoe.pkgo
gtp.pkgo
mmio { // Expose GTP protocol
groupi REG w 0x300 x05 0x0 0xb
// add UDP 2153
write REG w 0x100 x32 0x0 0xcf58b
…
compiler
Intel Customermpls.pkgo 3
8

Dynamic Device Personalization
Without vs. With, GTP-U example
Default FVL6 processing (without Personalization profile):
DA SA IPv4 UDP GTP TEID IPv4 TCP PAY
Parsed fields Payload
FVL6 with Personalization profile applied:
RSS
RSS
GTP-U is unknown flow type, so no RSS, FDIR or other filters are possible on encapsulated frame
GTP-U flow type is defined, encapsulated frame fields (including GTP TEID) can be used for RSS, FDIR.
Encapsulated frame type is indicated on RX descriptor, for example, TCP in GTP-U or GTP-U echo message
GTP-C flow type is defined as well and has separate RSS/FDIR configuration.
RSS
RSS
RSS
Packet type 24: UDP in IPv4
Packet type 172: TCP in GTP-U
RSS
RSS

GTP Profile Example
Before GTP Profile is Applied
 31 udp
 33 tcp
 34 sctp
 35 ip
 36 ipf
 41 udp6
 43 tcp6
 44 sctp6
 45 ip6
 46 ipf6
 63 l2
Before GTP Profile is Applied
 22 gtpu ipv4
 23 gtpu ipv6
 24 gtpu
 25 gtpc
 31 udp
 33 tcp
 34 sctp
 35 ip
 36 ipf
 41 udp6
 43 tcp6
 44 sctp6
 45 ip6
 46 ipf6
 63 l2

Dynamic Device Personalization Demo

Without Dynamic Device Personalization GTP Profile Loaded

With Dynamic Device Personalization GT Profile Loaded

With Dynamic Device Personalization RSS Queue Regions Assigned

With Dynamic Device Personalization More RSS Queue Per Regions Assigned

Dynamic Device Personalization Commands in DPDK testpmd
4.4. Display Functions
4.4.10. ddp get list
Get loaded dynamic device personalization (DDP) package info list:
testpmd> ddp get list (port_id)
4.4.11. ddp get info
Display information about dynamic device personalization (DDP) profile:
testpmd> ddp get info (profile_path)
4.5. Configuration Functions
4.5.88. ddp add
Load a dynamic device personalization (DDP) profile and store backup profile:
testpmd> ddp add (port_id) (profile_path[,backup_profile_path])
4.5.89. ddp del
Delete a dynamic device personalization profile and restore backup profile:
testpmd> ddp del (port_id) (backup_profile_path)
http://dpdk.org/doc/guides/testpmd_app_ug/testpmd_funcs.html#configuration-functions

rte_pmd_i40e_process_ddp_package()
download dynamic device personalization profile and register it**
int rte_pmd_i40e_process_ddp_package(
uint8_t port, // DPDK port index to download DDP package to
uint8_t *buff, // buffer with the package in the memory
uint32_t size, // size of the buffer
rte_pmd_i40e_package_op op // 17.05 supports only RTE_PMD_I40E_PKG_OP_WR_ADD
);
int rte_pmd_i40e_get_ddp_list()
get list of registered profile(s)**
int int rte_pmd_i40e_get_ddp_list (
uint8_t port, // DPDK port index to get list from
uint8_t *buff, // buffer to store list of registered profiles (rte_pmd_i40e_profile_list *)
uint32_t size // size of the buffer
);
* Requires XL710 firmware version 6.01
** HW could register up to 16 profiles, but SW should limit number of loaded profiles to one to avoid profiles overlapping
DPDK 17.05 APIs (rte_pmd_i40e.h)*
18

DPDK 17.08 APIs (rte_pmd_i40e.h)*
rte_pmd_i40e_process_ddp_package()
download dynamic device personalization profile and register it (RTE_PMD_I40E_PKG_OP_WR_ADD)
rollback dynamic device personalization profile and un-register it (RTE_PMD_I40E_PKG_OP_WR_DEL)
rte_pmd_i40e_get_ddp_info()
request information about a profile without downloading it to the NIC
int rte_pmd_i40e_get_ddp_info(
uint8_t *pkg_buff, // buffer with the package in the memory
uint32_t pkg_size, // size of the package buffer
uint8_t *info_buff, // buffer to store information to
uint32_t info_size, // size of the information buffer
enum rte_pmd_i40e_package_info type // information type: header, devlist, ...
);
Note: in case if profile adds new flow types/packet types, these types should be enabled in DPDK separately
NEW:
* Requires XL710 firmware version 6.01
19

Identify
Feature
Develop
Prototype
PoC with
Customers
FVL6 Delivery
Q2 2017
New Protocols Dynamically
Defined via DDP Profiles
Dynamic Device Personalization (DDP) – Protocol Profile Development Plan
20
Dynamic Device Personalization profiles for Intel® Ethernet 700 Series addresses
additional protocol support and allows early adoption of new technologies.
Intel® Ethernet 700 Series
2016
Pre-FVL6 Gap:
Protocols supported
Limited to firmware only
Intel® Ethernet 700 Series
FlexRAN /
MEC / EPC
GTP
RoE
BNG
PPPoE
CMTS
L2TPv3
Network
Programming
NHS
SRv6
Custom
VXLAN+IP payload
vRouter
MPLSoGRE
MPLSoUPD
New protocols via
Dynamic Device
Personalization
Application
Proposed
Protocols

Load Distribution Software
Runs on CPU Core
Intel® Ethernet
700 Series
Worker
1
Worker
2
Worker
3
Worker
4
From 40GbE
Traffic Generator
Worker
0
Load Distribution Core
Load Distribution
Runs on XL710 Personalization Profile
Worker
1
Worker
2
Worker
3
Worker
4
Load Distribution using
MPLSoGRE DDP
Worker
0
Intel® Ethernet
700 Series
From 40GbE
Traffic Generator
Profiles for Wireline
Profiles for Wireless Profiles for GiLAN
Profiles for Enterprise
21
Network Landscape with
Microwave
Mobile
Backhaul
BRAS
Wireless Core GiLAN
IMS
DSLAM
CDN
IPFE
GTP-U
QinQ,
GRE,
MPLS Martini
VXLAN
Load distribution
(stateless hashing)
on any packet’s fields
QinQ
SCTP
NSH
NSH
NSHoUDP
IPSec – ESP stateless hashing
ARP
Load balancing
(stateless hashing)
on GTP TEID,
IPSec ESP SPI
VXLAN-gpe
Geneve
CAPWAP
MPLS
Without DDP
Software
Bottleneck
Profiles can be loaded separately per-device, so the same server with
multiple NICs can be used for different workload in a very dynamic way
DDP Removes
Bottleneck
VXLAN + GTP

Intel® Ethernet Converged
Network Adapter XL710-QDA2
Port Speeds: 10Gb/40GbE
Intel® Ethernet Network
Adapter XXV710-DA1
Port Speeds: 1Gb/10Gb/25GbE
Network Adapter X710-T4
Network Adapter X710-DA4
PCIExpress*
EthernetAdapters
22
Intel®Ethernet700Seriesnetworkadapters
OpenComputeProject
EthernetAdapters
Intel® Ethernet Network Adapter
XXV710-DA1 for OCP
Intel® Ethernet Network
Adapter XXV710-DA2
Network Adapter X710-DA2
Intel® Ethernet Server Adapter
XL710-QDA2 for OCP
Network Adapter XL710-QDA1
XL710-QDA1 for OCP
X710-DA2 for OCP
10Gb Ethernet 10Gb or 25Gb Ethernet 4x10Gb or 40Gb Ethernet
Optimized Performance and Enhanced Usability
for Cloud and Network Functions Virtualization (NFV)

Dynamic Device Personalization Summary
Dynamic Device Personalization can be used on any OS supported by DPDK
 For example, QUIC protocol classification on both Windows and Linux Oses
Dynamic Device Personalization allows a DPDK application to enable identification of
new protocols, for example, GTP, PPPoE, QUIC, MPLSoGRE, SRv6, etc without requiring a
reboot due to changing hardware or firmware updates.
Allows for existing Parse Graphs to be modified to support new usage models as they
are defined enabling faster time to market for DPDK enable applications.
DPDK on Microsoft Windows DPDK on Linux
23

Dynamic Device Personalization Resources
Public Resources
 Intel® Developer Zone Article
– https://software.intel.com/en-us/articles/dynamic-device-
personalization-for-intel-ethernet-700-series
 DPDK Summit Nov 2017 presentation (YouTube)
– https://www.youtube.com/watch?v=X8aMDdAnnBI
 DPDK Summit Nov 2017 Slides
– https://www.slideshare.net/LF_DPDK/lfdpdk17flexible-and-
extensible-support-for-new-protocol-processing-with-dpdk-
using-dynamic-device-personalization
24

Without vs. With, PPPoE example
Default FVL6 processing (without personalization profile):
DA SA PPPoE IPv4 UDP PAY
FVL6 with personalization profile applied:
RSS
RSS
RSS
PPPoE is unknown flow type, so no RSS, FDIR or other filters are possible on inner packet fields.
FVL classifies PPPoE (Discovery and Session) packets as L2 payload.
With a DDP profile, PPPoE header can be parsed and encapsulated packet classified.
For example, for PPPoE session with UDP in IPv4, as shown above, RSS and FDIR filters can be used on UDP.
If required, RSS, FDIR and Tunnel filters can use PPPoE Session ID as well.
RSS
DA SA PPPoE IPv4 UDP PAYsession

Without vs. With, MPLSoGRE example
Default FVL6 processing (without personalization profile):
DA SA IPv4 GRE MPLS DA SA IPv4 UDP PAY
FVL6 with personalization profile applied:
DA SA IPv4 GRE MPLS DA SA IPv4 UDP PAY
RSS
RSS
RSS
MPLSoGRE is unknown flow type, so no RSS, FDIR or other filters are possible. Packet type set to 43.
With MPLSoGRE flow type is defined by encapsulated frame, for example, UDP in IPv4 as shown above
so all filters available for UDP can be applied to the packets: RSS, FDIR. Packet type 59-72.
RSS, FDIR and Tunnel filters can use MPLS label as tunnel (tenant) ID.
RSS

Q&A, generic
Q How profiles will be distributed?
A
There a two main ways of distributing profiles:
1. as a separate binary file and loaded by application
2. it can be precompiled to a driver, and selected on device initialization stage
For the trials/test with a customer we use the first option, as profile configuration can change
during pre-release.
After full validation, release version can be precompiled with a driver (kernel or DPDK) or an
application to be selected at initialization.
Q Can profile be applied to only one port/PF/VF of a multi-port device?
A
No, profiles affect all port on a device. In virtual environment not only PFs but all VFs will be
affected as well. Which PF is used to apply a profile does not matter.
Q Do we need to reload a profile after NIC reset or, once loaded, profile stays in the NVM?
A Profiles are not persistent, so must be reloaded after any reset of the NIC.
Q
Can we use profiles on any other i40e compatible devices, for example, on Intel® Ethernet
Connection X722 ports of Intel® Chipset C620?
A No, the Intel Ethernet Connection X722 does not support profiles.
28

Q&A, generic
Q Does firmware version 5 supports dynamic device personalization profiles?
A No, dynamic device personalization will be available starting from firmware version 6
Q How many profiles can be loaded simultaneously on one device?
A
Thou the Network Adapters can provide information about up to 16 loaded profiles. Multiple
profiles are not recommended. In case if, for some reason, multiple profiles are needed, it
should be treated very carefully to make sure that profiles are not overlapping and to avoid
any unexpected behaviour.
Q Will I (as a customer) be able to create profiles by myself?
A
No, profile programming requires very deep knowledge of the device internals and can be
done only by Intel engineers.
Q Can I apply profile while traffic is running through the device or I have to stop device first?
A You can apply/remove profile with traffic running, but some packet loss should be expected.
29

Enabling new protocol processing with DPDK using Dynamic Device Personalization

Default firmware processing (WITHOUT Personalization profile)
GTP-U is unknown flow type, so no RSS, FDIR or other filters are possible on encapsulated frame
RSS
RSS
Packet type 24: UDP in IPv4
RSS

GTP-U flow type is defined, encapsulated frame fields
(including GTP TEID) can be used for RSS, FDIR
Encapsulated frame type is indicated on RX descriptor,
for example, TCP in GTP-U or GTP-U echo message
GTP-C flow type is defined as well and has separate
RSS/FDIR configuration
Dynamic Device Personalization GTP Profile applied:
RSS
RSS
RSS
RSS

RSS configured separating traffic across 16 queues, assigning each of the
packet types to individual traffic classes and queue regions
GTP Demo RSS Configuration
// 31 ipv4-udp region 1, queues 1-2
// 41 ipv6-udp region 2, queues 3-4
// 34 SCTP region 3, queue 4
// 24 GTPU region 4, queue 6 (GTPU with message types != 0xFF)
// 25 GTPC region 5, queue 7
// 22 GTPU_IPV4 region 6, queues 8-11
// 23 GTPU_IPV6 region 7, queues 12-15
Dynamic Device Personalization GTP Profile applied with RSS:
RSS
RSS
RSS
RSS

Enabling new protocol processing with DPDK using Dynamic Device Personalization

More Related Content

What's hot (20)

Similar to Enabling new protocol processing with DPDK using Dynamic Device Personalization (20)

More from Michelle Holley (20)

Recently uploaded (20)

Enabling new protocol processing with DPDK using Dynamic Device Personalization