P4 Tutorial

Welcome
Noa Zilberman
University of Cambridge

Copyright © 2018 – P4.org

Goals
• Learn P4 Language
◦ Traditional applications
◦ Novel applications
• Learn P4 software tools
◦ P4 Compiler
◦ BMv2
◦ P4Runtime
• Learn about P4 hardware targets
◦ mini-workshop featuring solutions by Barefoot, Netronome, Netcope
and NetFPGA.
• Networking (the other kind)
• Have fun!
Copyright © 2018 – P4.org
Introduction to Data Plane Programming
Language Basics
Break
Software Tools & P4 Runtime
Lunch
Group Programming Exercise
Break
Architectures Mini-Workshop
Reception

Copyright © 2018 – P4.org

Instructors

Nate Foster Theo Jepsen Milad Sharif

Cornell USI Lugano Barefoot Networks

Robert Soulẻ Pietro Bressana

USI Lugano Copyright © 2018 – P4.org USI Lugano
Thank you

• Instructors & TAs

◦ Nate Foster
◦ Theo Jepsen
◦ Milad Sharif
◦ Robert Soulẻ
◦ Pietro Bressana
• Mini-Workshop speakers
◦ Pavel Benacek
◦ Jaco Joubert
• You!

Copyright © 2018 – P4.org

P4 Language Tutorial

Copyright © 2018 – P4.org

What is Data Plane Programming?
• Why program the Data Plane?

Copyright © 2018 – P4.org 2

Software Defined Networking: Logically Centralized Control
• Main contributions Control Function (“App”) xm
◦ OpenFlow = standardized model
■ match/action abstraction
◦ OpenFlow = standardized protocol to interact with Northbound I/F
switch
■ download flow table entries, query statistics, etc.
◦ Concept of logically centralized control via a SDN
single entity (“SDN controller”) x1
■ Simplifies control plane – e.g. compute optimal paths at Controller
one location (controller), vs. waiting for distributed routing
algorithms to converge
Southbound
• Issues
◦ Data-plane protocol evolution requires changes to
I/F
standards (12 → 40 OpenFlow match types)
◦ Limited interoperability between vendors => xn
southbound I/F differences handled at controller
(OpenFlow / netconf / JSON / XML variants)

Copyright © 2018 – P4.org

Switch 3
Status Quo: Bottom-up design

Switch OS
Network Demands

?
“This is how I know to
Run-time API
Driver

process packets”
(i.e. the ASIC datasheet
makes the rules)
Fixed-function ASIC
Copyright © 2018 – P4.org 4
A Better Approach: Top-down design

Switch OS
Network Demands

Feedback

Run-time API
Driver

P4

“This is how I want the

network to behave and how to
switch packets…”
(the user / controller
makes the rules)
P4 Programmable Device
Copyright © 2018 – P4.org 5
Benefits of Data Plane Programmability
• New Features – Add new protocols
• Reduce complexity – Remove unused protocols
• Efficient use of resources – flexible use of tables
• Greater visibility – New diagnostic techniques, telemetry, etc.
• SW style development – rapid design cycle, fast innovation, fix data
plane bugs in the field
• You keep your own ideas

Think programming rather than protocols…

Copyright © 2018 – P4.org 6

Programmable Network Devices
• PISA: Flexible Match+Action ASICs
◦ Intel Flexpipe, Cisco Doppler, Cavium (Xpliant), Barefoot Tofino, …
• NPU
◦ EZchip, Netronome, …
• CPU
◦ Open Vswitch, eBPF, DPDK, VPP…
• FPGA
◦ Xilinx, Altera, …

These devices let us tell them how to process packets.

Copyright © 2018 – P4.org 7

What can you do with P4?
• Layer 4 Load Balancer – SilkRoad[1]
• Low Latency Congestion Control – NDP[2]
• In-band Network Telemetry – INT[3]
• Fast In-Network cache for key-value stores – NetCache[4]
• Consensus at network speed – NetPaxos[5]
• Aggregation for MapReduce Applications [6]
• … and much more
[1] Miao, Rui, et al. "SilkRoad: Making Stateful Layer-4 Load Balancing Fast and Cheap Using Switching ASICs." SIGCOMM, 2017.
[2] Handley, Mark, et al. "Re-architecting datacenter networks and stacks for low latency and high performance.” SIGCOMM, 2017.
[4] Kim, Changhoon, et al. "In-band network telemetry via programmable dataplanes.” SIGCOMM. 2015.
[3] Xin Jin et al. “NetCache: Balancing Key-Value Stores with Fast In-Network Caching.” To appear at SOSP 2017
[5] Dang, Huynh Tu, et al. "NetPaxos: Consensus at network speed.” SIGCOMM, 2015.
[6] Sapio, Amedeo, et al. "In-Network Computation is a Dumb Idea Whose Time Has Come." Hot Topics in Networks. ACM, 2017.
Copyright © 2018 – P4.org 8
Brief History and Trivia

• May 2013: Initial idea and the name “P4”

• July 2014: First paper (SIGCOMM CCR)
• Aug 2014: First P414 Draft Specification (v0.9.8)
• Sep 2014: P414 Specification released (v1.0.0)
• Jan 2015: P414 v1.0.1
• Mar 2015: P414 v1.0.2
• Nov 2016: P414 v1.0.3
• May 2017: P414 v1.0.4

• Apr 2016: P416 – first commits

• Dec 2016: First P416 Draft Specification
• May 2017: P416 Specification released

Copyright © 2018 – P4.org 9

P4_16 Data Plane Model

Copyright © 2018 – P4.org 10

PISA: Protocol-Independent Switch Architecture

Programmer defines the

tables and the exact Programmer declares
Programmer declares the processing algorithm how the output packet
headers that should be will look on the wire
recognized and their order in
the packet

Programmable Match-Action Pipeline

Programmable Programmable
Parser Deparser

Copyright © 2018 – P4.org 11

PISA in Action
• Packet is parsed into individual headers (parsed representation)
• Headers and intermediate results can be used for matching and
actions
• Headers can be modified, added or removed
• Packet is deparsed (serialized)
Programmable Match-Action Pipeline
Programmable Programmable
Parser Deparser

Copyright © 2017 – P4.org 12

P416 Language Elements

State machine,
Parsers bitfield extraction

Tables, Actions,
Controls control flow
statements

Basic operations
Expressions and operators

Bistrings, headers,
Data Types structures, arrays

Architecture Programmable blocks

and their interfaces
Description
Support for specialized
Extern Libraries components

Copyright © 2017 – P4.org 13

P4_16 Approach

Term Explanation

P4 Target An embodiment of a specific hardware implementation

Provides an interface to program a target via some set of P4-programmable components,

P4 Architecture externs, fixed components

Community-Developed Vendor-supplied

P416
Language
P416 Core
Library
Extern Architecture
Libraries Definition

Copyright © 2018 – P4.org 14

Example Architectures and Targets
V1Model

TM

SimpleSumeSwitch

TM

Portable Switch Architecture (PSA)

TM Anything

Copyright © 2018 – P4.org 15

Programming a P4 Target
User supplied

Control Plane

RUNTIME
P4 Program P4 Compiler Add/remove Extern Packet-in/out
table entries control

CPU port
P4 Architecture Target-specific Extern
configuration Load Tables Data Plane
Model objects
binary

Target
Vendor supplied

Copyright © 2018 – P4.org 16

Lab 1: Basics

Copyright © 2018 – P4.org 17

Before we start...
•Install VM image (Look for instructor with USB sticks)
•Please make sure that your VM is up to date
◦$ cd ~/tutorials && git pull
•We’ll be using several software tools pre-installed on the VM
◦Bmv2: a P4 software switch
◦p4c: the reference P4 compiler
◦Mininet: a lightweight network emulation environment
•Each directory contains a few scripts
◦$ make : compiles P4 program, execute on Bmv2 in Mininet, populate tables
◦*.py: send and receive test packets in Mininet
•Exercises
◦Each example comes with an incomplete implementation; your job is to finish it!
◦Look for “TODOs” (or peek at the P4 code in solution/ if you must)
Copyright © 2018 – P4.org 18
V1Model Architecture
• Implemented on top of Bmv2’s simple_switch target

Parser Checksum Verification / Checksum Update / Deparser

Ingress Match-Action Egress Match-Action

Traffic
Manager

Copyright © 2017 – P4.org 19

V1Model Standard Metadata
struct standard_metadata_t { • ingress_port - the port on which
bit<9> ingress_port;
bit<9> egress_spec; the packet arrived
bit<9> egress_port;
bit<32> clone_spec; • egress_spec - the port to which
bit<32> instance_type;
bit<1> drop;
bit<16> recirculate_port;
the packet should be sent to
bit<32> packet_length;
bit<32> enq_timestamp;
• egress_port - the port on which
bit<19> enq_qdepth;
bit<32> deq_timedelta;
the packet is departing from (read
bit<19> deq_qdepth;
bit<48> ingress_global_timestamp; only in egress pipeline)
bit<32> lf_field_list;
bit<16> mcast_grp;
bit<1> resubmit_flag;
bit<16> egress_rid;
bit<1> checksum_error;
}

Copyright © 2018 – P4.org 20

P416 Program Template (V1Model)
#include <core.p4> /* EGRESS PROCESSING */
#include <v1model.p4> control MyEgress(inout headers hdr,
/* HEADERS */ inout metadata meta,
struct metadata { ... } inout standard_metadata_t std_meta) {
struct headers { ...
ethernet_t ethernet; }
ipv4_t ipv4; /* CHECKSUM UPDATE */
} control MyComputeChecksum(inout headers hdr,
/* PARSER */ inout metadata meta) {
parser MyParser(packet_in packet, ...
out headers hdr, }
inout metadata meta, /* DEPARSER */
inout standard_metadata_t smeta) { control MyDeparser(inout headers hdr,
... inout metadata meta) {
} ...
/* CHECKSUM VERIFICATION */ }
control MyVerifyChecksum(in headers hdr, /* SWITCH */
inout metadata meta) { V1Switch(
... MyParser(),
} MyVerifyChecksum(),
/* INGRESS PROCESSING */ MyIngress(),
control MyIngress(inout headers hdr, MyEgress(),
inout metadata meta, MyComputeChecksum(),
inout standard_metadata_t std_meta) { MyDeparser()
... ) main;
}

Copyright © 2018 – P4.org 21

P416 Hello World (V1Model)
#include <core.p4>
#include <v1model.p4> control MyEgress(inout headers hdr,
struct metadata {} inout metadata meta,
struct headers {} inout standard_metadata_t standard_metadata) {
apply { }
parser MyParser(packet_in packet, }
out headers hdr,
inout metadata meta, control MyComputeChecksum(inout headers hdr, inout metadata
inout standard_metadata_t standard_metadata) { meta) {
apply { }
state start { transition accept; } }
}
control MyDeparser(packet_out packet, in headers hdr) {
control MyVerifyChecksum(inout headers hdr, inout metadata apply { }
meta) { apply { } } }

control MyIngress(inout headers hdr, V1Switch(

inout metadata meta, MyParser(),
inout standard_metadata_t standard_metadata) { MyVerifyChecksum(),
apply { MyIngress(),
if (standard_metadata.ingress_port == 1) { MyEgress(),
standard_metadata.egress_spec = 2; MyComputeChecksum(),
} else if (standard_metadata.ingress_port == 2) { MyDeparser()
standard_metadata.egress_spec = 1; ) main;
}
}
}
Copyright © 2018 – P4.org 22
P416 Hello World (V1Model)
#include <core.p4>
#include <v1model.p4> control MyEgress(inout headers hdr,
struct metadata {} inout metadata meta,
struct headers {} inout standard_metadata_t standard_metadata) {
apply { }
parser MyParser(packet_in packet, out headers hdr, }
inout metadata meta,
inout standard_metadata_t standard_metadata) { control MyVerifyChecksum(inout headers hdr, inout metadata
state start { transition accept; } meta) { apply { } }
}
control MyComputeChecksum(inout headers hdr, inout metadata
control MyIngress(inout headers hdr, inout metadata meta, meta) { apply { } }
inout standard_metadata_t standard_metadata) {
action set_egress_spec(bit<9> port) { control MyDeparser(packet_out packet, in headers hdr) {
standard_metadata.egress_spec = port; apply { }
} }
table forward {
key = { standard_metadata.ingress_port: exact; } V1Switch( MyParser(), MyVerifyChecksum(), MyIngress(),
actions = { MyEgress(), MyComputeChecksum(), MyDeparser() ) main;
set_egress_spec;
NoAction; Key Action Name Action Data
}
size = 1024; 1 set_egress_spec 2
default_action = NoAction();
}
apply { forward.apply(); } 2 set_egress_spec 1
}
Copyright © 2018 – P4.org 23
Running Example: Basic Forwarding
•We’ll use a simple application as a running example—a basic
router—to illustrate the main features of P416

•Basic router functionality:

◦Parse Ethernet and IPv4 headers from packet
◦Find destination in IPv4 routing table
◦Update source / destination MAC addresses
◦Decrement time-to-live (TTL) field
◦Set the egress port
◦Deparse headers back into a packet

•We’ve written some starter code for you (basic.p4) and

implemented a static control plane
Copyright © 2018 – P4.org 24
Basic Forwarding: Topology

h1 h2
(10.0.1.1) (10.0.2.2)
s1 s2

s3

h3
(10.0.3.3)
Copyright © 2018 – P4.org 25
P416 Types (Basic and Header Types)
typedef bit<48> macAddr_t; Basic Types
typedef bit<32> ip4Addr_t;
header ethernet_t { • bit<n>: Unsigned integer (bitstring) of size n
macAddr_t dstAddr; • bit is the same as bit<1>
macAddr_t srcAddr;
• int<n>: Signed integer of size n (>=2)
bit<16> etherType;
} • varbit<n>: Variable-length bitstring
header ipv4_t {
bit<4> version;
Header Types: Ordered collection of members
bit<4> ihl;
bit<8> diffserv; • Can contain bit<n>, int<n>, and varbit<n>
bit<16> totalLen; • Byte-aligned
bit<16> identification;
bit<3> flags;
• Can be valid or invalid
bit<13> fragOffset; • Provides several operations to test and set validity bit:
bit<8> ttl; isValid(), setValid(), and setInvalid()
bit<8> protocol;
bit<16> hdrChecksum;
ip4Addr_t srcAddr; Typedef: Alternative name for a type
ip4Addr_t dstAddr;
}

Copyright © 2018 – P4.org 26

P416 Types (Other Types)
/* Architecture */
struct standard_metadata_t {
Other useful types
bit<9> ingress_port;
bit<9> egress_spec; • Struct: Unordered collection of members (with no
bit<9> egress_port;
bit<32> clone_spec; alignment restrictions)
bit<32> instance_type;
bit<1> drop; • Header Stack: array of headers
bit<16> recirculate_port;
bit<32> packet_length; • Header Union: one of several headers
...
}

/* User program */
struct metadata {
...
}
struct headers {
ethernet_t ethernet;
ipv4_t ipv4;
}

Copyright © 2018 – P4.org 27

P416 Parsers
• Parsers are functions that map packets
into headers and metadata, written in a start
state machine style
• Every parser has three predefined states
◦ start
◦ accept
◦ reject
• Other states may be defined by the
programmer
• In each state, execute zero or more
statements, and then transition to
accept reject
another state (loops are OK)

Copyright © 2018 – P4.org 28

Parsers (V1Model)
/* From core.p4 */
extern packet_in {
void extract<T>(out T hdr); MyParser
void extract<T>(out T variableSizeHeader,
in bit<32> variableFieldSizeInBits); packet_in hdr
T lookahead<T>();
void advance(in bit<32> sizeInBits);
bit<32> length();
} meta meta
/* User Program */
parser MyParser(packet_in packet,
The platform Initializes
out headers hdr, User Metadata to 0
inout metadata meta,
inout standard_metadata_t std_meta) {

state start { standard_meta

packet.extract(hdr.ethernet);
transition accept;
}

Copyright © 2018 – P4.org 29

Select Statement
state start { P416 has a select statement that can be
transition parse_ethernet; used to branch in a parser
}
Similar to case statements in C or Java,
state parse_ethernet { but without “fall-through behavior”—i.e.,
packet.extract(hdr.ethernet);
break statements are not needed
transition select(hdr.ethernet.etherType) {
0x800: parse_ipv4;
default: accept;
In parsers it is often necessary to branch
} based on some of the bits just parsed
}
For example, etherType determines the
format of the rest of the packet

Match patterns can either be literals or

simple computations such as masks

Copyright © 2018 – P4.org 30

Coding Break

Copyright © 2018 – P4.org 31

P416 Controls
• Similar to C functions (without loops)

• Can declare variables, create tables, instantiate externs, etc.

• Functionality specified by code in apply statement

• Represent all kinds of processing that are expressible as DAG:

◦ Match-Action Pipelines
◦ Deparsers
◦ Additional forms of packet processing (updating checksums)

• Interfaces with other blocks are governed by user- and

architecture-specified types (typically headers and metadata)
Copyright © 2018 – P4.org 32
Example: Reflector (V1Model)
control MyIngress(inout headers hdr, Desired Behavior:
inout metadata meta,
inout standard_metadata_t std_meta) {
/* Declarations region */ • Swap source and
bit<48> tmp; destination MAC
apply { addresses
/* Control Flow */
tmp = hdr.ethernet.dstAddr;
hdr.ethernet.dstAddr = hdr.ethernet.srcAddr;
• Bounce the packet back
hdr.ethernet.srcAddr = tmp; out on the physical port
std_meta.egress_spec = std_meta.ingress_port; that it came into the
}
}
switch on

Copyright © 2018 – P4.org

Example: Simple Actions
control MyIngress(inout headers hdr, • Very similar to C functions
inout metadata meta, • Can be declared inside a control or
inout standard_metadata_t std_meta) { globally
• Parameters have type and direction
action swap_mac(inout bit<48> src,
inout bit<48> dst) {
• Variables can be instantiated inside
bit<48> tmp = src; • Many standard arithmetic and
src = dst; logical operations are supported
dst = tmp; ◦ +, -, *
} ◦ ~, &, |, ^, >>, <<
◦ ==, !=, >, >=, <, <=
apply { ◦ No division/modulo
swap_mac(hdr.ethernet.srcAddr, • Non-standard operations:
hdr.ethernet.dstAddr); ◦ Bit-slicing: [m:l] (works as l-value too)
std_meta.egress_spec = std_meta.ingress_port; ◦ Bit Concatenation: ++
}
}

Copyright © 2018 – P4.org

P416 Tables
• The fundamental unit of a Match-Action Pipeline
◦ Specifies what data to match on and match kind
◦ Specifies a list of possible actions
◦ Optionally specifies a number of table properties
■ Size
■ Default action
■ Static entries
■ etc.
• Each table contains one or more entries (rules)
• An entry contains:
◦ A specific key to match on
◦ A single action that is executed when a packet matches the entry
◦ Action data (possibly empty)

Copyright © 2018 – P4.org 35

Tables: Match-Action Processing
Control Plane

(DataPlane)
Parameters

Directional
Headers and Metadata
Lookup Key

Hit
Key Action ID Action Data
Action

ID
Action ID
Code

Selector
Hit/Miss

(Action Data)
Directionless

Parameters
Data

Headers and Headers and

Metadata
Default Default
Action Metadata
(Input) Action ID Action Data
Execution (Output)

Copyright © 2018 – P4.org

Example: IPv4_LPM Table
• Data Plane (P4) Program
◦ Defines the format of the table
10.0.1.1 10.0.1.2 ■ Key Fields
■ Actions
■ Action Data
◦ Performs the lookup
◦ Executes the chosen action
• Control Plane (IP stack,
Key Action Action Data Routing protocols)
10.0.1.1/32 ipv4_forward dstAddr=00:00:00:00:01:01 ◦ Populates table entries with
port=1 specific information
10.0.1.2/32 drop ■ Based on the configuration
*` NoAction ■ Based on automatic discovery
■ Based on protocol calculations

Copyright © 2018 – P4.org 37

IPv4_LPM Table
table ipv4_lpm {
key = {
hdr.ipv4.dstAddr: lpm;
}
actions = {
ipv4_forward;
drop;
NoAction;
}
size = 1024;
default_action = NoAction();
}

Copyright © 2018 – P4.org 38

Match Kinds
/* core.p4 */ • The type match_kind is special in P4
match_kind {
exact, • The standard library (core.p4) defines
ternary, three standard match kinds
lpm ◦ Exact match
} ◦ Ternary match
◦ LPM match
/* v1model.p4 */
match_kind { • The architecture (v1model.p4) defines
range, two additional match kinds:
selector ◦ range
} ◦ selector

/* Some other architecture */ • Other architectures may define (and

match_kind { provide implementation for) additional
regexp, match kinds
fuzzy
}
Copyright © 2018 – P4.org 39
Defining Actions for L3 forwarding
/* core.p4 */ • Actions can have two different

(DataPlane)
Parameters

Directional
action NoAction() { types of parameters
} ◦ Directional (from the Data Plane)
◦ Directionless (from the Control
/* basic.p4 */ Plane)
action drop() { • Actions that are called directly:
mark_to_drop(); ◦ Only use directional parameters Action
} • Actions used in tables: Code

◦ Typically use directionless

/* basic.p4 */ parameters

(Action Data)
Directionless

Parameters
action ipv4_forward(macAddr_t dstAddr, ◦ May sometimes use directional
bit<9> port) { parameters too
...
}
Action
Execution

Copyright © 2018 – P4.org 40

Applying Tables in Controls

control MyIngress(inout headers hdr,

inout metadata meta,
inout standard_metadata_t standard_metadata) {
table ipv4_lpm {
...
}
apply {
...
ipv4_lpm.apply();
...
}
}

Copyright © 2018 – P4.org 41

P416 Deparsing
/* From core.p4 */ • Assembles the headers back
extern packet_out {
void emit<T>(in T hdr); into a well-formed packet
}
• Expressed as a control function
/* User Program */
control DeparserImpl(packet_out packet, ◦ No need for another construct!
in headers hdr) {
apply {
... • packet_out extern is defined in
packet.emit(hdr.ethernet); core.p4: emit(hdr): serializes
... header if it is valid
}
}
• Advantages:
• Makes deparsing explicit...
...but decouples from parsing
Copyright © 2018 – P4.org 42
Coding Break

Copyright © 2018 – P4.org 43

Basic Tunneling
•Add support for basic tunneling to the basic IP router

•Define a new header type (myTunnel) to encapsulate the IP

packet

•myTunnel header includes:

○ proto_id : type of packet being encapsulated
○ dst_id : ID of destination host

•Modify the switch to perform routing using the myTunnel header

Copyright © 2018 – P4.org 44

Basic Tunneling TODO List
• Define myTunnel_t header type and add to headers struct
• Update parser
• Define myTunnel_forward action
• Define myTunnel_exact table
• Update table application logic in MyIngress apply statement
• Update deparser
• Adding forwarding rules

Copyright © 2018 – P4.org 45

Basic Forwarding: Topology

h1 h2
1 1
(10.0.1.1) 2 2 (10.0.2.2)
(dst_id: 1) s1 3 3 s2 (dst_id: 2)

2 3

1
s3

h3
(10.0.3.3) (dst_id: 3)
Copyright © 2018 – P4.org 46
Coding Break

Copyright © 2018 – P4.org 47

FAQs
• Can I apply a table multiple times in my P4 Program?
◦ No (except via resubmit / recirculate)
• Can I modify table entries from my P4 Program?
◦ No (except for direct counters)
• What happens upon reaching the reject state of the parser?
◦ Architecture dependent
• How much of the packet can I parse?
◦ Architecture dependent

Copyright © 2018 – P4.org 48

Fin!

Copyright © 2018 – P4.org 49

Debugging
• Bmv2 maintains logs that keep track of
control MyIngress(...) { how packets are processed in detail
table debug { • /tmp/p4s.s1.log
key = { • /tmp/p4s.s2.log
std_meta.egress_spec : exact; • /tmp/p4s.s3.log
}
actions = { } • Can manually add information to the
}
logs by using a dummy debug table that
apply {
reads headers and metadata of interest
...
debug.apply();
} •[15:16:48.145] [bmv2] [D]
} [thread 4090] [96.0] [cxt 0]
Looking up key:
* std_meta.egress_spec : 2

Copyright © 2018 – P4.org 50

Lab 2: P4Runtime

1
Copyright © 2018 – P4.org, ONF
P4 Software Tools

2
Copyright © 2018 – P4.org, ONF
Makefile: under the hood
P4Runtime static controller
runtime.json
P4Runtime client
test.p4

P4Runtime server

p4c-bm2-ss L

simple_switch_grpc (BMv2)
o
TM g

Egress
Ingress
P4
D Debugger
test.json test.p4info e
b
u
g

Parser Deparser
Packet Packet
generator Port Interface sniffer
veth0..n
Linux Kernel
3
Copyright © 2018 – P4.org, ONF
Step 1: P4 Program compilation

test.p4 $ p4c-bm2-ss --p4v 16 \

-o test.json \
--p4runtime-file test.p4info \
--p4runtime-format text \
test.p4
p4c-bm2-ss

test.json test.p4info

4
Copyright © 2018 – P4.org, ONF
Step 2: Preparing veth Interfaces

test.p4 $ sudo ~/p4lang/tutorials/examples/veth_setup.sh

# ip link add name veth0 type veth peer name veth1

# for iface in “veth0 veth1”; do
ip link set dev ${iface} up
sysctl net.ipv6.conf.${iface}.disable_ipv6=1
TOE_OPTIONS="rx tx sg tso ufo gso gro lro rxvlan txvlan rxhash”
p4c-bm2-ss for TOE_OPTION in $TOE_OPTIONS; do
/sbin/ethtool --offload $intf "$TOE_OPTION”
done
done

test.json test.p4info veth

0 2 4 2n

Linux
Kernel
2n
veth 1 3 5 +1

5
Copyright © 2018 – P4.org, ONF
Step 3: Starting BMv2
$ sudo simple_switch_grpc --log-console --dump-packet-data 64 \
–i 0@veth0 -i 1@veth2 … [--pcap] --no-p4 \
test.p4 -- --grpc-server-addr 0.0.0.0:50051 --cpu-port 255 \
test.json

P4Runtime server (gRPC)

p4c-bm2-ss L

simple_switch_grpc (BMv2)
o
TM g

Egress
Ingress
D
test.json test.p4info e
b
u
g

Parser Deparser

Port Interface
veth0..n
Linux Kernel
6
Copyright © 2018 – P4.org, ONF
Step 4: Starting P4Runtime static controller
P4Runtime static controller
$ python $(RUN_SCRIPT) -t runtime.json
$(TOPO) $(run_args) P4Runtime client (gRPC)

test.p4 P4Runtime server (gRPC)

● P4Runtime server
L address and port

simple_switch_grpc (BMv2)
TM o ● BMv2 JSON and
g
P4Info to deploy
● Table entries
p4c-bm2-ss

Egress
Ingress
D
e
b
u
g

test.json test.p4info Parser Deparser

Port Interface
veth0..n
Linux Kernel
7
Copyright © 2018 – P4.org, ONF
P4Runtime
• API overview
• Workflow
• Exercise - Tunneling

8
Copyright © 2018 – P4.org, ONF
Runtime control of P4 data planes
User supplied

Control Plane
P4 Program P4 Compiler Add/remove Extern Packet-in/out
table entries control

CPU port
P4 Architecture Target-specific Extern
configuration Load Tables Data Plane
Model objects
binary

Vendor supplied Focus of this

session
9
Copyright © 2018 – P4.org, ONF
Existing approaches to runtime control
• P4 compiler auto-generated runtime APIs
◦ Program-dependent -- hard to provision new P4 program without restarting
the control plane!

• BMv2 CLI
◦ Program-independent, but target-specific -- control plane not portable!

• OpenFlow
◦ Target-independent, but protocol-dependent -- protocol headers and
actions baked in the specification!

• OCP Switch Abstraction Interface (SAI)

◦ Target-independent, but protocol-dependent
10
Copyright © 2018 – P4.org, ONF
Why do we need another data plane control API?

11
Copyright © 2018 – P4.org, ONF
Properties of a runtime control API

API Target-independent Protocol-independent

P4 compiler
auto-generated ✔ ✘
BMv2 CLI ✘ ✔
OpenFlow ✔ ✘
SAI ✔ ✘
P4Runtime ✔ ✔

12
Copyright © 2018 – P4.org, ONF
What is P4Runtime?
• Framework for runtime control of P4 targets
◦ Open-source API + server implementation
https://github.com/p4lang/PI
◦ Initial contribution by Google and Barefoot

• Work-in-progress by the p4.org API WG

◦ Draft of version 1.0 available

• Protobuf-based API definition

◦ p4runtime.proto
◦ gRPC transport

• P4 program-independent
◦ API doesn’t change with the P4 program

• Enables field-reconfigurability P4 target

◦ Ability to push new P4 program without recompiling the
software stack of target switches
13
Copyright © 2018 – P4.org, ONF
Protocol Buffers (protobuf) Basics
• Language for describing data for syntax = "proto3";
serialization in a structured way message Person {
string name = 1;
• Common binary wire-format int32 id = 2;
string email = 3;
• Language-neutral
enum PhoneType {
◦ Code generators for: Action Script, C, MOBILE = 0;
C++, C#, Clojure, Lisp, D, Dart, Erlang, HOME = 1;
Go, Haskell, Java, Javascript, Lua, WORK = 2;
}
Objective C, OCaml, Perl, PHP, Python,
Ruby, Rust, Scala, Swift, Visual Basic, ... message PhoneNumber {
string number = 1;
• Platform-neutral PhoneType type = 2;
}
• Extensible and backwards compatible repeated PhoneNumber phone = 4;
}
• Strongly typed

14
Copyright © 2018 – P4.org, ONF
gRPC Basics
● Use Protocol Buffers to define service API and messages
● Automatically generate native stubs in:
○ C / C++
○ C#
○ Dart
○ Go
○ Java
○ Node.js
○ PHP
○ Python
○ Ruby
● Transport over HTTP/2.0 and TLS
○ Efficient single TCP connection implementation that supports
bidirectional streaming
15
Copyright © 2018 – P4.org, ONF
gRPC Service Example
// The greeter service definition.
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (HelloReply) {}
}

// The request message containing the user's name.

message HelloRequest {
string name = 1;
}

// The response message containing the greetings

message HelloReply {
string message = 1;
}

More details here: https://grpc.io/docs/guides/

16
Copyright © 2018 – P4.org, ONF
P4Runtime Service
Enables a local or remote entity to arbitrate mastership, load the
pipeline/program, send/receive packets, and read and write
forwarding table entries, counters, and other P4 entities.
service P4Runtime {
rpc Write(WriteRequest) returns (WriteResponse) {}
rpc Read(ReadRequest) returns (stream ReadResponse) {}
rpc SetForwardingPipelineConfig(SetForwardingPipelineConfigRequest)
returns (SetForwardingPipelineConfigResponse) {}
rpc GetForwardingPipelineConfig(GetForwardingPipelineConfigRequest)
returns (GetForwardingPipelineConfigResponse) {}
rpc StreamChannel(stream StreamMessageRequest)
returns (stream StreamMessageResponse) {}
}

17
Copyright © 2018 – P4.org, ONF
P4Runtime Service

P4Runtime Protobuf Definition:

https://github.com/p4lang/p4runtime/blob/master/proto/p4/v1/p4runtime.proto

Service Specification:
Working draft of version 1.0 is available now
https://p4.org/p4-spec/docs/P4Runtime-v1.0.0.pdf

18
Copyright © 2018 – P4.org, ONF
P4Runtime Write Request
message WriteRequest { message Entity {
uint64 device_id = 1; oneof entity {
uint64 role_id = 2; ExternEntry extern_entry = 1;
Uint128 election_id = 3; TableEntry table_entry = 2;
repeated Update updates = 4; ActionProfileMember
} action_profile_member = 3;
ActionProfileGroup
action_profile_group = 4;
message Update { MeterEntry meter_entry = 5;
enum Type { DirectMeterEntry direct_meter_entry = 6;
UNSPECIFIED = 0; CounterEntry counter_entry = 7;
INSERT = 1; DirectCounterEntry direct_counter_entry = 8;
MODIFY = 2; PacketReplicationEngineEntry
DELETE = 3; packet_replication_engine_entry = 9;
} ValueSetEntry value_set_entry = 10;
Type type = 1; RegisterEntry register_entry = 11;
Entity entity = 2; }
} }

19
Copyright © 2018 – P4.org, ONF
P4Runtime Table Entry
p4runtime.proto simplified excerpts:
message TableEntry { message FieldMatch {
To add a table entry, the control
uint32 table_id; uint32 field_id; plane needs to know:
repeated FieldMatch match; message Exact {
Action action; bytes value;
int32 priority; } • IDs of P4 entities
... message Ternary { ◦ Tables, field matches, actions,
} bytes value; params, etc.
bytes mask;
}
message Action { ... • Field matches for the
uint32 action_id; oneof field_match_type { particular table
message Param { Exact exact;
◦ Match type, bitwidth, etc.
uint32 param_id; Ternary ternary;
bytes value; ...
}
} • Parameters for the particular
}
repeated Param params; action
}

Full protobuf definition: • Other P4 program attributes

https://github.com/p4lang/PI/blob/master/proto/p4/p4runtime.proto

20
Copyright © 2018 – P4.org, ONF
P4Runtime workflow
P4Info test.p4

• Captures P4 program attributes

p4c-bm2-ss
needed at runtime (compiler)
◦ IDs for tables, actions, params, etc.
◦ Table structure, action parameters, etc.

test.p4info
• Protobuf-based format

• Target-independent compiler output test.json

◦ Same P4Info for BMv2, ASIC, etc.

Full P4Info protobuf specification:

BMv2
https://github.com/p4lang/p4runtime/blob/master/proto/p4/config/v1/p4info.proto
21
Copyright © 2018 – P4.org, ONF
P4Info example
basic_router.p4 basic_router.p4info
... actions {
id: 16786453
action ipv4_forward(bit<48> dstAddr, name: "ipv4_forward"
bit<9> port) { params {
/* Action implementation */ id: 1
} name: "dstAddr"
bitwidth: 48
... ...
id: 2
table ipv4_lpm { name: "port"
key = { bitwidth: 9
hdr.ipv4.dstAddr: lpm; }
} }
actions = { ...
ipv4_forward; tables {
... P4 compiler id: 33581985
} name: "ipv4_lpm"
... match_fields {
} id: 1
name: "hdr.ipv4.dstAddr"
bitwidth: 32
match_type: LPM
}
action_ref_id: 16786453
}
22
Copyright © 2018 – P4.org, ONF
P4Runtime Table Entry Example
basic_router.p4 Protobuf message
action ipv4_forward(bit<48> dstAddr, table_entry {
bit<9> port) { table_id: 33581985
/* Action implementation */ match {
} field_id: 1
table ipv4_lpm { lpm {
key = { value: "\n\000\001\001"
hdr.ipv4.dstAddr: lpm; Control plane prefix_len: 32
} generates }
actions = { }
ipv4_forward; action {
... action_id: 16786453
} params {
... param_id: 1
} value: "\000\000\000\000\000\n"
}
params {
param_id: 2
Logical view of table entry value: "\000\007"
}
hdr.ipv4.dstAddr=10.0.1.1/32 }
-> ipv4_forward(00:00:00:00:00:10, 7) }

23
Copyright © 2018 – P4.org, ONF
P4Runtime SetPipelineConfig
message SetForwardingPipelineConfigRequest { test.p4
enum Action {
UNSPECIFIED = 0; Pipeline config
VERIFY = 1; p4info json
VERIFY_AND_SAVE = 2;
VERIFY_AND_COMMIT = 3; p4c-bm2-ss SetPipelineConfig()
COMMIT = 4; (compiler)
Pipeline config bits
RECONCILE_AND_COMMIT = 5;
}
uint64 device_id = 1;
uint64 role_id = 2;
Uint128 election_id = 3; BMv2
Action action = 4;
ForwardingPipelineConfig config = 5;
} message ForwardingPipelineConfig {
config.P4Info p4info = 1;
// Target-specific P4 configuration.
bytes p4_device_config = 2;
}
24
Copyright © 2018 – P4.org, ONF
P4Runtime StreamChannel
// Packet sent from the controller to the switch.
message StreamMessageRequest { message PacketOut {
oneof update { bytes payload = 1;
MasterArbitrationUpdate // This will be based on P4 header annotated as
arbitration = 1; // @controller_header("packet_out").
PacketOut packet = 2; // At most one P4 header can have this annotation.
DigestListAck digest_ack = 3; repeated PacketMetadata metadata = 2;
} }
}
// Packet sent from the switch to the controller.
message StreamMessageResponse { message PacketIn {
oneof update { bytes payload = 1;
MasterArbitrationUpdate // This will be based on P4 header annotated as
arbitration = 1; // @controller_header("packet_in").
PacketIn packet = 2; // At most one P4 header can have this annotation.
DigestList digest = 3; repeated PacketMetadata metadata = 2;
} }
}

25
Copyright © 2018 – P4.org, ONF
P4Runtime Common Parameters
• device_id
◦ Specifies the specific forwarding chip or software bridge
◦ Set to 0 for single chip platforms
• role_id
◦ Corresponds to a role with specific capabilities (i.e. what operations, P4
entities, behaviors, etc. are in the scope of a given role)
◦ Role definition is currently agreed upon between control and data planes
offline
◦ Default role_id (0) has full pipeline access
• election_id
◦ P4Runtime supports mastership on a per-role basis
◦ Client with the highest election ID is referred to as the "master", while all
other clients are referred to as "slaves"
◦ Set to 0 for single instance controllers
26
Copyright © 2018 – P4.org, ONF
Mastership Arbitration
• Upon connecting to the device, the client (e.g. controller)
needs to open a StreamChannel
• The client must advertise its role_id and election_id using a
MasterArbitrationUpdate message
◦ If role_id is not set, it implies the default role and will be granted full
pipeline access
◦ The election_id is opaque to the server and determined by the control
plane (can be omitted for single-instance control plane)
• The switch marks the client for each role with the highest
election_id as master
• Master can:
◦ Perform Write requests
◦ Receive PacketIn messages
◦ Send PacketOut messages 27
Copyright © 2018 – P4.org, ONF
Remote control
table_entry {
table_id: 33581985
Target-independent P4-defined
match { protobuf format OSPF BGP etc.
field_id: 1
lpm {
custom protocol
value: "\f\000\...
prefix_len: 8
p4info
}
}
action {
Remote control plane
action_id: 16786453
params {
param_id: 1
value: "\000\0...
}
params {
param_id: 2
value: 7
}
}
}

p4info p4info p4info

Vendor A Vendor B Vendor C

28
Copyright © 2018 – P4.org, ONF
Local control
table_entry {
Same table_id: 33581985
match {
P4-defined
etc.
target-independent field_id: 1
OSPF BGP lpm {
protocol protobuf format value: "\f\000\...
prefix_len: 8
p4info }
Local control plane }
action {
action_id: 16786453
params {
param_id: 1
p4info value: "\000\0...
}
params {
param_id: 2
value: 7
}
}
}

P4 target

The P4Runtime API can be used equally well

by a remote or local control plane
29
Copyright © 2018 – P4.org, ONF
P4Runtime on Fixed-Function Chips
TODO pull some stratum slides

30
Copyright © 2018 – P4.org, ONF
 l ide ogle
S o
G
o m
fr ● P4 program tailored to apps / role - does not describe the hardware
● Switch maps program to fixed-function ASIC
● Enables portability

Control

Logical

Physical

ASIC 1 ASIC 2
P4Runtime API recap
Things we covered: What we didn’t cover:
• P4Info • How to control other P4 entities
• Table entries ◦ Externs, counters, meters
• Set pipeline config • Packet-in/out support
• Controller replication
◦ Via master-slave arbitration
• Batched reads/writes
• Switch configuration
◦ Outside the P4Runtime scope
◦ Achieved with other mechanisms
■ e.g., OpenConfig and gNMI

Work-in-progress by the p4.org API WG

Expect API changes in the future
32
Copyright © 2018 – P4.org, ONF
 P4Runtime workflow

test.p4
P4 compiler generates 2 files:
1. Target-specific binaries
○ Used to configure switch pipeline
p4c
(e.g. binary config for ASIC, bitstream for FPGA, etc.)
(compiler)
2. P4Info file
○ Captures P4 program attributes needed to
runtime control
■ Tables, actions, parameters, etc. test.p4info

○ Protobuf-based format
○ Target-independent compiler output
■ Same P4Info for SW switch, ASIC, etc. test.bin

Full P4Info protobuf specification:

https://github.com/p4lang/PI/blob/master/proto/p4/config/p4info.proto
BMv2
33
Copyright © 2018 – P4.org, ONF
P4Runtime exercise

34
Copyright © 2018 – P4.org, ONF
Exercise Overview
Controller’s responsibilities:
1. Establish a gRPC connection to the
switches for the P4Runtime service mycontroller.py
2. Push the P4 program to each switch
3. Write the tunnel forwarding rules:
a. myTunnel_ingress rule to
encapsulate packets on the ingress
switch
b. myTunnel_forward rule to
forward packets on the ingress
switch
c. myTunnel_egress rule to
decapsulate and forward packets
on the egress switch
4. Read the tunnel ingress and egress
counters every 2 seconds

35
Copyright © 2018 – P4.org, ONF
Getting started
The source code has already been downloaded on your VM:
~/tutorials/exercises/p4runtime

You should start by reading the README.md

In this exercise, you will need to complete the

implementation of writeTunnelRules
in mycontroller.py

You will need two Terminal windows: one for

your dataplane network (Mininet) that you will
start using make, and the other is for your
controller program.

To find the source code:

https://github.com/p4lang/tutorials/

36
Copyright © 2018 – P4.org, ONF
Lab 3: Advanced P4 Features

Copyright © 2018 – P4.org

Plan
In this afternoon lab session, we'll explore some of
P4's advanced features and see how to implement
functionality commonly found in modern data planes.

● Source Routing: implement simple form of source

routing using P4's header stacks
● Load Balancing: implement ECMP-like functionality
using P4's extern functions
● Network Telemetry: implement fine-grained telemetry
directly in the data plane

Copyright © 2018 – P4.org 2

Source Routing
2,3,2,2,1, {payload}

h1 2 2 1 h2
(10.0.1.1) 1
(10.0.2.2)
s1 3 3 s2

2 3

1 s3

3
h3
(10.0.3.3)
Copyright © 2018 – P4.org
Source Routing: Packet Format
#define MAX_HOPS 9 • Parse source routes only if etherType is
0x1234
const bit<16> TYPE_IPV4 = 0x800;
const bit<16> TYPE_SRCROUTING = 0x1234;
header srcRoute_t { • The special value bos == 1 indicates the
bit<1> bos; “bottom of stack”
bit<15> port;
}
• Forward packets using source routes,
struct headers { and also decrement IPv4 TTL
ethernet_t ethernet;
srcRoute_t[MAX_HOPS] srcRoutes; • Drop the packet if source routes are not
ipv4_t ipv4; valid
}

• Hint: Use the next, pop_front primitives

packet.extract(hdr.srcRoutes.next)
hdr.srcRoutes.pop_front(1)
Copyright © 2018 – P4.org 4
Simple Load Balancing
h1 → 10.0.0.1

h1 h2
(10.0.1.1) (10.0.2.2)
s1 s2

Virtual IP (10.0.0.1)
shared by h2 and h3

s3

h3
(10.0.3.3)
Copyright © 2018 – P4.org 5
Hashing (V1Model)
enum HashAlgorithm {
csum16,
Computes the hash of data
xor16, (using algo) modulo max
crc32,
crc32_custom, and adds it to base
crc16,
crc16_custom,
random, Uses type variables (like
identity
}
C++ templates / Java
extern void hash<O, T, D, M>( Generics) to allow hashing
out O result,
in HashAlgorithm algo, primitive to be used with
in T base, many different types.
in D data,
in M max);

Copyright © 2018 – P4.org 6

Monitoring & Debugging

Congestion!
Fine … h2
h1
(10.0.1.1) (10.0.2.2)

h11 h22
(10.0.1.11) s1 s2 (10.0.2.22)

Hosts can infer

congestion: Switches can
● RTT help!
● Packet drops s3
● Dup Acks

h3
(10.0.3.3)

Copyright © 2018 – P4.org 7

Monitoring & Debugging

s1
Copyright © 2018 – P4.org 8
Multi-Route Inspection
Switch ID, queue h2
h1 depth, … (anything)
(10.0.2.2)
(10.0.1.1) pkt

s1 s2
h22
h11 (10.0.2.22)
(10.0.1.11)

The most verbose ECN s3

Packets carry their own logs! h3
(10.0.3.3)

Copyright © 2018 – P4.org 9

Multi-Route Inspect: Packet Format
header mri_t {
• Header validity opertions:
bit<16> count;
o hdr.setValid(): add_header
}
o hdr.setInvalid(): remove_header
o hdr.isValid():test validity
header switch_t {
switchID_t swid;
• Header Stacks
qdepth_t qdepth;
o hdr[CNT] stk;
}
• Header Stacks in Parsers
o stk.next
struct headers {
o stk.last
ethernet_t ethernet;
o stk.lastIndex
ipv4_t ipv4; • Header Stacks in Controls
ipv4_option_t ipv4_option; o stk[i]
mri_t mri; o stk.size
switch_t[MAX_HOPS] swtraces; o stk.push_front(int count)
} o stk.pop_front(int count)

Copyright © 2018 – P4.org 10

Header verification
/* Standard errors, defined in core.p4 */
error {
NoError, // no error
PacketTooShort, // not enough bits in packet for extract
NoMatch, // match expression has no matches
StackOutOfBounds, // reference to invalid element of a header stack
OverwritingHeader, // one header is extracted twice
HeaderTooShort, // extracting too many bits in a varbit field
ParserTimeout // parser execution time limit exceeded
}
/* Additional error added by the programmer */
error { IPv4BadHeader }
...
state parse_ipv4 {
packet.extract(hdr.ipv4);
verify(hdr.ipv4.version == 4, error.IPv4BadHeader);
transition accept;
}

Copyright © 2018 – P4.org 11

Wrapping up & Next Steps

Copyright © 2018 – P4.org 1

Why P416?
• Clearly defined semantics
◦ You can describe what your data plane program is doing
• Expressive
◦ Supports a wide range of architectures through standard methodology
• High-level, Target-independent
◦ Uses conventional constructs
◦ Compiler manages the resources and deals with the hardware
• Type-safe
◦ Enforces good software design practices and eliminates “stupid” bugs
• Agility
◦ High-speed networking devices become as flexible as any software
• Insight
◦ Freely mixing packet headers and intermediate results

Copyright © 2018 – P4.org 2

Things we covered
• The P4 ”world view”
◦ Protocol-Independent Packet Processing
◦ Language/Architecture Separation
◦ If you can interface with it, it can be used
• Key data types
• Constructs for packet parsing
◦ State machine-style programming
• Constructs for packet processing
◦ Actions, tables and controls
• Packet deparsing
• Architectures & Programs

Copyright © 2018 – P4.org 3

Things we didn’t cover
• Mechanisms for modularity
◦ Instantiating and invoking parsers or controls
• Details of variable-length field processing
◦ Parsing and deparsing of options and TLVs
• Architecture definition constructs
◦ How these “templated” definitions are created
• Advanced features
◦ How to do learning, multicast, cloning, resubmitting
◦ Header unions
• Control plane interface

Copyright © 2018 – P4.org 4

 The P4 Language Consortium

• Consortium of academic
and industry members

• Open source, evolving,

domain-specific language

• Permissive Apache license,

code on GitHub today

• Membership is free:
contributions are welcome

• Independent, set up as a
California nonprofit

Copyright © 2018 – P4.org 5