MPLS

a mini-course
Presented by:
Yaakov (J) Stein
CTO
July 2018

Your Network’s Edge

MPLS Slide 1
 Course Outline
1. Forwarding and routing
2. Forwarding Equivalence Classes
3. Problems with IP routing
4. Label Switching
5. MPLS
6. MPLS control plane(s)
7. LDP and BGP – in-depth slides
8. MPLS-TP (including the GACh, OAM, APS)
9. Pseudowires
10. Some PW-specific mechanisms
11. PWE3 control protocol
12. TDM PWs
13. Ethernet PWs (and VPWS, VPLS)
14. Other PW types
MPLS Slide 2
Forwarding and routing

MPLS Slide 3
 Buses (e.g., Ethernet LANs)
not for for me!! not for not for
me me me

not for not for not for

me me me

The simplest way to interconnect hosts is via a bus

• each message is sent to all possible recipients
• all receivers check the message’s destination address
• if message destination address does not match receiver’s address
– message ignored (except in promiscuous mode)
• if message destination address matches receiver’s address
– message received
MPLS Slide 4
 Packet Switched Networks

Bus topologies are only practical for very small networks (LANs)
Ethernet fixes this by defining a bridge that can filter traffic
Packet Switched Networks contain Network Elements (routers, switches)
that perform two distinct functions :
• forwarding (data plane component)
• routing (control plane component)
From the forwarding point of view
there are two different types of Packet Switched Network:
• ConnectionLess (e.g., IP)
• Connection Oriented (e.g., ATM, MPLS???)
MPLS Slide 5
 Connectionless Forwarding

host host
CL forwarder (router)
A PSN is connectionless (CL) if
• no setup is required before sending a packet
each router makes an independent forwarding decision
• packets are self-describing
packet inserted anywhere will be properly forwarded
Note:
– the address must have global significance
– IP only runs between routers
it relies on a L2 protocol (Ethernet, PPP) from router to host

MPLS Slide 6
 IP Routing
• Distance Vector (Bellman-Ford), e.g. RIP
– send <addr,cost> to neighbors
– routers maintain cost to all destinations
– need to solve “count to  problem“
• Path Vector, e.g. BGP
– send <addr,cost,path> to neighbors
– similar to distance vector, but w/o “count to  problem“
– like distance vector has slow convergence*
– doesn’t require consistent topology
– can support hierarchical topology => exterior protocol (EGP)
• Link State, e.g. OSPF, IS-IS
– send <neighbor-addr,cost> to all routers
– determine entire flat network topology (Dijkstra’s algorithm)
– fast convergence*, guaranteed loopless => interior routing protocol (IGP)

*convergence time is the time taken until all routers work consistently
before convergence is complete packets may be misforwarded, and there may be loops

MPLS Slide 7
 IP Forwarding
IPv4 and IPv6 are CL PSNs

what field(s) are parsed for forwarding?

field used depends on application
• regular: use longest destination prefix match
• ToS: use longest destination prefix match + exact match on ToS
• multicast: use longest/exact match on source/group address

how is forwarding performed?

forwarding algorithm depends on routing algorithm!
• for Distance Vector - forward by cost
• for Path Vector - forward by cost and path
• for Link State - Dijkstra’s algorithm

MPLS Slide 8
 Connection Oriented Forwarding
CO forwarder (switch)

1 1
2 2
3 3
4 4
5 5

input ports output ports

each forwarder maintains forwarding table (or table per input port)

control component:
• route must be set-up (table must be updated) before data sent
• set-up may be manual or signaled
• once route no longer needed it should be torn-down

MPLS Slide 9
 CO Forwarding
CO addresses need not have global significance
by using locally defined addresses:
• addresses are smaller in size
• lookup is faster
• no need for global allocation mechanism (purchasing, DHCP)
• no need to maintain global database
L2 forwarding is based on address read and swap

12 17 23

Note:
when addresses are purely local
CO forwarding is often called called L2 (link layer) forwarding
MPLS Slide 10
 CO Forwarding Table
input port input address output port output address

1 21 2 21

1 37 1 21

2 12 5 12

3 5 5 12

4 15 3 37

Note: never really have an input port column

either it is irrelevant (CO address is per switch not per input port),
or if needed (e.g. ATM) we keep separate tables per input port (more efficient)
MPLS Slide 11
 CO Control Plane
For CO forwarding we need to find and set-up a path
These can be done via
– manual configuration (strict explicit route)
– use routing protocols (e.g. P-NNI for ATM)
CO routing protocols search for a global path
– hence they can guarantee global characteristics (SLA)
– pure CL routing can not guarantee path QoS
Information source usually knows the desired characteristics
so usually use source routing
– can force route to go through specific forwarders (loose explicit route)
– can reject forwarders that can not guarantee needed characteristic
– can request resource reservation in selected forwarders

MPLS Slide 12
Forwarding Equivalence Classes

MPLS Slide 13
 Forwarding Equivalence Classes
Simple routers typically search for IP prefix
and immediately perform forwarding
e.g., the UNIX router LSDB Dijkstra returns forwarding information
Today’s routers decouple (as much as possible)
• routing protocol(s) (building LSDB, RIBs)
• building FIB
• packet parsing/classification/search
• forwarding decision
All packets that are to be forwarded in the same way
are grouped together to form a FEC
Thus the search algorithm returns a FEC label
and then a second lookup is performed
to find the forwarding information

MPLS Slide 14
 Equivalence Classes
In set theory we define an Equivalence Class as:
set of elements that can be considered equivalent for some purpose
Theorem reflexive: a~a
symmetric : a ~ b  b ~ a
any equality relation (e.g., common features) transitive : a ~ b and b ~ c  a ~ c

divides elements into non-overlapping equivalence classes

Example:
equality modulo 3 for positive integers
A = B (mod 3) if and only if A = 3a + c and B = 3b + c
or (A-B) divides by 3
there are three equivalence classes :
{ 0, 3, 6, 9, 12, … }
{1, 4, 7, 10, 13, …}
{2, 5, 8, 11, 14, … }
note that every positive integer is in exactly one EC

MPLS Slide 15
 Forwarding Equivalence Classes
A FEC is the set of all packets that are to be treated in the same way
By the theorem every packet belongs to one unique FEC
So the router’s forwarding job is now :
1) parse packet
2) search and classify packet as belong to a particular FEC
3) forward based on FEC’s forwarding information
Packets in the same FEC should follow the same path
but in IP this is not directly enforced
since each successive router reclassifies the packet’s FEC
If the router could insert information into the packet
informing the next router of its FEC
• this would save a lot of processing at the following routers
• the subsequent forwarding would be CO instead of CL
Unfortunately, this is impossible (without label switching)!
MPLS Slide 16
 FECs

What constitutes a FEC ?

For plain IP routing
all packets w/ same destination IP prefix
that prefix being the longest in the routing table

We would like more control

• coarsest granularity IP address
all packets with a destination address
served by a given router IP prefix
• finest granularity
all packets from given source socket
to given destination socket IP prefix +
with specified handling requirements ToS/DSCP
MPLS Slide 17
Problems with IP routing

MPLS Slide 18
 Problems with IP routing
IP is great – but not perfect !

• scalability
– router table overload
– routing convergence slow-down
– increase in queuing time and routing traffic
– problems specific to underlying L2 technologies
• hard to implement load balancing
• QoS and Traffic Engineering
• problem of routing changes
• difficulties in routing protocol update
• lack of VPN services

MPLS Slide 19
 Scalability
When IP was first conceived, scalability was not a problem
but as number of hosts increases, routing shows stress
Simplistic example
– N hosts
– each router serves M hosts
– each router entry takes a bytes
hence
– router table size a N ~ N
– N / M ~ N routers (more routers => slower convergence)
– packet processing time* ~N (since have to examine entire table)
– ~N routers send to ~N routers tables of size ~N
so routing table update traffic increases ~N 3 (or ~N 4 )

* IP routing requires 1000s of clocks per decision

MPLS Slide 20
 L2 Backbone Doesn’t Help!
Instead of expensive and slow IP routers
operators once used faster and cheaper ATM switches in core

ATM

Classical IP
over ATM

but this actually makes things worse!

ATM switches are transparent to routers
ATM switches do not participate in IP routing protocols
so every IP router becomes logically adjacent to every other
and we need ~ N2 ATM VCs !
If only the ATM switches could understand IP routing protocols…
Unfortunately, this is impossible (without label switching)! MPLS Slide 21
 Load Balancing
“fish diagram”
A D
1 traffic from A to G = 1Gb
C G traffic from B to G = 500Mb
all links 1Gb except EF - 500 Mb
B 0.5 E F

were C,D,E, and F ATM switches there would be no problem

(1Gb over ACDG, 500Mb over BCEFG)
with standard IP hop-count cost function, all traffic over CDG
resulting in 1.5Gb there (congestion) and CEFG idle
with administrative cost on CDG we can force all the traffic to CEFG
even worse congestion !
finally with administrative cost and Equal Cost MultiPath (ECMP)
750 Mb over CDG and CEFG, link EF is still congested !
solution requires traffic engineering
It would be great if we could add TE to IP
Unfortunately, this is hard (without label switching)!
MPLS Slide 22
 QoS and TE
CL networks can not guarantee path QoS
you can’t reserve resources for handling a packet
since you don’t know where the packet is going to go !
But other protocols in the IP suite can help
TCP adds CO layer compensating for loss and mis-ordering
but that is correcting for mishaps that already occurred
IntServ (RSVP) sets up path with reserved resources
but that modifies IP so much that it never caught on
DiffServ (DSCP) prioritizes packets
but that doesn’t provide any guarantees
So IP network managers use network engineering
– throw BW at the problem
rather than traffic engineering
– optimally exploiting the BW that is there
TE is great, but not possible in IP (without label switching)!
MPLS Slide 23
 Routing Changes
IP routing is satisfactory in the steady state
but what happens when something changes?
Any change in routing information
(new router, router failure, new inter-router connection, etc)
necessitates updating of tables of all routers
Convergence is generally slow
A change in the routing protocol is even worse
(e.g. Bellman-Ford to present, classed to classless, IPv4 to IPv6)
since it necessitates upgrade of all router software
Upgrades may have to be simultaneous
What we need is a more complete separation
of forwarding from routing functionality (ForCES)
Unfortunately, this separation is only recently starting to appear!

MPLS Slide 24
 VPN Services
192.115.243.19 192.115.243.79

IP

192.115.243.19

IP was designed to interconnect networks

not to provide VPN services
When we connect routers from different customers
the security isolation is weak
LANs may use non globally unique addresses (present solution - NAT)
Interconnect may entail complex provider-customer relationships

MPLS Slide 25
Label Switching

MPLS Slide 26
 Solution - Label Switching
CO forwarder (switch) CL forwarder (router) LSR

label switching adds the strength of CO to CL forwarding

label switching involves three stages:
– routing (topology determination) using L3 protocols
– path setup (label binding and distribution) perhaps using new protocol(s)
– data forwarding

label switching the solution to all of the above problems

– speeds up forwarding
– decreases forwarding table size (by using local labels)
– enables support for QoS and arbitrary granularity FECs
– load balancing by explicitly setting up paths
– complete separation of routing and forwarding algorithms
no new routing algorithm needed
but new signaling algorithm may be needed

MPLS Slide 27
 Where is it?

Unlike TCP, the label switching CO layer lies under the CL layer
If there is a broadcast L2 (e.g. Ethernet), the CO layer lies above it
higher layers
layer 3 (e.g. IP)
label switching (layer 2.5) shim header
layer 2 (e.g. ethernet)
physical layer

Hence, label switching is sometimes called layer 2.5 switching

MPLS Slide 28
 Labels
A label is a short, fixed length, structure-less address
The following are not labels:
• telephone number (not fixed length, country-code+area-code+local-number)
• Ethernet address (too long, note vendor-code is not meaningful structure)
• IP address (too long, has fields)
• ATM address (has VP/VC)
Not an explicit requirement, but normally only local in significance
Label(s) added to CL packet, in addition to L3 address
Layer 2.5 forwarding
• requires a flow setup process and signaling protocol
• may find a different route than the L3 forwarding
– and thus support higher granularity FECs
• may be faster than L3 forwarding

MPLS Slide 29
 Label Switching Architecture
downstream direction
upstream direction
Label Switched Path

L3 link L3 link
ingress egress
L3 router Label Edge Router Label Edge Router L3 router
Label Switched Routers

Label switching is needed in the core, access can be L3 forwarding*

Core interfaces the access at the edge (ingress, egress)
LSR router that can* perform label switching
LER LSR with non-MPLS neighbors (LSR at edge of core network)
LSP unidirectional path used by label switched forwarding (ingress to egress)

* not every packet needs label switching

MPLS Slide 30
 Label Switched Forwarding
LSP needs to be setup before data is forwarded
and should be torn down once no longer needed
LSR performs
– label switched forwarding* for labeled packets
label space may be
– per platform (unique to LSR) or
– per port (unique to input interface - like ATM)
LSRs optionally support L3 forwarding for unlabeled packets
ingress LER
– assigns packet* to FEC * once packet is assigned to a FEC and labeled
– labels packet no other LSR looks at the L3 headers
– forwards it downstream using label switching
egress LER
– removes label
– forwards packet using L3 forwarding
– exception: PHP (discussed later)
MPLS Slide 31
 Hierarchical Forwarding
Many networks use hierarchical routing
– decreases router table size
– increases forwarding speed
– decreases routing convergence time

telephone numbers Country-Code Area-Code Exchange-Code Line-Number

972 2 588 9159

host … SLD TLD

Internet URLs myrad . rad . co . il

ATM VC
VC VP
VC

Ethernet/802.3 address space is flat (even though written in byte fields)

MPLS Slide 32
 IP Routing Hierarchy
IPv4
– originally flat space (1st come - 1st) until router tables exploded
– then 3 classes
– now CIDR <RFC1519> A 0 net host24
7

B 10 net14 host16 C 110 net22 host8

AS=Autonomous System ASany host

IP exploits hierarchy by employing interior/exterior routing protocols

IP can even support arbitrary levels of hierarchy
by advertising aggregated addresses AS12
13 13
AS AS
AS14

but the exploitation is not optimal

MPLS Slide 33
 Label Stacks
since labels are structure-less, the label space is flat
label switching can support arbitrary levels of hierarchy
by using a label stack

top label
another label
yet another label
bottom label

label forwarding based only on top label

before forwarding, three possibilities (listed in NHLFE) :
– read top label and pop
– read top label and swap
– read top label, swap, and push new label(s)

MPLS Slide 34
Example Uses of Label Stack

MPLS Slide 35
 Fast ReRoute
IP has no inherent recovery method (like SDH)
in order to ensure resilience we can provide local detours

to reroute quickly we pre-prepare labels for the bypass links

when link is down
change fwd table
swap swap
10 11 12

swap swap
+
push swap 14 pop
11
13 11 from here on
no difference! *
11

protection LSP
* label space per LSR
not per input port MPLS Slide 36
 Label Switched VPNs

C C C
C CE
CE
C AC C
AC
customer 1 network P P P customer 2 network
PE PE
P P
AC
AC
C C C
C provider network CE
CE
C C
Key
C Customer router customer 1 network
customer 2 network CE Customer Edge router
P Provider router
PE Provider Edge router
MPLS Slide 37
 Label Switched VPNs (cont.)
If customers 1 and 2 use overlapping IP addresses
C-routers have inconsistent tables egress PE
Ingress PE (LER) inserts two labels egress CE
IP header
Only PEs know about customers payload
P-routers see only the label of the egress PE-router
– P-routers don’t see IP addresses, so there is no ambiguity
– they don’t know about VPNs at all
– no need to understand customer configuration
– smaller tables
– no rerouting if customer reconfigures
Ingress PE router only knows about CE routers
– no need to understand customer configuration (C-routers)

MPLS Slide 38
 PWE 3

P P
router router

PE P
PE
native router router router native
service
service
A tunnel may
P
contain many router
PWs

PW label is not a real label tunnel label(s)

it just identifies native service instance PW label
P routers don’t know about PWs PW control word
just how to get to egress PE payload
With MS-PWs, PW labels becomes real labels
MPLS Slide 39
 Segment Routing
Sometimes we need to force a packet to travel a particular path
for example, in order to have it inspected or manipulated
IP had a source routing option (specified by end host)
but it raised security concerns (RFCs 5095, 7126)
and was deprecated
MPLS label stacking provides a natural alternative
source specification is specified by the ingress router SID 1
requires deep label stacks SID 2
...
The principle is simple (this is only one way to perform SR) SID n
• IGP advertises label for each adjacency (segment) payload
• ingress router inserts a label for each segment to be traversed
• each router pops a label (or leaves it w/o swapping)
• packet arrives at egress router its label stack depleted
MPLS Slide 40
Label (20b) TC(3b) S(1b) TTL (8b)

MPLS

MPLS Slide 41
 MPLS history
Many different label switching schemes were invented
• Cell Switching Router (Toshiba) <RFC 2098,2129>
• IP Switching (Ipsilon, bought by Nokia) <RFC 2297>
• Tag Switching (Cisco) <RFC 2105>
• Aggregate Route-based IP Switching (IBM)
• IP Navigator (Cascade bought by Ascend bought by Lucent)

so the IETF decided to standardize a single method

This method is called MPLS

MPLS Slide 42
 MPLS

Of all possible label switching technologies

what is special about MPLS ?
• multiprotocol - from above and below
• label shim header (label may also be in legacy L2)
• single forwarding algorithm, including for multicast and TE
• control plane consisting of
– topology discovery via IP routing protocols
– and label distribution via
• piggybacked on existing routing protocols
• via the Label Distribution Protocol
• via RSVP-TE (and historically CRLDP) for Traffic Engineering

MPLS Slide 43
MultiProtocol Label Switching

(everything)

IPv4 IPv6 MPLS PWs

MPLS

Ethernet SDH/OTN
(GFP, HDLC)
legacy
(ATM, FR)

MPLS Slide 44
 MPLS Shim Header
Label (20b) TC(3b) S(1b) TTL (8b)

when a shim header is used, its format is:

Special (reserved) labels
Label there are 2 different labels (+ 2
20 20
multicast labels) 0 IPv4 explicit null
1 router alert
2 IPv6 explicit null
3 implicit null
Traffic Class (ex-EXP) 7 entropy label indicator
was CoS in Cisco Tag Switching 13 MPLS-TP GAL
may influence packet queuing 14 Y.1711 OAM label
QoS may be via E-LSP or L-LSP
S=0 top label
Stack bit S=1 indicates bottom of label stack S=0 another label
TTL decrementing hop count S=0 yet another label
used to eliminate infinite routing loops
S=1 bottom label
and for MPLS traceroute
generally copied from/to IP TTL field
MPLS Slide 45
 Single Forwarding Algorithm
IP uses different forwarding algorithms
for unicast, unicast w/ ToS, multicast, etc.
LSR uses one forwarding algorithm (LER is more complicated)
– read top label L for TTL processing see RFC 3443
– consult Incoming Label Map (forwarding table) [Cisco terminology LFIB]
– perform label stack operation (pop L, swap L - M, swap L - M and push N)
– forward based on L’s Next Hop Label Forwarding Entry
NHLFE contains:
– next hop (output port, IP address of next LSR)
• if next hop is the LSR itself then operation must be pop
• for multicast there may be multiple next hops, and packet is replicated
– label stack operation to be performed
– any other info needed to forward (e.g. L2 format, how label is encoded)
ILM contains:
– a NHLFE for each incoming label
– possibly multiple NHLFEs for a label, but only one used per packet
MPLS Slide 46
 LER Forwarding Algorithm
LER’s forwarding algorithm is more complex
• check if packet is labeled or not
• if labeled
– then forward as LSR
– else [Cisco terminology LIB]
• lookup destination IP address in FEC-To-NHLFE Map
• if in FTN
– then prepend label and forward using LSR algorithm
– else forward using IP forwarding

IP pure IP link MPLS link

router LER LSR

MPLS Slide 47
 MPLS flavors
We now distinguish four flavors of MPLS :
1. plain vanilla MPLS (usually with LDP, perhaps with RSVP-TE for FRR)
not true CO – pinned to route not to NEs
used in Internet core
2. MPLS for L3VPN services (RFC 4364 <ex-2547> using BGP)
used to deliver VPN services to businesses
3. MPLS-TE (currently with RSVP-TE)
true CO with resource reservation
used when SLA guarantees given
4. MPLS-TP (usually with management system, can use RSVP-TE)
does not assume the existence of IP forwarding plane
does not require control plane – can work with management OSS
implements OAM and APS functionality
MPLS Slide 48
Special mechanisms

MPLS Slide 49
 Penultimate Hop Popping
PH
IP link
I E CE
MPLS domain

the egress LER E also may have to work overtime:

– read top label
– lookup label in ILM
– find that in NHLFE that the label must be popped
– lookup IP address in IP routing
– forward to CE using IP forwarding
we can save a lookup (and the first 3 steps) by performing PHP
but pay in loss of OAM capabilities
penultimate LSR PH performs the following:
– read top label
– lookup label in ILM
– pop label revealing IP address of CE router
– forward to CE using IP forwarding MPLS Slide 50
 Route Aggregation (VC merge)
net 1 IP link 12
1
13
31 32 IP link
E 3 net 3
IP link 22 23
net 2 2 MPLS domain

traffic from both network 1 and network 2 is destined for network 3

scalability advantages
– fewer labels
– conserve table memory
disadvantages
– IP forwarding may be required
– OAM backwards trail is destroyed

MPLS Slide 51
 Entropy Labels
Ethernet and IP enable load balancing using LAG and ECMP
which map flows based hashing header fields
MPLS labels have no spare information = entropy
and peeking under the MPLS requires DPI (warning - layer violation)
One approach is to use multiple different labels for the same FEC
but that increases control plane complexity
RFC 6790 defines entropy labels (and RFC 6391 defines a FAT PW)
• ingress LER hashes IP header fields as for ECMP
• it pushes an entropy label
• it pushes the Entropy Label Indicator (reserved label 7)
• it pushes the MPLS transport label
RFC 8012 defines ping and traceroute mechanisms
based on entropy labels
MPLS Slide 52
 FAT PW and Entropy Labels
The Flow Aware Transport PW label and the entropy label
• are never signaled (but their capability is signaled)
• are used as key for ECMP-like hashing
• packets in a single flow share the same entropy label
• are slightly different in format and placement

MPLS label(s) MPLS label(s)

PW label ELI (label = 7, S=0))
FAT label (S=1, TTL=1) Entropy label (TTL=0)
PW Control Word optionally more label(s)
PW payload payload

MPLS Slide 53
 I need a label

I allocated 13

A 13 data B

MPLS control plane

MPLS Slide 54
 Data and control planes

control plane

• all IP routing protocols (OSPF, BGP, etc)

• procedure to bind label to FEC (label assignment)
• protocol to distribute label binding information
• procedure to create forwarding table

user (data) plane

• procedure to label incoming packet
• forwarding procedure
– forwarding table lookup
– label stack operations

MPLS Slide 55
 Label Distribution Protocols
When an LSR creates/removes a FEC - label binding
it needs to inform other LSRs of its decision
MPLS allows piggybacking label distribution on routing protocols
– protocols already in use (don’t need to invent or deploy)
– eliminates race conditions (when route or binding, but not both, defined)
– ensures consistency between binding and routing information
– only for distance vector or path vector routing protocols (not OSPF, IS-IS)
– not all routing protocols are sufficiently extensible (RIP isn’t)
– has been implemented for BGP-4
MPLS WG invented a new protocol LDP for “plain” label distribution
– messages sent reliably using TCP/IP
– messages encoded in TLVs
– discovery mechanism to find other LSRs
… and extended RSVP to LSPs for QoS - RSVP-TE
New approaches use OpenFlow (SDN) and OSPF (for segment routing)

MPLS Slide 56
 LER Architecture
control plane

IP IP routing protocols
IP routers
routing
tables label binding and
distribution protocols
free label table
LSRs

user (data) plane

FTN
IP
forwarding
table egress LER

MPLS
labeling procedure forwarding
table

MPLS Slide 57
 All the Tables
FEC table FEC protocol input port handling

192.115/16 IPv4 2 best-effort

Free Labels 128-200 presently free

FTN FEC port/label in port/label out

192.115/16 2/17 3/137

port/label in port/label out next hop operation

ILM
NHLFE 2/17 3/137 5.4.3.2 swap

MPLS Slide 58
 Binding and Distribution Options
label binding (assignment)
– per port or per LSR label space
– control driven vs. data driven (traffic driven)
– liberal vs. conservative label retention

label distribution (advertisement)

– downstream vs. upstream
– downstream on-demand (dod) vs.
downstream unsolicited (du)
– independent vs. ordered

MPLS Slide 59
 Per Port Label Space
LSR may have a separate label space for each input port (I/F)
or a single common label space
or any combination of the two
Separate labels spaces means separate forwarding tables per port
ATM LSR had only per port label spaces (leads to interleave problem)
per port label spaces increases number of available labels
common label space facilitates several MPLS mechanisms (e.g. FRR)

MPLS Slide 60
 Control vs. Data Driven
there are two philosophies as to when to create a binding
data-driven (traffic-driven) binding (Toshiba CSR, Ipsilon IP-Switching)
automatically create binding when data packets arrive
(from first packet?, after enough packets? when tear LSP down?)
control-driven binding (Cisco Tag Switching, IBM ARIS)
create binding when routing updates arrive
(only update when topology changes? update upon request?)

although not specifically stated in the architecture document

MPLS assumes control driven binding *

* two implementations of control driven:

– topology-driven (routing tables are consulted)
– control-traffic driven (only routing update messages are used)

MPLS Slide 61
 Liberal vs. Conservative Retention

LSR receives “advertisements” (label distribution messages)

from other LSRs
conservative label retention:
LSR retains only label-to-FEC bindings that are presently needed
liberal label retention
LSR stores all bindings received (more labels need to be maintained)
using liberal retention can speed response to topology changes
LSRs must agree upon mode to be used

A advertises label B
B is previous hop LSR
A
but C retains label anyway
later routing change makes C the previous hop C
C immediately can start forwarding
MPLS Slide 62
 Downstream vs. Upstream
downstream

binding means to allocate a label to a FEC

LSR allocates the label from “free label” pool
Which LSR allocates the label ?
Unicast MPLS uses downstream binding
• label allocated by LSR downstream from the LSR that prepends it
• label distribution information flows upstream
reverse in direction from data packets
to set up LSP through link from LSR A to LSR B
LSR B binds label 13 to FEC label 13
B advertises label to LSR A
LSR A sends packets with label 13 to B A 13 B
MPLS Slide 63
 On-demand vs. Unsolicited

downstream on-demand label distribution:

LSRs may explicitly request a label from its downstream LSR
unsolicited label distribution:
LSR distributes binding to upstream LSR w/o a request
(e.g. based on time interval, or upon receipt of topology change)
LSR may support on-demand, unsolicited, or both
adjacent LSRs must agree upon which mode to be used
LSR A needs to send a packet to LSR B
LSR A requests a label from LSR B
B binds label 13 to the FEC
B distributes the label I need a label
A starts sending data with label 13 I allocated 13

A 13 data B
MPLS Slide 64
 Independent vs. Ordered
independent binding (Tag Switching)
– each LSR makes independent decision to bind and distribute
ordered binding (ARIS)
• egress LSR binds first and distributes binding to neighbors
• LSR that believes that it should be the penultimate LSR
binds and distributes to its neighbors
• binding proceeds in orderly fashion until ingress LSR is reached
LSRs must agree upon mode to be used
B sees that it is egress LSR for 192.115.6
B allocates label 13
B distributes label to C and D 13
C distributes label to E
D 192.115/16
B A
E C 13

MPLS Slide 65
 LDP tasks
A label distribution protocol is a signaling protocol
that can perform the following tasks:

• discover LSR peers

• initiate and maintain LDP session
• signal label request
• advertise binding
• signal label withdrawal
• loop prevention
• explicit routing
• resource reservation

MPLS Slide 66
Label Distribution Protocols
Label distribution can be performed using various protocols
There are presently the following options:
• Management protocols
• LDP
– MPLS-enhanced IP networks
– used as basis for PWE3 control protocol
• BGP4-MPLS
– mainly for RFC 4364 VPNs
• RSVP-TE
– traffic engineering support
• CR-LDP
– constraint based (no longer recommended by IETF)

MPLS Slide 67
 LDP and BGP
slides for those who love protocols

MPLS Slide 68
 LDP vs. BGP
both use TCP for reliable transport (LDP uses UDP for hellos)
both are hard-state protocols
both use TLV format for parameters

BGP LDP
multiprotocol (IPv4, IPv6, IPX, MPLS) MPLS only
highly complex protocol simpler protocol
provides routing / label distribution only label distribution
built-in autodiscovery mechanism no built-in autodiscovery

MPLS Slide 69
 LDP
Major focus of the IETF MPLS WG was the design of LDP
based on similar TDP from Cisco
LDP sets up a bidirectional LDP session
both sides can request or advertise labels
LDP usually uses TCP
– needs reliable transport (e.g. what happens if miss a binding)
– needs in-order delivery (e.g. binding+withdrawal)
– hard to develop new reliable transport protocols
– single acknowledgement timer for session
– piggybacking ACK on data packets
Use UDP for discovery (hello) messages
Periodic keepalive messages (if not received, session terminated)
All messages encoded in TLV (Type Length Value) form

MPLS Slide 70
 LDP Setup

Hello (UDP)
Discovery
Hello (UDP)

Initialization (TCP)
Session
Initialization (TCP)

Label Request (TCP)

Distribution

Label Distribution (TCP)

MPLS Slide 71
 Discovery Phase
• LSR periodically multicast transmits hello to “LDP discovery” UDP port
– to “all routers on subnet” multicast group
– to preconfigured IP address (when not all LSRs on same subnet)
(extended discovery) “targeted LDP”
• LSRs listen on this UDP port for hello messages
• Hello message contains:
– hold time
– LSR Identifier
• when LSR receives Hello from another LSR
– it opens a TCP connection to that other LSR (if needed)
or (for extended discovery)
– it unicast transmits a hello back to the other LSR
• LDP session can now be established

MPLS Slide 72
 Session Initialization

• The LSR with higher identifier sends (TCP)

a session initialization message to the other LSR
• session initialization message contains:
– LDP Protocol version
– label distribution and control method
– timer values
– label space ranges
• If receiving LSR accepts these parameters
then it transmits a KeepAlive
else it transmits a reject

MPLS Slide 73
 Distribution Messages

• label mapping
– downstream LSR advertisement of a label mapping for a FEC
two FEC types: host address, IP address prefix
• label withdrawal
– reverse of mapping message
– downstream LSR informs upstream LSR
that it has revoked a previous binding
– upstream LSR can not longer use the label
• label release
– upstream LSR informs downstream LSR
that it no longer needs a binding
– typically when downstream is no longer next hop
and operating in conservative retention mode

MPLS Slide 74
 Request Messages
In dod mode upstream LSR must request binding
Upstream LSR sends label request message when:
– FEC in FEC table
next hop LSR is LDP peer
FEC not in forwarding table
– FEC next hop changes
upstream LSR doesn’t have a mapping from new next hop
– receives FEC label request from upstream LDP peer
next hop LSR is LDP peer
upstream LSR doesn’t have a mapping from next hop
Upstream LSR sends label request abort message when:
– upstream LSR needs to revoke request before satisfied
for example, next hop LSR for FEC has changed

MPLS Slide 75
 Notifications
There are two types of notifications:
– error notifications (fatal errors - terminate session)
– advisory notifications (status messages)
LSR sends notification messages when:
– received LDP message with unsupported protocol version
– received LDP message with unknown type
– KeepAlive timer expired
– session initialization fails due to unacceptable parameters
– etc.

MPLS Slide 76
 LDP state machine
• LSR periodically transmits hello UDP messages
– multicast to “all routers on subnet” group
– targeted to preconfigured IP address
• LSRs listen on this UDP port for hello messages
• When LSR receives hello from another LSR
– it opens a TCP connection to that other LSR
or (for extended discovery)
– it unicast transmits a hello back to the other LSR
• LSR with higher ID sends session initialization
• Other LSR LDP accepts (sends keepalive) or rejects
• Informative or keepalive messages sent

3.2
MPLS Slide 77
 LDP packet format
header (10B)

version length LDP-ID message TLVs

(2B) (2B) (6B) (variable)

version – presently 1
length - PDU length, excluding version and length fields
LDP-ID – identifies label space of sending LDP peer
– LSR-ID(4B) globally unique LSR ID
– label space ID (2B) for per-port label spaces
(zero for per-platform label spaces)
message TLVs – zero or more message TLVs (see next page)

MPLS Slide 78
 LDP message TLVs

mandatory optional
type length message-ID parameter parameter
U
(15b) (2B) (4B) TLVs TLVs
(variable) (variable)

U – unknown message bit

if message type unknown to receiver
U=0 – receiver returns notification to sender
U=1 – receiver silently ignores
length - message length, excluding type and length fields
message-ID – unique ID for message (for matching with returned notification)
If there are mandatory parameters
they must appear in a specific order
optional parameters may appear in any order
MPLS Slide 79
 All LDP message types
Hello (0x0100)
Initialization (0x0200)
KeepAlive (0x0201)
Notification (0x0001)
Address (0x0300)
Address Withdraw (0x0301)
Label Mapping (0x0400)
Label Withdraw (0x0402)
Label Request (0x0401)
Label Release (0x0403)
Label Abort Request (0x0404)

MPLS Slide 80
 LDP parameter (sub)TLVs
type length
U F
(14b) (2B) value

U – unknown message bit

if message type unknown to receiver:
U=0 – receiver returns notification to sender
U=1 – receiver silently ignores
F – forward unknown message bit
if U=1 and message type unknown to receiver
F=0 – do not forward
F=1 – forward
type - TLV type (FEC TLV, label TLV, address list TLV, hop count TLV, path vector TLV, status TLV )
length - length of value field in bytes
MPLS Slide 81
 FEC TLV
type (2B)
length (2B)
U=0 F=0 type=0x0100

FEC element 1

…
there may be more than one FEC element for mapping messages only
the FEC elements are not themselves TLVs (no length needed),
instead
– wildcard FEC (0x01)
– prefix FEC (0x02) + address family (IPv4, IPv6, Ethernet, E.164, etc.) +
prefix length in bits + prefix
– host address FEC (0x03) + address family + length + address

MPLS Slide 82
 Generic Label TLV

type (2B)
length (2B)
U=0 F=0 type=0x0200

label (20 bits)

this is the generic label TLV

there are also special label TLVs for ATM and FR based MPLS

MPLS Slide 83
 Status TLV

type (2B)
length (2B)
U F type=0x0300
E F status code data (30b)

message ID (32b) message type (16b)

Status TLVs : mandatory parameters in notification messages

optional parameters in other messages
U=0 when status in notification message, else U=1
E - fatal error bit E=1 for fatal error E=0 for advisory notification
the two F bits are equal and have the normal meaning
status code data = 0 means success
message ID - the message to which the status refers
message type - the message type to which the status refers

MPLS Slide 84
 Example full message -
label mapping
mapping message length=24 message-ID
U=0 type = 0x0400 (2B) (4B)

FEC TLV (2B)

length=8 (2B)
U=0 F=0 type=0x0100

prefix FEC element (8B)

type = 0x02 family=1(IPv4) prefix-length=16 prefix=192.115/16

label TLV (2B)

length=4 (2B)
U=0 F=0 type=0x0200

label = 17

MPLS Slide 85
 BGP4 Label Distribution
BGP peers exchange VPN routes
can easily associate a label with these routes
all BGP procedures are immediately available for use
for label distribution messages
BGP4 is a very extensible protocol
– multiprotocol extensions support address families
(originally for IPv4,IPv6, etc)
– MPLS defines a new address family

MPLS Slide 86
 BGP
header (19B)

marker length type data

(16B) (2B) (1B) (variable)

marker can be used for authentication (TCP MD5 signature)

length is total BGP PDU length, including header
type
– OPEN (for session initialization)
– UPDATE (add, change and withdraw routes)
– NOTIFICATION (return error messages, terminate session)
– KEEPALIVE (heartbeat)
KEEPALIVE packet consists of 19B header only

MPLS Slide 87
 BGP state machine

• idle – no session (awaiting session initialization)

• connect – attempting to connect to peer
• active – started TCP 3-way handshake (router busy)
• open sent – have sent OPEN message
• open confirm – after receiving TCP SYN for OPEN message
• established – BGP session up and running

MPLS Slide 88
 BGP OPEN

version my AS hold time BGP-ID op len opt parameters

(1B) (2B) (2B) (2B) (1B) (variable)

version (3 or 4) must be 4 for MPLS

my AS – identifier of autonomous system
hold time – max time (sec) between receipt of messages
BGP ID – sender’s BGP identifier
op len – length (bytes) of optional parameters
opt parameters - TLVs

MPLS Slide 89
 BGP UPDATE
WR len withdrawn PA len path NLRI
routes attributes
(2B) (2B) (var) (var)
(var)

Withdrawn Routes – list of routes no longer to be used (NLRI format- see below)
Path Attributes – route specific information (see next page)
Network Layer Reachability Information – (classless) routing information

len prefix
(1B) (variable)

the NLRI is a list of address-prefixes

each prefix must be masked from the left to the length specified

MPLS Slide 90
 BGP UPDATE - Path Attributes
flags type code
(1B) (1B)

flags
O – optional/well-known bit
if 1 must be recognized by all BGP implementations
if W=1 and unrecognized attribute, BGP sends notification and session closed
T – transitive/nontransitive bit
if 1 and attribute unrecognized it is passed along, else silently ignored
well-known attributes are always transitive

C – complete/partial bit (for optional transitive attributes only)

L – attribute length bit (=0 attribute length is 1B, =1 length is 2B)

type code
ORIGIN, AS_PATH, NEXT_HOP, MED, LOCAL_PREF,
AGGREGATOR, COMMUNITY, ORIGINATOR_ID…

MPLS Slide 91
 BGP NOTIFICATON

error error data

code subcode (var)
(1B) (2B)

all notification messages cause BGP session to close

error codes include:
– message header error
– open message error
– update message error
– hold timer expired
– state machine error
– other fatal error

MPLS Slide 92
MPLS-TP

MPLS Slide 93
Background
IP is the most popular packet-switched protocol

MPLS and Ethernet are the most popular server layers under IP
but neither is a transport network

At least some Service Providers want a

• packet-based transport network
• similar to present transport networks
• optimized for carrying IP

MPLS Slide 94
Characteristics of transport networks
1. High availability
1. Fault Management OAM
2. Automatic Protection Switching
2. Efficient utilization, SLA support, and QoS mechanisms
1. high determinism
2. Connection Oriented behavior
3. Performance Management OAM
3. Management plane (optionally control plane)
1. configuration management similar to traditional
2. efficient provisioning of p2p, p2m and m2m services
4. Scalability - must scale well with increase in
1. end-points
2. services
3. bandwidth
MPLS Slide 95
Possible solutions

There are two popular server network protocols for carrying IP

• Ethernet
• MPLS
(in the past there were ATM, frame relay, IP over SDH, etc.)
Extensions to both were proposed :
• Provider Backbone Transport (which became PBB-TE)
• Transport-MPLS (which became MPLS-TP)
PBT advanced in IEEE standardization (802.1ah + 802.1Qay)
but is now dead in the market
MPLS-TP was developed by both the IETF and the ITU-T
which eventually led to 2 incompatible versions

MPLS Slide 96
 MPLS-TP
MPLS-TP is a profile of MPLS, that is
• it reuses existing MPLS standards
• its data plane is a (minimal) subset of the full MPLS data plane
• it interoperates with existing MPLS (and PWE) protocols
without gateways

TP is similar to other transport networks (including look and feel)

TP is multi-vendor (in a single domain and between domains)
TP supports static provisioning via management plane
a control plane is defined but not mandatory to use
TP networks can be configured and operate w/o IP forwarding
TP’s data plane is physically/logically separated from
management/control planes
MPLS Slide 97
 The OAM issue
Since it strives to be a carrier-grade transport network
TP has strong OAM requirements
OAM has been the most contentious issue in standardization
OAM is carried in a Generic Associated Channel (GACh)
Two different OAM protocols break MPLS-TP into two distinct flavors
1. IETF bases its OAM on Bidirectional Forwarding Detection
BFD is a “hello” protocol originally between routers
before TP IETF standardized it for IP, MPLS, and PWs (in VCCV)
2. ITU bases its OAM on Ethernet OAM Y.1731 (802.1ag)
The mechanisms do not interoperate

MPLS Slide 98
The APS issue
MPLS-TP requires linear and ring protection mechanisms
Similar to what happened in OAM
the IETF and ITU developed different APS
The ITU adapted Ethernet APS mechanisms to MPLS
The IETF developed new mechanisms with the same functionality
The mechanisms can not interoperate

MPLS Slide 99
 Planes
TP supports static provisioning via management plane
a control plane (CP) is defined but not mandatory to use
TP networks can be configured and operate w/o IP forwarding
TP’s data plane is physically/logically separated from
management/control planes
Data plane continues to operate normally (forwarding, OAM, APS)
even if the management/control plane that configured it fails
TP can always distinguish user packets from control/management

MPLS Slide 100

 Data plane
TP is a CO PS network
TP defines PWs, LSPs, and segments (single links of LSP or PW path)
TP clients: IP, Ethernet, MPLS, MPLS-TP and can be extended to others
TP servers: Ethernet, MPLS-TP, SDH, OTN
TP supports
• traffic-engineered p2p and p2mp transport paths
• unidirectional/co-routed bidirectional/associated bidirectional flows
• mesh, ring, interconnected ring topologies
TP paths must be identifiable by a single label
The path’s source must be identifiable at destination
TP P2MP can exploit P2MP capabilities of a server layer
TP mechanisms can detect sub-SLA performance
MPLS Slide 101
 Topologies and connection types

TP paths are strictly Connection Oriented

and may be Traffic Engineered
TP supports :
• unidirectional p2p and p2mp connections
• co-routed bidirectional p2p paths
• associated bidirectional point-to-point transport p2p paths
TP should safeguard against forwarding loops
TP paths can span multiple (non-homogenous) domains
TP supports rings (with at least 16 nodes)
TP supports arbitrarily interconnected rings (1 or 2 interconnections)

MPLS Slide 102

 QoS

The main aim of TP is to enable SPs to guarantee SLAs

Thus QoS mechanisms are an essential part of TP
These mechanisms include:
• DiffServ traffic types and traffic class separation
• provisioning end-to-end bandwidth
• flexible BW allocation
• support for delay- and jitter- sensitive services
• guarantee of fair access to shared resources
• guaranteed resources for control/management-plane
traffic, regardless of the amount of data-plane traffic

MPLS Slide 103

 OAM
TP OAM applies to PWs, LSPs, and to segments, and may cross domains
TP OAM works independently and distinguishably at any label-stack depth
TP OAM fate-shares with user traffic, but is distinguishable from user traffic
TP OAM functionality can be configured by management or control plane
It should be possible to change configuration without impacting user traffic
Supported functionality:
• proactive CC
• proactive CV
• on-demand route tracing
• on-demand diagnostics (e.g., intrusive loopback)
• on-demand lock (administratively configured test state)
• proactive defect reporting (FDI and RDI)
• proactive client failure indication (CSF)
• proactive or on-demand packet loss measurement
• on-demand (and proactive) 1-way and 2-way delay measurement
TP OAM must not cause network congestion
MEPs and MIPs are defined
MPLS Slide 104
APS
TP APS is similar to APS in other transport networks
APS may be triggered by lower-layer/OAM/mngt/control plane
APS mechanism should be the same for p2p and p2mp
link, segment, and end-end protection are possible
Requirements:
• standard 50 ms switching time for 1200 km
• 100% protection must be supported
• priority logic is required but extra traffic is not required
• it must be possible to preconfigure protection paths
• it must be possible to test/validate protection mechanisms
• race conditions with other layers must be avoided
Protection types
• revertive/nonrevertive
• uni and bidi 1+1 for p2p
• uni 1+1 for p2mp
• bidi 1:n (including 1:1) for p2p
• uni 1:n for p2mp MPLS Slide 105
Control and Management
Every MPLS-TP network element must connect
(directly or indirectly) to an Operations System
When the connection is indirect, there must be a
Management Communication Channel
When there is a control plane, there is also a
Signaling Communication Channel
TP management plane functionality includes:
• configuration management (of system, CP, paths, OAM, APS)
• fault management (supervision, validation, alarm handling)
• performance management (characterization, measurement)
• security management
TP defines a control plane (but it is not mandatory to use)
• for setting up LSPs MPLS-TP uses
• RSVP-TE and extensions
• OSPF-TE (RFC 4203 and 5392) or ISIS-TE
• for setting up PWs MPLS-TP uses the PWE3 control protocol RFC4447
MPLS Slide 106
Identifiers
In order to configure, manage, and monitor network elements
they require unique identifiers
In IP networks, IP addresses serve as a unique identifiers
but MPLS-TP must function without IP
PWs set up by PWE3 control protocol have unique identifiers
RFC 4447 defines Attachment Individual Identifiers
In carrier networks network elements can be uniquely identified by
Country_Code:ICC:Node_ID
Country_Code is two upper case letters defined in ISO 3166-1
ICC is a string of one to six alphabetic/numeric characters
Node_ID is a unique 32-bit unsigned integer

For MPLS-TP any of these can be used

MPLS Slide 107

Generic Associated Channel
MPLS-TP must be able to forward
management and control plane messages
without an IP forwarding plane
MPLS-TP must be able to inject OAM messages
that fate-share with the user traffic
MPLS-TP needs to send status indications
MPLS-TP must support APS protocol messages
How are all these messages sent ?

MPLS Slide 108

 Associated channels
PWs have an Associated Channel (ACh)
in which one can place OAM (VCCV)
that will fate-share with user traffic
The ACh is defined in RFC 4385
and is based on use of the PWE3 Control Word
0001 VER RES=0        Channel Type
MPLS-TP also needs an ACh for its OAM
but MPLS LSPs do not have a CW!
Y.1711 defined a mechanism for MPLS (pre-TP) OAM
based on use of reserved label 14 and an OAM type code
The ITU wanted to use this mechanism for T-MPLS as well
but the IETF did something a little bit different

MPLS Slide 109

 GACh
RFC 5586 defines the Generic Associated Channel (GACh)
based on the Generic Associated channel Label (GAL)
For the simplest case :

MPLS label TC S TTL MPLS label stack

GAL label = 13 TC S TTL GAL
0001 0000 RESERVED Channel Type ACH header

Zero or more ACh TLVs

GACh message

MPLS Slide 110

 What can be carried in the GACh ?
Defined Channel Types (IANA registry) :
Value Description TLVs Reference
0x0000 Reserved
0x0001 MCC No RFC5718
0x0002 SCC No RFC5718
0x0007 BFD w/o IP header No RFC5885
0x0021 IPv4 packet No RFC4385
0x0057 IPv6 packet No RFC4385
0x0058 Fault OAM No RFC 6427
0x7FF8-0x7FFF Experimental Use RFC5586

The GACh can thus be used for:

1. OAM (FM/PM) – using BFD, Y.1731, … (see next chapter)
2. status signaling for static (non-LDP) PWs
3. management traffic (e.g., when no IP forwarding plane)
4. control traffic (e.g., when no IP forwarding plane)
5. other uses ? MPLS Slide 111
 Which OAM ?
So what OAM do we put into the GACh ?
There are two possibilities:
1. Bidirectional Forwarding Detection
BFD is a “hello” protocol originally between routers
before TP IETF standardized it for IP, MPLS, and PWs (in VCCV)
• RFC 5880 (draft-ietf-bfd-base)
• RFC 5881 (draft-ietf-bfd-v4v6-1hop)
• RFC 5882 (draft-ietf-bfd-generic)
• RFC 5883 (draft-ietf-bfd-multihop)
• RFC 5884 (draft-ietf-bfd-mpls)
2. Y.1731 (802.1ag)
Y.1731 is an ITU/IEEE OAM protocol for Ethernet OAM
end-end OAM with FM and PM (ITU-only) capabilities
proposed as an alternative to LSP-ping and BFD in VCCV
MPLS Slide 112
 BFD - review
Originally developed by Juniper and Cisco
to detect failures in the bidirectional path between routers
faster than via routing protocol hellos
thus reducing routing processing load as hello rates can be reduced
Light-weight liveliness protocol
control packets sent in both directions at negotiated rate
rate specified in msec
optional echo mode for two-way failure detection
runs in data plane like OAM, but unlike router hellos,
simple fixed-field encoding to facilitate HW implementation
no neighbor discovery (sessions triggered by routing protocol)
Since BFD can be the payload of any encapsulating protocol
so easily extended to new cases:
physical links, tunnels, LSPs, multihop routed paths, …
MPLS Slide 113
 BFD details
Modes
Async mode – each side periodically sends control packets
Demand mode – side does not send control packet unless polled
Echo mode – echo packet returned to sender

States
Down – just created or no connectivity
Init – during 3-way handshake (set-up or tear-down)
Up – connectivity
AdminDown – administratively down for indefinite period
does not imply lack of connectivity!

MPLS Slide 114

BFD format

format of echo packet need not be defined

BFD control packet (without optional Authentication) :

Vers Diag Sta|P|F|C|A|D|M Detect Mult Length

My Discriminator
Your Discriminator
Desired Min TX Interval
Required Min RX Interval
Required Min Echo RX Interval
MPLS Slide 115
 BFD control packet – explanations
Vers : version = 1
Diag : diagnostic code specifying the reason for the last state change
0 -- No Diagnostic 1 -- Control Detection Time Expired
2 -- Echo Function Failed 3 -- Neighbor Signaled Session Down
4 -- Forwarding Plane Reset 5 -- Path Down
6 -- Concatenated Path Down 7 -- Administratively Down
8 -- Reverse Concatenated Path Down 9-31 -- Reserved
Sta: current BFD session state as seen by the transmitting system
0 – AdminDown 1 -- Down 2 -- Init 3 -- Up
P: Poll. Sender requests verification of connectivity or of parameter change, expects an “F” packet in reply
F: Final Sender is responding to a received poll.
C: Control plane independent - sender BFD in data plane, continues to function even if control plane fails
A: Authentication present
D: Demand – sender wishes to operate in Demand mode, asks remote not to send control packets
M: Multipoint - for p2mp applications
Detect Mult : Detection time multiplier (e.g., 3). Number of Tx intervals for detection in async mode
Length : length of packet in bytes
My Discriminator : unique nonzero value used to demux BFD sessions between the same endpoints
Your Discriminator : discriminator received from the remote or zero if unknown
Desired Min TX Interval : minimum interval (msec) that can send
Required Min RX Interval : minimal interval (msec) that can receive
0 means do not send periodic control packets.
Required Min Echo RX Interval : minimum supported interval (msec) between received echo packets
if zero, echo mode is not supported.

MPLS Slide 116

 Encapsulations
single hop IP
UDP dest port = 3784 for control packets, 3785 for echo packets
UDP source port from dynamic range
TTL=255 (for security)
multihop IP
UDP dest port = 4784 for control packets, echo mode forbidden
UDP source port from dynamic range
TTL does not provide security
PW
PW label + any of the 3 VCCV CC types but always with the CW
4 CV types – (fault only or fault+status) * (with/without UDP/IP headers) – indicated in CW
only async mode, discriminator=0, capabilities signaled in PWE control protocol
MPLS
label stack of FEC being monitored
MPLS TTL set to expire
BFD triggered by LSP ping
UDP/IP BFD control packet inside MPLS
async mode only
bootstrapped with LSP ping echo request/reply messages
containing discriminators in TLV type 15
MPLS Slide 117
 Y.1731 – brief review
Developed by the ITU and IEEE as 802.1ag (CFM)
and supported by the MEF
Designed as a full multi-level carrier-grade OAM solution
Introduced new concepts, such as MEPs, MIPS, …
Supports CC, CV, AIS, LB, LT, placket loss, delay, PDV, …

Unfortunately, Y.1731 is tightly coupled with Ethernet

• EtherType identifies Y.1731 packet
• DAs identifies entities such as MEPs and MIPS
• MEL identifies level
not easy to drop Y.1731 PDUs into other protocols

MPLS Slide 118

 Y.1731 format

after DA, SA, optionally VLANs, comes Ethertype (8902)

and the following PDU

MEL VER OPCODE FLAGS TLV-OFF

(3b) (5b) (1B) (1B) (1B)

if there are sequence numbers/timestamp(s), they are next

then come TLVs (after offset), the “end TLV”, followed by the FCS
TLVs have 1B type and 2B length fields
there may or not be a value field
the “end-TLV” has type = 0 and no length or value fields

MPLS Slide 119

Y.1731 PDU types
opcode OAM Type DA
1 CCM M1 or U
3 LBM M1 or U
2 LBR U
5 LTM M2
4 LTR U
6-31 RES IEEE
32-63 unused RES ITU-T
33 AIS M1 or U
35 LCK M1or U
37 TST M1 or U
39 Linear APS M1or U
40 Ring APS M1or U
41 MCC M1 or U
43 LMM M1 or U
42 LMR U DA
45 1DM M1 or U
47 DMM M1 or U
46 DMR UA
49 EXM
48 EXR
50 VSR
51 VSM
52 CSF M1 or U
54 SLR U
55 SLM U
56 LLR
57 LLM
64-255 RES IEEE
MPLS Slide 120
and the winner is …
So, for MPLS-TP there are two options
1. BFD +  The IETF chose this route
extensible to new encapsulations
not a full OAM protocol
already runs on LSRs
and deployed in MPLS core networks
extend BFD (and LSP-ping) to become a full FM OAM protocol
and invent new protocols as needed
2. Y.1731  The ITU-T chose this route
full OAM protocol
not easily extensible to MPLS
already runs on switches
and deployed in carrier Ethernet networks
create a new encapsulation and reuse all functionality
MPLS Slide 121
 The IETF OAM - overview

All functionality runs over the GAL/GACh

• RFC 6428 leverages BFD for CC, CV and RDI

• RFC 6426 leverages LSP-ping for on demand CV
• RFC 6435 new lock instruct and loopback protocol
• RFC 6427 new fault (AIS, link-down) reporting protocol
• RFC 6374 new PM protocol
• RFC 6375 simplified subset of loss-delay protocol

Let’s see a few of these …

MPLS Slide 122

 The IETF CC and RDI message

from draft-ietf-mpls-tp-cc-cv-rdi

CC packet

GAL Label (13) TC S=1 TTL GAL

0001 VER 00000000 CC channel type GACh

BFD control packet BFD

RDI indicated in BFD control packet by

Diag=8 -- Reverse Concatenated Path Down

MPLS Slide 123

 The IETF CV message
from draft-ietf-mpls-tp-cc-cv-rdi
CV packet

GAL Label (13) TC S=1 TTL GAL

0001 VER 00000000 CV channel type GACh

BFD control packet BFD

Type= 1)segment 2)LSP 3) PW Length

MEP
Source ID
node identifier TLV

MPLS Slide 124

 The IETF on-demand CV message
from draft-ietf-mpls-tp-on-demand-cv
on-demand CV packet (several encaps possible)

GAL Label (13) TC S=1 TTL GAL

0001 VER 00000000 channel type GACh

RFC 4379 packet LSP-

ping
return path is in MPLS (no IP forwarding …)
three encapsulations
– LSP-ping UDP/IP packet in MPLS (RFC 4379 )
– LSP-ping packet in UDP/IP in GACh (channel type 0x21 or 0x57)
– “raw” LSP-ping packet in GACh (new channel type)
new TLVs are defined
MPLS Slide 125
 The IETF fault message
from draft-ietf-mpls-tp-fault
fault management packet

GAL Label (13) TC S=1 TTL GAL

0001 VER 00000000 FM channel type GACh

LR
Vers RES Msg Type Flags Refresh
Timer
FM
TLV Length TLVs message

L flag used for AIS R flag removes previous fault condition

TLVs indicate the nodes/interfaces and conditions
MPLS Slide 126
 The IETF loss and delay PM
RFC 6374 defines 4 new GACh types
Value Description TLVs Reference
0x000A Direct Loss Measurement (DLM) No RFC6374
0x000B Inferred Loss Measurement (ILM) No RFC6374
0x000C Delay Measurement (DM) No RFC6374
0x000D Inferred Loss and Delay Measurement (ILM+DM) No RFC6374

• the same packet format is used for query and response

a flag bit distinguishes between the two
• direct mode = use of counters for accurate loss measurement
• inferred mode = use of synthetic packets
• for loss measurement counters are carried in the OAM packets
• delay measurement timestamps may be
1588 format (default) or
NTP format
These messages are for MPLS in general
Profile for TP (where no ECMP, PHP, etc) is available
MPLS Slide 127
The ITU-T Y.1731-based OAM
Defined in G.8113.1 and draft-bhh-mpls-tp-oam
Y.1731 PDUs are placed after GAL
ACh channel type (not allocated by IANA) identifies PDUs

GAL Label (13) TC S=1 TTL GAL

0001 VER 00000000 allocated channel type GACh

MEL VER OPCODE FLAGS TLV-OFF

Y.1731
Y.1731 PDU with (ICC-based or IP-based) MEG ID

MPLS Slide 128

 MPLS-TP resilience
Since it strives to be a carrier-grade transport network
TP has strong protection switching requirements
APS has been almost as contentious issue as OAM
and indeed the arguments are inter-related
RFC 6372 gives a general framework
and differentiates between
– linear
– shared-mesh and
– ring
protection

MPLS Slide 129

 Linear protection – IETF style
from RFC 6378
• 1+1, 1:1, 1:n and uni/bidi are supported
• APS signaling protocol (for all modes except 1+1 uni)
is single-phase
and called the Protection State Coordination protocol
• PSC messages are sent over the protection channel
• APS messages are sent over the GACh with a single channel type
message functions identified by a request field
• 6 states: normal, protecting due to failure, admin protecting,
WTR, protection path unavailable, DNR
• when revertive, a WTR timer is used

MPLS Slide 130

 PSC message format
GAL Label (13) TC S=1 TTL GAL

0001 VER 00000000 PSC channel type GACh

Ver Request PT R Res FPath Path

TLV Length Res PSC

Optional TLVs

Request : NR, SF, SD, manual switch, forced switch, lockout, WTR, DNR
PT = Protection Type : uni 1+1, bidi 1+1, bidi 1:1/1:n
R = Revertive
FPath = which path has fault Path = which data path is on protection channel
MPLS Slide 131
 Linear protection – ITU style
from RFC 7347 (Pre-standard Linear Protection Switching)

Similar to previous, but uses Y.1731/G.8031 format

GAL Label (13) TC S=1 TTL GAL

0001 VER 00000000 allocated channel type GACh

MEL VER OPCODE=39 FLAGS=0 OFFSET=4

G.8031
req prot requested bridged
state type sig sig reserved

END=0

MPLS Slide 132

 Ring protection
RFC 6974
Between any 2 LSRs can define a Sub-Path Maintenance Entity
So between 2 LSRs on a ring there are 2 SPMEs –
we define 1 as the working channel and 1 as the protection channel
Now we re-use the linear protection mechanisms, including the PSC protocol
draft-helvoort-mpls-tp-ring-protection-switching
Both counter-rotating rings carry working and protection traffic
The bandwidth on each ring is divided
X BW is dedicated to working traffic and Y dedicated to protection traffic
The protection bandwidth of one ring is used to protect the other ring
Each node should have information about the sequence of ring nodes
MPLS-TP Ring Protection Switching is G.8032-like, but forwards non-NR msgs

MPLS Slide 133

Security

MPLS Slide 134

 MPLS security
RFC 5920 provides a security framework for MPLS networks
MPLS was designed for core networks
its security design is soft on the inside and hard on the outside
If an attacker gains physical access to an MPLS network node
(it is enough for there to be an open switch port)
he can inject fake MPLS packets with arbitrary labels
(there is high probability of guessing a valid label)
Once a packet is inside the network
there are no mechanisms to identify and block it
(no sanity checking, no header field to authenticate)
In addition there are issues with MPLS control protocols
At high rates this injection can overwhelm forwarding resources
MPLS Slide 135
 Source authentication
Ethernet and IP packets contain Source Addresses (SA)
but these can be readily forged
Message Authentication Codes (MACs)
can be used to authenticate a packet’s source
that is, to prove that the SA correctly indicates the source
Source authentication is used in IPsec and MACsec
along with integrity (anti-tampering)
and optionally confidentiality (encryption)
MPLS packets contain no source address
and thus there is no way to verify who sent them
This opens MPLS to injection and label swap attacks

MPLS Slide 136

 Control plane attacks
Some MPLS control protocols (e.g., RSVP-TE) are soft-state
and others (e.g., LDP) withdraw LSPs
when contact with a peer is lost
Intermittently deleting even a few consecutive keepalive packets
can thus cause a massive denial of service
More complex attacks can poison the LFIB
These attacks are not relevant for MPLS-TP w/o a control plane

MPLS Slide 137

 CE CE
ACs ACs
CE PE PE CE

CE CE

Pseudowires

MPLS Slide 138

 Communications services
There are many kinds of customer traffic
(voice, video, file-transfer, BE data, web browsing, etc.)
Historically, Service Providers built networks and sold services
optimized for a specific traffic type or types
Thus, we ended up with (too) many communications protocols
• IPv4, IPv6, Ethernet, MPLS, ATM, frame relay,
• CLNS, E1 , E3, T1, T3, SDH, OTN, CPRI, PPP,
• fiber channel, Controller Area Network, Profinet, …
However, SPs with one type of network infrastructure
want to fully exploit it to carry all types of network traffic

MPLS Slide 139

 Interworking
As transport using different network protocols
are often sold as different services (although the customer shouldn’t care)
one frequently needs to connect networks using different protocols

network network
A B

This connection is called interworking

The protocol converter goes by various names :
– interworking function (IWF)
– gateway (GW)
– edge device (e.g., PE)

MPLS Slide 140

 Service Interworking
One way to connect two different networks
is by full conversion of the Native Service formats
This is called service interworking

Native service Native

Service interworking Service
A function B

The service interworking IWF fully terminates the native services

NS A header(s) NS B header(s)

user data user data

MPLS Slide 141

 Tunneling
Network interworking (tunneling) is a simpler alternative
that is applicable when both end-points sit on the same NS
For example, if we wish to interconnect two Ethernet LANs
using an MPLS infrastructure network

native SP native
service service
network network network

Note that the native service protocol is not terminated

SP headers

NS headers NS headers NS headers

user data user data user data

MPLS Slide 142
 Emulation (wire service)
Since both ends of a tunnel are of the same native service
the tunnel seems to be transparent (a virtual wire)
In fact, the goal of network interworking (wire) service
is to emulate the native service
i.e., to be as transparent as possible
however, the emulation will not always be perfect !

customer customer
network leased line network

customer provider
customer
network network
network
tunnel
MPLS Slide 143
 Emulation Edge to Edge

customer customer
network native service link or path network

end to end service

edge to edge

provider
NS link network
customer NS link
customer
network network

emulated link

provider network edge provider network edge

MPLS Slide 144
 Pseudowire
a Packet Switched Network (PSN) is
– a network that forwards packets
– e.g., IPv4, IPv6, MPLS, Ethernet*
a pseudowire (PW) is a mechanism to tunnel through a PSN
PWs have been defined for :
• MPLS
• L2TPv3 over IP
TDM PWs have also been defined for :
• Ethernet (only TDM PWs)
• UDP/IP (only TDM PWs)

PWs enable exploitation of a single converged network

PWs are bidirectional (unlike MPLS LSPs)
PW architecture is an extension of VPN architecture

MPLS Slide 145

 Provider Network Architecture
a provider network is composed of:
•Provider routers or switches (P routers)
•Provider Edge routers or switches (PE routers)

P P
router router

Attachment P PE Attachment
PE
router router Circuit
Circuit router

P
router

MPLS Slide 146

 VPN architecture

C C C
C CE
CE
C C

customer 1 network P P P customer 2 network

PE PE
P P

C C C
C provider network CE
CE
C Key C
C Customer router
customer 2 network CE Customer Edge router customer 1 network
P Provider router
PE Provider Edge router
MPLS Slide 147
 Pseudowire Emulation
Edge to Edge
PWE3
Customer
Edge
provider’s
(CE) PSN Customer
Edge
Customer
Edge Provider Provider (CE)
Edge Edge
(CE)
(PE) (PE) Customer
Customer Edge
native
Edge service
native Pseudowires AC (CE)
(CE) service
AC (PWs)

MPLS Slide 148

 Some PWE3 ideas
• Edge-to-edge emulation and maintenance of PWs
– tunnel creation and placement out of scope
– PSN is responsible for differentiation between PWs
e.g., in MPLS there is a PW label, in UDP/IP a PW port number
• Network interworking, not service interworking
• Native service emulation need not be perfect
e.g., timing of a recovered TDM PW may not be the same as the timing of a native TDM circuit
– imperfection documented in applicability statements
• Must not exert controls on underlying PSN
– but diffserv, RSVP-TE can be used
• Must not redefine native service functionalities
– use Native Service Processing functions
• The PWE3 encapsulation consists of a 4-byte Control Word
– initially each native service encap defined a different CW
– RAD once had its own TDMoIP CW
– perhaps my most important contribution to PWE3 was the single CW format
• Like MPLS packets, PW packets are not self-describing
MPLS Slide 149
 MPLS PWs
Basic idea behind MPLS PWs :
LSRs forward based on ToS label
LSR don’t look inside the packet (e.g., for an IP address)
So we can carry anything inside the MPLS packet – not only IP !
The IETF PWE3 WG focused on MPLS (and the L2TPext WG on L2TPv3)
Why ?
• Emulated services often have QoS and TE requirements
– IP is basically a “best effort” service (RSVP extensions not prevalent)
– MPLS can provide TE guarantees (via RSVP-TE)
• IP provides no standard “bundle” multiplexing method
– UDP/TCP ports provide application multiplexing
– RTP uses ports in a nonstandard way
– L2TP includes a multiplexing mechanism
– MPLS label stack provides natural multiplexing method
We won’t delve into L2TPv3 PWs in this mini-course
MPLS Slide 150
 Simplistic MPLS solution
CE
CE
ACs ACs
CE PE P P PE CE

CE CE

Each customer network mapped to pair of (unidirectional) LSPs

Each native packet/frame encapsulated with MPLS label
Scaling problem:
• requires huge number of LSPs in provider network
overburdening P-router LFIBs
• P-routers need to be aware of customer network details

MPLS Slide 151

 (Martini) Pseudowires
CE CE
ACs transport tunnel ACs
CE PE PE CE

CE CE

MPLS (tunnel) label

Transport MPLS tunnel set up between PEs PW (inner) label
Multiple PWs may be set up inside tunnel payload
PW label is always Bottom of Stack (S=1)
Native packet/frame encapsulated with 2 labels
P-routers are completely unaware of individual PWs

inner label outer label(s) MPLS-f

Translations: interworking label transport label(s) ITU-T
PW label tunnel label(s) IETF
MPLS Slide 152
 Martini PWs (cont.)

P P
router router
PE PE
P
native router router native
router
service service
AC tunnel AC
P
containing router
many PWs

P routers use the tunnel label to forward the PW packet to the egress PE
The inner label is never used as an MPLS label
• no forwarding decisions based on it
• only used by PE to connect to the correct native service port
But this changes in MultiSegment PWs
MPLS Slide 153
 Generic PWE packet format
PSN / multiplexing

optional RTP header

optional control word (CW)

higher layers

PW payload

We call everything above the PW payload - the encap

MPLS Slide 154

 Example formats
MPLS PSN

tunnel PW control
Payload
label(s) label word

L2TPv3 PSN

IP header (5*4 B)

session ID (4 B)

optional cookie (4 or 8 B)
control word (4 B)

Payload

MPLS Slide 155

 Standard PWE Control Word
000x flags FRG  Length            Sequence Number  
000x
– 0000 for PW user traffic
– 0001 for PW Associated Channel (ACh), e.g., for VCCV OAM
Flags (4 b)
– only a few native service encaps define flags
– used to transport native service fault indications
FRG
– some native service encaps use to indicate payload fragmentation
• 00 = unfragmented 01 = 1st fragment
• 10 = last fragment 11 = intermediate fragment
Length (6 b)
– used when packet may be padded by lower layers
Sequence Number (16 b)
– may be used to detect packet loss / misordering
MPLS Slide 156
 The first nibble
Most native service encaps
– do not define CW flags
– do not use FRG or Length
– do not mandate use of the SN
So what is the CW for ?
The important part of the 4 bytes is the first nibble !
In RAD’s original CW, there was a PID here
In IPv4 over MPLS the first nibble after the BoS is 0100
In IPv6 over MPLS the first nibble after the BoS is 0110

In PW packets the first nibble after the BoS is 0000 or 0001

thus identifying the MPLS packet as carrying PW traffic (not IP)
ECMP mechanisms rudely recurse down the label stack
and peek at the first nibble to differentiate between IP and PW
MPLS Slide 157
 P P P P
T-PE S-PE T-PE
P P P P P P

Some PW-specific mechanisms

MPLS Slide 158

Associated Channel and VCCV
PWs have an Associated Channel for OAM, etc.
Main use is for VC Connectivity Verification - VCCV
VC is an old name for PW
CV is an incorrect name for Continuity Check
VCCV runs inside PW (same PW label) so that it fate shares
Different Control Word format
0001 VER RES=0       Channel Type (0x07-raw 0x21-IPv4 0x57-IPv6)

Inside VCCV several different OAM mechanisms may be used:

– ICMP
– LSP ping (RFC 4379)
– BFD (one-way or echo)

The ACh has been extended for MPLS-TP (to become the GACh)

MPLS Slide 159

 VCCV Channel Types
In RAD’s original proposal, there was an OAM PW (with a special label)
that monitored all PWs in the same tunnel
The PW ACh fate-shares with PW traffic
so how do we recognize a VCCV packet ?
RFC 5085 defines 4 Control Channel types
Type 1 in-band VCCV (only when there is a CW)
CW first nibble 0001
Type 2 out-of-band VCCV
MPLS Router Alert Label (label=1) above the PW label
Type 3 TTL expiry (TTL=1 at destination PE)
Type 4 new VCCV channel type
Generic ACh Label (label=13) under the PW label
MPLS Slide 160
 MS-PWs

P P P P
T-PE S-PE T-PE
P P P P P P

Single-Segment PW (SS-PW) requires the PEs to see each other

With multiple PSN domains this may not be the case
MultiSegment PW (MS-PW)
Terminal-PEs interconnect via stitching-PE
PW label becomes a true MPLS label (switching, swapping)
When there is more than one S-PE
need to ensure that the 2 LSPs traverse the same one

MPLS Slide 161

 FAT PWs
IP ECMP functions by hashing specific key fields in the IP header
MPLS ECMP hashes on the label stack
but since the BoS is the closest to the individual flows
it gives the finest granularity
There is a proposal for allowing a special entropy label in MPLS
The Flow Aware Transport PW mechanism adds a flow label
below the PW label (NB the PW label is no longer BoS)
label random (high entropy) but not a reserved label
true flows must be consistently mapped to the same flow label
TTL=1 so that it is never accidentally used for label switching
Unlike RAD’s proposal for PW bonding
there is only a single PW
but standard MPLS ECMP causes load balancing
MPLS Slide 162
 PW redundancy
Since PWs are used by SPs for transport
Automatic Protection Switching may be needed
For SS-PWs, the MPLS network protects the tunnels (e.g., FRR)
But how do we protect against PE or AC failure ?
For MS-PWs how do we protect against S-PE failure ?

The basic idea is

• dual homing a CE to >1 PE
• and setting up primary and secondary PWs (control protocol extensions)
• monitoring PWs using VCCV
• triggering protection switch using (new bits in) PW status messages

MPLS Slide 163

PWE Control Protocol

MPLS Slide 164

 PW set-up
PWs can be set up
• statically
• via the PWE3 control protocol (RFC 4447) based on LDP
• via BGP (for L3VPN or L2VPN)
No matter how set up
• need to place the PW into an MPLS tunnel
• A PW is a bidirectional entity (two LSPs in opposite directions)
so we need to associate the two directions

PE PE

MPLS Slide 165

 PWE3 (Martini) control protocol
PWE control protocol (RFC 4447) is used to set up / configure PWs
It is used only by PW end-points (PEs in standard model)
intermediate nodes (e.g. P routers) don’t participate or see it

The PWE3 control protocol is based on LDP

– PWs do not support QoS
– targeted LDP is used to communicate with opposite end-point
– 2 new FECs for PWs
– new TLVs added for PW-specific functionality
– associates two labels with PW

P P
PE PE
P P P
MPLS Slide 166
 LDP extensions
RFC 4447 does not define any new LDP messages
It defines 2 new FEC types (see next page)
• PWid FEC (128)
• Generalized ID FEC (129)

It defines 3 new TLVs

• Notification PW Interface Parameters (in FEC element)
• FEC PW Grouping ID (in FEC element)
• PW Status (in Notification message)
– Status TLV contains more information than Label Withdraw message

MPLS Slide 167

 PW FEC types
A PW is a bidirectional entity (two LSPs in opposite directions)
so we need to associate the two directions
2 new LDP FEC types are defined
LDP previously defined 3 FEC types : wildcard(1) prefix(2) host address(3)
– PWid FEC (128)
– Generalized ID FEC (129)
FEC 128
– both end-points of PW must be provisioned with a unique (32b) value
WARNING: no relationship between this value and label
– each PW end-point independently initiates LSP set up
– LSPs associated to form the PW

FEC 129
– used when autodiscovering PW end-points
– each end-point has Attachment Identifier (see next page)

MPLS Slide 168

 Generalized PW ID
Each forwarder (PW AC) has a PE-unique Attachment Identifier (AI)
<PE, AI> must be globally unique
It is frequently useful to group a set of forwarders into a attachment group
where PWs may only be set up among members of a group
then Attachment Identifier (AI) consists of
– Attachment Group Identifier (AGI) (which is basically a VPN-id)
– Attachment Individual Identifier (AII)
the LSPs making up the (two directions of the) PW are
< PE1, (AGI, AII1), PE2, (AGI, AII2) > and
< PE2, (AGI, AII2), PE1, (AGI, AII1) >

we also need to define

– Source Attachment Identifier (SAI = AGI+SAII)
– Target Attachment Identifier (TAI = AGI+TAII)
receiving PE can map TAI uniquely to AC

MPLS Slide 169

headers TDM payload

TDM PWs

MPLS Slide 170

 What is TDM ?
By TDM we mean synchronous transport at
• one of the PDH (G.702) rates
• one of the SDH (G.707) rates
TDM networks can themselves carry
• telephony traffic
• data traffic
• video
Native TDM networks
• circuit switching ensures signal integrity
• very high reliability (“five nines”)
• low delay and no noticeable echo for telephony 
• timing information transported over the network
• mature signaling protocols (over 3000 features)

MPLS Slide 171

 PDH rates
level

0 64 kbps

* 30 * 24
* 24
1 E1 2.048 Mbps T1 1.544 Mbps J1 1.544 Mbps

*4 *4 *4

2 E2 8.448 Mbps T2 6.312 Mbps J2 6.312 Mbps

*4 *7 *5

3 E3 34.368 Mbps T3 44.736 Mbps J3 32.064 Mbps

* 4 *6 *3

4 E4 139.264 Mbps T4 274.176 Mbps J4 97.728 Mbps

CEPT N.A. Japan

MPLS Slide 172
 TDM Structure
handling of TDM depends on its structure
unstructured TDM (TDM = arbitrary stream of bits)
…
structured TDM
framed (8000 frames per second)
S S S
Y Y Y
N N N
C C C
channelized (single byte timeslots)

SYNC TS1 TS2 TS3 … signaling

bits
… TSn
(1 byte)
multiframed

frame frame frame … frame

multiframe
MPLS Slide 173
 TDM transport types

headers TDM payload

Structure-agnostic transport (SAToP – RFC4553)

• for unstructured TDM
• even if there is structure, we ignore it
• simplest way of making payload
• OK if network is well-engineered
Structure-aware transport (TDMoIP, CESoPSN)
• take TDM structure into account
• must decide which level of structure (frame, multiframe, …)
• can overcome PSN impairments (PDV, packet loss, etc)

MPLS Slide 174

Structure aware encapsulations

Structure-locked encapsulation (CESoPSN)

headers TDM structure TDM structure TDM structure TDM structure

Structure-indicated encapsulation (TDMoIP – AAL1 mode)

headers AAL1 subframe AAL1 subframe AAL1 subframe AAL1 subframe

Structure-reassembled encapsulation (TDMoIP – AAL2 mode)

headers AAL2 minicell AAL2 minicell AAL2 minicell AAL2 minicell

MPLS Slide 175

 TDM PW layering structure

PSN / multiplexing

Optional RTP header

PWE3 Control Word

higher layers

SAToP CESoPSN AAL1 AAL2 HDLC

AAL1/CESoPSN used for preconfigured setup

AAL2 used for dynamic bandwidth
HDLC used for CCS signaling

MPLS Slide 176

 TDMoIP Control Word
0000 flags 0 0   Length            Sequence Number  

0 0 0 0 / FORMID (4 b)
– was used to indicate TDMoIP mode (AAL1, AAL1 - CAS, AAL2, HDLC)
– ensures differentiation between IP and MPLS PSNs
Flags (4 b)
– L bit (Local failure)
– R bit (Remote failure)
– M field (2 b)
Length (6 b) used when packet may be padded by lower layer
Sequence Number (16 b) used to detect packet loss / misordering

MPLS Slide 177

 Optional explicit timing
VoIP uses RTP (Real-Time Protocol)
RTP can be used to transport timing across IP networks
It does this by providing:
• a 16 bit sequence number
• a 32 bit timestamp
at the expense of 12 additional overhead bytes per packet
Accurate timing is important in telephony
and IP networks add packet delay variation (PDV)
For TDM PWs, only the timestamp is needed (SN is in CW)
– encodes time of sending using clock N*8kHz
– Note
• this is NOT the normal RTP clock (number of samples)
• rather with respect to a common clock
MPLS Slide 178
 CESoPSN mode

PSN CW T1/E1 frame … T1/E1 frame

Can efficiently handle fractional T1/E1

FRG field in CW enables support of multiframe
 CAS signaling uses a superframe (16/24 frames)
 Superframe/multiframe integrity must be respected

Octet aligned mode for T1 (24 bytes plus one bit per frame)

MPLS Slide 179

 TDMoIP AAL1 mode
Packet loss, misorder, PDV problems can be solved by:
• adding a packet sequence number
• adding a pointer to the next multi-frame boundary
• only sending timeslots in use
• allowing multiple frames per packet

UDP/IP seqnum ptr T1/E1 frames (only timeslots in use)

(with CRC)
for example 7 @ TS1 TS2 TS5 TS7 TS1 TS2 TS5 TS7

Good idea! This is AAL1 !

using precisely AAL1 enabled service interworking with ATM networks
MPLS Slide 180
 TDMoIP AAL2 mode

TDM frame TDM frame TDM frame TDM frame TDM frame
1 1 1 2 2 2 3 3 3 4 4 4 5 5 5

PSN hdrs CW hdr 1 2 3 4 5 hdr 1 2 3 4 5 hdr 1 2 3 4 5

TS1 TS2 TS3

• Each minicell consists of a header and buffered data

• Minicell header contains:
– CID (Channel IDentifier)
– LI (Length Indicator) = length-1
– UUI (User-User Indication) counter + payload type ID
using precisely AAL2 enabled service interworking with ATM networks
MPLS Slide 181
 CEP
Circuit Emulation over Packet is an SONET/SDH PW
Encapsulate Synchronous Payload Envelope fragment
Structure Pointer in CW points to J1 byte in STM frame

PSN layers
Optional RTP header
CEP Control Word

CEP CW
E1 flags4 structure ptr13   Suppressable Sequence Number 14 

MPLS Slide 182

Ethernet PWs

MPLS Slide 183

 Traditional WAN architecture
Ethernet layer is terminated
• only higher layer (e.g., IP) is transported
• the traffic is no longer Ethernet at all
Ethernet Ethernet

WAN

not Ethernet
Ethernet header
• removed at ingress, and
• new header added at egress
This is not transparent Ethernet LAN interconnect
• Ethernet LANs with many higher layer packet types
can’t be interconnected
• raw L2 Ethernet frames can not be sent
MPLS Slide 184
 Tunneling Ethernet frames

Users with multiple Ethernet sites may want to connect their LANs
so that all locations appear to be on the same LAN
This requires tunneling of all Ethernet L2 frames (not only IP)
between one LAN and another
The entire Ethernet frame needs to be preserved
(except perhaps the FCS, which may be regenerated at egress)

Ethernet Ethernet
X

Ethernet inside X

MPLS Slide 185

 Ethernet PW (RFC 4448)
While PWs were designed for legacy traffic
Ethernet PWs are now the most popular type
This is because native Ethernet transport is limited in range
RFC 4448 specifies:
• Control word is optional
even if control word is used, sequence number if optional
• Standard mode – FCS is stripped and regenerated
FCS retention mode (RFC 4720) allows retaining FCS
• Can transport tagged or untagged Ethernet frames
if tagged :
encapsulation can be raw mode or tagged mode
tagged mode processes (inserts/swaps/removes) service delimiting tags

MPLS Slide 186

 Ethernet Pseudowire packet

tunnel PW control
single Ethernet Frame
label label word

Ethernet Frame usually has FCS stripped

SP tag may also be stripped

optional control word

generation and processing of sequence number is optional

0000 reserved          Sequence Number (16b) 

MPLS Slide 187

 VPWS

AC AC
CE PE PE CE

provider
network

Based on Ethernet PWs one can provide a

Virtual Private Wire Service or a
Virtual Private LAN Service (L2VPN)
VPWS emulates a wire supporting the Ethernet physical layer
• set up MPLS tunnel between PEs
• set up Ethernet PW inside tunnel
CEs appear to be connected by a single L2 circuit
(can also make VPWS for ATM, FR, etc.)
MPLS Slide 188
 VPLS
AC
PE CE

AC
CE PE

for clarity only one VPN is shown

PE AC CE

VPLS emulates a LAN over an MPLS network

• set up MPLS tunnel between every pair of PEs (full mesh)
• set up Ethernet PW inside tunnels, for each VPN instance
CEs appear to be connected by a single LAN

What is the difference between this and a L3 VPN service ?

MPLS Slide 189
 L2VPN vs. L3VPN

PE CE

CE PE
? PE
L2VPN: MPLS network → giant switch
L3VPN: MPLS network → giant router

CE

in L2VPN CEs appear to be connected by single Ethernet network

PEs are transparent to L3 routing protocols
CEs are routing peers
in L3VPN CE routers appear to be connected by a single IP network
CE is routing peer of PE, not remote CE
PE maintains routing table for each VPN

MPLS Slide 190

 CE CE
ACs ACs
CE PE PE CE

CE CE

Other PW types

MPLS Slide 191

 What else is there ?
For what native service types have PW encaps been defined ?

• TDM (SONET/SDH, E1, T1, E3, T3)

• Ethernet
• ATM (port mode, cell mode, AAL5-specific modes)
• Frame Relay
• HDLC / PPP
• IP PW (in 4447, never became a normative RFC)
• Fiber channel
• Generic packet PW

MPLS Slide 192

 ATM PWs (RFC 4717)
• N:1 (Martini mode)
– control word optional
– remove HEC to form 52-byte cells
– pack 1 or more cells into MPLS packet (may mix VPI/VCI)
• 1:1 (ATM forum mode)
– special 3-byte control word required
– 1-byte header for each cell
– pack 1 or more cells into MPLS packet
– may mix VCI for given VPI (but then need to insert VCI)
• SDU (for AAL5 only)
– control word with 4 flags required (SN optional)
– payload is the complete SDU (no trailer)
• PDU (for AAL5 only)
– special 3-byte control word required
– 1-byte header after CW
– payload is N*48-byte PDU
• port mode (RFC 4816) - format same as N:1
MPLS Slide 193
 Frame Relay PWs (RFC 4619)
2 encapsulations :
• port mode (many-to-one mapping) 1 PW per all FR VCs
identical to HDLC PW encapsulation RFC4618
• 1:1 mode – one PW for each FR VC

Mandatory (for 1:1) CW contains 4 flags

• F FECN (Forward Explicit Congestion Notification)
• B BECN (Backward Explicit Congestion Notification)
• D DE bit (Discard Eligibility)
• C C/R (Command/Response)
SN optional and FRG extensively used

MPLS Slide 194

 Packet PW (RFC 6658)
Generic packet service to carry
any packets exchanged between adjacent LSRs, such as
• IP packets (user, system)
• Ethernet packets (e.g., IS-IS, LLDP, Ethernet OAM)
To mux different packet types we use raw Ethernet PW
although there is no real Ethernet interface
MAY use local MAC addresses
or MAY use (IANA allocated) MAC addresses PacketPWEthA / PacketPWEthB

MPLS Slide 195

 Fiber Channel PW (RFC 6307)
FC is a high-speed communications link used for SANs
Fibre Channel Over TCP/IP (FCIP) (RFC3821) needs TCP
to retransmit dropped frames and ensure order
MPLS networks may have very low loss and misorder rates
FC PW transparently transports FC traffic over MPLS
with simpler processing than FCIP
However, FC PWs have more complex NSP than other PWs

MPLS Slide 196

 DetNet PW
Recently, PWs have been proposed as a mechanism
for Deterministic Networking
For example, for high reliability MPLS-based DetNets
employ a replication and elimination function
that requires a sequence number per packet
This can be supplied by a PW-style control word
(Details still in being worked out)

MPLS Slide 197