H Scheduling
H Scheduling
Hierarchical Scheduling
LY
N
O
SE
U
AL
N
R
TE
IN
Hierarchical Scheduling
LY
N
O
SE
U
AL
Hierarchical scheduling, like port-based scheduling, occurs toward the end of the class of service
(CoS) process, on the outbound side. Once packets have been sent through the fabric to the egress
R
interface, the device puts the traffic into queues where it can be scheduled and shaped. Congestion
control is performed by a random early detection, or RED, algorithm. With hierarchical scheduling,
TE
the device can provide CoS treatment for multiple subscribers, groups of subscribers, and classes of
traffic within a subscriber at the same time.
Note
IN
2 www.juniper.net
Hierarchical Scheduling
LY
N
O
SE
U
AL
Like port-based scheduling, the primary function of hierarchical scheduling (also referred to as
hierarchical class of service, or H-CoS) is to define the parameters for how queues treat traffic.
R
Scheduling determines the order in which packets are transmitted, the rate at which packets are
transmitted, the number of packets the system can buffer, and differential treatment of packets in
TE
similar to port-based CoS, providing preference, prioritization, and congestion control, although the
specific capabilities at each level vary.
H-CoS can help to centralize CoS enforcement and application, managing traffic flows for
downstream devices that lack CoS significant functionality. H-CoS can also help reduce configuration
and management overhead, because it can reduce the need to design and configure complex CoS
features on downstream devices.
www.juniper.net 3
Hierarchical Scheduling
LY
N
O
SE
U
AL
Networks MX Series 3D Universal Edge Routers, enhanced queuing dense port concentrators
(EQ-DPCs) and modular port concentrators (EQ-MPCs) provide H-CoS functionality. On Juniper
TE
Networks M Series Multiservice Edge Routers and Juniper Networks T Series Core Routers, a variety
of queuing-related physical interface cards (PICs) provide enhanced queuing functionality.
EQ-DPC as a Baseline
IN
H-CoS functionality varies slightly across different hardware. For purposes of simplicity and
completeness this material is based on the EQ-DPC, which currently provides the most complete
H-CoS feature set. The EQ-MPC and Enhanced Intelligent Queuing 2 (IQ2E) PIC have very similar
functionality, with slight variations, while the other PICs have varied subsets of H-CoS features
.
Note
For detailed information on the differences in
CoS functionality across hardware components,
check the Junos Class of Service Configuration
Guide.
4 www.juniper.net
Hierarchical Scheduling
LY
N
O
SE
U
AL
Terminology: Part 1
N
A customer VLAN (C-VLAN), defined by IEEE 802.1ad, is a stacked VLAN that contains an outer tag
corresponding to the service VLAN, and an inner tag corresponding to the C-VLAN. A C-VLAN often
R
corresponds to customer premises equipment (CPE). Scheduling and shaping is often used on a
C-VLAN to establish minimum and maximum bandwidth limits for a customer.
TE
A service VLAN (S-VLAN), defined by IEEE 802.1ad, is the outer tag of a stacked VLAN. It often
corresponds to a network aggregation device such as a digital subscriber line access multiplexer
(DSLAM). Scheduling and shaping is often established for an S-VLAN to provide CoS for downstream
devices with little buffering and simple schedulers.
IN
www.juniper.net 5
Hierarchical Scheduling
LY
N
O
SE
U
AL
Terminology: Part 2
N
A traffic control profile is a configuration template used to set CoS parameters on a scheduler. It
provides a flexible and simple mechanism to define scheduling properties of ports, interface sets,
R
and logical interfaces (VLANs). Once defined, you can assign a traffic control profile to different
entities that have the same CoS profile.
TE
An interface set is a user-defined group of logical interfaces (VLANs) within a port. Interface sets
establish the set and reference a traffic control profile.
The committed information rate (CIR) is a configured guaranteed rate assigned to an interface set or
logical interface (VLAN). Physical interfaces implicitly have a CIR.
IN
The peak information rate (PIR) is a configured maximum rate for a port, interface set, or logical
interface (VLAN). The PIR is also referred to as the shaping rate.
6 www.juniper.net
Hierarchical Scheduling
LY
N
O
SE
U
AL
Hierarchical scheduling provides up to four levels of CoS treatment, as shown on the slide, on a
per-port basis. H-CoS supports up to eight queues per logical interface (VLAN), with each queue
R
www.juniper.net 7
Hierarchical Scheduling
LY
N
O
SE
U
AL
The slide shows the various levels of CoS that are available with H-CoS.
R
Each port can have one or more interface sets. Ports support shaping, to a reduce bandwidth lower
than its line-rate.
TE
Each interface set can have one or more logical interfaces (VLANs). Scheduling among the interface
sets of a port is based on a configured CIR and PIR.
Each VLAN has up to 8 queues. Scheduling among VLANs is based on the CIR and the PIR. The
IN
queues provide scheduling of traffic flows, with transmission based on priority, tunable buffers to
provide the ability to absorb bursts or control latency, as well as weighted random early detection
(WRED) to provide congestion control and avoidance.
8 www.juniper.net
Hierarchical Scheduling
LY
N
O
SE
U
AL
The Junos OS supports two hierarchical scheduling modes, as shown on the slide. These modes are
illustrated on the following pages.
R
TE
IN
www.juniper.net 9
Hierarchical Scheduling
LY
N
O
SE
U
AL
Per-unit scheduler mode is illustrated on the slide. The key difference between this mode and
hierarchical scheduler mode is the lack of interface sets.
R
TE
IN
10 www.juniper.net
Hierarchical Scheduling
LY
N
O
SE
U
AL
• Interface sets;
• Shaping per VLAN, and up to 8 queues per VLAN;
• Sharing of schedulers and queues among VLANs; and
IN
www.juniper.net 11
Hierarchical Scheduling
LY
N
O
SE
U
AL
To enable scheduler mode, add the appropriate statement (shown on the slide) at the physical
interface level of the configuration hierarchy. You can configure scheduler modes on a per-port basis.
R
TE
IN
12 www.juniper.net
Hierarchical Scheduling
LY
N
O
SE
U
AL
Level 1—Shaping
N
Level 1 includes the physical interface, or ports. Ports support shaping, which is the maximum rate
for the interface. A shaper permits a port to transmit traffic until the configured PIR, or shaping rate,
R
is reached. Once a port has exceeded its PIR, it is not allowed to transmit packets until traffic falls
back below the PIR. Packets sent to a port that has exceeded its shaping rate are not dropped, but
TE
www.juniper.net 13
Hierarchical Scheduling
LY
N
O
SE
U
AL
To configure CoS for a port within an H-CoS context, use a traffic control profile to set the PIR. Then
apply the profile to a port under the class-of-service stanza using the
R
output-traffic-control-profile statement.
TE
IN
14 www.juniper.net
Hierarchical Scheduling
LY
N
O
SE
U
AL
Level 2 includes interface sets. Two ways exist to combine logical interfaces or VLANs to form
interface sets: select individual logical interfaces (VLANs), or identify a common group of VLANs by
R
Usage Notes
• You cannot use interface ranges within an interface set configuration.
• You cannot specify an interface set mixing logical interface and S-VLAN tags.
IN
• Members of an interface set cannot span multiple physical interfaces. Only one physical
interface is allowed to appear in an interface set.
• A logical interface or S-VLAN tag can only belong to one interface set.
www.juniper.net 15
Hierarchical Scheduling
LY
N
O
SE
U
AL
Level 2—Shaping
N
The CIR and PIR provide guaranteed and maximum rates for the interface set.
R
Scheduling
Scheduling at the interface set level is based on CIR and PIR configured for the interface set. The
TE
system uses these values to determine the relative weight of an interface set with respect to the
other interface sets belonging to the same port.
The CIR and PIR determine bandwidth sharing among the interface sets within a port as follows:
IN
• If any interface set within a port has CIR defined, bandwidth sharing among the
interface sets is based on the CIR of the interface sets. This method of sharing is
referred to as CIR mode.
• If none of the interface sets within a port has CIR defined, bandwidth sharing is based
on the PIR of the interface sets. This method of sharing is referred to as PIR mode.
The PIR is the maximum bandwidth of the interface set in both modes. Once an interface set has
reached its PIR, it is not picked for transmission.
16 www.juniper.net
Hierarchical Scheduling
LY
N
O
SE
U
AL
To configure CoS for an interface set, use a traffic control profile to set the CIR and PIR. Then apply
the profile to an interface set under the class-of-service stanza using the
R
output-traffic-control-profile statement.
TE
IN
www.juniper.net 17
Hierarchical Scheduling
LY
N
O
SE
U
AL
Level 3—Shaping
N
Level 3 includes logical interfaces, or VLANs. The CIR and PIR regulate the bandwidth allocated to a
logical interface (VLAN). Because hierarchical scheduling provides a set of queues for each VLAN, a
R
Scheduling
Scheduling at the logical interface (VLAN) level is based on CIR and PIR. As with interface sets, the
system uses these values to determine the VLAN’s relative weight with respect to the other VLANs on
the same port.
IN
Bandwidth sharing among VLANs within a port is determined in a similar manner as interface sets:
• If any VLAN under a port is configured with CIR, bandwidth is shared among the VLANs
in proportion to their CIR. This method of sharing is referred to as CIR mode.
• If none of the VLANs under a port is configured with CIR, bandwidth is shared in
proportion to their PIR. This method of sharing is referred to as PIR mode.
The PIR is the maximum bandwidth of the VLAN in both modes. Once an interface set has reached
its PIR, it is not picked for transmission.
18 www.juniper.net
Hierarchical Scheduling
LY
N
O
SE
U
AL
To configure CoS for a logical interface or VLAN, use a traffic control profile to set the CIR and PIR.
Within the profile, specify a scheduler map to associate the VLAN to its queues, and then apply the
R
Note
If you do not include the guaranteed-rate
statement, the logical interface receives a
IN
www.juniper.net 19
Hierarchical Scheduling
LY
N
O
SE
U
AL
Level 4—Scheduling
N
Level 4 includes the actual queues that schedule traffic flows. The schedulers at this level are the
same ones used with port-level scheduling. They support a wide range of configuration options and
R
scheduling mechanisms to provide fine-grained control over the scheduling, buffering, and
congestion control and avoidance characteristics for a VLAN.
TE
Because H-CoS involves the interaction of multiple levels of CoS treatment, some scheduler
functionality is slightly different than when used for port-level scheduling. These variations are
discussed later in this material.
IN
20 www.juniper.net
Hierarchical Scheduling
LY
N
O
SE
U
AL
To configure queue scheduling, configure one or more schedulers and drop profiles. Include the
transmission rate, queue priority, delay buffers, drop profiles, and drop profile maps, and then
R
configure a scheduler map to associate the schedulers to the desired forwarding classes on the
device.
TE
With port-level scheduling, you applied the scheduler map to an interface. With hierarchical
scheduling, you apply the scheduler map to a logical interface or VLAN. Note that this step was done
earlier, within the configuration of logical interfaces and VLANs (level 3), by referencing the scheduler
map from within a traffic control profile, and then applying the profile to an interface under the
IN
class-of-service stanza.
www.juniper.net 21
Hierarchical Scheduling
LY
N
O
SE
U
AL
In the example on the slide, a port is configured with interface sets, VLANs, and queues as follows:
R
• Each VLAN has three queues with transmission rates shown on the slide.
This example is designed to illustrate the relevance and application of CIR and PIR by showing how
the values at one level can affect the values at other levels.
22 www.juniper.net
Hierarchical Scheduling
LY
N
O
SE
U
AL
Using the scenario on the previous page, the slide describes the ratios used to share bandwidth
among interface sets and VLANs.
R
TE
IN
www.juniper.net 23
Hierarchical Scheduling
LY
N
O
SE
U
AL
Throughput Results
N
The slide further describes how bandwidth is used, and how excess bandwidth shared among
interface sets and VLANs. The ratios are displayed as CIR values.
R
TE
IN
24 www.juniper.net
Hierarchical Scheduling
LY
N
O
SE
U
AL
Throughput Calculations
N
The bullets on the slide display the full calculation process for queue zero on VLAN 10. You can apply
the same methodology to calculate effective throughput for other VLANs and their queues. The table
R
provides a complete list of effective throughput for the port, interface sets, VLANs, and queues.
TE
IN
www.juniper.net 25
Hierarchical Scheduling
LY
N
O
SE
U
AL
Remaining Traffic
N
So far this material has discussed explicitly applying traffic control profiles to interface sets and
individual VLANs. However, the Junos OS also provides an efficient way to group VLANs together with
R
a single configuration statement and implicitly apply a traffic control profile to those groups. The
Junos OS groups VLANs with no associated CoS profile on a per-interface set basis, with one
TE
Remaining Schedulers
A remaining scheduler is identical to an independent VLAN scheduler with regard to scheduling
IN
properties and bandwidth sharing. You can configure it with CIR, PIR, delay buffer rate, and queuing
parameters. A VLAN that does not have an associated CoS profile (traffic control profile) is assigned
to a remaining scheduler.
26 www.juniper.net
Hierarchical Scheduling
LY
N
O
SE
U
AL
Remaining VLANs
N
The slide illustrates the concept of remaining traffic and remaining schedulers. In the top half of the
slide, groups of VLANs under interface sets share common schedulers and one set of queues. VLANs
R
12, 13, and 14 under interface set A, and VLANs 21, 22, and 23 under interface set B, use their
respective interface set’s remaining scheduler. At the bottom of the slide, a group of VLANs not
TE
associated with an interface is set to share one scheduler and one set of queues. VLANs 31 and 32
use the port’s remaining scheduler.
IN
www.juniper.net 27
Hierarchical Scheduling
LY
N
O
SE
U
AL
You configure CoS for remaining traffic within an interface set much like a logical interface or VLAN.
Use a traffic control profile to set the CIR and PIR for the VLAN group. Within the profile, specify a
R
scheduler map to associate the VLAN group to its queues, and then apply the profile to the
appropriate interface set under the class-of-service stanza. Note that you must use the
TE
Note
IN
28 www.juniper.net
Hierarchical Scheduling
LY
N
O
SE
U
AL
Port-level CoS configuration for remaining traffic is largely the same as for interface sets. Use a
traffic control profile to specify CIR, PIR, and a scheduler map for the VLAN group, and then apply the
R
profile, this time to the appropriate physical interface under the class-of-service stanza. Once
again, note that you must use the output-traffic-control-profile-remaining
TE
statement to encompass remaining VLANs under the port that are not part of interface groups.
Note
Without a guaranteed rate, remaining traffic
IN
www.juniper.net 29
Hierarchical Scheduling
LY
N
O
SE
U
AL
With multiple levels of hierarchy, VLANs compete for a port’s bandwidth. This is in contrast to
port-level queuing, where queues compete for a port’s bandwidth. In port-level queuing, a queue’s
R
bandwidth utilization determines its priority region. For hierarchical queuing, it is imperative that the
bandwidth utilization of a VLAN influence the priorities of its queues. Queues belonging to VLANs
TE
using bandwidth below their configured guaranteed rate should be given higher priority over queues
of VLANs using bandwidth over their configured guaranteed rate. If a VLAN is using bandwidth over
its guaranteed rate, the priority of its queues must be lowered.
A queue has two priority levels, based on the bandwidth utilization of its VLAN. When a VLAN has
IN
utilized bandwidth less than its configured guaranteed rate, its queues operate with their configured
priority. When a VLAN exceeds its configured guaranteed rate, its queues operate in the excess
priority region.
To ensure that low latency traffic always meets strict latency and jitter requirements, strict-high
queues always operate with strict-high priority regardless of bandwidth utilization of its VLAN. Note,
however, that a strict-high queue is still bounded by the shaping rate of its VLAN. All other queues are
assigned a common excess priority when the VLAN exceeds its guaranteed rate.
30 www.juniper.net
Hierarchical Scheduling
LY
N
O
SE
U
AL
The scheduling algorithm used for H-CoS includes a combination of PQ-DWRR and Intelligent
R
the priority of queues waiting to send packets is considered at all levels in the scheduling hierarchy.
A high priority queue needs to wait only for other high priority queues before being picked for
transmission again.
The priority queuing process for H-CoS includes VLANs and interface sets competing together as
IN
follows:
• Transmit packets from all strict-high and high priority queues of all VLANs and interface
sets;
• Transmit packets from all medium-high and medium-low priority queues;
• Transmit packets from all low priority queues;
• Transmit packets from all excess priority queues; and
An interface set or VLAN that has reached its PIR is not eligible to be picked for transmission.
www.juniper.net 31
Hierarchical Scheduling
LY
N
O
SE
U
AL
Any bandwidth left after all VLANs have used their guaranteed bandwidths is referred to as excess
bandwidth. Excess bandwidth can be shared among the VLANs in proportion to their CIR or PIR, or it
R
override the port setting by configuring the mode at the interface set level.
IN
32 www.juniper.net
Hierarchical Scheduling
LY
N
O
SE
U
AL
The default excess bandwidth sharing mode shares bandwidth among VLANs in proportion of CIR. If
no CIR is defined, PIR is used. To provide fine-grained control over the sharing of the bandwidth,
R
configure maximum queue bandwidth to enable the Junos OS to make accurate scheduler
calculations. This configuration allows the Junos OS to fine-tune the schedulers to provide a high
TE
In the example on the slide, the maximum queue bandwidth of the port is 14 Mbps. The interface set
is configured to share excess bandwidth equally among its VLANs.
Note
For enhanced IQ PICs, use the excess-rate
statement to determine the amount of excess
bandwidth to share.
www.juniper.net 33
Hierarchical Scheduling
LY
N
O
SE
U
AL
Buffer Size
N
The amount of buffer that is allocated to all the queues of a VLAN is defined using the traffic control
profile of the VLAN. Unlike port-level queuing, the bandwidth of a VLAN is not implicitly determined—
R
it needs to be configured.
The rate used to determine the delay buffer allocated to a VLAN is configured as part of the traffic
TE
control profile of the VLAN. You can configure the buffer rate explicitly using the
delay-buffer-rate statement, or the system can calculate the rate implicitly based on the
guaranteed rate or shaping rate configured in the VLAN’s traffic control profile.
The maximum buffer size for EQ-DPCs is 500 ms, which is equivalent to 500 Mbps or 62.5 MB. The
IN
34 www.juniper.net
Hierarchical Scheduling
LY
N
O
SE
U
AL
To configure delay buffers, add a buffer delay rate to a traffic control profile. Then apply the profile to
a logical interface (VLAN) under the class-of-service stanza using the
R
output-traffic-control-profile statement. With the delay buffer rate set, you can
configure schedulers as usual, including buffer size.
TE
In practice, the logical interface’s delay buffer rate becomes the reference point for the schedulers.
In the example on the slide, the logical interface’s traffic control profile indicates a reference
bandwidth for buffering purposes of 40 Mbps. The traffic control profile also includes a scheduler
map that associates the two schedulers shown. The schedulers specify that their effective buffer
IN
delays should be 70% and 30% of the maximum buffer size (500 ms), or 350 ms and 150 ms. These
values equate to 1.75 MB and 0.75 MB, respectively.
The primary mechanism used to determine the amount of delay buffer allocated to a VLAN is the
delay buffer rate in the traffic control profile of the VLAN. If no delay buffer rate is configured, the
guaranteed rate configured in the traffic control profile of the VLAN is used. If no CIR is configured,
the shaping rate configured in the traffic control profile of the VLAN is used.
Note
If you do not include the delay-buffer-rate
statement, the logical interface receives a
minimal delay buffer rate equal to 2 MTU-sized
packets.
www.juniper.net 35
Hierarchical Scheduling
LY
N
O
SE
U
AL
Levels 1, 2, and 3 support allocation of buffer space. Each level uses the level below as its
bandwidth reference. However, note that the buffers at each level must not exceed the level below.
R
The Junos OS provides statistical multiplexing when not all VLANs are using their allocated share of
the memory. The amount of delay buffer provisioned can be over-subscribed by configuring the delay
TE
36 www.juniper.net
Hierarchical Scheduling
LY
N
O
SE
U
AL
Drop Profiles
N
The general function of RED drop profiles—to provide congestion control by selectively dropping
packets based on user-configurable drop profiles—is the same as for port-level scheduling.
R
Configuration is largely the same as well, but with some changes in implementation.
The drop probability is determined by analyzing the queue depth. When the queue buffer utilization
TE
is between 0 and the first configured value (the minimum queue depth), the packet is not dropped.
When the packet is between the minimum queue depth and a second configured value (the
maximum queue depth) the drop probability is linear, from the minimum queue depth up to the
maximum queue depth. Above the maximum queue depth, the drop probability is 100%.
IN
In the example on the slide, when the queue depth is between 0% and 25% full, there is no chance
of a drop. When the fill level is between 25% and 90% a linear increase in drop probability occurs.
When the queue depth reaches 90% full, the drop probability is 90%. When the queue depth is
above 90%, drop probability is immediately 100%.
When configuring drop profiles in an H-CoS context, follow these guidelines:
• Use only the segmented configuration method;
• Configure only two entries (representing minimum and maximum queue depths); and
• In the first entry, configure the drop probability to 0 (the second entry can be whatever
you want).
www.juniper.net 37
Hierarchical Scheduling
LY
N
O
SE
U
AL
The slide lists the general configuration steps to implement hierarchical schedulers on Junos
devices. The following pages expand these steps.
R
TE
IN
38 www.juniper.net
Hierarchical Scheduling
LY
N
O
SE
U
AL
The slide illustrates the requirements of a configuration scenario. The following pages shows how to
configure a device to meet these requirements.
R
TE
IN
www.juniper.net 39
Hierarchical Scheduling
LY
N
O
SE
U
AL
The slide shows the configuration of the physical interface. In this case, the ge-0/0/0 interface uses
R
the hierarchical-scheduler statement. Note that without this statement, H-CoS does not
work.
TE
IN
40 www.juniper.net
Hierarchical Scheduling
LY
N
O
SE
U
AL
The slide shows the configuration of the interface set. In this case, the set contains logical interfaces
R
(VLANs).
TE
IN
www.juniper.net 41
Hierarchical Scheduling
LY
N
O
SE
U
AL
The slide shows the configuration of the schedulers (there are no drop profiles in this example).
Three schedulers are configured, reserving 20%, 30%, and 50%, respectively, of transmission rate
R
42 www.juniper.net
Hierarchical Scheduling
LY
N
O
SE
U
AL
Schedulers are designed to manage traffic for a particular forwarding class, or queue. However,
schedulers don’t actually do anything on their own. Scheduler maps provide the link between
R
www.juniper.net 43
Hierarchical Scheduling
LY
N
O
SE
U
AL
In an H-CoS context, scheduler maps must be associated to a VLAN by being referenced within a
traffic control profile (rather than referenced by an interface, as with port-based queuing).
R
The slide shows the traffic control profiles configuration. All profiles have a CIR and PIR, and each
VLAN profile specifies a scheduler map, to associate a VLAN with its queues.
TE
IN
44 www.juniper.net
Hierarchical Scheduling
LY
N
O
SE
U
AL
The final configuration step is to apply each traffic control profile to an interface, in the outbound
direction.
R
The slide shows the port and interface set configuration. The port and interface set have statements
for how to share excess bandwidth. The interface set has a statement associating a traffic control
TE
profile. The port and interface set also each have statements that apply a traffic control profile to
remaining traffic.
IN
www.juniper.net 45
Hierarchical Scheduling
LY
N
O
SE
U
AL
This page continues from the previous one, applying traffic control profiles to interfaces in the
outbound direction.
R
The slide shows the logical interface (VLAN) configuration. Each unit has a statement associating a
traffic control profile to its respective VLAN.
TE
IN
46 www.juniper.net