vMX

Getting Started Guide for KVM

Release

17.1

Modified: 2017-03-02

Copyright 2017, Juniper Networks, Inc.

Juniper Networks, Inc.
1133 Innovation Way
Sunnyvale, California 94089
USA
408-745-2000
www.juniper.net
Copyright 2017, Juniper Networks, Inc. All rights reserved.

Juniper Networks, Junos, Steel-Belted Radius, NetScreen, and ScreenOS are registered trademarks of Juniper Networks, Inc. in the United
States and other countries. The Juniper Networks Logo, the Junos logo, and JunosE are trademarks of Juniper Networks, Inc. All other
trademarks, service marks, registered trademarks, or registered service marks are the property of their respective owners.

Juniper Networks assumes no responsibility for any inaccuracies in this document. Juniper Networks reserves the right to change, modify,
transfer, or otherwise revise this publication without notice.

vMX Getting Started Guide for KVM

17.1
Copyright 2017, Juniper Networks, Inc.
All rights reserved.

The information in this document is current as of the date on the title page.

YEAR 2000 NOTICE

Juniper Networks hardware and software products are Year 2000 compliant. Junos OS has no known time-related limitations through the
year 2038. However, the NTP application is known to have some difficulty in the year 2036.

END USER LICENSE AGREEMENT

The Juniper Networks product that is the subject of this technical documentation consists of (or is intended for use with) Juniper Networks
software. Use of such software is subject to the terms and conditions of the End User License Agreement (EULA) posted at
http://www.juniper.net/support/eula.html. By downloading, installing or using such software, you agree to the terms and conditions of
that EULA.

ii Copyright 2017, Juniper Networks, Inc.

Table of Contents
About the Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Documentation and Release Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Documentation Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Documentation Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Requesting Technical Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv
Self-Help Online Tools and Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv
Opening a Case with JTAC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv
Chapter 1 vMX Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
vMX Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Benefits and Uses of vMX Routers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Architecture of a vMX Instance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Virtual Network Interfaces for vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
VM CPU Socket Awareness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Traffic Flow in a vMX Router . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
CPU Affinity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Automation for vMX Routers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Chapter 2 Setting Up vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Minimum Hardware and Software Requirements . . . . . . . . . . . . . . . . . . . . . . . . . 25
vMX Package Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Chapter 3 Installing vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Preparing the Ubuntu Host to Install vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Upgrading the Kernel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Upgrading to libvirt 1.2.19 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Uninstalling Previous Versions of libvirt from /usr/local . . . . . . . . . . . . . . . . . 32
Updating Drivers for the X710 NIC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Preparing the Red Hat Enterprise Linux Host to Install vMX . . . . . . . . . . . . . . . . . 33
Preparing the CentOS Host to Install vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Installing vMX for Different Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Installing vMX for Lab Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Installing vMX for Low-Bandwidth Applications . . . . . . . . . . . . . . . . . . . . . . . 38
Installing vMX for High-Bandwidth Applications . . . . . . . . . . . . . . . . . . . . . . 39
Specifying vMX Configuration File Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Configuring the Host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Configuring the VCP VM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
Configuring the VFP VM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
Configuring Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Deploying vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

Copyright 2017, Juniper Networks, Inc. iii

Getting Started Guide for KVM

Connecting to VMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Logging In to VCP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Logging In to VFP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Managing vMX Licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Adding a License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Deleting a License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
Viewing the Chassis Serial ID for vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Chapter 4 Managing vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Managing vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Controlling vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Deploying vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Managing vMX Deployments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Specifying the Temporary File Directory . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Specifying the Environment File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Configuring Logging Options for vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Connecting to Console Port for the VMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Getting Help for the Script Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Binding virtio Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Setting Up the Device Bindings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Creating Device Bindings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Deleting Device Bindings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Verifying Device Bindings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Chapter 5 Configuring vMX Chassis-Level Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Configuring the Number of Active Ports on vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Naming the Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Configuring the Media MTU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Enabling Performance Mode or Lite Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Tuning Performance Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
Chapter 6 Class of Service for vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
CoS on vMX Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
CoS Features and Limitations on vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
Configuring Hierarchical CoS on vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
Enabling Flexible Queuing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Mapping Forwarding Classes to Queues on vMX . . . . . . . . . . . . . . . . . . . . . . 64
Configuring Traffic Control Profiles for vMX . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Configuring Schedulers on vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Example: Configuring Hierarchical CoS on vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
Packet Loss Priority and Drop Profiles on vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
Managing Congestion Using Drop Profiles and Packet Loss Priorities on vMX . . . 70
Configuring Drop Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
Configuring Schedulers with Drop Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
Chapter 7 Troubleshooting vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Viewing VFP Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Viewing VFP Log Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
Troubleshooting VFP and VCP Connection Establishment . . . . . . . . . . . . . . . . . . 75

iv Copyright 2017, Juniper Networks, Inc.

 Table of Contents

Verifying Whether VMs Are Running . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

Viewing CPU Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
Verifying BIOS Settings for SR-IOV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

Copyright 2017, Juniper Networks, Inc. v

Getting Started Guide for KVM

vi Copyright 2017, Juniper Networks, Inc.

List of Figures
Chapter 1 vMX Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Figure 1: vMX Instance in a Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Figure 2: NIC Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Chapter 4 Managing vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Figure 3: Binding Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

Copyright 2017, Juniper Networks, Inc. vii

Getting Started Guide for KVM

viii Copyright 2017, Juniper Networks, Inc.

List of Tables
About the Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Table 1: Notice Icons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii
Table 2: Text and Syntax Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii
Chapter 1 vMX Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Table 3: Considerations for Choosing a Virtualization Technique . . . . . . . . . . . . . . 21
Table 4: Application Packages for Licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Chapter 2 Setting Up vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Table 5: Minimum Hardware Requirements for vMX . . . . . . . . . . . . . . . . . . . . . . . 25
Table 6: Minimum Software Requirements for Ubuntu . . . . . . . . . . . . . . . . . . . . . 26
Table 7: Minimum Software Requirements for Red Hat Enterprise Linux . . . . . . . . 27
Table 8: Minimum Software Requirements for CentOS . . . . . . . . . . . . . . . . . . . . . 27
Table 9: vMX Package Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Chapter 3 Installing vMX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Table 10: Sample Configurations for Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Copyright 2017, Juniper Networks, Inc. ix

Getting Started Guide for KVM

x Copyright 2017, Juniper Networks, Inc.

About the Documentation

Documentation and Release Notes on page xi

Documentation Conventions on page xi
Documentation Feedback on page xiii
Requesting Technical Support on page xiv

Documentation and Release Notes

To obtain the most current version of all Juniper Networks technical documentation,
see the product documentation page on the Juniper Networks website at
http://www.juniper.net/techpubs/.

If the information in the latest release notes differs from the information in the
documentation, follow the product Release Notes.

Juniper Networks Books publishes books by Juniper Networks engineers and subject
matter experts. These books go beyond the technical documentation to explore the
nuances of network architecture, deployment, and administration. The current list can
be viewed at http://www.juniper.net/books.

Documentation Conventions

Table 1 on page xii defines notice icons used in this guide.

Copyright 2017, Juniper Networks, Inc. xi

Getting Started Guide for KVM

Table 1: Notice Icons

Icon Meaning Description

Informational note Indicates important features or instructions.

Caution Indicates a situation that might result in loss of data or hardware damage.

Warning Alerts you to the risk of personal injury or death.

Laser warning Alerts you to the risk of personal injury from a laser.

Tip Indicates helpful information.

Best practice Alerts you to a recommended use or implementation.

Table 2 on page xii defines the text and syntax conventions used in this guide.

Table 2: Text and Syntax Conventions

Convention Description Examples

Bold text like this Represents text that you type. To enter configuration mode, type the
configure command:

user@host> configure

Fixed-width text like this Represents output that appears on the user@host> show chassis alarms
terminal screen.
No alarms currently active

Italic text like this Introduces or emphasizes important A policy term is a named structure
new terms. that defines match conditions and
Identifies guide names. actions.
Junos OS CLI User Guide
Identifies RFC and Internet draft titles.
RFC 1997, BGP Communities Attribute

Italic text like this Represents variables (options for which Configure the machines domain name:
you substitute a value) in commands or
configuration statements. [edit]
root@# set system domain-name
domain-name

xii Copyright 2017, Juniper Networks, Inc.

 About the Documentation

Table 2: Text and Syntax Conventions (continued)

Convention Description Examples

Text like this Represents names of configuration To configure a stub area, include the
statements, commands, files, and stub statement at the [edit protocols
directories; configuration hierarchy levels; ospf area area-id] hierarchy level.
or labels on routing platform The console port is labeled CONSOLE.
components.

< > (angle brackets) Encloses optional keywords or variables. stub <default-metric metric>;

| (pipe symbol) Indicates a choice between the mutually broadcast | multicast

exclusive keywords or variables on either
side of the symbol. The set of choices is (string1 | string2 | string3)
often enclosed in parentheses for clarity.

# (pound sign) Indicates a comment specified on the rsvp { # Required for dynamic MPLS only
same line as the configuration statement
to which it applies.

[ ] (square brackets) Encloses a variable for which you can community name members [
substitute one or more values. community-ids ]

Indention and braces ( { } ) Identifies a level in the configuration [edit]

hierarchy. routing-options {
static {
route default {
; (semicolon) Identifies a leaf statement at a
nexthop address;
configuration hierarchy level.
retain;
}
}
}

GUI Conventions
Bold text like this Represents graphical user interface (GUI) In the Logical Interfaces box, select
items you click or select. All Interfaces.
To cancel the configuration, click
Cancel.

> (bold right angle bracket) Separates levels in a hierarchy of menu In the configuration editor hierarchy,
selections. select Protocols>Ospf.

Documentation Feedback

We encourage you to provide feedback, comments, and suggestions so that we can

improve the documentation. You can provide feedback by using either of the following
methods:

Online feedback rating systemOn any page of the Juniper Networks TechLibrary site
at http://www.juniper.net/techpubs/index.html, simply click the stars to rate the content,
and use the pop-up form to provide us with information about your experience.
Alternately, you can use the online feedback form at
http://www.juniper.net/techpubs/feedback/.

Copyright 2017, Juniper Networks, Inc. xiii

Getting Started Guide for KVM

E-mailSend your comments to [email protected]. Include the document

or topic name, URL or page number, and software version (if applicable).

Requesting Technical Support

Technical product support is available through the Juniper Networks Technical Assistance
Center (JTAC). If you are a customer with an active J-Care or Partner Support Service
support contract, or are covered under warranty, and need post-sales technical support,
you can access our tools and resources online or open a case with JTAC.

JTAC policiesFor a complete understanding of our JTAC procedures and policies,

review the JTAC User Guide located at
http://www.juniper.net/us/en/local/pdf/resource-guides/7100059-en.pdf.

Product warrantiesFor product warranty information, visit

http://www.juniper.net/support/warranty/.

JTAC hours of operationThe JTAC centers have resources available 24 hours a day,
7 days a week, 365 days a year.

Self-Help Online Tools and Resources

For quick and easy problem resolution, Juniper Networks has designed an online
self-service portal called the Customer Support Center (CSC) that provides you with the
following features:

Find CSC offerings: http://www.juniper.net/customers/support/

Search for known bugs: http://www2.juniper.net/kb/

Find product documentation: http://www.juniper.net/techpubs/

Find solutions and answer questions using our Knowledge Base: http://kb.juniper.net/

Download the latest versions of software and review release notes:

http://www.juniper.net/customers/csc/software/

Search technical bulletins for relevant hardware and software notifications:

http://kb.juniper.net/InfoCenter/

Join and participate in the Juniper Networks Community Forum:

http://www.juniper.net/company/communities/

Open a case online in the CSC Case Management tool: http://www.juniper.net/cm/

To verify service entitlement by product serial number, use our Serial Number Entitlement
(SNE) Tool: https://tools.juniper.net/SerialNumberEntitlementSearch/

Opening a Case with JTAC

You can open a case with JTAC on the Web or by telephone.

Use the Case Management tool in the CSC at http://www.juniper.net/cm/.

Call 1-888-314-JTAC (1-888-314-5822 toll-free in the USA, Canada, and Mexico).

xiv Copyright 2017, Juniper Networks, Inc.

 About the Documentation

For international or direct-dial options in countries without toll-free numbers, see

http://www.juniper.net/support/requesting-support.html.

Copyright 2017, Juniper Networks, Inc. xv

Getting Started Guide for KVM

xvi Copyright 2017, Juniper Networks, Inc.

CHAPTER 1

vMX Overview

vMX Overview on page 17

Benefits and Uses of vMX Routers on page 18
Architecture of a vMX Instance on page 19
Virtual Network Interfaces for vMX on page 21
VM CPU Socket Awareness on page 21
Automation for vMX Routers on page 23
Licensing on page 23

vMX Overview

The vMX router is a virtual version of the MX Series 3D Universal Edge Router. Like the
MX Series router, the vMX router runs the Junos operating system (Junos OS) and supports
Junos OS packet handling and forwarding modeled after the Trio chipset. Configuration
and management of vMX routers are the same as for physical MX Series routers, allowing
you to add the vMX router to a network without having to update your operations support
systems (OSS).

You install vMX software components on an industry-standard x86 server running a

hypervisor, either the kernel-based virtual machine (KVM) hypervisor or the VMware ESXi
hypervisor.

For servers running the KVM hypervisor, you also run the Linux operating system and
applicable third-party software. vMX software components come in one software package
that you install by running an orchestration script included with the package. The
orchestration script uses a configuration file that you customize for your vMX deployment.
You can install multiple vMX instances on one server.

For servers running the ESXi hypervisor, you run the applicable third-party software.

Related Architecture of a vMX Instance on page 19

Documentation

Copyright 2017, Juniper Networks, Inc. 17

Getting Started Guide for KVM

Benefits and Uses of vMX Routers

You can use virtual devices to lower your capital expenditure and operating costs,
sometimes through automating network operations. Even without automation, use of
the vMX application on standard x86 servers enables you to:

Quickly introduce new services

More easily deliver customized and personalized services to customers

Scale operations to push IP services closer to customers or to manage network growth

when growth forecasts are low or uncertain

Quickly expand service offerings into new sites

A well designed automation strategy decreases costs as well as increasing network

efficiency. By automating network tasks with the vMX router, you can:

Simplify network operations

Quickly deploy new vMX instances

Efficiently install a default Junos OS configuration on all or selected vMX instances

Quickly reconfigure existing vMX routers

You can deploy the vMX router to meet some specific network edge requirements, such
as:

Network simulation

Temporary deployment until a physical MX Series router is available

Related Automation for vMX Routers on page 23

Documentation

18 Copyright 2017, Juniper Networks, Inc.

 Chapter 1: vMX Overview

Architecture of a vMX Instance

The vMX architecture is organized in layers:

The vMX router at the top

Third-party software and the hypervisor in the middle

The x86 server in the physical layer at the bottom

Figure 1 on page 19 illustrates the architecture of a single vMX instance inside a server.

Figure 1: vMX Instance in a Server

The physical layer of the server contains the physical NICs, CPUs, memory, and Ethernet
management port. The host contains applicable third-party software and the hypervisor.

The vMX instance contains two separate virtual machines (VMs), one for the virtual
forwarding plane (VFP) and one for the virtual control plane (VCP). The VFP VM runs
the virtual Trio forwarding plane software and the VCP VM runs Junos OS.

The hypervisor presents the physical NIC to the VFP VM as a virtual NIC. Each virtual NIC
maps to a vMX interface. Figure 2 on page 20 illustrates the mapping.

Copyright 2017, Juniper Networks, Inc. 19

Getting Started Guide for KVM

Figure 2: NIC Mapping

After the vMX instance is created, you use the Junos OS CLI to configure these vMX
interfaces in the VCP. The vMX router supports the following types of interface names:

Gigabit Ethernet (ge)

10-Gigabit Ethernet (xe)

100-Gigabit Ethernet (et)

NOTE: vMX interfaces configured with the Junos OS CLI and the underlying
physical NIC on the server are independent of each other in terms of interface
type (for example, ge-0/0/0 can get mapped to a 10-Gigabit NIC).

The VCP VM and VFP VM require Layer 2 connectivity to communicate with each other.
An internal bridge that is local to the server for each vMX instance enables this
communication.

The VCP VM and VFP VM also require Layer 2 connectivity to communicate with the
Ethernet management port on the server. You must specify virtual Ethernet interfaces
with unique IP addresses and MAC addresses for both the VFP and VCP to set up an
external bridge for a vMX instance. Ethernet management traffic for all vMX instances
enters the server through the Ethernet management port.

The way network traffic passes from the physical NIC to the virtual NIC depends on the
virtualization technique that you configure.

vMX can be configured to run in two modes depending on the use case:

Lite modeNeeds fewer resources in terms of CPU and memory to run at lower
bandwidth.

Performance modeNeeds higher resources in terms of CPU and memory to run at

higher bandwidth.

20 Copyright 2017, Juniper Networks, Inc.

 Chapter 1: vMX Overview

NOTE: Performance mode is the default mode.

Related Naming the Interfaces on page 57

Documentation
Enabling Performance Mode or Lite Mode on page 58

Virtual Network Interfaces for vMX on page 21

Virtual Network Interfaces for vMX

In a virtual environment, packet input and output capabilities play a significant role in the
performance of the packet processing functionality inside the virtual machine, specifically
the VFP VM. VFP supports two types of virtual network interfaces:

ParavirtualizedParavirtualized network interfaces use network drivers in the guest

OS and host OS that interact with the virtual environment and communicate effectively
to give higher performance than fully emulated interfaces. In KVM, the supported
paravirtualized interface is virtio. For VMware, VMXNET3 is supported.

PCI passthroughPCI passthrough enables PCI devices such as network interfaces to

appear as if they were physically attached to the guest operating system, bypassing
the hypervisor and providing a high rate of data transfer. The physical network interfaces
support single root I/O virtualization (SR-IOV) capability and can be connected to the
VMs using PCI passthrough.

Choose the type based on how you want to use the vMX router. SeeTable 3 on page 21.

Table 3: Considerations for Choosing a Virtualization Technique

Consideration Paravirtualization Technique PCI Passthrough Technique

Interfaces virtio (for KVM), VMXNET3 (for VMware) SR-IOV

Use Cases Network simulation Static vMX deployments

Low-throughput applications High-throughput applications

Host Requirements No requirements specific to this technique Physical NIC must support PCI passthrough

Related Architecture of a vMX Instance on page 19

Documentation

VM CPU Socket Awareness

The x86 server architecture consists of multiple sockets and multiple cores within a
socket. Each socket also has memory that is used to store packets during I/O transfers
from the NIC to the host. To efficiently read packets from memory, guest applications
and associated peripherals (such as the NIC) should reside within a single socket. A

Copyright 2017, Juniper Networks, Inc. 21

Getting Started Guide for KVM

penalty is associated with spanning CPU sockets for memory accesses, which might
result in non-deterministic performance.

For vMX, we recommend that the vMX instance always runs within a single socket. CPU
pinning is a commonly used technique to ensure the virtual CPUs (vCPUs) associated
with a VM are pinned to a specific core so that the VM utilizes resources on a single socket.
The vMX orchestration script automatically performs CPU pinning.

Traffic Flow in a vMX Router

The VFP consists of the following functional components:

Receive thread (RX): RX moves packets from the NIC to the VFP. It performs
preclassification to ensure host-bound packets receive priority.

Worker thread: The Worker performs lookup and tasks associated with packet
manipulation and processing. It is the equivalent of the lookup ASIC on the physical
MX Series router.

Transmit thread (TX): TX moves packets from the Worker to the physical NIC.

The RX and TX components are assigned to the same core (I/O core). If there are enough
cores available for the VFP, the QoS scheduler can be allocated separate cores. If there
are not enough cores available, the QoS scheduler shares the TX core.

CPU Affinity
CPU affinity ensures that the virtual CPUs of the VM are pinned to a specific physical
core, which improves the VM performance by using the CPU cache efficiently.

The optimal number of vCPUs needed for performance mode:

VCP1

VFP (without CoS)(2.5 * number-of-ports) + 3

VFP (with CoS)(3.5 * number-of-ports) + 3

The minimum number of vCPUs needed for lite mode:

VCP1

VFP3

Related Architecture of a vMX Instance on page 19

Documentation

22 Copyright 2017, Juniper Networks, Inc.

 Chapter 1: vMX Overview

Automation for vMX Routers

Automating network tasks simplifies network configuration, provisioning, and

maintenance. Because the vMX software uses the same Junos OS software as MX Series
routers and other Juniper Networks routing devices, vMX supports the same automation
tools as Junos OS. In addition, you can use standard automation tools to deploy the vMX,
as you do other virtualized software.

Related Benefits and Uses of vMX Routers on page 18

Documentation

Licensing

Licenses are required for using vMX features. When you order licenses, this information
is bound to a customer ID. If you did not order the licenses, contact your account team
or Juniper Networks Customer Care for assistance. When you order a license, you receive
instructions for generating license activation keys on the Juniper Networks License
Management System.

The vMX licenses are based on application packages and processing capacity.
Table 4 on page 23 describes the features available with application packages.

Table 4: Application Packages for Licenses

Application Package Features

BASE IP routing with 32,000 routes in the forwarding table

Basic Layer 2 functionality, Layer 2 bridging and switching

ADVANCE Features in the BASE application package

IP routing with routes up to platform scale in the forwarding table

IP and MPLS switching for unicast and multicast applications

Layer 2 features include Layer 2 VPN, VPLS, EVPN, and Layer 2 Circuit

PREMIUM Features in the BASE and ADVANCE application packages

Layer 3 VPN for IP and multicast

An application package is associated with a bandwidth license. vMX provides full-duplex

bandwidth in the following capacities: 100 Mbps, 250 Mbps, 500 Mbps, 1 Gbps, 5 Gbps,
10 Gbps, and 40 Gbps. Bandwidth licenses that are not associated with a specific
application package apply to all application packages. Bandwidth licenses are additive.
For example, if you add a 500 Mbps license and a 1 Gbps license, you are entitled to use
1.5 Gbps of capacity.

You can download the vMX software BASE application package with 1 Mbps bandwidth
and evaluate it without a license. To use additional features, you must order the

Copyright 2017, Juniper Networks, Inc. 23

Getting Started Guide for KVM

appropriate license. If you delete all valid licenses, you can only use the BASE application
package with 1 Mbps bandwidth.

If you upgrade from a BASE package license to an ADVANCE or PREMIUM package license
or if you downgrade from an ADVANCE or PREMIUM package license to a BASE package
license, you must restart the routing protocol process. If your configuration has logical
systems, you must restart the routing protocol process for all logical systems.

If you need to move your vMX installation to another host, you must remove vMX from
the current host before installing vMX and adding the license on the new host.

Related Managing vMX Licenses on page 45

Documentation

24 Copyright 2017, Juniper Networks, Inc.

CHAPTER 2

Setting Up vMX

Minimum Hardware and Software Requirements on page 25

vMX Package Contents on page 27

Minimum Hardware and Software Requirements

Table 5 on page 25 lists the hardware requirements.

Table 5: Minimum Hardware Requirements for vMX

Description Value

Sample system configuration For lab simulation and low performance (less than 100 Mbps) use cases,
any x86 processor (Intel or AMD) with VT-d capability.

For all other use cases, Intel Ivy Bridge processors or later are required.
Example of Ivy Bridge processor: Intel Xeon E5-2667 v2 @ 3.30 GHz 25 MB
Cache

For single root I/O virtualization (SR-IOV) NIC type, use Intel 82599-based
PCI-Express cards (10 Gbps) and Ivy Bridge processors.

Number of cores For lab simulation use case applications (lite mode): Minimum of 4

NOTE: Performance mode is the default mode 1 for VCP

and the minimum value is based on one port. 3 for VFP

To calculate the optimal number of vCPUs For low-bandwidth (virtio) or high-bandwidth (SR-IOV) applications
needed by VFP for performance mode: (performance mode): Minimum of 8

Without CoS(2.5 * number-of-ports) + 3 1 for VCP

With CoS(3.5 * number-of-ports) + 3 7 for VFP

Copyright 2017, Juniper Networks, Inc. 25

Getting Started Guide for KVM

Table 5: Minimum Hardware Requirements for vMX (continued)

Description Value

Memory For lab simulation use case applications (lite mode): Minimum of 5 GB

NOTE: Performance mode is the default mode. 1 GB for VCP

4 GB for VFP

For low-bandwidth or high-bandwidth applications (performance mode):

Minimum of 16 GB

4 GB for VCP
12 GB for VFP

Additional 2 GB recommended for host OS

Storage Local or NAS

Other requirements Intel VT-d capability

Hyperthreading (recommended)

AES-NI

Table 6 on page 26 lists the software requirements.

Table 6: Minimum Software Requirements for Ubuntu

Description Value

Operating system Ubuntu 14.04.1 LTS (recommended host OS)

Linux 3.13.0-32-generic

Virtualization QEMU-KVM 2.0.0+dfsg-2ubuntu1.11 or later

Required packages bridge-utils qemu-kvm libvirt-bin python python-netifaces vnc4server

libyaml-dev python-yaml numactl libparted0-dev libpciaccess-dev
NOTE: Other additional packages might be libnuma-dev libyajl-dev libxml2-dev libglib2.0-dev libnl-dev python-pip
required to satisfy all dependencies. python-dev libxml2-dev libxslt-dev

NOTE: libvirt 1.2.19

NOTE: Use the apt-get install pkg name command to install a package.

Table 7 on page 27 lists the software requirements for Red Hat Enterprise Linux.

26 Copyright 2017, Juniper Networks, Inc.

 Chapter 2: Setting Up vMX

Table 7: Minimum Software Requirements for Red Hat Enterprise Linux

Description Value

Operating system Red Hat Enterprise Linux 7.2

Linux 3.10.0-327.4.5 or later

Virtualization QEMU-KVM 1.5.3 or later

Required packages python27-python-pip python27-python-devel numactl-libs libpciaccess-devel

parted-devel yajl-devel libxml2-devel glib2-devel libnl-devel libxslt-devel
NOTE: SR-IOV requires these packages: libyaml-devel numactl-devel redhat-lsb kmod-ixgbe libvirt-daemon-kvm
kernel-devel gcc numactl telnet net-tools

NOTE: libvirt 1.2.17 or later

NOTE: Use the yum install pkg name command to install a package.

Table 8 on page 27 lists the software requirements for CentOS.

Table 8: Minimum Software Requirements for CentOS

Description Value

Operating system CentOS 7.2

Virtualization QEMU-KVM 1.5.3 or later

Required packages python27-python-pip python27-python-devel numactl-libs libpciaccess-devel parted-devel

yajl-devel libxml2-devel glib2-devel libnl-devel libxslt-devel libyaml-devel numactl-devel
redhat-lsb kmod-ixgbe libvirt-daemon-kvm numactl telnet net-tools

NOTE: libvirt 1.2.19

To avoid any conflicts, install libvirt 1.2.19 instead of updating from libvirt 1.2.17.

NOTE: Use the yum install pkg name command to install a package.

Related Preparing the Ubuntu Host to Install vMX on page 29

Documentation
Preparing the Red Hat Enterprise Linux Host to Install vMX on page 33

Preparing the CentOS Host to Install vMX on page 34

Installing vMX for Different Use Cases on page 36

vMX Package Contents

Table 9 on page 28 lists the contents of the vMX package.

Copyright 2017, Juniper Networks, Inc. 27

Getting Started Guide for KVM

Table 9: vMX Package Contents

Filename Description

images/junos-vmx-x86-64-*.qcow2 Software image files for VCP.

images/vmxhdd.img Software image file for VCP file storage.

images/vFPC_*.img Software image file for VFP.

config/vmx.conf Configuration file for defining vMX parameters. See Specifying vMX Configuration
File Parameters on page 41 for more information.

config/vmx-junosdev.conf Configuration file for binding devices (for virtio NICs).

env/ OS environment settings.

scripts/ Juniper Networks orchestration scripts.

vmx.sh Main orchestration script. See Managing vMX on page 49 for more information
about command options.

NOTE: Only English locale is supported for using the vmx.sh script.

The orchestration script is only supported on Ubuntu.

NOTE: Modified IXGBE drivers are included in the package. Multicast

promiscuous mode for Virtual Functions is needed to receive control traffic
that comes with broadcast MAC addresses. The reference driver does not
come with this mode set, so the IXGBE drivers in this package contain certain
modifications to overcome this limitation.

Related Installing vMX for Different Use Cases on page 36

Documentation
Specifying vMX Configuration File Parameters on page 41

Managing vMX on page 49

28 Copyright 2017, Juniper Networks, Inc.

CHAPTER 3

Installing vMX

Preparing the Ubuntu Host to Install vMX on page 29

Preparing the Red Hat Enterprise Linux Host to Install vMX on page 33
Preparing the CentOS Host to Install vMX on page 34
Installing vMX for Different Use Cases on page 36
Specifying vMX Configuration File Parameters on page 41
Deploying vMX on page 44
Connecting to VMs on page 44
Managing vMX Licenses on page 45
Viewing the Chassis Serial ID for vMX on page 47

Preparing the Ubuntu Host to Install vMX

To prepare the Ubuntu host system for installing vMX:

1. Meet the minimum software and OS requirements described in Table 6 on page 26.
See Upgrading the Kernel on page 30 and Upgrading to libvirt 1.2.19 on page 31.

If you are using Intel XL710 PCI-Express family cards, make sure you update the drivers.
See Updating Drivers for the X710 NIC on page 32.

2. Enable Intel VT-d in BIOS. (We recommend that you verify the process with the vendor
because different systems have different methods to enable VT-d.)

Refer to the procedure to enable VT-d available on the Intel Website.

3. Disable KSM by setting KSM_ENABLED=0 in /etc/default/qemu-kvm.

4. Disable APIC virtualization by editing the /etc/modprobe.d/qemu-system-x86.conf

file and adding enable_apicv=0 to the line containing options kvm_intel.

options kvm_intel nested=1 enable_apicv=0

5. Restart the host to disable KSM and APIC virtualization.

6. If you are using SR-IOV, you must perform this step.

Copyright 2017, Juniper Networks, Inc. 29

Getting Started Guide for KVM

NOTE: You must remove any previous installation with an external bridge
in /etc/network/interfaces and revert to using the original management
interface. Make sure that the ifconfig -a command does not show external
bridges before you proceed with the installation.

To determine whether an external bridge is displayed, use the ifconfig

command to see the management interface. To confirm that this interface
is used for an external bridge group, use the brctl show command to see
whether the management interface is listed as an external bridge.

Enable SR-IOV capability by turning on intel_iommu=on in the /etc/default/grub

directory.

GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on"

Append the intel_iommu=on string to any existing text for the

GRUB_CMDLINE_LINUX_DEFAULT parameter.

Run the update-grub command followed by the reboot command.

7. For optimal performance, we recommend you configure the size of Huge Pages to be
1G on the host and make sure the NUMA node for the VFP has at least 16 1G Huge
Pages. To configure the size of Huge Pages, add the following line in /etc/default/grub:

GRUB_CMDLINE_LINUX="default_hugepagesz=1G hugepagesz=1G
hugepages=number-of-huge-pages"

The number of Huge Pages must be at least (16G * number-of-numa-sockets).

8. Run the modprobe kvm-intel command before you install vMX.

To meet the minimum software and OS requirements, you might need to perform these
tasks:

Upgrading the Kernel on page 30

Upgrading to libvirt 1.2.19 on page 31
Uninstalling Previous Versions of libvirt from /usr/local on page 32
Updating Drivers for the X710 NIC on page 32

Upgrading the Kernel

To upgrade the kernel:

1. Run the following commands:

apt-get install linux-firmware
apt-get install linux-image-3.13.0.32-generic
apt-get install linux-image-extra-3.13.0.32-generic
apt-get install linux-headers-3.13.0.32-generic

2. Restart the system.

30 Copyright 2017, Juniper Networks, Inc.

 Chapter 3: Installing vMX

NOTE: If you are using Ubuntu 14.04.1 LTS, which comes with
3.13.0-32-generic, you can skip this step. Ubuntu 14.04 comes with a lower
version of kernel (Linux 3.13.0-24-generic) than the recommended version
(Linux 3.13.0-32-generic).

Upgrading to libvirt 1.2.19

Ubuntu 14.04 supports libvirt 1.2.2 (which works for VFP lite mode). If you are using the
VFP performance mode or deploying multiple vMX instances using the VFP lite mode,
you must upgrade to libvirt 1.2.19.

NOTE: If previous versions of libvirt were installed in /usr/local, there might

be a library conflict. To resolve the conflict before you upgrade libvirt, see
Uninstalling Previous Versions of libvirt from /usr/local on page 32.

To upgrade libvirt:

1. Make sure that you install all the packages listed in Table 6 on page 26.

2. Navigate to the /tmp directory using the cd /tmp command.

3. Get the libvirt-1.2.19 source code by using the command

wget http://libvirt.org/sources/libvirt-1.2.19.tar.gz.

4. Uncompress and untar the file using the tar xzvf libvirt-1.2.19.tar.gz command.

5. Navigate to the libvirt-1.2.19 directory using the cd libvirt-1.2.19 command.

6. Stop libvirtd with the service libvirt-bin stop command.

7. Run the ./configure --prefix=/usr --localstatedir=/ --with-numactl command.

8. Run the make command.

9. Run the make install command.

10. Make sure that the libvirtd daemon is running. (Use the service libvirt-bin start command
to start it again. If it does not start, use the /usr/sbin/libvirtd -d command.)

root@vmx-server:~# ps aux | grep libvirtd

root 1509 0.0 0.0 372564 16452 ? Sl 10:25 0:00
/usr/sbin/libvirtd -d

11. Verify that the versions of libvirtd and virsh are 1.2.19.

root@vmx-server:~# /usr/sbin/libvirtd --version

libvirtd (libvirt) 1.2.19
root@vmx-server:~# /usr/bin/virsh --version
1.2.19
root@vmx-server:~#

The system displays the code compilation log.

Copyright 2017, Juniper Networks, Inc. 31

Getting Started Guide for KVM

NOTE: If you cannot deploy vMX after upgrading libvirt, bring down the virbr0
bridge with the ifconfig virbr0 down command and delete the bridge with the
brctl delbr virbr0 command.

Uninstalling Previous Versions of libvirt from /usr/local

If previous versions of libvirt were installed in /usr/local, there might be a library conflict.
To resolve the conflict:

1. If you previously installed directly from source, download the old version of the libvirt
source. For example:

cd /tmp
wget http://libvirt.org/sources/libvirt-1.2.8.tar.gz
tar zxvf libvirt-1.2.8.tar.gz
cd libvirt-1.2.8

2. Uninstall the old version from the original source.

./configure --prefix=/usr/local --with-numactl

service libvirt-bin stop
make uninstall

3. Remove the old libraries from /usr/local.

/bin/rm rf /usr/local/lib/libvirt*
ldconfig

Updating Drivers for the X710 NIC

If you are using Intel XL710 PCI-Express family NICs, make sure you update the drivers
before you install vMX.

To update the drivers:

1. Download the vMX software package as root and uncompress the package.

tar xzvf package-name

2. Install the i40e driver from the installation directory.

cd drivers/i40e-1.3.46/src
make install

3. Install the latest i40evf driver from Intel.

For example, the following commands download and install Version 1.4.15:

cd /tmp
wget https://downloadmirror.intel.com/24693/eng/i40evf-1.4.15.tar.gz
tar zxvf i40evf-1.4.15.tar.gz
cd i40evf-1.4.15/src
make install

4. Update initrd with the drivers.

update-initramfs -u -k 'uname -r'

32 Copyright 2017, Juniper Networks, Inc.

 Chapter 3: Installing vMX

5. Activate the new driver.

rmmod i40e
modprobe i40e

Related Installing vMX for Different Use Cases on page 36

Documentation

Preparing the Red Hat Enterprise Linux Host to Install vMX

To prepare the host system running Red Hat Enterprise Linux for installing vMX:

1. Meet the minimum software and OS requirements described in Table 7 on page 27.

2. Enable hyperthreading and VT-d in BIOS.

If you are using SR-IOV, enable SR-IOV in BIOS.

We recommend that you verify the process with the vendor because different systems
have different methods to access and change BIOS settings.

3. During the OS installation, select the Virtualization Host and Virtualization Platform
software collections.

If you did not select these software collections during the GUI installation, use the
following commands to install them:

yum groupinstall "virtualization host"

yum groupinstall "virtualization platform"

4. Register your host using your Red Hat account credentials. Enable the appropriate
repositories.

subscription-manager register --username username --password password --auto-attach

subscription-manager repos --enable rhel-server-rhscl-7-rpms
subscription-manager repos --enable rhel-7-server-extras-rpms
subscription-manager repos --enable rhel-7-server-rhn-tools-beta-rpms
subscription-manager repos --enable rhel-7-server-optional-rpms

5. Update currently installed packages.

yum upgrade

6. Install the required packages.

yum install python27-python-pip python27-python-devel numactl-libs libpciaccess-devel

parted-devel yajl-devel libxml2-devel glib2-devel libnl-devel libxslt-devel
libyaml-devel numactl-devel redhat-lsb kmod-ixgbe libvirt-daemon-kvm numactl
telnet net-tools

7. (Optional) If you are using SR-IOV, you must install these packages and enable SR-IOV
capability.

yum install kernel-devel gcc

grubby --args="intel_iommu=on" --update-kernel=ALL

Reboot and log in again.

8. Link the qemu-kvm binary to the qemu-system-x86_64 file.

Copyright 2017, Juniper Networks, Inc. 33

Getting Started Guide for KVM

ln -s /usr/libexec/qemu-kvm /usr/bin/qemu-system-x86_64

9. Set up the path for the correct Python release and install the PyYAML library.

PATH=/opt/rh/python27/root/usr/bin:$PATH
export PATH
pip install netifaces pyyaml

10. If you have installed any Red Hat OpenStack libraries, you must change
script/templates/red_{vPFE, vRE}-ref.xml to use <type arch='x86_64'
machine='pc-0.13'>hvm</type> as the machine type.

11. Stop and disable Network Manager.

systemctl disable NetworkManager

systemctl stop NetworkManager

If you cannot stop Network Manager, you can prevent resolv.conf from being
overwritten with the chattr +I /etc/resolv.conf command.

12. Ensure that the build directory is readable by the QEMU user.

chmod -R o+r,o+x build-directory-pathname

As an alternative, you can configure QEMU to run as the root user by setting the
/etc/libvirt/qemu.conf file to user=root.

You can now install vMX.

NOTE: When you install vMX with the sh vmx.sh -lv --install command, you
might see a kernel version mismatch warning. You can ignore this warning.

Related Installing vMX for Different Use Cases on page 36

Documentation

Preparing the CentOS Host to Install vMX

To prepare the host system running CentOS for installing vMX:

1. Meet the minimum software and OS requirements described in Table 8 on page 27.

2. Enable hyperthreading and VT-d in BIOS.

If you are using SR-IOV, enable SR-IOV in BIOS.

We recommend that you verify the process with the vendor because different systems
have different methods to access and change BIOS settings.

3. During the OS installation, select the Virtualization Host and Virtualization Platform
software collections.

If you did not select these software collections during the GUI installation, use the
following commands to install them:

yum groupinstall "virtualization host"

yum groupinstall "virtualization platform"

34 Copyright 2017, Juniper Networks, Inc.

 Chapter 3: Installing vMX

4. Enable the appropriate repositories.

yum install -y "http://www.elrepo.org/elrepo-release-7.0-2.el7.elrepo.noarch.rpm"

yum install centos-release-scl

5. Update currently installed packages.

yum upgrade

6. Install the required packages.

yum install python27-python-pip python27-python-devel numactl-libs libpciaccess-devel

parted-devel yajl-devel libxml2-devel glib2-devel libnl-devel libxslt-devel
libyaml-devel numactl-devel redhat-lsb kmod-ixgbe libvirt-daemon-kvm numactl
telnet net-tools

7. (Optional) If you are using SR-IOV, you must install these packages and enable SR-IOV
capability.

yum install kernel-devel gcc

grubby --args="intel_iommu=on" --update-kernel=ALL

Reboot and log in again.

8. Link the qemu-kvm binary to the qemu-system-x86_64 file.

ln -s /usr/libexec/qemu-kvm /usr/bin/qemu-system-x86_64

9. Set up the path for the correct Python release and install the PyYAML library.

PATH=/opt/rh/python27/root/usr/bin:$PATH
export PATH
pip install netifaces pyyaml

10. If you have installed any Red Hat OpenStack libraries, you must change
script/templates/red_{vPFE, vRE}-ref.xml to use <type arch='x86_64'
machine='pc-0.13'>hvm</type> as the machine type.

11. Stop and disable Network Manager.

systemctl disable NetworkManager

systemctl stop NetworkManager

If you cannot stop Network Manager, you can prevent resolv.conf from being
overwritten with the chattr +I /etc/resolv.conf command.

12. Ensure that the build directory is readable by the QEMU user.

chmod -R o+r,o+x build-directory-pathname

As an alternative, you can configure QEMU to run as the root user by setting the
/etc/libvirt/qemu.conf file to user=root.

13. Add this line to the end of the /etc/profile file.

export PATH=/opt/rh/python27/root/usr/bin:$PATH

You can now install vMX.

NOTE: When you install vMX with the sh vmx.sh -lv --install command, you
might see a kernel version mismatch warning. You can ignore this warning.

Copyright 2017, Juniper Networks, Inc. 35

Getting Started Guide for KVM

Related Installing vMX for Different Use Cases on page 36

Documentation

Installing vMX for Different Use Cases

Installing vMX is different for specific use cases. Table 10 on page 36 lists the sample
configuration requirements for some vMX use cases.

Table 10: Sample Configurations for Use Cases

Use Case Minimum vCPUs Minimum Memory NIC Device Type

Lab simulation 4: 5 GB: virtio

1 for VCP 1 GB for VCP
Up to 100 Mbps performance 3 for VFP 4 GB for VFP

Low-bandwidth applications 8: 16 GB: virtio

1 for VCP 4 GB for VCP
Up to 3 Gbps performance 7 for VFP 12 GB for VFP

High-bandwidth applications 8: 16 GB SR-IOV

or performance testing 1 for VCP 4 GB for VCP
7 for VFP 12 GB for VFP
For 3 Gbps and beyond
performance

To install vMX for a particular use case, perform one of the following tasks:

Installing vMX for Lab Simulation on page 36

Installing vMX for Low-Bandwidth Applications on page 38
Installing vMX for High-Bandwidth Applications on page 39

Installing vMX for Lab Simulation

The use case for lab simulation uses the virtio NIC.

To install vMX for the lab simulation (less than 100 Mbps) application use case:

1. Download the vMX software package as root and uncompress the package.

tar xzvf package-name

2. Change directory to the location of the uncompressed vMX package.

cd package-location

3. Edit the config/vmx.conf text file with a text editor to configure a single vMX instance.

Ensure the following parameter is set properly in the vMX configuration file:

device-type : virtio

See Specifying vMX Configuration File Parameters on page 41.

36 Copyright 2017, Juniper Networks, Inc.

 Chapter 3: Installing vMX

4. Run the ./vmx.sh -lv --install script to deploy the vMX instance specified by the
config/vmx.conf startup configuration file and provide verbose-level logging to a file.
See Deploying vMX on page 44.

5. From the VCP, enable lite mode for the VFP.

user@vmx# set chassis fpc 0 lite-mode

Here is a sample vMX startup configuration file using the virtio device type for lab
simulation:
---
#Configuration on the host side - management interface, VM images etc.
HOST:
identifier : vmx1 # Maximum 4 characters
host-management-interface : eth0
routing-engine-image : "/home/vmx/vmxlite/images/junos-vmx-x86-64.qcow2"

routing-engine-hdd : "/home/vmx/vmxlite/images/vmxhdd.img"
forwarding-engine-image : "/home/vmx/vmxlite/images/vFPC.img"

---
#External bridge configuration
BRIDGES:
- type : external
name : br-ext # Max 10 characters

---
#vRE VM parameters
CONTROL_PLANE:
vcpus : 1
memory-mb : 1024
console_port: 8601

interfaces :
- type : static
ipaddr : 10.102.144.94
macaddr : "0A:00:DD:C0:DE:0E"

---
#vPFE VM parameters
FORWARDING_PLANE:
memory-mb : 4096
vcpus : 3
console_port: 8602
device-type : virtio

interfaces :
- type : static
ipaddr : 10.102.144.98
macaddr : "0A:00:DD:C0:DE:10"

---
#Interfaces
JUNOS_DEVICES:
- interface : ge-0/0/0
mac-address : "02:06:0A:0E:FF:F0"
description : "ge-0/0/0 interface"

- interface : ge-0/0/1

Copyright 2017, Juniper Networks, Inc. 37

Getting Started Guide for KVM

mac-address : "02:06:0A:0E:FF:F1"
description : "ge-0/0/1 interface"

Installing vMX for Low-Bandwidth Applications

The use case for low-bandwidth applications uses virtio or SR-IOV NICs.

To install vMX for the low-bandwidth (up to 3 Gbps) application use case:

1. Download the vMX software package as root and uncompress the package.

tar xzvf package-name

2. Change directory to the location of the uncompressed vMX package.

cd package-location

3. Edit the config/vmx.conf text file with a text editor to configure a single vMX instance.

Ensure the following parameter is set properly in the vMX configuration file:

device-type: virtio or device-type: sriov

See Specifying vMX Configuration File Parameters on page 41.

4. Run the ./vmx.sh -lv --install script to deploy the vMX instance specified by the
config/vmx.conf startup configuration file and provide verbose-level logging to a file.
See Deploying vMX on page 44.

5. From the VCP, enable performance mode for the VFP.

user@vmx# set chassis fpc 0 performance-mode

Here is a sample vMX startup configuration file using the virtio device type for
low-bandwidth applications:
---
#Configuration on the host side - management interface, VM images etc.
HOST:
identifier : vmx1 # Maximum 4 characters
host-management-interface : eth0
routing-engine-image : "/home/vmx/vmx/images/junos-vmx-x86-64.qcow2"
routing-engine-hdd : "/home/vmx/vmx/images/vmxhdd.img"
forwarding-engine-image : "/home/vmx/vmx/images/vFPC.img"

---
#External bridge configuration
BRIDGES:
- type : external
name : br-ext # Max 10 characters

---
#vRE VM parameters
CONTROL_PLANE:
vcpus : 1
memory-mb : 4096
console_port: 8601

interfaces :
- type : static
ipaddr : 10.102.144.94
macaddr : "0A:00:DD:C0:DE:0E"

38 Copyright 2017, Juniper Networks, Inc.

 Chapter 3: Installing vMX

---
#vPFE VM parameters
FORWARDING_PLANE:
memory-mb : 12288
vcpus : 10
console_port: 8602
device-type : virtio

interfaces :
- type : static
ipaddr : 10.102.144.98
macaddr : "0A:00:DD:C0:DE:10"

---
#Interfaces
JUNOS_DEVICES:
- interface : ge-0/0/0
mac-address : "02:06:0A:0E:FF:F0"
description : "ge-0/0/0 interface"

- interface : ge-0/0/1
mac-address : "02:06:0A:0E:FF:F1"
description : "ge-0/0/1 interface"

Installing vMX for High-Bandwidth Applications

The use case for high-bandwidth applications uses the SR-IOV NICs.

To install vMX for the high-bandwidth (above 3 Gbps) application use case:

1. Download the vMX software package as root and uncompress the package.

tar xzvf package-name

2. Change directory to the location of the uncompressed vMX package.

cd package-location

3. Edit the config/vmx.conf text file with a text editor to configure a single vMX instance.

Ensure the following parameter is set properly in the vMX configuration file:

device-type: sriov

See Specifying vMX Configuration File Parameters on page 41.

4. Run the ./vmx.sh -lv --install script to deploy the vMX instance specified by the
config/vmx.conf startup configuration file and provide verbose-level logging to a file.
See Deploying vMX on page 44.

5. From the VCP, enable performance mode for the VFP.

user@vmx# set chassis fpc 0 performance-mode

Here is a sample vMX startup configuration file using the SR-IOV device type:
---
#Configuration on the host side - management interface, VM images etc.
HOST:
identifier : vmx1 # Maximum 4 characters
host-management-interface : eth0

Copyright 2017, Juniper Networks, Inc. 39

Getting Started Guide for KVM

routing-engine-image : "/home/vmx/images/junos-vmx-x86-64.qcow2"
routing-engine-hdd : "/home/vmx/images/vmxhdd.img"
forwarding-engine-image : "/home/vmx/images/vFPC.img"

---
#External bridge configuration
BRIDGES:
- type : external
name : br-ext # Max 10 characters

---
#VCP VM parameters
CONTROL_PLANE:
vcpus : 1
memory-mb : 4096
console_port: 8601

interfaces :
- type : static
ipaddr : 10.102.144.94
macaddr : "0A:00:DD:C0:DE:0E"

---
#VFP VM parameters
FORWARDING_PLANE:
memory-mb : 12288
vcpus : 10
console_port: 8602
device-type : sriov

interfaces :
- type : static
ipaddr : 10.102.144.98
macaddr : "0A:00:DD:C0:DE:10"

---
#Interfaces
JUNOS_DEVICES:
- interface : ge-0/0/0
port-speed-mbps : 10000
nic : eth1
mtu : 2000 # DO NOT EDIT
virtual-function : 0
mac-address : "02:06:0A:0E:FF:F0"
description : "ge-0/0/0 connects to eth1"

- interface : ge-0/0/1
port-speed-mbps : 10000
nic : eth2
mtu : 2000 # DO NOT EDIT
virtual-function : 0
mac-address : "02:06:0A:0E:FF:F1"
description : "ge-0/0/1 connects to eth2"

Related Preparing the Ubuntu Host to Install vMX on page 29

Documentation
Specifying vMX Configuration File Parameters on page 41

Deploying vMX on page 44

40 Copyright 2017, Juniper Networks, Inc.

 Chapter 3: Installing vMX

Specifying vMX Configuration File Parameters

The parameters required to configure vMX are defined in the startup configuration file.
The configuration file is in YAML format. The default file is config/vmx.conf. You can save
your configuration file to a different name for different instances.

To customize the configuration, perform these tasks:

Configuring the Host on page 41

Configuring the VCP VM on page 42
Configuring the VFP VM on page 42
Configuring Interfaces on page 43

Configuring the Host

To configure the host environment, you must change the identifier for each vMX instance
and you must provide the correct path for the images.

To configure the host, navigate to Host and specify the following parameters:

identifierName of the vMX instance, maximum of four alphanumeric characters.

host-management-interfaceName of the physical NIC on the host device that is used

for management access (eth0).

NOTE: The interfaces for HOST:host-management-interface,

CONTROL_PLANE, and FORWARDING_PLANE must be on the same subnet.

routing-engine-imageAbsolute path to the junos-vmx-x86-64-*.qcow2 file for

launching VCP.

routing-engine-hddAbsolute path to the vmxhdd.img file for VCP storage.

forwarding-engine-imageAbsolute path to the vFPC-*.img file for launching VFP.

make-local-copy-of-images(Optional) Makes a local copy of the VCP and VFP images

and uses the local copy to launch vMX. Default value is yes.

NOTE: Copy the image file from its default location to ensure that the
scripts do not try to use the same image file concurrently.

make-local-copy-of-vmxhdd(Optional) Makes a local copy of the VCP storage image

and uses the local copy to launch vMX. Default value is yes.

NOTE: Copy the image file from its default location to ensure that the
scripts do not try to use the same image file concurrently.

Copyright 2017, Juniper Networks, Inc. 41

Getting Started Guide for KVM

Configuring the VCP VM

To configure the VCP VM, you must change the IP address and you must make sure the
console port is not being used by another vMX instance or another server.

To configure the VCP VM, navigate to CONTROL_PLANE and specify the following
parameters:

vcpusNumber of vCPUs for the VCP, default is 1.

memory-mbAmount of memory for the VCP, default is 2 GB.

console_portKVM TCP-based console port. It must be a unique number.

console_listen(Optional) IP address for the interface from which the console can be
accessed; default is 127.0.0.1, which only allows access from within the host. To allow
access from any interfaces, specify 0.0.0.0.

ipaddrManagement IP address for the VCP VM (fxp0). Navigate to interfaces > type
(static) > ipaddr to modify this parameter.

NOTE: The interfaces for HOST:host-management-interface,

CONTROL_PLANE, and FORWARDING_PLANE must be on the same subnet.

Configuring the VFP VM

Before you configure the VFP VM, consider the following:

You must make sure the console port is not being used by another vMX instance or
another server.

To disable network access to the VFP console, do not configure an IP address.

Based on your requirements, you might want to change the memory, number of vCPUs,
and the device type. See Table 10 on page 36 for some sample configuration
requirements.

To configure the VFP VM, navigate to FORWARDING_PLANE and specify the following
parameters:

memory-mbAmount of memory for the VFP, default is 6 GB.

vcpusNumber of vCPUs for the VFP, default is 3.

console_portKVM TCP-based console port. It must be a unique number.

console_listen(Optional) IP address for the interface from which the console can be
accessed; default is 127.0.0.1, which only allows access from within the host. To allow
access from any interfaces, specify 0.0.0.0.

device-typeNIC interface type, either sriov or virtio.

ipaddrManagement IP address for the VFP VM (eth0). Navigate to interfaces > type
(static) > ipaddr to modify this parameter.

42 Copyright 2017, Juniper Networks, Inc.

 Chapter 3: Installing vMX

NOTE: The interfaces for HOST:host-management-interface,

CONTROL_PLANE, and FORWARDING_PLANE must be on the same subnet.

Configuring Interfaces
The JUNOS_DEVICES interface names correspond to the Linux physical NIC names on
the host. Bring up the Linux physical NIC ports that are defined in this section before
proceeding. For example, use the ifconfig eth9 up command to bring up the NIC ports on
the eth9 interface.

To configure interfaces for virtio device types, you must specify the interface and the
MAC address.

To configure interfaces for SR-IOV device types, you must specify the interface, the NIC,
and the MAC address.

To configure the routed interfaces, navigate to JUNOS_DEVICES and specify the following
parameters:

interfaceName of the interface on the vMX.

NOTE: The interface names that are defined in the vmx.conf file must be
contiguous starting from ge-0/0/0. The total number of interfaces
supported is 10, up to ge-0/0/9.

port-speed-mbps(SR-IOV only) Port speed for the physical NIC, default is 10000
Mbps.

nic(SR-IOV only) Name of the physical NIC.

NOTE: Depending on the version of udev, you can rename the classic Linux
standard ethXX names. See Predictable Network Interface Names for more
information.

mtu(SR-IOV only) MTU value, default is 2000 and maximum is 9500.

To change the MTU configuration for virtio device types, modify the mtu parameter in
the device binding file (vmx-junosdev.conf).

virtual-function(SR-IOV only) Child unit of the physical NIC, default is 0.

mac-addressUnicast MAC address for the physical NIC.

descriptionDescription of the mapping.

Related Preparing the Ubuntu Host to Install vMX on page 29

Documentation
Installing vMX for Different Use Cases on page 36

Copyright 2017, Juniper Networks, Inc. 43

Getting Started Guide for KVM

Deploying vMX

Running the ./vmx.sh --install script to deploy vMX sets up:

Host NICs

SR-IOV or virtio device types

External and internal bridges (br-ext and br-int)

NOTE: When br-ext is being created, access to the management port may
be frozen temporarily when it gets attached to the br-ext bridge.

Using the --install option also launches the VCP and VFP VMs.

We recommend you deploy the vMX by running the ./vmx.sh -lv --install script to provide
verbose-level logging to a file for the deployment of the vMX instance.

NOTE: Only English locale is supported for using the vmx.sh script on Ubuntu.

NOTE: If you cannot deploy vMX after upgrading libvirt, bring down the virbr0
bridge with the ifconfig virbr0 down command and delete the bridge with the
brctl delbr virbr0 command.

Related Specifying vMX Configuration File Parameters on page 41

Documentation
Managing vMX on page 49

Connecting to VMs

Perform these tasks to connect to the virtual machines for first-time configuration, to
enable access by other means (like Telnet or SSH):

Logging In to VCP on page 44

Logging In to VFP on page 45

Logging In to VCP
You can access the serial console using the ./vmx.sh --console vcp vmx-id command,
where vmx-id is the vMX identifier specified in the startup configuration file, and log in
with the username root and no password.

To disconnect from the console, log out of the session and press Ctrl + ]. At the telnet>
prompt, type close and press Enter.

44 Copyright 2017, Juniper Networks, Inc.

 Chapter 3: Installing vMX

Logging In to VFP
You can access the serial console using the ./vmx.sh --console vfp vmx-id command,
where vmx-id is the vMX identifier specified in the startup configuration file, and log in
with the username root and password root.

You can connect to VFP using the SSH protocol. Use the IP address defined under
FORWARDING_PLANE in the vmx.conf file. For security reasons, you cannot connect to
VFP using the Telnet protocol.

NOTE: Configure the management IP address on eth0 and routing for

reachability by using the Linux commands ifconfig and route add on VFP VM.

To disconnect from the console, log out of the session and press Ctrl + ]. At the telnet>
prompt, type close and press Enter.

Related Deploying vMX on page 44

Documentation
Managing vMX on page 49

Managing vMX Licenses

You must add a license to use vMX features. The licensed features are enforced based
on the license you purchased.

If you upgrade from a BASE package license to an ADVANCE or PREMIUM package license
or if you downgrade from an ADVANCE or PREMIUM package license to a BASE package
license, you must restart the routing protocol process (restart routing). If your configuration
has logical systems, you must restart the routing protocol process for all logical systems
(restart routing logical-system logical-system-name).

If you need to move your vMX installation to another host, you must remove vMX from
the current host before installing vMX and adding the license on the new host.

Adding a License on page 45

Deleting a License on page 46

Adding a License
To add a license key to the vMX:

1. Copy the license activation key file to the VCP and add the license key by specifying
the filename.

user@vmx> request system license add filename

Or, you can copy and paste the license activation key directly to add the license key.
For example:
user@vmx> request system license add terminal
E408408918 aeaqib qcsbja okbuqe rcmxnq vjocwf uxfsta

Copyright 2017, Juniper Networks, Inc. 45

Getting Started Guide for KVM

z5ufjb kdrmt6 57bimv 2f3ddp qttcdn 627q4a

jx4s5x hiri

2. Verify that the license is installed. VMX-BANDWIDTH indicates the licensed bandwidth
(in Mbps) and VMX-SCALE indicates the application package. (VMX-SCALE 1 is the
BASE package, VMX-SCALE 2 is the ADVANCE package, and VMX-SCALE 3 is the
PREMIUM package.) This information is also listed as Features in the Licenses installed
section. For example, this output indicates that the 40G perpetual license for the
PREMIUM application package is installed.

user@vmx> show system license

License usage:
Licenses Licenses Licenses Expiry
Feature name used installed needed
scale-subscriber 0 1000 0 permanent
scale-l2tp 0 1000 0 permanent
scale-mobile-ip 0 1000 0 permanent
VMX-BANDWIDTH 40000 40000 0 permanent
VMX-SCALE 3 3 0 permanent

Licenses installed:
License identifier: JUNOS640113
License version: 4
Software Serial Number: 1012620150123J
Customer ID: vMX-Juniper
Features:
vmx-bandwidth-40g - vmx-bandwidth-40g
permanent
vmx-feature-premium - vmx-feature-premium
permanent

3. Verify the configured bandwidth for PFE traffic matches the licensed bandwidth
(VMX-BANDWIDTH). The current and average bandwidth are also displayed.

user@vmx> show pfe statistics traffic bandwidth

Configured Bandwidth : 40000000000 bps
Bandwidth : 0 bps
Average Bandwidth : 0 bps

Deleting a License
To delete a vMX license:

1. Display the installed licenses.

user@vmx> show system license installed

License identifier: JUNOS640113
License version: 4
Features:
vmx-bandwidth-40g - vmx-bandwidth-40g
permanent
vmx-feature-premium - vmx-feature-premium
permanent

2. Delete the license.

user@vmx> request system license delete license-identifier

For example:

user@vmx> request system license delete JUNOS640113

46 Copyright 2017, Juniper Networks, Inc.

 Chapter 3: Installing vMX

3. Verify that the license is deleted.

user@vmx> show system license

License usage:
Licenses Licenses Licenses Expiry
Feature name used installed needed
scale-subscriber 0 1000 0 permanent
scale-l2tp 0 1000 0 permanent
scale-mobile-ip 0 1000 0 permanent

Licenses installed: none

Viewing the Chassis Serial ID for vMX

You can view the chassis serial ID by using the show chassis hardware command from
the CLI, or by using the sysctl hw.chassis.serialid command from shell.

user@vmx> show chassis hardware

Hardware inventory:
Item Version Part number Serial number Description
Chassis VM54599D128A VMX
Midplane
Routing Engine RE-VMX
CB 0 VMX SCB
FPC 0 Virtual FPC
CPU

Copyright 2017, Juniper Networks, Inc. 47

Getting Started Guide for KVM

48 Copyright 2017, Juniper Networks, Inc.

CHAPTER 4

Managing vMX

Managing vMX on page 49

Binding virtio Devices on page 52

Managing vMX

NOTE: Only English locale is supported for using the vmx.sh script on Ubuntu.

After you install and deploy vMX, you can use the vmx.sh script with different options to
perform these tasks:

Controlling vMX on page 49

Configuring Logging Options for vMX on page 51
Connecting to Console Port for the VMs on page 51
Getting Help for the Script Options on page 52

Controlling vMX
When you are controlling vMX with the vmx.sh script, you can perform these tasks:

Deploying vMX on page 49

Managing vMX Deployments on page 50
Specifying the Temporary File Directory on page 50
Specifying the Environment File on page 51

Deploying vMX

NOTE: You must be logged in as root to use the control options.

To deploy vMX, use these options with the vmx.sh script:

--cfg fileUse the specified vMX startup configuration file. The default file is
config/vmx.conf.

Copyright 2017, Juniper Networks, Inc. 49

Getting Started Guide for KVM

--installStart vMX by setting up the environment, driver dependencies, and memory

requirements and deploying the vMX. If you do not specify a startup configuration
file with the --cfg option, the default file is used.

NOTE: If you cannot deploy vMX after upgrading libvirt, bring down the virbr0
bridge with the ifconfig virbr0 down command and delete the bridge with the
brctl delbr virbr0 command.

This example deploys a new vMX instance specified by the my-vmx.cfg configuration file
and provides verbose-level logging to a file:

./vmx.sh -lv --install --cfg config/my-vmx.cfg

Managing vMX Deployments

NOTE: You must be logged in as root to use the control options.

Use these options with the vmx.sh script to stop, start, restart, verify, and clean up an
existing vMX:

--cfg fileUse the specified vMX startup configuration file. The default file is
config/vmx.conf.

--cleanupStop vMX and clean up relevant information about the vMX instance. It also
tears down the Linux bridges and other dependencies. If you do not specify a startup
configuration file with the --cfg option, the default file is used.

--restartStop and start a running vMX. This option is useful for redeploying a vMX that
has parameter changes in the startup configuration file. If you do not specify a startup
configuration file with the --cfg option, the default file is used.

--startStart the vMX that was running and stopped. If you do not specify a startup
configuration file with the --cfg option, the default file is used.

--statusVerify the status of a deployed vMX. If you do not specify a startup configuration
file with the --cfg option, the default file is used.

--stopStop vMX without cleaning up build files so that the vMX can be started quickly
without setup performed by the --install option.

This example tears down an existing vMX instance specified by the my-vmx.cfg
configuration file:

./vmx.sh --cleanup --cfg config/my-vmx.cfg

Specifying the Temporary File Directory

NOTE: You must be logged in as root to use the control options.

50 Copyright 2017, Juniper Networks, Inc.

 Chapter 4: Managing vMX

To specify the directory used for temporary files, run the ./vmx.sh build directory script.
The default directory is build/vmx-id, where vmx-id is the vMX identifier specified in the
startup configuration file.

By default, copies of the VCP and VFP images are copied to this directory. We recommend
that you do not change the make-local-copy-of-images and make-local-copy-of-vmxhdd
parameters when specifying startup configuration file parameters for the host.

Specifying the Environment File

NOTE: You must be logged in as root to use the control options.

To specify the environment file (.env), run the ./vmx.sh env file script. The default file is
env/ubuntu_sriov.env.

Configuring Logging Options for vMX

You can enable logging options. It is especially useful when used with the control options,
such as --install.

Use these options to configure logging:

-lEnable logging to a file in the specified build directory. The default directory is
build/vmx-id/logs, where vmx-id is the vMX identifier specified in the startup
configuration file. By default, logging is disabled.

-lvEnable logging with verbose details.

-lvfEnable logging with verbose details to the foreground (standard output).

This example deploys a new vMX instance specified by the my-vmx.cfg configuration file
and provides verbose-level logging to a file:

./vmx.sh -lv --install --cfg config/my-vmx.cfg

Connecting to Console Port for the VMs

Use these options with the vmx.sh script to connect to the console of the VCP or VFP of
the specified vMX:

--console vcp [vmx-id]Connect to the console of the VCP for the specified vMX. The
vMX identifier is specified in the startup configuration file.

--console vfp [vmx-id]Connect to the console of the VFP for the specified vMX. The
vMX identifier is specified in the startup configuration file.

This example connects to the console of the VCP for the vMX instance specified by the
vmx1 identifier:

./vmx.sh --console vcp vmx1

Copyright 2017, Juniper Networks, Inc. 51

Getting Started Guide for KVM

Getting Help for the Script Options

To obtain on-line help for the vmx.sh script options, run the ./vmx.sh --help script.

Related Deploying vMX on page 44

Documentation
Connecting to VMs on page 44

Binding virtio Devices

For configurations using virtio device types, you can bind multiple vMX instances together
on the same system if the host has enough CPU and memory to support the vMX
instances. You configure each vMX instance with a different startup configuration file.

If you are deploying multiple vMX instances, make sure:

The VM identifiers are unique across all instances.

The console ports of the VCP and the VFP are unique across all instances.

The external static IP address of the VCP and the VFP are unique across all instances.

The MAC addresses of the VCP and the VFP are unique across all instances, whenever
specified.

NOTE: All VMs share the same management domain. The physical
management interface (for example, eth0) is also part of this global external
bridge.

You can connect virtio NICs in the vMX to physical NICs or virtio NICs in another vMX by
binding these devices as shown in Figure 3 on page 52.

Figure 3: Binding Devices

52 Copyright 2017, Juniper Networks, Inc.

 Chapter 4: Managing vMX

To manage device bindings, perform these tasks:

Setting Up the Device Bindings on page 53

Creating Device Bindings on page 55
Deleting Device Bindings on page 55
Verifying Device Bindings on page 55

Setting Up the Device Bindings

The parameters required to configure vMX to bind devices are defined in the device-binding
file. The device-binding file is in YAML format. The default file is config/vmx-junosdev.conf.

The device-binding file defines the endpoints of each link originating from the VFP of a
vMX. One endpoint must be a device using virtio NICs. The other endpoint can be a physical
NIC, a virtio NIC in another vMX instance, or a Linux bridge.

To bind the vMX instances together:

1. Edit the config/vmx-junosdev.conf file to set up the communication between the vMX
instances.

2. Modify the link_name to the name of the Linux bridge (as shown by the brctl show
command). The link name can be 15 characters long. It must be unique for each bridge.
If more than two interfaces (virtual or physical) are connected by a Linux bridge, then
the bridge name is derived from the dev_name of the common endpoint for the
connected devices.

3. Specify the mtu to change the MTU value for virtio device types from the default of
1500. The maximum value is 9500.

To change the MTU configuration for SR-IOV device types, modify the mtu parameter
in the startup configuration file (vmx.conf).

4. Specify the endpoints for vMX devices (junos_dev type) by customizing these
parameters:

typeType of device is junos_dev.

vm-nameName of the vMX identifier specified in the startup configuration file for
that vMX instance.

dev-nameName of the interface on vMX as specified in the startup configuration

file.

5. Specify the endpoints for physical NICs (host_dev type) by customizing these
parameters:

typeType of device is host_dev.

dev-nameName of the physical NIC on the host.

6. Specify the endpoints for bridges (bridge_dev type) by customizing these parameters:

typeType of device is bridge_dev.

Copyright 2017, Juniper Networks, Inc. 53

Getting Started Guide for KVM

dev-nameName of the Linux bridge.

7. If you have multiple device-binding files, save them with different names.

Here is a sample vMX device-binding file:

interfaces :

- link_name : link_host
mtu : 1500
endpoint_1 :
- type : junos_dev
vm_name : vmx1
dev_name : ge-0/0/0
endpoint_2 :
- type : host_dev
dev_name : int2

- link_name : link_vmx_12
mtu : 1500
endpoint_1 :
- type : junos_dev
vm_name : vmx1
dev_name : ge-0/0/1
endpoint_2 :
- type : junos_dev
vm_name : vmx2
dev_name : ge-0/0/0

- link_name : bridge_vmx_123
endpoint_1 :
- type : junos_dev
vm_name : vmx1
dev_name : ge-0/0/2
endpoint_2 :
- type : bridge_dev
dev_name : bridge1

- link_name : bridge_vmx_123
endpoint_1 :
- type : junos_dev
vm_name : vmx2
dev_name : ge-0/0/1
endpoint_2 :
- type : bridge_dev
dev_name : bridge1

- link_name : bridge_vmx_123
endpoint_1 :
- type : junos_dev
vm_name : vmx3
dev_name : ge-0/0/0
endpoint_2 :
- type : bridge_dev
dev_name : bridge1

The link_host entry shows how to connect the ge-0/0/0 interface on vmx1 to the physical
NIC. The link_vmx_12 entry shows how to connect two interfaces on vmx1 and vmx2 to
each other. The bridge_vmx_123 entries show how to connect the interfaces on vmx1,
vmx2, and vmx3 with a bridge.

54 Copyright 2017, Juniper Networks, Inc.

 Chapter 4: Managing vMX

Creating Device Bindings

NOTE: You must be logged in as root to bind devices.

To bind devices with virtio NICs to other devices, define your devices in the vMX
device-binding file and run the ./vmx.sh --bind-dev -cfg device-binding-file script to create
the device binding. If you do not specify a file, the default file is config/vmx-junosdev.conf.

This example creates device bindings with the specified device-binding file:

./vmx.sh --bind-dev -cfg config/vmx1-junosdev.conf

Deleting Device Bindings

NOTE: You must be logged in as root to unbind devices.

To unbind devices, run the ./vmx.sh --unbind-dev -cfg device-binding-file script to delete
the device bindings created with the --bind-dev option. If you do not specify a file, the
default file is config/vmx-junosdev.conf.

This example deletes device bindings with the specified device-binding file:

./vmx.sh --unbind-dev -cfg config/vmx1-junosdev.conf

Verifying Device Bindings

NOTE: You must be logged in as root to bind devices.

To verify the status of device bindings created with the --bind-dev option, run the ./vmx.sh
--bind-check -cfg device-binding-file script. If you do not specify a file, the default file is
config/vmx-junosdev.conf.

This example verifies the status of the device bindings for the specified device-binding
file:

./vmx.sh --bind-check -cfg config/vmx1-junosdev.conf

Related Installing vMX for Different Use Cases on page 36

Documentation
Specifying vMX Configuration File Parameters on page 41

Copyright 2017, Juniper Networks, Inc. 55

Getting Started Guide for KVM

56 Copyright 2017, Juniper Networks, Inc.

CHAPTER 5

Configuring vMX Chassis-Level Features

Configuring the Number of Active Ports on vMX on page 57

Naming the Interfaces on page 57
Configuring the Media MTU on page 58
Enabling Performance Mode or Lite Mode on page 58
Tuning Performance Mode on page 59

Configuring the Number of Active Ports on vMX

You can specify the number of active ports for vMX. The default number of ports is 10,
but you can specify any value in the range of 1 through 23. You can change this number
if you want to limit the number of Ethernet interfaces in the VCP VM to match the number
of NICs added to the VFP VM.

To specify the number of active ports, configure the number of ports at the [edit chassis
fpc 0 pic 0] hierarchy level.

[edit]
user@vmx# set chassis fpc 0 pic 0 number-of-ports

Naming the Interfaces

vMX supports the following interface types:

Gigabit Ethernet (ge)

10-Gigabit Ethernet (xe)

100-Gigabit Ethernet (et)

By default, the interfaces come up as ge interfaces with 1 Gbps bandwidth in the Junos
OS configuration. The default port speed values for the interface types are 1 Gbps (ge),
10 Gbps (xe), and 100 Gbps (et). If you do not enable schedulers, the speed is only for
display purposes and is not enforced. If you enable schedulers, the transmit rate of the
port is limited to the speed unless it is overridden by the shaping rate in the CoS
configuration.

Copyright 2017, Juniper Networks, Inc. 57

Getting Started Guide for KVM

To specify the interface types, configure the interface type at the [edit chassis fpc 0 pic
0] hierarchy level.

[edit]
user@vmx# set chassis fpc 0 pic 0 interface-type (ge | xe | et)

Configuring the Media MTU

For vMX, you can configure the media MTU in the range 256 through 9500.

NOTE: For VMware, the maximum value is 9000.

You configure the MTU by including the mtu statement at the [edit interface
interface-name] hierarchy level.

[edit]
user@vmx# set interface ge-0/0/0 mtu bytes

Enabling Performance Mode or Lite Mode

vMX can be configured to run in two modes depending on the use case.

Lite modeNeeds fewer resources in terms of CPU and memory to run at lower
bandwidth.

Performance modeNeeds higher resources in terms of CPU and memory to run at

higher bandwidth.

NOTE: Performance mode is enabled implicitly by default.

When you enable performance mode, make sure you have configured the
proper number of vCPUs and memory for your VMs based on your use case.

You can explicitly enable performance mode by including the performance-mode

statement at the [edit chassis fpc 0] hierarchy level.

[edit]
user@vmx# set chassis fpc 0 performance-mode
If you are using paravirtualized network interfaces such as virtio (for KVM) or VMXNET3
(for VMware) for lab simulation use cases, you can disable performance mode by including
the lite-mode statement at the [edit chassis fpc 0] hierarchy level.

[edit]
user@vmx# set chassis fpc 0 lite-mode

NOTE: We recommend that you enable hyperthreading in BIOS. We

recommend that you verify the process with the vendor because different
systems have different methods to enable hyperthreading.

58 Copyright 2017, Juniper Networks, Inc.

 Chapter 5: Configuring vMX Chassis-Level Features

Related Tuning Performance Mode on page 59

Documentation

Tuning Performance Mode

To tune performance mode for unicast traffic, you can specify the number of Workers
dedicated to processing multicast and control traffic. You can specify any value in the
range of 0 through 15. The default of 0 specifies that all available Workers are used to
process all traffic.

The number of dedicated Workers specified in relation to the number of available Workers
results in the following behavior:

If the number of dedicated Workers is greater than or equal to the number of available
Workers, then all available Workers are used to process all traffic.

If the number of dedicated Workers is less than the number of available Workers, then
the first set of available Workers (equal to the specified number of dedicated Workers)
is used to process multicast and control traffic while the remaining available Workers
are used to process flow cache traffic.

To specify the number of dedicated Workers for processing multicast and control traffic,
configure the number of Workers at the [edit chassis fpc 0 performance-mode] hierarchy
level.
[edit]
user@vmx# set chassis fpc 0 performance-mode number-of-ucode-workers number-workers

NOTE: Changing the number of Workers reboots the FPC.

Related Enabling Performance Mode or Lite Mode on page 58

Documentation

Copyright 2017, Juniper Networks, Inc. 59

Getting Started Guide for KVM

60 Copyright 2017, Juniper Networks, Inc.

CHAPTER 6

Class of Service for vMX

CoS on vMX Overview on page 61

CoS Features and Limitations on vMX on page 62
Configuring Hierarchical CoS on vMX on page 63
Example: Configuring Hierarchical CoS on vMX on page 65
Packet Loss Priority and Drop Profiles on vMX on page 69
Managing Congestion Using Drop Profiles and Packet Loss Priorities on vMX on page 70

CoS on vMX Overview

vMX supports two-level hierarchical scheduling (per-unit scheduler or hierarchical

scheduler) with VLAN queuing. Each VLAN (logical interface) uses three traffic classes
and eight queues.

vMX supports shaping at the traffic class level, not at the queue level. A traffic class is a
bundle of queues with fixed priority. The next level in the hierarchy is the VLAN (logical
interface), which is a bundle of traffic classes.

vMX has the following fixed priorities and queues for these traffic classes:

Traffic Class 1: High (strict priority)

Queue 0

Queue 6

Traffic Class 2: Medium (strict priority)

Queue 1

Queue 7

Traffic Class 3: Low

Queue 2

Queue 3

Queue 4

Queue 5

Copyright 2017, Juniper Networks, Inc. 61

Getting Started Guide for KVM

NOTE: Both Traffic Class 1 and Traffic Class 2 follow strict priority, so all
excess traffic is discarded as tail drops. However, Traffic Class 3 does not
follow strict priority, so the shaping rate is set to the shaping rate of the VLAN.

All queues in the same traffic class have equal priority, so the scheduler pulls
packets from each queue in the traffic class based on weighted round robin
(WRR) for the VLAN.

All configured forwarding classes must be mapped to one of the queues.

These features are not supported:

Weighted random early detection (WRED)

Queue buffer size configuration

NOTE: No commit errors are displayed for unsupported features.

Related CoS Features and Limitations on vMX on page 62

Documentation
Packet Loss Priority and Drop Profiles on vMX on page 69

CoS Features and Limitations on vMX

vMX has the following limitations for CoS support:

Schedulers support only the transmit-rate and excess-rate statements. Only weights
are supported at the queue level, so transmission rate and excess rate are used for
calculating queue weights.

If transmit-rate percent is configured at the queue level, then configure guaranteed

rate at the VLAN level.

NOTE: Guaranteed rate is not supported, but it is used to calculate queue

weights.

If you only configure transmit rate, queue weights are calculated based on the
transmission rate.

If you only configure excess rate, queue weights are calculated based on the excess
rate.

If you configure both transmit rate and excess rate, queue weights are calculated
based on the excess rate.

If you configure the excess rate for one queue, the excess rate is expected for all the
queues to compute the weights. If the excess rate is not configured, the default
weight of 1 is used.

62 Copyright 2017, Juniper Networks, Inc.

 Chapter 6: Class of Service for vMX

NOTE: To get the expected behavior, you must configure the excess rate
for all queues.

Traffic control profiles support only the shaping-rate and scheduler-map statements.

If a traffic control profile has a default scheduler map, you must configure the
guaranteed rate.

For high- and medium-priority traffic classes, the transmission rate is the shaping rate.

For low-priority queues, the shaping rate for the VLAN is used for the queue. As a result,
the low-priority queues can burst up to the configured shaping rate for the VLAN. The
transmission rate is used as the WRR weight when there is more than one queue
configured for a given priority.

Some considerations for the high- and medium-priority traffic classes:

All excess traffic from the traffic classes for high- and medium-priority queues are
discarded as tail drops.

For high- and medium-priority traffic classes, the transmission rate is the shaping rate.

If the transmission rate is not configured and the shaping rate is configured, then the
queue weight is calculated based upon the configured shaping rate.

If you configure the transmission rate for both queues of the same traffic class, the
shaping rate of the traffic class is the sum of the individual transmission rates of the
queues for that traffic class.

If a queue is not configured, its transmission rate is set to zero.

If no queues are configured, the shaping rate of the VLAN is applied to the traffic class
as the transmission rate.

If any of the queues in the traffic class is configured, the shaping rate of the VLAN is
set to the guaranteed rate of the configured queue. If a queue is not configured, the
guaranteed rate is set to zero by default.

If the sum of the rates of the individual queues in a traffic class exceeds the shaping
rate of the VLAN, the shaping rate of the VLAN is used as the shaping rate of the traffic
class.

Related Configuring Hierarchical CoS on vMX on page 63

Documentation
CoS on vMX Overview on page 61

Configuring Hierarchical CoS on vMX

To configure hierarchical CoS, perform these tasks:

Enabling Flexible Queuing on page 64

Mapping Forwarding Classes to Queues on vMX on page 64

Copyright 2017, Juniper Networks, Inc. 63

Getting Started Guide for KVM

Configuring Traffic Control Profiles for vMX on page 64

Configuring Schedulers on vMX on page 64

Enabling Flexible Queuing

Hierarchical CoS is disabled by default. To enable hierarchical CoS, include the
flexible-queuing-mode statement at the [edit chassis fpc 0] hierarchy level and restart
the FPC.
[edit]
user@vmx# set chassis fpc 0 flexible-queuing-mode

Mapping Forwarding Classes to Queues on vMX

You must map all configured forwarding classes to one of the queues.
[edit]
user@vmx# set class-of-service forwarding-classes class class-name queue-num queue-number

Configuring Traffic Control Profiles for vMX

Traffic control profiles support only the shaping-rate and scheduler-map statements for
vMX.

To specify the shaping rate, include the shaping-rate statement at the [edit class-of-service
traffic-control-profiles profile-name] hierarchy level.

[edit]
user@vmx# set class-of-service traffic-control-profiles profile-name shaping-rate rate

To specify the scheduler map, include the scheduler-map statement at the [edit
class-of-service traffic-control-profiles profile-name] hierarchy level.

[edit]
user@vmx# set class-of-service traffic-control-profiles profile-name scheduler-map map-name

Configuring Schedulers on vMX

The scheduler map contains the mapping of forwarding classes to their schedulers. The
scheduler defines the properties for the queue.

Schedulers support only the transmit-rate and excess-rate proportion statements for
vMX.

To specify the transmission rate, include the transmit-rate statement at the [edit
class-of-service schedulers scheduler-name] hierarchy level.

[edit]
user@vmx# set class-of-service schedulers scheduler-name transmit-rate rate

BEST PRACTICE: Guaranteed rate is not supported, so there is no reserved

bandwidth for the VLAN. To get the expected behavior, we recommend that
you configure the transmit rate to be the guaranteed rate.

64 Copyright 2017, Juniper Networks, Inc.

 Chapter 6: Class of Service for vMX

To specify the proportion of the excess bandwidth to share, include the excess-rate
proportion statement at the [edit class-of-service schedulers scheduler-name] hierarchy
level. The value is in the range of 0 through 1000.
[edit]
user@vmx# set class-of-service schedulers scheduler-name excess-rate proportion value

If you configure the excess rate for one queue, the excess rate is expected for all the
queues to compute the weights. If the excess rate is not configured, the default weight
of 1 is used.

NOTE: To get the expected behavior, you must configure the excess rate for
all queues.

For example, if you configure excess rate for the low-priority queues, configure
the same excess rate for the high- and medium-priority queues.

Related Example: Configuring Hierarchical CoS on vMX on page 65

Documentation
CoS on vMX Overview on page 61

CoS Features and Limitations on vMX on page 62

Example: Configuring Hierarchical CoS on vMX

This example describes how to configure hierarchical CoS on vMX with eight queues.

Requirements on page 65
Overview on page 65
Configuration on page 65

Requirements
This example uses the following hardware and software components:

Junos OS Release 16.2

vMX Release 16.2

Overview
This example configures two-level hierarchical schedulers with specified transmission
rates.

Configuration
Configuring the Chassis on page 66
Applying Shaping and Scheduling to VLANs on page 66

Copyright 2017, Juniper Networks, Inc. 65

Getting Started Guide for KVM

Configuring the Chassis

CLI Quick [edit]

Configuration set chassis fpc 0 flexible-queuing-mode

Step-by-Step To enable hierarchical CoS on the chassis:

Procedure
1. Enable flexible queuing mode on the chassis.

[edit]
user@vmx# set chassis fpc 0 flexible-queuing-mode

2. Restart the FPC.

[edit]
user@vmx# run request chassis fpc restart slot 0

Applying Shaping and Scheduling to VLANs

CLI Quick [edit]

Configuration set class-of-service forwarding-classes class voice1 queue-num 0
set class-of-service forwarding-classes class video1 queue-num 1
set class-of-service forwarding-classes class data1 queue-num 2
set class-of-service forwarding-classes class data2 queue-num 3
set class-of-service forwarding-classes class data3 queue-num 4
set class-of-service forwarding-classes class data4 queue-num 5
set class-of-service forwarding-classes class voice2 queue-num 6
set class-of-service forwarding-classes class video2 queue-num 7
set interfaces ge-0/0/0 hierarchical-scheduler maximum-hierarchy-levels 2
set interfaces ge-0/0/0 vlan-tagging
set interfaces ge-0/0/0 unit 100 vlan-id 100
set interfaces ge-0/0/0 unit 100 family inet address 10.2.2.1/24
set interfaces ge-0/0/1 hierarchical-scheduler maximum-hierarchy-levels 2
set interfaces ge-0/0/1 vlan-tagging
set interfaces ge-0/0/1 unit 100 vlan-id 100
set interfaces ge-0/0/1 unit 100 family inet address 10.1.1.1/24
set class-of-service classifiers inet-precedence vlan_tos forwarding-class voice1 loss-priority
low code-points 000
set class-of-service classifiers inet-precedence vlan_tos forwarding-class video1 loss-priority
low code-points 001
set class-of-service classifiers inet-precedence vlan_tos forwarding-class data1 loss-priority
low code-points 010
set class-of-service classifiers inet-precedence vlan_tos forwarding-class data2 loss-priority
low code-points 011
set class-of-service classifiers inet-precedence vlan_tos forwarding-class data3 loss-priority
low code-points 100
set class-of-service classifiers inet-precedence vlan_tos forwarding-class data4 loss-priority
low code-points 101
set class-of-service classifiers inet-precedence vlan_tos forwarding-class voice2 loss-priority
low code-points 110
set class-of-service classifiers inet-precedence vlan_tos forwarding-class video2
loss-priority low code-points 111
set class-of-service traffic-control-profiles ge_0_0_1_vlan_100_tcp shaping-rate 50m
set class-of-service traffic-control-profiles ge_0_0_1_vlan_100_tcp scheduler-map
vlan_smap

66 Copyright 2017, Juniper Networks, Inc.

 Chapter 6: Class of Service for vMX

set class-of-service interfaces ge-0/0/1 unit 100 output-traffic-control-profile

ge_0_0_1_vlan_100_tcp
set class-of-service interfaces ge-0/0/0 unit 100 classifiers inet-precedence vlan_tos
set class-of-service scheduler-maps vlan_smap forwarding-class voice1 scheduler
sched_voice1
set class-of-service scheduler-maps vlan_smap forwarding-class video1 scheduler
sched_video1
set class-of-service scheduler-maps vlan_smap forwarding-class data1 scheduler
sched_data1
set class-of-service scheduler-maps vlan_smap forwarding-class data2 scheduler
sched_data2
set class-of-service scheduler-maps vlan_smap forwarding-class data3 scheduler
sched_data3
set class-of-service scheduler-maps vlan_smap forwarding-class data4 scheduler
sched_data4
set class-of-service scheduler-maps vlan_smap forwarding-class voice2 scheduler
sched_voice2
set class-of-service scheduler-maps vlan_smap forwarding-class video2 scheduler
sched_video2
set class-of-service schedulers sched_voice1 transmit-rate 15m
set class-of-service schedulers sched_video1 transmit-rate 15m
set class-of-service schedulers sched_data1 transmit-rate 5m
set class-of-service schedulers sched_data2 transmit-rate 5m
set class-of-service schedulers sched_data3 transmit-rate 5m
set class-of-service schedulers sched_data4 transmit-rate 5m
set class-of-service schedulers sched_voice2 transmit-rate 10m
set class-of-service schedulers sched_video2 transmit-rate 10m

Step-by-Step To apply shaping and scheduling:

Procedure
1. Map the forwarding classes to their respective queues.

[edit]
user@vmx# set class-of-service forwarding-classes class voice1 queue-num 0
user@vmx# set class-of-service forwarding-classes class video1 queue-num 1
user@vmx# set class-of-service forwarding-classes class data1 queue-num 2
user@vmx# set class-of-service forwarding-classes class data2 queue-num 3
user@vmx# set class-of-service forwarding-classes class data3 queue-num 4
user@vmx# set class-of-service forwarding-classes class data4 queue-num 5
user@vmx# set class-of-service forwarding-classes class voice2 queue-num 6
user@vmx# set class-of-service forwarding-classes class video2 queue-num 7

2. Configure the interfaces to enable two-level hierarchical scheduling and apply

scheduling to the VLANs.

[edit]
user@vmx# set interfaces ge-0/0/0 hierarchical-scheduler
maximum-hierarchy-levels 2
user@vmx# set interfaces ge-0/0/0 vlan-tagging
user@vmx# set interfaces ge-0/0/0 unit 100 vlan-id 100
user@vmx# set interfaces ge-0/0/0 unit 100 family inet address 10.2.2.1/24
user@vmx# set interfaces ge-0/0/1 hierarchical-scheduler
maximum-hierarchy-levels 2
user@vmx# set interfaces ge-0/0/1 vlan-tagging
user@vmx# set interfaces ge-0/0/1 unit 100 vlan-id 100
user@vmx# set interfaces ge-0/0/1 unit 100 family inet address 10.1.1.1/24

Copyright 2017, Juniper Networks, Inc. 67

Getting Started Guide for KVM

3. Configure the classifiers.

[edit]
user@vmx# set class-of-service classifiers inet-precedence vlan_tos
forwarding-class voice1 loss-priority low code-points 000
user@vmx# set class-of-service classifiers inet-precedence vlan_tos
forwarding-class video1 loss-priority low code-points 001
user@vmx# set class-of-service classifiers inet-precedence vlan_tos
forwarding-class data1 loss-priority low code-points 010
user@vmx# set class-of-service classifiers inet-precedence vlan_tos
forwarding-class data2 loss-priority low code-points 011
user@vmx# set class-of-service classifiers inet-precedence vlan_tos
forwarding-class data3 loss-priority low code-points 100
user@vmx# set class-of-service classifiers inet-precedence vlan_tos
forwarding-class data4 loss-priority low code-points 101
user@vmx# set class-of-service classifiers inet-precedence vlan_tos
forwarding-class voice2 loss-priority low code-points 110
user@vmx# set class-of-service classifiers inet-precedence vlan_tos
forwarding-class video2 loss-priority low code-points 111

4. Configure the traffic control profiles.

[edit]
user@vmx# set class-of-service traffic-control-profiles ge_0_0_1_vlan_100_tcp
shaping-rate 50m
user@vmx# set class-of-service traffic-control-profiles ge_0_0_1_vlan_100_tcp
scheduler-map vlan_smap

5. Map the traffic control profiles to their respective interface.

[edit]
user@vmx# set class-of-service interfaces ge-0/0/1 unit 100
output-traffic-control-profile ge_0_0_1_vlan_100_tcp
user@vmx# set class-of-service interfaces ge-0/0/0 unit 100 classifiers
inet-precedence vlan_tos

6. Configure the scheduler maps.

[edit]
user@vmx# set class-of-service scheduler-maps vlan_smap forwarding-class voice1
scheduler sched_voice1
user@vmx# set class-of-service scheduler-maps vlan_smap forwarding-class video1
scheduler sched_video1
user@vmx# set class-of-service scheduler-maps vlan_smap forwarding-class data1
scheduler sched_data1
user@vmx# set class-of-service scheduler-maps vlan_smap forwarding-class data2
scheduler sched_data2
user@vmx# set class-of-service scheduler-maps vlan_smap forwarding-class data3
scheduler sched_data3
user@vmx# set class-of-service scheduler-maps vlan_smap forwarding-class data4
scheduler sched_data4
user@vmx# set class-of-service scheduler-maps vlan_smap forwarding-class voice2
scheduler sched_voice2
user@vmx# set class-of-service scheduler-maps vlan_smap forwarding-class video2
scheduler sched_video2

7. Configure the schedulers.

[edit]

68 Copyright 2017, Juniper Networks, Inc.

 Chapter 6: Class of Service for vMX

user@vmx# set class-of-service schedulers sched_voice1 transmit-rate 15m

user@vmx# set class-of-service schedulers sched_video1 transmit-rate 15m
user@vmx# set class-of-service schedulers sched_data1 transmit-rate 5m
user@vmx# set class-of-service schedulers sched_data2 transmit-rate 5m
user@vmx# set class-of-service schedulers sched_data3 transmit-rate 5m
user@vmx# set class-of-service schedulers sched_data4 transmit-rate 5m
user@vmx# set class-of-service schedulers sched_voice2 transmit-rate 10m
user@vmx# set class-of-service schedulers sched_video2 transmit-rate 10m

Related Configuring Hierarchical CoS on vMX on page 63

Documentation

Packet Loss Priority and Drop Profiles on vMX

vMX handles packet priorities within a queue by assigning a threshold to each loss priority
within a queue and dropping new packets of that loss priority level when the queue depth
exceeds the threshold. When the queue becomes oversubscribed, packets of lower
priority are dropped to ensure that there is room in the queue for packets of higher priority.

Packet loss priority has four loss priority levels:

low

medium-low

medium-high

high

vMX supports three thresholds so the medium-low and medium-high loss priority levels
are grouped together. vMX maps the packet loss priority to tricolor marking as follows:

Packet Loss Priority Color

low green

medium-low yellow

medium-high yellow

high red

vMX drop profiles define the threshold within a queue for a given loss priority as the fill
level value associated with the drop probability of 100 percent. If you do not specify a
drop probability of 100 percent in the drop profile, the threshold defaults to 100 percent.
All other fill level values are ignored. These drop profiles can be referenced by the
scheduler to evaluate packets with different loss priority settings.

You can set packet loss priority for packets using behavior aggregate (BA) classifiers,
firewall filters, or firewall policers.

Copyright 2017, Juniper Networks, Inc. 69

Getting Started Guide for KVM

Limitations
vMX has the following limitations for supporting drop profiles and packet loss priority:

If you do not apply drop profiles to the queue, then packets are tail dropped.

The show interface queue command does not display separate drop rates for the
medium-high PLP and medium-low PLP because they both map to yellow. All yellow
drop rates appear as medium-high drops.

Related Managing Congestion Using Drop Profiles and Packet Loss Priorities on vMX on page 70
Documentation
CoS on vMX Overview on page 61

Managing Congestion Using Drop Profiles and Packet Loss Priorities on vMX

When you are configuring CoS, you can manage congestion by configuring drop profiles
to specify the thresholds for packet loss priority. You reference the drop profiles in the
scheduler configuration to assign a drop profile to the loss priority setting.

To configure how packet loss priority is handled for queues, perform these tasks:

Configuring Drop Profiles on page 70

Configuring Schedulers with Drop Profiles on page 71

Configuring Drop Profiles

Drop profiles specify the threshold for a given loss priority.

NOTE: The threshold for the loss priority assigned this drop profile is the
fill-level value associated with the drop-probability of 100. If you do not specify
a drop probability of 100 percent in the drop profile, the fill level defaults to
100 percent. All other fill levels are ignored.

To specify the drop profile, include the drop-profiles statement at the [edit
class-of-service] hierarchy level.

[edit]
user@vmx# set class-of-service drop-profiles profile-name

To specify the threshold for the loss priority, include the fill-level and drop-probability
statements at the [edit class-of-service drop-profiles profile-name] hierarchy level.

[edit class-of-service drop-profiles profile-name]

user@vmx# set fill-level percentage drop-probability percentage

For example, the dpLow drop profile specifies a threshold of 100 percent, the dpMed drop
profile specifies a threshold of 75 percent, and the dpHigh drop profile specifies a threshold
of 50 percent.
[edit]

70 Copyright 2017, Juniper Networks, Inc.

 Chapter 6: Class of Service for vMX

user@vmx# set class-of-service drop-profiles dpLow fill-level 100 drop-probability 100

user@vmx# set class-of-service drop-profiles dpMed fill-level 75 drop-probability 100
user@vmx# set class-of-service drop-profiles dpHigh fill-level 50 drop-probability 100

Configuring Schedulers with Drop Profiles

The drop profile map contains the mapping of loss priority and protocol type to configured
drop profiles. You can associate multiple drop profile maps with a scheduler.

NOTE: If you do not apply drop profiles to the queue, then packets are tail
dropped.

To specify the drop profile map, include the drop-profile-map statement at the [edit
class-of-service schedulers scheduler-name] hierarchy level.

[edit class-of-service schedulers scheduler-name]

user@vmx# set drop-profile-map loss-priority (any | low | medium-low | medium-high |
high) protocol any drop-profile profile-name

For example, the sched-be scheduler applies the dpLow drop profile to packets with low
loss priority for any protocol type, applies the dpMed drop profile to packets with
medium-high loss priority for any protocol type, and applies the dpHigh drop profile to
packets with high loss priority for any protocol type.
[edit class-of-service schedulers sched-be]
user@vmx# set drop-profile-map loss-priority low protocol any drop-profile dpLow
user@vmx# set drop-profile-map loss-priority medium-high protocol any drop-profile dpMed
user@vmx# set drop-profile-map loss-priority high protocol any drop-profile dpHigh

Related Packet Loss Priority and Drop Profiles on vMX on page 69

Documentation

Copyright 2017, Juniper Networks, Inc. 71

Getting Started Guide for KVM

72 Copyright 2017, Juniper Networks, Inc.

CHAPTER 7

Troubleshooting vMX

Viewing VFP Statistics on page 73

Viewing VFP Log Files on page 74
Troubleshooting VFP and VCP Connection Establishment on page 75
Verifying Whether VMs Are Running on page 76
Viewing CPU Information on page 76
Verifying BIOS Settings for SR-IOV on page 77

Viewing VFP Statistics

You can view the VFP statistics from a Web browser. The displayed statistics are not
absolute counters; they are relative to the start of the HTTP session and start as all zero
counters.

The RPIO Stats and Hostif Stats sections provide statistics about the internal
communication between the VCP and the VFP. The RPIO session uses ports 3000 and
3001 and the HostIF session uses port 3002.

The Port Stats section provides statistics about the packets received from and transmitted
to the NIC interfaces.

There is a receive (rx) and transmit (tx) line for each port. Port 0 maps to the ge-0/0/0
interface, port 1 maps to the ge-0/0/1 interface, and so forth. rx0 displays statistics for
packets received from port 0 and tx1 displays statistics for packets transmitted to port
1.

Errors are miscellaneous errors reported by the physical layer NIC.

The Ring Stats section provides statistics about packet processing.

There is an I/O thread (io) for packets received from a port.

There is a Worker thread (wk) for each CPU core.

Copyright 2017, Juniper Networks, Inc. 73

Getting Started Guide for KVM

The host interface (host) sends protocol packets to the VCP.

The queue processes the packets. The columns provide this information about the
queues:

The Producer and Consumer columns display the source and destination that
generate packets for this queue. The values can be io, wk, tx, or host.

The Priority column displays the priority of the queue. The values can be Normal or
High (only for control packets).

The Free and Used columns display the queue occupancy. The queue has 1024
entries.

The Enqueues and Dequeues columns display the number of queue operations.

The Drops column indicates whether the queue is being drained fast enough.

To view the statistics:

1. By default, you cannot log in to the Web browser window without configuring the
username and password credentials and enabling HTTP access.

From the VFP console, configure the username and password by invoking the
/home/pfe/riot/vfp_util.sh -setpass command.

root@vfp-vmx1:/home/pfe/riot# ./vfp_util.sh -setpass

Enter new Username: pfe

Enter new Password:
Re-enter Password:

Password successfully changed

root@vfp-vmx1:/home/pfe/riot#

To enable HTTP access, invoke this command.

root@vfp-vmx1:/home/pfe/riot# ./vfp_util.sh -http_enable

2. Navigate to http://vfp-mgmt-ip:8080/, where vfp-mgmt-ip is the management IP

address for the VFP VM.

3. When prompted, enter pfe as the username and the password configured in Step 1.

4. View the statistics displayed in the browser window.

5. After troubleshooting, you can disable HTTP access to improve security with this
command:

root@vfp-vmx1:/home/pfe/riot# ./vfp_util.sh -http_disable

Viewing VFP Log Files

The VFP saves the following files:

VFP log files are saved in the /var/log directory.

VFP crash files are automatically saved in the VCP /var/crash directory.

74 Copyright 2017, Juniper Networks, Inc.

 Chapter 7: Troubleshooting vMX

To view the VFP log or crash files:

1. Log in to the VFP console by using the ./vmx.sh --console vfp vmx-id command, where
vmx-id is the vMX identifier specified in the startup configuration file.

2. Navigate to the appropriate directory to determine whether there are any files to view.

# cd /var/crash
# ls -l
-rwxr-xr-x 1 root root 864678 Jan 4 02:14 core.riot.1420366466.8271.gz

3. (Optional) If necessary, unzip the file and view it using GDB.

# gunzip core.riot.1420366466.8271.gz
# gdb /build/app core.riot.1420366466.8271

The VFP is configured for remote logging of the /var/log/messages directory. You can
configure the VCP syslog facility to record the VFP log messages:

user@vmx# set system syslog file messages any any

user@vmx# set system syslog server routing-instances all
user@vmx# set services app-engine monitor-cpu 50 100
user@vmx# commit

Related Connecting to VMs on page 44

Documentation

Troubleshooting VFP and VCP Connection Establishment

Purpose When the VCP and VFP connection is established, the show interfaces terse command
in the VCP CLI displays the ge-0/0/x interfaces and the following messages appear in
the VFP syslog file:
RPIO: Accepted connection from 128.0.0.1:50896 <-> vPFE:3000
RPIO: Accepted connection from 128.0.0.1:56098 <-> vPFE:3000
HOSTIF: Accepted connection

If the VCP cannot connect to the VFP, the VFP syslog file does not display the RPIO and
HOSTIF messages.

Action Run the request chassis fpc slot 0 restart command from the VCP CLI. If an FPC is in
transition error message is displayed, then run restart chassis-control.

If these commands do not correct the problem, verify whether the VCP can ping the VFP
from the routing-instance __juniper_private1__. The three management interfaces (for the
host, VCP VM, and VFP VM) connected to the internal bridge should be able to reach
each other. For example:
root> ping 128.0.0.16 routing-instance __juniper_private1__
PING 128.0.0.16 (128.0.0.16): 56 data bytes
64 bytes from 128.0.0.16: icmp_seq=0 ttl=64 time=0.273 ms
64 bytes from 128.0.0.16: icmp_seq=1 ttl=64 time=0.606 ms

Copyright 2017, Juniper Networks, Inc. 75

Getting Started Guide for KVM

If the VCP cannot ping the VFP, perform these tasks:

1. Use the brctl show command to verify the bridge configuration and connected
interfaces.

2. Verify that the startup configuration file is correct.

3. Verify that the VFP and the VCP VMs are up and the correct IP addresses are assigned.

4. Restart the FPC from the VCP VM.

5. Restart the chassis management process from VCP VM.

6. Stop and start the VFP VM.

7. Stop and start the VCP VM.

8. Restart the host.

If the problem persists, contact the Juniper Networks Technical Assistance Center (JTAC).

Related Connecting to VMs on page 44

Documentation

Verifying Whether VMs Are Running

To verify that the VMs are running after vMX is installed, use the virsh list command. The
virsh list command displays the name and state of the VM. The state can be: running,
idle, paused, shutdown, crashed, or dying.

You can stop and start VMs with the following virsh commands.

virsh destroyForcefully stop a VM while leaving its resources intact.

virsh startStart an inactive VM that was defined previously.

Related Connecting to VMs on page 44

Documentation

Viewing CPU Information

On the host server, use the lscpu command to display CPU information. The output
displays such information as the total number of CPUs, the number of cores per socket,
and the number of CPU sockets. For example:

root@vmx-host:~# lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 40
On-line CPU(s) list: 0-39
Thread(s) per core: 1
Core(s) per socket: 10
Socket(s): 4
NUMA node(s): 4
Vendor ID: GenuineIntel

76 Copyright 2017, Juniper Networks, Inc.

 Chapter 7: Troubleshooting vMX

CPU family: 6
Model: 62
Stepping: 7
CPU MHz: 3191.766
BogoMIPS: 6385.87
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 38400K
NUMA node0 CPU(s): 0,4,8,12,16,20,24,28,32,36
NUMA node1 CPU(s): 1,5,9,13,17,21,25,29,33,37
NUMA node2 CPU(s): 2,6,10,14,18,22,26,30,34,38
NUMA node3 CPU(s): 3,7,11,15,19,23,27,31,35,39

Related Minimum Hardware and Software Requirements on page 25

Documentation

Verifying BIOS Settings for SR-IOV

If you are having problems with the SR-IOV ports, make sure BIOS has the following
settings:

SR-IOV is enabled.

VT-d is enabled.

Hyperthreading is enabled.

We recommend that you verify the process with the vendor because different systems
have different methods to access and change BIOS settings.

Related Preparing the Ubuntu Host to Install vMX on page 29

Documentation

Copyright 2017, Juniper Networks, Inc. 77

Getting Started Guide for KVM

78 Copyright 2017, Juniper Networks, Inc.