0% found this document useful (0 votes)

184 views

Administration Guide

Uploaded by

samyxs

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

184 views

Administration Guide

Uploaded by

samyxs

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 505

VMware Avi Load

Balancer Administration
Guide
VMware Avi Load Balancer 30.2
VMware Avi Load Balancer Administration Guide

You can find the most up-to-date technical documentation on the VMware by Broadcom website at:

https://docs.vmware.com/

VMware by Broadcom
3401 Hillview Ave.
Palo Alto, CA 94304
www.vmware.com

©
Copyright 2024 Broadcom. All Rights Reserved. The term “Broadcom” refers to Broadcom Inc. and/or its
subsidiaries. For more information, go to https://www.broadcom.com. All trademarks, trade names, service
marks, and logos referenced herein belong to their respective companies.

VMware by Broadcom 2
Contents

About This Guide 8

1 Web Interface Access Settings 9

2 CLI Access Settings 15

Access the Controller CLI 15
CLI Options 16
Install the Avi Load Balancer CLI Shell 20
CLI - Linux Command Line Mode 23
CLI - Script Mode 25
CLI Top-Level Commands 26
Common Avi Load Balancer CLI Tasks 36
CLI and Shell Prompts Available for Avi Load Balancer 42
Access Using Remote Authentication 43
Accessing Avi Load Balancer Linux CLI as a Non-Admin User Using an SSH client 43
Instructions 43
Adding Required HMAC to the Allowed HMACs Lists using Avi Load Balancer UI 45
Securing Management IP Access 46
Certificates 52
Changing the Default Certificate of the Controller 52
Renew Default (Self-Signed) Certificates on Avi Load Balancer 53
Configure Stronger SSL Cipher 55
User Interface Inaccessible Due to Wrong Certificate 56
Import Encrypted Private Key to Install an SSL Certificate 57
Venafi Integration 59
Integrating Let's Encrypt Certificate Authority with Avi Load Balancer 62
Certificate Management Integration for CSR Automation 68
SSL Certificates 71

3 Licensing 89
Avi Load Balancer Editions 89
Avi Load Balancer Enterprise with Cloud Services Edition 92
Avi Load Balancer-Enterprise Edition 93
Avi Load Balancer - Basic Edition 111
Avi Load Balancer Essentials for Tanzu 117
Terms of Avi Load Balancer Software License 121
Removing Trial Licenses from an Avi Load Balancer Controller 122

VMware by Broadcom 3
VMware Avi Load Balancer Administration Guide

Updating License on each Node in an Avi Load Balancer Controller Cluster 124

4 Controller Settings 125

DNS/ NTP Settings 125
Time synchronization for the Service Engine 132
Email-SMTP Settings 134

5 Avi Load Balancer Controller Cluster 138

Deploying an Avi Load Balancer Controller Cluster 142
Using the UI 142
Using the CLI 142
Deploying an Avi Load Balancer Controller Cluster (Additional) 143
Deploy SEs in Different Data Centers from Controllers 145
Clustering Patched Controllers 145
Controller Cluster IP 146
Cluster IP Advertisement 147
OpenStack Controller Cluster 147
Configure the Cluster IP Using the UI 150
Assigning Avi Load Balancer Controller IP in Different Deployments 150
Cluster Configuration with DNS Hostnames 152
Changing Avi Load Balancer Controller Cluster Configuration 153
Remove both Followers (Dismantling the Cluster) Using the UI 153
Change a Follower Node Using the UI 154
Replace the Leader Node 156
Replace a Follower Node with a New Node 156
Updating the Configuration Following Avi Load Balancer Controller IP Address Change 157
Configure the Script 157
Updating IP Information for a Single-node Deployment 158
Updating IP Information for an Avi Load Balancer Controller Cluster 158
Change Controller IPs on Nutanix Cluster 159
Customizing Avi Load Balancer Configuration 159
Clustering Avi Load Balancer Controllers of Different Networks 165
High Availability for Avi Load Balancer Controllers 166
Operation of Avi Load Balancer Controller HA 166
Converting a Single-Node Deployment to a 3-node Cluster 167
Control Plane High Availability 168
Operation of Avi Load Balancer Controller High Availability 168
Converting a Single-Node Deployment to a Three-node Cluster 170
Switching Among Controllers Using the Controller Site Selector 171
Controller Cluster Deployment Across Two Availability Zones 175
Impact of a Controller Failure 177

VMware by Broadcom 4
VMware Avi Load Balancer Administration Guide

Recover a Non-Operational Controller Cluster 178

API - Configuring the Avi Load Balancer Controller Cluster 179
Avi Load Balancer ControllerCluster Configuration in AWS 182
Configure Azure Cluster IP 183
Changing Controller IP Addresses in a Bare-metal Environment 184
OpenStack Network Configuration for Avi Load Balancer Controller Cluster 185
FAQs on Controller Cluster 189
FAQs on Database Replication 192

6 Tenant Settings 195

Create a Tenant 196
Add an Existing User to a Tenant 197
Enable Use Single Role for All Tenants 197
Configuring Tenant Settings 197
Using the UI 198
Using the CLI 198
Switching Between Tenants 199
Sharing Certificates Across Tenants 200
Tenant-Scoped Clouds 209
All Tenants View 211
Using the UI 211
Using the CLI 211
Using the API 211
Tenants Versus SE Group Isolation 212
Sharing Admin Profiles Across Tenants 213

7 User Authentication and Authorization 215

User Accounts 216
User Profile 224
User Account Management 229
User Roles 229
Create a Role 230
Assign a Role to a Local User Account 232
Manage User Roles 233
HTTP Basic Authentication for API Queries 233
Token Authentication 234
Access the CLI using the Token 236
Remote Authentication 237
Authentication Profile 238
LDAP Authentication Profile 238
TACACS+ Authentication Profile 249

VMware by Broadcom 5
VMware Avi Load Balancer Administration Guide

SAML Authentication Profile 261

Troubleshooting 277
Auth Mapping Profile 278
Tenant and Role Mapping Examples 282
User Account Security 316
User Account Self-Service 316
User Account Lockout 317
User Credentials Timeout 318
Strong Default Admin Password 319
Strong Password Enforcement 320
Password History Enforcement 321
Extended Granular RBAC 324
Using Markers 325
Supported Objects 328
Label-based Permissions for Cloud Objects 329
Unsupported Permissions 330
Caveats and Limitations 330
Granular Role-Based Access Controls Per Field 330

8 Avi Load Balancer Controller Operations 333

Backup and Restore of Avi Load Balancer Configuration 335
Backing Up the Avi Load Balancer Configuration 336
Upgrade and Patches 340
Upgrade Overview 341
Patch Overview 342
Prerequisites 342
Determine Software Version 345
Pre/Post Upgrade Object Operational State Verification 346
Upgrade Guide 352
Upgrading SE Group 408
Patch Guide 413
Rollback 427
Rollback - Patch 428
Reset to Factory Defaults 430
Locking a Linux System to a Specific OS Version 432
Increasing Docker Container Size 433

9 Avi Conversion Tool Appliance 436

Deploying Avi Conversion Tool Appliance 439
Accessing Avi Conversion Tool 440
Avi Conversion Tool CLI Access 440

VMware by Broadcom 6
VMware Avi Load Balancer Administration Guide

Using Avi Conversion Tool 444

Using Avi Conversion Tool in Offline Mode 464
iRules Automatic Translation 469

10 Testing Avi Load Balancer with Load Generators 478

11 Maintaining Avi Load Balancer Controller Single Node Cluster Availability using
VMware 480

12 Avi Load Balancer SaaS 486

Configuring Avi Load Balancer SaaS Controller Using Service Account 491
Accessing the Avi Load Balancer SaaS REST API 493
Accessing the Avi Load Balancer SaaS CLI Using SSH 495

13 SAML Support for Avi Load Balancer SDK 496

14 Avi Load Balancer Go SDK Support 500

VMware by Broadcom 7
About This Guide

The Avi Load Balancer Administration Guide provides information about configuring and
managing networking for VMware Avi Load Balancer (formerly known as Avi Vantage). This
guide helps you set up user accounts and manage roles and permissions, and understand how to
protect the network from unauthorized users. This guide also discusses how to control, manage,
and monitor applications within Avi Load Balancer and control user access to objects. Further,
you can schedule periodic backups and restore data in case of an unlikely disaster.

Intended Audience
This information is intended for administrator users who want to configure Avi Load Balancer.
The information is written for experienced system administrators who are familiar with virtual
machine technology, networking, and security operations.

VMware Technical Publications Glossary

VMware Technical Publications provides a glossary of terms that might be unfamiliar to you. For
definitions of terms as they are used in VMware technical documentation, see VMware Glossary.

VMware by Broadcom 8
Web Interface Access Settings
1
Access Settings allows users to view or edit settings mainly related to accessing the Avi Load
Balancer Controller from the outside.

Navigate to Administration > System Settings > EDIT > Access.

VMware by Broadcom 9
VMware Avi Load Balancer Administration Guide

The following options are classified as System Access Settings:

n Enable HTTP Access to System: It allows HTTP access to the Avi Load Balancer web
interface and REST API. This option is insecure and not recommended.

VMware by Broadcom 10
VMware Avi Load Balancer Administration Guide

n Enable HTTPS Access to System: Enables SSL/TLS access to Avi Load Balancer GUI and
REST API. When the option is enabled, the SSL Profile and SSL/TLS Certificate fields must be
populated.

n Redirect HTTP to HTTPS: When HTTP Access to System is disabled, enabling this option will
automatically redirect administrators to the HTTPS interface for the web interface and API.

n SSL Profile: Select an SSL Profile to complete the HTTPS Access. This profile is from
Templates > Security > SSL Profiles, which is also referenced by SSL-enabled virtual services.

n SSL/TLS Certificate: Select an SSL certificate from SSL/TLS Certificate drop-down menu to
present to clients connecting to the web interface. RSA and Elliptic Curve (EC) are supported.

n Allow Basic Authentication: Uses HTTP to prompt the Avi Load Balancer user for a
username and password and to return the values to Avi Load Balancer for authentication
and authorization.

n Allowed Ciphers: List of the ciphers supported for HTTP basic authentication.

n Allowed HMACs: List of the HMACs supported for HTTP basic authentication.

The following option is related to SNMP:

n SNMP: Select None, SNMP V2, or SNMP V3 as required. The string to be furnished by an
external SNMP v2c manager wishing to query the SNMP daemon running on the Avi Load
Balancer Controller leader. For more information, see SNMP Support in Avi Load Balancer.

The Client Management Access to Avi Load Balancer Controller lists four different client types
that are authorized system users:

1 SSH Clients

2 CLI Shell Clients

3 External HTTP(S) Clients

4 External SNMP Clients

Note Enter the controller management IPs for HTTP(s) settings using the field Allowed External
HTTP(S) Clients. Internal Analytics APIs will fail if the management IPs of the cluster nodes are not
included in the list of IPs allowed for external HTTP(s).

Each one can flexibly specify clients by IP address and/or string/IP groups.

The following options govern two Banners that Avi Load Balancer will display if set:

n Message of the Day: Displayed to users after a successful login, be it through the UI, CLI, or
SSH.

n Login Banner: Displayed before logging in through SSH or UI.

Read the following topics next:

n Login Banner and Message of the Day

VMware by Broadcom 11
VMware Avi Load Balancer Administration Guide

Login Banner and Message of the Day

This section discusses the steps to configure Message of the Day and Login Banner.

Avi Load Balancer supports the configuration of the following types of management greeting
messages:

Message of the Day

A greeting that appears to users who log in through the Avi Load Balancer UI, SSH, or CLI.

Message that appears before login through the web interface or CLI.

Configure Message of the Day and Login Banner

1 Navigate to Administration > System Settings.

2 Click the edit icon next to System Settings to display the EDIT SYSTEM SETTINGS pop-up
window.

3 In the Message of the Day and Login Banner fields, enter the text messages to be displayed
after login through the web interface, SSH, or CLI.

4 Click Save.

The changes done in the previous steps will reflect in the next login session. Logout from the Avi
Load Balancer UI, and login again to see the changes reflected as shown below

VMware by Broadcom 12
VMware Avi Load Balancer Administration Guide

VMware by Broadcom 13
VMware Avi Load Balancer Administration Guide

VMware by Broadcom 14
CLI Access Settings
2
When accessing the Avi Load Balancer CLI, an administrator needs to SSH through port 22 to the
IP address of an Avi Load Balancer Controller or the cluster IP.

Avi Load Balancer Controller has two levels of CLI access. The first level of access is when the
administrator connects to the Avi Load Balancer Controller and logs in, and they are granted
access to a Linux Bash shell. The admin can then access the Avi Load Balancer CLI shell through
a procedure in the Access the Controller CLI section. In some authentication modes, non-admin
accounts cannot access Linux, and are instead forwarded directly to the shell.

Read the following topics next:

n Access the Controller CLI

n CLI Options

n CLI and Shell Prompts Available for Avi Load Balancer

n Access Using Remote Authentication

n Accessing Avi Load Balancer Linux CLI as a Non-Admin User Using an SSH client

n Adding Required HMAC to the Allowed HMACs Lists using Avi Load Balancer UI

n Securing Management IP Access

n Certificates

Access the Controller CLI

Local admin accounts default to Linux bash. In this case, enter the Avi Load Balancer shell by
typing shell.

username@avi:~$ | shell

To exit the shell, enter in to Linux, type exit.

: > | bash

VMware by Broadcom 15
VMware Avi Load Balancer Administration Guide

To switch between Linux Bash and Shell, type exit.

username@avi:~$ | exit

Note The SE can be accessed using CLI. However, this is not recommended except for
troubleshooting.

CLI Options
Avi Load Balancer can be managed through GUI, RESTful API, or CLI. Both the GUI and CLI are
built on top of the API, which means every CLI command maps to a corresponding API call (or
calls) to be run.

Interfacing to Avi Load Balancer through CLI Commands

In Avi Load Balancer, there are three entities with which the authorized user can have a CLI-
based conversation:

1 A Controller’s Linux operating system

2 Avi Load Balancer processes running on a Controller

3 An SE’s Linux operating system

Conversations with the first two entities are common. Conversations of the third kind
are infrequent and typically undertaken by customer support personnel for troubleshooting
purposes.

CLI Conversations with the Controller’s Operating System

Below is a command-line example:

The one argument (user@hostname) passed to the ssh command is a combination of admin and
the Controller’s IP address, 10.144.130.195. Every Avi Load Balancer Controller recognizes the
admin user; others can be defined if necessary. The response to the password prompt is not
echoed. However, in this example, it is represented by XXXXX.

The bash command-line interpreter gives access to the Controller’s underlying operating
system and file system. One use case would be to analyze various logs in the /opt/avi/log
and /var/log/upstart directories.

$>ssh [email protected]
Avi Cloud Controller
Avi Networks software, Copyright (C) 2013-2017 by Avi Networks, Inc.
All rights reserved.
Management: 100.65.9.10/20 UP
Gateway: 100.65.15.254 UP
[email protected]@100.65.9.10
Avi Cloud Controller
Avi Networks software, Copyright (C) 2013-2017 by Avi Networks, Inc.
All rights reserved.

VMware by Broadcom 16
VMware Avi Load Balancer Administration Guide

Management: 100.65.9.10/20 UP
Gateway: 100.65.15.254 UP
[email protected]@100.65.9.10
Avi Cloud Controller
Avi Networks software, Copyright (C) 2013-2017 by Avi Networks, Inc.
All rights reserved.
Management: 100.65.9.10/20 UP
Gateway: 100.65.15.254 UP
[email protected]'s password:
The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under
license. Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or the GNU
Lesser General Public License (LGPL) Version 2.1. A copy of each
such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://www.opensource.org/licenses/lgpl-2.1.php
Last login: Wed Apr 5 16:24:41 2023 from 100.65.1.5
admin@100-65-9-10:~$

CLI Conversations with Avi Load Balancer Processes Running on the

Controller
Avi Load Balancer-specific commands are not directly accessible from the Controller’s bash
interface.

Enter the Avi Load Balancer shell command-line interpreter by typing the shell command in
response to the bash prompt and input the login credentials as follows:

shell
Login: admin
Password: XXXXX

In response to the shell prompt, press the Tab key twice to reveal the commands specific to Avi
Load Balancer. In the CLI snippet shown below, TABTAB represents the Tab key pressed twice.

TABTAB
attach forcedelete reprogram terminal
clear gslb resync test
configure import retryplacement upgrade
controller migrate rollback upload
convert nsx rotatekeys upload_to_avi
debug passwd scalein verifylogin
delete reboot scaleout vinfra
do rediscover show watch
exec redistribute switchover
export reimage switchto

The show command helps analyze various issues and collect information. For example, show
virtualservice my-vs displays essential data about the virtual service named my-vs.

For insight into what the aforementioned commands do, see CLI Top-Level Commands.

VMware by Broadcom 17
VMware Avi Load Balancer Administration Guide

To exit the Avi Load Balancer shell and return to the Controller’s bash prompt, type bash.

bash

CLI Conversations with an Avi Load Balancer SE’s Linux Operating

System
Note While it is possible to access a SE's CLI directly, it must only be used for basic
troubleshooting. All configuration management must instead be done from and by the Controller.

SSH'ing to a SE or running the attach serviceengine name-of-service-engine command from

the Avi Load Balancer shell places one in the SE's Linux CLI. There is no equivalent to the
Controller's shell prompt; there are no show commands in the SE CLI. Use the SE's CLI to look into
SE-specific logs in various directories, such as /opt/avi/log.

Navigation and Help

After entering into the Avi Load Balancer shell, press the TAB key twice to see a list of available
commands.

n While typing a command, pressing the TAB key auto-completes the command. Double TAB
returns a list of available options for the command in the left column. Most options include a
brief help description, which is shown in the right column.

export configuration
export configuration serviceengine
export serviceengine ova file from controller virtualservice
export virtual service

n Commands or parameters might require multiple words or options. If there is only a single
word or option, pressing TAB auto-completes the next word in the command.

export configuration [TAB]

export configuration file [TAB]
WORD (required)
export configuration file mybackup
Completed writing the export configuration to mybackup

Other navigational commands:

n The up-arrow key cycles through and enables the reuse of previously run commands.

n The history command presents commands in a list format.

n Pipe filters results, as in

n | grep address

n | more

These are useful in watch commands too.

VMware by Broadcom 18
VMware Avi Load Balancer Administration Guide

Sub-mode Navigation
Many Avi Load Balancer CLI commands contain sub-modes, which are nested sub-sections of the
current command.

To enter the sub-mode, enter the relevant command. Within the context of a sub-mode, changes
are not committed until explicitly saved. Type save to exit the sub-mode while committing
changes.

To exit the sub-mode without saving changes, type cancel. When in a sub-mode or a nested
sub-mode, the command prompt will change to reflect the current sub-mode.

debug virtualservice Test-VS

debug_ip
cancel
cancel

It is possible to enter a command which enters a sub-mode while also adding applicable
flags. This will simultaneously navigate into the sub-mode and run the command. Subsequent
commands within the sub-mode do not use the initial sub-mode command.

debug_ip addrs 10.1.1.1

addrs 10.1.1.2
save

The where Command

When operating within a sub-mode, multiple changes can be made to parameters. To see the
current status of the configured parameters, use the where command.

debug_ip addrs 10.1.1.1

addrs 10.1.1.2
addrs 10.1.1.3
where
Tenant: admin
+----------+----------+
| Field | Value |
+----------+----------+
| addrs[1] | 10.1.1.1 |
| addrs[2] | 10.1.1.2 |
| addrs[3] | 10.1.1.3 |
+----------+----------+

VMware by Broadcom 19
VMware Avi Load Balancer Administration Guide

Revealing the REST API Calls Behind CLI Commands

Any Avi Load Balancer CLI command may include the --api-detail flag, which echoes the API
call (or calls) the command is performing. The command runs as it would without this flag. This
can be useful when building API-driven automation scripts.

show serviceengine --api-detail

REST API Request
API: /api/serviceengine?owned_by_controller=True&join_subresources=runtime

API-echoed output may be enabled for every command run during a single CLI session by typing
the terminal display_api_details command, as shown below:

terminal display_api_details
show serviceengine
REST API Request
Method: GET

API: /api/serviceengine?owned_by_controller=True&join_subresources=runtime

Install the Avi Load Balancer CLI Shell

Avi Load Balancer can be managed through the web interface, REST API, or command-line
interface (CLI). This section explains how to install the CLI shell onto a client's PC.

The CLI shell provides access to the Avi Load Balancer Controller through a PC client version of
the Controller’s CLI. Two versions of the CLI shell installation package are available:

n avi_shell-18.2.1-9010.tar.gz (or later) – Can be used with all infrastructure types. For installing
this version of the CLI shell package, continue referring to the later Install the CLI Shell on a
Ubuntu Docker Container.

n avi_lbaas-18.2.1-9010.tar.gz (or later) – Can be used if the infrastructure type is OpenStack

and Keystone support is enabled. (This option is chosen during the initial Avi Load Balancer
Controller setup and can also be configured later). To install this version of the CLI
shell, see Installing LBaaS Driver CLI Shell for OpenStack topic in the VMware Avi Load
BalancerInstallation Guide.

Note The CLI packages are available on the VMware customer portal, below the Avi Load
Balancer Controller download option for each version.

For more information on VMware's customer portal, see VMware CUSTOMER CONNECT.

VMware by Broadcom 20
VMware Avi Load Balancer Administration Guide

Requirements to Enable Remote CLI Shell

CLI shell server listens on TCP port 5054. To use the remote CLI shell, port 5054 must be
permitted in the firewall rules between the CLI client and the Avi Load Balancer Controller.

OS Versions Supported
Versions of the CLI shell available for Linux and Mac are as follows:

n Linux Ubuntu Docker container

n Linux (not in a Docker container)

n Mac

The steps are the same for each OS.

Prerequisites
The Avi Load Balancer CLI shell requires the following software:

1 pip (install package manager for Python).

2 Virtual environment (virtualenv): Command syntax included below.

3 Avi Load Balancer CLI shell installation file: From AWS S3.

The following sections provide steps for installing the Avi Load Balancer CLI shell.

Install the CLI Shell on a Ubuntu Docker Container

To install the Avi Load Balancer CLI shell on a Ubuntu Docker container, download the shell
package onto the host and enter the following commands. Edit the “/tmp” in “/tmp ubuntu” to
the directory where the image is downloaded.

docker run -it -v /tmp:/tmp ubuntu

sudo apt-get update
sudo apt-get install python-pip

Log in to the CLI Shell

avi_shell --address 10.10.10.99

The IPADDR is the Controller's IP address (10.10.10.99 in this example).

After logging in, Avi Load Balancer CLI commands can be entered into the shell. The show
version controller command in the following example displays the Avi Load Balancer version:

show version controller

+-----------------+---------------------------------------+
| Controller Name | Version |
+-----------------+---------------------------------------+

VMware by Broadcom 21
VMware Avi Load Balancer Administration Guide

| 10.10.25.44 | 18.2.1 (9010) 2019-12-03 22:42:48 UTC |

+-----------------+---------------------------------------+

Leave the CLI Virtual Environment

Exit the CLI shell virtual environment as shown below.

deactivate

Restart the CLI Shell

After the CLI shell is installed, enter the following command to start it the next time.

$> avi_shell/bin/avi_shell

show version controller

+-----------------+---------------------------------------+
| Controller Name | Version |
+-----------------+---------------------------------------+
| node-1 | 18.2.1 (9010) 2019-12-04 16:45:38 UTC |
| node-2 | 18.2.1 (9010) 2019-12-04 16:45:38 UTC |
| node-3 | 18.2.1 (9010) 2019-12-04 16:45:38 UTC |
+-----------------+---------------------------------------+

Install the CLI Shell on Linux or Mac

This section provides steps for installing the Avi Load Balancer CLI shell onto the Linux or
Macintosh. The steps are the same for either OS.

Procedure

1 Install pip, if not already installed.

2 Install virtualenv, if not already installed.

sudo pip install virtualenv

Downloading virtualenv-14.0.6-py2.py3-none-any.whl (1.8MB)
100% |████████████████████████████████| 1.8MB 178kB/s
Installing collected packages: virtualenv
Successfully installed virtualenv-14.0.6

3 Create a virtual environment for the CLI shell.

virtualenv avi_shell
New python executable in /home/user/git/clean/avi-dev/build/avi_shell/bin/python
Installing setuptools, pip, wheel...done.

4 Go to the CLI shell virtual environment.

cd avi_shell/
source ./bin/activate

VMware by Broadcom 22
VMware Avi Load Balancer Administration Guide

5 Install the CLI shell package.

pip install /tmp/avi_shell-18.2.1-9010.tar.gz

Processing /tmp/avi_shell-18.2.1-9010.tar.gz
Collecting cmd2==0.6.8 (from shell-client==18.2)
Collecting iso8601==0.1.11 (from shell-client==18.2)
Using cached iso8601-0.1.11-py2.py3-none-any.whl
...
...
Successfully installed cmd2-0.6.8 commentjson-0.6 iso8601-0.1.11 prettytable-0.7.2
pyparsing-2.1.0 pytz-2015.7 requests-2.9.1 requests-toolbelt-0.5.1 shell-client-17.2.17
urllib3-1.14 virtualenv-13.1.2 wheel-0.26.0 wrapt-1.10.6

Note For installing a CLI shell to manage an OpenStack write access mode deployment with
Keystone support enabled, see Install the LBaaS CLI Shell (OpenStack with Keystone) topic in
the VMware Avi Load BalancerInstallation Guide.

CLI - Linux Command Line Mode

The CLI mode, linux_command_line allows the user to interact with the CLI similar to a traditional
Linux command line program, where all the arguments are presented in --parameter value
format. Multi-line configuration of object uses a Bash heredoc style.

The CLI enhancements are introduced with the following considerations:

n Easy automation using the CLI

n Search for specific keywords in the configuration using the CLI

n Quick cut-and-paste to make modifications or resolve some issues with the CLI

In the linux_command_line mode, the entire object is presented in the form of --parameter
value, where each parameter represents a field in the object specified in a fully qualified path. An
example of this is:

IP address of a server in a pool is specified as --servers.index.ip. This is equivalent to referring

to this attribute as pool[‘servers’][index][‘ip’].

terminal mode linux_command_line

terminal --linux_cmd_output single_line
show pool p2
--uuid pool-b0cb56dc-cc24-4b87-9d19-7bf790d2e582 --default_server_port
80 --graceful_disable_timeout 1 --connection_ramp_duration 10 --
max_concurrent_connections_per_server 0 --servers.1.ip 1.1.1.1 --servers.1.hostname
1.1.1.1 --servers.1.enabled True --servers.1.ratio 1 --servers.1.verify_network
False --servers.1.resolve_server_by_dns False --servers.1.static False --
servers.1.rewrite_host_header False --servers.2.ip 2.2.2.2 --servers.2.hostname
2.2.2.2 --servers.2.enabled True --servers.2.ratio 1 --servers.2.verify_network
False --servers.2.resolve_server_by_dns False --servers.2.static False --
servers.2.rewrite_host_header False --servers.3.ip 3.3.3.3 --servers.3.hostname
3.3.3.3 --servers.3.enabled True --servers.3.ratio 1 --servers.3.verify_network
False --servers.3.resolve_server_by_dns False --servers.3.static False
--servers.3.rewrite_host_header False --server_count 3 --lb_algorithm

VMware by Broadcom 23
VMware Avi Load Balancer Administration Guide

lb_algorithm_least_connections --inline_health_monitor True --use_service_port False

--capacity_estimation False --server_auto_scale False --vrf_ref global --
fewest_tasks_feedback_delay 10 --enabled True --request_queue_enabled False
--request_queue_depth 128 --host_check_enabled False --sni_enabled True --
rewrite_host_header_to_sni False --rewrite_host_header_to_server_name False --tenant_ref
admin --cloud_ref Default-Cloud

Note The index starts with 1 in the CLI but is converted to 0 based on the API interactions.

Configure using the Linux Command Line Mode

configure pool p2 --uuid pool-7fce112f-c340-4c31-8c65-95e52f4b85ba --servers.1.ip 3.3.3.3 --
servers.1.hostname 3.3.3.3 --servers.2.ip 2.2.2.2 --servers.2.hostname 2.2.2.2 --servers.3.ip
1.1.1.1 --servers.3.hostname 1.1.1.1 --servers.4.ip 4.4.4.4 --servers.4.hostname 4.4.4.4 --
server_count 4 --tenant_ref admin
Updating an existing object.
p2 --uuid pool-7fce112f-c340-4c31-8c65-95e52f4b85ba --servers.1.ip 3.3.3.3 --
servers.1.hostname 3.3.3.3 --servers.2.ip 2.2.2.2 --servers.2.hostname 2.2.2.2 --servers.3.ip
1.1.1.1 --servers.3.hostname 1.1.1.1 --servers.4.ip 4.4.4.4 --servers.4.hostname 4.4.4.4 --
server_count 4 --tenant_ref admin

Some objects are relatively large, so interacting with the CLI with all these parameters specified
in a single line becomes cumbersome. To address this, use multi_line, the CLI setting to change
the Linux command output. This enables the CLI interaction one parameter per line.

Consider the example given below:

terminal --linux_cmd_output multi_line

show pool p2
--uuid pool-7fce112f-c340-4c31-8c65-95e52f4b85ba
--servers.1.ip 3.3.3.3
--servers.1.hostname 3.3.3.3
--servers.2.ip 2.2.2.2
--servers.2.hostname 2.2.2.2
--servers.3.ip 1.1.1.1
--servers.3.hostname 1.1.1.1
--server_count 3
--tenant_ref admin

configure pool p2 << END

--uuid pool-7fce112f-c340-4c31-8c65-95e52f4b85ba
--servers.1.ip 3.3.3.3
--servers.1.hostname 3.3.3.3
--servers.2.ip 2.2.2.2
--servers.2.hostname 2.2.2.2
--servers.3.ip 1.1.1.1
--servers.3.hostname 1.1.1.1
--servers.4.ip 4.4.4.4
--servers.4.hostname 4.4.4.4
--tenant_ref admin
END

VMware by Broadcom 24
VMware Avi Load Balancer Administration Guide

Updating an existing object.

CLI - Script Mode

In script mode, the input and output of the CLI command are expected to be in YAML format.
This is a direct conversion of what is presented in the API input and output as JSON into YAML.

YAML was chosen for presentation as it allows an easier interaction without having to consider
issues about the syntax such as commas, quotes, curly braces, and so on. It is easier to work
with YAML primarily for cut-and-paste and incremental changes to an existing object. Multi-line
configuration of object uses a Bash heredoc style. You can find more information on this under
https://en.wikipedia.org/wiki/Here_document.

terminal mode script

show pool p1
name: p1
server_count: 3
servers:
- hostname: 1.1.1.1
ip:
addr: 1.1.1.1
type: V4
- hostname: 2.2.2.2
ip:
addr: 2.2.2.2
type: V4
tenant_ref: https://localhost/api/tenant/admin
uuid: pool-b0cb56dc-cc24-4b87-9d19-7bf790d2e582

Configure a Pool using the Script Mode

configure pool p1 << END
name: p1
server_count: 3
servers:
- hostname: 1.1.1.1
ip:
addr: 1.1.1.1
type: V4
- hostname: 2.2.2.2
ip:
addr: 2.2.2.2

VMware by Broadcom 25
VMware Avi Load Balancer Administration Guide

type: V4
- hostname: 3.3.3.3
ip:
addr: 3.3.3.3
type: V4
tenant_ref: https://localhost/api/tenant/admin
uuid: pool-b0cb56dc-cc24-4b87-9d19-7bf790d2e582
END
Updating an existing object
name: p1
server_count: 3
servers:
- hostname: 1.1.1.1
ip:
addr: 1.1.1.1
type: V4
- hostname: 2.2.2.2
ip:
addr: 2.2.2.2
type: V4
- hostname: 3.3.3.3
ip:
addr: 3.3.3.3
type: V4
tenant_ref: https://localhost/api/tenant/admin
uuid: pool-b0cb56dc-cc24-4b87-9d19-7bf790d2e582

CLI Top-Level Commands

This section elaborates the top-level commands and their application.

Following are top-level commands shown when pressing the Tab twice from the shell:

Command Description

attach Connect to a remote Controller or SE. Similar to SSH.

clear Clear the statistics or value of a designated object.

configure Create or modify a new or existing object, such as a

virtual service, pool, profile, and so on.

convert Import and convert a configuration from non-VMware

load balancers.

copy Copy a file, such as a packet capture or tech-support file.

debug Change debug settings or perform packet captures.

delete Delete an object. Some objects may have dependencies

which must be resolved first.

do Execute a show command without exiting the current

location or sub-mode.

export Backup the system configuration or a single virtual service

configuration.

VMware by Broadcom 26
VMware Avi Load Balancer Administration Guide

Command Description

import Import a backed up (exported) complete or virtual service

specific configuration.

initialplacement Move a virtual service using manual placement mode to a

different SE.

purge Remove a file, such as a packet capture or tech-support

file.

rebalance Realign the SEs handing virtual services within an SE

group.

reboot Reboot part or all of the system. It can also wipe

configuration.

rediscover VMware-specific: Initiate discovery of networks and VMs.

restart Deactivate, then re-activate a virtual service.

scalein Reduce by one the number of SEs handling a virtual

service in manual placement mode.

scaleout Increase by one the number of SEs handling a virtual

service in manual placement mode.

show Show detailed information and stats on any Avi Load

Balancer object.

switchto Switch into a different tenant.

terminal Alter the shell's terminal settings.

upgrade Initiate an upgrade of the Avi Load Balancer system.

upload Upload a specified tech-support debug file to VMware.

verifylogin Validate login settings to a remote orchestrator such as

vCenter, APIC, or OpenStack.

watch Update the result of a command, such as show, every few

seconds.

attach
Description Connect to a remote Controller or SE. Similar to SSH.

Example Attach serviceengine Avi-se-arjnz

Top Level Flags:

controller Attach to a Controller shell.

serviceengine Attach to a SE shell.

configure
Description Create or modify a new or existing object, such as a virtual service, Pool, or Profile.

Example : > configure pool Test

VMware by Broadcom 27
VMware Avi Load Balancer Administration Guide

Top-Level Flags:

actiongroupconfig Create or modify an Action Group Config.

alertconfig Create or modify an Alert Config.

alertemailconfig Create or modify an Alert Email Config.

alertscriptconfig Create or modify an Alert Script Config.

alertsyslogconfig Create or modify an Alert Syslog Config.

analyticsprofile Create or modify an Analytics Profile.

application Create or modify an Application.

applicationpersistenceprofile Create or modify an Application Persistence Profile.

applicationprofile Create or modify an Application Profile.

authprofile Create or modify an Auth Profile.

cloud Create or modify a Cloud.

cluster Create or modify a Cluster.

controller Create or modify Controller properties.

healthmonitor Create or modify a Health Monitor.

httppolicyset Create or modify an HTTP Policy Set.

ipaddrgroup Create or modify an IP Address Group.

network Create or modify a Network.

networkprofile Create or modify a Network Profile.

networksecuritypolicy Create or modify a Network Security Policy.

pkiprofile Create or modify a PKI Profile.

pool Create or modify a Pool.

role Create or modify a Role.

serviceengine Create or modify a Service Engine.

serviceenginegroup Create or modify a Service Engine Group.

serviceengineproperties Create or modify Service Engine properties.

sslkeyandcertificate Create or modify an SSL Key and Certificate Request.

sslprofile Create or modify an SSL Profile.

stringgroup Create or modify a String Group.

systemconfiguration Create or modify a System Configuration.

tenant Create or modify a Tenant.

user Create or modify a User.

virtualservice Create or modify a Virtual Service.

vrfcontext Create or modify a VRF Context.

vsdatascriptset Create or modify a virtual service DataScript Set.

VMware by Broadcom 28
VMware Avi Load Balancer Administration Guide

convert
Description Import and convert a configuration from non-VMware load balancers. Supports conversion from F5
BIG-IP Local Traffic Manager configuration. Imported Virtual Services start in a disabled state to avoid
IP conflicts.

Example convert bigip_configuration

Top-Level Flags:

bigip_ip_addr BIGIP IP address

filename Avi Load Balancer config file name.

password BIGIP Password

username BIGIP Username

virtualservername Convert virtualserver. Name is of the form /partition/virtualservername

copy
Description Copy a file, such as a packet capture or tech-support file.

Example copy file source/tmp/old destination /tmp/new

Top-Level Flag:

source The source of the original file and path.

destination The destination of the new file and path.

debug
Description Change debug settings or perform packet captures.

Example : > debug virtualservice Test-VS

Top-Level Flags:

controller Controller-specific debug options.

serviceengine Service Engine-specific debug options.

virtualservice Virtual service specific debug options, including packet capture.

delete
Description Delete an object. Some objects may have dependencies that must be resolved first.

Example delete pool Test-Pool

Top-Level Flags:

actiongroupconfig Delete Action Group Config.

alert Delete Alert.

alertconfig Delete Alert Config.

alertemailconfig Delete Alert Email Config.

alertscriptconfig Delete Alert Script Config.

alertsyslogconfig Delete Alert Syslog Config.

VMware by Broadcom 29
VMware Avi Load Balancer Administration Guide

application Delete Application.

applicationpersistenceprofile Delete Application Persistence Profile.

applicationprofile Delete Application Profile.

authprofile Delete Auth Profile.

cloud Delete Cloud.

healthmonitor Delete Health Monitor.

httppolicyset Delete HTTP Policy Set.

ipaddrgroup Delete IP Address Group.

networkprofile Delete Network Profile.

networksecuritypolicy Delete Network Security Policy.

pkiprofile Delete PKI Profile.

pool Delete Pool.

role Delete Role.

serviceengine Delete Service Engine.

serviceenginegroup Delete Service Engine Group.

sslkeyandcertificate Delete SSL Key and Certificate Request.

sslprofile Delete SSL Profile.

stringgroup Delete String Group.

tenant Delete Tenant.

user Delete User.

virtualservice Delete Virtual Service.

vrfcontext Delete VRF Context.

vsdatascriptset Delete virtual service DataScript Set.

do
Description Execute a show command without exiting the current location or sub-model.

Example do show debug flags virtualservice Test-VS

Top-Level Flags:

show Show detailed information and stats for any Avi Load Balancer object.

export
Description Back up the system configuration or a single virtual service configuration.

Example export configuration file /tmp/backup_config

export virtualservice Test-VS file /tmp/Test-VS

Top-Level Flags:

configuration Export the entire Avi Load Balancer configuration in JSON format.

VMware by Broadcom 30
VMware Avi Load Balancer Administration Guide

serviceengine Export the SE OVA file from Controller for manual install.

virtualservice Export a specific virtual service configuration file including child objects.

import
Description Import a backed up (exported) complete or virtual service specific configuration.

Example import configuration file /tmp/backup_config

Top-Level Flags:

initialplacement
Description Move a virtual service using manual placement mode to a different SE.

Example initialplacement

virtualservice Test-VS

servicengine Avi-se-arjni

Top-Level Flags:

virtualservice Specify the virtual service to be assigned to an SE.

serviceengine Specify the name of the SE to receive the virtual service.

migrate
Description Move a virtual service using manual placement (No Access or Read Access) mode to a different SE.

Example migrate

virtualservice Test-VS

serviceengine Avi-se-arjni

Top-Level Flags:

from_se_ref Specify the name of the source SE that has the virtual service.

to_host_ref An option used with to_new_se, specifying the host upon which to create the new SE.

to_new_se Create a new SE and migrate to it.

to_se_ref Migrate to a specific existing SE.

purge
Description Remove a file, such as a packet capture or tech-support file.

Example purge file source /tmp/backup_config

Top-Level Flags:

rebalance
Description In auto scale mode, sets the frequency upon which the Controller will Inspect an SE group to see
if a virtual service to SE mapping mudt be adjusted, potentially resulting in a scale in, scale out, or
migration.

Example rebalance interval 10 se_group_ref My_SE_Group

VMware by Broadcom 31
VMware Avi Load Balancer Administration Guide

Top-Level Flags:

interval The frequency, in minutes. Default is 5.

se_group_ref The name of the SE group to alter.

reboot
Description Reboot part or all of the Avi Load Balancer system. It can also delete configuration. With no flags
specified, all Controllers and SEs are rebooted.

Example reboot

Top-Level Flags:

clean Reset the Avi Load Balancer system's configuration and reboot the cluster. Consider making a backup
first.

node Reboot the virtual machine of a Controller within a cluster.

serviceengine Reboot a specific SE. Virtual service disruption will depend on the high availability settings for the SE
group.

rediscover
Description VMware-specific: Initiate discovery of networks and VMs.

Example rediscover vcenter My-vCenter

Top-Level Flags:

restart
Description Disable then immediately re-enable a virtual service.

Example restart virtualservice Test-VS

Top-Level Flags:

scalein
Description Reduce by one the number of SEs handling a virtual service in manual placement mode. There must be
a minimum of one SE.

Example scalein virtualservice Test-VS

Top-Level Flags:

from_se_ref Specify a non-primary SE to stop using for this virtual service.

scalein_primary Migrate from the primary SE and discontinue use of the SE for this virtual service.

scaleout
Description Increase by one the number of SEs handling a virtual service in manual placement.

Example scaleout virtualservice Test-VS

Top-Level Flags:

to_host_ref An option used with to_new_se, specifying where to create the new SE.

VMware by Broadcom 32
VMware Avi Load Balancer Administration Guide

to_new_se Create a new SE and scale out to it.

to_se_ref Scale out to an existing SE.

show
Description Show detailed information and stats on any Avi Load Balancer object.

Example show virtualservice Test-VS summary

Top-Level Flags:

actiongroupconfig Show info on an Action Group Config.

alert Show info on an Alert.

alertconfig Show info on an Alert Config.

alertemailconfig Show info on an Alert Email Config.

alertscriptconfig Show info on an Alert Script Config.

alertsyslogconfig Show info on the an Syslog Config.

analyticsprofile Show info on an Analytics Profile.

apic Show info on the APIC Graph Instances.

application Show info on an Application Folder.

applicationpersistenceprofile Show info on an Application Persistence Profile.

applicationprofile Show info on an Application Profile.

authprofile Show info on an Auth Profile.

backups Show available backup files.

cloud Show info on the Cloud.

cluster Show info on the Cluster.

config-consistency-check Show config-consistency-check status.

config_events Show info on the Event Log.

configuration Show configuration.

controller Show Controller properties.

cpuusage Show Controller CPU usage.

debug Show Virtual Service capture file.

debug-log Show Service Engine debugs.

diskusage Show Controller disk usage.

events Show info on an Event Log.

file Show files.

healthmonitor Show info on a Health Monitor.

httppolicyset Show info on an HTTP Policy Set.

ipaddrgroup Show info on an IP Address Group.

jobs Show all duration based jobs pending expiry.

VMware by Broadcom 33
VMware Avi Load Balancer Administration Guide

License Show info on the Controller License.

logcontrollermapping Show mapping of Log Controllers for each virtual service.

logs-status Show logs subsystem status.

memoryusage Show Controller memory usage.

metricsmgr Show info on a Metrics Entity Runtime.

network Show info on a Network.

networkprofile Show info on a Network Profile.

networksecuritypolicy Show info on a Network Security Policy.

openstack_audit Show OpenStack LBaaS vs Avi config audit reports.

pkiprofile Show info on a PKI Profile.

placement Show info on a Rm VRF Proto.

pool Show info on a Pool.

role Show info on a Role.

serviceengine Show info on a Service Engine.

serviceenginegroup Show info on a Service Engine Group.

serviceengineproperties Show the Service Engine Properties.

seupgrade Show an ongoing SE upgrade status.

sslkeyandcertificate Show info on an SSL Key and Certificate.

sslprofile Show info on an SSL Profile.

stringgroup Show info on a String Group.

systemconfiguration Show info on the System Configuration.

systemconfigurationruntime Show info on the System Configuration Runtime.

tech-support Show full tech support.

tenant Show info on a Tenant.

terminal Show the terminal settings.

transaction Show more info on Transaction Stats.

upgrade Show upgrade status if one is in-progress.

user Show info on the User.

vcenter Show info on the specified VMs.

version Show a Controller node's version.

vinfra Show info on the VI Datastore Contents.

virtualservice Show info on a Virtual Service.

vrfcontext Show info on a VRF Context.

vsdatascriptset Show info on the virtual service DataScript Set.

VMware by Broadcom 34
VMware Avi Load Balancer Administration Guide

switchto
Description Switch into a different tenant.

Example switchto tenant Tenant2

Top-Level Flags:

terminal
Description Alter the shell's terminal settings.

Example terminal session_timeout 240

Top-Level Flags:

length Number of rows to show for pagination output. Greater than will pipe to more. Choose 0 for no
pagination.

session_timeout Alter the default 15 min timeout to keep an idle terminal session open.

timezone Display timestamps in specified time zone.

unhide Commands show additional flags. It is not recommended for casual use.

upgrade
Description Initiate an upgrade of the Avi Load Balancer system. This may be done passively (by migrating each
SE while upgrading) or disruptively (fast, but it will terminate existing client connections then begin the
upgrade).

Example upgrade system image_path /tmp/new_file

Top-Level Flags:

system Upgrade the Avi Load Balancer system.

ui Upgrade the Avi Load Balancer UI.

upload
Description Generate and upload a tech-support debug file to VMware.

Example upload tech-support debuglogs filter exclude_logs

Top-Level Flags:

exclude_archive Exclude archived backups.

exclude_logs Exclude client log files (virtual service logs), which could be quite large.

include_se Include (non-significant) logs may be stored on the SEs.

VMware by Broadcom 35
VMware Avi Load Balancer Administration Guide

verifylogin
Description Validate the username, password, and path to a remote orchestrator such as vCenter, APIC, or
OpenStack.

Example verifylogin

vcenter username

admin password secret

vcenter_url 10.1.1.1/login

Top-Level Flags:

apic Verify access to an APIC controller.

openstack Verify access to OpenStack.

vcenter Verify access to a VMware vCenter controller.

watch
Description Updates the result of a command such as show every few seconds.

Example watch show pool Test-pool server detail | grep -e 'ip\| open_conns'

Top-Level Flags:

show Select a valid show command syntax to update.

Common Avi Load Balancer CLI Tasks

This section illustrates examples of executing common management tasks with the Avi Load
Balancer CLI.

CLI: Virtual Service and Pool Creation

During virtual service creation, a pool must be configured. This pool has to be created before
creating the virtual service.

Create the following using the CLI:

1 The Pool

2 The VsVIP

3 The Virtual Service

Note The examples below shows only the minimum configuration required to successfully
deploy a new application. Many additional options are available for customizing virtual services
and pools.

Example: Create the Pool

: > configure pool Test
: pool> servers ip 10.1.1.100 port 80
: pool:servers> save
: pool> servers ip 10.1.1.101 port 80
: pool:servers> save

VMware by Broadcom 36
VMware Avi Load Balancer Administration Guide

: pool> where
Tenant: admin
+------------+------------+
| Field | Value |
+------------+------------+
| name | Test |
| servers[1] | |
| ip | 10.1.1.100 |
| port | 80 |
| servers[2] | |
| ip | 10.1.1.101 |
| port | 80 |
+------------+------------+
: pool> save

Example: Create the VsVIP

: > configure vsvip Test_vsVip
: vsvip> vip ip_address 10.10.10.10
: vsvip:vip> save
: vsvip> save

Example: Create the Virtual Service

: > configure virtualservice Test_VS
: virtualservice> vsvip_ref Test_vsVip
: virtualservice> pool_ref Test
: virtualservice> services port 80
: virtualservice:services> save
: virtualservice> save

Modify Pool Servers

A common task is to add and remove servers from an existing pool or to activate or deactivate
servers within the pool. While servers can be added to a pool without specifying a port (they will
inherit the port of the pool or virtual service), the CLI requires the IP and port to manipulate the
server for tasks such as enable and disable.

The following commands will delete the .101 server and add a new .102 server to the pool
named Test.

configure pool Test

no servers ip 10.1.1.101 port 80
servers ip 10.1.1.102 port 80
[admin]: pool:servers save

The following commands will enter the sub-mode for the .100 server and disable it.

ip 10.1.1.100 port 80
where
Tenant: admin
+-------+------------+

VMware by Broadcom 37
VMware Avi Load Balancer Administration Guide

| Field | Value |
+-------+------------+
| ip | 10.1.1.100 |
| port | 80 |
+-------+------------+
no enabled
+---------+------------+
| Field | Value |
+---------+------------+
| ip | 10.1.1.100 |
| port | 80 |
| enabled | False |
+---------+------------+
save

Export / Import Configuration

The Avi Load Balancer Controller has a database to store its configuration information. This
includes all configurations related to tenants, virtual services, pools, policies, and accounts.
Export this configuration as a JSON file for backup or to migrate parts of the config to another
Avi Load Balancer Controller.

The configuration may be backed up or moved to another Controller cluster through the CLI. The
exported configuration might be the entire system configuration, a more limited version based on
the access rights of the current user and tenant of the administrator, or a single virtual service
and its child properties.

To export only a single virtual service and its child objects, such as pools, use the virtualservice
flag instead of the configuration flag with the export command.

The following commands will export the configuration to a file named config_export, SCP it to a
remote location, and then return it to the Avi Load Balancer shell.

export configuration file config_export

Completed writing the export configuration to config_export
bash

pwd
/home/admin
ls
config_export
scp ./config_export [email protected]:/root
[email protected]'s password:
config_export 100% 232KB 431.8KB/s 00:00
exit

Before starting a configuration import:

n Ensure that all Controller members of the cluster are up and the cluster leader has the
following configuration:

n Admin Account

VMware by Broadcom 38
VMware Avi Load Balancer Administration Guide

n Cluster Configuration

n OpenStack Infrastructure (OpenStack only)

n Edit the configuration file and remove the _last_modified field.

The following commands will import a backed-up configuration to a Controller cluster.

import configuration file /home/admin/myconfig keep_uuid

Successfully imported the configuration file

You can restore this configuration information to an empty Avi Load Balancer Controller. The
restoration steps are:

n Deploy three new Controller nodes; the image type (OVA, qcow2, ami) should match that of
the original Controller node from which the configuration was exported.

n Choose one Controller as the leader and go through the initial setup page to enter the initial
setup information and the redundant Controller cluster information.

n Use the CLI or API to import the configuration.

After these steps, the cluster will return to the same running state as the original one.

Export Configuration Filters

You can also tune the exported file by using filters. The following filters are supported:

Filter Name Description

full_system Exports the complete configuration (including persistent

runtime) in the format documented in Restore
Configuration. This configuration cannot be imported
through the CLI or the REST API.

obj_type Limits exported objects to a specific set of object types.

recurse Exports the entire tree of dependencies (including some

runtime) for objects in filtered config.

passphrase Encrypts sensitive data. The same passphrase must be

provided to import the configuration.

search Only exports objects that contain a certain string.

skip_default Prunes defaulted values of objects in exported

configuration.

tenant_ref Only exports objects in the specified tenant.

Example: Examples
To export all virtual services and dependencies, use the following command:

export configuration file config\_export obj\_type virtualservice recurse

VMware by Broadcom 39
VMware Avi Load Balancer Administration Guide

To export all objects in tenant t1 and filter out defaulted values, use the following command:

export configuration file config\_export tenant\_ref t1 skip\_default

Packet Capture
The Avi Load Balancer Controller provides a convenient mechanism to capture data plane traffic
processed by SEs. An administrator can initiate a capture command from the Controller CLI
while the actual capture occurs on the SEs. The packet capture is saved in pcap format on
the Controller. The SE captures packets on both the client-side and server-side of the SE. If a
virtual service is scaled across multiple SEs, then all applicable SEs will participate in the packet
capture. The Controller will aggregate the captures and sort the entries according to time. Once
a capture is complete, it is uploaded to a personal computer or other systems for analysis with an
appropriate tool such as tcpdump or Wireshark.

Enter the packet capture sub-mode for the specified virtual service:

debug virtualservice Test-VS

Parameters may be defined for the packet capture. By default, the capture is performed within
the context of the selected virtual service, and it is also performed on all SEs handling the virtual
service traffic, including the packets from the client and server sides.

Parameter Name Description

capture_params duration Time, in minutes. Default is unlimited.

capture_params num_pkts Maximum number of packets to collect. Default is

unlimited.

capture_params pkt_size Packet size, or snap length, to capture. Default is

unlimited.

debug_ip addrs IP4 Address format <x.x.x.x>

debug_ip prefixes IP4 Prefix format <x.x.x.x/x>

The debug_ip command enters a sub-mode. This allows multiple IP addresses or IP subnets to
be entered (omit the debug_ip for subsequent entries). Save to commit the desired IPs and
return to the previous menu.

Note By default, no maximum packets, or duration of time to be captured are defined. It is

recommended to include a maximum packet capture, as shown in the following example. Without
a limit, the capture will run until the SE disk is filled, potentially disrupting the service.

VMware by Broadcom 40
VMware Avi Load Balancer Administration Guide

Specify parameters, including the maximum number of packets to capture:

capture_params num_pkts 1000

debug_ip addrs 10.10.10.10
save

Begin capturing based on the previously configured parameters:

Re-enter the packet capture sub-mode and stop an ongoing packet capture:

debug virtualservice Test-VS

no capture
save

Export the packet capture to a remote system to view it using tools such as tcpdump or
Wireshark:

show debug virtualservice Test-VS capture

Please specify the destination directory: /tmp
Downloaded the attachment to /tmp/vs_virtualservice.20141205_192033.pcap
bash
scp /tmp/vs_virtualservice.192033.pcap [email protected]:/tmp

Monitor Concurrent Connections to a Server

To track the number of concurrent connections to a server, the server must be viewed within the
context of a pool. By prepending the watch command in front of the show, it is possible to see a
near real-time updates of the connection count. Keep in mind there is some delay between the
time the connection is established and the polling interval when the SE sends information back to
the Controller. Cancel the watch with Ctrl+C.

watch show pool Test-pool server detail | grep -e 'ip\|open_conns'

| ip_addr | 10.1.1.17 |
| open_conns | 50 |
| ip_addr | 10.1.1.16 |
| open_conns | 49 |

VMware by Broadcom 41
VMware Avi Load Balancer Administration Guide

CLI and Shell Prompts Available for Avi Load Balancer

The authorized user may engage in three conversations through two command-line interfaces.
Conversations with the first two entities are common. Conversations of the third kind are
infrequent and typically undertaken by Avi Load Balancer Customer Support personnel for
troubleshooting purposes.

In conversation Your commands address With this frequency

1 Avi Load Balancer Controller's Linux Frequently Used

OS using Bash

2 Avi Load Balancer processes running Frequently Used

on Controller using the Avi Load
Balancer shell

3 The SE’s Linux operating system Rarely

Accessing Avi Load Balancer Controller using Linux Shell

n Access the Avi Load Balancer Controller’s OS through the bash Linux CLI.

n Either SSH (Secure Shell) to the Controller or access it through the console from an
orchestrator such as vCenter.

n Example: If the Controller’s address is 10.144.130.195, type the ssh command: ssh
[email protected], and when prompted, supply the password for the admin user.

Avi Load Balancer Controller CLI is used to analyze various logs in the /opt/avi/log
and /var/log/upstart directories.

Accessing Avi Load Balancer Controller using Avi Load Balancer

Shell
n Start with conversation 1.

n To enter Avi Load Balancer-specific commands, type the shell command and furnish
credentials.

n For example: show virtualservice <name-of-virtual-service>

Accessing Avi Load Balancer Controller using Avi Load Balancer

Service Engines
n SSH to an Avi Load Balancer SE or execute the attached service engine <name-of-service-
engine> command from the Avi Load Balancer shell to enter the SE’s Linux CLI.

n Use it to look into SE-specific logs in various directories, such as /opt/avi/log.

Shell prompt access is not available for Avi Load Balancer Service Engines. Avi Load Balancer
SE’s Linux CLI does not provide the option to run show commands.

For more information, see CLI Options.

VMware by Broadcom 42
VMware Avi Load Balancer Administration Guide

Access Using Remote Authentication

The following users' SSH into the Controller using local or remote authentication:

n Admin

n CLI

The admin account is the standard administrative account for the system and is maintained as a
locally authenticated account, even in a system configured for remote auth.

The cli account has no password. Any non-admin account must use the account name cli, which
will forward the user through Linux to the Avi Load Balancer shell. From the shell, the user must
log in through their standard account. Non-admin accounts do not have access to Linux.

For more information on remote authentication, see Authentication Profile.

Accessing Avi Load Balancer Linux CLI as a Non-Admin User

Using an SSH client
When a non-admin user logs in to the Avi Load Balancer Linux CLI, the non-admin user can have
admin-like privileges.

An SSH session to Linux CLI is available only for admin username. Even if a user is configured as
a super-user, the user cannot log in to Linux CLI. Users other than admin, including super-users
(whether local or remote), can only log in using cli@<Avi Controller IP> command.

If a non-admin user, even if it is configured as a super-user, tries to SSH to Avi Load Balancer
Controller IP address, the system will return an Access Denied error, as shown below.

login as: testuser

Instructions
Follow the steps to SSH into Avi Load Balancer CLI using a non-admin user account.

In this example, the non-admin user is configured as a super-user too.

n Open an SSH client and use the cli@<Avi Controller IP> command. Replace the Avi Load
Balancer Controller IP with the IP of the Controller for which access is required.

VMware by Broadcom 43
VMware Avi Load Balancer Administration Guide

n Provide the credentials when prompted for a username. In the below example, a user account
with the username testuser is used, which is also configured as a super-user on Avi Load
Balancer.

Using username "cli".

Avi Cloud Controller
Avi Networks software, Copyright (C) 2013-2017 by Avi Networks, Inc.
All rights reserved.
Version: 17.1.8
Date: 2017-09-21 06:03:07 UTC
Build: 9020
Management: 10.10.1.1/23 UP
Gateway: 10.10.1.1 UP
The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under
license. Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or the GNU
Lesser General Public License (LGPL) Version 2.1. A copy of each
such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://www.opensource.org/licenses/lgpl-2.1.php
Last login: Fri Oct 27 10:32:02 2017 from 10.10.8.11
Launching a CLI shell in a container
No handlers could be found for logger "docker.auth.auth"
Login: testuser
Password:

n After providing the password, as shown in the above CLI snippet, you can get access to the
Avi Load Balancer shell.

[admin:avi-controller]: >

From the Avi Load Balancer shell prompt, you can run all the show commands and shell
commands.

Checking Logs Using a Super-User Account

Use the account mentioned in the previous steps and use the attach controller
<controller-ip> command to go to the linux bash prompt. As it is a container with no
persistent storage, none of the log files are visible when the ls command is used.

[admin:avi-controller]: > bash

root@04de723c268a:/#
root@04de723c268a:/# cd /opt
root@04de723c268a:/opt# ls
root@04de723c268a:/opt# <- No directory in /opt as seen here

VMware by Broadcom 44
VMware Avi Load Balancer Administration Guide

Using Username avidebuguser

A non-admin user (who is also a super-user) can be associated with the Avi Load Balancer
Controller by using attach <Avi Controller IP> command. This will provide the Controller
container access to the user as an avidebuguser. The avidebuguser is also a sudo user. Attach
option is available only if the user (local or remote) is configured as a super-user.

[admin:avi-controller]: > attach controller 10.10.1.10

No handlers could be found for logger "root"
Warning: Permanently added '10.10.1.10' (ECDSA) to the list of known hosts.
Avi Cloud Controller
Avi Networks software, Copyright (C) 2013-2017 by Avi Networks, Inc.
All rights reserved.
Version: 17.1.8
Date: 2017-09-21 06:03:07 UTC
Build: 9020
Management: 10.10.1.10/23 UP
Gateway: 10.10.1.1 UP
Esx and OpenstackWelcome, this is your controller!!!
The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under
license. Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or the GNU
Lesser General Public License (LGPL) Version 2.1. A copy of each
such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://www.opensource.org/licenses/lgpl-2.1.php
Last login: Fri Oct 27 10:32:36 2017 from 172.17.0.2
avidebuguser@avi-controller:~$

Use the ls command to check the log files as shown below.

avidebuguser@avi-controller-2:/opt$ ls
*avi zookeeper-3.4.6*

avidebuguser@avi-controller-2:/opt/avi/log$ pwd
/opt/avi/log

Additional Information
For more information on Avi Load Balancer Linux CLI and Avi Load Balancer CLI access, see CLI -
Linux Command Line Mode and Chapter 2 CLI Access Settings.

Adding Required HMAC to the Allowed HMACs Lists using

Avi Load Balancer UI
Sometimes the virtual machine (VM) for the Avi Load Balancer Controller is accessible through
the UI but is not accessible through SSH, for example, Putty (Windows) and Microsoft
Certification Test Tool for Azure (Windows).

VMware by Broadcom 45
VMware Avi Load Balancer Administration Guide

By default, Avi Load Balancer does not allow hmac-sha1-96 for management access to Avi Load
Balancer Controller and Service Engines. To access the Avi Load Balancer Controller through
an SSH Client (for example, Putty), which needs to use hmac-sha1-96, add hmac-sha1-96 to
the Allowed HMACs using UI. If there is no entry in the Allowed HMACs, all the default HMACs are
allowed.

1 Navigate to Administration > System Settings, and click edit.

2 Under Access, add hmac-sha1-96 to the Allowed HMACs option.

3 Click Save.

Once the required HMAC is allowed for the management access to the Avi Load Balancer
Controller, SSH will be accessible to the various SSH clients. For the list of allowed HMACs for
the management access of the Avi Load Balancer Controller, see Restricting the Allowed HMACs.

Securing Management IP Access

This topic describes how to restrict access to both the Avi Load Balancer Controller’s
management interfaces and the list of ciphers and HMACs that are allowed for management
sessions.

By default, the Avi Load Balancer Controller does not restrict the client IP addresses that are
allowed to attempt access to the Controller through its management interfaces. The Avi Load
Balancer provides a way to define the set of IP addresses allowed to attempt management
access to the Controller.

Additionally, access through all the management interfaces can be further restricted by explicitly
specifying the ciphers and keyed-hash message authentication codes (HMACs) that are allowed.

To establish a management connection with the Controller through any management interface,
the client must support a cipher and HMAC that are allowed by the Controller.

Note
n When configuring the IP access list for API and SSH, ensure that the Controller nodes can talk
to each other by either placing them on the same subnet or explicitly adding the Controller IP
addresses.

n Avi Load Balancer does not currently support SSH and System Internal Access control types
if controller IPs are FQDN based.

IP Access Lists
When the configuration is modified to specify allowed IP addresses for accessing specific
management interfaces, the Avi Load Balancer programs the Linux IP tables on the Avi Load
Balancer Controller host to allow the specified IP addresses at the network layer.

VMware by Broadcom 46
VMware Avi Load Balancer Administration Guide

Separate IP access lists can be configured for each of the following management interfaces:

n REST API / Web interface (or any script or other means of automation that uses the REST
API)

n SSH daemon

n CLI Shell (allows remote CLI access)

n SNMP

The following is a logical view of the management interfaces.

NSX Advanced
Load Balancer Controller
(or cluster)

REST API
(cmds internally
converted)

API / Web
SSH (CLI) CLI Shell SNMP
Interface

Note Avi Load Balancer internally converts commands that are entered through the web
interface or the CLI shell into REST API commands. Similarly, the system responses are converted
back into web interface or CLI format when presented in return to the Avi Load Balancer user. To
restrict IP access through the interfaces, separate IP lists must be entered for each interface.

IP Access List Format

For each management interface, the set of client IP addresses that are allowed to access that
interface can be specified in any of the following ways:

n Individual IP addresses; example: 1.1.1.1, 2.2.2.2

n IP address ranges; example: 1.1.1.1-1.1.1.100

n IP address prefixes (subnets); example: 1.1.1.0/24

n IP address groups

Caution
n When changing the IP access list for the management interface, ensure to include the IP
address in the access list. Else, the management session will end after the change is saved.

n If no IP addresses are added to the access list of the management service, any IP address is
allowed to attempt access to that service.

VMware by Broadcom 47
VMware Avi Load Balancer Administration Guide

Restricting IP Access to Management Interfaces through Web

Interface
1 Navigate to Administration > System Settings.

2 Click the edit icon to open the EDIT SYSTEM SETTINGS pop-up window.

VMware by Broadcom 48
VMware Avi Load Balancer Administration Guide

3 In the Client Management Access section, configure the fields Allowed SSH Clients, Allowed
CLI Shell Clients, Allowed External HTTP(S) Clients, and Allowed External SNMP Clients as
required, and enter or select the IP addresses that are allowed access.

n Host, range, or subnet: Choose Select From Available or Enter Custom Value to activate
the field, and select or type the address(es). If you are listing multiple addresses, use
commas to delimit them. For entering a range, use a hyphen between the starting
(lowest) and ending (highest) addresses.

n IP Group: Select the IP group (if already configured) or click Create to create the group
(list). Enter the group name and address information and click Save to return to the EDIT
SYSTEM SETTINGS pop-up window.

Note Enter the Controller management IPs for HTTP(s) settings using the field Allowed
External HTTP(S) Clients. Internal Analytics APIs fail if the management IPs of the cluster
nodes are not included in the list of IPs allowed for external HTTP(s).

4 After specifying the allowed client IP addresses for each management service, go to the next
section or click Save to save the changes and close the popup.

Restricting the Allowed Ciphers

By default, Avi Load Balancer allows management sessions to use any of the following ciphers:

n aes128-ctr

n aes256-ctr

n arcfour256

n arcfour128

n aes128-cbc

n 3des-cbc

n blowfish-cbc

n aes192-cbc

n aes256-cbc

Ciphers arcfour128 and arcfour256 are not supported. To restrict access to a subset of these
ciphers, specify the individual ciphers:

n In the Update System Access Settings popup, in the Allowed Ciphers field, enter the cipher
names. The names must be spelled as shown above. Use commas between the names.

n Go to the next section or click Save.

Restricting the Allowed HMACs

The Avi Load Balancer allows management sessions to use any of the following HMACs:

n hmac-md5

VMware by Broadcom 49
VMware Avi Load Balancer Administration Guide

n hmac-md5-96

n hmac-sha1

n [email protected]

n hmac-sha2-512

n [email protected]

To support fewer HMACs, specify the individual HMACs:

1 In the EDIT SYSTEM SETTINGS pop-up window, in the Allowed HMACs field, enter the HMAC
names. The names must be spelled as shown above. Use commas between the names.

2 After updating, click Save to save the changes and close the popup.

REST API -IP Addresses That Are Allowed Management Access

The following request to the Avi Load Balancer REST API retrieves the current system settings,
including the mgmt_ip_access_control section. This section specifies the client IP addresses that
are allowed to access the Controller through the management services.

In this example, access through the web interface or REST API is restricted to addresses in
the 10.10.0.0/16 subnet, and to IP addresses in the range 3.3.3.1-100. IP access for the other
management services is not included in this output, because IP access has not been explicitly
defined for them.

API: GET /api/systemconfiguration

Data:
{
"email_configuration": {
"from_email": "[email protected]",
"mail_server_name": "localhost",
"smtp_type": "SMTP_NONE",
"mail_server_port": 25
},
...

"mgmt_ip_access_control": {
"api_access": {

VMware by Broadcom 50
VMware Avi Load Balancer Administration Guide

"ranges": [
{
"begin": {
"type": "V4",
"addr": "3.3.3.0"
},
"end": {
"type": "V4",
"addr": "3.3.3.100"
}
}
],
"prefixes": [
{
"ip_addr": {
"type": "V4",
"addr": "10.10.0.0"
},
"mask": 16
}
],
"match_criteria": "IS_IN"
}
},
}

Allowed Ciphers and HMACs

The following REST API request retrieves the list of allowed ciphers and HMACs.

API: GET /api/systemconfiguration

Data:
{
"email_configuration": {
"from_email": "[email protected]",
"mail_server_name": "localhost",
"smtp_type": "SMTP_NONE",
"mail_server_port": 25
},
...

"ssh_ciphers": [
"aes128-ctr",
"aes256-ctr",
"aes192-cbc",
"aes256-cbc"
],

"ssh_hmacs": [
"hmac-md5",
"hmac-md5-96",
"hmac-sha1",
"[email protected]",
"hmac-sha2-512"

VMware by Broadcom 51
VMware Avi Load Balancer Administration Guide

],
}

For more information, see Renew Default (Self-Signed) Certificates on Avi Load Balancer.

For more information on how to Change the Default Portal Certificates, see Change the Default
Certificate of the Controller topic in the VMware Avi Load BalancerConfiguration Guide.

Certificates

Changing the Default Certificate of the Controller

The Avi Load Balancer Controller can be accessed through the UI using the default certificate
associated with it, but users get a warning message regarding certificate mismatch or certificate
trust. To avoid a browser warning message while accessing the Controller, install the complete
certificate chain matching the FQDN of the Controller and replace the default Controller
certificate with the new certificate.

To change the default certificate for the Controller, and import or create a new Controller
certificate.

Procedure

1 Navigate to Templates > Security > SSL/TLS Certificates, click Create and select Controller
Certificate.

2 Create or Import the existing certificate.

3 Navigate to Administration > System Settings, and click the edit icon to edit the System
Settings.

4 Replace the default or existing certificate with the new certificate in the SSL/TLS Certificate
drop-down menu.

5 Click Save.

VMware by Broadcom 52
VMware Avi Load Balancer Administration Guide

Results

Note To avoid any certificate trust issue, make sure that the certificate chain is complete on
the Controller and the client browser. Install the complete certificate chain (the root and the
intermediate certificates) on the Controller and the client browser. Try accessing the Controller
using the UI to confirm it is opening without any error.

Renew Default (Self-Signed) Certificates on Avi Load Balancer

The default certificate on Avi Load Balancer is self-signed. This section explains how to replace
the default certificate when the certificate is expired or is about to expire. The following steps
can also be used to replace the self-signed certificate with a third-party signed certificate.

Prerequisites
OpenSSL 1.1.x or later.

Configure using Avi Load Balancer UI

n Navigate to Templates > Security > SSL/TLS Certificates and click the Export icon (right) of
System-Default-Cert entry.

n Copy data from the Key and Certificate fields to two new files using the COPY TO
CLIPBOARD option. Name the new files as system-default.key and system-default.cer,
respectively.

VMware by Broadcom 53
VMware Avi Load Balancer Administration Guide

Configure using OpenSSL

n Run the following command in OpenSSL to check the expiration date of the certificate.

openssl x509 -in system-default.cer -noout -enddate

n Run the following command to generate a new CSR with the system-default.key.

openssl req -new -key system-default.key -out system-default.csr

n Run the following command to generate a new certificate with the new expiration date. In this
example, the new certificate is named as system-default2.cer.

openssl x509 -req -days 365 -in system-default.csr -signkey system-default.key -out system-
default2.cer

n Verify the expiration date on the new certificate (system-default2.cer).

openssl x509 -in system-default2.cer -noout -enddate

Configure using Avi Load Balancer CLI and UI

n Copy the system-default2.cer and the system-default.key to the Controller.

VMware by Broadcom 54
VMware Avi Load Balancer Administration Guide

Optional Step: Before performing the next steps, you can deactivate any virtual services that
are configured to use the System-Default-Cert.

n Log in to the CLI, and execute the following command to perform the changes for the default
certificate on Avi Load Balancer (System-Default-Cert).

[admin:cntrl1]: > configure sslkeyandcertificate System-Default-Cert

n Execute the certificate command and click Enter. Run certificate file <path to system-
default2.cer>/system-default2.cer. Enter the save command to save the changes.

[admin-cntrl1]: sslkeyandcertificate> certificate

[admin-cntrl1]: sslkeyandcertificate:certificate> certificate file:<path to system-
default2.cer>/system-default2.cer
[admin-cntrl1]: sslkeyandcertificate> save

n Enter the key file <path to system-default.key>/system-default.key. Enter the save

command.

[admin-cntrl1]: sslkeyandcertificate> key file:<path to system-default.key>/system-

default.key
[admin-cntrl1]: sslkeyandcertificate> save

n Enable the virtual services if they were deactivated before the changes (optional).

n Log in to the UI, navigate to Templates > Security > SSL/ TLS Certificates and check the
expiry date for the renewed certificate.

Configure Stronger SSL Cipher

This section explains how to Configure Stronger SSL Cipher.

Strength
SSL ciphers are defined by the Templates > Security > SSL/TLS Profile. Within a profile, List view
and string view are the two modes for configuring ciphers.

For more information, see Apple’s App Transport Security topic in the VMware Avi Load
BalancerConfiguration Guide.

SSL Rating
Modifying or reordering the list will alter the associated SSL rating in the top-right corner of the
SSL/ TLS Profile edit window. This provides insight into the encryption performance, security,
and client compatibility of the selected ciphers. This ranking is only made against the validated
ciphers from the List View mode.

VMware by Broadcom 55
VMware Avi Load Balancer Administration Guide

List View
The default cipher list view shows common ciphers in order of priority. Enable or deactivate
ciphers using the check box, and reorder them using the up/ down arrows or drag and drop. List
view provides a static list of validated ciphers. If alternate ciphers not listed are required, consider
using String View. The ciphers included in this list are considered reasonably strong. If a cipher
is later deemed to be insecure or less secure, its security score rating will drop to indicate it has
fallen out of favor.

String View
The second cipher configuration mode allows accepted ciphers to be added as a string, similar to
the OpenSSL syntax for viewing and setting ciphers. For this mode, Avi Load Balancer accepts all
TLS 1.0 - 1.2, and Elliptic Curve ciphers from https://www.openssl.org/docs/man1.0.2/apps/
ciphers.html. In this mode, the administrator must determine if the enabled ciphers are secure.
You can set strong security by employing a known cipher suite, such as HIGH.

ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA:ECDHE-ECDSA-AES256

User Interface Inaccessible Due to Wrong Certificate

There can be instances where the Avi Load Balancer user interface is not accessible, but the
Controller is reachable through CLI access.

VMware by Broadcom 56
VMware Avi Load Balancer Administration Guide

Resolution
If the Avi Load Balancer UI is associated with a revoked certificate, the UI becomes inaccessible.
The CLI can still be accessed as it uses SSH. Add the new certificate to the default NGNIX
configuration file. Use the /etc/nginx/sites-enabled/default command to enable the
Controller to use the correct certificate.

server {
listen 443;
server_tokens off;
more_clear_headers Server;
add_header Strict-Transport-Security "max-age=31536000;
includeSubdomains";
ssl on;
ssl_certificate /var/lib/avi/nginxd/certstore/server_00.cert;
ssl_certificate_key /var/lib/avi/nginxd/certstore/server_00.key;
ssl_certificate /var/lib/avi/nginxd/certstore/server_01.cert;
ssl_certificate_key /var/lib/avi/nginxd/certstore/server_01.key;

Test the reachability of the Avi Load Balancer UI by using the following curl command.

> curl -k -v <URL>

Import Encrypted Private Key to Install an SSL Certificate

You can import an encrypted private key to install an SSL certificate on Avi Load Balancer.
Password associated with an encrypted private key can be provided using the Key Passphrase
option, available in the Avi Load Balancer UI while creating an SSL certificate. Avi Load Balancer
uses the provided passphrase to decrypt the private key while installing the SSL certificate.

The steps to validate the Key Passphrase provided during the SSL certificate creation are as
follows:

Procedure

1 Navigate to Templates > Security > SSL /TLS Certificates.

2 Click Create and choose Application Certificate or Controller Certificate according to the
requirement.

3 From the Type drop-down menu, select Import.

VMware by Broadcom 57
VMware Avi Load Balancer Administration Guide

4 Paste the key text in the Upload or Paste Key (PEM) or PKCS12 File text box.

5 Enter the password in the Key Passphrase.

6 Click Validate to get the password validated by Avi Load Balancer.

Example: Sample Encrypted and Decrypted SSL key

Below is an example of an encrypted SSL key.

VMware by Broadcom 58
VMware Avi Load Balancer Administration Guide

Note A decrypted SSL key file starts with BEGIN PRIVATE KEY as follows.

Venafi Integration
The Avi Load Balancer can be set up to integrate with the Venafi Trust Protection Platform™ for
automation of SSL and TLS certificate life-cycle management. All certificates will be protected
and controlled through TPP. This process is transparent to the Avi Load Balancer Controllers.

Note Minimum Venafi release is Trust Protection Platform 16.3.

Configuration
The Venafi Trust Protection Platform leverages the Avi Load Balancer REST API for all
communications, including creating and updating SSL certificate and keys. No configuration
changes are required on Avi Load Balancer. TPP requires a user name, password, and an IP
address of the Controller. If the Controller is configured with a floating Controller cluster IP, this
address must be used. If no floating IP is configured, the IP address of any Controller can be
used.

For help in configuring Trust Protection Platform, contact the Venafi support team. You can get
the latest version of the required driver, along with instructions for TPP configuration from the
team.

VMware by Broadcom 59
VMware Avi Load Balancer Administration Guide

Workflow
When a new application is created or a new SSL/ TLS certificate is required, use the following
workflow.

n From the Trust Protection Platform, create a new certificate for the Avi Load Balancer. Behind
the scenes, the following happens:

n TPP sends the API calls necessary to generate a CSR on Avi Load Balancer and import it.

n TPP forwards the CSR to the configured certificate authority.

n TPP receives the signed certificate and key from the CA.

n TPP pushes the certificate and key to Avi Load Balancer, along with any required chain
certificates.

n From the Avi Load Balancer, create a new application virtual service, attaching the certificate.

n Any subsequent updates to the certificate received by the Controller will automatically and
transparently be pushed to Service Engines using the certificate.

n TPP will learn the mapping of the virtual service name to the certificate. It will automatically
handle certificate and chain certificate renewals.

IP Access
As a best practice, you can lock down the Avi Load Balancer administrative UI to known IP
addresses. If this has been done, ensure that you include the source IP addresses of the Trust
Protection Platform in the allowed-IP list of Avi Load Balancer for administrative access.

Admin Access

VMware by Broadcom 60
VMware Avi Load Balancer Administration Guide

TPP can use any administrator account that has write-access to SSL/ TLS certificates for the
required tenants. Best practice is to create a new role for SSL administration, with only write
access for SSL/ TLS Certificates enabled. Create a new admin account mapped to this role. This
limits the exposure of this account, and provides better audit logs.

Onboard discovery
The Onboard Discovery feature automates the process of importing certificates into Trust
Protection Platform from network devices where you can monitor, validate, and provision them.

In case of having virtual services setup on Controller with SSL support and a blank setup or
outdated configurations on Venafi TPP, onboard discovery job can be used to generate the
corresponding certificate and application objects for the virtual services in Venafi TPP with pre-
defined parameter values. This makes provisioning possible immediately after the discovery step.
The job can be run automatically at regular intervals of time from Venafi TPP.

Provisioning
Provisioning is the process of pushing attached certificate in Avi Load Balancer Adaptable
App in Venafi TPP to the corresponding virtual service under corresponding tenant, based on
parameters of the Adaptable App. Provisioning is automatically triggered when a certificate is
renewed in Venafi TPP.

The following are the two types of provisioning:

n Central

n In central provisioning, since Venafi completely takes care of generation of certificate, the
certificate is already ready while pushing.

n New certificates are uploaded to Controller with the naming convention: <Common
Name><Valid To as yyMMMdd><Last 4 of Serial>.

n Remote

n In remote provisioning, CSR is generated by the Avi Load Balancer and imported to
Venafi using API calls, which is then used by Venafi to generate certificate.

n New certificates are uploaded to the Controller with the naming convention: <Common
Name>RGEN<Current Date/Time as yyMMdd-HHmmss>.

When a new application is created or a new SSL/ TLS certificate is required, use the following
workflow.

n From the Trust Protection Platform, create a new certificate for Avi Load Balancer. Behind the
scenes, the following happens:

n TPP sends the API calls necessary to generate a CSR on Avi Load Balancer and import it.

n TPP forwards the CSR to the configured certificate authority.

n TPP receives the signed certificate and key from the CA.

VMware by Broadcom 61
VMware Avi Load Balancer Administration Guide

n TPP pushes the certificate and key to the Avi Load Balancer, along with any required
chain certificate.

n From the Avi Load Balancer, create a new application virtual service, attaching the certificate.

n Any subsequent updates to the certificate received by the Controller will automatically and
transparently be pushed to Service Engines using the certificate.

n TPP will learn the mapping of the virtual service name to the certificate. It will automatically
handle certificate and chain certificate renewals.

Integrating Let's Encrypt Certificate Authority with Avi Load Balancer

Let’s Encrypt is a free, automated (automates both issuing and renewing the certificate) and
open certificate authority. The following section describes integration of Let's Encrypt Certificate
Authority with Avi Load Balancer.

SSL/TLS protocol helps keeping an internet connection secure and safeguard any sensitive
data sent between two machines, systems or devices, preventing intruders from reading, and
modifying any information transferred between two machines, systems or devices. Though
SSL/TLS certificate ensures secure, encrypted connections between systems, following are some
challenges around it.

n Manually getting a certificate.

n The cost associated with a certificate signed by CA.

Let’s Encrypt resolves the above challenges. For more information, see Let’s Encrypt.

Working with Let’s Encrypt

Before issuing a certificate, Let’s Encrypt servers validate that the requester controls the domain
names in that certificate using challenges as defined by the ACME standard. Let’s Encrypt uses
the ACME protocol to verify that you control a given domain name and to issue you a certificate.
There are different ways that the agent or client can prove control of the domain.

n Provisioning a DNS record under the domain (as per CSR’s common name).

n Provisioning an HTTP resource under a well-known URI.

The Avi Load Balancer supports HTTP-01 challenge for domain validation.

HTTP-01 Challenge

n Let’s Encrypt gives a token to the ACME client that puts a file on the web server at http://
<YOUR_DOMAIN>/well-known/acme-challenge/<TOKEN>. This file contains the token and a
thumbprint of account key.

n Once the ACME client tells Let’s Encrypt that the file is ready, Let’s Encrypt tries retrieving it
(potentially multiple times from multiple vantage points).

VMware by Broadcom 62
VMware Avi Load Balancer Administration Guide

n If validation checks get the right responses from the web server, the validation is considered
successful, and a certificate is issued.

Note
n As Let’s Encrypt CA communicates on port 80 for HTTP-01 challenge, port 80 must be
opened on the firewall and Let’s Encrypt CA must be able to reach user network (network
where the Avi Load Balancer System is deployed). Let’s Encrypt CA connects through public
network to user’s Avi Load Balancer System on port 80.

n The script automatically creates a virtual service on port 80 for the respective virtual service
listening on port 443/custom SSL Port, only if there is no virtual service already listening on
port 80.

For more information on domain validation, see the following links.

n Challenge Types

n Let’s Encrypt - How It Works

Configuring Let’s Encrypt

Following is the configuration summary for the Let’s Encrypt integration with the Avi Load
Balancer.

1 Get the script which would assist in getting and renewing the certificate.

2 Add the script as controller script on the Avi Load Balancer System.

3 Add user account with customer (limited access only).

4 Create certificate management profile on Avi Load Balancer System.

5 Add virtual service on Avi Load Balancer System.

6 Make sure that the FQDN resolves to public IP and port 80 is open at Firewall.

7 Create CSR and select the configured certificate management profile.

8 Review the list of certificates. Let’s Encrypt CA would push signed certificate.

9 Associate the certificate to the configured virtual service.

Configuring the Avi Load Balancer System

To configure Let’s Encrypt for Avi Load Balancer.

1 Download the script available at letsencrypt_mgmt_profile. To download the file, click Raw
option. Copy the code available.

2 In the Avi Load Balancer, navigate to Templates > Scripts > ControlScripts and click Create.

3 Add a meaningful name and paste the code copied in step 1 in the Import or Paste Control
Script field. Save the configuration.

VMware by Broadcom 63
VMware Avi Load Balancer Administration Guide

4 Configure a custom role by navigating to Administration > Roles. Ensure that read and write
access is enabled for Virtual Service, Application Profile, SSL/TLS Certificates and Certificate
Management Profile, for this role.

5 Add a user by navigating to Administration > Users > CREATE, enter all the required details
and select the configured custom role.

VMware by Broadcom 64
VMware Avi Load Balancer Administration Guide

6 Navigate to Templates > Security > Certificate Management and click Create.

7 Enter a meaningful name, select the configured control script and enable custom parameters,
add custom parameters as shown in the following example:

VMware by Broadcom 65
VMware Avi Load Balancer Administration Guide

Note It is recommended not to use admin account. Always add a user account which has
custom role (with limited access).

8 Navigate to Templates > Security > SSL/TLS Certificates, click Create and select Application
Certificate.

9 Enter meaningful name, common name, select the configured certificate management profile,
add all relevant details and save the configuration.

VMware by Broadcom 66
VMware Avi Load Balancer Administration Guide

Ensure that a virtual service is configured with the Application Domain Name as Common Name
(CN) of certificate. CN of certificate must match with the Application Domain Name of the virtual
service. FQDN (CN of certificate or Application Domain Name of virtual service must resolve to IP
address and the domain must be reachable).

After few minutes, review the list of the certificates. You can see the certificate pushed by Let’s
Encrypt CA. Associate the certificate to the configured virtual service.

Logs
To view the logs, enable non-significant logs for the configured virtual service and generate the
certificate. Following is an example of the log.

VMware by Broadcom 67
VMware Avi Load Balancer Administration Guide

Automation of certificate renewal

Controllerproperties has the configuration for the ssl_certificate_expiry_warning_days. In
the default configuration, the value for ssl_certificate_expiry_warning_days is 30 days, 7 days,
and 1 day. This setting can be modified if required. When certificate renewal is required (based
on configured settings), the script gets activated and automatically takes care of certificate
renewal.

Note Let’s Encrypt CA imposes a rate limit. So ensure that the renewal of the certificate does
not hit the rate limit.

Additional Information
n For more information on rate limit, see Rate Limits.

n For more information on SSL/TLS Certificate, see SSL Certificates.

Certificate Management Integration for CSR Automation

Avi Load Balancer supports automation of the process for requesting and installing a certificate
signed by a certificate authority (CA). This feature handles initial certificate registration and
renewal of certificates based on certificate expiration. This topic explains configuring certificate
management integration and automatic certificate renewal.

Create a certificate management profile which provides a way to configure a path to a certificate
script. Along with the set of parameters the script needs (CSR, common name, and others)
to integrate with a certificate management service within the customer’s internal network.
The script itself is left opaque by design to accommodate the various certificate management
services different customers may have.

VMware by Broadcom 68
VMware Avi Load Balancer Administration Guide

For SSL certificate configuration, select CSR and fill in the necessary fields for the certificate,
and select the certificate management profile to which this certificate is bound. The Avi Load
Balancer Controller will then use the CSR and the script to obtain the certificate and also renew
the certificate upon expiration. As a part of the renewal process, a new key pair is generated and
a certificate corresponding to this is obtained from the certificate management service.

As a part of the SSL certificate configuration, only select CSR, fill in the necessary fields for the
certificate, and select the certificate management profile to which this certificate is bound. The
Avi Load Balancer Controller will then use the CSR and the script to obtain the certificate and
also renew the certificate upon expiration. As a part of the renewal process, a new key pair is
generated and a certificate corresponding to this is obtained from the certificate management
service.

Without the addition of this automation, the process for sending the CSR to the external CA, then
installing the signed certificate and keys, must be performed by the Avi Load Balancer user.

Configuring Certificate Management Integration

This section explains the details to configure certificate management integration.

The following are the steps to configure certificate management integration:

1 Prepare a Python script that defines a certificate_request() method. The method must
accept the following input as a dictionary:

a CSR

b Hostname for the Common Name field.

c Parameters defined in the certificate management profile.

2 Create a certificate management profile that calls the script.

For more information, click here.

Prepare the Script

The script must use the def certificate_request command. For instance,

def certificate_request(csr, common_name, args_dict):

"""
Check if a token exists that can be used:
If not, authenticate against the service with the provided credentials.
Invoke the certificate request and get back a valid certificate.
Inputs:
@csr : Certificate signing request string. This is a multi-line string output like what
you get from openssl.
@common_name: Common name of the subject.
@args_dict: Dictionary of the key value pairs from the certificate management profile.
"""

The specific parameter values to be passed to the script are specified within the certificate
management profile.

VMware by Broadcom 69
VMware Avi Load Balancer Administration Guide

Sensitive Parameters
For parameters that are sensitive, for instance, passwords, the values can be hidden. Marking a
parameter sensitive prevents its value from being displayed in the web interface or being passed
by the API.

Dynamic Parameter
The value for a certificate management parameter can be assigned within the profile or within
individual CSRs.

n If the parameter value is assigned within the profile, the value applies to all CSRs generated
using this profile.

n To dynamically assign a parameter’s value, indicate that the parameter is dynamic within the
certificate management profile. This leaves the parameter’s value unassigned. In this case, the
dynamic parameter’s value is assigned when creating an individual CSR using the profile. The
parameter value applies only to that CSR.

Create the Certificate Management Profile

This section explains how to create the certificate management profile.

Procedure

1 Navigate to Templates > Security > Certificate Management and click Create.

2 Specify the name for the profile.

3 Select an alert script configuration object type for the certificate management profile from
the drop-down list.

4 If the profile need to pass some parameter values to the script, check Enable Custom
Parameters box, and specify their names and values.

In this example, the location (URL) of the CA service and the login credentials for the service,
will be passed to the script. For parameters that are sensitive, such as, passwords, select
Sensitive. Marking a parameter sensitive prevents its value from being displayed in the web
interface or being passed by the API. For parameters that are to be dynamically assigned
during CSR creation, select Dynamic. This leaves the parameter unassigned within the profile.

5 Click Save.

Use the Certificate Management Profile to get Signed Certificates

After adding the script and creating the certificate management profile, the profile can be used
to easily obtain and install CA-signed certificates.

Procedure

1 Navigate to Templates > Security > SSL/TLS Certificates. Select Application Certificate
option from the CREATE dropdown menu.

2 Specify the certificate name and select the type as CSR in the Type field.

VMware by Broadcom 70
VMware Avi Load Balancer Administration Guide

3 Select the profile configured in the previous section from the Certificate Management Profile
dropdown menu.

4 Specify the certificate details and click Save

The Avi Load Balancer Controller generates a key pair and CSR, executes the script to
request the CA-signed certificate from the Avi Load Balancer PKI service, and saves the
signed certificate in persistent storage.

Automatic Certificate Renewal

This section explains about automatic certificate renewal.

You can choose to customize when certificate expiry notifications are sent; see the topic
Certificate Management Integration for CSR Automation section. If the certificate management
profile is configured for a certificate, a renewal is attempted in the last-but-one interval. By
default, Avi Load Balancer Controller generates events 30 days, seven days, and one day before
expiry. In this setting, certificate renewal will be attempted seven days before expiry.

If the certificate management profile is configured for automatic certificate renewal, a renewal is
attempted just prior to the penultimate notification (in the above example, that will be just prior
to the seven-day notification). If the renewal succeeds, the last two notifications are not sent. If
the renewal fails, the penultimate notification is sent. Thereafter, if a manual renewal succeeds
prior to the last notification, it is skipped. Otherwise, the final notification will be sent (with no
accompanying final attempt to renew).

When a certificate renewal occurs, a new expiration date is set and yet another notification
schedule is established per the values within the ssl_certificate_expiry_warning_days array in
force at the time.

SSL Certificates
Avi Load Balancer supports terminating client SSL and TLS connections at the virtual service,
which requires it to send a certificate to clients that authenticate the site and establishes secure
communications.

A virtual service that handles secure connections requires the following:

n SSL/ TLS profile

n This determines the supported ciphers and versions. For more information on SSL/ TLS
profiles, see SSL/TLS Profile topic in the VMware Avi Load BalancerConfiguration Guide.

n SSL Certificate

n This is presented to clients connecting to the site.

n SSL certificates can be used to present to administrators connecting to the Avi Load
Balancer web interface or API, and also for the Avi Load Balancer SE to present to
servers when SE-to-server encryption is required with client (the SE) authentication.

VMware by Broadcom 71
VMware Avi Load Balancer Administration Guide

The SSL/TLS Certificates page on Templates > Security > SSL/TLS Certificates allows the
import, export, and generation of new SSL certificates or certificate requests. Newly-created
certificates may be either self-signed by Avi Load Balancer or created as a Certificate Signing
Request (CSR) that must be sent to a trusted Certificate Authority (CA), that generates a trusted
certificate.

n Creating a self-signed certificate generates both the certificate and a corresponding private
key.

n Imported existing certificates are not valid until a matching key has been supplied.

n Avi Load Balancer supports PEM and PKCS #12 formats for certificates.

SSL/ TLS Certificates Page

Navigate to Templates > Security > SSL/TLS Certificates to open the SSL/TLS Certificates page.
This tab includes the following functionalities:

n Search: Search across the list of objects.

n Create: Shows the list of certificates from the drop-down menu to create a certificate.

n Edit: Opens the Edit Certificatewindow. Only incomplete certificates that do not have a
corresponding key can be edited.

n Export: The down arrow icon exports a certificate and corresponding private key.

n Renew: Renew Certificate objects.

n Delete: A certificate may only be deleted if it is not currently assigned to a virtual service. An
error message will indicate the virtual service referencing the certificate.

The table on this tab contains the following information for each certificate:

n Name: This displays the name of the certificate. Mouse over the name of the certificate
will display any intermediate certificate that has been automatically associated with the
certificate.

n Status: This shows the status of the certificate. Status in 'green' indicates it is good, in
'yellow'/orange/red' indicates the certificate is expiring soon or has already expired, and
'gray' indication, if the certificate is incomplete.

VMware by Broadcom 72
VMware Avi Load Balancer Administration Guide

n Common Name: This displays the fully qualified name of the site to which the certificate
applies. This entry must match the hostname the client will enter in their browser in order for
the site to be considered trusted.

n Issuer Name: This displays the name of the certificate authority.

n Algorithm: This displays the algorithm as either EC (Elliptic Curve) or RSA.

n Self Signed: This displays whether the certificate is self-signed by Avi Load Balancer or
signed by a Certificate Authority.

n Valid Until: This displays the date and time when the certificate expires.

Create Certificate
Navigate to Templates > Security > SSL/TLS Certificates. Click Create to open the New
Certificate (SSL/ TLS)window.

When creating a new certificate, you can select any of the following certificates:

n Root/Intermediate CA Certificate: This certificate is used to automatically create the

certificate chain for application certificates. There are no configuration options other than
importing the certificate through a file or pasting the text. The root/ intermediate certificate
will show up in a separate table at the bottom of the SSL Certificates page. It is
recommended to import the root/intermediate certificate prior to importing an application
certificate that relies on the intermediate for the chain.

n Application Certificate: This certificate is used for normal SSL termination and decryption on
Avi Load Balancer. This option is also used to import or create a client certificate for Avi Load
Balancer to present to a back-end server when it needs to authenticate itself.

n Controller Certificate: This certificate is used for the GUI and API for the Controller cluster.
Once uploaded, select the certification through Administration > Settings > Access Settings.

To create a new certificate, specify the following:

n Name: Specify the name of the certificate.

VMware by Broadcom 73
VMware Avi Load Balancer Administration Guide

n Type: Select the type of certificate to create from the drop-down menu. The following are the
options:

n Self Signed: Quickly create a test certificate that is signed by Avi Load Balancer. Client
browsers will display an error that the certificate is not trusted. If the HTTP application
profile has HTTP Strict Transport Security (HSTS) enabled, clients will not be able to
access a site with a self signed certificate.

n CSR: Create a valid certificate by first creating the certificate request. This request must
be sent to a certificate authority, which will send back a valid certificate that must be
imported back into Avi Load Balancer.

n Import: Import a completed certificate that was either received from a certificate
authority or exported from another server.

n Common Name: Specify the fully qualified name of the site, such as www.vmware.com. This
entry must match the hostname the client entered in the browser in order for the site to be
considered trusted.

n Input the required information required for the type of certificate you are creating:

n Self-Signed Certificates

n CSR Certificates

n Importing Certificates

VMware by Broadcom 74
VMware Avi Load Balancer Administration Guide

Note The OCSP stapling can be enabled and configured using the UI. For more
information, see OCSP Stapling in NSX Advanced Load Balancer topic in the VMware Avi Load
BalancerConfiguration Guide.

VMware by Broadcom 75
VMware Avi Load Balancer Administration Guide

Self-Signed Certificates
Avi Load Balancer can generate self-signed certificates. Client browsers do not trust these
certificates and will warn the user that the virtual service’s certificate is not part of a trust chain.

Self-signed certificates are good for testing or environments where administrators control the
clients and can safely bypass the browser’s security alerts. Public websites must never use
self-signed certificates.

If you have selected Self Signed option from Type drop-down menu in the New Certificate
window, then specify the following details:

n Common Name- Specify the fully-qualified name of the site, such as www.vmware.com. For
the site to be considered trusted, this entry must match the hostname that the client entered
in the browser.

n Organization: Company or entity registering the certificate, such as Avi Load Balancer
Networks, Inc. (optional).

n Organization Unit: Group within the organization that is responsible for the certificate, such
as Development (optional).

n Country: Country in which the organization is located (optional).

n State Name or Province: State in which the organization is located (optional).

VMware by Broadcom 76
VMware Avi Load Balancer Administration Guide

n Locality or City: City of the organization (optional).

n Email: The email contact for the certificate (optional).

n Subject Alternate Name (SAN)- The Subject Alternate Name (SAN) lets you specify
additional host names to be protected by a single SSL certificate, such as example.com and
example.org. The are essentially the alternative identities of the subject that is specified in
the Certificate.

n Algorithm: Select either EC (Elliptic Curve) or RSA. RSA is older and considered less secure
than EC, but is more compatible with a broader array of older browsers. EC is new, less
expensive computationally, and generally more secure; however, it is not yet accepted by all
clients. Avi Load Balancer allows a virtual service to be configured with two certificates at a
time, one each of RSA and EC. This will enable it to negotiate the optimal algorithm with the
client. If the client supports EC, then the Avi Load Balancer will prefer this algorithm, which
gives the benefit of natively supporting Perfect Forward Secrecy for better security.

n Key Size: Select the level of encryption to be used for handshakes, as follows:

Higher values may provide better encryption but increase the CPU resources required by
both Avi Load Balancer and the client.

n 2048-bit is recommended for RSA certificates. Higher values may provide stronger
encryption, but dramatically increase the CPU resources required by both Avi Load
Balancer and the client. For stronger encryption, use ECC certificates instead.

n SECP256R1 is recommended for EC certificates.

n Enable HSM Certificate: Rather than store the private key locally on the Controller or
Service Engine, it is maintained in an external hardware security module. This option enables
referencing an HSM profile containing information about communicating with the HSM.

n After specifying the necessary details, click SAVE.

CSR Certificates
The Certificate Signing Request (CSR) is the first of three steps involved in creating a valid
SSL/ TLS certificate. The request contains the same parameters as a Self-Signed Certificate.
However, Avi Load Balancer does not sign the completed certificate. Rather, it must be signed by
a Certificate Authority that is trusted by client browsers.

You can select CSR option from Type drop-down menu in the New Certificate window. The
configuration options for a certificate signing request are the same as for self-signed certificates.
See the descriptions above for definition of each field.

Once completed, forward the completed CSR to any trusted Certificate Authority (CA), such as
Thawte or Verisign by selecting the Certificate Signing Request at the bottom left of the Add
Certificate popup. Then either copy-paste it directly to the CA’s website or save it to a file for
later use.

VMware by Broadcom 77
VMware Avi Load Balancer Administration Guide

Once the CA issues the completed certificate, either paste or upload it into Certificate field at the
bottom right of Add Certificate popup. The certificate is not usable on Avi Load Balancer until
this step is complete.

Note It can take several days for the CA to return the finished certificate. Meanwhile, you can
close the New Certificate window to return to the SSL/TLS Certificates page. The new certificate
will appear in the table with the notation Awaiting Certificate Valid Until column.

When you receive the completed certificate, click Edit icon for the certificate to open the Edit
Certificate, and then paste the certificate and click Save to generate the CSR certificate. Avi Load
Balancer will generate a key from the completed certificate automatically.

Import Certificates
You can directly import an existing PEM or PKCS /#12 SSL/ TLS certificate into Avi Load Balancer
(such as from another server or load balancer). A certificate will have a corresponding private
key, which must also be imported.

Note Avi Load Balancer generates the key for self-signed or CSR certificates automatically.

The following are the steps to import the certificate:

1 Navigate to Templates > Security > SSL/TLS Certificates.

2 Click CREATE and select the certificate type such as Application Certificate.

3 Click Type and select Import. Certificate or Private Key can be imported by copying-pasting
or uploading a file.

n PEM File – PEM files contain certificate or private key in plain-text Base64 encoded format.
Certificate and private key can be provided in separate PEM files or combined in a single PEM
file.

n If certificate and private key are provided in a single PEM file, navigate to Paste Key text box
and add the private key by following any one of the methods listed below:

n Upload File: Click the Upload File button, select the PEM or PKCS12 file, then click
Validate button to parse the file. If the upload is successful then, the Key field will be
populated.

n Paste: Copy and paste a PEM key into the Key field. Ensure that you do not introduce
extra characters in the text, which can occur while using some e mail clients or rich text
editors. If you copy and paste the key and certificate together as one file then, click
Validate button to parse the text and populate the Certificate field.

n If certificate and private key are provided in two separate PEM files, follow the below steps to
import each individually:

n Certificate - Add the certificate in the Paste Certificate text box by copying-pasting or file
upload, as described above.

VMware by Broadcom 78
VMware Avi Load Balancer Administration Guide

n Key – Add the private key in the Paste Key field by copying-pasting or file upload.

n PKCS#12 File - PKCS #12 files contain both the certificate and key, PKCS #12 is a binary
format, which cannot be copied-pasted, and hence it can be uploaded only. Navigate to the
Paste Key and follow the below step to import the PKCS #12 file.

n Upload File - Click the Import File button, select the PKCS #12 file, click the Validate button
to parse the file. If the upload is successful, both the Key and Certificate fields will be
populated.

n Key Passphrase: You can also add and validate a key passphrase to encrypt the private key.

n Import: Select Import to finish adding the new certificate and key. The key will be embedded
with the certificate and treated as one object within the Avi Load Balancer UI.

Certificate Authority
Certificates require a trusted chain of authority to be considered as valid. If the certificate
used is directly generated by a certificate authority that is known to all client browsers then,
no certificate chain is required. However, if there are multiple levels required, an intermediate
certificate may be necessary. Clients will often traverse the path indicated by the certificate to
validate on their own if no chain certificate is presented by a site, but this adds additional DNS
lookups and time for the initial site load. The ideal scenario is to present the chain certificates
along with the site cert.

If a chain certificate, or rather a certificate for a certificate authority, is uploaded through the
Certificate > Import in the certificates page, it will be added to the Certificate Authority section.
Avi Load Balancer will automatically build the certificate chain if it detects a next link in the chain
exists.

To validate a certificate that has been attached to a chain certificate, hover the pointer over the
certificate’s name in the SSL Certificates table at the top of the page. Avi Load Balancer supports
multiple chain paths. Each may share the same CA issuer, or they can be chained to different
issuers.

SSL/ TLS Profile

Avi Load Balancer supports the ability to terminate SSL connections between the client and the
virtual service, and to enable encryption between Avi Load Balancer and the back-end servers.
The SSL/ TLS profile contains the list of accepted SSL versions and the prioritized list of SSL
ciphers.

Both an SSL/ TLS profile and an SSL certificate must be assigned to the virtual service while
configuring it to terminate client SSL/ TLS connections. If you encrypt traffic between Avi
Load Balancer and the servers then, an SSL/ TLS profile must be assigned to the pool. While
creating a new virtual service through the basic mode, the default system SSL/TLS profile is used
automatically.

VMware by Broadcom 79
VMware Avi Load Balancer Administration Guide

SSL termination can be performed on any service port. However, browsers assume that the
default port as 443. The best practice is to configure a virtual service to accept both HTTP and
HTTPS by creating a service on port 80, by selecting the + icon to add an additional service port,
and then set the new service port to 443 with SSL enabled. A redirect from HTTP to HTTPS is
generally preferable, which can be done through a policy or by using the System-HTTP-Secure
application profile.

Each SSL/ TLS profile contains default groupings of supported SSL ciphers and versions that may
be used with RSA or an Elliptic Curve certificate, or both. Ensure that any new SSL/TLS profile
you create, includes ciphers that are appropriate for the certificate type that will be used later.
The default SSL/ TLS profiles included with Avi Load Balancer provides a broad range of security.
For instance, the Standard Profile works for typical deployments.

Creating a new SSL/ TLS profile or using an existing profile entails various trade-offs between
security, compatibility, and computational expense. For instance, increasing the list of accepted
ciphers and SSL versions increases the compatibility with clients while also lowering security
potentially.

Note Avi Load Balancer can accommodate a broader set of security needs within a client
community by associating multiple SSL profiles with a single virtual service, and have the Service
Engines choose which to use based on the client’s IP address. For more information, see 'Client-
IP-based SSL Profiles' section in this guide.

SSL Profile Settings

Navigate to Templates > Security > SSL/TLS Profile.

n Search: Searches across the list of objects.

n Create: Opens the new application or systemprofile.

n Edit: Opens the existing profile to edit.

n Delete: An SSL/ TLS profile can only be deleted, if it is not currently assigned to a virtual
service. An error message will indicate the virtual service referencing the profile. The default
system profiles can be modified, but not deleted.

The table on this tab provides the following information for each SSL/ TLS profile:

n Name: Name of the profile.

n Accepted Versions: SSL and TLS versions accepted by the profile.

VMware by Broadcom 80
VMware Avi Load Balancer Administration Guide

Create an SSL Profile

The following are the fields to be edited in the Create / Edit SSL page:

n Name: Specify a unique name for the SSL/ TLS Profile.

n Type: Select the type of profile from the drop-down menu.

VMware by Broadcom 81
VMware Avi Load Balancer Administration Guide

n Accepted Versions: Select one or more SSL/ TLS versions from the drop-down menu to add
to this profile. Chronologically, TLS v1.0 is the oldest supported, and TLS v1.2 is the newest.
SSL v3.0 is no longer support as of Avi Load Balancer v15.2. In general with SSL, older
versions have many known vulnerabilities while newer versions have many undiscovered
vulnerabilities. As with any security, Avi Load Balancer recommends diligence to understand
the dynamic nature of security and to ensure that Avi Load Balancer is always up to date.
Some SSL ciphers are dependent on specific versions of SSL or TLS supported. For more
information, see OpenSSL.

n Accepted Ciphers: Enter the list of accepted ciphers in the Accepted Ciphers field. Each
cipher entered must conform to the cipher suite names listed at OpenSSL. Separate each
cipher with a colon. For instance, AES:3DES means that this Profile will accept the AES
and 3DES ciphers. When negotiating ciphers with the client, Avi Load Balancer will prefer
ciphers in the order listed. You may use an SSL/ TLS profile with both an RSA and an Elliptic
Curve certificate. These two types of certificates can use different types of ciphers, so it is
important to incorporate ciphers for both types. Selecting only the most secure ciphers may
incur higher CPU load on Avi Load Balancer and may also reduce compatibility with older
browsers.

PKI Profile
For more information on PKI Profile, see PKI Profile and PKI Profile Settings topics in the VMware
Avi Load BalancerConfiguration Guide.

Certificate Management
To create a new certificate, follow the steps below:

1 From the UI, navigate to the Templates > Security > Certificate Management.

2 Click CREATE.

3 In the New Certificate Management screen, enter the Name of the profile.

4 In the Control Script field, select the required alert script configuration, as required.

Note Click Create button in the drop down, to create a new Control Script (if required).

5 If the profile needs to pass some parameter values to the script, select Enable Custom
Parameters.

6 Enter the Name and Value for the parameters.

VMware by Broadcom 82
VMware Avi Load Balancer Administration Guide

Note Re-upload the Control Script, if the file has been modified after uploading for the
changes to reflect.

The location (URL) of the Trust Anchor service and the login credentials for the service, will
be passed to the script. For parameters that are sensitive (for instance, passwords), select
the Sensitive check box.

Marking a parameter sensitive prevents its value from being displayed in the web interface
or being passed by the API. For parameters that are to be dynamically assigned during CSR
creation, select the Dynamic check box. This leaves the parameter unassigned within the
profile.

7 Click Save.

VMware by Broadcom 83
VMware Avi Load Balancer Administration Guide

Authentication Profile
The Authentication profile (“auth profile”) allows configuration of clients into a virtual service
through HTTP basic authentication.

The authentication profile is enabled through the HTTP basic authentication setting of a virtual
service’s Advanced Properties tab.

Avi Load Balancer also supports client authentication through SSL client certificates, which is
configured the HTTP Profile’s Authentication section.

Auth Profile Settings

Select Templates > Security > Auth Profile to open the Auth tab. This tab includes the following
functions:

n Search: Search across the list of objects.

n Create: Opens the Create/ Edit window.

n Edit: Opens the Create/ Edit window.

VMware by Broadcom 84
VMware Avi Load Balancer Administration Guide

VMware by Broadcom 85
VMware Avi Load Balancer Administration Guide

n Delete: An Auth profile may only be deleted if it is not currently assigned to a virtual service
or in use by Avi Load Balancer for administrative authentication.

The table on this tab provides the following information for each authorization profile:

n Name: Name of the profile.

n Type: The Type will be LDAP.

Create an Authentication Profile

The following fields are available during creation or editing of an authentication profile:

n Name: Enter a unique name.

n LDAP Servers: Configure one or more LDAP servers by adding their IP addresses.

n LDAP Port: This is the service port to use when communicating with the LDAP servers. This is
typically 389 for LDAP or 636 for LDAPS (SSL).

n Secure LDAP using TLS: Enable startTLS for secure communications with the LDAP servers.
This may require a service port change.

n Base DN: LDAP Directory Base Distinguished Name. Used as default for settings where DN is
required but was not populated like User or Group Search DN.

n Anonymous Bind: Minimal LDAP settings that are required to verify User authentication
credentials by binding to LDAP server. This option is useful when you do not have access
to administrator account on the LDAP server.

n User DN Pattern: LDAP user DN pattern is used to bind an LDAP user after replacing the
user token with real user name. The pattern must match the user record path in the LDAP
server. For instance, cn=,ou=People,dc=myorg,dc=com is a pattern where we expect to
find all user records under ou “People”. When searching LDAP for a specific user, we
replace the token with user name.

n User Token: An LDAP token is replaced with real user name in the user DN pattern. For
instance, inUser DN Pattern is configured as “cn=-user-,ou=People,dc=myorg,dc=com”,
the token value must be -user-.

n User ID Attribute: LDAP user ID attribute is the login attribute that uniquely identifies a
single user record. The value of this attribute must match the user name used at the login
prompt.

VMware by Broadcom 86
VMware Avi Load Balancer Administration Guide

n User Attributes: LDAP user attributes to fetch on a successful user bind. These attributes
are used only for debugging purpose.

n Administrator Bind: LDAP administrator credentials configured under LDAP Directory

Settings below are used to bind Avi Load Balancer as an admin when querying LDAP for
Users or Groups.

n Admin Bind DN: Full DN of LDAP administrator. Admin bind DN is used to bind to an
LDAP server. Administrators must have sufficient privileges to search for users under user
search DN or groups under group search DN.

n Admin Bind Password: Administrator password. Password expiration or change is not

handled. The password is hidden from Rest API and CLI.

n User Search DN: LDAP user search DN is the root of search for a given user in the
LDAP directory. Only user records present in this LDAP directory sub-tree are allowed for
authentication. Base DN value is used if this value is not configured.

n Group Search DN: LDAP group search DN is the root of search for a given group in the
LDAP directory. Only matching groups present in this LDAP directory sub-tree will be
checked for user membership. Base DN value is used if this value is not configured.

n User Search Scope: LDAP user search scope defines how deep to search for the user
starting from user search DN. The options are search at base, search one level below
or search the entire subtree. The default option is to search one-level deep under user
search DN.

n Group Search Scope: LDAP group search scope defines how deep to search for the
group starting from the group search DN. The default value is the entire subtree.

n User ID Attribute: LDAP user ID attribute is the login attribute that uniquely identifies a
single user record. The value of this attribute must match the user name used at the login
prompt.

n Group Member Attribute: LDAP group attribute that identifies each of the group
members. For instance, member and memberUid are commonly used attributes.

n User Attributes: LDAP user attributes to fetch on a successful user bind. These attributes
are only for debugging.

n Insert HTTP Header for Client UserID: Insert HTTP header into the client request before it is
sent to the destination server. This field is used to name the header. The value will be the
client’s User ID. This same User ID value will also be used to populate the User ID field in the
Virtual Service’s logs.

n Required User Group Membership: User must be a member of these groups. Each group is
identified by the DN. For instance, cn=testgroup,ou=groups,dc=LDAP,dc=example,dc=com

n Auth Credentials Cache Expiration: The maximum allowed length of time a client’s
authentication is cached.

VMware by Broadcom 87
VMware Avi Load Balancer Administration Guide

n Group Member Attribute Is Full DN: Group member entries contain full DNs instead of only
User ID attribute values.

VMware by Broadcom 88
Licensing
3
The Avi Load Balancer software requires a license to enable and utilize the available load
balancing features. This section lists the Avi Load Balancer license editions and describes the
steps to obtain, add, and manage them.

Read the following topics next:

n Avi Load Balancer Editions

n Terms of Avi Load Balancer Software License

n Removing Trial Licenses from an Avi Load Balancer Controller

n Updating License on each Node in an Avi Load Balancer Controller Cluster

Avi Load Balancer Editions

The following are the various editions supported on Avi Load Balancer:

n VMware Avi Load Balancer Enterprise with Cloud Services Edition

n VMware Avi Load Balancer Enterprise Edition

n VMware Avi Load Balancer – Basic Edition

n VMware Avi Load Balancer Essentials for Tanzu

Note The Basic Edition of Avi Load Balancer (formerly known as NSX Advanced Load Balancer)
has reached its End of Availability (EoA) date on Jan 30th, 2024. The changes only apply to the
Basic Edition of Avi Load Balancer. For additional details, please visit VMware Knowledge Base.

The next section explains the detailed feature list for each of the editions.

VMware by Broadcom 89
VMware Avi Load Balancer Administration Guide

Feature Comparison
Enterprise with Cloud
Categories Feature Essentials Basic Enterprise Services Edition

Local Traffic L4 LB TCP & UDP TCP & UDP TCP, UDP, DNS, SIP, TCP, UDP, DNS, SIP,
Managemen Fast Path Fast Path & RADIUS DSR, TLS support, RADIUS DSR, TLS support,
t No SSL/TLS ProxyNo L4 PROXY protocol support PROXY protocol support
SSL/TLS

L7 LB X Limited to HTTP/2, content rewrite, HTTP/2, content rewrite,

basic L7 compression, caching compression, caching
features and
no HTTP/2

L3/4 Policies X Policies Full-featured, including Full-featured, including

limited to DataScripts and live IP DataScripts and live IP
connection Reputation Reputation
drops
No
DataScript
support

L7 Policies X Basic match Match on: IP Reputation Match on: IP Reputation

and actions DB, string groups, and so DB, string groups, and so
on on
Actions to: Rate-limit, ICAP, Actions to: Rate-limit, ICAP,
and so on and so on

Global Global X X Enterprise grade GSLB Enterprise grade GSLB

Traffic Server Load
Managemen Balancing
t

Application SSL/TLS X Very limited Dynamic CRLs, OCSP Dynamic CRLs, OCSP
Security feature-set stapling, TLSv1.3, HSM, and stapling, TLSv1.3, HSM, and
cert management cert management

Application X X Rate limiters can be applied Rate limiters can be applied

Rate for L4, L7, DNS, and WAF for L4, L7, DNS, and WAF
Limiting

DDoS X X L3, L4, and L7 DDoS L3, L4, and L7 DDoS

Protection protection protection

Intelligent X X CRS, learning/PSM, CRS, learning/PSM,

WAF IPReputation, application IPReputation, application
signatures signatures, bot detection
For more information on
bot detection, see BOT
Management Service topic
in the VMware Avi Load
BalancerCloud Services
Guide

Container Service Yes Yes Yes Yes

Ingress Type LB

VMware by Broadcom 90
VMware Avi Load Balancer Administration Guide

Enterprise with Cloud

Categories Feature Essentials Basic Enterprise Services Edition

Container X X Full Ingress Full Ingress

Ingress DNS and GSLB integration DNS and GSLB integration
Multi K8s cluster & multi-AZ Multi K8s cluster & multi-AZ
support support

Software Administrati Basic LB Basic LB Fully multi-tenant & Fully multi-tenant &
Defined on administrati administrati granular RBAC granular RBAC
Platform on on Rich alerts and events Rich alerts and events
Various 3rd part Various 3rd part
integrations integrations
Integrations with various Integrations with various
IDPs IDPs

Scale and Active/ Active/ Active/Active deployments Active/Active deployments

HA Standby Standby BGP and ECMP support BGP and ECMP support
only only Autoscale of load balancers Autoscale of load balancers

Flexible X X Flexible LCM across Flexible LCM across

upgrades tenants/clouds/se-groups tenants/clouds/se-groups

Ecosystem No Access No Access Native integrations with Native integrations with

automation Cloud Cloud all major public & private all major public & private
vCenter NSX-T clouds and container clouds and container
Cloud orchestration platforms orchestration platforms

Pulse X X Enhanced case Enhanced case

management management
Live Security Threat
Note Proactive case
Updates
creation is supported.

For more information on

Proactive case creation,
see Proactive Support
topic in the VMware
Avi Load BalancerCloud
Services Guide
Live Security Threat
Updates

Advanced Application X X Advanced application Advanced application

Analytics Visibility/ telemetry telemetry
Analytics Log insights Log insights

Functioning of Licensing Editions

The following are a few points on how the licensing editions work in the Avi Load Balancer
Controller:

n The Enterprise with Cloud Services Edition is the default licensing tier for an Avi Load
Balancer Controller. A new Controller is set up to be in the Enterprise with Cloud Services
Edition licensing tier.

VMware by Broadcom 91
VMware Avi Load Balancer Administration Guide

n Customers can request trial service units (saas units) valid for up to 90 days to try out
features in the Cloud Services edition. Contact the Avi Load Balancer representative for more
details.

n VMware continues to ship trial licenses for the Enterprise edition as part of the packaging.
The customer must change the license tier to Enterprise to utilize these licenses.

n The Avi Load Balancer can be set up in any one of the licensing tier. However, the entire
Controller cluster operates in the same license edition tier to which all applications and SEs
belong..

n The License tier for the Controller can be changed from one edition to another without any
disruptions of traffic or downtime.

Avi Load Balancer and NSX Editions

Two different NSX Advanced Load Balancer entitlements are available with VMware NSX:

n Avi Load Balancer-Basic Edition

n Avi Load Balancer-Enterprise Edition

For more information on Enterprise and Basic Editions, see Avi Load Balancer-Enterprise Edition
and Avi Load Balancer - Basic Edition.

Starting 8th May 2023, specific NSX Per-Core Term bundles started bundling Avi Load Balancer
with a 250:1 ratio.

To activate Avi Load Balancer Enterprise licenses that are included as part of NSX, access
VMware Customer Connect to obtain the Serial Keys. The Avi Load Balancer Controller will
decode the Serial Keys and assign the appropriate number of Service Units based on the
specified ratio. The license will be accounted as Service Units from the Controller perspective.

For more information on NSX editions and product, see the NSX+ Features and NSX Features
sections on VMware NSX Overview.

Note Since the NSX bundle SKUs are based on cores, the same amount of NSX and Avi Load
Balancer for NSX cores can be seen on the Order and VMware Customer Connect. To calculate
the amount of Service Units available as part of the entitlement, the amount of cores must be
divided by 250 and rounded up.

Avi Load Balancer Enterprise with Cloud Services Edition

The Avi Load Balancer with Cloud Services provides multi-cloud load balancing, web application
firewall, application analytics, and container services from the data center to the cloud, with
enhanced operations delivered through SaaS.

For more information, see VMware Avi Load Balancer Cloud Services Guide.

VMware by Broadcom 92
VMware Avi Load Balancer Administration Guide

Avi Load Balancer-Enterprise Edition

The new license enforcement model enables product enhancement to incorporate unified
licensing around Service Units.

The Enterprise Edition license is based on Service Units. On upgrade to Avi Load Balancer release
20.1.1, the licenses prior to Avi Load Balancer release 20.1.1 are converted to Service Unit licenses.

Starting 8th May 2023, Avi Load Balancer-Enterprise Edition is included as part of specific NSX
Per-core Term bundles. Contact your VMware representative for more details on Avi Load
Balancer entitlements that are part of NSX Bundles.

If you are upgrading from version 18.x to Avi Load Balancer version 22.1.3, use the following table
to understand how the various license types are converted to the Enterprise Edition licenses.

License prior to Avi Load Balancer 20.1.1 (1 unit) Corresponding Service Unit License

Socket 5

Core (Avi Vantage Core - Prior to 2020) 1

Host 1

25 Mbps Bandwidth 0.4

200 Mbps Bandwidth 0.7

Depending on the configuration of Avi Load Balancer, the appropriate number of Service units
are consumed by the SEs.

The configuration depends on the following parameters, which are explained in the later sections:

n Bandwidth restriction

n Per-app mode

n Virtual Machines parameters

n Bare-metal parameters

n Cloud-specific configuration

The license is downloaded from the Avi Load Balancer web portal and is administered centrally
by the Avi Load Balancer Controller to all SEs. Licenses are consumed by the SEs and not by the
Controllers.

The following environments support Avi Load Balancer-Enterprise Edition:

n Azure VMware Solution

n Google Cloud VMware Engine

n Oracle Cloud VMware Solution

n VMware Cloud on AWS

VMware by Broadcom 93
VMware Avi Load Balancer Administration Guide

The license edition provides the following advantages:

n Aligns Avi Load Balancer with VMware price list

n Simplifies commercial model

n Provides flexibility – Same Service Unit license can be used on any ecosystem and using any
configuration

n Simplifies reporting and projection

Enterprise Edition License

The Enterprise Edition is a fully-featured Avi Load Balancer license that includes load balancing,
GSLB, WAF, and all other features available on Avi Load Balancer.

Note
n Future-dated license is not supported.

n Enterprise Edition comes with two units of a free perpetual license. This license does not
include product support and is only intended for PoC or trial use.

License Duration
Avi Load Balancer is available in the following license types, based on the duration for which the
license is valid.

License Type Description

Term n Available for 1-year and 3-year pre-paid terms

n It has all the features such as load balancing, GSLB, WAF, and so on
n Features related to Cloud Services are not available (Cloud Services features such as
Centralized Licensing are available in Subscription SKUs only)
n Support included

Perpetual n Perpetual License with features like Term

n Also needs additional per-year Support and Subscription (SnS)

Note Perpetual SKUs are deprecated in December 2021.

Subscription n Available for 1-year and 3-year pre-paid terms

n Includes all features and the Avi Load Balancer Cloud Services features (such as
Centralized Licensing)
n Includes Support

Obtaining an Avi Load Balancer License

Avi Load Balancer licenses can be downloaded by authorized persons from the customer portal
or obtained by contacting the Avi Load Balancer sales team.

To activate a license, you must first add the license to the Avi Load Balancer Controller.

VMware by Broadcom 94
VMware Avi Load Balancer Administration Guide

Adding a New License

You can add a new license using one of the following options:

Using VMware Serial Key

You can activate the license manually by entering the VMware serial key.

After adding a 25-character VMware serial key, the system will display information about the
license, such as start date, expiry date, resource count, and so on, available in the Avi Load
Balancer Controller license list.

Note Subscription-based serial keys with more than 90-day validity can have trouble functioning
on the Controllers running 18.2.10 or lower versions.

The start date of such licences appears to be in the future, even when the serial key is correctly
added. For instance, if the serial key is issued on 01-JAN-2021 for a 1-year validity, the start date
can appear as 01-OCT-2021. However, the expiration date will be accurately set to 31-DEC-2021.
You must update to version 18.2.11 or later to fix it.

You must reconfigure (by deleting the added but unusable serial key) such future-dated serial
keys after the upgrade.

Using Avi Load Balancer License File

After obtaining a license from Avi Load Balancer, you can activate the license by adding it to the
Avi Load Balancer Controller.

Procedure

1 Navigate to Administration > Licensing.

2 Click Upload a License File (.lic).

VMware by Broadcom 95
VMware Avi Load Balancer Administration Guide

3 Click IMPORT FILE and navigate to the file.

4 After the license file is uploaded, the new license appears in the license list of the Controller.

Results

The system displays information about the license, including the start date and expiration date.

Divide or Combine Licenses

You can divide or combine VMware serial key licenses.

For more information on divide and combine licenses, see How to Divide or Combine License
Keys guide.

Note If you are using old YAML-based licenses generated by the Avi Load Balancer, contact
your accounts team or VMware support for the new ones.

Various Parameters and Configuration Impacting License Consumption

The following is the list of configurations and parameters that impact Enterprise Edition license
allocation:

n Bandwidth mode licensing

n Per-App mode licensing

n Bare-metal server licensing

n Public cloud licensing

n Azure cloud licensing

VMware by Broadcom 96
VMware Avi Load Balancer Administration Guide

Bandwidth Mode Licensing

Bandwidth mode licensing is used to restrict the use of bandwidth on SEs. Using this licensing
method, the number of datapath processes on SEs is restricted. The SEs no longer require a
separate bandwidth-based license to enable the bandwidth mode. When the bandwidth mode is
enabled on the SEs, the bandwidth is consumed from the central pool of licenses, which is the
capacity of the Service Units license.

Note
n 1 Gbps bandwidth mode is no longer available.

n Per-App mode is not supported on SE Groups when the Bandwidth mode is enabled.

Bandwidth Mode Consumed License of Service Units Number of Datapath Processes

25 Mbps 0.4 1

200 Mbps 0.7 2

Per-App Mode Licensing

In the Per-App mode licensing, only two virtual services are supported per SE.

The key points regarding Per-app Mode licensing are as follows:

n It is not supported for SEs enabled with the Bandwidth Mode.

n It is not supported by the Azure Marketplace.

n It cannot be used to host DNS Virtual Services (both standalone DNS and GSLB).

n 0.25 Service Units license is required to license one core of load balancing license.

n For an SE VM with eight physical cores, two Service Units licenses are required for the
load balancing.

n For SNI or EVH shared virtual services, Per-App is limited to only one parent and one child
virtual service.

Bare-Metal Server Licensing

The following are the key points regarding Bare-Metal server licensing:

n The socket license is no longer available.

n When you upgrade a bare-metal server, one socket license gets converted to five Service
Unit licenses.

n Bare-metal servers are also licensed following the Service units.

n The number of Service Unit licenses consumed by a bare-metal server depends on the
following:

n the number of sockets associated with the bare-metal server

n the number of cores associated with each socket

VMware by Broadcom 97
VMware Avi Load Balancer Administration Guide

n Calculating Service Units license for various bare-metal servers based on their configuration:

n For 16 cores, dual-socket, bare-metal server (16 cores on each socket), the following is
applicable:

n For each socket, and five licenses are required, as up to 20 cores, five Service Units
licenses are required for one socket.

n After the limit of 20 cores per socket, one Service Units license for every four additional
cores is required.

n For a bare-metal server configured with 24 cores, six Service Units licenses are
required.

n For a bare-metal server configured with 28 cores, seven Service Units licenses are
required.

n For a dual-socket, 28 core bare-metal servers, and 14 Service Units licenses are
required.

Public Cloud Licensing

The following are the key points regarding the public cloud licensing for the SEs:

n 1 Gbps bandwidth SKU is no longer available.

n Hyper-threaded vCPUs are not accounted for licensing, regardless of the ecosystems.

n Ability to define custom number of datapath processes (number of cores used for load
balancing). It allows for cost-effective use of larger instances in a public cloud.

n You can change bandwidth configuration in SE Group even when it has SEs in it.

n SE might have to be rebooted for the bandwidth configuration to take effect.

n The restrictions based on the instance flavor has been removed.

Azure Marketplace Licensing

The Enterprise Edition license impacts the Azure Marketplace licensing or Azure pay-as-you-go
(Azure PAYG) licensing in the following ways:

n 1 Gbps bandwidth SKU is not supported.

n Supported license types are as follows:

n 25 Mbps bandwidth license

n 200 Mbps bandwidth license

n Existing SEs on 1 Gbps SKU will continue to be billed following the older SKU.

n The bandwidth restriction for the SEs is removed.

n New SEs creation with 1 Gbps SKU is not supported.

n Instance-level restriction is removed. Any instance type can be used to deploy an Avi Load
Balancer Controller.

VMware by Broadcom 98
VMware Avi Load Balancer Administration Guide

For the PAYG licenses, the recommended sizes for an Avi Load Balancer SE are as follows:

Maximum vCPU Allowed

License Unit for Service Engine Recommended VM Sizes Other Allowed VM Sizes

25 Mbps 2 F2s_V2, DS2_v3 Any VM <= 2 vCPUs

200 Mbps 4 F4s_v2, DS4_v3 Any VM <= 4 vCPUs

For more information on PAYG licenses, see Azure Pay-as-you-go (Azure PAYG) section in the
VMware Avi Load BalancerInstallation Guide.

Updating an Existing License

Updating an existing license follows the same process as adding a new license. When a license
file has the same license ID (with an updated limit and expiration date), uploading this license file
will update the existing entry.

Upgrading to Enterprise Edition

All the previously deployed licenses are converted to equivalent Service unit-based licenses
when an Avi Load Balancer Controller is upgraded.

The points to consider while upgrading to an Enterprise Edition license are as follows:

n The number of licenses consumed will be based on the number of SE_DPs running and not
the number of cores on the SE.

n SEs associated with the Controller before the upgrade will continue to work. After the
upgrade, all previous license types will be migrated to Service Units licenses. The number
of consumed Enterprise Edition licenses is based on the number of SE_DPs running on the
SEs and not on the number of cores on the SEs. Modification of the number of datapath
processes for the SE is supported.

n A 1 Gbps bandwidth license is no longer available.

n The Enterprise Edition license supports a 10% buffer, over and above the licensed resources
on the Controller. This 10% buffer is relative to the Service Units license available on the
Controller. It is recommended to use the buffer allowance temporarily and for emergency
requirements like a scale-out event.

n For a Controller with 20 Service Units licenses, the effective Service Units after adding the
buffer allowance is 22 Service Units (20 + 2).

Impacts on Service Engine Group

For SE groups, all the existing Enterprise_18 edition licenses are migrated to the Enterprise
Edition license. All the existing license types are migrated to the Service Units licenses, except for
pay-as-you-go (Azure marketplace).

A 1 Gbps bandwidth is set to unlimited, and the corresponding number of SE_DPs is set to 4. The
25 Mbps bandwidth and 200 Mbps bandwidth licenses are not migrated.

VMware by Broadcom 99
VMware Avi Load Balancer Administration Guide

Impact on Cloud Objects

The default licensing tier for the Avi Load Balancer Controller cluster is updated to Enterprise.

Migration of Service Engines using 1 Gbps Bandwidth

One unit of 1 Gbps bandwidth license is converted to four units of Service Units licenses.

The following are applicable to the SE groups on which the SEs (with 1 Gbps bandwidth license)
are running.

n The bandwidth mode is set to unlimited.

n Bandwidth restriction is removed.

n SE is set to consume only four Service Units licenses, as the value of the max_se_dp setting on
the SE Group is set to 4. The number of cores used for load balancing is set to 4. Therefore, a
maximum of four Service Units licenses are used.

n 1 Gbps license is no longer available.

VMware Customer Connect Portal Self-Registration

This section lists step-by-step instructions to self-register using the customer portal.

Registration
n Navigate to the home page and click Support. This guides you to the VMware Customer
Connectportal. Click Register to navigate to the Login Information page.

VMware by Broadcom 100

VMware Avi Load Balancer Administration Guide

n Fill in the required details in the Login Information page.

n Fill out the mandatory fields and click REGISTER.

n Upon submission of the form, an activation email is sent to the provided email address.

Activation
n Type the Authentication Code received in the email, in the space provided.

VMware by Broadcom 101

VMware Avi Load Balancer Administration Guide

n If activation is successful, you are redirected to the Profile Activated page and subsequently
to the vmware Customer Connect login page as shown below.

VMware by Broadcom 102

VMware Avi Load Balancer Administration Guide

Additional Configuration Options

The following are the additional configuration options:

Limiting Number of Service Units Used by SEs

The number of Service Units used for the load balancing can be customized by using the
max_num_se_dps option on the SE group. Customization of the number of cores for the load
balancing feature enables the use of larger VM sizes but with fewer cores for load balancing.

The use of a customized core has the following benefits:

n Saving on licensing costs.

n Useful in cases where a larger VM is required for better packet per second (PPS) or VIP/VS
density.

VMware by Broadcom 103

VMware Avi Load Balancer Administration Guide

SE creates one datapath process per core. To restrict the number of cores used for
load balancing, set the max_num_se_dps setting to the desired value on the SE Group. The
max_num_se_dps parameter represents all cores on the VM that is used for load-balancing and
its default value is zero.

Note
n The configuration of the max_num_se_dps parameter is supported on all the ecosystems.
The number of Service Units licenses assigned to the SE depends on the number of SE_DPs
running. By default, an eight-core SE will consume eight Service Units licenses. However, if
max_num_se_dps is set to 4, even on an eight-core SE VM, only four Service Units licenses are
consumed.

n When an SE group with a 1 GB bandwidth limit is migrated, the bandwidth limit is set to
unlimited. The value of the max_num_se_dps attribute is set to 4. For all the other SE Groups,
the value of max_num_se_dps is set to 0.

n Hyper-threaded cores are not accounted for licensing on all the ecosystems. For more
information on Service Engine groups, see Service Engine Group topic in the VMware Avi
Load BalancerConfiguration Guide.

Checking License Usage

To check license usage information:

1 Navigate to Administration > Licensing.

The system displays the license limits and current usage.

Note To check license usage information using CLI, use show license ledger details
command.

VMware by Broadcom 104

VMware Avi Load Balancer Administration Guide

Reaching the License Limit

When a license limit is reached, you can neither create nor register new Avi Load Balancer SEs
with the Avi Load Balancer Controller. However, you can use the existing SEs for existing or new
virtual services.

In the case of container cloud environments, after the license limit has been reached, you cannot
create new SEs even if you add new hosts to the cluster. Any new tasks or containers created
on new container hosts can have degraded access to the existing load-balanced applications
because SEs are not present on those hosts.

License Expiration
If multiple licenses are attached to the Avi Load Balancer, and one of the licenses expire, the
system immediately prevents the capacity from exceeding the total resources allotted by the
remaining valid licenses.

When all applied licenses expire, the Avi Load Balancer returns to an unlicensed state. In the
unlicensed state, the following changes become applicable:

n There is no impact on existing virtual services.

n You cannot create new SEs.

n You can still create and edit virtual services and other load-balancing configurations.

n The system defaults to the free, perpetual limits.

n When a valid license is reapplied, the system will immediately be able to utilize the total
resources allocated by the new license.

n If you have an Enterprise with Cloud license tier and the license expires, an SE reboot will fail.

New licenses can be managed and downloaded from the Customer Portal. The interval for
expiration notification is two months.

License Expiry Events

The Avi Load Balancer automatically generates a LICENSE_EXPIRY event once every day,
starting one month before expiry. The events are included in the events table.

Procedure

1 Navigate to Operations > Events > All Events.

2 Search for license_expiry under Event Code (event_id) in the table.

3 click the + sign at the right of the row.

Results

The details of the particular event are displayed.

VMware by Broadcom 105

VMware Avi Load Balancer Administration Guide

License Expiry Alerts

In addition to system-generated license expiry events, you can also configure the Avi Load
Balancer to generate license-expiration alerts. Based on the alert configuration, the alerts are
triggered by license expiration events and are defined as you prefer, namely, through emails,
Syslog messages, SNMP traps, and ControlScripts.

FAQs on Expired License

This section answers FAQs on expired Avi Load Balancer licenses.

1 Why does the Avi Load Balancer generate alerts for the expired license even though the valid
license is applied?

Each license present on Avi Load Balancer is considered as a separate object. The Avi Load
Balancer will generate alerts for both types of licenses – active license and expired license.
To stop generation of alerts for the expired licenses, delete or uninstall the expired licenses
installed on Avi Load Balancer.

2 What is the procedure to delete the expired licenses?

Follow the steps mentioned below to delete the expired license:

a Navigate to Administration > Licensing.

b Select the expired license from the list, and click Delete to uninstall the expired license.

3 Is it safe to delete the expired license?

Yes, it is safe to delete the expired licenses from the Avi Load Balancer.

Configuration Objects Restricted in the Avi Load Balancer UI for Essentials and
Basic License
This section elaborates the configuration objects restricted in the Avi Load Balancer UI for
Essentials and Basic License.

Avi Load Balancer supports the following license types:

n VMware Avi Load Balancer Enterprise Edition - It is a fully-featured VMware Avi Load
Balancer license, which includes load balancing, GSLB, WAF, and all other features available
on Avi Load Balancer.

n VMware Avi Load Balancer - Basic Edition

n Avi Load Balancer Essentials for Tanzu - VMware Avi Load Balancer essentials for Tanzu
provides Layer 4 load balancing services for Tanzu Basic and Standard users.

VMware by Broadcom 106

VMware Avi Load Balancer Administration Guide

In VMware Avi Load Balancer essentials for Tanzu, a few configuration options are restricted or
not editable in the Avi Load Balancer UI. Some configuration knobs allow the create and edit
options using UI in the essentials edition, whereas others only have the view access.

Avi Load Balancer UI Object List for Essentials, Basic, and Enterprise Edition
See the below table for more information on permissions available for various UI objects in
VMware Avi Load Balancer essentials for Tanzu, VMware Avi Load Balancer - Basic Edition, and
VMware Avi Load Balancer Enterprise Edition.

Permissions(Essentia Permissions(Enterpri
Menu Sub Menu ls)* Permissions(Basic)* se)

Applications Dashboard View View View

Virtual Service View Create/Edit/View Create/Edit/View

VS VIPs View Create/Edit/View Create/Edit/View

Pools View Create/Edit/View Create/Edit/View

Pool Groups View Create/Edit/View Create/Edit/View

Operations Alerts - All Alerts View Create/Edit/View Create/Edit/View

Alerts - Alert Config View Create/Edit/View Create/Edit/View

Alerts - Alert Actions View Create/Edit/View Create/Edit/View

Events - All Events View Create/Edit/View Create/Edit/View

Events - Config Audit View Create/Edit/View Create/Edit/View

Trail

Notifications - Syslog View Create/Edit/View Create/Edit/View

Notifications - Email View Create/Edit/View Create/Edit/View

Notifications - SNMP View Create/Edit/View Create/Edit/View

Trap

Traffic Capture - Create/Edit/View Create/Edit/View Create/Edit/View

Traffic Capture

Templates Profile - Application View Create/Edit/View Create/Edit/View

Profile - TCP/UDP View Create/Edit/View Create/Edit/View

Profile - Persistence View Create/Edit/View Create/Edit/View

Profile - Health View Create/Edit/View Create/Edit/View

Monitor

Profile - Analytics View Create/Edit/View Create/Edit/View

Profile - IPAM/DNS Create/Edit/View Create/Edit/View Create/Edit/View

Profiles

VMware by Broadcom 107

VMware Avi Load Balancer Administration Guide

Permissions(Essentia Permissions(Enterpri
Menu Sub Menu ls)* Permissions(Basic)* se)

Profile - Custom View Create/Edit/View Create/Edit/View

IPAM/DNS

Profile - Traffic Clone View Create/Edit/View Create/Edit/View

Profile - ICAP Profile View Create/Edit/View Create/Edit/View

Templates Policies - NAT Policy View Create/Edit/View Create/Edit/View

Policies - L4 Policy View Create/Edit/View Create/Edit/View

Set

Templates Groups - IP Groups View Create/Edit/View Create/Edit/View

Groups - String View Create/Edit/View Create/Edit/View

Groups

Templates Security - SSL/TLS Create/Edit/View Create/Edit/View Create/Edit/View

Certificates

Security - SSL/TLS View Create/Edit/View Create/Edit/View

Profile

Security - PKI Profile View Create/Edit/View Create/Edit/View

Security - Auth View Create/Edit/View Create/Edit/View

Profile

Security - JWT View Create/Edit/View Create/Edit/View

Server Profile

Security - Ping View Create/Edit/View Create/Edit/View

Access Agent

Security - Certificate View Create/Edit/View Create/Edit/View

Management

Security - IP View Create/Edit/View Create/Edit/View

Reputation

Security - Geo DB View Create/Edit/View Create/Edit/View

Security - HSM Group View Create/Edit/View Create/Edit/View

Security - SSO Policy View Create/Edit/View Create/Edit/View

Security - Bot View Create/Edit/View Create/Edit/View

Management Policy

Templates Scripts - DataScripts View Create/Edit/View Create/Edit/View

Scripts - View Create/Edit/View Create/Edit/View

ControlScripts

Scripts - View Create/Edit/View Create/Edit/View

ProtocolParserScripts

Templates Autoscale - Server View Create/Edit/View Create/Edit/View

Autoscale

VMware by Broadcom 108

VMware Avi Load Balancer Administration Guide

Permissions(Essentia Permissions(Enterpri
Menu Sub Menu ls)* Permissions(Basic)* se)

Autoscale - Launch View Create/Edit/View Create/Edit/View

Config

Autoscale - View Create/Edit/View Create/Edit/View

WebHook

Autoscale - View Create/Edit/View Create/Edit/View

PoolGroup
Deployment

Templates WAF - WAF Policy View Create/Edit/View Create/Edit/View

WAF - WAF Profile View Create/Edit/View Create/Edit/View

WAF - Application View Create/Edit/View Create/Edit/View

Rules

WAF - Application View Create/Edit/View Create/Edit/View

Rules

WAF - CRS View Create/Edit/View Create/Edit/View

Templates Error Page - Error View Create/Edit/View Create/Edit/View

Page Profile

Error Page - Error View Create/Edit/View Create/Edit/View

Page Body

Infrastructure Dashboard View View View

Clouds Create/Edit/View Create/Edit/View Create/Edit/View

Cloud Resources - View Create/Edit/View Create/Edit/View

Service Engine

Cloud Resources - Create/Edit/View Create/Edit/View Create/Edit/View

Service Engine Group

Cloud Resources - Edit/View Create/Edit/View Create/Edit/View

Networks

Cloud Resources - Create/Edit/View Create/Edit/View Create/Edit/View

VRF Context

GSLB - Site View Create/Edit/View Create/Edit/View

Configuration

Administration Accounts - Users View Create/Edit/View Create/Edit/View

Accounts - User View Create/Edit/View Create/Edit/View

Profiles

Accounts - Roles View Create/Edit/View Create/Edit/View

Accounts - Tenants View Create/Edit/View Create/Edit/View

Administration Settings Edit/View Edit/View Edit/View

– Authentication/
Authorization

VMware by Broadcom 109

VMware Avi Load Balancer Administration Guide

Permissions(Essentia Permissions(Enterpri
Menu Sub Menu ls)* Permissions(Basic)* se)

Settings – Access Edit/View Edit/View Edit/View

Settings

Settings – DNS/NTP Edit/View Edit/View Edit/View

Settings – Licensing Edit/View Edit/View Edit/View

Settings – Email/ Edit/View Edit/View Edit/View

SMTP

Settings – Tenancy Edit/View Edit/View Edit/View

Mode

Settings – Upload Edit/View Edit/View Edit/View

HSM Packages

Settings – DNS View Edit/View Edit/View

Service

Settings – Cloud View Create/Edit/View Create/Edit/View

Services

Administration Controller - Nodes Edit/View Edit/View Edit/View

Controller - Events View Create/Edit/View Create/Edit/View

Controller - Alerts View Create/Edit/View Create/Edit/View

Controller - Software View Create/Edit/View Create/Edit/View

Controller - System Edit/View Create/Edit/View Create/Edit/View

Update

Controller - SEG View Create/Edit/View Create/Edit/View

Update

Administration System - Edit/View Edit/View Edit/View

Configuration Backup

System - Tech Create/Edit/View Create/Edit/View Create/Edit/View

Support

System - Crash View Create/Edit/View Create/Edit/View

Reports

Administration User Credential - View Create/Edit/View Create/Edit/View

User Credentials

Administration Support - Cases View Create/Edit/View Create/Edit/View

VMware by Broadcom 110

VMware Avi Load Balancer Administration Guide

Note The above table lists all the UI options, but not all features are supported with the
Essentials and Basic editions.

Some of the examples are as follows:

n WAF and GSLB are not supported with the essentials edition and the Basic Edition, but WAF
and GSLB are available with the Enterprise edition.

n Layer 7 load balancing is not available with the Essentials edition, limited capability for Layer
7 load balancing is available with the Basic edition, and fully-featured Layer 7 support is
available with the Enterprise edition.

n Similarly, only the vCenter cloud and No Orchestrator clouds can be created or edited with
the essentials and the Basic editions. Whereas the Enterprise Edition supports clouds, like
Google, Azure, AWS, and Oracle.

n Only HTTP and Layer 4 profiles can be created or edited in the Basic edition.

n The GSLB Services option is available under the Applications tab only when the GSLB feature
is configured on the Avi Load Balancer Controller. GSLB is available with the Enterprise
edition only.

In the UI, you might be able to edit or view the options, but to check if a feature is supported for
your Avi Load Balancer edition, see Avi Load Balancer Editions.

Avi Load Balancer - Basic Edition

This topic explains how the Avi Load Balancer - Basic edition is deployed.

Note
n The Basic Edition of Avi Load Balancer (formerly known as NSX Advanced Load Balancer)
has reached its End of Availability (EoA) date on Jan 30th, 2024. The changes only apply to
the Basic Edition of Avi Load Balancer. For additional details, please visit VMware Knowledge
Base.

n Avi Load Balancer-Basic edition is not available as a separate SKU and is not saleable.

n Avi Load Balancer-Basic Edition supports the following deployment modes:

n NSX-T Cloud Connector

n No Orchestrator Mode

For more information on setting up Avi Load Balancer - Basic Edition, see Installing Avi Load
Balancer - Basic Edition.

Feature Comparison (Essentials vs Basic vs Enterprise)

For more information on the feature list for each of the editions, See Avi Load Balancer Editions.

Installing Avi Load Balancer - Basic Edition

This topic explains how the Avi Load Balancer - Basic edition is installed.

VMware by Broadcom 111

VMware Avi Load Balancer Administration Guide

Installing the Avi Load Balancer Controller

The Avi Load Balancer Controller OVA can be downloaded from My VMware customer portal. SE
creation will be handled by the Avi Load Balancer Controller.

Deploying Avi Load Balancer Controller

Log in to the vCenter server through a vSphere Web Client. Use the client to deploy Avi Load
Balancer Controller OVA file by following the steps mentioned below:

1 Click the File option in the top menu and select Deploy OVF Template.

2 Follow the Deploy OVA Template wizard instructions:

a Select Thick Provision Lazy Zeroed for disk format.

b Select a port group for Destination Networks in Network Mapping. This port group will be
used by the Avi Load Balancer Controller to communicate with vCenter.

c Specify the management IP address and default gateway. In the case of DHCP, leave this
field empty (Only static IP addresses must be used in production environment).

d Leave the Sysadmin login authentication key field blank.

3 Power on the VM.

Repeat the above steps to deploy three Avi Load Balancer Controller VMs to create an Avi Load
Balancer Controller cluster set-up.

Performing the Avi Load Balancer Controller Initial Setup

You can change or customize settings following initial deployment using the Avi Load Balancer
ControllerUI. Navigate to the Avi Load Balancer Controller IP on the browser.

Note While the system is booting up, a 503 status code or a page with following message will
appear:
n Controller is not yet ready. Please try again after a couple of minutes.
Wait for about 5 to 10 minutes and refresh the page. Follow the instructions below for the
set-up wizard.

To perform the Avi Load Balancer Controller initial setup:

n Configure the basic system settings:

n Administrator account.

Note A valid email address is required for the admin password reset in case of user
account lockout.

n DNS and NTP server information

VMware by Broadcom 112

VMware Avi Load Balancer Administration Guide

n Configure Email or SMTP information

n Select the No Orchestrator option for Orchestrator Integration.

n Select No for Support multiple Tenants? to complete the set-up.

Preparing Avi Load Balancer - Basic Edition

The Avi Load Balancer Controller boots up in the Enterprise Edition by default. Delete the
features that are not supported in the Basic Edition before changing the license tier.

Perform Configuration Audit

Run the configuration audit API to identify the Enterprise Edition features that need to be
removed.

All configuration violations need to be corrected before license tier change is allowed.

The following is a sample output of the configuration audit API performed right after a fresh
install.

[admin:cntrl]: > show configuration audit tier basic

The following are the observations from the above output:

n The default Service Engine Group (Default-Group) has a default HA mode of N+M. The Avi
Load Balancer Basic Edition only supports the legacy Active/Standby HA mode and this must
be changed.

n The Avi Load Balancer Basic Edition does not support configurable cache memory, and this
value must be edited to zero.

VMware by Broadcom 113

VMware Avi Load Balancer Administration Guide

n The Avi Load Balancer Basic Edition does not support health monitoring from the standby SE,
and this must be disabled.

The above command will display any features that must be disabled before proceeding, if the Avi
Load Balancer Controller is used and other features are configured.

Setup Service Engine Group to Legacy Active/ Standby

Before configuring the Avi Load Balancer Controller to be in the Basic Edition tier, the Service
Engine group needs to be re-configured to adhere to the Basic Edition restrictions. Log in to the
Avi Load Balancer UI using the Controller IP address on your browser.

Navigate to Infrastructure > Cloud Resources > Service Engine Group, edit the Default-Group
and perform the following steps:

n Under Placement, select the Active/Standby option from High Availability Mode.

n Deselect the Enable Health Monitoring on Standby SE(s) check box.

n Under Resources, set 0 as the value for Memory for Caching.

n Click Save.

Run the configuration audit API to validate that there are no configuration errors.

[admin:ctrl]: > show configuration audit tier basic

License audit passed successfully
[admin:ctrl]: >

Perform Configuration Audit

The Avi Load Balancer Controller enforces feature restrictions for each edition. Therefore,
configuration needs to adhere to Basic edition for the transition from Enterprise edition to Basic
edition to be allowed.

The following is the sample output for the configuration audit API. The following output
exhibits the configuration and features which are not supported in the Basic Edition. Perform
the required actions as per the output, before proceeding with the license edition change.

[admin:ctrl]: > show configuration audit tier basic

VMware by Broadcom 114

VMware Avi Load Balancer Administration Guide

BASIC license tier. Allowed value: False. |

| | | Field VirtualService.enable_autogw cannot have True as
its value in BASIC license tier. Allowed value: False. |
| | | Field VirtualService.content_rewrite cannot be set in
BASIC license tier. |
| | | Field VirtualService.sideband_profile cannot be set in
BASIC license tier. |
| |
|
|
| ServiceEngineGroup | Default-Group | Field ServiceEngineGroup.ha_mode cannot have
HA_MODE_SHARED as its value in BASIC license tier. Allowed value(s):
HA_MODE_LEGACY_ACTIVE_STANDBY. |
| | | Field ServiceEngineGroup.hm_on_standby cannot have
True as its value in BASIC license tier. Allowed value: False.
|
| | | Field ServiceEngineGroup.app_cache_percent cannot have
10 as its value in BASIC license tier. Allowed value: 0. |
| |
|
|
| Pool | vs-1-pool | Field Pool.connection_ramp_duration cannot have 10 as
its value in BASIC license tier. Allowed value: 0. |
| |
|
|
+--------------------+---------------
+---------------------------------------------------------------------------------------------
-----------------------------------------------------+

The following are the additional points to check while changing the default licensing tier from the
Enterprise Edition to the Basic Edition.

n Save the backup configuration before changing the licensing edition.

n The changing of license editions is supported only for Avi Load Balancer Controllers and
SEs on the same base version. The Controller and SEs on different patch versions are also
supported for the license tier change option.

Prerequisites

A config audit tool is introduced to check the configurations that are not supported for the target
edition. Run the configuration audit API to identify the Enterprise Edition features that need to be
removed and resolve them before changing the edition from Enterprise to Basic.

The configuration audit API performs licensing checks on all the objects in the system and reports
various violations. It ignores the violations in system default objects because they are handled
internally.

VMware by Broadcom 115

VMware Avi Load Balancer Administration Guide

Use the Avi Load Balancer CLI or the REST API to check the unsupported configuration and take
the required actions:

n CLI: show configuration audit tier <tier_name>

n REST API: /api/config-audit/tier/ <tier_name>

Converting from Basic Edition to Enterprise Edition

There is no disruption of traffic or downtime while changing the licensing tier from the Basic
Edition to the Enterprise Edition. Once the Controller is in the Enterprise edition, all the features
available on the Avi Load Balancer can be utilized. Please note the Basic Edition of Avi Load
Balancer (formerly known as NSX Advanced Load Balancer) has reached its End of Availability
(EoA) date on Jan 30th, 2024. The changes only apply to the Basic Edition of Avi Load Balancer.
For additional details, please visit https://kb.vmware.com/s/article/96386.

Procedure

1 Check the available Enterprise Service Units on the Controller. The available Enterprise
Service Units must be sufficient to license the existing SEs. Changing from the Basic Edition to
the Enterprise Edition is only allowed if sufficient licenses are available on the Controller.

2 Import and upload the Avi Load Balancer Enterprise license using UI if the Enterprise license
is not available on the Controller.

3 Set the value of the default_license_tier to the enterprise.

[admin:ctrl]: > configure systemconfiguration

[admin:ctrl]: systemconfiguration > default_license_tier enterprise
overwriting the previous entered value for default_license_tier
[admin:ctrl]: systemconfiguration > save

4 Once the mode change is successful, the SEs are licensed with the Enterprise Service Units,
and the Basic Service Units are freed up.

Results

The new system-default objects are created according to the Enterprise edition support. The
configurations and features available on the Controller will be as per the Enterprise Edition
feature enforcement. All the features on Avi Load Balancer is available when the Avi Load
Balancer is in the Enterprise Edition.

FAQ
This topic answers basic questions about Avi Load Balancer-Basic edition.

n Would deploying x Basic LB SEs and y Enterprise LB SEs on the same Avi Load Balancer
Controller be allowed?

No, the Basic edition and the Enterprise edition licenses cannot exist on the same Avi Load
Balancer Controller.

n Will Avi Load Balancer-Basic edition allow AKO for ingress?

VMware by Broadcom 116

VMware Avi Load Balancer Administration Guide

No. Enterprise edition license is required to use AKO.

n Is Per-App supported in the Basic Edition?

No. Customers can create multiple SE Groups if this functionality is needed.

n Can a Controller be upgraded from Basic edition to Enterprise edition?

Yes, the Basic to Enterprise edition upgrade on the Avi Load Balancer Controller is allowed.
The only requirement is to have sufficient Enterprise edition licenses to license the existing
SEs in the cluster. Upgrade to the Enterprise edition would be seamless and cause zero traffic
disruption.

Avi Load Balancer Essentials for Tanzu

This topic explains how the Avi Load Balancer Essentials for Tanzu is deployed.

Note
n Avi Load Balancer Essentials for Tanzu is not an SKU and is not saleable.

n Avi Load Balancer Essentials for Tanzu does not require any licenses to be used.

n Content Library is not supported with Avi Load Balancer Essentials for Tanzu.

Licensing Requirement
Any VMware NSX Tanzu customer with active entitlements for Basic or Standard SKUs is eligible
to use VMware Avi Load Balancer Essentials for Tanzu. When the Avi Load Balancer Controller
is in the Essentials Edition, any number of SEs can be used without requirements for any license,
but it requires an active support entitlement for VMware Tanzu Basic or Standard.

Converting from Enterprise Edition to Essentials Edition

Follow the steps mentioned in the following sections to change an Avi Load Balancer Controller
from the Enterprise Edition to the essentials Edition.

Configuration Objects Restricted in the Avi Load Balancer Essentials UI

In VMware Avi Load Balancer essentials for Tanzu, a few configuration options are restricted
or not editable in the Avi Load Balancer UI. For more information on permissions available for
various UI objects in VMware Avi Load Balancer essentials for Tanzu, see Configuration Objects
Restricted in the Avi Load Balancer UI for Essentials and Basic License.

Check Licenses and Import NSX Serial Key

Setting up an Avi Load Balancer Controller in the Essentials Edition does not require any licenses.

VMware by Broadcom 117

VMware Avi Load Balancer Administration Guide

Perform Configuration Audit

Avi Load Balancer Controller enforces feature restrictions for each edition. Therefore, the
configuration needs to adhere to Essentials Edition for allowing the transition from Enterprise
Edition to Essentials Edition.

The following is the sample output for the configuration audit command when the target tier
is set as essentials. The following output exhibits the configurations and features which are
not supported in the Essentials Edition. Take the required actions as per the output of the
configuration output command and then change the license tier edition.

[admin:ctrl]: > show configuration audit tier essentials

+--------------------+---------------
+---------------------------------------------------------------------------------------------
-----------------------------------------------------+
| Object Type | Name | License
Violations
|
+--------------------+---------------
+---------------------------------------------------------------------------------------------
-----------------------------------------------------+
| VirtualService | vs-1 | Object refers to system default object marked for
removal: StringGroup System-Rewritable-Content-
Types |
| | | Field
VirtualService.analytics_policy.full_client_logs.enabled cannot have True as its value in
ESSENTIALS license tier. Allowed value: False. |
| | | Field VirtualService.enable_autogw cannot have True
as its value in ESSENTIALS license tier. Allowed value:
False. |
| | | Field VirtualService.content_rewrite cannot be set
in ESSENTIALS license
tier. |
| | | Field VirtualService.sideband_profile cannot be set
in ESSENTIALS license
tier. |
| |
|
|
| ServiceEngineGroup | Default-Group | Field ServiceEngineGroup.ha_mode cannot have
HA_MODE_SHARED as its value in ESSENTIALS license tier. Allowed value(s):
HA_MODE_LEGACY_ACTIVE_STANDBY. |
| | | Field ServiceEngineGroup.hm_on_standby cannot have
True as its value in ESSENTIALS license tier. Allowed value:
False. |
| | | Field ServiceEngineGroup.app_cache_percent cannot
have 10 as its value in ESSENTIALS license tier. Allowed value:
0. |
| |
|
|
| Pool | vs-1-pool | Field Pool.connection_ramp_duration cannot have 10
as its value in ESSENTIALS license tier. Allowed value:
0. |

VMware by Broadcom 118

VMware Avi Load Balancer Administration Guide

| |
|
|
+--------------------+---------------
+---------------------------------------------------------------------------------------------
-----------------------------------------------------+

The following are the additional points to check while changing the licensing tier from the
Enterprise edition to the Essentials edition:

n Save the backup configuration before changing the licensing edition.

n Ensure that the base versions of Avi Load Balancer Controllers and Avi Load Balancer SEs are
same.

Note Controllers and SEs can run on different patch versions.

Prerequisites

A config audit tool is introduced to check the configurations that are not supported for the
target edition. Run the config audit tool to check the violations in the configurations. Resolve
all the violations before proceeding to change from the Enterprise Edition to the Essentials
edition. The configuration audit API performs licensing checks on all the objects in the system
and reports various violations. It ignores the violations in system default objects as they are
handled internally.

Use the following Avi Load Balancer CLI or the REST API to check the unsupported configuration
and take the required actions.

n Avi Load Balancer CLI: show configuration audit tier <tier_name>

n Avi Load Balancer REST API: /api/config-audit/tier/ <tier_name>

Changing Licensing Tier from the Enterprise Edition to the Essentials Edition
Once all the features and configurations specific to the Enterprise edition are removed, the Avi
Load Balancer can be switched to the desired licensing tier.

You can change the licensing tier from the Enterprise edition to the Essential edition as follows.

1 Navigate to Administration > Licensing.

2 Click gear icon next to Licensing.

3 Select Essentials License.

4 Click Save.

After changing the licensing tier to the essentials edition, SEs do not require any licenses.
The Enterprise Service Units are freed up and are no more used when Avi Load Balancer
Controller runs in the essentials edition. The Controller will start enforcing essentials edition
feature restrictions.

VMware by Broadcom 119

VMware Avi Load Balancer Administration Guide

The new System-Default objects are created according to the Avi Load Balancer Essentials for
Tanzu edition support.

The events related to the system objects and the configuration changes are available under
Operations > Events. Various events related to the deletion of network profiles, WAF Profiles,
and so on are listed here.

Converting from Essentials Edition to Enterprise Edition

There is no disruption of traffic or downtime while changing the licensing tier from the essentials
edition to the Enterprise edition. Once the Avi Load Balancer Controller is in the Enterprise
edition, all the features available on the Avi Load Balancer can be utilized.

Follow the steps mentioned below to set up the Controller in the Enterprise edition.

Procedure

1 Check the available Enterprise Service Units on the Controller. The available Enterprise
Service Units must be sufficient to license the existing SEs. Changing from the essentials
edition to the Enterprise edition is only allowed if sufficient licenses are available on the
Controller.

2 Import and upload the Avi Load Balancer Enterprise license using UI if the Enterprise license
is not available on the Controller.

3 Change the system mode to Enterprise mode.

[admin:ctrl]: > configure systemconfiguration

[admin:ctrl]: systemconfiguration > default_license_tier enterprise
overwriting the previous entered value for default_license_tier
[admin:ctrl]: systemconfiguration > save

4 Once the mode change is successful, the SEs are entitled to the Enterprise Service Units.

5 The new system-default objects are created according to the Enterprise edition support. The
configurations and features available on the Controller will be as per the Enterprise Edition
feature enforcement. All the features on Avi Load Balancer are available when the Avi Load
Balancer is in the Enterprise edition.

How to Get Support

A customer that has an active entitlement for VMware Tanzu Basic, or VMware Tanzu Standard
is eligible for NSX ALB essentials for Tanzu support. This can be supported by the Avi Load
Balancer and VMware Technical Support teams.
FAQs
n Is Avi Load Balancer Essentials an SKU?

No, Avi Load Balancer Essentials do not require any licenses. Support entitlement is through
VMware Tanzu SKUs (Basic or Standard).

n Can a Controller be upgraded from the Essentials Edition to the Enterprise Edition?

VMware by Broadcom 120

VMware Avi Load Balancer Administration Guide

Yes, the Essentials to Enterprise Edition upgrade is allowed on the Controller. The only
requirement is to have sufficient Enterprise Edition licenses to license the existing SEs in
the Controller Cluster. Upgrade to Enterprise Edition can be seamless and cause zero traffic
disruption.

Terms of Avi Load Balancer Software License

Avi Load Balancer licenses are based on annual or multi-year subscription. The licenses are valid
for a year and can be renewed on a yearly basis.

Note For complete information on procuring, installing, and managing a license, see Chapter 3
Licensing.

Avi Load Balancer License Types

License Type Infrastructure Description

Data Plane (Avi Load Balancer Service Engine)

vCPU VMs, Baremetal, Container, Defines the maximum number of vCPUs (cores) across
Cisco CSP, Private, and all SEs. This is the default license mode. The SE vCPU
public cloud allocation is configured in the SE group.
n For example, a 10-vCPU license allows up to ten 1-
vCPU SEs, or five 2-vCPU SEs, or a mix of four 1-vCPU
SEs and three 2-vCPU SEs (across two SE groups),
and so on.
Per-VS license mode: If an SE group is configured in per-
app SE mode, a vCPU counts at a 25% rate for licensing
usage. For example, a 2-vCPU SE in the per-app SE group
utilizes a 0.5 vCPU license (2 * 0.25). Per-app SE mode
is limited to a maximum of two virtual services per SE.
This mode is suitable for deployments with dedicated per-
application load balancers. Per-VS license mode is not
supported for DNS virtual services.

Socket Baremetal, Cisco CSP Defines the total number of sockets that an SE can be
deployed on. This license is applicable for processors with
up to 14 cores.

High-core socket Baremetal, Cisco CSP Defines the total number of sockets that an SE can be
deployed on. This license is applicable for processors with
16 cores or more.

Bandwidth VMs, Baremetal, Container, Defines bandwidth license applicable to a single SE.
Cisco CSP, Private, and Available in the options of 25Mbps, 200Mbps, and 1Gbps
public cloud (for public cloud only). Multiple Bandwidth licenses cannot
be aggregated on a single SE.

Container-Core Container East-West Defines the maximum number of vCPUs (cores) across all
SEs deployed for East-West traffic in a Container Cluster.
This license is used for East-West traffic in a service mesh.

VMware by Broadcom 121

VMware Avi Load Balancer Administration Guide

License Type Infrastructure Description

Data Plane (Avi Load Balancer Service Engine)

Control Plane (Avi Load Balancer Controller)

SaaS unit SaaS Defines the number of Avi Load Balancer SaaS units
needed in the SaaS instance to support the required
number of SEs and application instances.

Note A free trial license with all features supported by an enterprise license is available for
evaluation. Contact the Avi Load Balancer representative for more details.

License Overage Allowance

The Avi Load Balancer allows an overage allowance for the vCPU and bandwidth licenses.

The main features of the overage allowance are as follows:

n Avi Load Balancer provides an allowance of 10% of the configured license capacity. This
allowance can be utilized once the regular capacity of the license is exhausted.

n If the 10% allowance results in a fractional number, it is rounded up to the next higher integer.

n In the case of bandwidth licenses, the allowance is for each bandwidth license SKU separately
(25M, 200M, 1G BW).

n For an Avi Load Balancer Controller having 15 vCPU licenses, the overage allowance is 1.5
vCPU (10% of 15vCPU). This value will be rounded up to two vCPUs. The effective license
value when the overage allowance is active is 17 vCPUs.

n For a Controller having 15 25M bandwidth licenses and ten 200M bandwidth licenses, the
allowance is 1.5 additional 25M bandwidth licenses (rounded up to two bandwidth licenses)
and one additional 200M bandwidth license.

Removing Trial Licenses from an Avi Load Balancer

Controller
Licenses on an Avi Load Balancer Controller can be checked by navigating to Administration >
Licensing on the UI. Trial licenses cannot be removed using UI or CLI. Avi Load Balancer REST API
is used to remove trial license.

Follow the instructions mentioned below to remove trial licenses from an Avi Load Balancer
Controller.

Instructions
1 Enable HTTP Basic authentication:

a Log into Avi Load Balancer UI, navigate to Administration > System Settings , and click
EDIT.

VMware by Broadcom 122

VMware Avi Load Balancer Administration Guide

b Under Access, select Allow Basic Authentication to activate HTTP basic authentication.

2 Use cURL command to delete the trial license

a Run the following cURL command from a host which can reach the Avi Load Balancer
Controller:

curl -k --user admin -X DELETE --header 'Accept: application/json' --header

'X-Avi-Version: <Controller-Version>' 'https://<controller-address>/api/license/Trial'

b Example for the cURL command is shown below:

curl -k --user admin -X DELETE --header 'Accept: application/json' --header

'X-Avi-Version: 21.1.1' 'https://1.2.3.4/api/license/Trial'

c Above command will prompt for the admin password. Enter the password for the admin
account.

3 Verify the changes:

a To verify that the trial license is removed from the system successfully, navigate to
Administration > Licensing in the Avi Load Balancer Controller UI and ensure that it is
not listed.

VMware by Broadcom 123

VMware Avi Load Balancer Administration Guide

Updating License on each Node in an Avi Load Balancer

Controller Cluster
It is not required to update the license on each node in an Avi Load Balancer Controller Cluster.
The leader node in a Controller Cluster pushes its license to the remaining (follower) nodes. The
leader node license in the Avi Load Balancer Controller Cluster can be updated using the Avi
Load Balancer UI.

Navigate to Administration > Licensing to update the existing license on a leader node in a
Controller Cluster.

For more information about License management on Avi Load Balancer, see Avi Load Balancer-
Enterprise Edition.

VMware by Broadcom 124

Controller Settings
4
The System Settings page of the Avi Load Balancer Controller UI provides the ability to configure
administrative settings for the Avi Load Balancer.

The following settings can be configured:

n Authentication/Authorization: Choose local or remote authentication and authorization of Avi

Load Balancer users. In the remote case, enables specification of an auth profile. For more
information, see Authentication Profile.

n Access Settings: Select options for securing management access to the Avi Load Balancer
system.

n DNS / NTP: Specify the Domain Name System (DNS) and Network Time Protocol (NTP)
servers used by Avi Load Balancer.

n Licensing: Display license information and upload license files.

n Email/SMTP: Specify the source for Simple Mail Transfer Protocol (SMTP) traffic from Avi
Load Balancer.

n Tenant Settings: Specify system-wide settings for multitenancy.

n SSH Key Settings: Create or import SSH keys.

n Upload HSM Packages: Upload a customer’s Hardware Security Module (HSM) software
package, to enable HSM features within Avi Load Balancer.

For more information, see Application Profile topic in the VMware Avi Load
BalancerConfiguration Guide.

Read the following topics next:

n DNS/ NTP Settings

n Email-SMTP Settings

DNS/ NTP Settings

NTP (Network Time Protocol) settings are critical for the functioning of the Avi Load Balancer.
The analytics functionality in the Controller relies on synchronization between the Controller in

VMware by Broadcom 125

VMware Avi Load Balancer Administration Guide

the cluster and SE. Controller synchronize time with the configured NTP servers and the SE in
turn synchronize time with the Controller.

Avi Load Balancer requires access to valid DNS and NTP servers for operation. Using an NTP
server is especially critical, without which Avi Load Balancer cannot function properly.

Port 123 must be open on the Avi Load Balancer Controller to receive timestamps over UDP.

Configure DNS/ NTP Settings using the UI

Configure NTP servers from the Avi Load Balancer UI as follows:

1 Navigate to Administration > System Settings.

2 Click the EDIT button to view the EDIT SYSTEM SETTINGS screen.

3 Under DNS/NTP tab, enter DNS Resolver(s). This is a comma-delimited list of DNS server IP
addresses. If a DNS server is not configured, Avi Load Balancer will not be able to accept
names for load-balanced servers, virtual services, mail servers, and similar inputs. You can
specify IPv4, IPv6 addresses and hostnames.

Note You can change from IPv4 to IPv6 endpoints on the Controller with v4 as primary and
v6 as secondary interface. The system needs to verify if IPv4 server address is allowed on
IPv4 Controller and IPv6 server address is allowed on IPv6 Controller.

4 Enter the Search Domain to use in DNS lookup.

5 Configure NTP Authentication Keys.

a Click Add.

b Enter the Key Number from the list of trusted keys used to authenticate this server.

c Select the Message Digest Algorithm used for NTP authentication. Message Digest (MD5)
and Secure Hash Algorithm (SHA1) are selected.

d Enter the NTP Authentication Key.

VMware by Broadcom 126

VMware Avi Load Balancer Administration Guide

6 Under NTP Servers, select the Key Number for NTP authentication and the IP address of the
NTP Servers.

7 Click SAVE.

Configure DNS/ NTP Settings using the CLI

Configure NTP servers from the CLI as follows.

: > configure systemconfiguration

VMware by Broadcom 127

VMware Avi Load Balancer Administration Guide

The DNS Search Domain is the local domain name, which will be appended to a name that is not
fully qualified. For instance, if the DNS search domain is set to avinetworks.com, and the name to
be resolved is www, Avi Load Balancer will look up www.avinetworks.com.

Configure DNS Settings using the CLI

The .local domains are not resolvable by default through the configured DNS server (local
domains are not routed to DNS servers). The search domains need to be configured explicitly
for .local domains to make lookups work within this DNS domain. Configure the DNS settings
from the CLI as shown below.

[admin:avictrl]: > configure systemconfiguration

[admin:avictrl]: systemconfiguration> dns_configuration
[admin:avictrl]: systemconfiguration:dns_configuration> search_domain "test.domain1.local
test.domain2.com"
Overwriting the previously entered value for search_domain
[admin:avictrl]: systemconfiguration:dns_configuration> save
[admin:avictrl]: systemconfiguration> save
+----------------------------------+------------------------------------+
| Field | Value |
+----------------------------------+------------------------------------+
| uuid | default |
| dns_configuration | |
| server_list[1] | 10.79.16.132 |
| search_domain | test.domain1.local test.domain2.com|
| ntp_configuration | |
| ntp_servers[1] | |

VMware by Broadcom 128

VMware Avi Load Balancer Administration Guide

Configure DNS Settings using the API

Configure NTP servers with the API as follows.

{
},
"ntp_configuration": {

VMware by Broadcom 129

VMware Avi Load Balancer Administration Guide

"ntp_server_list": [
{
"type": "V4",
"addr": "23.239.26.89"
},
{
"type": "V4",
"addr": "69.89.207.99"
}
]
}
}

Configuring NTP Authentication using the CLI

NTP authentication can be enabled using either the CLI or the REST API. With NTP authentication,
one can specify a set of trusted authentication keys and configure each NTP server peer with
a specific authentication key. The NTP authentication key object consists of a key number, key
algorithm (SHA1 or MD5) and the key itself.

Configure NTP and NTP authentication with the CLI as follows.

[admin:10-10-25-45]: > configure systemconfiguration

[admin:10-10-25-45]: systemconfiguration> ntp_configuration
[admin:10-10-25-45]: systemconfiguration:ntp_configuration> ntp_authentication_keys
key_number 1 algorithm ntp_auth_algorithm_md5 key "=I&FBDl,WM,en5Mn~DaG"
New object being created
[admin:10-10-25-45]: systemconfiguration:ntp_configuration:ntp_authentication_keys> exit
[admin:10-10-25-45]: systemconfiguration:ntp_configuration> ntp_authentication_keys
key_number 5 algorithm ntp_auth_algorithm_sha1 key ff9a0d589668a0f66649abbd7dfb388d841f1f44
New object being created
[admin:10-10-25-45]: systemconfiguration:ntp_configuration:ntp_authentication_keys> exit
[admin:10-10-25-45]: systemconfiguration:ntp_configuration> exit
[admin:10-10-25-45]: systemconfiguration:ntp_configuration> ntp_servers server 23.239.26.89
New object being created
[admin:10-10-25-45]: systemconfiguration:ntp_configuration:ntp_servers> exit
[admin:10-10-25-45]: systemconfiguration:ntp_configuration> ntp_servers server 69.89.207.99
key_number 5
New object being created
[admin:10-10-25-45]: systemconfiguration:ntp_configuration:ntp_servers> exit
[admin:10-10-25-45]: systemconfiguration:ntp_configuration> exit
[admin:10-10-25-45]: systemconfiguration> exit
+-------------------------------------+------------------------------------------+
| Field | Value |
+-------------------------------------+------------------------------------------+
| uuid | default |
| dns_configuration | |
| search_domain | |
| ntp_configuration | |
| ntp_authentication_keys[1] | |
| key_number | 1 |
| algorithm | NTP_AUTH_ALGORITHM_MD5 |
| key | =I&FBDl,WM,en5Mn~DaG |
| ntp_authentication_keys[2] | |

VMware by Broadcom 130

VMware Avi Load Balancer Administration Guide

Configuring NTP Authentication using the API

Configure NTP and NTP authentication with the API as follows.

{
},
"ntp_configuration": {
"ntp_servers": [
{
"server": {
"type": "V4",
"addr": "23.239.26.89"
}
},
{
"key_number": 5,
"server": {
"type": "V4",
"addr": "69.89.207.99"
}

VMware by Broadcom 131

VMware Avi Load Balancer Administration Guide

}
],
"ntp_authentication_keys": [
{
"key_number": 1,
"algorithm": "NTP_AUTH_ALGORITHM_MD5",
"key": "=I&FBDl,WM,en5Mn~DaG"
},
{
"key_number": 5,
"algorithm": "NTP_AUTH_ALGORITHM_SHA1",
"key": "ff9a0d589668a0f66649abbd7dfb388d841f1f44"
}
]
}
}

Time synchronization for the Service Engine

Avi Load Balancer supports configuring NTP servers on SEs. Prior to this, NTP synchronization for
SEs relied on the Controller as the NTP server and performed time synchronization on UDP port
123 over the management interface. The SE presumes network connectivity with the Controller.

NTP Configuration for SE deployed as a Virtual Machine

The SE NTP servers can be configured for the virtual machine and LSC-based deployments. The
SE will synchronize time with the configured servers at start-up and periodically monitor the time
sync status.

NTP Configuration for Virtual Machine Deployments

When SE is deployed as a virtual machine, NTP servers can be configured using any of the
methods below and applies configuration in the following order of priority:

DHCP

If DHCP through dhclient provides NTP servers over the management interface, the SE uses
DHCP provided NTP servers as configuration for SE NTP synchronization.

Cloud Configuration
If DHCP does not provide NTP servers, NTP servers are acquired from the cloud
configuration. NTP server's configuration through cloud configuration is a bootup property,
and the SE must be restarted to apply this configuration.

Controller

NTP servers configuration is through system configuration

NTP Server configuration through Cloud configuration using CLI is as follows:

[admin:ctrl]: > configure cloud Default-Cloud

Updating an existing object. Currently, the object is:

VMware by Broadcom 132

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 133

VMware Avi Load Balancer Administration Guide

NTP Configuration for LSC (Baremetal) Deployments

When SE is deployed as a container on the Bare Metal, the administrator is required to configure
the NTP servers on the host.

LSC NTP synchronization only supports the following NTP daemons:

ntpd

The Network Time Protocol daemon (ntpd) is an operating system program that maintains
the system time in synchronization with time servers using the NTP.

chronyd

chronyd is another implementation of the NTP and is used:

n To synchronize the system clock with NTP servers.

n To synchronize the system clock with a reference clock, for instance, a GPS receiver.

n To synchronize the system clock with a manual time input.

SE NTP operation
In both modes of deployments (SE deployed as a VM or as a container on LSC), SE periodically
verifies if the NTP daemons can acquire and time sync with configured servers, and if SE is
unable to sync time with the configured servers, an event is raised. The event is periodically
repeated in 15 minutes unless the NTP time is synchronised.

Email-SMTP Settings
The Avi Load Balancer proactively sends emails for password reset operations, or as a result of
a triggered alert action that calls for email notifications to be sent to administrators or monitoring
systems. Emails are sent from the Controller, which requires the Controller to have DNS and
network access to a destination email server. When the Controller sends an email, the email is
sourced from the SMTP source.

Note The Avi Load Balancer supports non-TLS SMTP servers.

Modify Email or SMTP Settings

The steps to view or modify the email or SMTP settings are as follows.

1 Navigate to Administration > System Settings.

2 To edit the settings, click EDIT.

3 Under Email/SMTP, select Anonymous SMTP, Local, SMTP, or None as required for SMTP
Source.

4 Configure the SMTP source settings.

5 Click SAVE.

VMware by Broadcom 134

VMware Avi Load Balancer Administration Guide

Configure Email or SMTP Source Settings using the UI

1 Select None from the SMTP Source option to not send emails.

2 Select Local from the SMTP Source option to send the email from a local host. Some
enterprise email servers might not accept this method.

a Enter From Address.

3 Select SMTP from the SMTP Source option to send the email from a remote SMTP
server. This method is generally more trusted by the staff of security-conscious enterprise
environments. Under SMTP, enter the following details:

a Enter Username to authenticate to the mail server.

b Enter Password to authenticate to the mail server.

c Enter SMTP Server to send the email from a server host.

d Enter SMTP Port number. The default service port for SMTP is 25.

e Enter From Address.

f Select Do not use TLS to disable the TLS connection to the mail server. By default, the
option is not selected.

VMware by Broadcom 135

VMware Avi Load Balancer Administration Guide

4 Select Anonymous SMTP from the SMTP Source option to log in to the SMTP server without
the need for a username and password. The SMTP server must allow anonymous access.
Under Anonymous SMTP Server, enter the following details:

a Enter From Address.

b Enter SMTP Server to send the email from a server host.

c Enter SMTP Port number. The default service port for SMTP is 25.

d Select Do not use TLS to disable the TLS connection to the mail server. By default, the
option is deselected.

Configure Email or SMTP Source Settings using the CLI

Email/ SMTP settings are configured under System configuration as shown below.

[admin:ctrl]: systemconfiguration> email_configuration

[admin:ctrl]: systemconfiguration> email_configuration from_name "Admin Avinetworks"
[admin:ctrl]: systemconfiguration:email_configuration> save
[admin:ctrl]: systemconfiguration> save

Time zone for e-mail Alert Notifications

By default, the Controller uses UTC time when generating timestamps in the alert notification
e-mails.

VMware by Broadcom 136

VMware Avi Load Balancer Administration Guide

To configure the timestamps to reflect a specific timezone, use the following configuration
through the CLI:

configure systemconfiguration
email_settings
email_timezone < timezone >
save
save

Using non-TLS SMTP Servers

To use non-TLS SMTP servers, the TLS option must be disabled implicitly.Log in to the CLI
and run the following commands using the configure systemconfiguration mode.configure
systemconfiguration email_configuration disable_tls save save

VMware by Broadcom 137

Avi Load Balancer Controller
Cluster 5
To ensure complete system redundancy, the Avi Load Balancer Controller must be highly
available. To provide high availability (HA) for the Controller, add two additional Controller nodes
to create a three-node Controller cluster.

HA of the Controller requires three separate Controller instances configured as a 3-node cluster.
Start with a single-node Controller deployment and use the following steps to add two additional
Controller nodes to form a 3-node cluster.

Note If the cluster is already deployed and you want to modify its node membership or
dismantle the cluster, see Changing Avi Load Balancer Controller Cluster Configuration.

Prerequisites for Cluster Deployment

A Controller cluster can have three nodes, one leader node, and two follower nodes.

Leader node

n The leader can be any single node with configuration or without configuration.

n The leader can have SEs connected.

n The node must have a static IP address.

n Using DHCP can cause issues when nodes reboot and their IP addresses change.

n The current release does not support the use of hostnames for cluster configuration.

Follower nodes

n An Avi Load Balancer Controller cluster can have 3 nodes, 1 leader node and 2 follower
nodes.

n Follower nodes must have only factory-default configuration.

n In AWS environments, follower nodes must have an initial password configured. For more
information, see Changes for Cluster Set-up for AWS Deployments topic in the VMware
Avi Load BalancerInstallation Guide.

n In all other environments, follower nodes must be using the default initial admin
credentials; do not run the initial setup wizard to set an admin password.

VMware by Broadcom 138

VMware Avi Load Balancer Administration Guide

n Follower nodes are expected to be running the same Avi Load Balancer base+patch
version as the leader.

n Follower typically is a VM or container created from the the Avi Load Balancer Controller
installation package.

n Each follower node must have a static IP address.

Cluster Compatibility and Upgrade Requirements

The following requirements are enforced for upgrade (brownfield) and greenfield deployments:

n All Controller nodes in a cluster must be homogeneous in terms of memory, CPU, and disk
configurations.

n All Controller nodes in a cluster need to be in the same software base and patch versions.

n All Controller nodes in a cluster must meet a minimum requirement of 6 cores and 32 GB
of memory. Due to this requirement, Controllers which are in the Essentials sizing cannot be
launched.

n Controllers will fail to upgrade if they are in the Essentials flavor (less than 6 cores or 32 GB of
memory).

n If FIPS mode is enforced, all Controller nodes in a cluster must also be FIPS-enabled.

A sample error message displayed when upgrade fails due to insufficient resources controller
trying to upgrade:

[admin:12345]: >show upgrade status detail filter controller|

| message |UC::[Thu Apr 18 07:27:32 2024] |
| |Error while running task: |
| |WaitForAllNodesAfterReboot |
| |{"Summary": ["Insufficient resources. |
| |Essentials Controller is deprecated. |
| |Minimum memory and CPU supported are 32GB |
| |and 6 cores, respectively."], |
| |"Detail": {"1 92.163.0.122": |
| |"[ CPU: 8, Memory: 23.47, Disk: 125.19, |
| |Major Version: {30.2.1 9000}, |
| |Patch Version: , Fips mode: false ]"}} |
| |Traceback (most recent call last): |
| | File "/opt/avi/python/lib/avi/upgrade/ |
| |upgrade_tasks.py", line 264, in start |
| | self.run() |
| | File "/opt/avi/python/lib/avi/upgrade/ |
| |upgrade_tasks.py", line 639, in run |
| | raise RuntimeError(result) |
| |RuntimeError: {"Summary": |
| |Essentials Controller is deprecated. |
| |Minimum memory and CPU supported are 32GB |
| |and 6 cores, respectively."], |
| |"Detail": {"192.163.0.122": |
| |"[ CPU: 8, Memory: 23.47,Disk: 125.19, |

VMware by Broadcom 139

VMware Avi Load Balancer Administration Guide

| |Major Version: {30.2.1 9000}, |

| |Patch Version: , Fips mode: false ]"}} |
+---------------+-------------------------------------------+

Node Compatibility Checks

Node compatibility checks are enforced when the cluster is formed as well as when a node tries
to rejoin a cluster after reboot.

n Performing on-the-fly changes to memory or CPU without rebooting controller nodes and not
adhering to homogeneity, may result in the loss of cluster quorum and subsequent failures.

n Changing or increasing resources one by one on Controllers of a cluster will fail as the cluster
cannot be formed with nodes having different resources. The recommended workflow is to
schedule a maintenance window before altering resources on the Controllers, shutting down
all the Controller nodes, resizing them, bringing them back up, and allowing them to rejoin the
cluster.

If the specified requirements are not met, the Controller will fail to form a cluster and the reason
for failure will be displayed as shown below:

[admin:1234]: > show cluster detail

| local_node_states[1] | |
| node_ip | 100.65.9.133 |
| state | CLUSTIFY_CHECK_FAILURE |
| reason | {'Summary': ['Cluster nodes need to be |
| |homogeneous regarding memory, CPU, disk, |
| |version, patch version, and FIPS mode.'] |
| |'Detail': {'100.65.9.132': '[ CPU: 5, |
| |Memory: 31.35, Disk: 125.19, |
| |Major Version:{30.2.1 9044},Patch Version:|
| |, Fips mode: false ]', '100.65.9.133': |
| |'[ CPU: 8, Memory: 31.35, Disk: 125.19, |
| |Major Version: {30.2.1 9044}, |
| | Fips mode: false ]'}} |
| local_node_states[2] | |
| node_ip | 100.65.9.131 |
| state | CLUSTIFY_CHECK_FAILURE |
| reason | {'Summary': ['Cluster nodes need to be |
| |homogeneous regarding memory, CPU, disk, |
| |version, patch version, and FIPS mode.'] |
| |'Detail': {'100.65.9.132': '[ CPU: 5, |
| |Memory: 31.35, Disk: 125.19, MajorVersion:|
| |{30.2.1 9044}, Patch Version: , Fips |
| |mode: false ]', '100.65.9.133': ' |
| |'[ CPU: 8, Memory: 31.35, Disk: 125.19, |
| |Major Version: {30.2.1 9044}, |
| | Fips mode: false ]'}} |
| local_node_states[3] | |
| node_ip |100.65.9.132 |
| state |CLUSTIFY_CHECK_FAILURE |
| reason | {'Summary': ['Insufficient resources. |
| | Essentials Controller is deprecated. |

VMware by Broadcom 140

VMware Avi Load Balancer Administration Guide

| | Minimum memory and CPU supported are |

| | 32GB and 6 cores, respectively.'], |
| | 'Detail': {'1 00.65.9.132': '[ CPU: 5, |
| | Memory: 31.35, Disk: 125.19, |
| | Major Version: {30.2.1 9044}, |
| | Patch Version: , Fips mode: false ]'}} |
+----------------------------+------------------------------------------+

Caveats
The current release does not support the use of hostnames for cluster configuration.

Read the following topics next:

n Deploying an Avi Load Balancer Controller Cluster

n Deploying an Avi Load Balancer Controller Cluster (Additional)

n Deploy SEs in Different Data Centers from Controllers

n Clustering Patched Controllers

n Controller Cluster IP

n Cluster Configuration with DNS Hostnames

n Changing Avi Load Balancer Controller Cluster Configuration

n Updating the Configuration Following Avi Load Balancer Controller IP Address Change

n Customizing Avi Load Balancer Configuration

n Clustering Avi Load Balancer Controllers of Different Networks

n High Availability for Avi Load Balancer Controllers

n Control Plane High Availability

n Switching Among Controllers Using the Controller Site Selector

n Controller Cluster Deployment Across Two Availability Zones

n Impact of a Controller Failure

n Recover a Non-Operational Controller Cluster

n API - Configuring the Avi Load Balancer Controller Cluster

n Avi Load Balancer ControllerCluster Configuration in AWS

n Configure Azure Cluster IP

n Changing Controller IP Addresses in a Bare-metal Environment

n OpenStack Network Configuration for Avi Load Balancer Controller Cluster

n FAQs on Controller Cluster

n FAQs on Database Replication

VMware by Broadcom 141

VMware Avi Load Balancer Administration Guide

Deploying an Avi Load Balancer Controller Cluster

To deploy an Avi Load Balancer Controller cluster, first deploy a single Avi Load Balancer
Controller node (the leader), and then add the follower nodes to the leader.

Ensure that after using the setup wizard to install Avi Load Balancer on the follower nodes, you
do not make any other configuration changes on these nodes.

For more information about initial cluster deployment, see deploying cluster topic in the VMware
Avi Load BalancerInstallation Guide.

Using the UI
The steps to deploy a cluster on the web interface of the Avi Load Balancer Controller that will
serve as the leader are as follows:

Procedure

1 Navigate to Administration > Controller > Nodes and click EDIT.

2 Specify the Controller Cluster name and IP address.

For more details on Controller cluster IP address, see Controller Cluster IP. This will be the
shared management IP address of the cluster.

3 Specify the host IP addresses of each of the 3 nodes in the Node IP field of each Cluster
Node.

n Controller Node-1: Host IP address of the leader node.

n Controller Node-2: Host IP address of one of the follower nodes.

n Controller Node-3: Host IP address of one of the other follower nodes.

4 Click SAVE.

Using the CLI

Note Ensure that you specify the host IP addresses of the Avi Load Balancer Controller nodes
instead of the IP addresses shown below:

Configure the cluster with the Controller nodes at [‘10.10.25.81’, ‘10.10.25.82’, ‘10.10.25.83’].

VMware by Broadcom 142

VMware Avi Load Balancer Administration Guide

To add or remove nodes from the cluster, bring this Controller down and spin it up with the new
configuration.

Waiting for the cluster to be ready...

Controller is ready.

Deploying an Avi Load Balancer Controller Cluster

(Additional)
Begin with a single Avi Load Balancer Controller that is already installed and will become the
leader node.

Procedure

1 Install two new Avi Load Balancer Controller nodes.

It is recommended that each Avi Load Balancer Controller should be on the same
management network as the Controller that was already installed. If they are not on the same
management network, See Clustering Avi Load Balancer Controllers of Different Networks.
Do not perform any configuration on these newly installed Avi Load Balancer Controller
nodes.

2 Select an IP address to use as the Controller Cluster IP.

For OpenStack deployments, choose one of the following options:

a Pick an unused IP address from the same network to which the Controllers are connected.
During clustering, the Controller will automatically create a Neutron port with the
Controller cluster IP as a fixed IP address. It ensures that Neutron does not assign that
IP address for another purpose.

b Explicitly create a Neutron port and use the IP address assigned to that port as the
cluster IP address.

3 Use a web browser to navigate to the management IP address of the Avi Load Balancer
Controller that was already installed.

VMware by Broadcom 143

VMware Avi Load Balancer Administration Guide

4 Navigate to Administration > Controller > Nodes and click Edit. The EDIT CLUSTER pop-up
window appears.

5 In the Controller Cluster IP field, enter the shared IP address for the Controller cluster.

6 Configure the Deploying an Avi Load Balancer Controller Cluster.

Note
n When deploying a Controller Cluster in AWS, enter the Password that was created
for the new Controller. For more information, see Changes for Cluster Set-up for AWS
Deployments topic in the VMware Avi Load BalancerInstallation Guide. For deployments
in all other environments, the Password can be left blank to use the default initial admin
credentials.

n Public IP is specified for all public clouds.

7 After these steps, the incumbent Controller becomes the leader of the cluster and invites
the other Controllers to the cluster as follower members. Avi Load Balancer then performs
a warm reboot of the cluster. This process can take 2-3 minutes. The configuration of the
leader Avi Load Balancer Controller node is synchronized to the new follower nodes when
the cluster comes online following the reboot.

a The leader Avi Load Balancer Controller assigns the Controller cluster IP address as a
secondary IP address to its management interface.

b For OpenStack deployments, the leader also performs a Neutron API call to set Controller
cluster IP address in the list of allowed-address-pairs for the Neutron port corresponding
to its management interface.

c To use FQDNs instead of IP address of the Controller nodes, see the following section.

What to do next

Primary (leader)

Cluster IP:
10.30.163.63
10.30.163.68 0110

Invite to join Invite to join

cluster cluster

0110 0110

10.30.163.64 10.30.163.65

The leader Controller assigns the Controller cluster IP address as a secondary IP address to its
management interface.

VMware by Broadcom 144

VMware Avi Load Balancer Administration Guide

For OpenStack deployments, the leader also performs a Neutron API call to set the Controller
cluster IP address in the list of allowed address pairs for the Neutron port corresponding to its
management interface.

To use FQDNs instead of IP addresses of controller nodes, see Cluster Configuration with DNS
Hostnames.

Deploy SEs in Different Data Centers from Controllers

There is no strict requirement that will prevent Controllers from managing remote SEs. However,
it is essential to understand the best practices and ensure the deployment will scale with
resilience.

It is recommended to deploy the Controller cluster and its Service Engines in the same data
center. Tasks such as SE deployment (copying the SE image), log and metric collection, config
propagation, and state backup (such as persistence tables) require connectivity. Increased
latency can impact these operations and cause potential failures for data plane traffic.

If the latency is low enough, and the intent is to have the Controllers manage SEs in two or
more data centers, redundant Controllers are recommended to be congregated in one data
center. If the connectivity to the remote data center is lost, those SEs will not be able to receive
configuration updates, but will continue to operate until connectivity is restored.

Note The RTT (round-trip time) value between an Avi Load Balancer Controller and SE must be
less than 70 ms.

Clustering Patched Controllers

Consider the special circumstances of a patched Controller.

For more information on patch process, see Patch Guide.

n The patch process requires logging into the Avi Load Balancer CLI as the admin user.

n Being able to log in as an admin user implies the factory-default admin password has been
changed, as explained in the Strong Default Admin Password section.

n As mentioned, follower nodes are expected to have no configuration, and no changes to the
Avi Load Balancer login credentials. If follower nodes do have credentials, a message of the
following form is displayed if one attempts to cluster them:

Cluster configuration failed. Unable to add node 1.2.3.4: Login with default credentials
failed, clean reboot the node and retry.

Therefore, before joining two patched Controller-followers to a patched Controller-leader, run

reboot clean commands on each follower.

n A Controller which has had its admin password changed has a configuration.

VMware by Broadcom 145

VMware Avi Load Balancer Administration Guide

Transition Process
n Trust is established between the leader and new followers over HTTPS.

n The leader exports cluster state information to each follower over SSH.

n Each node locally applies the cluster state information, and restarts services.

n Existing Avi Load Balancer SEs, originally connected to only the leader node, detect the
cluster membership changes and connect to all the Avi Load Balancer Controller nodes in the
cluster.

Note During this transition, the REST API is not available. Generally, the transition takes four to
ten minutes.

Transition Status
The progress of transition to a 3-node cluster is indicated by the following status messages:

n Configuration is in progress: During this phase, Avi Load Balancer verifies the
configuration and verifies the state information on each of the new nodes. Only nodes with
no state can be added to the cluster. Avi Load Balancer also updates the cluster database,
and launches a configuration script on each of the nodes.

n Inaccessible: Avi Load Balancer Controller processes are restarted.

n Restoring: Avi Load Balancer configuration is synchronized across all nodes.

n Waiting for Service Engines to connect: Avi Load Balancer waits for any known SEs
that have not yet connected to and registered with any of the Avi Load Balancer Controller
nodes.

n Ready

If dismantling a cluster (returning to only a single Avi Load Balancer Controller node), the status
messages are the same. However, during the Configuration is in progress phase, the
state information on the followers is ignored.

Controller Cluster IP
The Avi Load Balancer Controller cluster IP address is a single IP address shared by multiple
Controllers within a cluster. It is the address to which the web interface, CLI commands, and REST
API calls are directed. As a best practice, to access the Controller, you must log in to the cluster
IP address instead of the IP addresses of individual Controller nodes.

For cluster IPs, the management IPs of the Controllers must all be in the same subnet.

Note You can use Route 53 with health checks to resolve the cluster's domain name
to a Controller IP address directly in AWS deployments where Controllers are on different
subnets. For complete information on cluster configuration in AWS, see Avi Load Balancer
ControllerCluster Configuration in AWS.

VMware by Broadcom 146

VMware Avi Load Balancer Administration Guide

Cluster IP Advertisement
The Avi Load Balancer Controller cluster IP is ARPed by the primary Controller (or leader,
depending on the infrastructure type) within the cluster. When another Controller becomes the
primary, it will send out a gratuitous ARP to claim ownership of the cluster IP.

Administrators can manage any of the Controllers within the cluster by directly accessing the
cluster IP address. The Controllers will handle all replication, so there is no requirement to make
changes specifically on the primary Controller.

Note In Avi Load Balancer, the cluster IP is not referred to as floating IP. The term floating IP
applies only to OpenStack.

OpenStack Controller Cluster

The two ways to configure OpenStack Controller cluster are as follows:

Method 1
The following are the steps to configure OpenStack Controller cluster:

1 Spin up three or more Avi Load Balancer Controllers on OpenStack.

2 Consider one Controller for cloud or cluster configuration. If Controller is not reachable from
outside OpenStack, assign Floating IP to Controller IP. You can access the controller using
Floating IP and configure Cluster VIP on the Controller.

avi-dev-venv) ~ $> neutron floatingip-create public

(avi-dev-venv) ~ $> neutron floatingip-associate 55cbe4ce-97d4-44fc-ad38-78faf0cbe2d7

2aecadeb-755a-495e-8f19-53301ee63d6b
Associated floating IP 55cbe4ce-97d4-44fc-ad38-78faf0cbe2d7

3 Configure the cloud, wait for the status to be green indicating that the installation is
successful.

VMware by Broadcom 147

VMware Avi Load Balancer Administration Guide

4 Create a port in OpenStack for cluster VIP. It should be in the Avi Load Balancer management
network.

avi-dev-venv) ~ $> neutron port-show cvip1

+-----------------------
+-----------------------------------------------------------------------------------+
| Field |
Value |
+-----------------------
+-----------------------------------------------------------------------------------+
| admin_state_up |
True |
| allowed_address_pairs | {"ip_address": "172.16.0.3", "mac_address":
""} |
| binding:host_id
| |
| binding:vif_details | {"port_filter":
true} |
| binding:vif_type |
vrouter |
| binding:vnic_type |
normal |
| description
| |
| device_id
| |
| device_owner
| |
| fixed_ips | {"subnet_id": "4982c62d-ada2-4067-879a-1c5b1ec94ec8",
"ip_address": "172.16.0.3"} |
| id | 1fed2319-a179-4dc5-
b9e5-49853606e7a8 |
| mac_address |
02:1f:ed:23:19:a1 |
| name |
cvip1 |
| network_id | 9feb21ba-6c14-44a3-
a478-1f09e16b60df |
| port_security_enabled |
True |
| security_groups | 367987f6-f373-4637-8867-
aa5b31dc60d2 |
| status |
DOWN |
| tenant_id |
037e661ac0cb44c89449e5e9b76b9a00 |
+-----------------------
+-----------------------------------------------------------------------------------+

VMware by Broadcom 148

VMware Avi Load Balancer Administration Guide

5 Assign Floating IP to cluster VIP if needed. If the Avi Load Balancer management network is
reachable from outside, Floating IP is not required.

(avi-dev-venv) ~ $> neutron floatingip-create public

(avi-dev-venv) ~ $> neutron floatingip-associate e5838127-f2f4-47d6-aaba-d5925d082514

1fed2319-a179-4dc5-b9e5-49853606e7a8
Associated floating IP e5838127-f2f4-47d6-aaba-d5925d082514

(avi-dev-venv) ~ $> neutron floatingip-list

+--------------------------------------+------------------+---------------------
+--------------------------------------+
| id | fixed_ip_address | floating_ip_address |
port_id |
+--------------------------------------+------------------+---------------------
+--------------------------------------+
| 55cbe4ce-97d4-44fc-ad38-78faf0cbe2d7 | 172.16.0.2 | 10.176.2.102 |
2aecadeb-755a-495e-8f19-53301ee63d6b |
| e5838127-f2f4-47d6-aaba-d5925d082514 | 172.16.0.3 | 10.176.2.104 |
1fed2319-a179-4dc5-b9e5-49853606e7a8 |
+--------------------------------------+------------------+---------------------
+--------------------------------------+

6 Configure cluster VIP in Avi Load Balancer Controller.

7 Use the cluster VIP or the cluster Floating IP to log in to Avi Load Balancer Controller.

8 Disassociate the Floating IP from Controller IP. It is an optional step. (Because it is done in
Step 2).

9 Add the other Controllers to the Cluster Configuration page.

Method 2
The following are the steps to configure OpenStack Controller cluster:

1 Spin up only one Controller.

2 Follow steps from 2 to 8 mentioned in method 1.

VMware by Broadcom 149

VMware Avi Load Balancer Administration Guide

3 Bring another set of Avi Load Balancer Controller nodes.

4 Add the Controller nodes in the cluster configuration.

Configure the Cluster IP Using the UI

The steps to add the cluster IP using the Avi Load Balancer UI are as follows.

Procedure

1 Navigate to Administration > Controller > Nodes and click EDIT.

2 Enter Name - Specify the name of the cluster.

3 Specify Controller Cluster IPv4 -Specify the new address.

4 Under Cluster Node, click ADD.

a Enter Node IP - Specify the IP address assigned to this Controller VM.

b Specify Hostname - Specify the host name assigned to this Controller VM.

c Enter Password - Specify the password to be used while authenticating with this node
(Not persisted).

d Enter Public IP Address - Specify the public IP address or hostname of the controller VM.
This field must be configured when the Service Engines (SEs) are unable to communicate
with the Controller private IP address directly, for example, when the SEs are deployed in
a remote environment and must access the Controller over the public internet through a
destination NAT. If SEs can directly communicate with the Controller Private IP, then the
Public IP field need not be configured.

e Click SAVE.

5 Click SAVE.

6 DNS host names may be specified in lieu of IP addresses as explained in Cluster Configuration
with DNS Hostnames.

Assigning Avi Load Balancer Controller IP in Different Deployments

For an Avi Load Balancer deployment in a DHCP environment, Controller IP is assigned using the
DHCP option. The process for assigning IP to Avi Load Balancer Controller differs based on the
configured cloud if Avi Load Balancer is configured using static IP.

Assigning Avi Load Balancer IP in VMware Deployment

In any VMware environment, Avi Load Balancer Controller IP address is assigned using OVF
properties during the installation. For more information, see Installing NSX Advanced Load
Balancer in VMware vSphere Environments topic in the VMware Avi Load BalancerInstallation
Guide.

VMware by Broadcom 150

VMware Avi Load Balancer Administration Guide

Assigning Avi Load Balancer Controller IP in AWS Deployment

In Amazon Web Services (AWS) or Cisco CSP integration, Avi Load Balancer Controller IP is
assigned using metadata. For more information, see Installing Avi Load Balancer in Amazon Web
Services.

The detailed information regarding assigning Avi Load Balancer Controller IP is available in
Deploying an EC2 Controller Instance section of Installing NSX Advanced Load Balancer in
Amazon Web Services topic in the VMware Avi Load BalancerInstallation Guide.

Assigning Avi Load Balancer Controller IP in Linux Server Cloud Deployment

In bare-metal integration or Linux Server Cloud integration (in the case of Docker), Avi Load
Balancer Controller IP is assigned while running the avi_baremetal_setup.py script. Enter the
Controller IP at the following prompt:

Please Enter Controller IP x.x.x.x

The following snapshot shows the section of the script when you will be prompted to enter Avi
Load Balancer Controller IP.

VMware by Broadcom 151

VMware Avi Load Balancer Administration Guide

For more information, see step number 7 of Installing NSX Advanced Load Balancer in a Linux
Server Cloud topic in the VMware Avi Load BalancerInstallation Guide.

Cluster Configuration with DNS Hostnames

Avi Load Balancer Controller cluster can be configured with DNS hostnames instead of IP
addresses.

Cluster configuration can be changed to use DNS hostnames instead of IP addresses by editing
node management IP or hostname in the Administration > Controller > Nodes page. It triggers a
new Cluster configuration process and updates all nodes to use DNS hostnames.

DNS is resolved when a secure connection is established between Controllers. This usually
happens when the cluster is configured or any of the nodes is rebooted. SEs resolve DNS
hostname of the Controller when establishing a secure connection to the Controller. This usually
happens when the SE is first created and when either an SE or a Controller reboots.

Note If cluster VIP is configured there must be a valid IP address only. An FQDN cannot be
configured for cluster VIP.

In addition, cluster configuration with DNS hostnames is not supported for write access vCenter
or NSX-T clouds where the Service Engines' Management IP address is statically allocated by the
Controller. It is supported for these cloud types if the Service Engines' Management IP address
assignment is using DHCP and the DHCP server provides DNS server information that allows the
Service Engine to resolve the Controller DNS hostnames.

VMware by Broadcom 152

VMware Avi Load Balancer Administration Guide

The cluster configuration process takes the DNS information of the leader node and updates
it in the new follower nodes. All nodes can reach the DNS server that is configured in the
Administration > System Settings > DNS/NTP page of the leader.

SEs can initiate connection using IP address from the VM metadata but are automatically
switched to use DNS hostname for the Controller nodes. So it is important for SEs to use the
correct DNS server to resolve Controller DNS hostname.

Changing Avi Load Balancer Controller Cluster

Configuration
This section discusses about how to change node membership or node IP address in an Avi Load
Balancer Controller cluster.

Note If you change the IP address of an Avi Load Balancer Controller node, you must run
a script to update the configuration. For more information, see Updating the Configuration
Following Avi Load Balancer Controller IP Address Change.

Remove both Followers (Dismantling the Cluster) Using the UI

The steps to remove both followers from an Avi Load Balancer Controller cluster are as follows:

Procedure

1 Navigate to Administration > Controller > Nodes.

2 Click EDIT.

3 Remove each of the follower IP addresses in the EDIT CLUSTER pop-up window.

4 Click SAVE.

Using the CLI

Note Ensure that you specify the host IP addresses of theAvi Load Balancer Controller nodes
instead of the IP addresses shown as below.

VMware by Broadcom 153

VMware Avi Load Balancer Administration Guide

Configuring the cluster with the Controller nodes at [10.10.25.81’].

If you add or remove nodes from the cluster, you need to bring down this Controller and then
spin up the Controller up with the new configuration.

Waiting for the cluster to be ready...

Controller is ready.

After saving,

n The cluster is dismantled and recovered into a single node.

n Each follower is re-imaged into its default state with no configuration and no access to the
leader.

n The leader holds the configuration. The SEs continue to connect to the leader.

For more information about the transition process, including descriptions of the status messages
that appear, see Clustering Patched Controllers.

Note During the transition from a cluster to a single node, the REST API will be unavailable for
around two minutes.

Change a Follower Node Using the UI

The steps to remove one of the follower nodes and add another one are as follows.

VMware by Broadcom 154

VMware Avi Load Balancer Administration Guide

Prerequisites

Note If the node is removed and replaced with a different node (different VM, container, or
bare-metal server), see Changing Avi Load Balancer Controller Cluster Configuration. The cluster
has to be Replace a Follower Node with a New Node, and recreated using the new node.

Procedure

1 Navigate to Administration > Controller > Nodes.

2 Click EDIT.

3 Edit the IP address for the follower node to change.

4 Click SAVE.

Using the CLI

Note Ensure that you enter the host IP addresses of the Avi Load Balancer Controller nodes
instead of the IP addresses shown in the example.

VMware by Broadcom 155

VMware Avi Load Balancer Administration Guide

Removed nodes with name=node-4

: cluster:nodes> save
: cluster> save

Configuring the cluster with the Controller nodes at [u’10.10.25.81’, ‘10.10.25.82’, ‘10.10.25.84’]. If
you add or remove nodes from the cluster, you should bring down this Controller and back up
with the new configuration.

Waiting for the cluster to be ready...

Controller is ready.

After saving,

n The removed follower node is sent an API request asking it to gracefully clear its state.

n Since the old follower is not always expected to clear its state, the leader will forcibly remove
it if necessary.

n The new follower node is added.

Replace the Leader Node

To replace the leader node, power it down to force one of the followers to take over leadership.

Procedure

1 Power down the leader node and leave it powered off for several minutes while one of the
followers assumes leadership.

2 Use the steps for adding a new follower node.

Note The leader cannot be directly removed. Instead, it must be replaced with another
leader.

To replace a follower, see Change a Follower Node Using the UI.

Replace a Follower Node with a New Node

This procedure is applicable for replacing a follower node with a completely different node that
is not currently in the cluster. For instance, these steps apply to replacing a follower node with a
new VM or bare metal server. The node cannot be replaced simply by inserting the node into the
network and editing the cluster configuration information.

Procedure

1 Log in to the leader node, and use the steps to dismantle the cluster by Remove both
Followers (Dismantling the Cluster) Using the UI nodes.

2 On the new node, install Avi Load Balancer. It involves spinning up a new virtual machine (VM)
or container from the Avi Load Balancer Controller package (OVA, QCOW or Docker image)
on the new node.

3 Follow the steps mentioned in the Deploying an Avi Load Balancer Controller Cluster section.

VMware by Broadcom 156

VMware Avi Load Balancer Administration Guide

Updating the Configuration Following Avi Load Balancer

Controller IP Address Change
The management IP addresses of each Avi Load Balancer Controller node should be static. It
applies to single-node deployments and 3-node deployments.

The cluster configuration and runtime configuration each contain the IP information for the
cluster. If the IP address of a leader or follower node changes (for example, due to DHCP), this
script must be run to update the IP information. The cluster will not function properly until the
cluster configuration is updated.

If the IP address of an Avi Load Balancer Controller node is changed for any reason (such as
DHCP), the following script must be used to update the cluster configuration. This applies to
single-node deployments and cluster deployments.

To repair the cluster configuration after an Avi Load Balancer Controller node’s IP address is
changed, run the change_ip.py script.

The script is located in the following directory:

/opt/avi/python/bin/cluster_mgr/change_ip.py

Note
1 The change IP script only changes the Avi Load Balancer cluster configuration. It does not
change the IP address of the host or the virtual machine on which Controller services are
running. For example, it does not update the /etc/network/interfaces file in a VMware-
hosted Controller. One must change the IP address for the VM in the vApp properties in
VMware.

2 Special consideration is required when changing the IP addresses of Controllers in a bare-

metal configuration. For more information, see Changing Controller IP Addresses in a Bare-
metal Environment.

Configure the Script

The script can run on the Avi Load Balancer Controller node whose management IP address
changed, or on another Avi Load Balancer Controller node within the same cluster.

Note Before running the script, make sure that new IPs are working on all nodes and are
reachable across nodes. If one or more IPs are not accessible, the script makes a best-effort
update, but upon restoring connectivity, there is no guarantee the cluster will be back in sync.

The script must run on one of the nodes that is in the cluster. If the script is run on a node that is
not in the cluster, the script will fail. The following is the list of parameters:

n -i ipaddr: Specifies the new IP address of the node on which the script is run.

n -o ipaddr: Specifies the IP address of another node in the cluster.

VMware by Broadcom 157

VMware Avi Load Balancer Administration Guide

n -m subnet-mask: If the subnet also changed, use this option to specify the new subnet. Enter
the mask in the following format: 255.255.255.0

n -g gateway-ipaddr: If the default gateway also changed, use this option to specify the new
gateway.

Note The -m and -g options apply to all IP addresses in the cluster.

Updating IP Information for a Single-node Deployment

To update Avi Load Balancer Controller IP information for a single-node deployment, use a
command string such as the following:

*change_ip.py -i **ipaddr*

This command is run on node 10.10.25.81. Whereas no other nodes are specified, this is assumed
to be a single-node cluster (just this Avi Load Balancer Controller).

username@avi:~$ ∣ change_ip.py -i 10.10.25.81

In the following example, the default gateway of the node also has changed.

username@avi:~$ ∣ change_ip.py -i 10.10.25.81 -g 10.10.10.1

Updating IP Information for an Avi Load Balancer Controller Cluster

To update Avi Load Balancer Controller IP information for a cluster, use a command string such
as the following:

Note Before executing change_ip.py, ensure that all new IPs are reachable from one another
over SSH ports (22 for regular, 5098 for containers).

change_ip.py -i ipaddr -o ipaddr -o ipaddr

This command is run on node 10.10.25.81, which is a member of a 3-node cluster that also
contains nodes 10.10.25.82 and 10.10.25.83.

username@avi:~$ ∣ change_ip.py -i 10.10.25.81 -o 10.10.25.82 -o 10.10.25.83

The script can run on any of the nodes in the cluster. The following example is run on node
10.10.25.82.

username@avi:~$ ∣ change_ip.py -i 10.10.25.82 -o 10.10.25.81 -o 10.10.25.83

Note After executing change_ip.py, in case of failure, use recover.py to convert nodes
to single nodes and create the 3-node cluster again. For more information, see Recover a Non-
Operational Controller Cluster.

To verify, go to the Controller nodes page and ensure all nodes are CLUSTER_ACTIVE.

VMware by Broadcom 158

VMware Avi Load Balancer Administration Guide

Change Controller IPs on Nutanix Cluster

For the Controller cluster to come up with the new IPs, follow the steps below.

1 Change the IP address of each Controller node within the cluster to the new IP by manually
editing the network-scripts on the host and changing the interface configuration.

For instance, /etc/network/interfaces/ file on Avi Load Balancer Controller virtual

machine must be modified as follows (if using static IP):

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
address <ipv4 address>
netmask 24
gateway <ipv4 gw>

2 Ensure that the new Controller IP addresses are reachable in the network from the other
Controller nodes.

3 Run /opt/avi/python/bin/cluster_mgr/change_ip.py on the Controller to reflect the

above IP address change.

4 Reboot the Controller.

The above procedure is for a single node cluster specified in step 2. For a three-node cluster
deployment, change the IPs on all the Controllers and run the command as shown below from
any Controller node to update Avi Load Balancer Controller IP information for a cluster:

username@avi:~$ change_ip.py -i ipaddr -o ipaddr -o ipaddr

Where,

n -i ipaddr: Specifies the new IP address of the node on which the script is run.

n -o ipaddr: Specifies the IP address of another node in the cluster.

n -m subnet-mask: If the subnet also changed, use this option to specify the new subnet. Enter
the mask in the following format: 255.255.255.0.

n -g gateway-ipaddr: If the default gateway also changed, use this option to specify the new
gateway.

Customizing Avi Load Balancer Configuration

Avi Load Balancer Controller cluster instances are frequently required to be deployed from
a common initial single Controller configuration that differs from the factory-default settings.
This can be accomplished by creating a JSON file with the required object definitions and then
using it to deploy subsequent controllers, which, as leaders, can then add followers to become

VMware by Broadcom 159

VMware Avi Load Balancer Administration Guide

clusters. You will have a set of identically initialized Controller clusters, ready to be individualized
as needed.

Create setup.json
In most cases, these objects can be created by referring to the Avi Load Balancer REST API
section.

The following example updates the system configuration by adding 8.8.8.8 to the DNS
configuration:

{
"SystemConfiguration": [
{
"dns_configuration": {
"search_domain": "",
"server_list": [
{
"type": "V4",
"addr": "8.8.8.8"
}
]
}
}
]
}

In the case of complex objects such as the SSLKeyAndCertificate object, the JSON file can
be created by running a diff command against two configuration files. In a typical deployment,
generating setup.json on a test Controller environment is recommended. This generated file
can then be used as a template for actual deployments. An Avi Load Balancer Controller
configuration snapshot can be taken using the export CLI command.

> export configuration file before.cfg

Please enter the passphrase to encrypt configuration:
Downloaded the attachment to before.cfg
Completed writing the export configuration to before.cfg

Configure objects using the UI or CLI, as required.

> export configuration file after.cfg

Please enter the passphrase to encrypt configuration:
Downloaded the attachment to after.cfg
Completed writing the export configuration to after.cfg

Beyond this, configuration diff can be taken using a Python script.Avi Load Balancer has written
expressly to customize initial configuration of another Controller.

/opt/avi/scripts/diff_config.py -f before.cfg -t after.cfg > setup.json

VMware by Broadcom 160

VMware Avi Load Balancer Administration Guide

User passwords can be encrypted using the following code when creating setup.json with the
User object.

/opt/avi/scripts/avi_passwd_tool.py --password admin --salt

fF6ngAb3pvPgpbkdf2_sha256$100000$fF6ngAb3pvPg$ijkEue1M9fR/
qsLVgzvPe7N0VvOxIjDiJVmK9NIx+0Q=$6$fF6ngAb3pvPg$CqAKtNRZtgXtJchrPmoxUgdLFM7rFGmta1tWb7sobQI4iS
ZAY2QuAOBNtboVGrmDYPMCvqXXH6lARr9RedCJT.

Deployment using setup.json

It is recommended to take a configuration backup before deploying the Controller using
the setup.json file created by the Python script. Use the following command to create an
encrypted backup of the existing configuration.

/opt/avi/python/bin/portal/manage.py export_configuration --file ~/setup-old.json --

passphrase secret

For a Mesos/Bare-Metal Deployment

Copy setup.json to the persistent directory in the host mounted as /vol in the Controller
container. If you are using the avi_baremetal script, the default location is /opt/avi/
controller/data on the host. When deploying the Controller as a container, setup.json can
be passed as an additional argument to the avi_baremetal_setup.py script. For example:

./avi_baremetal_setup.py -c -cc 4 -cm 12 -i 10.10.22.108 -m 10.10.22.108 --setup-json /root/

configs/avi-setup.json

For a Controller Deployment as a Virtual Machine

Wait until the Controller comes up. Place the config file on the Controller as /var/lib/avi/etc/
setup.json (note the filename). Upon reboot or fresh-start, the Avi Load Balancer Controller will
self-configure using the provided setup.json file.

reboot

For an OpenStack Deployment

UserData config size is limited to 48 Kb if the size of setup.json is within allowable limits.

># cfgdrv userdata

>nova boot --config-drive true --image avicontroller --key-name mykey --flavor 4 --user-data /
root/my-avi-config.json --nic net-id=7402bf4f-240f-4172-99c1-90000ea45f86 avicontrollers

># metasvc userdata

>nova boot --config-drive false --image avicontroller --key-name mykey --flavor 4 --user-
data /root/my-avi-config.json --nic net-id=7402bf4f-240f-4172-99c1-90000ea45f86 avicontrollers

VMware by Broadcom 161

VMware Avi Load Balancer Administration Guide

If the setup.json size is bigger then than the allowable limit, setup.json can be uploaded and
referred to in the deployment phase.

User data can refer to the file using the “url” or “file” tag. Following is an example of my-avi-
config-url.json with URL.

{
"META": {
"init_config": {
"url": "https://s3-us-west-2.amazonaws.com/avi-controller-configs/linuxserver-
awsipam-setup.json"
}
}
}

Following is an example of my-avi-config-url.json with filepath.

{
"META": {
"init_config": {
"file": "/vol/linuxserver-awsipam-setup.json"
}
}
}

Following is an example of deployment.

># cfgdrv userdata indirection

>nova boot --config-drive true --image avicontroller --key-name mykey --flavor 4 --user-data /
root/my-avi-config-url.json --nic net-id=7402bf4f-240f-4172-99c1-90000ea45f86 --min-count=3 --
max-count=3 avicontrollers

># metasvc userdata indirection

nova boot --config-drive false --image avicontroller --key-name mykey --flavor 4 --user-data /
root/my-avi-config-url.json --nic net-id=7402bf4f-240f-4172-99c1-90000ea45f86 --min-count=3 --
max-count=3 avicontrollers

For an AWS Deployment

UserData config size is limited to 16Kb. If the size of setup.json is within allowable limits,
cut-paste the my-avi-config.json into the user-data section during launch from AWS Web
Console.

# metasvc userdata ec2-run-instances ami-b7ea27d7 -f /root/my-avi-config.json -t c4.2xlarge

-s subnet-62f1b707 -g sg-642d8d02

If the setup.json size is bigger than the allowable limit, cut-paste the my-avi-config-
url.json into the user-data section during launch from AWS Web Console.

# metasvc userdata indirection ec2-run-instances ami-b7ea27d7 -f /root/my-avi-config-url.json

-t c4.2xlarge -s subnet-62f1b707 -g sg-642d8d02

VMware by Broadcom 162

VMware Avi Load Balancer Administration Guide

my-avi-config-url.json follows similar formats as discussed in the OpenStack section.

Following is a sample my-avi-config-url.json file for the S3 bucket.

{
"META": {
"init_config": {
"s3": "avi-controller-configs/linuxserver-awsipam-setup.json"
}
}
}

For uploading setup.json on the S3 bucket:

n Public : use the url style or s3 style

n Private through RBAC on VM: use the s3 style. The VM role must have s3:GetObject action
allowed to be able to s3-get the object using IAM.

n Private through RBAC on S3-bucket: use the s3 style. The VM role must have AWS access.
The S3 bucket must have permissions for the account or user or VM role to download the
object.

Example Bucket Policy:

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AddPerm",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::139284885014:role/BM-AviController-Role",
"arn:aws:iam::139284885014:root"
]
},
"Action": "s3:*",
"Resource": "arn:aws:s3:::avi-controller-configs/*"
}
]
}

For an Azure Deployment

There are two ways to provide initial configuration in an Azure environment:

n Using Azure CLI

n Using ARM (Azure Resource Manager) template

VMware by Broadcom 163

VMware Avi Load Balancer Administration Guide

Using Azure CLI

If the Avi Load Balancer Controller is deployed using Azure CLI, the JSON file can be provided
during the deployment.

az vm create --resource-group rahulr-jenkins-resource-group --location centralus --image

avi-networks:avi-vantage-adc:avi-vantage-adc-byol:17.2.7 --name Avi-Test-Controller --size
Standard_F8s --subnet /subscriptions/<subscription_id>/resourceGroups/<resource_group_name>/
providers/Microsoft.Network/virtualNetworks/<vnet_name>/subnets/rahulr-subnet --public-ip-
address "" --nsg "" --custom-data ./initial_config.json

Using Azure ARM Template

If the Avi Load Balancer Controller is deployed using the ARM template, the JSON data is
provided as the Custom Data on the Custom deployment page of the Azure portal. Navigate
to Home > Templates > avi-cluster-managed-disks-market place > Custom deployment.

Use the required JSON template in the Custom Data field. For reference, the below JSON
template is for adding 8.8.8.8 to DNS configuration. Copy the JSON configuration mentioned
below, and add it to the Custom Data field in the ARM template.

{
"SystemConfiguration": [
{
"dns_configuration": {
"search_domain": "",
"server_list": [
{
"type": "V4",

VMware by Broadcom 164

VMware Avi Load Balancer Administration Guide

"addr": "8.8.8.8"
}
]
}
}
]
}

Clustering Avi Load Balancer Controllers of Different

Networks
Controller clusters provide high availability (HA), redundancy, and increased analytic workload
scale.

Controllers communicate with each other over a single management IP address, the Controller
Cluster IP. They also use this path to communicate with all SEs within the fabric.

Although Controllers do not have to exist within the same limitations, consider the following
conditions:

n Controllers must be within the same region (ideally the same data center). It helps
synchronize the databases and perform actions such as log indexing and data retrieval.

n Controllers have the option of sharing a cluster IP address. The cluster IP address is owned
by the primary Controller within the cluster. To share an IP address, all Controllers must have
a NIC in the same network.

n Each Controller must have access to the IP addresses of other Controllers through configured
network routes.

n RTT (round-trip time) value between two Controller nodes must be less than 20 milliseconds.

Considerations
AWS

AWS Availability Zones (AZs) provide redundancy and separate fault domains. All AWS
regions support a minimum of two AZs. To leverage HA provided by AWS AZs, it is
recommended to deploy different Avi Load Balancer Controller instances of a cluster in
different AZs.

Azure

The Controller cluster must be running inside the Azure cloud. Additionally, consider the
following information:

n Azure credentials (username and password or application ID) which have contri butor
privilege access over the Controller cluster VMs and AviController role access over the
virtual network that is hosting the Controller cluster.

VMware by Broadcom 165

VMware Avi Load Balancer Administration Guide

n Subscription_id of the subscription where the Controller virtual machines are running.

OpenStack

OpenStack requires Avi Load Balancer to maintain a cluster IP address. So, Avi Load
Balancer deployed into an OpenStack cloud does not support clustering of Avi Load Balancer
Controllers present in different networks.

High Availability for Avi Load Balancer Controllers

Avi Load Balancer can run with a single Controller (single-node deployment) or a three-node
Controller cluster.

In a deployment that uses a single Controller, that Controller performs all administrative functions
and all analytics data gathering and processing.

Adding two additional nodes to create a three-node cluster provides node-level redundancy for
the Controller and maximizes performance for CPU-intensive analytics functions. Whereas the
lone Controller in a single-node deployment performs all administrative functions and analytics
data collection and processing, these tasks are distributed in three-node cluster.

In a three-node Controller cluster, one node is the primary (leader) node and performs the
administrative functions. The other two nodes are followers (secondaries), and perform data
collection for analytics, in addition to standing by as backups for the leader.

Operation of Avi Load Balancer Controller HA

This section explains how HA operates within an Avi Load Balancer Controller cluster.

Quorum
Avi Load Balancer Controller-level HA requires a quorum of Controller nodes to be up. In a
three-node Controller cluster, quorum can be maintained if at least two of the three Controller
nodes are up. If one of the Controllers fails, the remaining two nodes continue service and Avi
Load Balancer continues to operate. However, if two of the three nodes go down, the entire
cluster goes down, and Avi Load Balancer stops working.

Failover
Each Controller node in a cluster periodically sends heartbeat messages to the other Controller
nodes in the cluster through an encrypted SSH tunnel using TCP port 22 (port 5098 if running as
Docker containers).

The heartbeat interval is ten seconds. The maximum number of consecutive heartbeat messages
that can be missed is four. If one of the Controllers does not hear from another Controller for 40
seconds (four missed heartbeats), the other Controller is assumed to be down.

If only one node is down, quorum is maintained, and the cluster can continue to operate.

n If a follower goes down but the leader node remains up, access to virtual services continues
without interruption.

VMware by Broadcom 166

VMware Avi Load Balancer Administration Guide

n If the primary (leader) node goes down, the member nodes form a new quorum and elect a
cluster leader. The election process takes about 50-60 seconds, and during this period, there
is no impact on the data plane. The SEs will continue to operate in the headless mode, but the
control plane service will be unavailable. During this period, users will be unable to create a
VIP through LBaaS or use the Avi Load Balancer UI, API, or CLI.

Converting a Single-Node Deployment to a 3-node Cluster

To convert a single-node Avi Load Balancer Controller deployment into a three-node
deployment, use the following steps.

In this procedure, the Controller node that is already deployed in the singe-node deployment is
referred to as the incumbent Controller.

Procedure

1 Install one new Avi Load Balancer Controller node. During installation, configure the following
settings:

n Node management IP address

n Gateway address

2 Connect the management interface of each new Controller node to the same network as the
incumbent Controller. After the incumbent Controller detects the two new Controller nodes,
the incumbent Controller will become the primary (leader) Controller for the three-node
cluster.

3 Use a web browser to navigate to the management IP address of the primary (leader)
Controller.

4 Navigate to Administration > Controller > Nodes, and click Edit. The Edit CLUSTER pop-up
window appears.

5 In the Controller Cluster IP field, enter the shared IP address for the Controller cluster.

6 In the Public IP Address or Hostname field, enter the management IP addresses of the new
Controller nodes.

Note Password for the admin account is required for each node of the cluster for
configuring cluster in AWS Cloud.

7 After these steps, the incumbent Controller becomes the primary (leader) for the cluster and
invites the other Controllers to the cluster as members. Avi Load Balancer then performs a
warm reboot of the cluster. This process can take two- three minutes. The configuration of
the primary (leader) Controller is synchronized to the new member nodes when the cluster
comes online following the reboot.

VMware by Broadcom 167

VMware Avi Load Balancer Administration Guide

Control Plane High Availability

You can deploy a set of three Avi Load Balancer Controllers as a High Availability cluster, as a
best practise. In a cluster deployment, one of the Controllers acts as a leader. It performs load
balancing and configuration management for the cluster. While the other two Controllers are
followers, they collaborate along with the leader to perform data collection from SEs and process
analytics data.

High Availability for Avi Load Balancer Controllers

Avi Load Balancer can run with a single Controller (single-node deployment) or with a three-node
Controller cluster. In a deployment that uses a single Controller, that Controller performs all the
administrative functions, it also gathers and processes all the analytics data.

You can create a three-node cluster by adding two additional nodes. This three-node cluster
provides a node-level redundancy for the Controller and maximizes performance for CPU-
intensive analytics functions. However, for a single Controller in a single-node deployment, it
performs all administrative functions; collects and processes all the analytics data. These tasks
are distributed in a three-node cluster.

Operation of Avi Load Balancer Controller High Availability

This section explains how high availability (HA) operates within Avi Load Balancer Controller
cluster.

Quorum
The Controller level HA requires a quorum of Avi Load Balancer Controller nodes to be up. In a
three-node Controller cluster, quorum can be maintained if at least two of the three Controller
nodes are up. If one of the Controllers fail, the remaining two nodes continues service and Avi
Load Balancer continues to operate. However, if two of the three nodes go down, then the entire
cluster goes down, and Avi Load Balancer stops working.

Failover
Each Controller node in a cluster periodically sends heartbeat messages to the other Controller
nodes in a cluster, through an encrypted SSH tunnel using TCP port 22 (port 5098 if running as
Docker containers).

VMware by Broadcom 168

VMware Avi Load Balancer Administration Guide

Primary (leader)

0110

Heartbeat
messages

0110 0110

If only one node is down, then the quorum is still maintained and the cluster can continue to
operate.

If a follower node goes down but the primary (leader) node remains up, then the access to virtual
services continues without any interruption.

Primary (leader)

0110

Member node down. 0110

No reply to heartbeats.

n If the primary (leader) node becomes unavailable because of events such as a hardware
failure or a snapshot of the leader node VM being taken, the member nodes will form a
new quorum and elect a cluster leader. The election process takes about 50-60 seconds and
during this period, there is no impact on the data plane. The SEs will continue to operate in
the Headless mode, but the control plane service will not be available. During this period,
you cannot create a VIP through LBaaS or use the Avi Load Balancer user interface, API, or
CLI.

VMware by Broadcom 169

VMware Avi Load Balancer Administration Guide

Primary (leader)
node down.
No reply to heartbeats.

Headless mode
(during election of
new primary/leader)
0110 0110

Converting a Single-Node Deployment to a Three-node Cluster

This section explains the process to convert a single-node deployment to a three-node cluster.

In this procedure, the Avi Load Balancer Controller node that is already deployed in the singe-
node deployment is referred to as the incumbent Avi Load Balancer Controller.

To convert a single-node Avi Load Balancer Controller deployment into a three-node

deployment, follow the steps below:

1 Install a single new Controller node. During installation, configure the following settings:

n Node management IP address

n Gateway address

2 Connect the management interface of each new Controller node to the same network as the
incumbent Controller. After the incumbent Controller detects the two new Controller nodes,
the incumbent Controller becomes the primary (leader) Controller node for the three-node
cluster.

3 Use a web browser to navigate to the management IP address of the primary (leader)
Controller node.

4 Navigate to Administration > Controller > Nodes and click Edit. The Edit Cluster window
appears.

5 In the Controller Cluster IP field, specify the shared IP address for the Controller cluster.

6 In the Public IP Address or Host Name field, specify the management IP address of the new
Controller node.

Note To configure cluster in AWS Cloud, each node of the cluster requires admin account
credentials.

The incumbent Controller becomes the primary (leader) for the cluster and invites the other
Controller to the cluster as members.

VMware by Broadcom 170

VMware Avi Load Balancer Administration Guide

The Avi Load Balancer then performs a warm reboot of the cluster. This process can take
two or three minutes. After the reboot, the configuration of the primary (leader) Controller is
synchronized to the new member nodes once the cluster appears online.

Primary (leader)

Cluster IP:
10.30.163.63
10.30.163.68 0110

Invite to join Invite to join

cluster cluster

0110 0110

10.30.163.64 10.30.163.65

To know more about cluster HA, see the links below:

n Controller Cluster IP

n Clustering Avi Load Balancer Controllers of Different Networks

n Impact of a Controller Failure

For more information on how to Enable Per-app SE mode for a Service Engine Group, see
Per-app SE Mode topic in the VMware Avi Load BalancerConfiguration Guide.

Switching Among Controllers Using the Controller Site

Selector
This section discusses the steps to configure Controller sites and then switching between them
through the Avi Load Balancer UI.

Synchronize the list of sites across all participating Controller sites using a script, Ansible or
Terraform, to create a primary list of Controller sites. Thereafter, this information is propagated
to every Controller deployment using one of the automation methods.

For example, to define the trio of Controller sites mentioned in the subsequent Controller
Site Selector Use section, you must have two JSON payloads to POST to each /api/
controllersite. The two POSTs from any given Controller must point to the other two. Thus, in
this example, there is a total of six POSTs.

Note Adding the local Controller as a Controller site entry is no longer required. If at least one
record does not match the current site, then Avi Load Balancer automatically shows the current
site’s hostname (if applicable) or IP address.

From the Earth site:

VMware by Broadcom 171

VMware Avi Load Balancer Administration Guide

"name": "Mars",

"address": "192.0.2.20"

}, {

"name": "Venus",

"address": "192.0.2.30"

From the Mars site:

"name": "Earth",

"address": "192.0.2.10"

}, {

"name": "Venus",

"address": "192.0.2.30"

From the Venus site:

"name": "Earth",

"address": "192.0.2.10"

}, {

"name": "Mars",

"address": "192.0.2.20"

VMware by Broadcom 172

VMware Avi Load Balancer Administration Guide

Example: POST (all releases)

VMware by Broadcom 173

VMware Avi Load Balancer Administration Guide

Note
n A Controller site maps to one Controller name and IP address. Thus, for the feature to work
with any Controller in a cluster, three local Controller names and corresponding IP addresses
must be configured as sites. In the above example, consider three POSTs as shown below:

"name": "Earth",

"address": "192.0.2.10"

}, {

"name": "Earth2",

"address": "192.0.2.11"

}, {

"name": "Earth3",

"address": "192.0.2.12"

n The REST API call api/controllersite reveals the set to which the user has the
PERMISSION_CONTROLLERSITE access right. Lack of PERMISSION_CONTROLLERSITE hides the
Controller site name and drop-down icon from the main menu bar.

n Controller sites can only be added/modified through the CRUD call at /api/controllersite
with X-Avi-Version set in the header. Adding a Controller site to a list of sites requires that at
least two fields be given in the payload, name and address.

n Site configuration is locally significant, that is, adding site2 to site1’s list does not automatically
result in site1 appearing in site2’s list.

Controller Site Selector Use

1 When there is but one Controller site record in the set, just the Controller’s name appears to
the left of the ellipsis (three dots).

2 On the other hand, if there are multiple records, a V-shaped drop-down icon appears to the
left of the ellipsis (three dots).

VMware by Broadcom 174

VMware Avi Load Balancer Administration Guide

3 Clicking on the drop-down icon reveals the Controller sites within the set to which the user
has access.

4 Hovering over a particular Controller name causes it to be highlighted. Select the Controller to
switch to that site.

5 The selected Controller opens in a new tab, while the previous tab remains open in the
background. The user can now communicate directly with the new Controller. Due to SSO
being active, no login screen was presented by the first Controller. The login was performed
automatically, behind the scenes.

Controller Cluster Deployment Across Two Availability

Zones
This section discusses Controller cluster deployments across two availability zones (AZ), what
happens during failures, and the recovery actions needed during certain failures.

The Avi Load Balancer Controller is deployed as a three-node cluster for high availability and
scalability. These nodes must be deployed such that the RTT (round-trip time) value between
two Controller nodes is less than 20 milliseconds. In public cloud deployments such as AWS, a
region has multiple availability zones. In such deployments, as a best practice, each Controller
node should be deployed in a separate AZ.

Controller cluster deployment considerations are straightforward where the region has three or
more AZs. However, in many disaster recovery (DR) deployments, there are only two AZ across
which the three Controller cluster nodes are deployed, as depicted below:

VMware by Broadcom 175

VMware Avi Load Balancer Administration Guide

Figure 5-1. Two Availability Zones within One Region

AZ-1 AZ-2

C1 C2 C3

SE SE SE SE SE SE

The Controller cluster will be UP in partition scenarios if at least two nodes are UP and connected.
SEs will attempt to connect to the active partition. If they cannot connect to the active partition,
they will operate without a Controller in a headless fashion and continue to serve application
traffic.

In the above deployment, if AZ-2 goes DOWN, C1 will be brought down due to a lack of quorum. In
such a scenario, manual intervention is needed to bring the Controller cluster UP.

At a high level, the manual workflow provides a way to recover the remaining node as a stand-
alone cluster and permits two new nodes to be added when appropriate. The procedure is
intentionally kept manual to force the user to recover the partitions carefully.

Recover a Non-Operational Cluster

Log in to Controller C1 and run the /opt/avi/scripts/recover_cluster.py script. It will
reconfigure C1 as a standalone cluster, preserve the configuration and analytics data (logs
and metrics), and bring the Controller cluster UP. Secure channel credentials for the other two
Controller nodes will be revoked, and SEs having connectivity with C1 will be brought UP. As part
of this, SEs will reconfigure themselves to connect only to C1.

What Happens When AZ-2 is Recovered

C2 and C3 will be able to form a Cluster. SEs in AZ-2 might connect to either partition (C1 or
C2+C3). At this time, both the partitions are active, which could cause disruption if they run for
too long. To prevent this, each Controller node monitors other Controller nodes in its configured
members list. If a node identifies that it is not present in the configured members list of the other
node, it will bring itself down for manual recovery. In this case, both C2 and C3 will detect that C1
has moved forward with manual recovery and will bring themselves down.

VMware by Broadcom 176

VMware Avi Load Balancer Administration Guide

Secure channel credentials for all SEs will be revoked. SEs connecting to C2 or C3 during this
time will detect that the Controller cluster is in manual recovery state; they will reboot themselves
to no longer serve any application traffic. Once an SE comes UP, it will try to connect to C1 to
establish normal operations. REST API commands sent to the Controllers, C2, and C3 will fail with
a status code of 520, indicating that these nodes must be reset to factory defaults using the
clean_cluster.py script.

Clean a Non-Operational Cluster

It is assumed that C2 and C3 will be brought down soon after AZ-2 becomes operational. Log
into C2 and C3 and run the script, /opt/avi/scripts/clean_cluster.py. This script will wipe out
all the configuration and analytics and bring it up as a standalone node. Once C2 and C3 are
cleaned, they can rejoin back to C1 to form a three-node cluster.

Impact of a Controller Failure

The impact of the loss of a Controller depends on whether the Controller is deployed as a
standalone or as part of a three-node Controller cluster. A Controller failure can occur for various
reasons, such as a software failure or failure in the underlying hardware. A Controller can also fail
by losing connectivity to the networks containing the rest of the Avi Load Balancer infrastructure.

Controller Cluster
In a Controller cluster, the three Controllers divide the workload among themselves. If one
Controller fails, the remaining two Controllers continue operations as normal with no impact to
data plane traffic.

If the failed Controller was the cluster leader, one of the remaining Controllers takes over as
cluster leader. This change in leadership has no direct impact on operations. If the Controller
cluster IP address has been created, that IP address will move to the new leader, which will begin
ARPing (advertising) the address.

Cluster Quorum
A Controller cluster requires at least two of its three nodes to be up, in order to maintain quorum.
This eliminates the “split brain” scenario in which two devices are simultaneously active (primary)
with potentially conflicting configuration updates.

If a Controller is still online, but has merely lost contact with the other Controllers, it will no
longer be part of the cluster until it has regained connectivity. So if one Controller and multiple
Service Engines are created in each of the three data centers, and one data center has lost
connectivity to the other two data centers, the Controller at the isolated data center will not
accept configuration changes as a standalone. The isolated Controller must either reconnect with
the cluster or be formally demoted to a standalone.

VMware by Broadcom 177

VMware Avi Load Balancer Administration Guide

Replacing a Failed Controller

If Controller failure is a permanent state, take the following actions to remove the Controller and
restore full high availability to the cluster.

1 If the Controller was in a virtual machine, delete the VM from the cloud orchestrator.

2 From the web interface of another Controller that is still up, delete the IP address of the failed
Controller. Navigate to Administration > Controller.

3 Install a new Controller.

Note Do not log into the web interface of the new Controller. Only perform initial setup such
as selecting the cloud orchestrator.

4 From the existing Controller, add the IP address of the new Controller. After a few minutes,
the status of the cluster must turn green.

Standalone Controller
In a standalone Controller configuration, a Controller failure leaves the Avi Load Balancer system
in a headless state. In a headless state, existing Service Engines (SEs) will continue to operate
with the last instructions they were given.

No new configuration changes are possible until a Controller is restored. This can be done
by rebuilding a new Controller, which is disruptive to existing connections, or by bringing the
existing Controller back online, which is transparent to existing data traffic.

While in a headless state, SEs will attempt to buffer client logs if sufficient disk space is available.
If a Controller is temporarily offline, such as when the Controller’s host is rebooted, the SEs will
reconnect after the Controller returns, allowing the buffered client logs to be retrieved.

Recover a Non-Operational Controller Cluster

When two of the three Avi Load Balancer Controller nodes within a cluster are permanently
down and not recoverable, the remaining Controller node in the cluster will be marked
operationally down due to the lack of a cluster quorum.

Note All SEs continue to operate in headless mode.

Follow the steps below to return to a highly available three-node cluster:

1 To recover the cluster, you must first convert the remaining healthy Controller node to a
single-node cluster configuration. Thereafter, two new nodes can be added to the cluster.

2 There are two ways of recovering a Controller, that is, with configuration and without
configuration. It is important to recover one node with configuration to ensure it is made
the Controller leader while other nodes are added as followers to the cluster:

n To recover a Controller with configuration, use the /opt/avi/scripts/

recover_cluster.py script.

VMware by Broadcom 178

VMware Avi Load Balancer Administration Guide

n To recover a Controller without configuration (essentially a factory reset; rarely

necessary), use the /opt/avi/scripts/clean_cluster.py script instead. It is not
reversible. The Controller will take a longer time to recreate the database. The/opt/avi/
scripts/clean_cluster.py script performs the below tasks:

n By default, this script reboots the connected SEs, unless the script is run with the
switch.

/opt/avi/scripts/clean_cluster.py --skip-se-reboot

n The only way to login to the Controller node after running the script is to reset the
admin password through the UI.

Typical Recovery
To convert the remaining Controller node to a single-node cluster while preserving the Avi Load
Balancer configuration, execute the following script from the root account. If you attempt to
execute it from a non-root account, the script will fail with a Permission denied message. Run
sudo and enter the admin password to be promoted to root before running the script.

root@controller1:/home/admin# /opt/avi/scripts/recover_cluster.py

The script will request confirmation as a precaution and remind the user must run the script as
root.

It is highly recommended to power off the other Controllers that were part of the cluster when
running the recover_cluster.py script. Failure to do so can put the current and other nodes in an
inoperable state.

The script stops all services on the Controller and restarts them. The Controller will be down and
inaccessible for a few minutes.

Once the script finishes, you can log into the Controller node as a single-node cluster. To make
this a highly available three-node cluster, add two new, unconfigured Controllers nodes to the
cluster.

Note Ensure that the Controllers are on the same base and patch version.

API - Configuring the Avi Load Balancer Controller Cluster

The Avi Load Balancer REST API provides commands for managing the Controller cluster. This
section discusses the steps to configure the Controller cluster using the API.

Getting the Avi Load Balancer Controller Cluster Configuration

To get the existing cluster configuration, send an API request as follows:

GET /api/cluster

VMware by Broadcom 179

VMware Avi Load Balancer Administration Guide

The following is a sample output.

{
url: "https://10.10.5.27/api/cluster",
uuid: "cluster-005056ac9e91",
name: "cluster-0-1",
tenant_ref: "https://10.10.5.27/api/tenant/admin",
virtual_ip: {
type: "V4",
addr: "10.10.5.27"
},
nodes: [
{
ip: {
type: "V4",
addr: "10.10.5.16"
},
vm_hostname: "node1.controller.local",
vm_uuid: "005056ac9e91",
name: "10.10.5.16",
}
]
}

In this example, the cluster contains only a single member. When Avi Load Balancer is installed,
the Controller created during installation is placed in a cluster as the primary Controller. The
management IP address is configured as the cluster IP address.

Adding Avi Load Balancer Controller Nodes to the Cluster

To create a 3-node cluster, send a request such as the following to add two more Controller
nodes to the cluster.

For controller-ip, specify the management IP address of the individual Controller node, not the
IP address to be assigned to the cluster. The cluster IP is specified under virtual_ip.

PUT /api/cluster

The following is a PUT Data example.

{
uuid: "cluster-005056ac9e91",
name: "cluster-0-1",
virtual_ip: {
type: "V4",
addr: "10.10.5.27"
},
nodes: [
{
ip: {
type: "V4",
addr: "10.10.5.16"
},
vm_hostname: "node1.controller.local",

VMware by Broadcom 180

VMware Avi Load Balancer Administration Guide

vm_uuid: "005056ac9e91",
name: "10.10.5.16",
},
{
ip: {
type: "V4",
addr: "10.10.5.15"
},
name: "10.10.5.15",
},
{
ip: {
type: "V4",
addr: "10.10.5.17"
},
name: "10.10.5.17",
}
]
}

Removing Avi Load Balancer Controller Nodes from the Cluster

To remove Controller nodes from the cluster, use a request such as the following.

PUT /api/cluster

The following is a PUT Data example.

{
uuid: "cluster-005056ac9e91",
name: "cluster-0-1",
virtual_ip: {
type: "V4",
addr: "10.10.5.27"
},
nodes: [
{
ip: {
type: "V4",
addr: "10.10.5.16"
},
name: "10.10.5.16",
}
]
}

Getting Runtime Information for the Cluster

The following request gets runtime information for the cluster.

The cluster is ready for operation when the cluster_state is CLUSTER_UP_HA_ACTIVE (for a 3-node
cluster) or CLUSTER_UP_NO_HA (for a 1-node cluster).

This example shows CLUSTER_UP_HA_ACTIVE.

VMware by Broadcom 181

VMware Avi Load Balancer Administration Guide

GET /api/cluster/runtime

{
node_info: {
uuid: "005056ac115e",
mgmt_ip: "10.10.5.15",
has_config: "True",
ip: "node2.controller.local",
vm_mor: "vm-54736",
version: "16.1.1(9014) 2016-03-26 01:05:26 UTC",
vm_uuid: "005056ac115e"
},
node_states: [
{
up_since: "2016-03-26 16:06:21",
state: "CLUSTER_ACTIVE",
role: "CLUSTER_LEADER",
name: "10.10.5.15"
},
{
up_since: "2016-03-26 16:07:08",
state: "CLUSTER_ACTIVE",
role: "CLUSTER_FOLLOWER",
name: "10.10.5.17"
},
{
up_since: "2016-03-26 16:07:31",
state: "CLUSTER_ACTIVE",
role: "CLUSTER_FOLLOWER",
name: "10.10.5.16"
}
],
service_states: [],
cluster_state: {
up_since: "2016-03-26 16:06:21",
progress: 100,
state: "CLUSTER_UP_HA_ACTIVE"
}
}

Avi Load Balancer ControllerCluster Configuration in AWS

The Avi Load Balancer Controller is the management and orchestration engine for Avi Load
Balancer ADC.

To provide HA and resilience, it is recommended to deploy a cluster of three Controller instances.

Once the Controller cluster is formed, the controllers synchronize the state, irrespective of the
Controller instance used to configure Avi Load Balancer features or retrieve operational data.

For more information about Controller cluster architecture, see High Availability for Avi Load
Balancer Controllers.

VMware by Broadcom 182

VMware Avi Load Balancer Administration Guide

In AWS environments, AWS Availability Zones (AZs) provide redundancy and separate fault
domains. All AWS regions support a minimum of two AZs. To leverage the HA provided by AWS
AZs, it is recommended to deploy different Controller instances of a cluster in different AZs.

Managing an Avi Load Balancer Controller Cluster across AZs

User queries Controller FQDN
Controller FQDN registered in
Route 53 with all Controller IPs
Response from DNS:
Controller 1/2/3 IP VPC

Avi Controller Cluster

Availablility Availablility Availablility

Zone 1 Zone 2 Zone 3

Avi Controller Avi Controller Avi Controller

Controller 1 IP Controller 2 IP Controller 3 IP

Each Avi Load Balancer Controller will receive an IP address from a different subnet given that an
AWS subnet does not span across AZs.

In this scenario, it is recommended to create a FQDN in AWS Route 53, and associate all three
Controller IPs with this FQDN. In addition, Route 53 health checks can be used in conjunction
with multivalue routing when the FQDN is added to a public zone. This ensures that only healthy
Controller IPs are returned.

Configure Azure Cluster IP

Avi Load Balancer Controller cluster IP configuration is integrated into the configuration workflow
and no longer requires ControlScripts.

Procedure

1 Configure MSI based authentication for the Controller virtual machine.

For more information on MSI based authentication, see Support for Managed Services
Identify (MSI) based Authentication for Microsoft Azure topic in the VMware Avi Load
BalancerInstallation Guide.

2 Configure the Controller cluster IP. To add the cluster IP within the UI, navigate to
Administration > Controller > Nodes > Edit. Enter the Name and the new address to the
Controller Cluster IP field.

VMware by Broadcom 183

VMware Avi Load Balancer Administration Guide

3 Click Save.

Note The configured cluster IP must belong to the same VNet as the Controller nodes.

4 Remove a pre-configured cluster IP. If a cluster IP was configured using ControlScripts for a
release prior to 17.2.5, follow the steps below to ensure that it is removed:

a Remove the ControlScript, if configured.

b Delete the cluster IP from the cluster IP configuration page.

c Remove the cluster IP from the Azure portal.

Changing Controller IP Addresses in a Bare-metal

Environment
Following are the steps to change the IP addresses of Controllers running in a bare-metal
(Docker) environment.

Procedure

1 Change the IP address of each Controller node within the cluster to the new IP by manually
editing the network-scripts on the host and changing the interface configuration.

2 Ensure that the new Controller IP addresses are reachable in the network from the other
Controller nodes.

3 Edit the /etc/systemd/system/avicontroller.service or /usr/sbin/avicontroller file on

each host running the Controller Docker container and overwrite the old Controller IP with the
new Controller IP wherever applicable.

The long command line below contains the Controller IP in a sample /etc/systemd/system/
avicontroller.service file.

code>ExecStartPre=/usr/bin/docker run --name=avicontroller -m 48g --cpu-period=100000

--cpu-quota=2400000 -p 5098:5098 -p 8443:8443 -p 5054:5054 -p 80:80
-p 443:443 -p 161:161/udp -d --privileged -e "CONTAINER_NAME=avicontroller"
-e "MANAGEMENT_IP=10.122.0.112" -e NUM_CPU=24 -e NUM_MEMG=48 -e
DISK_GB=30 -e CNTRL_SSH_PORT=5098 -e SYSINT_PORT=8443 -e HTTP_PORT=80 -e
HTTPS_PORT=443 -v /:/hostroot/ -v /dev:/dev -v /var/run/docker.sock:/var/run/docker.sock
-v /run/xtables.lock:/run/xtables.lock -v /opt/avi/controller/data:/vol/ avinetworks/
controller:18.2.5-5116-20190708.063533</code>

4 On each Controller host the administrator would run this trio of commands:

systemctl daemon-relaod
systemctl stop avicontroller
systemctl start avicontroller

VMware by Broadcom 184

VMware Avi Load Balancer Administration Guide

5 Corresponding to the sample in step 3, on any one of the Controller containers, the
administrator would run these commands:

cd /opt/avi/python/bin
python change_ip.py -i 10.122.0.111 -o 10.122.0.112 -o 10.122.0.113

Results

The Controller cluster comes back up with the new IPs.

OpenStack Network Configuration for Avi Load Balancer

Controller Cluster
This section explains how to configure a cluster in Avi Load Balancer for an OpenStack cloud.
To provide Avi Load Balancer Controller HA, add two additional Controller nodes to create a
three-node Controller cluster.

For more information on deploying a cluster, see Chapter 5 Avi Load Balancer Controller Cluster.

Prerequisites for Cluster Deployment

There are certain prerequisites defined for the leader and follower nodes in a cluster. For
complete information, see Prerequisites for Cluster Deployment.

From an OpenStack perspective, consider the following:

1 A Neutron port is created and is available for cluster VIP.

2 A floating IP is available for Neutron port.

Deploying an Avi Load Balancer Controller Cluster

For complete information on configuring Controller’s management interfaces and cluster IP, see
Deploying an Avi Load Balancer Controller Cluster (Additional).

The following sections are for creating OpenStack floating IP and binding that with the cluster IP:

Write Mode
1 Access OpenStack Horizon CLI.

a List the Network

openstack network list — This indicates the configured requisite networks.

root@openstack-mitaka:/root# openstack network list

+--------------------------------------+---------------
+------------------------------------------------------+
| id | name
| subnets |
+--------------------------------------+---------------
+------------------------------------------------------+
| 10a514a3-d843-499d-80fd-28274d4a4912 | webserver-

VMware by Broadcom 185

VMware Avi Load Balancer Administration Guide

net | 3ebfb2ef-9b47-44f7-9da5-5245e1d0ed53 192.168.10.0/24 |

b Create a floating IP.

openstack floating ip create provider1 — provider1 is the network used.

root@openstack-mitaka:/root# openstack floating ip create provider1

New floating IP is created.

c Get the port-id for cluster IP.

openstack port list -c ID -c 'Fixed IP Addresses'|grep 172.16.0.65

95665123-64a4-453a-abde-70fdb3d2ae2a| ip_address='172.16.0.65',
subnet_id='5785c1cf-a222-4b0a-9343-003153f37a65'

d Associate the cluster IP with the floating IP.

Using the port-id from the command above (95665123-64a4-453a-abde-70fdb3d2ae2a in

this case), associate it with the floating IP created in step b.

root@openstack-mitaka:/root# openstack floating ip set --port 95665123-64a4-453a-

abde-70fdb3d2ae2a 4ec57a12-7357-461a-80f6-d87ae7536335

VMware by Broadcom 186

VMware Avi Load Balancer Administration Guide

2 Add the cluster IP and the secondary IP for the cluster leader.

root@172-16-0-66:~# ip a
eth0: (BROADCAST,MULTICAST,UP,LOWER_UP) mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:50:56:bd:5a:0f brd ff:ff:ff:ff:ff:ff
inet 172.16.0.66/24 brd 172.16.0.255 scope global eth0
valid_lft forever preferred_lft forever
inet 172.16.0.65/32 scope global eth0:1 Cluster IP

No-Access Mode
For OpenStack No-Access cloud type, the AAP entries need to be configured manually using the
following command.

An example is shown in the code block below.

root@openstack-mitaka:/root# openstack port set --allowed--address ip-address=172.16.0.133

Controller_Port

root@openstack-mitaka:/root# openstack port set --allowed--address ip-address=172.16.0.133

d0bf0bda-02e2-46bf-abd2-0d05cc4654df
root@openstack-mitaka:/root# openstack port show d0bf0bda-02e2-46bf-abd2-0d05cc4654df
+-------------------------------
+-----------------------------------------------------------------------------------+
| Field |
Value |
+--------------------------
+----------------------------------------------------------------------------------------+
| admin_state_up |
True |
| allowed_address_pairs | {"ip_address": "172.16.0.131", "mac_address":
"fa:16:3e:47:6b:70"} |
| binding:host_id | openstack-
mitaka |
| binding:profile |
{} |
| binding:vif_details | {"port_filter":
true} |
| binding:vif_type |
bridge |
| binding:vnic_type |
normal |
| created_at |
2018-01-12T13:58:02 |
| description
| |
| device_id | 2adedfc3-75d6-4296-ad18-
bfc38873485c |

VMware by Broadcom 187

VMware Avi Load Balancer Administration Guide

| device_owner |
compute:nova |
| extra_dhcp_opts
| |
| fixed_ips | {"subnet_id": "5785c1cf-a222-4b0a-9343-003153f37a65",
"ip_address": "172.16.0.133"} |
| id | d0bf0bda-02e2-46bf-
abd2-0d05cc4654df |
| mac_address |
fa:16:3e:47:6b:70 |
| name
| |
| network_id | dd9dab27-9228-4765-96f2-
d56194136ba0 |
| port_security_enabled |
True |
| security_groups | 3cc1092e-538c-4ff7-b4ac-
eeff84731f75 |
| status |
ACTIVE |
| tenant_id |
904fb201a92f443297bffca3b354d52d |
| updated_at |
2018-01-12T14:19:06 |
+--------------------------
+----------------------------------------------------------------------------------------+

Create the neutron port for the VIP by using the following command.

openstack port create --network "neutron_network_name" --

allowed-address mac-address="fa:16:3e:52:81:03",ip-address="172.16.0.63" --allowed-
address mac-address="fa:16:3e:52:81:04",ip-address="172.16.0.64" --allowed-address mac-
address="fa:16:3e:52:81:06",ip-address="172.16.0.66" --fixed-ip ip-address="172.16.0.65" --
project "904fb201a92f443297bffca3b354d52d"

The following is an example.

openstack port create --network "neutron_network_name" --

allowed-address mac-address="controller_mac1",ip-address="controller_ip1" --allowed-
address mac-address="controller_mac2",ip-address="controller_ip2" --allowed-address mac-
address="controller_mac3",ip-address="controller_ip3" --fixed-ip ip-address="cluster_ip" --
project "project-id"

Note When the leader Controller fails (or reboots), a follower Controller will take over the
cluster IP (in this case, 172.16.0.65), and the mapping between floating IP (10.130.170.86) and
cluster IP (172.16.0.65) will not change. Therefore without intervention, the floating IP and cluster
IP association will work as expected.

VMware by Broadcom 188

VMware Avi Load Balancer Administration Guide

FAQs on Controller Cluster

This topic answers basic questions about Avi Load Balancer Controller Clusters.

n How many nodes are supported in an Avi Load Balancer Controller cluster?

Avi Load Balancer Controller clusters can include either 1 (standalone mode) or 3 Controller
nodes in a cluster.

n How many nodes are needed for the Avi Load Balancer Controller cluster to be operational?

A three-node cluster requires a quorum (majority) of the nodes to be present for the cluster
to be operational. It means at least two nodes must be up.

n Can you explain how the three nodes in an Avi Load Balancer Controller are used during
normal operation?

Among the three nodes, a leader node is elected and orchestrates the process startup or
shutdown across all the active members of the cluster. Configuration and metrics databases
are configured to be active on the leader node and placed in standby mode on the follower
nodes.

Streaming replication is configured to synchronize the active database to the standby

databases. Analytics functionality is shared among all the nodes of the cluster. A given virtual
service streams the logs and metrics to one of the nodes of the cluster.

n What happens if a follower node goes down?

This node is removed from the active member list. The work that was performed on this node
is re-distributed among the remaining nodes in the Avi Load Balancer Controller cluster.

n What happens if the leader node goes down?

One of the follower nodes is promoted as a leader. This triggers a warm restart of the
processes among the remaining nodes in the cluster. The warm standby is required to
promote the standby configuration and metrics databases from standby to active on the
new leader.

Note During warm restart, the Controller REST API is unavailable for a period of 2-3 minutes.

n What happens if two nodes go down?

The entire cluster becomes non-operational until at least one of the nodes up. A quorum (two
out of three) of the nodes must be up for the cluster to be operational.

n Will there be a disruption to traffic during cluster convergence (single or multiple nodes go
down and come back)?

When single or multiple nodes go down and come back, the cluster is non-operational, the
SEs continue to forward traffic for the configured virtual services. This is referred to as
headless mode.

VMware by Broadcom 189

VMware Avi Load Balancer Administration Guide

The analytics (logs and metrics) are buffered on the SEs. When the Controller cluster is
once again operational, the cluster re-synchronizes with the SEs and initiates the collection of
metrics and logs. Data plane traffic continues to flow normally throughout this time.

n Recover a non-operational cluster where two of the three nodes are permanently down and
not recoverable

For detailed information, see Recover a Non-Operational Controller Cluster.

n Recover the system if all three Avi Load Balancer Controller nodes are permanently down
and not recoverable.

For detailed information, see Backup and Restore of Avi Load Balancer Configuration.

n Re-configure the Controller Cluster membership.

For detailed information, see Changing Avi Load Balancer Controller Cluster Configuration.

n Can the Avi Load Balancer Controller nodes in a cluster have different resource allocations
(memory, CPU, and drive space)?

It is recommended to allocate these same resources to each of the three nodes within the
Controller cluster. In course of time, if there is a need to re-size (change resource allocations)
the Avi Load Balancer Controller in the cluster, to accommodate growth, change resource
allocations on one Avi Load Balancer Controller node at a time. However, it is expected that
all nodes within the cluster will eventually be resized to the same allocations.

n Can the Avi Load Balancer Controllers in a cluster be in different subnets?

Yes. This configuration is supported and can even be preferred for certain topologies for
improved fault tolerance. However, a limitation with placing the Controllers in separate
subnets is that the cluster IP address is not supported. The cluster IP address requires all
Controller nodes to be in the same subnet.

n Can the cluster leader be changed manually?

No. Currently, this is not a supported operation. The leader is chosen during initial
deployment of the cluster or when recovering a fully down cluster. However, the leader
cannot be manually changed while the cluster is operational.

Avi Load Balancer Controller Controllers participate in election of their leader, and
automatically decides who the new leader must be in case of failure.

n What timers are used during cluster failover? Are the cluster failover timers configurable?

Leader failure

VMware by Broadcom 190

VMware Avi Load Balancer Administration Guide

If the leader Controller of the cluster fails, this triggers an internal warm restart of the
Controller processes. Typically, this takes around 2-3 minutes after it is detected that the
leader has failed.

Graceful reboot of leader

Failure detection is almost instantaneous in cases of a graceful reboot.

Ungraceful power-off of leader

In the case of an ungraceful power-off of the leader, it can take up to 30 seconds to detect
that the leader has failed.

These timers are tuned based on testing but are not configurable.

n Can configuration changes be made directly on follower Controllers?

Yes. Nearly any configuration changes can be made on any of the Controller nodes.

Exceptions:

Configuration of the cluster itself (node IP addresses and cluster address).

Upgrade, which is performed only on the leader node.

n What is the recommended cluster deployment option for multiple data centers in different
regions?

Avi Load Balancer recommends deploying the Controller Cluster only within a single region.
However, the member nodes are deployed across multiple availability zones within the
region. If there are multiple regions, it is recommended to deploy one Controller Cluster
per region. (For more information, see Clustering Avi Load Balancer Controllers of Different
Networks).

n How to import and export Avi Load Balancer metrics database?

Prior to upgrading to version 22.1.2, it is recommended to export the metrics database. To

export the metrics database, copy the /var/lib/postgresql/10/pg_metrics/ directory
and store it outside the Controller VM. In case you need to roll back to a previous release, the
exported metrics can be imported. Thus, preventing the loss of data.

To import the metrics database,

n Stop the process supervisor by running the following command first on the follower
nodes and then on the leader.

systemctl stop process-supervisor

n Move the copied pg_metrics directory to /var/lib/postgresql/10/pg_metrics/ on

the leader.

VMware by Broadcom 191

VMware Avi Load Balancer Administration Guide

n Run the following to first start the process supervisor on leader and then on follower so
that the imported pg_metrics data is replicated from leader to the followers.

python3 /opt/avi/scripts/pg_rec_fix_cluster.py --nodelist leader_ip follower_1_ip

follower_2_ip

Note It will take some time for the cluster to become HA_ACTIVE based on the metrics
database size.

FAQs on Database Replication

This section discusses the FAQs and resolutions on Database Replication.

What are the observations leader node disconnects from the Avi
Load Balancer cluster?
Following are the observations when a leader node disconnects from the Avi Load Balancer
cluster:

1 The leader node restarts and breaks the Avi Load Balancer cluster due to a lost quorum.

2 One of the follower nodes posts a message that the DB replication is not completed and it
cannot become a leader due to that.

3 Another follower node will post DB replication from the leader and becomes the
leader node.

What will happen when none of the follower is not able to complete
database replication in an Avi Load Balancer cluster?
1 On the follower node, the following process is used to replicate the database:

Check if streaming replication of the database can be enabled. In general, it is expected to

succeed as Avi Load Balancer allows for a reasonable number of WAL record files to satisfy
this. If this fails or if it is the first time a follower node is added to the cluster, the follower
node does a full backup, which usually takes time.

2 Once the streaming replication is set up, Avi Load Balancer monitors that the replication
continues to occur every minute. If it detects a failure five consecutive times, then it declares
that the replication has failed and attempt a full replication.

3 On trying to do a full replication, Avi Load Balancer always does a full copy of the entire
database data directory into the next directory. Only when it is complete it moves this to
the current directory for the database. If there is a failure as a part of the full copy of the
database files, then it will continue to have a valid, current directory.

VMware by Broadcom 192

VMware Avi Load Balancer Administration Guide

4 On deciding to do a full backup, it is essential to record this to ensure that this node does
not become a leader if there is a leader failure during this time. With this scheme, a follower
node must always be in sync and can take over as a leader. If the full sync fails, the safety
mechanism will prevent the node from taking over as leader, but with manual intervention,
you can promote one of the nodes as the leader.

Does the cluster_mgr logs show the sync up every minute once the
full replication attempts?
Logs for the full replication process can be found in /var/lib/avi/log/
postgres_service_main.log and postgres_service_metrics.log.

What are the common causes of replication failure?

Permanent connectivity issues can cause replication failures. Streaming replication always
succeeds when the postgres on the follower node can recover from its checkpoint and is behind
the leader. In some instances, when the postgres is not brought down gracefully, it may not be
able to replicate from leader. This will force us to do a full backup from the leader.

Which directory is the ‘current’ directory and which directory is the

‘next’ directory during the database replication process?
For config database: next is /var/lib/postgresql/9.3/main_backup and curl is /var/lib/
postgresql/9.3/main/.

For metrics database: next is /var/lib/postgresql/9.3/pg_metrics/metrics_backup

and /var/lib/postgresql/9.3/pg_metrics/metrics.

What are the symptoms of replication failures? Are they only

noticeable during leader election? Are internal events generated for
such events?
DB REPLICATION FAILED events are observed on the Avi Load Balancer, when it detects
replication failures.

Is there any relation to cluster convergence? If so, what are the

recent improvements on this in detail?
When Avi Load Balancer detects replication failure, a full backup is triggered. As a part of this,
it touches a file in /var/lib/avi/etc to indicate that the replication has not been completed.
So, the specific node does not participate in becoming a leader if there is a convergence. Also,
the state for this service goes to COPYING_DB_FROM_LEADER, which will move the node state
from CLUSTER_ACTIVE to CLUSTER_STARTING. The overall cluster state is not observed as
CLUSTER_HA_ACTIVE.

VMware by Broadcom 193

VMware Avi Load Balancer Administration Guide

Does the recent versions of Avi Load Balancer still use zookeeper ?
If not, why does it still run on Avi Load Balancer and what has
changed?
The recent versions of the Avi Load Balancer do not use zookeeper. The zookeeper process
is run for backward compatibility. When Avi Load Balancer is upgraded from previous versions
to 17.2, the Controllers will have to publish the leadership information in ZK for the SEs that
are not yet in 17.2. Once the upgrade is complete, SE will use the new scheme for leadership
notifications. At some point, ZK will be removed when all upgrades will be from 17.2.x.

VMware by Broadcom 194

Tenant Settings
6
A tenant is an isolated instance of Avi Load Balancer. Each Avi Load Balancer user account is
associated with one or more tenants.

The tenant associated with a user account defines the resources that the user can access within
the Avi Load Balancer. When a user logs in, the Avi Load Balancer restricts access to only those
resources that are in the same tenant. If a user is associated with multiple tenants, each tenant
still isolates the resources that belong to that tenant from the resources in other tenants. To
access resources in another tenant, the user must switch the focus of the management session
to the other tenant.

Note
n For information on switching a management session from one tenant to another, see
Switching Between Tenants.

n For information on creating a cloud inside a tenant, see Tenant-Scoped Clouds.

n Certificates from the admin tenant can be shared by non-admin tenants. For more
information, see Sharing Certificates Across Tenants.

Default Tenant
By default, all resources belong to a single, global tenant, namely, admin. The admin tenant
contains all Avi Load Balancer resources.

The default admin user account belongs to the admin tenant and so, can access all resources.

If no additional tenants are created, all new Avi Load Balancer user accounts are automatically
added to the admin tenant.

Tenant-to-Role Mapping
Within each user account, the role selected for the user is mapped to a tenant. If only one tenant
is defined (the default admin tenant), this tenant is automatically mapped to the role selected
for the user. It allows the user to access all resources to the extent (write, read, or no access)
allowed by their role.

VMware by Broadcom 195

VMware Avi Load Balancer Administration Guide

Creating additional tenants allows a user account to have multiple roles. In this case, each role
can be mapped individually to a tenant within the user account. Optionally, a single role can be
mapped to all tenants.

If a single role is mapped to all tenants, the user can switch the management session to other
tenants as needed.

All Tenants View for Super Users

The Avi Load Balancer user accounts that are enabled for superuser access automatically have
access to a special read-only tenant: All Tenants. The All Tenants view provides read-only access
to all resources within the Avi Load Balancer.

The All Tenants View tenant cannot be mapped to any roles within a user account. The All
Tenants tenant is automatically made available to all superuser accounts.

Read the following topics next:

n Create a Tenant

n Add an Existing User to a Tenant

n Enable Use Single Role for All Tenants

n Configuring Tenant Settings

n Switching Between Tenants

n Sharing Certificates Across Tenants

n Tenant-Scoped Clouds

n All Tenants View

n Tenants Versus SE Group Isolation

n Sharing Admin Profiles Across Tenants

Create a Tenant
This task helps you to create a tenant.

Procedure

1 Navigate to Administration > Accounts > Tenants, and click Create.

2 Enter a name for the new tenant.

3 Enter an optional description.

4 By default, Tenant Access to Provider Service Engine is selected.

VMware by Broadcom 196

VMware Avi Load Balancer Administration Guide

5 Select Tenant VRF if required.

When Per Tenant IP Domain is selected, each tenant gets its own routing domain that is
not shared with any other tenant. When Share IP Domain across all tenants is selected, all
tenants share the same routing domain.

6 Click Save.

The new tenant appears on the tenant table.

Note The admin account is automatically added to each new tenant.

Add an Existing User to a Tenant

This task helps you to add an existing user to a tenant.

Procedure

1 Navigate to Administration > Accounts > Users.

2 Click the edit icon next to the user name.

3 Under Tenant & Role, click Add and select the role from the Role drop-down menu.

4 Click Add and select the new tenant.

a Click Create if the new tenant does not exist.

A new set of mapping fields appears.

5 Select the mapping that is no longer needed and click REMOVE.

6 Click Save.

Results

The tenants table reappears, displaying the change.

Enable Use Single Role for All Tenants

This option allows the user account to access any tenant from the same role. In this case, the
default tenant must be selected. The default tenant is the one into which the user is placed after
logging in.

Configuring Tenant Settings

A tenant is a logical grouping of configuration. An administrator might belong to one or multiple
tenants, but can only view and edit one tenant at a time. Each tenant has its own isolated control
plane configuration.

Each tenant can optionally have its own isolated data plane. This will depend on the global
configuration of the Avi Load Balancer deployment.

VMware by Broadcom 197

VMware Avi Load Balancer Administration Guide

Tenants can be deployed within a Provider Context or a Tenant Context:

n Provider Context mode: SE groups are shared across tenants

n Tenant Context mode: SE groups are exclusive to each tenant

In Avi Load Balancer, Virtual Routing Forwarding (VRF) contexts can be created to isolate traffic
within a system.

Using the UI
This task helps you to configure the tenant settings using UI.

Procedure

1 Navigate to Administration > System Settings > EDIT > Tenancy Mode.

2 Select IP Route Domain as Per Tenant or Share Across Tenants as required.

3 Select the context mode as applicable:

n Service Engines are managed within the tenant context, not shared across tenants to
enable the Tenant Context mode.

n Service Engines are managed within the provider context, shared across tenants to
enable the Provider Context mode.

4 Select Read or No Access as required in the Tenant Service Engine Access field.

5 Click Save.

Note The configuration made using the UI will remain as the global configuration when
creating a tenant. These values can be overridden for a specific tenant using the CLI or API.

Using the CLI

VMware by Broadcom 198

VMware Avi Load Balancer Administration Guide

Field Value Description

se_in_provider_context True SEs are managed within the provider

context, shared across tenants

se_in_provider_context False SEs are managed within the tenant

context, not shared across tenants

tenant_access_to_provider_se True Tenant has read access to SEs

tenant_access_to_provider_se False Tenant has no access to SEs

tenant_vrf True Configure IP route domain per tenant.

The system creates a unique VRF for
each tenant that is only usable within
that tenant

tenant_vrf False Share IP route domain across tenants.

VRFs are defined in the admin tenant
and are usable by any tenant

Note In the global configuration, se_in_provider_context is set to True. However, this can be
changed for a tenant.

Switching Between Tenants

User accounts that have access to multiple tenants are placed into a single, default tenant after
log in.

Procedure

1 In the UI, navigate to Applications > Dashboard.

VMware by Broadcom 199

VMware Avi Load Balancer Administration Guide

2 Click the username in the upper right corner of the display, and select All Tenants from the
drop-down menu.

The resources of newly selected tenant are now accessible through the web interface instead
of the previously selected tenant.

Results

Note If you are logged in with a superuser account, a read-only All Tenants View tenant is also
available.

Note When you switch the tenant, the corresponding Application dashboard opens.

Sharing Certificates Across Tenants

In Avi Load Balancer, certificates from the admin tenant can be shared by non-admin tenants
when the shared_ssl_certificates flag is set to True in the Controller.

Default Behavior
System default certificates are used by objects in any tenant. For example, these include
System-Default-Cert, System-Default-Cert-EC, System-Default-Portal-Cert, System-Default-
Portal-Cert-EC256, System-Default-Root-CA, and System-Default-Secure-Channel-Cert, a set
of objects that can be expected to expand over time. Objects created in a specific tenant
(including the admin tenant) can only be viewed and used in their respective tenant. Certificates
are automatically chained and will only be chained to certificates in the respective tenant.

Shared SSL Certificates

In Avi Load Balancer, the shared_ssl_certificates are added to the Controller Properties
object. By default, this is set to False. If shared_ssl_certificates is set to True, the following
behavior applies:

n All certificates from the admin tenant are viewed from non-admin tenants.

n Certificates from the admin tenant can be used in non-admin objects, that is, virtual services,
pools, and so on.

n Application certificates in non-admin tenants will be chained to issuer certificates in the admin
tenant.

n Avi Load Balancer will not chain certificates from the admin tenant to issuer certificates in
non-admin tenants. As a result, if there is an Intermediate certificate in the admin tenant and
the corresponding CA certificate is in the non-admin tenant, these objects will not be linked.

n If there are any cross-tenant links (that is, an Intermediate certificate in the admin tenant and
Application certificate in the non-admin tenant), the Avi Load Balancer will prevent changing
the shared_ssl_certificates flag.

VMware by Broadcom 200

VMware Avi Load Balancer Administration Guide

n For unchained Application certificate in a non-admin tenant and the corresponding

Intermediate certificate in the admin tenant, the user toggles the shared_ssl_certificates
flag from False to True, and the Intermediate and Application certificates will not be chained.
If you want these certificates to be chained, delete and recreate the application certificate.

n You can configure this feature using Avi Load Balancer REST API or CLI. This feature is
currently not supported on the Avi Load Balancer UI.

Note
n When certificate sharing is enabled in Avi Load Balancer before version 21.1.4, the certificate
with the most days to expiry is always selected.

n When certificate sharing is enabled in Avi Load Balancer version 21.1.4, the Intermediate or
CA certificate with the highest expiry in the current tenancy is always selected. If the current
tenant has no Intermediate or CA, the corresponding Intermediate or CA from the admin
tenant (if any) is selected.

Usage Guidelines
The following guidelines are applicable as the certificates in the admin tenant can be chained to
any certificate in the system:

n Toggle the shared_ssl_certificates flag to True and create shared Intermediate or Root
certificates in the admin tenant before creating Application certificates.

n Application certificates must be in the tenant with the corresponding application.

n Although certificate additions or updates in the admin tenant are CPU-intensive, these actions
must have minimal impact, as they are infrequent operations.

CLI Configuration
[admin:10-10-28-16]: > configure controller
properties

Updating an existing object. Currently, the object is:

+--------------------------------------------+--------------------+
| Field | Value |
+--------------------------------------------+--------------------+
| uuid | global |
| unresponsive_se_reboot | 300 sec |
| crashed_se_reboot | 900 sec |
| se_offline_del | 172000 sec |
| vs_se_create_fail | 1500 sec |
| vs_se_vnic_fail | 300 sec |
| vs_se_bootup_fail | 480 sec |
| se_vnic_cooldown | 120 sec |
| vs_se_vnic_ip_fail | 120 sec |
| fatal_error_lease_time | 120 sec |
| upgrade_lease_time | 360 sec |
| query_host_fail | 180 sec |

VMware by Broadcom 201

VMware Avi Load Balancer Administration Guide

| vnic_op_fail_time | 180 sec |

VMware by Broadcom 202

VMware Avi Load Balancer Administration Guide

| unresponsive_se_reboot | 300 sec |

VMware by Broadcom 203

VMware Avi Load Balancer Administration Guide

[admin:10-10-28-16]: sslkeyandcertificate> certificate

[admin:10-10-28-16]: sslkeyandcertificate:certificate> certificate --
-----BEGIN
CERTIFICATE-----

[280/18075]
MIIFZzCCA0+gAwIBAgICEAIwDQYJKoZIhvcNAQELBQAwPzELMAkGA1UEBhMCVVMx
CzAJBgNVBAgMAkNBMQwwCgYDVQQKDANBdmkxFTATBgNVBAMMDEludGVybWVkaWF0
ZTAeFw0xNzEyMjAyMzM0MzVaFw0zNzEyMTUyMzM0MzVaMEkxCzAJBgNVBAYTAlVT
MQswCQYDVQQIDAJDQTEMMAoGA1UECgwDQXZpMR8wHQYDVQQDDBZTYW1lLU5hbWUt
SW50ZXJtZWRpYXRlMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAy0kq
S48Ngxg1KJ1hmwMxbSEJnGuz0bfxf/FbcVK0OQZzOfl7K1nrg8CIjLyywEkgzBqf
/b1GwEwNRNvCxAgIP78kCw39chdGzW2jRcjiWPV6OrOizrkXHKlhCJ7LnONSeQH1
rGehFSzpLT8g6KY+DCkeVQBVscV4cFJFTL484EoOhgxMuqj0jij3T+GctqsK5p2Y
VCy71ZEbJvvET3x6/rDNIJU9njJxCvlJyk3T78sTSsW7+xjhCRVsvBAHyUhUGWuC
9ol6EcJdOBUVUKIJX8t+qT1iGtMEd2oV0rUv+2cvHJrhZW24BSVnebW05n32z9Je
oPcHgdrH0ZJN9O0DV46QP1HTdVe7GvY1Fd+UjUFh4oIjwQyYSpO/smBHUffCmtyX
wljCbmjYM2yKyQe04C/+s8ZO+AFFtqx6srvnElQTXtfxkTWYPSrodDKmxqY81aR9
TFd5wWtApMeFT9DK5dDlneBpqn0gDE+JixlEx+pEZM6SDdO1arAg3PKZotuzndo0
1c0mqG6Lp5r464xi5g4kbPHNe1PFe+2tDCEW9BuYADe0v8PvpHMbGJNxOt+w8CcV
R/muH/KoKYs8Y9Ej03MRob1r7Xpv4/NO/1KLHhggxlihiUib1GVDguRNJmMYloo+
8FfoSMixPRJxUg03yZA479e4QSNI+5AryxzohXUCAwEAAaNjMGEwHQYDVR0OBBYE
FEamhG8kGg3PCElsgH8XYIWO04BXMB8GA1UdIwQYMBaAFEqs0+NaumvRXZkP+sTw
NMNvbr61MA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3
DQEBCwUAA4ICAQCVeQKhOIDK0z8XdohL/vkypGGayBtU17lFfwZEiWuIeeLnZDrB
vzz1T1j91tx6MBWFEbP2FoJYCwaU9YSuOP8mhtmJM4v1MgC3aOGdMa3nKo2PbS9M
ECMLFB6Jpo7zVjVxwEz7WXA7/YJgR0g5ft/turJnbbUis0K0FBO/aYzc9gyBvg8I
GTB6GX6DDNuwT5EOjkynT3SqnRrnD2piZ0oQ2IIDMaYm/r/DFaMLoU6GRLmj74N0
P3Lefks4JX5C2KKEuM3/6/udMlmNrObjkIACe34icImkdxSXjmKj8Mg4YG8PBRU1
/j1yizB6GokGq2//0BkRMzBLJfUifOVa9mH/C303kA/CvJ42nQyDPLU77nunng3f
T//+/dQYk+OuMTTuVul2WSef+wW+kEspE8uTo/GH1ZmMRV0T7aPxt8/ASDbhEcQM
Okhbo49AhxuTHlOWS3xKxVIbxJ4P/P0v8c5bb/4D5gdGgBCoXQptiBRtS2suBt1M
g0eCtusMuUqPkwB5o5IU2MPGbHiiPzB4up5ZJHYe97rtKduM1cD+0v+w7ZxDrqdD
ebfAJjqaZLKNWEmy5fYt0lWUgDsA8aWUSLN2j/R3BbtXHcClmsZap3CSFzJlhbPz
9tQBVsfx6UJYZR2eAXTpEtEMYous6tKHcRS04/mCPBq+WhoYG39aX85g2Q==
-----END CERTIFICATE-----
END
[admin:10-10-28-16]: sslkeyandcertificate:certificate> save
[admin:10-10-28-16]: sslkeyandcertificate> save
+------------------------
+------------------------------------------------------------------------------+
| Field |
Value |
+------------------------
+------------------------------------------------------------------------------+
| uuid | sslkeyandcertificate-2348ba24-1a56-4e9d-9833-
c8c3c1158714 |
| name | admin-
intermediate |

VMware by Broadcom 204

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 205

VMware Avi Load Balancer Administration Guide

[t1:10-10-28-16]: sslkeyandcertificate> key --

VMware by Broadcom 206

VMware Avi Load Balancer Administration Guide

jXs64nmmO/kGyGAXduIEoA4POMj7OIZUn8FlSvn0U5gUDUHlfuewuCzfRE0D8+x0
O1n06vhD4II59Z13T7IOAywvdio5p0ZBOVnxFNJRJ1oizqHIywgGKOOnqj59iBr9
pw/LTthM1mozfUxVYxftSwDr91C7PTaYDE9prmw8wH1TL6I4skxKRVmagFwY0rtr
ViNhNigPjUB3xlEtv6RuwFeEmfcZMzkLCAoXbg1yv6Av5tGJwdCVwDwrpP6I/FHz
PwQdFmZRGZJI8QqdEcWYI/ewXYevCfDrQIWH+gFVIQKBgQDrcCmclzSqQt4xczJ2
ajXAaxnxLSJC/WYOIsIp3L5b/gqs+SUAIJXoVZMinOcygtJs3J4f0Zuy9NkddNn9
JVeMXs7rr7quXKSzX0100acB1NR4Sfq1RWboOxoiSgrUSx8D/ooaJE0JSlj0DtHl
+FVlSECAK2wpM8dFEMf9cAEIeQKBgQDOoRlQzkdnoDVL+gyIXnsA3ArnXDcig1x1
tSj0VqCEaGHhjngYHsmissaIw9ABlwZkt9maylX9PrLaAceGXPzeBvlK0PcgImZ+
2hYVp00znj4//JOsFe9joruKfaXrTLPvY8N0jYAmip6FJJ1eq4x8rL8gU/NdlMQf
5diVimhizQKBgCGs82bAgfnwgpOUJJ2nZ3TUXOuQRxxJ3nUbJ6aROnEyDxjash4o
iwimZNtIkhE5gRutGrj2ZEzelMeP1TZORw1+6h3wDsWt3qkBcrTI4Bh09scV3dRb
zvJcscpByPbAn/kUSXCfzJ0Nk1elXwSD1sMb6I3sqBXkoBYS5mgrwxoRAoGAXJmB
uN7YzS3U9LmYiDyfLyFtmYWQB92KwA1xzx5LTUtiIi0w0M5rWoh3xK7MNwoxiU2D
LYVjx9wjVuPZQPPHNtE1Qzwmo7YG7O5bW1TgmjNeflp463PhFmvFVCk/BBYZxTyW
SVNojN0ucUiZZeXHTdA0zw4QUG3s/saIq2udoDkCgYBS9FJxYZV/3eWZTV7E8RHO
4ABpujonzZcrxB/pIlQJhehVABopbMAGE0aGc7gGacu0DKsLNYL8Wkdqgs6WN9Yo
erlGXlJelgs4CSlZulInntFgdqC9Rj0sHjx6gCVEgg1lGkB++YrCLj2YuYN7L9JW
wk/YYUmjGLjqcHvBNDl0Gw==
-----END PRIVATE KEY-----
END
[t1:10-10-28-16]: sslkeyandcertificate> certificate
[t1:10-10-28-16]: sslkeyandcertificate:certificate> certificate --
-----BEGIN CERTIFICATE-----
MIIFAzCCAuugAwIBAgICEAEwDQYJKoZIhvcNAQELBQAwSTELMAkGA1UEBhMCVVMx
CzAJBgNVBAgMAkNBMQwwCgYDVQQKDANBdmkxHzAdBgNVBAMMFlNhbWUtTmFtZS1J
bnRlcm1lZGlhdGUwHhcNMTcxMjIwMjMzNDU2WhcNMzcxMjE1MjMzNDU2WjA3MQsw
CQYDVQQGEwJVUzELMAkGA1UECAwCQ0ExDDAKBgNVBAoMA0F2aTENMAsGA1UEAwwE
QXBwMTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAL4Iak5x+rG5S+Ww
JrX6JhS7sTKu8k5sNJ2ishuNu7mpemovuVJwZCrq6FsoowAYz8kiIjvmE4B4c32F
hr/S/wGd1X8AnXMakrcrlqG5V+u2w0geTqrRJLDdi0Hrf/yeHBSLP9kGC2F8In96
ugbtEESHy46k+Fcsl/zyjVQ4XkVxusWxbmq9dAcxFIorYj3AWRJKA0wKg4Y4HtZf
7poMbRkBm316hCQ9A07HD10Rq+o5kWqQdNKTJ9bIe7DrTl2ZGzYLPwhkyGERm+ot
Sl7ocwWinHBPO3/k3JbQUklbUrzVZLjppv+04g4tuSA1X3AtYw14q8ARbNg841JA
PME6GuUCAwEAAaOCAQUwggEBMAkGA1UdEwQCMAAwEQYJYIZIAYb4QgEBBAQDAgZA
MDMGCWCGSAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBTZXJ2ZXIgQ2VydGlm
aWNhdGUwHQYDVR0OBBYEFBFU4BzZ35LC8gZlhXQG6pB9sDxNMGgGA1UdIwRhMF+A
FEamhG8kGg3PCElsgH8XYIWO04BXoUOkQTA/MQswCQYDVQQGEwJVUzELMAkGA1UE
CAwCQ0ExDDAKBgNVBAoMA0F2aTEVMBMGA1UEAwwMSW50ZXJtZWRpYXRlggIQAjAO
BgNVHQ8BAf8EBAMCBaAwEwYDVR0lBAwwCgYIKwYBBQUHAwEwDQYJKoZIhvcNAQEL
BQADggIBAAfp7d3STNGOQvPxMb+w9b4MjxXAdcFLCLiowcnh6wRS5/ALIjr+7oAt
5T+SzFx1jiZltRf7wk5Ot48+lKwSJ93oaqow82QAZZFeNvkYecL/HHqW7squC7Su
lmdxQ0DT/fkpedKu3koWjUvf90zb/LotdKN9GN4R2KwKY+p/73w1cDMxyqyPiSOH
dCX1fkG1du4HEujG+zVTlEO5Wc94zer4+C9g/QwTVkBH11MOLd9RSlStadYzy8Qs
wu1pPEXZbePA7urZGqgiYUTYKbW+Ck/EKqt8NxvyqvmYBqmfuEnOW1W7XH7Zlzli
dAFEfZ5U9we1YlduDT7KUHizBn8Uex1O1TCjn2XMt+5KhJ8yfNjqbwTyg7G1pHcG
ifl+u/PYTyrLwnf0s09/iw27oacSczDxB/yRe5W6wmhsgL0Rry1tZvAcIHPR2c5t
xstiAJVZVp+WSqJRbCR+KZYZS7IX3J09gtZy8ZDaEhCGtiE/liin4yxLEP4cgbCd
ctIdYP+3pYFC7Ij4BvT+cHtKFAIQ8gD3pSx+NHjX/cWnhjQIo4ljt+ash9YQz+70
hbsp3zDB+Qbnc6j1MuITHQneKKxVPBkvYK7bcqKmKRfjOIpFgtClWd9+YRBriBKo
CayuZ7LuJYYgVqnU6waCJaA9eZC/BSNUqqHzBYV49oBUpyDIWOTW
-----END CERTIFICATE-----
END
[t1:10-10-28-16]: sslkeyandcertificate:certificate> save
[t1:10-10-28-16]: sslkeyandcertificate> save

VMware by Broadcom 207

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 208

VMware Avi Load Balancer Administration Guide

Tenant-Scoped Clouds
For additional configuration flexibility, isolation, and delegation of ADC administration, the Avi
Load Balancer supports the creation of a cloud inside a tenant.

The Tenant-scoped clouds feature is currently restricted to AWS, Azure, Linux Server Clouds,
NSX-T, and no-orchestrator clouds. Attempts to choose other cloud types result in an error code.

For more information on no-orchestrator (no-access clouds), in which adding, removing, or

modifying properties of an SE requires an administrator’s manual intervention, see Orchestrator
Access Modes topic in the VMware Avi Load BalancerInstallation Guide.

Features
Configuration Flexibility

VMware by Broadcom 209

VMware Avi Load Balancer Administration Guide

Users with Tenant-Admin role access can create/read/update/delete (CRUD) cloud

infrastructure (VRF contexts, SE groups, and SEs) within their tenant (the one to which the
cloud is scoped). In addition, tenant administrators can access and use the infrastructure of
clouds in the admin tenant, as before, based on tenant settings. If the infrastructure in an
admin-tenant cloud is configured in provider mode, tenant administrators can use it. It is used
in addition to, and independent of any tenant-scoped clouds under their direct control.

n Provider Mode — It is a configuration in which SEs are shared across tenants. In provider
mode, all the network resources of the cloud remain in the admin tenant and cannot
be moved. To configure VRF contexts and move port groups into them, the Avi Load
Balancer user must have write privileges for the admin tenant.

Isolation

The tenant-scoped cloud and its associated objects (VRF context, SEs, SE groups) are only
visible in the cloud’s tenant. They are not visible in other tenants or the admin tenant.

Administration

The admin user is authorized and responsible for performing the highest-level configuration
operations, such as user creation and authorization, tenant creation, and all infrastructure
associated with the admin tenant. However, the admin user can and likely prefers to delegate
the creation of clouds scoped to their respective tenants to tenant administrators.

When creating a tenant-scoped cloud, Avi Load Balancer automatically:

1 Creates two VRF contexts for the cloud in the particular tenant. One is assigned data-plane
traffic and the other is control-plane (management) traffic.

2 Creates the first SE group for the cloud in this tenant (it will be named Default-Group).

3 Changes default permissions to enable the user having the Tenant-Admin role for creating
and managing tenant-scoped clouds and associated objects.

Avi Load Balancer UI Tenant-Scoped Cloud Creation

n Log in to the admin user, click admin on the top right of the toolbar to show the complete
list of tenants defined and select a tenant from the All Tenants drop-down menu. The
administrator's intent is to switch the context to that tenant.

n When a new tenant is selected, the name of the selected tenant is displayed in place of
admin on the top right of the toolbar.

Note The admin user is still logged in, and only the tenant context is changed.

n After navigating to Infrastructure > Clouds, click the CREATE drop-down menu. Select the
desired cloud the user intends to create.

Note It is not necessary for the admin user to create the cloud within the tenant.

n Cloud creation can proceed as normal.

VMware by Broadcom 210

VMware Avi Load Balancer Administration Guide

n Typically, the tenant administrator of the selected tenant is logged in. No other tenants
are listed when the selected tenant is clicked in the top right corner of the toolbar. It is
preferrable to be limited to a single tenant.

n The user can also opt to create other clouds with the selected tenant.

Note The steps to create a cloud are same as for clouds that are not a tenant-scoped. It
depends on the tenant context of the user.

All Tenants View

The All Tenants view provides a global view of all Avi Load Balancer objects across all tenants.
This includes all the tenants, including the admin tenant.

The All Tenants view is intended for viewing, updating, and deleting objects. New objects cannot
be created in this view.

Note The All Tenants view is supported only for administrative user accounts configured to be
a superuser. To use the feature, you must log in with an Avi Load Balancer user account that has
the role System-Admin and the superuser attribute enabled.

To switch the focus of the management session to the All Tenants view, use any of the following
methods.

Using the UI
This task helps you to switch the focus of the management to the All tenants View using UI.

Procedure

1 Click the username in the upper right corner of the display.

2 Select All Tenants from the drop-down menu.

Using the CLI

This task helps you to switch the focus of the management to the All tenants View using CLI.

Enter the following command.

switchto tenant *
Switching to tenant *

Specifying /* (wildcard) instead of a tenant name gives access to objects across all tenants.

Using the API

This task helps you to switch the focus of the management to the All tenants View using API.

Set the value of X-Avi-Tenant in the HTTP request header to /* (wildcard).

VMware by Broadcom 211

VMware Avi Load Balancer Administration Guide

Tenants Versus SE Group Isolation

There are multiple ways to create isolation within the Avi Load Balancer. This section discusses
the difference between tenants and SE groups and their relationship to data plane isolation and
control plane isolation.

Isolation Tenant: Provider Context Tenant: Tenant Context SE Group

Control Plane Yes Yes No

Data Plane No Yes Yes

Service Engine Groups

SE groups are an inherent method of grouping SEs to provide data plane isolation. A single
tenant can have one or more SE groups. Multiple tenants can also point to one or more SE
Groups. Only one SE group can serve a virtual service. If one of its SEs fails, another SE can take
over within the same SE group. SEs in other SE groups cannot be pulled in to provide capacity
for another SE group. This ensures data plane isolation.

Example: Example 1:

Tenant 1 Tenant 2 Tenant 3

SE Group 1 SE Group 2 SE Group 3 SE Group 4

An administrator manages an application in both test and production environments. The virtual
service of each application must be deployed on a different SE group. For ease of management,
both applications can be in the same tenant (tenant 2 in the diagram), though arguments can be
made for separating these different environments into two separated tenants (such as tenant 1
and 3 in the diagram).

Example: Example 2:

Tenant 1 Tenant 2

SE Group 1

VMware by Broadcom 212

VMware Avi Load Balancer Administration Guide

A cloud service provider manages multiple customer applications. Each customer is assigned to
a unique tenant, ensuring complete configuration isolation. The service provider can allow all
tenants to have isolated SEs, or they can choose to place multiple tenants in the same SE group
to reduce idle resources.

Sharing Admin Profiles Across Tenants

Profile objects in the admin tenant are shared across all tenants in the system. This includes
objects visible under the Templates section of the Avi Load Balancer UI such as Health Monitor,
Application Profile, SSL Profile, Network Profile, PKI Profile, and so on.

The Avi Load Balancer defines a set of system default profiles as a part of the installation.
Though these objects can be edited, they can neither be renamed nor deleted. In addition to
these profiles, the administrator can create, edit and delete custom profile objects suited to their
specific deployment. Both the system default profiles and any custom profiles that are created in
the admin tenant are automatically shared across all the tenants in the system. Tenants are able
to view and use these profiles, but cannot delete the shared admin profiles.

Tenants can chose to override the default profile parameters to that which best suits their
application deployment. A tenant can do this by either editing the shared admin profile or
creating new custom profiles under their tenant context. Editing the shared admin profile creates
a copy-on-write effect, such that a new profile with the same name (but a different UUID) is
created in the specific tenant’s context. Making changes to this profile object does not affect
other tenants in the system. Deleting such a profile created using copy-on-write brings the
shared admin profile back into the tenant context.

The following example demonstrates the sharing functionality.

Navigate to Templates > Profiles > Application. In the Application Profile page, a couple of
custom profiles have been created by the administrator. Note that the default profiles (System-
xxx) cannot be deleted (check box is disabled for selection), but the custom profiles can be
selected for deletion (adjacent check box can be selected).

VMware by Broadcom 213

VMware Avi Load Balancer Administration Guide

In the context of tenant avidev, though the shared admin profiles are visible, none of the profiles
can be deleted.

By clicking the edit icon, tenant avidev edits the System-HTTP and Custom-HTTP profiles.

The formerly disabled check boxes turn selectable, indicating that the profiles have undergone a
copy-on-write operation and can now be selected for deletion.

Tenant avidev deletes the System-HTTP and Custom-HTTP profiles. The shared admin profiles
are now visible and cannot be deleted.

VMware by Broadcom 214

User Authentication and
Authorization 7
To access Avi Load Balancer through the GUI, REST API, or CLI, a valid user account is required.
Each user is assigned a role which grants permissions and access to read or write to the objects
in Avi Load Balancer. You can restrict accounts to specific tenants and grant configure different
roles within each tenant.

User accounts are maintained in two categories within Avi Load Balancer using an external
authentication, authorization, and accounting (AAA) server. Depending on how users are
authenticated, the two categories are as follows:

1 Local users

2 Remote users

Local Users

Local users are required to provide the username and password. The user can access CLI
without entering a password by providing a valid SSH key. Local users must belong to a
defined user group on the system.

Remote Users

Remote users are authenticated remotely on a service provided by LDAP, Tacas+, or SAML
servers. Remote users need not belong to a user group on the system.

When both the local and remote user accounts are configured, Avi Load Balancer authenticates
the credentials locally first and then authenticates the remote user account.

Read the following topics next:

n User Accounts

n User Roles

n HTTP Basic Authentication for API Queries

n Token Authentication

n Remote Authentication

n Authentication Profile

n Auth Mapping Profile

n User Account Security

VMware by Broadcom 215

VMware Avi Load Balancer Administration Guide

n Extended Granular RBAC

User Accounts
A valid account is required for access to Avi Load Balancer through the web interface, REST API,
or CLI. To configure or manage Avi Load Balancer user accounts, you need a user account with
write access to the Accounts section. This is defined by the role assigned to the user account.
The admin user account is a unique account used for initial setup of Avi Load Balancer.

Default System Accounts

By default, the Avi Load Balancer installation includes a number of pre-configured user accounts
that serve specific purposes. Unlike custom user accounts, these accounts cannot be deactivated
or removed.

admin

n The admin account exists on both the Controller and Service Engine of Avi Load Balancer.

n It is the default administrator user-name for the system and cannot be changed.

n The default shell is Linux bash.

n From Linux prompt, use shell command to access the CLI shell.

n admin is the only Avi Load Balancer account with its password automatically
synchronized with Linux.

n admin account has the Super User Accounts field enabled in the Controller. Any changes
to the admin account are not allowed.

n User is in the sudoers list.

n Default password for admin user - The initial default password of the Controller admin
user is changed from admin to a strong password. This password is available in the
portal where release images are uploaded and is accessible only to customers having
an account on the portal. Additionally, SSH access to the Controller with this default
password is not allowed until the user changes the default password of the admin user.
Once the password is changed, SSH access to the admin user is permitted. For more
information on default password, see Strong Default Admin Password.

n Password is synchronized to the SEs.

n Account has super-user status with full access to all tenants.

VMware by Broadcom 216

VMware Avi Load Balancer Administration Guide

n This account is always authenticated through the local user-db. It does not use any
configured remote authentication.

cli

n This account is used to launch the CLI shell by logging into the Controller. Users will SSH
to a Controller IP address, use cli as the username at the Linux prompt, and then be
presented with the Avi Load Balancer CLI shell access username and password prompt,
which requires their custom credentials.

n It is password-less from the Linux perspective with the CLI shell as the default shell that
prompts for a user name/password.

n CLI shell is launched in a container with no persistent storage.

aviseuser

n This account exists on the Controller and SE.

n It is the internal user for SE-to-Controller communication through SSH tunnel.

n No password is required. It uses unique key-pair per SE.

n User is not in the sudoers list on the Controller.

n User is in the sudoers list on the SEs.

avictlruser

n This account exists on Controller only.

n It is the internal user for Controller-to-Controller communication through SSH.

n No password is required. It uses unique key-pair per Controller.

n User is in sudoers list.

Create a Local User Account

1 Navigate to Administration > System Settings > EDIT.

2 Under Authentication, select Local. For more information on configuring Access Settings, see
Chapter 1 Web Interface Access Settings.

3 Navigate to Administration > Accounts > Users and click Create.

VMware by Broadcom 217

VMware Avi Load Balancer Administration Guide

4 If the user requires the same privileges as the admin account, select the Super User check
box. For more information, see Super User Accounts.

5 In the Name field, enter the full name of the user.

6 In the Username field, enter the name that the user will supply when signing into Avi Load
Balancer, such as jdoe or [email protected].

7 In the Password field, either enter a case-sensitive password or click the GENERATE button
to create a random password for the new user.

8 In the Email field, enter the email address of the user. This field is used when a user loses
their password and requests to have it reset.

9 From the User Profile drop-down menu, select Default-User-Account-Profile or No-Lockout-

User-Account-Profile as required. For more information, see User Profile.

10 Under Tenant & Role, select the areas of the Avi Load Balancer system to which the user
account will be allowed access. For each system area, the role defines whether the user
account has read, write, or no access. Avi Load Balancer comes with predefined roles. In
addition, users who have write access to the Accounts section of Avi Load Balancer can
customize the predefined roles and create new roles. For more information, see User Roles.

11 Click Save.

Modifying the Admin User Account

The admin user account can be modified only from the EDIT MY ACCOUNT screen:

1. Click the user icon at the right-hand top corner.

2. Click My Account.

VMware by Broadcom 218

VMware Avi Load Balancer Administration Guide

3. Modify the details as required in the EDIT MY ACCOUNT screen.

VMware by Broadcom 219

VMware Avi Load Balancer Administration Guide

4. Click Save.

Note Modifying the admin user account from the EDIT USER screen (Administration > Accounts
> Users) displays the error: Cannot make changes to the admin user.

User Account Table

To view the Avi Load Balancer user accounts that are in its local user database, navigate to
Administration > Accounts > Users.

VMware by Broadcom 220

VMware Avi Load Balancer Administration Guide

For each local user account, the following information is listed:

Field Description

Username The account name used to log in to Avi Load Balancer

through its management interfaces, that is, web interface,
REST API, or CLI.

Super User Super user access provides write access to all resources
within Avi Load Balancer.

Status Diplays the status of the user account, whether it is

Active or Suspended.

Full Name Full name of the user.

Email Email address of the user.

Tenant (Role) Access settings (write, read, or no access) for each type
of resource within Avi Load Balancer.

Last Signed In System time on the Avi Load Balancer Controller when
the user most recently logged in.

Super User Accounts

The user accounts in Avi Load Balancer can be enabled for Super User access. Super User
access provides write access to all resources within the Avi Load Balancer and also automatically
provides access to all tenants.

The admin account that is created during the installation of the Controller automatically has
Super User access enabled.

Optionally, super user access can be enabled in other user accounts, on an individual basis.

To enable super access for a Avi Load Balancer user account, select the Super User check box in
the account configuration.

VMware by Broadcom 221

VMware Avi Load Balancer Administration Guide

A non-admin user that is also a super-user can be associated with the Avi Load Balancer
Controller by using the attach <Avi Controller IP> command. This provides the Controller
container access to the user as an avidebuguser. The avidebuguser is also a sudo user. Attach
option is available only if the local or remote user is configured as a super-user.

For detailed information about configuring user accounts, see Create a User Account.

SSH Access for the Super User

During debugging, there is a need to access different Avi Load Balancer entities such as
Controllers and Service Engines. The Avi Load Balancer includes a mechanism whereby the
Controller can self-authenticate a super-user access to any entity in the system.

A user can attach to the Service Engine using the attach serviceengine <se_ip> command.
The admin user is authenticated using a token and logs into the SE as admin.

A sample illustration is as follows:

login as: admin

Pre-authentication banner message from server:
|
| Avi Cloud Controller
|
| Avi Networks software, Copyright (C) 2013-2017 by Avi Networks, Inc.
| All rights reserved.
|
| Management: 10.10.10.1/22 UP
| Gateway: 10.79.186.1 UP
|
| Controller 1 demo environment
|
End of banner message from server
[email protected]'s password:
The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under
license. Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or the GNU
Lesser General Public License (LGPL) Version 2.1. A copy of each
such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://www.opensource.org/licenses/lgpl-2.1.php
Last login: Wed Aug 17 14:32:15 2022 from 10.16.113.99
admin@10-79-187-241:~$

SSH Users and Keys

The Controller and SEs use SSH for secure management communication. The communication falls
into two categories, Controller-to-SE, and administrator-to-Controller.

VMware by Broadcom 222

VMware Avi Load Balancer Administration Guide

Controller-to-SE Communication
This requires an SSH user who exists on both the Controller and the SEs, and a copy of the
SSH user’s public key on the Avi Load Balancer SEs. Though SSH setup is automated for some
installation types such as installation into VMware with write access, other installation types
require manual setup of these SSH resources. For more information, see Public Key Management
on SE Hosts topic in the VMware Avi Load BalancerInstallation Guide.

Create SSH User

To create an SSH user on the Avi Load Balancer Controller, use the following steps.

Note An SSH user and key that already exists can be used. They must still be added to the
Controller using these steps. When creating the user account, the existing key for the user can
be added by copying and pasting it or by importing the key file.

1 Navigate to Administration > Accounts > User Credentials.

2 Click Create.

3 Enter the SSH user name (for example, root) in the Name field.

4 Select SSH as Credentials Type.

5 In Authentication section, select SSH Key.

6 In Keys section, select Generate SSH Key Value Pair to create a key pair for the user.

7 Click Generate and Save.

Note If the user already exists, you can add the user to the Avi Load Balancer by entering the
user name, selecting Import Private Key, or either copying and pasting or importing the key file.

Download Public Key File

After creating an SSH user on the Avi Load Balancer Controller, perform the following steps to
download the user public key from the Controller.

1 Navigate to Administration > Accounts > User Credentials.

2 Click the three dots next to the SSH user and select Download Public Key.

Restricting admin SSH access to Service Engines

By default, the user can connect directly to an SE via SSH using the system’s admin credentials.

If there is a security requirement to disallow this connectivity, it is possible to disable this access
using the following CLI configuration:

[admin:<controller-ip>]: > configure serviceengineproperties

[admin:<controller-ip>]: seproperties> se_runtime_properties
[admin:<controller-ip>]: seproperties:se_runtime_properties> no admin_ssh_enabled

VMware by Broadcom 223

VMware Avi Load Balancer Administration Guide

[admin:<controller-ip>]: seproperties:se_runtime_properties> save

[admin:<controller-ip>]: seproperties> save

Securing Remote Access to Service Engines

With direct SSH access to SEs disabled, it is still possible for a user with the Super User privilege
to remotely access a Service Engine's shell through a secure tunnel from the Controller using the
attach serviceengine command from the Controller CLI. This will automatically log in to the
SE using an internal user account (avidebuguser) via PKI authentication - the Super User does
not need to know the default admin account credentials. The avidebuguser account also has
passwordless sudoer privileges.

Users without the Super User privilege cannot establish remote connections to an SE through
this method, even if the user has the default admin account credentials.

The attach serviceengine command can only be used from a CLI shell running directly on the
associated Controller. A remote shell or a shell launched in a container on the Controller (via SSH
with the cli username) will not be able to attach to the service engine.

A user with the Super User privilege can launch a CLI shell directly on the Controller without
requiring to know the default admin password as follows:

ssh cli@<controller-ip>

(Log in to the container CLI shell with Super User account.)

[admin:<controller-ip>]: > attach controller <controller-ip>

avidebuguser@<controller-ip>:~$ shell

(Log in to the second non-container CLI shell with Super User account.)

The Super User can now attach to service engines:

[admin:<controller-ip>]: > attach serviceengine <service-engine>

For more information, refer to the following topics:

n Accessing Avi Load Balancer Linux CLI as a Non-Admin User Using an SSH client

n Super User Accounts

n Chapter 2 CLI Access Settings

n SSH Access for the Super User

User Profile
The User Profiles option is used to control a user access to Avi Load Balancer. It is used to
control various attributes related to the user account.

VMware by Broadcom 224

VMware Avi Load Balancer Administration Guide

The two default user profiles are a follows:

1 Default-User-Account-Profile

2 No-Lockout-User-Account-Profile

By default, all the users in the system are associated with Default-User-Account-Profile.

Create a User Profile

The steps to create a new user profile are as follows.

Procedure

1 From the User Profiles screen, click CREATE.

You can view the New User Profile screen.

2 Enter a Name.

3 Under Max Password History Count, enter the maximum number of passwords to be
maintained in the password history.

4 In the Max Login Failure Count, enter the maximum number of login attempts allowed.

5 Enter the Account Lock Timeout period in minutes.

6 UnderLogin Failure Count Expiry Window, enter the time (in minutes) within which only
login_failure_attempts from the past will be considered for account lockout. Set this to 0
if you do not want to do a time-based account lockout and consider all failed login attempts
irrespective of time frame.

7 In the Max Concurrent Sessions, enter the maximum number of concurrent sessions allowed.

8 Enter Credentials Timeout Threshold (in days), after which the credentials are invalid.

The New User Profile screen is as shown below:

VMware by Broadcom 225

VMware Avi Load Balancer Administration Guide

9 Click Save.

What to do next

To completely avoid the risk of your account getting locked, you can use the Configure No-
Lockout-User-Account-Profile on Avi Load Balancer. By default, this user profile has Max login
Failure Count and Login Failure Count Expiry Window set to 0.

Maximum Concurrent Login Sessions

This feature prevents having more than a configurable number of concurrent sessions. The
default value is set to 0, which means the concurrent session check is bypassed. Additional logins
are prevented if the maximum concurrent session count has been reached. Beyond this, the user
can log in using shell --clear-sessions which will invalidate all the active sessions.

The administrator controls this feature through Avi Load Balancer CLI or REST API. The setting
for it is maintained within the UserAccountProfile object. By default, all the users in the system
are attached to Default-User-Account-Profile, as shown below. The admin can create a new user
account profile with different thresholds if required.

admin:10-10-24-52]: > show useraccountprofile Default-User-Account-Profile

VMware by Broadcom 226

VMware Avi Load Balancer Administration Guide

If the threshold has been reached, the user can choose to invalidate existing sessions using the
--clear-sessions option of the shell command.

root@10-10-24-52:/home/admin# shell
Login: admin
Password:

Caution Maximum concurrent session count has been reached. Clear the sessions using shell
–clear-sessions

root@10-10-24-52:/home/admin/# shell –clear-sessions Login: admin Password:

[admin:10-10-24-52]: >

The sessions are considered invalid when one of the following is observed:

n The user’s UI session(s) end, and a login screen is displayed.

n The user’s CLI sessions will end with no indication on-screen. The next command typed will
trigger the re-validation of a new CLI session, and the command will be executed.

n A REST API user’s next API call will fail to validate. Or, if the REST API user is executing calls
within a session, the session is ended.

Configure No-Lockout-User-Account-Profile on Avi Load Balancer

The main difference between the Default-User-Account-Profile and No-Lockout-User-Account-
Profile is the value set for Max Login Failure Count. The Max Login Failure Count is the number
of login attempts allowed before the lockout of the user account. By default, this value is set to 3
for the default profile.

For the no lockout user profile, Max login Failure Count is set to 0. It means that a user can have
unlimited login failures without the risk of an account getting locked.

In GSLB deployments, it is recommended to use No-Lockout-User-Account-Profile. It prevents

the locking of the user account due to various reasons. Sometimes, an admin account gets
locked when one node of the GSLB pair tries to reach another node with the admin credentials,
but the other node is not reachable.

Configuring using UI
To check or edit the attributes for No-Lockout-User-Account-Profile, navigate to Administration
> Accounts > User Profiles and click the pencil icon on the right side of No-Lockout-User-
Account-Profile.

Note You can use the existing No-Lockout-User-Account-Profile available or create a new
one. Max Login Failure Count must be set to 0 for any profile to work like a No-Lockout-User-
Account-Profile.

To create a new user, follow the below steps:

1 Login to the Avi Load Balancer Controller using admin credentials.

VMware by Broadcom 227

VMware Avi Load Balancer Administration Guide

2 Navigate to Administration > Accounts > Users.

3 Click Create.

4 Provide the username of your choice.

5 Select the No-Lockout-User-Account-Profile from the drop-down menu for User Profile.

6 Use desired Tenant and Role for this new user.

7 Click Save.

Note Use the same user created in the previous step while doing GSLB configuration. For
example, the user admin must be replaced with the newly created user (GSLB-User) with No-
Lockout-User-Account-Profile.

For more information, see GSLB Site Configuration.

VMware by Broadcom 228

VMware Avi Load Balancer Administration Guide

Manage User Profiles

The User Profiles screen lists all the user accounts created.

Procedure

1 Navigate to Administration > Accounts > User Profiles.

2 From this screen, click CREATE to create new user profiles, or edit/DELETE existing user
profiles, as required.

User Account Management

Avi Load Balancer user accounts with write access to the Accounts section can perform the
following management actions on other user accounts:

Field Description

ACTIVE Reactivates the selected user accounts.

SUSPEND Deactivates the selected user accounts. A suspended

user cannot access Avi Load Balancer through any of its
management interfaces. When the user attempts to log in
to Avi Load Balancer, a notice is displayed to inform them
of the account suspension, and access is denied.

DELETE Removes the selected user accounts from Avi Load

Balancer.

User Roles
The role, in combination with the tenant, comprises the authorization settings for an Avi Load
Balancer user. When assigned to a user, the role defines the type of access the user has to each

VMware by Broadcom 229

VMware Avi Load Balancer Administration Guide

area of the NSX Advanced Load Balancer system. Roles provide granular RBAC within Avi Load
Balancer.

Access Types
For each Avi Load Balancer resource (object type) and within each group of resources (system
area), the user can have the following privileges:

n Write: The user has full access to create, read, modify, and delete resources.

n Read: The user can only read the existing configuration of resources. For example, the user
can see how a virtual service is configured and view the health and analytics data of the
virtual service but is unable to modify the configuration or delete the virtual service.

n No Access: The user has no access to the resources and cannot even read or list these
resources.

n Assorted: The user has a mixture of the above privileges for different resources within the
system area.

Create a Role
If none of the pre-defined roles fit the access requirements for some user accounts, you can
create custom roles according to the requirement.

Procedure

1 Navigate to Administration > Accounts > Roles.

2 Click CREATE to open the New Role screen.

3 Under General, enter a Name for the role.

VMware by Broadcom 230

VMware Avi Load Balancer Administration Guide

4 Configure the access for each system area under Role Access. By default, all the roles are set
to no access.

a To set access to all the resources within a system area, click the required access icon on
the title row of the system row.

To provide write access to all the profiles, click the write icon on the title row of the
Profiles system area:

b To give write, read, or no access for some resources, click the relevant icon for each
resource. Automatically, the assorted access is enabled on the title row of the system
area, indicating multiple types of access.

5 Optionally, configure Label Filters for the role to provide object-based granular access.

a Click Add to open the LABEL FILTER sub-screen.

b Enter the Name for the filter.

VMware by Broadcom 231

VMware Avi Load Balancer Administration Guide

c Select Enable Label Filter to apply the label.

d Enter the Key.

e Select the Criteria to match the key:value pair.

f Under the Values, click Add to add values.

g Click Save.

6 Click Save to save the new role.

Note Roles can only be created in the admin tenants.

Assign a Role to a Local User Account

You can assign a role to local and remote user accounts (LDAP, TACACS+). The procedure differs
depending on where the account is maintained.

Roles can be assigned to a user account when the account is created or at any time later. In
either case, select the Role from the Role drop-down menu in the configuration popup for the
user account.

Note User accounts are case-sensitive. LDAP users are an exception.

Procedure

1 Navigate to Administration > Accounts > Users.

2 To configure a new account, click Create. To edit an existing account, click the edit icon.

3 Under Tenant & Role, click Add and select the Role from the Roles drop-down menu. To
create a custom role, click Create.

VMware by Broadcom 232

VMware Avi Load Balancer Administration Guide

Manage User Roles

Each user must be associated with at least one role. The role can be either predefined or a
custom role.

Procedure

1 From the UI, navigate to Administration > Accounts > Roles.

The pre-defined roles in Avi Load Balancer are listed. Each user role has different levels of
access to different settings in Avi Load Balancer.

2 Click the + icon in the table to expand the access settings for a role.

3 Click the edit icon against a role to configure the access settings for the role according to the
requirement.

Results

If multi-tenancy is configured, a user can be assigned to more than one tenant and have a
separate role for each tenant.

HTTP Basic Authentication for API Queries

API calls from a client to the Avi Load Balancer Controller must first be authenticated, either
by HTTP session-based authentication or HTTP basic authentication. By default, HTTP basic
authentication is disabled.

Authenticated API calls are still subject to normal authentication settings, regardless of the
method used. The user account used for authentication can be validated by the Controller
through a local database or remote (such as LDAP), can be limited to a specific tenant, or have
limited roles or access levels. HTTP basic authentication is deactivated by default for increased
security.

Enabling HTTP Basic Authentication using CLI

HTTP Basic Authentication can be activated using the following CLI commands.

bash# shell
: > configure systemconfiguration
: systemconfiguration> portal_configuration
: systemconfiguration:portal_configuration> allow_basic_authentication
Overwriting the previously entered value for allow_basic_authentication
: systemconfiguration:portal_configuration> exit
: systemconfiguration> exit

Truncated view of the results:

VMware by Broadcom 233

VMware Avi Load Balancer Administration Guide

Activating HTTP Basic Authentication through API

HTTP Basic Authentication method can be activated by setting “allow_basic_authentication” =
True in “/api/systemconfiguration” under “portal_configuration”.

Truncated view of the results:

--snip--
docker_mode: false,
portal_configuration:
{
use_uuid_from_input: false,
redirect_to_https: true,
sslprofile_ref: "https://10.10.5.27/api/sslprofile/sslprofile-0-1",
allow_basic_authentication: true,
enable_clickjacking_protection: true,
enable_https: true,
sslkeyandcertificate_refs:
[
"https://10.1.1.10/api/sslkeyandcertificate/sslkeyandcertificate-ae6c1033-859b"
],
password_strength_check: false,
enable_http: false
},
--snip--

Token Authentication
For debugging or even normal day-to-day operations, accessing the Controller’s CLI is often
needed using SSH. To access the Avi Load Balancer Controller through SSH, the system admin
or a registered user must have a valid token. Note that this token is not the same as the Oauth
token. It is an alternative to a password. The system admin can create a temporary token for a
user to access resources for a few hours. After that time expires, the token will not work and
the user loses access, so there is no need to delete the token. Once a token is created, you can
initiate an SSH connection to the Controller using CLI as the SSH user. A CLI shell is created. Once
the shell has been created, a login prompt will be displayed. Provide the required username and
the token as the password.

To generate the SSO token through the UI, perform the following steps.

VMware by Broadcom 234

VMware Avi Load Balancer Administration Guide

Procedure

1 Login to the Avi Load Balancer UI.

2 Click the three dots in the top right corner of the dashboard.

3 Select Generate Token from the drop down menu.

A pop-up screen appears as shown below.

VMware by Broadcom 235

VMware Avi Load Balancer Administration Guide

4 Enter the Token Lifetime for the token’s validity in hours and click Generate.

Note
n To generate a single use token, enter 0.

n The maximum value that can be entered in this field is 87600 hours.

n In case another token is generated before the first one expires, the first token still remains
valid.

5 Copy the generated token for CLI/SSH access.

6 To delete a token, for example, if you want to decommission an active token, use the DELETE
command in the CLI.

Access the CLI using the Token

This section explains the steps to access CLI/SSH using the generated SSO token.

Procedure

1 Open an SSH or Putty session to [email protected]</code>.

2 Login with your username.

3 Paste the token that was generated using the CLI as the Password.

Results

You have now successfully logged into the Controller using your account or Service Account.

VMware by Broadcom 236

VMware Avi Load Balancer Administration Guide

Remote Authentication
Authentication Profile discusses different ways of remote authentication supported in Avi Load
Balancer.

Note If remote authentication (LDAP, TACACS, SAML, and more) is enabled, you can deactivate
local authentication in the Avi Load Balancer Controller by deactivating the Enable Local User
Login option in the Edit System Settings screen.

Configuring Remote Authentication

To enable remote authentication and map user roles, follow the steps below.

1 Log in to the Avi Load Balancer Controller with admin credentials.

2 Navigate to Administration > System Settings and click Edit.

3 Under Authentication, select Remote.

4 Select the Enable Local User Login option. If this option is not selected and there is a
configuration issue, you will not be able to log back into the Controller.

5 Under Auth Profiles & Mapping Profiles, click Add.

6 Select the required Auth Profile and Mapping Profile.

7 Click Save.

VMware by Broadcom 237

VMware Avi Load Balancer Administration Guide

Authentication Profile
Authentication is the process of validating the identity of a user. This is to make sure that the
user connecting to a system is authorized or allowed to do so. In the last couple of years, there
has been a significant shift towards federated authentication, to centralize the authentication
process by delegating the process of authentication to an identity provider.

To configure an Auth profile, perform the following steps.

1 Navigate to the Templates > Security > Auth Profile and click Create.

2 Enter a name for the profile and select the Type.

3 Configure the settings specific to the type of profile selected.

4 Click Save.

LDAP Authentication Profile

Avi Load Balancer supports user authentication using Lightweight Directory Access Protocol
(LDAP). LDAP is a commonly used protocol for accessing a directory service. A directory service
is a hierarchical object oriented database view of an authentication system. LDAP settings can be
configured in an authentication profile.

Creating an LDAP Auth Profile

To create an LDAP Auth Profile, perform the following steps.

1 Navigate to Templates > Security > Auth Profile.

2 Under the General tab, enter the Name of the profile and select the Type of the Auth Profile
as LDAP.

3 Configure the LDAP settings.

VMware by Broadcom 238

VMware Avi Load Balancer Administration Guide

4 Click Save.

Configuring LDAP Settings

1 Under the LDAP tab, click Add to enter the LDAP server IPs.

Note Assume there are three LDAP servers. When a user logs in, the Controller validates
the credentials against the first LDAP server. If a match is detected, the Controller stops
further checks. However, if no match is found, the Controller continues to validate against
the remaining LDAP servers in the configured order until a match is found. If no match is
identified across all servers, an error is displayed.

2 Select LDAPS or None from the LDAP Connection Security Mode drop-down menu as
required. Selecting None defaults to LDAP.

3 Enter the Port on which Avi Load Balancer queries the LDAP servers. Use port 389 for LDAP
and 636 for LDAPS (SSL).

Note Configuring the port does not determine if the LDAP security mode is LDAPS. Hence,
explicitly select LDAPS as the LDAP Connection Security Mode to ensure secure connection
between an LDAP client and server.

4 In the Base DN field, enter the top level path in the directory server object hierarchy.

5 Select the LDAP Bind Settings.

Administrator Bind

The administrator account provided will be used to search for users and user group
memberships across the LDAP server.

Note Administrator Bind is the recommended LDAP Bind option.

VMware by Broadcom 239

VMware Avi Load Balancer Administration Guide

An LDAP Admin Bind DN (Distinguished Name) is a unique identifier used to authenticate an

administrative user to an LDAP server. It is typically set up during the initial configuration of
the LDAP server and is used to perform administrative tasks:

n Managing the directory structure.

n Adding or modifying user entries and configuring the server itself.

The Admin Bind DN is essential for performing administrative tasks. It is important to

safeguard the associated credentials to prevent unauthorized access and maintain the
security of the LDAP directory. It is also important to ensure that the Admin Bind DN
has sufficient privileges to perform the required operations, for example, in OpenLDAP, the
Admin Bind DN is often set to cn=admin,dc=example,dc=com.

where,

n dc=example

n dc=com is the base DN of the LDAP server

The Admin Bind DN and Admin Bind Password are the credentials used to authenticate
the administrative user to an LDAP server. The Admin Bind Password is associated with an
administrative user's DN. The account used must have access to search the directory tree for
both users and user groups.

Avi Load Balancer uses the configuration to search for users or groups. An LDAP search
typically requires:

n Top-level directory hierarchy (search DN) to start the search.

n The scope value limits the search to one of the following: base (one-level deep) or entire
subtree.

n Filter to match only on entries of a given class or category.

To configure Administrator Bind, enter the following:

VMware by Broadcom 240

VMware Avi Load Balancer Administration Guide

Field Description

User Search DN LDAP User Search DN is the root of search for a given
user in the LDAP directory. It helps ensure that the
search is efficient and focused only on the relevant part
of the directory tree where user information is stored.
Only user records present in this LDAP directory sub-
tree are allowed for authentication. Base DN value is
used if this value is not configured.

User Search Scope LDAP User Search Scope determines the extent of
search for the user starting from user search DN.
The different levels of user search available are:
n Base - The search is limited to the specified base
DN.
n One Level - The search is limited to the specified
base DN and its immediate child entries.
n Subtree - The search is performed on the entire
tree, starting from the specified base DN.

User ID Attribute LDAP User ID Attribute is the login attribute that

uniquely identifies a single user record. The value of this
attribute must match the user name used at the login
prompt.

Group Search DN LDAP Group Search DN is the root of search for

a given group in the LDAP directory. Only matching
groups present in this LDAP directory sub-tree will
be checked for user membership. If this value is not
defined, the base DN value is used.

Group Search Scope Define the LDAP Group Search Scope using filters for
the group starting from the group search DN.
The different levels of group search available are:
n Scope Base
n Scope One Level
n Scope Subtree

Group Filter LDAP Group Filter is a search filter used to focus

the search results to specific group entries within the
directory. A search can find many different types of
objects under the search DN. Specify the Group Filter
to pick up objects specific to the group only. The
default group filter is (objectClass=*).
Avi Load Balancer appends a user-specific group
membership filter to the configured group filter to
check a specific user’s group membership.

VMware by Broadcom 241

VMware Avi Load Balancer Administration Guide

Field Description

Group Member Attribute LDAP Group Member Attribute is a multi-valued

attribute that can store the DNs of the users who
are members of the group. It is used for retrieving
data about group members and managing group
membership.
For example, member and memberUid are commonly
used attributes.

Enable Full DN For Group Member Attribute Enable this option if the group member entries have full
DNs instead of just user ID attributes.
For example, a configured group filter
(objectClass=group) is extended by Avi Load Balancer
to a full filter when user “bob” logs in to
something such as the following: (&(objectClass=group)
(member=bob)).

Anonymous Bind

Anonymous Bind is a method of connecting to an LDAP server without providing any

credentials. It is useful when the user has no access to an administrator account on the LDAP
server(s). However, the level of access granted to an anonymous bind is typically limited and
depends on the configuration of the LDAP server.

To configure Anonymous Bind, enter the following:

Field Description

User DN Pattern User DN Pattern is used to bind an LDAP user after

replacing the user token with the real user name. The
pattern must match the user record path in the LDAP
server.

User Token User Token is replaced with the real user name in the
user DN pattern.
For example, a User DN Pattern could be
uid={0},ou=Users,dc=example,dc=com.

where,
{0}=Username

User ID Attribute User ID Attribute is the unique property that identifies

a single user record. This value must match with the
user name used at the login prompt.

Note Authentication profiles that use anonymous bind cannot be used for role or tenant
mapping.

1 An LDAP profile configured with Administrator Bind is as shown below:

VMware by Broadcom 242

VMware Avi Load Balancer Administration Guide

2 Enable Ignore Referrals for the group search to skip referrals links that connect to another
LDAP server. This option is deselected by default.

Note Enabling the option Ignore Referrals can quicken the group searches by preventing
delay in LDAP group search due to unnecessary referral searches. For more information, see
RFC4511.

3 Under User, define the scope of search for users who log into Avi Load Balancer.

4 Click SAVE.

Note IPv6 snmptrap/syslog/email endpoints on a Controller is supported with a IPv6

address.

LDAP Authentication Profile Testing

Avi Load Balancer provides an option for testing authentication profiles configured on the Avi
Load Balancer Controller.

VMware by Broadcom 243

VMware Avi Load Balancer Administration Guide

Testing an Authentication Profile

To verify the LDAP profile, perform the following steps.

1 Navigate to Templates > Security > Auth Profile.

2 Click the verify icon to view the VERIFY AUTH PROFILE.

Note Depending on whether the LDAP Auth Profile has Administrator Bind or Anonymous
Bind configured, the VERIFY AUTH PROFILE screen displays a different set of options.

3 Enter the details required in the VERIFY AUTH PROFILE.

Verifying LDAP Profile with Anonymous Bind

If the LDAP authentication profile is configured to use anonymous binding for authentication
requests, the popup for testing the profile prompts for the LDAP user’s user name and password.

VMware by Broadcom 244

VMware Avi Load Balancer Administration Guide

In the VERIFY AUTH PROFILE screen, enter the Username and Password , and click Verify.

Testing whether a user can bind successfully verifies that the LDAP authentication profile is
configured correctly to authenticate users with the same user DN pattern.

Verifying LDAP Profile with Administrator Bind

If the LDAP authentication profile is configured to use administrator binding for authentication
requests, one of the following types of information can be specified on the verification popup for
the profile.

In the VERIFY AUTH PROFILE screen, select the required option to verify and enter the
Username.

VMware by Broadcom 245

VMware Avi Load Balancer Administration Guide

Test user entry

n Searches the LDAP server’s database for the specified user name, and returns the
corresponding user entry from the LDAP database.

n This option is useful for listing all attribute key-value pairs for any given user. The user
search settings configured in the authentication profile are used.

n If the Username field is left empty, Avi Load Balancer pulls the entire list of user records
from the LDAP database.

Test user group membership

n Lists all group memberships for the specified user. The group search settings configured
in the authentication profile are used.

n If the Username field is left empty, all groups are returned.

Test base DN

n Returns all objects under the base DN.

n This option is useful for testing administrator permissions and for reading the DN tree of
the LDAP server.

This process can identify some common error scenarios like:

n LDAP server IP/port is incorrect

VMware by Broadcom 246

VMware Avi Load Balancer Administration Guide

n Bad user name or user search settings are incorrect

n User is either not a member of any group or the group search settings are incorrect

LDAP Configuration Examples

This section has several sample LDAP profile configurations that can be used in different
scenarios.

For more information on creating an LDAP profile, see Configuring LDAP Settings.

Example: Example 1

Use Case Binding Option Configuration Details

Active Directory common settings Administrator Bind With the configuration for
administrator bind, the group
membership includes the full DN.

The LDAP profile configured with Administrator Bind and user search configuration is as shown
below:

The Group search configuration for the LDAP profile is as shown below:

VMware by Broadcom 247

VMware Avi Load Balancer Administration Guide

Note The User ID Attribute and the Group Member Attribute must match as configured in the
Active Directory. The quickest way to verify is to check the response by testing the Base DN.

Example: Example 2

Use Case Binding Option Configuration Details

Active Directory common settings Anonymous Bind If the LDAP/AD user can bind
with the DN [email protected] and
password, it validates the user login.

The LDAP Profile configured with Anonymous Bind is as shown below:

VMware by Broadcom 248

VMware Avi Load Balancer Administration Guide

TACACS+ Authentication Profile

Avi Load Balancer supports authentication and authorization of Avi Load Balancer users with
Terminal Access Controller Access Control System (TACACS+). TACACS+ is an open standards
protocol that handles authentication, authorization, and accounting.

Creating a TACACS Plus Profile

In Avi Load Balancer, TACACS+ settings are configured in an auth profile. To create a TACACS+
auth profile, complete the following steps.

1 Navigate to the Templates > Security > Auth Profile.

2 Click Create to view the CREATE AUTH PROFILE screen.

3 Select the Type as TACACS_PLUS.

4 Under the TACACS+ tab, click Add to enter TACACS+ server IP address.

Note You can add multiple servers. If the first server does not respond, Avi Load Balancer
tries the next server. If there is no response from this server as well, Avi Load Balancer tries
to connect to the next server. Each server is tried once. If all the servers fail or timeout, the
request is failed.

5 Enter the TACACS+ server Port. By default, the value is 49.

6 Enter the TACACS+ server shared secret as Password.

7 Select the TACACS+ Service type used in all authentication and authorization queries.

VMware by Broadcom 249

VMware Avi Load Balancer Administration Guide

8 Under Authorization Attributes, click Add and enter the set of attribute value pairs under
Name and Value to identify the host. The TACACS+ server configures user-level authorization
based on these attributes. For example, Cisco Access Control Servers (ACSs) typically expect
authorization attribute values for “service” and “protocol” to be populated to identify and
authorize an Avi Load Balancer user. Authorization attributes from a TACACS+ server can be
used to map users to various roles and tenants.

9 If the attribute is required, check Mandatory.

10 Click Save.

Authentication and Authorization

Authentication and authorization of an Avi Load Balancer user with TACACS+ takes place as
follows:

1 AUTHEN START packet from Avi Load Balancer to TACACS+ server. Contains:

n action=login

n authen_type=ascii

n service=

n user=

n remote_addr=

2 AUTHEN REPLY packet from TACACS+ server to Avi Load Balancer. It contains status of type
GETPASS indicating that password needs to be supplied for the user message field with text
“Password.”

VMware by Broadcom 250

VMware Avi Load Balancer Administration Guide

3 AUTHEN CONTINUE packet from Avi Load Balancer to TACACS+ server. It contains user
message field with actual password from user.

4 AUTHEN REPLY packet from TACACS+ server to Avi Load Balancer. Contains:

n SUCCESS status if password is valid and user is allowed

n FAILED status

5 AUTHOR START packet from Avi Load Balancer to TACACS+ server. Contains:

n User name of the user

n Remote address of the user

n Authorization attribute name, value and whether or not they are mandatory

n An authorization attribute string “abc=xyz” that indicates an attribute named “abc” is

mandatory and has value “xyz”

n An authorization attribute string “abc*xyz” that indicates an attribute named “abc” is

optional and has value “xyz”

6 AUTHOR REPLY packet from TACACS+ server to Avi Load Balancer. It contains one of the
following:

n PASS_ADD or PASS_REPL status, which authorizes the successfully authenticated user

with attribute value pairs to be added or replaced.

n FAIL status, indicating the user is not authorized.

Encryption
All TACACS+ packets are encrypted, whereas the 12-byte header is passed in the clear.
Encryption is part of the TACACS+ standard and is compatible with all TACACS+ servers.

Error Handling
If an error is indicated in the Status field of any reply packet during this process, the user login is
rejected and results in a failure.

TACACS+ Configuration Examples

This section explains the examples of TACACS+ Configuration.

ISE TACACS+ Server

Cisco ISE is a security policy management platform that provides secure access to network
resources. Cisco ISE functions as a policy decision point and enables enterprises to ensure
compliance, enhance infrastructure security, and streamline service operations.

To set up an ISE TACACS+ server as a remote authentication and authorization system for Avi
Load Balancer, follow the steps given below:

The ISE server is generally configured with external Identity Sources (in this case OpenLDAP).

VMware by Broadcom 251

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 252

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 253

VMware Avi Load Balancer Administration Guide

n The ISE LDAP settings used to fetch LDAP groups and use them for Authorization conditions
are as shown below:

n ISE Authorization conditions added for Users in the AD groups.

VMware by Broadcom 254

VMware Avi Load Balancer Administration Guide

n ISE server recognizes all Avi Load Balancer Controller cluster nodes as valid Network Devices.

VMware by Broadcom 255

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 256

VMware Avi Load Balancer Administration Guide

n Configuring ISE requires shell profiles and TACACS+ profiles.

VMware by Broadcom 257

VMware Avi Load Balancer Administration Guide

n ISE device policy sets default condition updated to assign different shell profiles based on
group membership.

n The Avi Load Balancer TACACS+ auth profile must be configured with the same shared
secret that was assigned to the device in ISE. The “service” attribute is generally required
to identify and authorize a Avi Load Balancer user. Authorization attributes from a TACACS+
server can be used to map Avi Load Balancer users to various roles and tenants.

In the case of an ACS server, service=avishell is required for user authorization; while in the
case of an ISE server, service=avishell is known to cause authorization failure.

VMware by Broadcom 258

VMware Avi Load Balancer Administration Guide

n Avi Load Balancer TACACS+ authorization role and tenant mapping configured to assign
different roles based on TACACS+ attribute value.

Shrubbery TAC_PLUS
n TAC_PLUS server is a much simpler alternative to ISE/ACS. This is relevant in development or
testing environments. Conceptually, users are assigned to groups and groups have request
and response attributes.

VMware by Broadcom 259

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 260

VMware Avi Load Balancer Administration Guide

The Avi Load Balancer TACACS+ auth profile can be configured in the same way as an ISE or
ACS.

SAML Authentication Profile

Security Assertion Markup Language (SAML) is an XML-based markup language for exchanging
authentication and authorization between an identity provider (IdP) and a Service Provider (SP).
SAML is important to your organization as you move more towards relying on hosted services, as
it provides a single point of authentication at IdP. The credentials are not stored on multiple
untrusted devices, as SAML is used to assert the identity. Another benefit lies in the user
experience, as users do not have to remember multiple credentials to access their apps. They
can access multiple applications with a single set of credentials by logging in once.

Configuring the Authentication Profile using UI

1 Navigate to Templates > Security > Auth Profile.

2 Click Create.

3 Enter the Name for the Auth profile.

4 Select SAML as the Type of Auth profile.

5 Select Use IDP Metadata or Use IDP Metadata URL as required and enter the metadata from
your IdP.

Note
a It is recommended to select Use IDP Metadata URL option. If you use both the options,
the Use IDP Metadata URL option takes precedence.

b Any node acting as a service provider must generate a metadata file for registration with
the IdP. The file contains configuration and integration details for SAML single sign-on.
Obtain the metadata file from your identity provider.

VMware by Broadcom 261

VMware Avi Load Balancer Administration Guide

6 Under Service Provider, select Use Cluster VIP or Use DNS FQDN, as required and enter
details.

Note
a Service provider metadata contains keys, services, and URLs defining SAML endpoints of
the Avi Load Balancer Controller. The Controller can be registered using its cluster VIP or
a DNS-resolvable FQDN.

1 If Use Cluster VIP is selected, then the cluster VIP is picked up automatically. Cluster
VIP is the virtual IP address of the Controller cluster.

2 If Use DNS FQDN is selected, Avi Load Balancer Controller prompts to provide an
FQDN. DNS FQDN is the FQDN name of the Controller cluster.

b Use user-provided entity ID option is not applicable for Controller Authentication. It is

only used for Client Authentication (L7 Virtual Service Authentication).

7 Click Save.

VMware by Broadcom 262

VMware Avi Load Balancer Administration Guide

Configuring the Authentication Profile using CLI

n Configure the Auth Profile using the CLI commands shown below:

[admin:saml-ctrlr-1]: > configure authprofile Saml-auth-profile

[admin:saml-ctrlr-1]: authprofile> type auth_profile_saml
[admin:saml-ctrlr-1]: authprofile> saml
[admin:saml-ctrlr-1]: authprofile:saml> sp
[admin:saml-ctrlr-1]: authprofile:saml:sp> saml_entity_type auth_saml_app_vs
[admin:saml-ctrlr-1]: authprofile:saml:sp> save
[admin:saml-ctrlr-1]: authprofile:saml> save
[admin:saml-ctrlr-1]: authprofile> saml
[admin:saml-ctrlr-1]: authprofile:saml> new idp
metadata : ‘metadata_string’

n A sample configuration is shown below:

metadata: ‘<?xml version="1.0" encoding="UTF-8"?><md:EntityDescriptor

xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata" entityID="http://www.okta.com/
exk2c02xxTcM9pIr0355"><md:IDPSSODescriptor WantAuthnRequestsSigned="false"
protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol"><md:KeyDescriptor
use="signing"><ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/
xmldsig#"><ds:X509Data><ds:X509Certificate>MIIDtjCCAp6gAwIBAgIGAWPuJWSOMA0GCSqGSIb3DQEBCwUA
MIGbMQswCQYDVQQGEwJVUzETMBEG
A1UECAwKQ2FsaWZvcm5pYTEWMBQGA1UEBwwNU2FuIEZyYW5jaXNjbzENMAsGA1UECgwET2t0YTEU
MBIGA1UECwwLU1NPUHJvdmlkZXIxHDAaBgNVBAMME2F2aW5ldHdvcmtzLWF1dGhsYWIxHDAaBgkq
hkiG9w0BCQEWDWluZm9Ab2t0YS5jb20wHhcNMTgwNjExMDkxOTE4WhcNMjgwNjExMDkyMDE3WjCB
mzELMAkGA1UEBhMCVVMxEzARBgNVBAgMCkNhbGlmb3JuaWExFjAUBgNVBAcMDVNhbiBGcmFuY2lz
Y28xDTALBgNVBAoMBE9rdGExFDASBgNVBAsMC1NTT1Byb3ZpZGVyMRwwGgYDVQQDDBNhdmluZXR3
b3Jrcy1hdXRobGFiMRwwGgYJKoZIhvcNAQkBFg1pbmZvQG9rdGEuY29tMIIBIjANBgkqhkiG9w0B
AQEFAAOCAQ8AMIIBCgKCAQEAiFKBy70aa5G2I5JH+uUqXef9jrhUtx6CX1nmrg26FXtsKYdjRm5v
otxbjfNdcXeXRXHu5scMwAgMy9EZM+AXehlm/qnahNWvEZ+YgPZS55UzkcSXJ30dl62kbUAyXxo3
CQQs+Hj5k7W0rcZAj405qxOZVgtkrs6cB3uS/pn02eV4EHA6ECReQLrEPFcy6zLZpIChbkzyz372
ZLbwMCSjF5DLh52MSGgWixwvs5Mq20WofBWMOnS0ofnZq6+TM6XK7P8VEQxJe37sWi0W+RrR6685
T+bnlM6GMg24wRHt/1fouUbZQuBgoc0/HNKywlO9BXLoJ9j02/VYn3Uex9bumQIDAQABMA0GCSqG
SIb3DQEBCwUAA4IBAQAmAh0fXL7gU1ivV3hWdl0AlLPENREAzKbHwuthtTySBr6rmreo6j8SvOMW
pKQzNznmzU3zyeLd96j6lfA7PIDGyBGmNB6z0Va0bPvOQe+a2f3/cmumVdrKFv7I5ZiR0UNbeBmG
BIeWkJ+Rx+FcaIzP2IiFddmvpdh1nLae7FS9F1jvnioSIwq2PlFZuMMFb2TrMXrqqEMp9CeGfEag
bjxQcWEW1ifNxeKrI/LcS5g5mTf4gx41bgo/w9x6MRsK+bIbYv680mdtb6LhWiT1lZU+ZAYJTKMr
HHoIxYFPW8Zcs7DGirOOYMbSU97G0rljQzbv9gcS+FhwPffBaHi3spk9</ds:X509Certificate></
ds:X509Data></ds:KeyInfo></
md:KeyDescriptor><md:NameIDFormat>urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified</
md:NameIDFormat><md:NameIDFormat>urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress</
md:NameIDFormat><md:SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-
POST" Location="https://avinetworks-authlab.okta.com/app/avinetworksorg108212_apmssotest_1/
exk2c02xxTcM9pIr0355/sso/saml"/><md:SingleSignOnService
Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect" Location="https://avinetworks-
authlab.okta.com/app/avinetworksorg108212_apmssotest_1/exk2c02xxTcM9pIr0355/sso/saml"/></
md:IDPSSODescriptor></md:EntityDescriptor>’

n Complete the CLI sequence with two save commands.

[admin:saml-ctrlr-1]: authprofile:saml> save

[admin:saml-ctrlr-1]: authprofile> save
+----------------------

VMware by Broadcom 263

VMware Avi Load Balancer Administration Guide

+----------------------------------------------------------------------------------+
| Field |
Value |
+----------------------
+----------------------------------------------------------------------------------+
| uuid |
authprofile-789ce4af-6b9d-4a73-bd26-d00f670a19c0 |
| name |
Saml-auth-profile |
| type |
AUTH_PROFILE_SAML |
| saml
| |
| idp
| |
| metadata |
<?xml version="1.0" encoding="UTF-8"?><md:EntityDescriptor xmlns:md="urn:oasis:n |
| | ames:tc:SAML:2.0:metadata"
entityID="http://www.okta.com/exk2c02xxTcM9pIr0355">< |
| | md:IDPSSODescriptor
WantAuthnRequestsSigned="false" protocolSupportEnumeration=" |
| | urn:oasis:names:tc:SAML:2.0:protocol"><md:KeyDescriptor
use="signing"><ds:KeyInf |
| | o
xmlns:ds="http://www.w3.org/2000/09/xmldsig#"><ds:X509Data><ds:X509Certificate |
| |
>MIIDtjCCAp6gAwIBAgIGAWPuJWSOMA0GCSqGSIb3DQEBCwUAMIGbMQswCQYDVQQGEwJVUzETMBEG A1 |
| |
UECAwKQ2FsaWZvcm5pYTEWMBQGA1UEBwwNU2FuIEZyYW5jaXNjbzENMAsGA1UECgwET2t0YTEU MBIGA |
| |
1UECwwLU1NPUHJvdmlkZXIxHDAaBgNVBAMME2F2aW5ldHdvcmtzLWF1dGhsYWIxHDAaBgkq hkiG9w0B |
| |
CQEWDWluZm9Ab2t0YS5jb20wHhcNMTgwNjExMDkxOTE4WhcNMjgwNjExMDkyMDE3WjCB mzELMAkGA1U |
| | EBhMCVVMxEzARBgNVBAgMCkNhbGlmb3JuaWExFjAUBgNVBAcMDVNhbiBGcmFuY2lz
Y28xDTALBgNVBA |
| | oMBE9rdGExFDASBgNVBAsMC1NTT1Byb3ZpZGVyMRwwGgYDVQQDDBNhdmluZXR3
b3Jrcy1hdXRobGFiM |
| | RwwGgYJKoZIhvcNAQkBFg1pbmZvQG9rdGEuY29tMIIBIjANBgkqhkiG9w0B
AQEFAAOCAQ8AMIIBCgKC |
| | AQEAiFKBy70aa5G2I5JH+uUqXef9jrhUtx6CX1nmrg26FXtsKYdjRm5v
otxbjfNdcXeXRXHu5scMwAg |
| | My9EZM+AXehlm/
qnahNWvEZ+YgPZS55UzkcSXJ30dl62kbUAyXxo3 CQQs+Hj5k7W0rcZAj405qxOZVg |
| | tkrs6cB3uS/
pn02eV4EHA6ECReQLrEPFcy6zLZpIChbkzyz372 ZLbwMCSjF5DLh52MSGgWixwvs5Mq2 |
| | 0WofBWMOnS0ofnZq6+TM6XK7P8VEQxJe37sWi0W+RrR6685
T+bnlM6GMg24wRHt/1fouUbZQuBgoc0/ |
| | HNKywlO9BXLoJ9j02/
VYn3Uex9bumQIDAQABMA0GCSqG SIb3DQEBCwUAA4IBAQAmAh0fXL7gU1ivV3h |
| | Wdl0AlLPENREAzKbHwuthtTySBr6rmreo6j8SvOMW
pKQzNznmzU3zyeLd96j6lfA7PIDGyBGmNB6z0V |
| | a0bPvOQe+a2f3/
cmumVdrKFv7I5ZiR0UNbeBmG BIeWkJ+Rx+FcaIzP2IiFddmvpdh1nLae7FS9F1jvn |
| | ioSIwq2PlFZuMMFb2TrMXrqqEMp9CeGfEag
bjxQcWEW1ifNxeKrI/LcS5g5mTf4gx41bgo/w9x6MRsK |

VMware by Broadcom 264

VMware Avi Load Balancer Administration Guide

| | +bIbYv680mdtb6LhWiT1lZU+ZAYJTKMr
HHoIxYFPW8Zcs7DGirOOYMbSU97G0rljQzbv9gcS+FhwPff |
| | BaHi3spk9</
ds:X509Certificate></ds:X509Data></ds:KeyInfo></md:KeyDescriptor><md: |
| | NameIDFormat>urn:oasis:names:tc:SAML:1.1:nameid-
format:unspecified</md:NameIDFor |
| | mat><md:NameIDFormat>urn:oasis:names:tc:SAML:1.1:nameid-
format:emailAddress</md: |
| | NameIDFormat><md:SingleSignOnService
Binding="urn:oasis:names:tc:SAML:2.0:bindin |
| | gs:HTTP-
POST" Location="https://avinetworks-authlab.okta.com/app/avinetworksorg1 |
| | 08212_apmssotest_1/
exk2c02xxTcM9pIr0355/sso/saml"/><md:SingleSignOnService Bindi |
| | ng="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-
Redirect" Location="https://avinet |
| | works-
authlab.okta.com/app/avinetworksorg108212_apmssotest_1/exk2c02xxTcM9pIr035 |
| | 5/sso/
saml"/></md:IDPSSODescriptor></md:EntityDescriptor> |
| sp
| |
| saml_entity_type |
AUTH_SAML_APP_VS |
| tenant_ref |
admin |
+----------------------
+----------------------------------------------------------------------------------+
[admin:saml-ctrlr-1]: >

Note For more information on CLI access for SAML using Token authentication, see Token
Authentication.

Verify Service Provider Settings for SAML Auth Profile

Avi Load Balancer supports single sign-on (SSO) to the Avi Load Balancer Controller’s UI using
SAML.

Service provider settings can be retrieved by clicking the verify button (Tick icon) on the Auth
Profile list page. This page contains the service provider Entity ID and the Single Sign on URL
generated by the Avi Load Balancer Controller.

VMware by Broadcom 265

VMware Avi Load Balancer Administration Guide

Local Admin or User Logon

After SAML-based access is enabled, you can have the Avi Load Balancer web console display
the logon UI without redirecting to the SAML authentication profile configured on the IdP logon
page. To accomplish this, send a URL to the Controller or cluster IP address in one of the
following two forms:

n https://ControllerIP/#!/login?local=1

n https://FQDN/#!/login?local=1

Configure SAML with Workspace One for Avi Load Balancer

SAML enables integration into VMware Workspace ONE (WS1) and takes advantage of the App
Catalog, network access restrictions, and step-up authentication when administrators sign in. In
this section, you will understand how to configure SAML for Avi Load Balancer control plane
authentication with WS1 access being the IDP.

Note You can configure granular role-based access control by adding application parameters
into the Workspace One Access catalog item and then mapping those parameters to different
roles in Avi Load Balancer. For more information, see Tenant and Role Mapping Examples.

Prerequisites
Before initiating the configuration, complete the following prerequisites:

n Configure a DNS record for the Avi Load Balancer Controller. This will be used for the fully
qualified domain name (FQDN) that is used when signing into the system.

n Get the Workspace ONE Access IDP metadata.

Follow the steps below to download the idp.xml file:

1 Log in to the Workspace ONE Access administrator console.

2 Navigate to the Catalog > Settings.

3 Click SAML Metadata under SaaS Apps.

4 In the Download SAML Metadata tab, click Copy URL next to Identity Provider (IdP)
metadata.

VMware by Broadcom 266

VMware Avi Load Balancer Administration Guide

SAML Configuration in Avi Load Balancer

To configure an authentication profile to support SAML on the Avi Load Balancer Controller,
follow the below:

1 Log in to the Avi Load Balancer Controller with admin credentials.

2 Navigate to the Templates > Security > Auth Profile and click CREATE.

3 Enter the Name of the auth profile.

4 Select SAML as the Type of auth profile.

5 Select Use IDP Metadata URL option and paste the URL in theEnter IDP Metadata URL field.

6 Under Service Provider, select Use DNS FQDN.

7 Enter the service provider organization details, as required.

8 Enter the FQDN to be used for the SAML configuration.

VMware by Broadcom 267

VMware Avi Load Balancer Administration Guide

9 Click Save.

Collecting Service Provider Metadata

Avi Load Balancer does not generate an xml file that can be imported into Workspace ONE
Access. So, the metadata must be entered manually. Collect the following details:

n Entity ID

n SSO URL

n Signing Certificate

Get the entity ID and the SSO URL from the VERIFY SERVICE PROVIDER SETTINGS screen as
shown below.

1 Navigate to Templates > Security > Auth Profile.

2 Identify the authentication profile created and click verify icon.

VMware by Broadcom 268

VMware Avi Load Balancer Administration Guide

3 The VERIFY SERVICE PROVIDER SETTINGS screen displays the Entity ID and the Single
Sign on URL. Copy this information and paste in a text editor.

Get the signing certificate from SSL/TLS Certificates as shown below.

1 From the Avi Load Balancer UI, navigate to Templates > Security > SSL/TLS Certificates.

2 Identify the System-Default-Portal-Cert and click Export icon.

VMware by Broadcom 269

VMware Avi Load Balancer Administration Guide

3 From the Export Certificate screen, click Copy to clipboard to copy the Key and the
Certificate.

4 Paste the details into a text editor.

5 Click DONE.

Configuring Auth Mapping Profile

1 From the Avi Load Balancer UI, navigate to Templates > Security > Auth Mapping Profile.

2 Click CREATE or edit an existing auth mapping rule.

3 Enter Name and Description.

4 Select SAML as the Type.

VMware by Broadcom 270

VMware Avi Load Balancer Administration Guide

Note Depending on the type selected, the auth profile settings are displayed.

5 Under Rules, Click ADD. A new Mapping Rule screen appears.

VMware by Broadcom 271

VMware Avi Load Balancer Administration Guide

6 Under Match, select the filter for the Attribute Match.

a Any: Users match regardless of attributes or their values.

b Contains: The user must have the specified attribute, and the attribute must have one of
the specified values.

c Does Not Contain: The user must not have the specified attribute and value(s).

d Regex:

7 Under Action, select Custom Mapping.

8 Select and configure User Tenant, User Role, or User Account Profile from the ADD drop-
down menu as required. For more information, see Configuring Default Tenant for Remote
Users, User Roles, and User Profile.

9 Click Save.

VMware by Broadcom 272

VMware Avi Load Balancer Administration Guide

Configuring Remote Authentication

By default, a Controller will have only local authentication established (Authentication/
Authorization: Local).

To configure remote authentication using the NSX Advanced Load Balancer UI, follow the steps
below:

1 Navigate to Administration > System Settings > EDIT > Authentication.

2 In the Edit System Settings screen, select Remote as the Authentication method.

3 SelectEnable Local User Login to allow users from the local user database to log in with their
user credentials.

4 Under Auth Profiles & Mapping Profiles, click Add.

5 From the Select Auth Profile drop-down menu, select the previously created remote Auth
Profile.

6 From the Select AUTH Mapping Profile drop-down menu, select an existing Mapping Profile
or click the vertical menu icon (three dots) to create a new Mapping Profile. For more
information, see Configuring Auth Mapping Profile.

7 Click Save.

Note Tenant and role mapping are available only with remote authentication.

Configuring the Avi Load Balancer Catalog Item in Workspace ONE Access
Once the SAML profile is created in the Avi Load Balancer Controller, the Workspace ONE
catalog entry can be created.

To create the Workspace ONE catalog entry, do the following:

1 Log in to your Workspace ONE Access administrator console.

VMware by Broadcom 273

VMware Avi Load Balancer Administration Guide

2 Navigate to the Catalog tab.

3 Click New.

4 In the New SaaS Application screen, enter a Name for the new Avi Load Balancer entry in the
App Catalog.

5 Under the Definition tab, if you have an icon to use, click Select File, upload the icon for the
application, and click Next to view the Configuration tab.

6 In the Configuration tab, enter the details as shown below:

Field Description

Single Sign-On URL Use the Single Sign on URL copied from the VERIFY
SERVICE PROVIDER SETTINGS screen in Avi Load
Balancer.

Note The trailing slash (/) after acs is mandatory.

Recipient URL Use the Single Sign-On URL.

Application ID Use the Entity ID copied from the VERIFY SERVICE

PROVIDER SETTINGS screen in Avi Load Balancer.

Username Format Unspecified.

Username Value ${user.email}.

Relay State URL The FQDN or IP address of your appliance.

The Configuration tab in the New SaaS Application screen is as shown below.

VMware by Broadcom 274

VMware Avi Load Balancer Administration Guide

7 Click Advanced Properties and configure the properties as shown below:

8 Copy the value of the System-Default-Portal-Cert certificate and paste it into the Request
Signature field.

VMware by Broadcom 275

VMware Avi Load Balancer Administration Guide

9 Enter the FQDN or IP address of the appliance as the Application Login URL. This enables
SP-initiated login workflows.

10 Click Next to select Access Policies to use for this application. This determines the rules used
for authentication and access to the application.

11 Click Next to review the Summary of the configuration.

VMware by Broadcom 276

VMware Avi Load Balancer Administration Guide

12 Click Save & Assign and select the users or groups that will have access to this application
and the deployment type.

13 Click Save.

Troubleshooting
This section explains the troubleshooting techniques for Authentication profile types.

SAML Authentication
Create Application on the IdP

A SAML application needs to be created on the IdP using the Controller-generated

information. The Entity ID and the Single Sign on URL required while creating the application
on IdP need to precisely match the Avi Load Balancer-generated configuration.

In the case of certain identity providers, IdP metadata can be retrieved after the SAML
application has been created. In those cases, the recommended workflow is to configure
the JumpCloud SAML application first and use the Avi Load Balancer-generated attributes to
create the SAML application.

Once the application has been created, the IdP metadata can be plugged into the
authentication profile. The authentication profile cannot be attached to the system
configuration without valid IdP metadata.

Note In Avi Load Balancer, both SAML assertion and response signing are mandatory for
successful SAML authentication.

VMware by Broadcom 277

VMware Avi Load Balancer Administration Guide

Avi Load Balancer has verified interoperability with the Google, Okta, and OneLogin IDPs.
Contact Avi Load Balancer sales team if you require integration with other IDPs.

When User tries to Authenticate with SSO and Encounters User has no privileges Error
Message
Following are a few reasons for the issue:

1 When group attributes are not retrieved from the backend.

2 When group attributes are retrieved from the backend, but no rule corresponding to that
attribute is created.

This issue can be fixed by enabling the option in JumpCloud SAML application to include
group attribute in the SAML requests.

Auth Mapping Profile

You can assign roles to users if LDAP, TACACS+, or SAML remote authentication is used.

Configuring LDAP or TACACS+ Users to Roles

Prerequisites

n Avi Load Balancer authentication or authorization must be set to remote, and an LDAP or
TACACS+ Auth profile is applied.

n Group names are case-sensitive for LDAP mapping.

Roles are assigned to users based on the following:

LDAP group

A role is assigned to users in any group or specifically to users who are either members or
not members of specific groups.

LDAP attributes

For users who match the LDAP group filter, the role is assigned to those who either do or do
not have specific attributes and values.

The mappings are configured within Avi Load Balancer rather than the LDAP or TACACS+ server.

To map LDAP or TACACS+ users to Avi Load Balancer roles, use the following steps.
Multiple mappings can be configured if needed for any combination of user group name and
attribute:value pair.

1 Navigate to Templates > Security > Auth Mapping Profile and click CREATE.

2 In the CREATE AUTH MAPPING PROFILE under the General tab, enter the profile Name.

VMware by Broadcom 278

VMware Avi Load Balancer Administration Guide

3 Select the Type of auth mapping profile to which these rules are applied and click ADD
button. Select LDAP to define LDAP Group Match.

Note Depending on the type selected, the auth profile settings are displayed.

4 Select the filter for the LDAP Group Match.

n Any: Users in any LDAP group match the filter.

n Member: Users must be members of the specified groups. When specifying a group, you
must use the common name (CN). If entering multiple group names, use commas between
the names.

n Not a Member: Users must not be members of the specified groups.

5 Select the filter for the Attribute Match.

n Any: Users match regardless of attributes or their values.

n Contains: The user must have the specified attribute, and the attribute must have one of
the specified values.

n Does Not Contain: The user must not have the specified attribute and value(s).

n Regex: Match succeeds if the regex matches any substring of the attribute or group.

6 Under Action, select Custom Mapping.Select User Role from the ADD drop-down menu.

7 Select the Type from the drop-down menu.

n All: Assigns all roles.

n Attribute Regex: Match succeeds if the regex matches any substring of the attribute.

n Attribute Value: Assigns the role whose name matches an attribute value from the LDAP
server.

Note Attribute Match Value is case sensitive. If there is a mismatch, the following error
message will be displayed: DEBUG [auth_rules.update_user_access:502] ldap: NO
Tenant/Role assigned

n Group Name: Assigns the role whose name matches a group name on the LDAP server.

n Group Regex: Match succeeds if the regex matches any substring of the group.

n Selected: Displays a Roles drop-down menu. Select the role from the menu.

8 If using multitenancy, users also can be mapped to tenants. Tenants can be mapped by
selecting User Tenant from the ADD drop-down menu. For more information, see Chapter 6
Tenant Settings.

9 Click Save. The new mapping appears in the Tenant and Role Mapping table.

Configuring SAML User to Roles

1 From the Avi Load Balancer UI, navigate to Templates > Security > Auth Mapping Profile.

VMware by Broadcom 279

VMware Avi Load Balancer Administration Guide

2 Click CREATE or edit an existing auth mapping rule.

3 Enter Name and Description.

4 Select SAML as the Type.

Note Depending on the type selected, the auth profile settings are displayed.

5 Under Rules, Click ADD. A new Mapping Rule screen appears.

VMware by Broadcom 280

VMware Avi Load Balancer Administration Guide

6 Under Match, select the filter for the Attribute Match.

a Any: Users match regardless of attributes or their values.

b Contains: The user must have the specified attribute, and the attribute must have one of
the specified values.

c Does Not Contain: The user must not have the specified attribute and value(s).

d Regex:

7 Under Action, select Custom Mapping.

9 Click Save.

VMware by Broadcom 281

VMware Avi Load Balancer Administration Guide

Tenant and Role Mapping Examples

Remote Auth requires the assignment of roles and tenants for every user login through the
authorization mapping rules. Authorization is assessed on every login, and the user record is
updated. Upon successful user login through an external authentication server, all mapping rules
are evaluated.Tenant and role pairs are added to the user access list.

Avi Load Balancer supports user profile mapping for remote users.

Configuring Remote Authentication

By default, a Controller will have only local authentication established (Authentication/
Authorization: Local).

To configure remote authentication using the Avi Load Balancer UI, follow the steps below:

1 Navigate to Administration > System Settings > EDIT > Authentication.

2 In the Edit System Settings screen, select Remote as the Authentication method.

3 Select Enable Local User Login to allow users from the local user database to log in with their
user credentials.

4 Under Auth Profiles & Mapping Profiles, click Add.

5 From the Select Auth Profile, select an existing remote auth profile or click the vertical menu
icon (three dots) to create a new auth profile.

6 Click Save.

Note Tenant and role mapping are available only with remote authentication.

VMware by Broadcom 282

VMware Avi Load Balancer Administration Guide

Multiple remote authentication profiles can be configured using the Avi Load Balancer UI under
the following conditions:

n Any number of auth profiles (more than 2) can be created provided they are of the same
type. Consider the following examples to understand this further:

Profile Types Result Reason

n TACACS Successful All profiles are of the same type

n TACACS
n TACACS

n TACACS Fails with an error All the profiles are not of the same
n TACACS type

n LDAP

n If there are only two auth profiles configured,

n The primary auth profile has to be SAML auth profile.

n Both profiles cannot be SAMl. So if SAML is the first auth profile, then the second profile
has to be any profile other than SAML

Consider the following examples to understand this further:

Primary Auth Profile Secondary Auth Profile Result Reason

SAML LDAP Successful The primary and the

secondary profiles are of
different types.

SAML SAML Fails with an error The primary and secondary

profiles are both of type
SAML.

LDAP TACACS Fails with an error The primary auth profile is

not of type SAML.

Any Group or Any Attribute Rule

A rule with any group or any attribute applies to all users and can be used as a default option.
The rule assigns every user to a least-privileged role and tenant.

Note The role and tenant need to configured to allow only the least privileges.

If the user is not assigned additional roles or tenant pairs, the least privileged access will take
effect after login.

VMware by Broadcom 283

VMware Avi Load Balancer Administration Guide

Super User Rule

A rule can be configured to assign Super User privileges to a user. This user will have access
to all tenants with the most privileged role. Once a user is super user, no other tenant or role
mapping assignments will make a difference to the user’s access.

VMware by Broadcom 284

VMware Avi Load Balancer Administration Guide

Attribute and Group Match

A mapping rule can be required to match both an attribute and group requirement. It will ensure
a more specific assignment of role(s) and tenant(s).

VMware by Broadcom 285

VMware Avi Load Balancer Administration Guide

Assign Matching Attribute Values

LDAP/TACACS+ attribute vantageRole for a user can have one or more values. For each value,
if there is a configured role with the same name, the role is assigned to the user with access to
all tenants. A user session can end up with multiple roles, and the most privileged role will take
effect.

VMware by Broadcom 286

VMware Avi Load Balancer Administration Guide

Assign Matching Group Names

A user can be a member of multiple LDAP or AD groups. For each group, if there is a configured
tenant, the user will be given access to the tenant with any other tenants the user might already
have obtained access to using matching rules.

Configuring Default Tenant for Remote Users

When configuring auth mapping rules, you can now select default tenants. When a remote user is
created, the tenant selected will be used as the default tenant.

1 From the Avi Load Balancer UI, navigate to Templates > Security > Auth Mapping Profile.

2 Click CREATE or edit an existing auth mapping rule.

3 Enter Name and Description.

VMware by Broadcom 287

VMware Avi Load Balancer Administration Guide

4 Select the Type as required.

5 Under Rules, Click ADD.

6 Under Action, select Custom Mapping.

7 Click ADD and select User Tenant drop-down menu

8 From Type drop-down menu, choose Selected.

9 Select the required Tenants.

Note If multiple tenants are created in the mapping rules, the first tenant selected is taken as
the default tenant.

10 Click Save.

Alternatively, default tenant for remote users can be configured using the field
default_tenant_ref through the CLI as shown below:

[admin:10-102-64-241]: authmappingprofile:mapping_rules> default_tenant_ref admin

[admin:10-102-64-241]: authmappingprofile:mapping_rules> save
[admin:10-102-64-241]: authmappingprofile> save
+----------------------+---------------------------------------------------------+
| Field | Value |
+----------------------+---------------------------------------------------------+
| uuid | authmappingprofile-46400ba5-9f05-4ec9-9464-e188010ce7b7 |

VMware by Broadcom 288

VMware Avi Load Balancer Administration Guide

Default tenant mapping based on the tenant selected, is as shown below:

User Tenant Behavior Default Tenant Description

All Admin All tenant values Any user-specified tenant

is used. In the absence
of user-defined tenants,
admin is selected as the
default tenant.

Admin Admin All tenant values Any user-specified tenant

is used. In the absence
of user-defined tenants,
admin is selected as the
default tenant.

Selected Tenant ref Only selected tenant values Any User-specified Tenant
that is part of the selected
tenant list is used. If not,
Must check will display an
error while creating a new
mapping.
For example, if the selected
tenant list has T1 and T2
but the user has provided
T3 as the default tenant,
then the must check will
display the following error:
Default tenant is not
in selected tenants
list.
If the user has not
selected any tenant but has
provided default tenant ref,
then the error Please add
at least one tenant
in the selected list is
displayed.

Attribute Regex Tenant ref All tenant values User-specified tenant.

VMware by Broadcom 289

VMware Avi Load Balancer Administration Guide

User Tenant Behavior Default Tenant Description

Attribute value Tenant ref All tenant values User-specified tenant.

Group name Tenant ref All tenant values User-specified tenant.

Note In case of Attribute Regex, Attribute value, Group Name, if the user has entered a tenant
that is not part of the user tenant list (retrieved at runtime on the basis of Attribute Regex,
Attribute Value, Group Name, Group Regex match), then the first entry in the retrieved list will be
assigned as the default tenant from the tenant list. For example, User1 can be assigned T1 and T2,
but T3 is provided as the default tenant. Depending on the first entry in the received tenant list,
either T1 or T2 will become the default tenant.

Example: Multiple Groups Mapping to Different Roles

This example illustrates the case of an IT team with three user groups — super-admins, app-
admins, and operations — where the following applies:

User Groups Roles

Super Admins The users can access all tenants and all settings.

Application Admins n The users can only create, read, update, and delete
virtual services and other profiles.
n The users can not create clouds.

Application Operators The users has read-only access.

n Navigate to Templates > Security > Auth Mapping Profile > CREATE. Select Type and click
ADD under the Rules section.

VMware by Broadcom 290

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 291

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 292

VMware Avi Load Balancer Administration Guide

n Navigate to Administration > System Settings > EDIT. Under

Authentication, select Remote and click ADD.

VMware by Broadcom 293

VMware Avi Load Balancer Administration Guide

Separate mapping rules are required to map users from each group to different role or tenant
assignments.

Example: Multiple Groups Mapping to Different Tenants

This example illustrates settings for an IT team that expects tenant isolation except for a few
super users.

User Groups Roles

Super Admins The users can access all tenants and all settings.

Tenant Application Admins The users have access to a few tenants — app owner for
a few tenants.

Tenant Application Operators The users have access to a few tenants — cannot modify
anything.

Tenant Application Admins or Operators The users have access to a few tenants as app owners
and other tenants as app operator folks.

VMware by Broadcom 294

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 295

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 296

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 297

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 298

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 299

VMware Avi Load Balancer Administration Guide

In this example, members of group Service Admins E have read or write access (Application-
Admin role) in tenants Tenant AE and Tenant SE, whereas they have read-only access
(Application-Operator role) in a few other tenants. Service Operators have only read-only access
in their respective tenants.

Example: Multiple Authorizations for a Single User

In this example, the login of user John Doe results in the user gaining access through multiple
authorization mapping rules.

Multiple mapping rules are configured based on various group and attribute criteria.

VMware by Broadcom 300

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 301

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 302

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 303

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 304

VMware Avi Load Balancer Administration Guide

The LDAP server is configured with user John Doe.

VMware by Broadcom 305

VMware Avi Load Balancer Administration Guide

The LDAP server can be configured with John Doe as a member of the groups - Enterprise
Admins and Service Operators.

After the user John Doe logs in, and all authorization rules are applied to the user session.
Multiple role or tenant combinations are used to determine user privileges during user login. The
user record shows the user successfully matched all four rules and role or tenant pairs were
appropriately applied.

Example: Multiple Authorizations Resulting in a Super User

In this example, the login of user John Doe results in the user becoming a super user.

Mapping rules make a member of the group Service Operators a super user.

VMware by Broadcom 306

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 307

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 308

VMware Avi Load Balancer Administration Guide

Due to the super user access, user John Doe gets access to all tenants with every role.

Example: No Authorizations for a Single User

In this example, the login of user John Doe results in the user not getting any roles or tenants.

Mapping rules are updated to keep user John Doe from having any privileges.

VMware by Broadcom 309

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 310

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 311

VMware Avi Load Balancer Administration Guide

When user John Doe logs in, the user interface reports no privileges to login.

VMware by Broadcom 312

VMware Avi Load Balancer Administration Guide

User record does not have any access entries.

[: > show user jdoe

Dynamic Tenant and Role Mappings

Avi Load Balancer supports dynamically mapping tenants and/or roles based on regular
expression matching against the user’s group memberships or user LDAP attributes.

The mapping is achieved using regex Named Capture Groups of the form (?P{tenant|
role}<match>).

VMware by Broadcom 313

VMware Avi Load Balancer Administration Guide

In the following example, the user is a member of the LDAP groups lb_app1234_admin and
lb_app7890_admin, where app1234 and app7890 correspond to the names of tenants defined
in Avi Load Balancer. The example will grant the user the specific Tenant-Admin role within these
tenants.

configure authmappingprofile authmapping

[admin:avicontroller]: > configure authmappingprofile authmapping
[admin:avicontroller]: authmappingprofile> mapping_rules index 1
New object being created
[admin:avicontroller]: authmappingprofile:mapping_rules> group_match
[admin:avicontroller]: authmappingprofile:mapping_rules:group_match> criteria auth_match_regex
[admin:avicontroller]: authmappingprofile:mapping_rules:group_match> groups "lb_(?P{tenant}
\w+)_admin"
[admin:avicontroller]: authmappingprofile:mapping_rules:group_match> save
[admin:avicontroller]: authmappingprofile:mapping_rules> assign_tenant
assign_matching_group_regex
[admin:avicontroller]: authmappingprofile:mapping_rules> assign_role assign_from_select_list
[admin:avicontroller]: authmappingprofile:mapping_rules> role_refs Tenant-Admin
[admin:avicontroller]: authmappingprofile:mapping_rules> save

Roles may also be dynamically mapped using the ?P{role} Named Capture Group. The example
below will map a user that is a member of a group named lb_app1234_appowner to the role
appowner and tenant app1234:

configure authmappingprofile authmapping

[admin:avicontroller]: > configure authmappingprofile authmapping
[admin:avicontroller]: authmappingprofile> mapping_rules index 1
New object being created
[admin:avicontroller]: authmappingprofile:mapping_rules> group_match
[admin:avicontroller]: authmappingprofile:mapping_rules:group_match> criteria auth_match_regex
[admin:avicontroller]: authmappingprofile:mapping_rules:group_match> groups "lb_(?P{tenant}
\w+)_(?P{role}\w+)"
[admin:avicontroller]: authmappingprofile:mapping_rules:group_match> save
[admin:avicontroller]: authmappingprofile:mapping_rules> assign_tenant
assign_matching_group_regex
[admin:avicontroller]: authmappingprofile:mapping_rules> assign_role
assign_matching_group_regex
[admin:avicontroller]: authmappingprofile:mapping_rules> save

Similarly, regex-based mapping based on the value of an LDAP attribute rather than group
membership is also supported. The example below will map a user with LDAP attribute
business_unit with value bu_sales to the tenant sales:

configure authmappingprofile authmapping

VMware by Broadcom 314

VMware Avi Load Balancer Administration Guide

P{tenant}\w+)"
[admin:avicontroller]: authmappingprofile:mapping_rules:attribute_match> save
[admin:avicontroller]: authmappingprofile:mapping_rules> assign_tenant
assign_matching_attribute_regex
[admin:avicontroller]: authmappingprofile:mapping_rules> tenant_attribute_name business_unit
[admin:avicontroller]: authmappingprofile:mapping_rules> assign_role assign_from_select_list
[admin:avicontroller]: authmappingprofile:mapping_rules> role_refs Tenant-Admin
[admin:avicontroller]: authmappingprofile:mapping_rules> save

Note: Currently group-based regex matching can only be configured through the CLI/API.

User Profile Mapping

Using user profile mapping, it is possible to choose a user profile for remote users based on
certain conditions.

Use the following CLI to configure the user profile:

[admin:123]: systemconfiguration> admin_auth_configuration

[admin:123]: systemconfiguration:admin_auth_configuration> mapping_rules index 1
[admin:123]: systemconfiguration:admin_auth_configuration:mapping_rules> assign_userprofile
assign_from_select_list
[admin:123]: systemconfiguration:admin_auth_configuration:mapping_rules> userprofile_ref
Tacacs-Userprofile
[admin:123]: systemconfiguration:admin_auth_configuration:mapping_rules> save

Note Ensure that the user profile is already created. For more information on how to create and
configure a user profile, see User Profile.

View the user profile configuration as shown below.

[admin:123]: > show systemconfiguration

|----------------------------------|------------------------------------|
|Field | Value |
|----------------------------------|------------------------------------|
| Truncated Output
|
| admin_auth_configuration | |
| auth_profile_ref | tacacs1 |
| mapping_rules[1] | |
| index | 1 |
| assign_tenant | ASSIGN_FROM_SELECT_LIST |
| tenant_attribute_name | |
| tenant_refs[1] | admin |
| assign_role | ASSIGN_FROM_SELECT_LIST |
| role_attribute_name | |
| role_refs[1] | Application-Admin |
| is_superuser | False |
| assign_userprofile | ASSIGN_FROM_SELECT_LIST |
| userprofile_ref | Tacacs-Userprofile |
| allow_local_user_login | True |
|----------------------------------|------------------------------------|

VMware by Broadcom 315

VMware Avi Load Balancer Administration Guide

User Account Security

The Avi Load Balancer includes several mechanisms to protect the control plane from overload
or abuse. Some of these are based on feedback from penetration test findings.

Details about these mechanisms are discussed in the following sections:

n Super User Accounts

n Strong Password Enforcement

n Password History Enforcement

n Password-less SSH-based Login for Admin User

n User Account Lockout

n Maximum Concurrent Login Sessions

n User Credentials Timeout

User Account Self-Service

This topic explains the steps to configure the user account.

Editing User Account

When logged into the web interface, an Avi Load Balancer user can view or change information
about their own account.

This applies only to accounts in local user database of Avi Load Balancer.

To configure your user account, use the following steps.

1 Log in to the web interface.

2 Click the user icon in the top, right corner of the application.

3 Click My Account.

4 In the EDIT MY ACCOUNT screen, enter the Name and Username.

5 Enter the Email address. This address is used by Avi Load Balancer when you request a
password recovery.

6 To change the password, enter the Old Password, the New Password, and re-enter the New
Password for confirmation.

7 Select the Time Zone. The format can be UTC Time or Local Time. The Avi Load Balancer
obtains the time from the specified NTP server. This format is used in the timestamps that
appear in logs or metrics.

8 Select the Language of display.

Note The UI supports Japanese and Simplified Chinese along with the default option English.

VMware by Broadcom 316

VMware Avi Load Balancer Administration Guide

9 Select Enable Tech Preview for Virtual Service Logs checkbox to replace the existing legacy
view. This option is deselected by default.

10 Under Session Timeout, enter the maximum number of minutes a management session can
remain idle before being terminated by the Controller.

11 Under DNS Refresh Period, enter the period for refresh pool and GSLB DNS job in minutes.

12 Click Save.

Configuring Idle Timeout for Admin Access to GUI and API

By default, the Controller terminates idle connections to the GUI and API. You can configure this
idle connection timeout value as a global setting, applied to all administrators across all tenants
for the Controller.

Only administrators with write access can modify this parameter by navigating to Administration
> Controller.

The account can timeout based on the global settings defined by an administrator with
appropriate privileges.

You can change this setting through the GUI by logging in to the Avi Load Balancer UI and
navigating to My Account > Session Timeout.

You can change this setting through the CLI by navigating to show controller properties
api_idle_timeout.

User Account Lockout

This feature prevents users from logging in after 20 failed attempts. The user account is locked
out for 30 minutes after the last failure login attempt. If the account has not been locked, the
running count of failed login attempts is reset to 0 after a valid login.

The administrator controls this feature through the Avi Load Balancer CLI or REST API. The
setting for it is maintained within the UserAccountProfile object. By default, all the users in
the system are attached to Default-User-Account-Profile, as shown in the following example.
If required, the admin can create a new user account profile with different thresholds.

Note This feature can be deactivated by setting the max_login_failure_count to zero.

admin:10-10-24-52]: > show useraccountprofile Default-User-Account-Profile

VMware by Broadcom 317

VMware Avi Load Balancer Administration Guide

| credentials_timeout_threshold | 0 |
+-------------------------------+---------------------------------------------------------+

To change user account lockout attributes, use the following.

[admin:10-10-24-52]: > configure useraccountprofile Default-User-Account-Profile

Updating an existing object. Currently, the object is:
[admin:10-10-24-52]: useraccountprofile> max_login_failure_count 30
Overwriting the previously entered value for max_login_failure_count
[admin:10-10-24-52]: useraccountprofile> account_lock_timeout 60
Overwriting the previously entered value for account_lock_timeout
[admin:10-10-24-52]: useraccountprofile> save
+-------------------------------+---------------------------------------------------------+
| Field | Value |
+-------------------------------+---------------------------------------------------------+
| uuid | useraccountprofile-6753548e-7ac5-4601-939b-ad4394405db4 |
| name | Default-User-Account-Profile |
| max_password_history_count | 0 |
| max_login_failure_count | 30 |
| account_lock_timeout | 60 |
| max_concurrent_sessions | 0 |
| credentials_timeout_threshold | 0 |
+-------------------------------+---------------------------------------------------------+

User Credentials Timeout

The admin can choose to expire user credentials after a configurable number of days. Once
credentials have expired, all API calls will error out. Only API or user account is supported at this
point to enable the user to change the password. If the user has configured an email address,
Forgot Password workflow can also be followed to reset the password.

The administrator controls this feature through the Avi Load Balancer CLI or REST API. The
setting for it is maintained within the UserAccountProfile object. By default, all the users in
the system are attached to Default-User-Account-Profile, as shown in the following example. If
required, the admin can create a new user account profile with different thresholds.

VMware by Broadcom 318

VMware Avi Load Balancer Administration Guide

Strong Default Admin Password

The initial default password of the admin user of the Controller is set to a strong password. This
password is available in the Avi Load Balancer portal, where release images are uploaded and are
accessible only to users having an account on the portal.

SSH access to the Controller with this default password is not allowed until the user changes the
default password of the admin user. Once the password is changed, SSH access to the admin
user is permitted. The primary reason behind this change is to mitigate the risk of having a
newly created Controller being attacked through SSH using the simple password. The password
can be changed through the Welcome screen in Avi Load Balancer UI, REST API, or through a
remote CLI shell. This change strikes a good balance between security and ease of automation
and keeps the existing automation workflows, such as the Avi Load Balancer UI welcome screen,
adding the Controller nodes to a cluster unchanged.

Users must change the password of the admin user to a strong password first before any other
change is implemented in the Controller.

Using the UI
If you use the Avi Load Balancer UI, a welcome screen is launched. The user’s first step is to
change the admin user password.

Using the CLI

To change the password using CLI, you need to launch a remote CLI shell as follows.

avi_shell --addr 10.10.25.44

[admin:10-10-25-44]: > passwd

Changing password for admin
(current) password:
Enter new password:
Retype new password:
Password successfully changed
[admin:10-10-25-44]: >

VMware by Broadcom 319

VMware Avi Load Balancer Administration Guide

Using the REST API

If you use the SDK or API directly, establish a login session with the Controller using the default
password and execute the API calls below to change the password.

PUT https://<controller-ip>/api/useraccount DATA: {“old_password”: <default password>,

“password”: <new password>}

Strong Password Enforcement

The default deployment of Avi Load Balancer creates an admin account for access to the
system. This initial account does not mandate any specific password requirements. Additional
user accounts can be created, either local (username/password) or remote accounts (which are
tied into an external auth system, such as LDAP). For local accounts, it is possible to enable
strong password enforcement. Enabling this option does not impact the passwords of existing
accounts. It only impacts newly created accounts, or existing accounts that are attempting to
change their password.

The strong password enforcement feature does not affect remotely authenticated accounts. It
also does not impact the password requirements for the underlying Linux operating system of
the Avi Load Balancer.

Password Requirements (Strong Password Enforcement Enabled)

Following are the list of password requirements:

n Minimum of 8 characters

n It must contain at least one occurrence of three of the following four categories:

n Uppercase letters

n Lowercase letters

n Digits

n Special characters

n Dictionary words are not allowed.

n Passwords that are easy to guess. For example, the password !QAZxsw2 is not allowed
because it is entered using 8 adjacent keys on the keyboard.

You can specify the minimum permissible password length when password complexity is
enforced. The configurable minimum password length is 8 characters by default, but can range
from 6 to 32 characters.

You can configure minimum password length using the following CLI command.

[admin]: > configure systemconfiguration

[admin]: systemconfiguration > portal_configuration
[admin]: systemconfiguration:portal_configuration > minimum_password_length <value>

VMware by Broadcom 320

VMware Avi Load Balancer Administration Guide

Strong password enforcement is enabled by default. You can deactivate it if required.

Note Only an account that has the System Administrator role can change this setting.

Enabling Strong Password Enforcement

Strong password enforcement can be enabled using the CLI commands shown below.

bash# shell
: > configure systemconfiguration
: systemconfiguration> portal_configuration
: systemconfiguration:portal_configuration> password_strength_check
Overwriting the previously entered value for password_strength_check
: systemconfiguration:portal_configuration> exit
: systemconfiguration> exit

The following is the truncated view of the results.

Deactivating Strong Password Enforcement

Warning Deactivating strong password enforcement is not recommended.

bash# shell
: > configure systemconfiguration
: systemconfiguration> portal_configuration
: systemconfiguration:portal_configuration> no password_strength_check
Overwriting the previously entered value for password_strength_check
: systemconfiguration:portal_configuration> exit
: systemconfiguration> exit

Password History Enforcement

This feature prevents a user from updating their password to some password that was used in
the recent past. A configurable number of previous password hashes are saved in the system.
Any future proposed password update is compared against this list and marked invalid if there is
a perfect match.

VMware by Broadcom 321

VMware Avi Load Balancer Administration Guide

The administrator controls this feature through Avi Load Balancer CLI or REST API. The settings
for this feature are maintained within the UserAccountProfile object. By default, all the users in
the system are attached to Default-User-Account-Profile, as shown below. If required, the admin
can create a new user account profile with different thresholds.

admin:10-10-24-52]: > show useraccountprofile Default-User-Account-Profile

Use the following CLI to change the password history count.

[admin:10-10-24-52]: > configure useraccountprofile Default-User-Account-Profile

Password-less SSH-based Login for Admin User

The Avi Load Balancer enables the admin user to log in using SSH keys, instead of having to
supply a password. This section details ways for password-less SSH-based login for admin user.

Password-less Login using Avi Load Balancer CLI

n To upload a key, use the following command.

upload adminkey public_key "<public key>

n To delete a specific key, use the following command.

delete adminkey public_key "<public key>"

n To delete all the keys, use the following command.

VMware by Broadcom 322

VMware Avi Load Balancer Administration Guide

delete adminkey

Password-less Login using Avi Load Balancer Rest API

n To upload a key, usee the following API request.

POST https://<controller-ip>/api/adminkey JSON data: {"key":"<public key>"}

n To delete a specific key, use the following API request.

DELETE https://<controller-ip>/api/adminkey?key=<public key>

n To delete all the keys, use the following API call.

DELETE https://<controller-ip>/api/adminkey?key

Password Recovery
This section discusses ways to recover passwords for different user types.

Reset a Known Password

Locally authenticated users can change their own password by logging in, clicking their user
name in the top-right corner of the GUI, and then clicking the My Account selection from the
menu. When the user account editor appears, the user enters the old password followed by the
new.

Note An administrator can change passwords of other users by navigating to Administration >
Accounts > Users, but can change their own password only through My Account page.

Local User Password Recovery

If a locally-authenticated user forgets their password, the following two options are available:

User-initiated

If SMTP email is configured for Avi Load Balancer, locally authenticated users can make
a reset-password request through the GUI. In the Avi Load Balancer UI, the Forgot Your
Password? link is available below the Password field. Clicking the link prompts the user for
the email address. If the provided email address is configured through the Administration >
Account > Users, an email containing a link to reset the password is sent to the email id.

If SMTP is not configured, the Forgot your password? link does not appear.

Note The user account must have a valid email configured for the account, and Avi Load
Balancer must have access to an SMTP server.

Admin-initiated

VMware by Broadcom 323

VMware Avi Load Balancer Administration Guide

Any administrator or user who has write privileges to user objects can change the password
for a local user. From the admin account, navigate to Administration > Account > Users and
edit the account to be reset. Input a new password or select the Generate button to create a
random password for the user.

Note Even if the password is reset on saving, the administrator must still copy and manually
send this new password to the user.

Remote User Password Recovery

Password recovery for users who are remotely authenticated through LDAP or TACACS must be
performed with the remote auth server. With remote authentication, the Avi Load Balancer does
not maintain user passwords and does not participate in the password recovery process.

Admin User Password Recovery

The procedure to reset the administrator password is the same as any locally-authenticated user.

Extended Granular RBAC

This section explains extended granular Role Based Access Control (RBAC) feature.

Note The terms labels and markers are used interchangeably throughout the section.

n Multiple values (value 1, value 2, value 3…), mapped to a single key for an object.

Example:

User 1 has to transfer a pre-prod application to prod, for which User 2 has access.

The object has label “app”:[“pre-prod”].

User 1 has access to “app”:”pre-prod”.

User 1 needs to transfer this object to “prod“.

In this case, the object has to be configured with “key” : “list of labels” for transferring the
application.

Any user who has access to the object with labels configured in the role, can use additional
labels which the user does not have access to.

Here, an additional label has to be applied for the object.

“app“: [“pre-prod”, “prod”] After the transfer, the new app owner (User 2) can remove “app“:
“pre-prod“ from the object to discontinue access to User 1.

n The access privilege granted by setting the allow_unlabelled_ access to true in the role
object depends on the privilege type of the role.

n A new object, LabelGroup is introduced to track the labels allowed or suggested for a given
tenant. For each tenant, there can be associated label groups. At the tenant level, the labels
can either be enforced or deprecated.

VMware by Broadcom 324

VMware Avi Load Balancer Administration Guide

Using Markers
The steps to configure the object using markers are as follows.

Configuring the Object

Let us consider configuring the pool object pool-123 with two owners, engineer, and marketing.
Here, “Key”: [“value1”, “value2”] :: “Owner”: [“eng”, “marketing”].

[admin:ctrl10]: > configure pool pool-123

[admin:ctrl10]: pool> markers
New object being created
[admin:ctrl10]: pool:markers> key owner
[admin:ctrl10]: pool:markers> values eng
[admin:ctrl10]: pool:markers> values marketing
[admin:ctrl10]: pool:markers> save
[admin:ctrl10]: pool> save

The pool configuration shows that the key and the corresponding values are assigned as shown
below.

VMware by Broadcom 325

VMware Avi Load Balancer Administration Guide

Creating Roles
Create the Role named Eng with write access to the pool object.

[admin:ctrl10.79.169.184]: > configure role role-eng

[admin:ctrl10.79.169.184]: role> privileges
New object being created
[admin:ctrl10.79.169.184]: role:privileges> type write_access
[admin:ctrl10.79.169.184]: role:privileges> resource permission_pool
[admin:ctrl10.79.169.184]: role:privileges> save
[admin:ctrl10.79.169.184]: role> filters
New object being created
[admin:ctrl10.79.169.184]: role:filters> match_operation role_filter_glob_match
[admin:ctrl10.79.169.184]: role:filters> match_label
[admin:ctrl10.79.169.184]: role:filters:match_label> key owner
[admin:ctrl10.79.169.184]: role:filters:match_label> values *eng*
[admin:ctrl10.79.169.184]: role:filters:match_label> save
[admin:ctrl10.79.169.184]: role:filters> save
[admin:ctrl10.79.169.184]: role> no allow_unlabelled_access
[admin:ctrl10.79.169.184]: role> save

The role is viewed as shown below.

Note For this role, allow_unlabelled_access is disabled. This means, the unlabelled objects are
not visible to the user. For unlabelled objects to be visible, this option has to be set to True.

VMware by Broadcom 326

VMware Avi Load Balancer Administration Guide

Similarly, the role marketing can be configured with the required permissions to the object.

Creating a Label Group

Create label group-123 which is a new object that holds a list of [“key1”: [“value1”, “value2’,
“value3”, …].

[admin:ctrl]: > configure labelgroup labelgroup-123

[admin:ctrl]: labelgroup> labels
New object being created
[admin:ctrl]: labelgroup:labels> match_operation role_filter_equals
[admin:ctrl]: labelgroup:labels> match_label
[admin:ctrl]: labelgroup:labels:match_label> key owner
[admin:ctrl1]: labelgroup:labels:match_label> values eng
[admin:ctrl1]: labelgroup:labels:match_label> values marketing
[admin:ctrl1]: labelgroup:labels:match_label> values testing
[admin:ctrl1]: labelgroup:labels:match_label> save
[admin:ctrl1]: labelgroup:labels> save
[admin:ctrl1]: labelgroup> save

The label group object is as shown below.

Associating Label Group to a Tenant

[admin:ctrl]: > configure tenant t-1
[admin:ctrl]: tenant> enforce_label_group
[admin:ctrl]: tenant> label_group_refs labelgroup-123
[admin:ctrl]: tenant> save

The configured tenant is as shown below.

VMware by Broadcom 327

VMware Avi Load Balancer Administration Guide

Creating an object with markers that does not qualify the assigned key:value rules in the label
group, displayed as error.

For example, if the pool object is configured with the marker “Key”: [“sales”], an error is
displayed as shown below:

[admin:ctrl]: > configure pool pool-4

[admin:ctrl]: pool> markers
New object being created
[admin:ctrl]: pool:markers> key owner
[admin:ctrl]: pool:markers> value sales
[admin:ctrl]: pool:markers> save
[admin:ctrl]: pool> save
Error: {"error": "Marker with key 'owner' to value 'sales' does not qualify the labelgroup
rules on this tenant."}

Supported Objects
The labels framework is supported for the following config objects:

n VIRTUALSERVICE

n POOL

n POOLGROUP

The other supported objects are as follows:

n NETWORKPROFILE

n HTTPPOLICYSET

n DNSPOLICY

n SECURITYPOLICY

n IPADDRGROUP

n STRINGGROUP

n SSLPROFILE

n SSLKEYANDCERTIFICATE

n NETWORKSECURITYPOLICY

n APPLICATIONPERSISTENCEPROFILE

n ANALYTICSPROFILE

n VSDATASCRIPTSET

VMware by Broadcom 328

VMware Avi Load Balancer Administration Guide

n PKIPROFILE

n SERVERAUTOSCALEPOLICY

n AUTOSCALELAUNCHCONFIG

n HARDWARESECURITYMODULEGROUP

n PRIORITYLABELS

n POOLGROUPDEPLOYMENTPOLICY

n GSLBSERVICE

n GSLBGEODBPROFILE

n GSLBAPPLICATIONPERSISTENCEPROFILE

n TRAFFICCLONEPROFILE

n WAFPOLICY

n WAFPROFILE

n ERRORPAGEPROFILE

n ERRORPAGEBODY

n L4POLICYSET

n WAFPOLICYPSMGROUP

n PINGACCESSAGENT

n NETWORKSERVICE

n NATPOLICY

n SSOPOLICY

n PROTOCOLPARSER

n IPREPUTATIONDB

n IpamDnsProviderProfile

n SERVICEENGINEGROUP

Label-based Permissions for Cloud Objects

Granular RBAC can be applied and enforced on cloud objects using the field
restrict_cloud_read_access in controllerproperties via the CLI.

This field is set to False by default. To enforce label-based permissions on cloud objects, set the
field restrict_cloud_read_access to True as shown below.

[admin:ctrl]: > configure controller properties

[admin:ctrl]: controllerproperties> restrict_cloud_read_access

VMware by Broadcom 329

VMware Avi Load Balancer Administration Guide

Overwriting the previously entered value for restrict_cloud_read_access

[admin:ctrl]: controllerproperties> save

Unsupported Permissions
The following permissions classified under Operations and Administrations are not supported for
role-based access control per app using labels:

n Operations

n PERMISSION_ALERT, PERMISSION_ALERTCONFIG, PERMISSION_ALERTEMAILCONFIG,

PERMISSION_ALERTSYSLOGCONFIG, PERMISSION_ACTIONGROUPCONFIG,
PERMISSION_SNMPTRAPPROFILE, PERMISSION_TRAFFIC_CAPTURE

n Administration

n PERMISSION_CONTROLLER, PERMISSION_INTERNAL, PERMISSION_TENANT,

PERMISSION_SYSTEMCONFIGURATION, PERMISSION_USER, PERMISSION_ROLE,
PERMISSION_UPGRADE, PERMISSION_REBOOT, PERMISSION_TECHSUPPORT

Caveats and Limitations

This section discusses the caveats and limitations in implementing this framework for RBAC.

n The number of labels for objects is limited to 4. There is no limit of values to each label or
marker key.

n A maximum of 4 filters can be applied per role.

n The number of characters for the labels key and value is limited to 128.

n A user with no filters can access all objects as per the privileges configured in the role (as per
the default behavior).

Granular Role-Based Access Controls Per Field

Avi Load Balancer provides RBAC to provide granular access to control, manage, and monitor
applications within Avi Load Balancer.

RBAC can be implemented at a field-level. This section covers the use of sub-resources to
implement RBAC per field.

Granular RBAC per-Field

Using Granular RBAC per Field, users can be allowed to update an object but restrict the updates
to a specific set of fields.

For example, allow users to:

n Enable or disable GSLB service groups, but restrict updating any other fields in the GSLB
object.

n Enable or disable a virtual service, but restrict updating any other virtual service
configuration.

VMware by Broadcom 330

VMware Avi Load Balancer Administration Guide

n Add, remove, or update the pool servers, but restrict updating any other pool configuration.

Sub-resources
To implement per-field RBAC, sub-resources for the existing resources are introduced. These
sub-resources are associated with a specific field, feature, or a set of fields within the object.
When a sub-resource is configured on a resource with write access, it will allow update to the
object only if those sub-resources are the only fields updated. Read access is allowed for the
full object, but delete and create are not allowed from that permission. Sub-resources can be
combined to allow users to configure multiple fields or features in an object.

To define access for sub-resources, the flags allow edit to only [subresource(s)] and
allow edit of entire object except for [subresource(s)] are introduced.

For example, configure a role with sub-resources as shown below.

[admin:10]: > configure role Pool-Enabled-Role

[admin:10]: role> privileges
New object being created
[admin:10]: role:privileges> type write_access
[admin:10]: role:privileges> resource permission_pool
[admin:10]: role:privileges> subresource
[admin:10]: role:privileges:subresource> subresources subresource_pool_enabled
[admin:10]: role:privileges:subresource> save
[admin:10]: role:privileges> save
[admin:10]: role> save

The pool is configured as shown below.

Sub-resources enable the user to execute a specific function within the object.

All available sub-resources are listed below.

Sub-resource Function

SUBRESOURCE_POOL_ENABLED Add/update/disable pool servers

SUBRESOURCE_POOL_SERVERS Add/update/remove pool servers

SUBRESOURCE_POOL_SERVER_ENABLED Enable/disable pool servers

VMware by Broadcom 331

VMware Avi Load Balancer Administration Guide

Sub-resource Function

SUBRESOURCE_VIRTUALSERVICE_ENABLED Enable/disable virtual servers

SUBRESOURCE_GSLBSERVICE_ENABLED Enable/disable GSLB service objects

SUBRESOURCE_GSLBSERVICE_GROUPS Update GSLBservice groups

SUBRESOURCE_GSLBSERVICE_GROUPS_ENABLED Enable/disable GSLBservice groups

SUBRESOURCE_GSLBSERVICE_GROUP_MEMBERS Update GSLBservice group members

SUBRESOURCE_GSLBSERVICE_GROUP_MEMBER_ENABL Enable/disable GSLBservice group members

SUBRESOURCE_VIRTUALSERVICE_AUTO_ALLOCATE_FL Enable/ disable Auto allocate floating IP

OATING_IP

Note If the access is not allowed for any field, creation of objects is not permitted as well.

VMware by Broadcom 332

Avi Load Balancer Controller
Operations 8
This section discusses the recommended steps and precautions required during the maintenance
operations of an Avi Load Balancer Controller.

Shutting Down or Rebooting an Avi Load Balancer

Controller
The Controller Virtual Machines (VMs) can be shut down or rebooted directly through the
hypervisor or a public cloud console. If the Controller is part of a cluster, it will attempt to rejoin
the cluster once it is operational. The Controller does not require user intervention for a reboot
operation in a cluster set-up.

You can reboot the Controller node using the following CLI:

reboot node nodename <node>

This will restart the virtual machine.

You can reboot the Controller node using the following API command:

/cluster/reboot/node

This will restart the virtual machine.

cluster/reboot

For more details on the CLI used here, see CLI Top-Level Commands section in this guide.

VMware by Broadcom 333

VMware Avi Load Balancer Administration Guide

Increasing Controller Disk Size of an Avi Load Balancer

Controller
There are scenarios where the Controller’s disk size configured initially is insufficient and requires
an increase in the disk size. The Controller VM’s disk can be resized directly using the hypervisor
or the public cloud console. If the Controller is part of a cluster, it is recommended to perform the
changes on the follower nodes, followed by the leader node.

Note
n Increasing overall disk capacity by adding a new, additional disk to the Controller VM is not
supported.

n It is recommended to configure the same disk size for all Controllers in a cluster.

Follow the below steps to increase Disk Size:

1 Shut down the Controller VM.

2 Increase the disk size through the hypervisor or public cloud console.

3 Start the Controller VM.

4 SSH to the Controller VM, and confirm that the new disk size using the command sudo df
-h.

5 Perform the above steps on the follower nodes first, followed by the leader nodes.

Increasing Memory and vCPU of an Avi Load Balancer

Controller
There are scenarios where the vCPU or memory configured initially is insufficient and requires an
increase in the allotment. Another related use case it so select a different flavor or size for the
Controller VM for a public cloud or OpenStack deployment. The resources can be edited using
the hypervisor, or the public cloud console.

If the Controller is part of a cluster, it is recommended to perform the changes on the follower
nodes, followed by the leader node.

Note
n It is recommended to configure the same resource parameters for all Controllers in a cluster.

Follow the below steps to increase vCPU or Memory of an Avi Load Balancer Controller:

1 Shut down the Controller VM.

2 Increase the vCPU or memory assigned to the VM.

3 In vCenter environments, increase the allocation using the settings.

4 In case of flavor-based selections, select the new flavor or VM size.

VMware by Broadcom 334

VMware Avi Load Balancer Administration Guide

5 Start the Controller VM.

6 Perform the above steps on the follower nodes, followed by the leader nodes.

Increasing Controller Disk Capacity

Following is the process by which disk capacity for Controllers in a cluster can be increased.

1 Power down one of the Controller followers.

2 Increase disk size for the physical or virtual machine using the tools appropriate to your
environment.

3 Power up the Controller on that machine.

4 Verify that it has joined into the cluster.

5 Repeat steps 1-4 for the second follower.

6 Power down the Controller cluster leader. One of the two already-upgraded followers
automatically become the new leader.

7 Proceed with steps 2-4 for the third and final Controller. The third controller must have joined
as the second follower.

Read the following topics next:

n Backup and Restore of Avi Load Balancer Configuration

n Upgrade and Patches

n Increasing Docker Container Size

Backup and Restore of Avi Load Balancer Configuration

Periodic backup of the Avi Load Balancer configuration database is recommended. This database
defines all clouds, all virtual services, all users, and so on.

Any user capable of logging into the admin tenant is authorized to perform a backup of the entire
configuration, that is, of all tenants. A restore operation spans all the same entities but can only
be performed by the administrator(s) capable of logging into one of the Controllers using SSH or
SCP.

It is a best practice to store backups in a safe external location in the unlikely event that a
disaster destroys the entire Avi Load Balancer Controller (or cluster), with no possibility of
remediation. A recommended backup schedule could be daily or even hourly, based on how
often the configuration changes.

VMware by Broadcom 335

VMware Avi Load Balancer Administration Guide

Backing Up the Avi Load Balancer Configuration

To back up the configuration, use the Avi Load Balancer UI, CLI commands, or API commands
shown in this section. Backups can be scheduled or on-demand.

Note When backing up Avi Load Balance via the CLI method for system restore, it is required
and recommended that you perform the backup with the full_system option enabled.

Scheduled Backup using the UI

To view or edit the configuration backup scheduler’s current settings, an admin-tenant user first
must navigate to Administration > Controller > Configuration Backup.

Note The scheduled backups get stored in /var/lib/avi/backups/ on all Avi Load Balancer
Controller in the cluster.

1 To configure the backup, click EDIT to view the EDIT CONFIGURATION BACKUP screen.

2 Select Enable Configuration Backup check box to schedule backups.

3 Enter the Passphrase and confirm the passphrase to encrypt all sensitive fields contained
within the backup. Choose a phrase that is not easy to guess and guard it carefully. Data
cannot be restored without the passphrase.

4 Enter a File Prefix to customize the backup archive filename.

5 Select a remote transfer Protocol from the drop-down menu.

a Secure Copy Protocol (SCP)

b Secure File Transfer Protocol (SFTP)

6 Under Scheduler, configure the following:

a Enter a value from 0 to 60 as the Frequency to determine how often backups are to be
taken. 0 indicates the backup sequence has no end time.

b Enter the Frequency Unit for backups to occur. By default, the Frequency Unit is taken
every day. Use this field to change the unit to minutes, hours, weeks, or months.

c Enter a number ranging from 0 to 20 as the Number of backups to store. 0 is equivalent

to deselect the Local option. The oldest backup is deleted after the most recent backup
successfully completes.

VMware by Broadcom 336

VMware Avi Load Balancer Administration Guide

7 You can choose to save local (on Controller) and remote backup locations independently. In
the Backup Destination section, configure the following:

a Select Enable Local Backups (On Controller), to preserve the number of indicated
backups on the Controller.

b Select Enable Remote Server Backup to save the backup on a remote server.

Note This option is deactivated by default, but it is recommended to specify a remote

destination in case the Controller cluster fails in a non-recoverable fashion.

c In the Server Address field, enter an FQDN or IP address reachable from the Controller.

d In the Home Directory field, provide a remote destination address with write permissions.

e Use the drop-down menu to select User Credentials from a previously-defined SSH user.
Alternatively, click the three dots at the end of the field to Create new user credentials.

f If you are creating a new SSH user, the following dialog is shown.

VMware by Broadcom 337

VMware Avi Load Balancer Administration Guide

Enter a username in the Name field. Avi Load Balancer will use this username to transfer
the file. In the Authentication field, select either SSH Key or Password and provide the
required information.

8 Click Save to complete configuration backup settings.

Scheduled Backup using the CLI

Note When backing up Avi Load Balance via the CLI method for system restore, it is required
and recommended that you perform the backup with the full_system option enabled.

[admin:10-102-64-234]: > show backupconfiguration Backup-Configuration

VMware by Broadcom 338

VMware Avi Load Balancer Administration Guide

You can specify the value of start_date_time from the CLI (not possible using the UI).

[admin:10-10-24-52]: > configure scheduler Default-Scheduler

[admin:10-10-24-52]: scheduler> no enabled
[admin:10-10-24-52]: scheduler> start_date_time 2017-05-11T00:00:00
Overwriting the previously entered value for start_date_time
[admin:10-10-24-52]: scheduler> save
[admin:10-10-24-52]: > configure scheduler Default-Scheduler
[admin:10-10-24-52]: scheduler> enabled
Overwriting the previously entered value for enabled
[admin:10-10-24-52]: scheduler> save

Scheduled Backup using the API

In this example, a PUT changes the scheduler frequency to one week.

PUT : api/scheduler/
{'_last_modified': u'1476209663670990',
'backup_config_ref': 'https://10.10.24.52/api/backupconfiguration/
backupconfiguration-5d65f12e-5da1-49e0-b703-ec65ae9a39c6',
'enabled': True,
'frequency': 1,
'frequency_unit': u'SCHEDULER_FREQUENCY_UNIT_WEEK',
'name': u'Default-Scheduler',
'run_mode': u'RUN_MODE_PERIODIC',
'scheduler_action': u'SCHEDULER_ACTION_BACKUP',
'start_date_time': u'2016-10-09T15:35:46.220623',
'tenant_ref': u'https://10.10.24.52/api/tenant/admin',
'url': 'https://10.10.24.52/api/scheduler/scheduler-b5f7e673-8818-44d1-8f74-45238cc08235',
'uuid': u'scheduler-b5f7e673-8818-44d1-8f74-45238cc08235'}

On-demand Backup using the CLI

To back up the Avi Load Balancer configuration on-demand, at any arbitrary time, use the
following CLI command.

: > export configuration file /tmp/avi_config.json full_system

Please enter the passphrase to encrypt configuration:
Retype passphrase:
Downloaded the attachment to /tmp/avi_config.json
Completed writing the export configuration to /tmp/avi_config.json

VMware by Broadcom 339

VMware Avi Load Balancer Administration Guide

On-demand Backup using the REST API

To back up the Avi Load Balancer configuration on-demand, at any arbitrary time, use the
following API request.

GET https://[CONTROLLER-IP]/api/configuration/export?full_system=true

To include a passphrase, use one of the following options.

GET https://[CONTROLLER-IP]/api/configuration/export?full_system=true&passphrase=[PASSPHRASE]

Use the following POST method and include passphrase in the JSON data.

POST https://[CONTROLLER-IP]/api/configuration/export?full_system=true JSON data:

{"passphrase":"[PASSPHRASE]"}

Make sure to replace [CONTROLLER-IP] with the IP address of the Avi Load Balancer Controller
(if using a single Controller node) or the IP address of the Controller cluster.

On-demand Backup Script Utilizing Python

For more information, see vmware/alb-sdk.

Configuring Backup using Amazon S3

A backup configuration can be stored on Amazon S3 bucket.

To enable backup configuration on Amazon S3, use the configure backupconfiguration

command and set the value of the upload_to_s3 flag to true.

Provide the value of the following attributes to save the backup file on the Amazon S3 bucket for
the required instance.

n aws_access_key : Access Key ID

n aws_secret_access : Secret Access Key

n aws_bucket_id : Name of the S3 bucket

[admin:10-1-1-1]: configure backupconfiguration Backup-Configuration

Note For enabling backup of the Avi Load Balancer Controller, you must have write access
permission to the S3 bucket.

For detailed information on the Access Key ID and Secret Access Key, see AWS User Cross-
Account AssumeRole topic in the VMware Avi Load BalancerInstallation Guide.

Upgrade and Patches

This section discusses about the upgrade and patches.

VMware by Broadcom 340

VMware Avi Load Balancer Administration Guide

Upgrade Overview
Avi Load Balancer supports flexible methods for upgrading the Avi Load Balancer system.

The followings are the main features for the Flexible Upgrades:

n Avi Load Balancer Controller can be upgraded/patched first and later the Service Engine
groups are upgraded .

n The upgrade is possible per SE group. The transition of all the SE groups to the new version
might occur over a long period.

n Upgrades of different SE groups are supported with different patch versions.

n Rollback to the previous versions of Avi Load Balancer is non-disruptive.

The following options are available with Flexible Upgrades.

Upgrades Patch Ugrades Rollback Rollback Patch

System (Avi Load Balancer System (Avi Load Balancer System (Avi Load Balancer System (Avi Load Balancer
Controller and SE Groups) Controller and SE Groups) Controller and SE Groups) Controller and SE Groups)

Avi Load Balancer Avi Load Balancer Controller Avi Load Balancer Avi Load Balancer Controller
Controller only only Controller only only

Some or all the SE groups Some or all the SE groups Some or all the SE groups Some or all the SE groups

Use Cases
n Scenarios where it is not possible to upgrade all SE groups to the newer version at the same
time due to various business reasons such as logistics, confidence in the new software, and
so on

n The configuration is blocked during the entire duration of the Avi Load Balancer Controller
and SE upgrade. This is not acceptable in many deployments. With the new upgrade feature,
the process is flexible and can be performed on an SE group basis. The configuration is
blocked for the entire duration if a system upgrade is performed till all SEs are upgraded

n Using SE groups for data plane separation. Based on the SE group segmentation, the
upgrade is performed depending on the following attributes:

n Application or product offering

n Tenant

n Production, pre-production, and development environments

n Cloud or environment (AWS, VMware, and so on.)

n Provide patches to only applications or SE groups that need them

n Flexible scheduling

n Self-service upgrades

VMware by Broadcom 341

VMware Avi Load Balancer Administration Guide

Patch Overview
Avi Load Balancer supports patch upgrades by which hotfixes are placed into effect.

The followings are the available options for patch upgrade:

n System — Patch upgrade for Avi Load Balancer Controller and all SE groups

n Controller — Patch upgrade for the Avi Load Balancer Controller

n SE group — Patch upgrade for some or all the SE groups

Note The following are a few points for a patch upgrade process:
n An image with a patch can be applied.

n The image and the patch must have the same base version.

n A patch cannot be applied without applying the image.

n Compatibility checks prevent incorrect patches from getting applied to different versions.

Prerequisites
This section discusses about the prerequisites of Image Management and Service.

Image Management and Service

Image service is the first step in the flexible upgrade workflow. Avi Load Balancer Controller
hosts images of different versions because SE groups could be potentially in different versions.

The Controller must have additional disk space to host these images.

Avi Load Balancer Controller images for the major versions include the following:

n controller.pkg (for VM-based Avi Load Balancer Controller)

n controller_docker.tgz (For Docker-based Controller)

Images for the patches include the following:

n avi_patch.pkg (Full package)

n controller_patch.pkg (Avi Load Balancer Controller package)

n se_patch.pkg (SE patch package)

As a part of the upload process, the image service extracts files, and metadata from the package.
This information is not only displayed to the user, but it is also used in the upgrade process.

Note
n Image service provides an ability to upload, query, and delete Avi Load Balancer image(s) to
the system.

n Image service supports the upload of Avi Load Balancer patch packages.

n Image upload can happen only on the cluster leader. It is not allowed from a cluster member.

VMware by Broadcom 342

VMware Avi Load Balancer Administration Guide

Image Bundling
Avi Load Balancer supports composite image or image bundle. The composite image consists of
the following:

n Base image – Controller image (controller_docker.tgz, controller.pkg, controller

ova, controller.qcow2, and so on).

n Controller package – It is an optional package.

n SE patch image – It is an optional package.

The upgrade workflow using the image bundle, or the composite image is the same as the
standard image. When using the image bundle for an upgrade, a patch image can be applied in
addition to the base image.

Note When upgrading from versions 18.x or lesser to 21.1 and higher, in the Avi Load
Balancer Controller, change the DefaultTimeoutStartSec (File: /etc/systemd/system.conf) to
120 seconds to avoid timeout during the upgrade.

Uploading Image Using the CLI

The CLI provides better control of the upgrade operations leading to a consistent and predictable
workflow.

For uploading the package, use the upload image filename <path-of-the-package>
command as shown below.

[admin:controller]: > upload image filename /tmp/controller.pkg

The following show command returns the details of the image metadata. Use the show image
<image-name> command as shown below.

[admin:-controller]: > show image

+-----------------------------+--------------------------------------------+----------------+
| Name | UUID | Status |
+-----------------------------+--------------------------------------------+----------------+
| 18.2.7-5000-20191009.205501 | image-fxxxx22-0f40-45de-8551-15xxxxxxx1fe | SYSERR_SUCCESS |
+-----------------------------+-----

For more information about differences in CLI commands and APIs, see Comparison Table for
Differences in CLIs Commands and APIs.

Uploading Image Service using the REST API

A POST operation is used for image upload. To get the image details in response, run a GET API
request.

n Use the following REST API to upload the image for controller.pkg.

n URI: /api/image

VMware by Broadcom 343

VMware Avi Load Balancer Administration Guide

n Method: POST

root@admin:-controller# curl -X POST -k https://10.58.3.27/api/image -u "admin:admin"

-F [email protected] -H "X-CSRFToken: P9e5H5aCtRwtAVIouzJW9eoxWNPSHptc" -H "Cookie:
csrftoken= P9e5H5aCtRwtAVIouzJW9eoxWNPSHptc; sessionid= 9ku3n1z3oqa62k7lwgbdm8zcatf75qag"
-H "Referer: https://10.58.3.27/api"

n Use the following REST API to upload the image for controller_patch.pkg.

root@admin:-controller# curl -X POST -k https://10.58.3.27/api/image

-u "admin:admin" -F file=@controller_patch.pkg -H "X-CSRFToken:
P9e5H5aCtRwtAVIouzJW9eoxWNPSHptc" -H "Cookie: csrftoken= P9e5H5aCtRwtAVIouzJW9eoxWNPSHptc;
sessionid= 9ku3n1z3oqa62k7lwgbdm8zcatf75qag" -H "Referer: https://10.58.3.27/api"

n Use the following API to delete the image provided, if it is not in use.

delete image <image-name>

Must-Checks for Upgrade

Prior to upgrade operations, various must-checks are run to check the mandatory and optional
requirements for upgrade. The output message is displayed either as an error message or a
warning message. Warnings can be skipped, whereas Errors cannot be over-ridden. CLI/API
provides the skip_warnings option to control the above behavior.

n For the CLI: This is directly integrated into the normal workflow, and there is no separate
command.

n For the REST API: Add /preview/ at the end of APIs to get previews for that particular flow.

Initiating an upgrade or patch through the CLI will halt at the UPGRADE_PRE_CHECK_WARNING state if
the command is executed without the skip_warnings parameter.

Review the warnings using the command show upgrade status detail filter
pre_check_status and take any necessary actions.

After addressing the warnings, you can run the upgrade or patch command again with the
skip_warnings parameter to allow the operation to continue.

[admin:10-10-10-1]: > upgrade system image_ref 30.3.3-7235-20230110.035149 skip_warnings

+-------------+------------------------------------------------------------------------------+
| Field | Value |
+-------------+------------------------------------------------------------------------------+
| status_code | SYSERR_UPGRADE_OPS_PREVIEW_RESPONSE |
| status | Checks preview for upgrade operations. |
| checks | |
| | Check Controller Cluster readiness for upgrade operations. |
| | Check and inform user to take a backup prior to upgrade operations. |
| | Check if se linux is enabled on controller nodes. |
| | Check if upgrade operation is already in progress. |
| | Check ServiceEngineGroup has an ongoing upgrade operation. |
| | Check image version compatibility for upgrade operations. |
| | Check ServiceEngine reachability for upgrade operations. |
| | Check ServiceEngine disk space for upgrade operations. |

VMware by Broadcom 344

VMware Avi Load Balancer Administration Guide

| | Check Controller Cluster disk space for upgrade operations. |

| | Check and inform Virtual Service(s) disruption for upgrade operations. |
| | Check idempotent operations for upgrade operations. |
| | Check active versions compatibility for upgrade operations. |
| | Check ServiceEngineGroup error recovery options prior to upgrade operations. |
| | Check Image state across Cluster members for upgrade operations. |
| | Checks for the patch in image bundle. |
| | Checks if Gslb Feature is enabled and provides feature specific messages. |
| | Checks the system configuration. |
| | Check total number of alerts for upgrade operations. |
| | Checks if the cloud api versions are compatible after upgrade. |
| | Checks if Docker version is compatible. |
| | Checks if configured IP type is DHCP or STATIC. |
| | Checks if se has a valid license state. |
+-------------+------------------------------------------------------------------------------+
Starting upgrade
+-------------
+---------------------------------------------------------------------------------------------
--------------+
| Field |
Value
|
+-------------
+---------------------------------------------------------------------------------------------
--------------+
| status_code |
SYSERR_UPGRADE_SYSTEM_STARTED
|
| status | 'Upgrade of System (Controller + All SEGroup(s)) started. Use 'show upgrade
status' to check the status.' |
+-------------
+---------------------------------------------------------------------------------------------
--------------+

Similarly, use the skip_warning option while performing the patch upgrade.

[admin:10-10-10-1]: > patch controller controller_patch_ref 23.1.1-7189-2p1-20221216.192828

skip_warnings
Previewing upgrade

Downloading Upgrade Images

Download upgrade images by logging in to the Broadcom Support portal, and opening the My
Downloads page.

Determine Software Version

This topic details the methods to determine the specific Avi Load Balancer software version
running on the Controller and its SEs.

VMware by Broadcom 345

VMware Avi Load Balancer Administration Guide

Finding Version through CLI

From the Controller shell, enter the following command.

: > show version controller

+-----------------+--------------------------------------+
| Controller Name | Version |
+-----------------+--------------------------------------+
| 10.130.162.14 | 15.2.4(9007) 2015-10-17 01:43:38 UTC |
+-----------------+--------------------------------------+

Alternatively, from bash, run the following command.

cat /bootstrap/VERSION

Finding Version through Web Interface

From the top right corner of the Controller web interface, click the user icon and select About
NSX ALB (Avi) from the drop-down menu.

Pre/Post Upgrade Object Operational State Verification

This section compares the pre/post-upgrade operational state using the CLI.

For software upgrade operations, the Avi Load Balancer Controller provides visibility into
the pre-operation and post-operation states of the system, summarizing and highlighting the
significant differences in the state or health of the system caused by the performed operation.

When performing a software upgrade, the Avi Load Balancer Controller will take an operational
state snapshot pre-upgrade and another post-upgrade to allow the admin to perform a quick
validation of the current state of their environment compared to the state before the upgrade.

The operational snapshots are taken for the following objects:

n Virtual Services

n Service Engines

n Pools

n GSLB Services

The operational snapshots are taken during the following software operations:

n Upgrade

n Patch

n Rollback Patch

VMware by Broadcom 346

VMware Avi Load Balancer Administration Guide

Compare Pre/Post Upgrade Operational State using the CLI

The following are the steps to compare pre/ post upgrade operational state using the CLI:

1 Check if all the state-diff operations exists in the system.

> show statediff

+----------------------------------------------------+----------------+
| Name | Status |
+----------------------------------------------------+----------------+
| upgrade_controller_fitb_2021-06-22-08:20:40.957288 | FB_IN_COMPLETED|
+----------------------------------------------------+----------------+

2 Check if the summarized state changes for a particular operation from the above listed
operations.

> show statediff upgrade_controller_fitb_2021-06-22-08:20:40.957288

+----------------+----------------+
| Object | States Changed |
+----------------+----------------+
| SERVICEENGINE | 0 |
| VIRTUALSERVICE | 1 | // Indicates 1 VS changed state for the queried
operation
| GSLBSERVICE | 0 |
| POOL | 0 |
+----------------+----------------+

3 Check for the detailed state changes for all entities.

> show statediff upgrade_controller_fitb_2021-06-22-08:20:40.957288 filter detail

VMware by Broadcom 347

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 348

VMware Avi Load Balancer Administration Guide

4 Check for the summary of state changes for all the objects of a particular type (virtual service
in this example).

> show statediff upgrade_controller_fitb_2021-06-22-08:20:40.957288 filter obj_type

fb_virtualservice
+------+-----------------------------------------------------+--------------
+----------------------+
| Name | Uuid | State Change |
Summary |
+------+-----------------------------------------------------+--------------
+----------------------+
| vs-2 | virtualservice-7e22eec4-fc6b-40d9-9306-84259f54948a | NO |
- |
| vs-1 | virtualservice-1aa470fa-e43f-47f3-b720-0836bf87bad8 | YES | OPER_UP ->
OPER_DOWN |
+------+-----------------------------------------------------+--------------
+----------------------+

5 Check for the detailed state changes for desired objects.

> show statediff upgrade_controller_fitb_2021-06-22-08:20:40.957288 filter virtualservice

VMware by Broadcom 349

VMware Avi Load Balancer Administration Guide

6 Check for the state changes for all objects under a particular serviceenginegroup(parent
obj) object.

> show statediff upgrade_controller_fitb_2021-06-22-08:20:40.957288 filter virtualservice

7 Clean up the changes, if required, using the following command.

> delete statediff upgrade_controller_fitb_2021-06-22-08:20:40.957288

Pre-upgrade Checklist
This section provides a list of checks to be performed before initiating upgrade of Avi Load
Balancer.

Avi Load Balancer supports a simple system upgrade method wherein all Avi Load Balancer
Controller nodes and SE nodes are upgraded to the newer software version in one upgrade
sequence. Following are the list of checks to be performed before upgrading the Avi Load
Balancer:

n Check if all the three nodes of the Controller cluster are Active.

VMware by Broadcom 350

VMware Avi Load Balancer Administration Guide

For more information about Controller clusters, see Chapter 5 Avi Load Balancer Controller
Cluster and Changing Avi Load Balancer Controller Cluster Configuration.

n Check if there is adequate disk space on all the Controller nodes before the upgrade. Ideally,
the disk usage must not be more than 60%. For more information, see NSX Advanced Load
Balancer Controller Sizing topic in the VMware Avi Load BalancerInstallation Guide.
n If the disk usage is more (greater than 80%), look for any core archives or packages uploaded
to the Controller nodes and clear them. Core archives are under /var/lib/avi/ archives.

n Ensure all servers are connected. Run Show serviceengine. Check if all the service engines
are UP unless they are disabled. For more information, see De-activating SE topic in the
VMware Avi Load BalancerMonitoring and Operability Guide. The upgrade command does a
pre-check to ensure that no Service Engines are disconnected from the Controller, so we do
not inadvertently finish the upgrade without these Service Engines.

n Ensure that all the SEs are on the version required. Run show version controller and show
version serviceengine.

n Check CPU and memory of all Controllers to ensure compliance with minimum
recommendations.

n Controllers will fail to upgrade if they are in the Essentials flavor (less than 6 cores or 32 GB of
memory).

Note For Avi Load Balancer (formerly Avi Vantage) lifecycle information, see the Product Life
Cycle Matrix and the Lifecycle Policy.

Post-upgrade Checklist
This section provides a list of checks to be performed after successful upgrade of Avi Load
Balancer.

n Execute show upgrade status to ensure that the upgrade is successful. The status should
be UPGRADE_COMPLETED.

n Execute show version controller and show version serviceengine to ensure that the
service engines are on the version expected.

n Ensure all the three nodes of the Controller cluster are active.

n Ensure that the Cloud Status is Placement Ready.

n Inspect the Application Dashboard to ensure the application’s health is as expected. If you
have traffic tools to test the availability of the virtual services, ensure that this is running
successfully. For more information, see Virtual Service Health Monitoring topic in the VMware
Avi Load BalancerMonitoring and Operability Guide.

VMware by Broadcom 351

VMware Avi Load Balancer Administration Guide

Upgrade Guide
This section explains the steps to upgrade Avi Load Balancer System (Avi Load Balancer
Controller and SE Groups)

Prechecks_only Flag
Starting with Avi Load Balancer Controller 22.1.6, a new flag prechecks_only is introduced for
all upgrade operations (upgrade, patch upgrade, rollback). The prechecks_only flag enables
upgrade APIs to perform upgrade checks prior to an upgrade maintenance window. Executing
the prechecks_only flag does not trigger any upgrade operation and it is safe to perform
upgrade operations with the prechecks_only flag outside of a maintenance window.

Use the upgrade controller image_ref <image_name> prechecks_only command to inititate the
prechecks_only operation for Avi Load Balancer Controller.

[admin:100-65-10-137]: > upgrade controller image_ref 22.1.6-8143-20240115.080016

prechecks_only
Previewing upgrade
+-------------
+-----------------------------------------------------------------------------------------+
| Field |
Value |
+-------------
+-----------------------------------------------------------------------------------------+
| status_code |
SYSERR_UPGRADE_OPS_PREVIEW_RESPONSE |
| status | Checks preview for upgrade
operations. |
| checks
| |
| | 'Checking Controller Cluster readiness for upgrade
operations.' |
| | 'Check if upgrade operation is already in
progress.' |
| | 'Checking active versions compatibility for upgrade
operations.' |
| | 'Checking if system in Avi-ESSENTIALS/BASIC mode and allows only permitted
operations.' |
| | 'Checking if the cloud api versions are compatible after
upgrade.' |
| | 'Checking ServiceEngineGroup has an ongoing upgrade
operation.' |
| | 'Checking Controller Cluster disk space for upgrade
operations.' |
| | 'Checking image version compatibility for upgrade
operations.' |
| | 'Checking idempotent operations for upgrade
operations.' |
| | 'Checking Image state across Cluster members for upgrade
operations.' |
| | 'Checking the system
configuration.' |

VMware by Broadcom 352

VMware Avi Load Balancer Administration Guide

| | 'Checking config
migration.' |
| | 'Checking if Gslb Maintenance Mode is enabled prior to
upgrade.' |
| | 'Checking if Gslb Feature is enabled and provides feature specific
messages.' |
| | 'Checking and inform user to take a backup prior to upgrade
operations.' |
| | 'Checking if Docker version is
compatible.' |
| | 'Checking for the mandatory patch in image
bundle.' |
| | 'Checking if configured IP type is DHCP or
STATIC.' |
| | 'Checking if se linux is enabled on controller
nodes.' |
| | 'Checking total number of alerts for upgrade
operations.' |
+-------------
+-----------------------------------------------------------------------------------------+
Starting upgrade
+--------------------
+---------------------------------------------------------------------------------------------
---+
| Field |
Value
|
+--------------------
+---------------------------------------------------------------------------------------------
---+
| status_code |
SYSERR_EVAL_UPGRADE_CONTROLLER_STARTED
|
| status | 'Pre-checks for Upgrade of Controller started. Use 'show upgrade
status' to check the status.' |
| system_report_uuid | systemreport-6f499bff-fcbd-4fca-
a43a-483d1764eed5 |
+--------------------
+---------------------------------------------------------------------------------------------
---+

Upgrading Avi Load Balancer System (Avi Load Balancer Controller and SE
Groups)
The configuration and placement of virtual services are blocked if it is a system-level upgrade
till all the SEs are upgraded. Once these operations are completed, configuration on Avi Load

VMware by Broadcom 353

VMware Avi Load Balancer Administration Guide

Balancer Controller (except the configuration of virtual service and VIP) is allowed, irrespective of
the SE group upgrade status.

Note
n It is recommended to increase the default timeout value from 90 seconds to 120 seconds
before performing the upgrade. The default timeout is available as the parameter name
DefaultTimeoutStartSec and at (File: /etc/systemd/system.conf).This is to avoid the
upgrade going to timeout.

n Controllers will fail to upgrade if they are in the Essentials flavor (less than 6 cores or 32 GB of
memory).

Number of Maximum Active Versions Consideration for Upgrade

Avi Load Balancer supports a maximum of 2 active image versions in the system at any time.

It is essential to consider various deployment scenarios when planning system upgrades. This
strategic approach aids in the effective execution of system upgrades.

Consider the following deployment scenarios:

n Case 1: The Controller's image version is 22.1.6, and one of SE Group's image versions is 22.1.5.
So, the number of active versions used in the system is 2 (22.1.5 and 22.1.6).

n In this case, the number of active versions of deployed images is 2. As the number of
active versions is equal to the maximum active version supported, upgrading Controller
to the next version is not supported. An upgrade to the next version, say 30.2.1 in this
case, means total number of active versions at a time are 30.2.1, 22.1.5, and 22.1.6. It is not
allowed as it exceeds maximum number active versions supported. Make sure to upgrade
all the SE Groups' image versions to 22.1.6 before attempting the upgrade operation to
30.2.1 version.

n Case 2: When all SE Groups' image versions are 22.1.6, and the Controller's image version is
also 22.1.6, the number of active versions is 1 (22.1.6).

n In this scenario, upgrading the Controller to the next version is supported, as the number
of active versions (1) is less than the maximum number of active versions supported (2).

Using the CLI

This section discusses CLI options to upgrade Avi Load Balancer system.

The auto-suggest option in the CLI provides available values on pressing tab on your keyboard.
Use skip_warnings option to skip any warnings and optional must checks.

The following are the various options available for Avi Load Balancer system upgrade:

n Use the upgrade system image_ref <image name> command to upgrade the system to a
base image.

[admin:-controller]: >upgrade system image_ref 18.2.6-9000-20191031.063017

VMware by Broadcom 354

VMware Avi Load Balancer Administration Guide

n Use the following to upgrade the system to a base image and a controller patch.

[admin:-controller]: >upgrade system image_ref 18.2.6-9134-20191101.042535

controller_patch_ref 18.2.6-9134-2p1-20190806.011824

n Use the following to upgrade the system to a base image and an SE patch.

[admin:-controller]: >upgrade system image_ref 18.2.6-9134-20191101.042535 se_patch_ref

18.2.6-9134-2p1-20190806.011824

n Use the following to upgrade the system to a base image, an Avi Load Balancer Controller
patch, and an SE patch.

[admin:-controller]: >upgrade system image_ref 18.2.6-9134-20191101.042535

controller_patch_ref 18.2.6-9134-2p1-20190806.011824 se_patch_ref
18.2.6-9134-2p1-20190806.011824

SE Upgrades
The Controller allows you to pick up the number of SE-groups per Controller node.

seupgrade_fabric_pool_size:

This property allows the Controller to pickup number of SE groups per Controller to upgrade.

For instance, if seupgrade_fabric_pool_size is set to 3, three SE-groups are picked up per

Controller, that means 9 SE groups across the cluster.

The default value of seupgrade_fabric_pool_size is 20. However, you can update this based on
the requirement or the load.

seupgrade_copy_pool_size:

This parameter defines the number of simultaneous SE image downloads in a SEGroup. It is

used to pace the SE downloads so that Controller network/ CPU bandwidth is a bounded
operation. A value of zero will disable the pacing scheme and all the SE(s) in the SEGroup will
attempt to download the image.

seupgrade_copy_pool_size = n, where ‘n’ is the number of SE within SE group will be picked for
copy.

For instance, if seupgrade_copy_pool_size = 3, the three SE in a picked up SE group will be

picked for copy.

The default value of seupgrade_copy_pool_size is 5. However, you can update this based on the
requirement or the load.

The following are the steps to configure this:

1 Configure the Controller properties.

2 Set seupgrade_fabric_pool_size <number>.

3 Set seupgrade_copy_pool_size <number>.

VMware by Broadcom 355

VMware Avi Load Balancer Administration Guide

4 Save.

[admin:ctrl]: > configure controller properties

[admin:ctrl]: controllerproperties> seupgrade_fabric_pool_size 2
Overwriting the previously entered value for seupgrade_fabric_pool_size
[admin:ctrl]: controllerproperties> seupgrade_copy_pool_size 2
Overwriting the previously entered value for seupgrade_copy_pool_size
[admin:ctrl]: controllerproperties> save

[admin:ctrl]: > show controller properties |grep pool

SE Upgrades
The Controller allows you to pick up the number of SE-groups per Controller node.

seupgrade_fabric_pool_size:

This property allows the Controller to pickup number of SE groups per Controller to upgrade.

For instance, if seupgrade_fabric_pool_size is set to 3, three SE-groups are picked up per

Controller, that means 9 SE groups across the cluster.

The default value of seupgrade_fabric_pool_size is 20. However, you can update this based on
the requirement or the load.

seupgrade_copy_pool_size:

This parameter defines the number of simultaneous SE image downloads in a SEGroup. It is

seupgrade_copy_pool_size = n, where ‘n’ is the number of SE within SE group will be picked for
copy.

For instance, if seupgrade_copy_pool_size = 3, the three SE in a picked up SE group will be

picked for copy.

The default value of seupgrade_copy_pool_size is 5. However, you can update this based on the
requirement or the load.

The following are the steps to configure this:

1 Configure the Controller properties.

2 Set seupgrade_fabric_pool_size <number>.

3 Set seupgrade_copy_pool_size <number>.

VMware by Broadcom 356

VMware Avi Load Balancer Administration Guide

4 Save.

[admin:ctrl]: > configure controller properties

[admin:ctrl]: > show controller properties |grep pool

Using the REST API

Image UUID is obtained by using the GET /api/image.

The following are the various REST API options available for the Avi Load Balancer system
upgrade:

n Use the following API to upgrade the system to a base image.

API:/api/upgrade

Method: POST

JSON Data:

{
'image_uuid': 'image-b8adc2bd-d27f-469d-b78d-5e2bc14a14e4',
'system': true
}

n Use the following API to upgrade the system to a base image and a controller patch.

API: /api/upgrade

Method: POST

JSON Data:

{
'image_uuid': 'image-b8adc2bd-d27f-469d-b78d-5e2bc14a14e4',
'controller_patch_uuid': 'image-e3aaad68-5aaf-485a-8bd9-1db3ec562d6a',
'system': true
}

n Use the following API to upgrade the system to a base image and an SE patch.

API:/api/upgrade

Method: POST

VMware by Broadcom 357

VMware Avi Load Balancer Administration Guide

JSON Data:

{
'image_uuid': 'image-b8adc2bd-d27f-469d-b78d-5e2bc14a14e4',
'system': true,
'se_patch_uuid': 'image-e3aaad68-5aaf-485a-8bd9-1db3ec562d6a',
'skip_warnings': True
}

n Use the following API to upgrade the system to a base image, an Avi Load Balancer
Controller patch, and an SE patch

API: /api/upgrade

Method: POST

JSON Data:

{
'image_uuid': 'image-b8adc2bd-d27f-469d-b78d-5e2bc14a14e4',
'controller_patch_uuid': 'image-e3aaad68-5aaf-485a-8bd9-1db3ec562d6a',
'system': true,
'se_patch_uuid': 'image-e88aaad68-5aaf-485a-8bd9-1db3ec562d6a'
}

Upgrading Avi Load Balancer Controller

This section explains about upgrading Avi Load Balancer Controller.

Prechecks_only Flag
Starting with Avi Load Balancer Controller 22.1.6, a new flag prechecks_only is introduced for all
upgrade operations (upgrade, patch upgrade, rollback).

Enabling the prechecks_only flag enables upgrade APIs to perform upgrade checks prior to an
upgrade maintenance window. When this flag is enabled, only the list of pre-check actions is
performed on Avi Load Balancer Controller Controller. Executing the prechecks_only flag does
not trigger any upgrade operation. Please note it is safe to perform upgrade operation with
'prechecks_only' flag outside of maintenance window.

Use the upgrade controller image_ref <image_name> prechecks_only command to inititate the
prechecks_only flag.

[admin:10-x-x-1]: > upgrade controller image_ref 22.1.6-8143-20240115.080016 prechecks_only

Previewing upgrade
+-------------
+-----------------------------------------------------------------------------------------+
| Field |
Value |
+-------------
+-----------------------------------------------------------------------------------------+
| status_code |
SYSERR_UPGRADE_OPS_PREVIEW_RESPONSE |

VMware by Broadcom 358

VMware Avi Load Balancer Administration Guide

| status |
Checks preview for upgrade operations. |
| checks
| |
| | 'Checking
Controller Cluster readiness for upgrade operations.' |
| | 'Check
if upgrade operation is already in progress.' |
| | 'Checking
active versions compatibility for upgrade operations.' |
| | 'Checking
if system in Avi-ESSENTIALS/BASIC mode and allows only permitted operations.' |
| | 'Checking
if the cloud api versions are compatible after upgrade.' |
| | 'Checking
ServiceEngineGroup has an ongoing upgrade operation.' |
| | 'Checking
Controller Cluster disk space for upgrade operations.' |
| | 'Checking
image version compatibility for upgrade operations.' |
| | 'Checking
idempotent operations for upgrade operations.' |
| | 'Checking
Image state across Cluster members for upgrade operations.' |
| | 'Checking
the system configuration.' |
| | 'Checking
config migration.' |
| | 'Checking
if Gslb Maintenance Mode is enabled prior to upgrade.' |
| | 'Checking
if Gslb Feature is enabled and provides feature specific messages.' |
| | 'Checking
and inform user to take a backup prior to upgrade operations.' |
| | 'Checking
if Docker version is compatible.' |
| | 'Checking
for the mandatory patch in image bundle.' |
| | 'Checking
if configured IP type is DHCP or STATIC.' |
| | 'Checking
if se linux is enabled on controller nodes.' |
| | 'Checking
total number of alerts for upgrade operations.' |
+-------------
+-----------------------------------------------------------------------------------------+

Note Presently, the prechecks_only flag is not available through Avi Load Balancer Controller UI.

Using the UI
The Avi Load Balancer supports improved and more flexible methods for upgrading the system.
The Controller supports all the upgrade workflows through the UI.

VMware by Broadcom 359

VMware Avi Load Balancer Administration Guide

Following are the additional features for Flexible Upgrades using the Avi Load Balancer UI:

n Upgrading System

n Upgrading only the Controller

n Upgrading the Controller and the Service Engine groups

n Upgrading only Service Engine Group – You can upgrade a single SE group or multiple SE
groups, as needed. The transition of all the SE groups to the new version can occur over a
long period. Upgrades of different SE groups are supported with different patch versions.

Note
n Introduces a new flag 'prechecks_only' for all upgrade operations (Upgrade, Patch, Rollback),
When prodivded only the pre-checks are performed and does not trigger upgrade operation.
It is safe to perform upgrade operation with 'prechecks_only' flag outside of maintenance
window.

n Controllers will fail to upgrade if they are in the Essentials flavor (less than 6 cores or 32 GB of
memory).

The following sections explain the different options available for the Avi Load Balancer software’s
upgrade process through the UI.
Upload the Software
To upload the Avi Load Balancer software:

1 Login to the Avi Load Balancer UI, and navigate to Administration > Controller > Software.
Click Upload From Computer to upload the Avi Load Balancer software to the Controller.

2 Once the file is selected, the upload of the software image starts. The status of the image
upload progress is displayed on the UI.

VMware by Broadcom 360

VMware Avi Load Balancer Administration Guide

3 After the image upload is complete, the software package is available under the Software
tab.

Upgrade the Avi Load Balancer System

1 Navigate to Administration > Controller > System Update, select the image uploaded in the
previous step and click UPGRADE to start the upgrade process.

2 Select the Upgrade All Service Engine Groups check-box for selecting the SE groups update
along with the Controller upgrade. Click Continue to proceed with the Controller and SE
groups software upgrades.

VMware by Broadcom 361

VMware Avi Load Balancer Administration Guide

The following screen appears for the final checks before the upgrade proceeds.

VMware by Broadcom 362

VMware Avi Load Balancer Administration Guide

3 Do not select the check-box for the SE group update if the requirement is to upgrade the
Controller only.

4 When selecting the SE group update, the following two options are available if the SE groups
updates fail:

n Continue – The upgrade process for SE groups continues in the event of an error, failure,
or any other issues.

n Suspend – The upgrade process will not progress in the event of an error, failure, or other
such issues.

5 Confirm that the backup of configuration has been performed, when prompted for.

6 The progress for the update process is available on the UI under the In Progress section.

7 Once the upgrade process is complete, the latest software version will be available under the
Administration > Controller > System Update tab. The current tag is displayed next to the
updated software version.

8 Once the Controller upgrade is successful, the Service Engine group upgrade is initiated,
as the SE group upgrade was chosen in step no. 2. The following screenshot exhibits the
message regarding the SE group update status.

VMware by Broadcom 363

VMware Avi Load Balancer Administration Guide

9 Once the SE group update is successful, the upgrade status gets changed to successful, as
shown below.

SE Group Update
Use the same steps as mentioned in the previous section to update one or more than one SE
group. Navigate to Administration > Controller > SEG Update, select the required SE group, and
click on UPGRADE to proceed with the update process.

Starting with Avi Load Balancer 22.1.3, SE group upgrade can be initiated in both admin and
non-admin tenants as required.

Using the CLI

n Use the upgrade controller image_ref <image name> command to upgrade the
Controller to a base image.

[admin:-controller]: >upgrade controller image_ref 18.2.6-9000-20191031.063017

VMware by Broadcom 364

VMware Avi Load Balancer Administration Guide

n Use the upgrade controller image_ref <image name>controller_patch_ref <patch

name> command to upgrade the Controller to a base image and an Avi Load Balancer
Controller patch.

[admin:-controller]: >upgrade controller image_ref 18.2.6-9134-20191101.042535

controller_patch_ref 18.2.6-9134-2p1-20190806.011824

Using the REST API

This section discusses about Using Avi Load Balancer REST API.

n Use the following API to upgrade the Controller to a base image.

API: /api/upgrade

Method: POST

JSON Data:

{
'image_uuid': 'image-b8adc2bd-d27f-469d-b78d-5e2bc14a14e4'
}

n Use the following API to upgrade the Controller to a base image and the Avi Load Balancer
Controller patch.

API: /api/upgrade

Method: POST

JSON Data:

{
'image_uuid': 'image-b8adc2bd-d27f-469d-b78d-5e2bc14a14e4',
'controller_patch_uuid': 'image-e3aaad68-5aaf-485a-8bd9-1db3ec562d6a'
}

Tracking Avi Load Balancer Controller and Service Engine Upgrade Status in
AWS Cloud
The upgrade version and the last upgrade date information are available for the Controller and
SEs for deployment in the AWS cloud.

Tags To Check Upgrade Status

The following tags are available for the Controller and SEs to track the upgrade status:

n AVI_VERSION - the latest Avi Load Balancer version for the Controller and Service Engine

n AVI_UPGRADED_ON - the date and the timestamp when the software version upgrade was
completed.

VMware by Broadcom 365

VMware Avi Load Balancer Administration Guide

Tagging Avi Load Balancer Controller

These tags are available only for Controllers, which are deployed with IAM roles attached.
Upgrade tags are not supported for the deployment of the Controller using the access key
and secret key. The upgrade version and the last upgrade date information are available for
Controller and SEs for deployment in the AWS cloud.

Tagging Avi Load Balancer SEs

SEs have these tags populated irrespective of the access methods used for AWS cloud creation.

Checking Upgrade Status

The tags are available in the AWS Console for the Controller and SE instances.

The above screenshot depicts Avi Load Balancer at version 21.1.1, and the upgrade was
successfully completed on 2021-07-21 at 06:26:10 UTC.

Note On initial deployment, AVI_UPGRADED_ON shows the build date for the Controller and SE
software.

Restore the VMware Avi Load Balancer Configuration

VMware Avi Load Balancer introduces an improved sytem restore capability in 30.2.1 version. The
new restore capability provides visibility over the progress of the restore process.

System restore now runs a series of pre-checks to ensure that the new Controller is compatible
with the Avi Load Balancer configuration being restored. The restore script used in the previous
versions of Avi Load Balancer Controller has been replaced with the CLI/API method. The earlier
restore process has been discontinued.

Note
n Restore should be done with single Controller and then form 3 node cluster (if needed).

n Make sure to use the configuration file that was exported with “full_system” option.

n SEs to connect back to Controllers post disaster recovery operation, make sure one of the
controller IPs matches with the previous controller IPs (before).

This restore process consists of importing of the backup configuration onto the Avi Load
Balancer Controller.

VMware by Broadcom 366

VMware Avi Load Balancer Administration Guide

Before you start system restore process, make sure the following files are available on Avi Load
Balancer Controller:

n Active Images in the System (Controller and SE Groups).

n Follow the steps mentioned in Upgrading Avi Controller to upload the System Images.

n Active Patches in the System (Controller and SeGroups).

n Follow the steps mentioned in Patch Upgrades for Avi Controller to upload the Patch
Images.

Identifying Images and Files required for Controller Restore:

n Perform prechecks_only operation for Controller restore. It is safe to perform this operation
at any time, it only runs the pre-checks and does not start restore operation.

n Run the following command in CLI:

n restore configuration file <path/to/avi_config.json> prechecks_only

n Use the show upgrade status comamnd to monitor the pre-check status and use show
upgrade status detail filter pre_check_status to view the pre-check warnings
& errors.

Note
n Uploading System Images and Patch Images are mandatory to perform Controller Restore
operation.

n Uploading of file Objects are optional. User may choose to upload them after restore if
required.

n Re-upload GSLB Geo-DB files after restore operation is completed in case of GSLB
deployment.

Follow the steps below to start the restore process:

1 Copy backup file using SCP. Use scp /var/backup/avi_config.json

admin@<controller-ip>://tmp/avi_config.json command to copy files to the
Controller.

2 Log in to the Avi Load Balancer Controller CLI using the administrator credentials.

3 Run the following restore command using CLI. This passphrase is the same password that
was used while creating the backup file.

restore configuration <path/to/file> passphrase <passphrase>

[admin:ctrl-3node1]: > restore configuration file /home/admin/config.json

Please enter the passphrase for the configuration: +-------------
+------------------------------------------------------------------------------------+
| Field |
Value
| +-------------

VMware by Broadcom 367

VMware Avi Load Balancer Administration Guide

+------------------------------------------------------------------------------------+
| checks
| |
| | 'Checking Controller Cluster readiness for Restore
operations.' | | | 'Check if upgrade operation is
already in progress.' | | | 'Checking
ServiceEngineGroup has an ongoing upgrade operation.' | |
| 'Checking if backup config version can be restored on current controller
version.' | | | 'Checking if backup config version has FIPS
enabled.' | | | 'Checking if all images
objects from the config are present on the controller.' | | |
'Checking if the configuation is valid.'
| | | 'Checking if FIPS mode of backup configuration against
controller environment.' | | | 'Checking Controller Cluster disk space
for Restore operations.' | | | 'Checking if active
versions post restore exceeds max allowed active versions.' | | |
'Checking if system package exists in disk.' |
| | 'Checking if migration across versions are required for restore
operation.' | | | 'Checking user consent prior to restore
operations.' | | | 'Checking if all file objects
from the config are present on the controller.' | | | 'Checking if
the Service Engines are attached to the controller.' | |
| 'Checking the system configuration.'
| | | 'Checking for the patch applied as part of
restore.' | | status | Checks preview for
the requested operation. | | status_code
| SYSERR_UPGRADE_OPS_PREVIEW_RESPONSE
| +-------------
+------------------------------------------------------------------------------------+
+--------+---------------------------------------------------------------------------------
+ | Field |
Value | +--------
+---------------------------------------------------------------------------------+ |
status | 'Restore of Controller started. Use 'show
upgrade status' to check the status.' | +--------
+---------------------------------------------------------------------------------+

Check the status of restore step using the show upgrade status command as shown below.

[admin:ctrl-3node1]: > show upgrade status

+---------------+--------+---------------+--------------------------------------
+--------------+-----------------------------+-------+--------+
| Name | Tenant | Cloud | State |
Operation | Image | Patch | Reason |
+---------------+--------+---------------+--------------------------------------
+--------------+-----------------------------+-------+--------+
| cluster-0-1 | admin | - | UPGRADE_PRE_CHECK_IN_PROGRESS :(95)% |
EVAL_RESTORE | 30.2.1-9044-20240404.235217 | - | - |
| Default-Group | admin | Default-Cloud | UPGRADE_FSM_INIT |
None | 30.2.1-9044-20240404.235217 | - | - |
| 100.65.9.248 | admin | Default-Cloud | UPGRADE_FSM_INIT |
None | 30.2.1-9044-20240404.235217 | - | - |
| 100.65.9.249 | admin | Default-Cloud | UPGRADE_FSM_INIT |
None | 30.2.1-9044-20240404.235217 | - | - |
+---------------+--------+---------------+--------------------------------------

VMware by Broadcom 368

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 369

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 370

VMware Avi Load Balancer Administration Guide

04:48:34 |
| end_time | 2024-04-05
04:48:34 |
| duration | 0
sec |
+--------------------
+----------------------------------------------------------------------------------+

In the below example, the pre checks for restore is failed with a status code. Check the reason
and take respective actions. In the below example, the restore is again resume after uploading
image 30.2.1-9044-20240404.235217.

[admin:ctrl-3node1]: > restore configuration file /home/admin/config.json skip_warnings

Please enter the passphrase for the configuration:
+-------------
+------------------------------------------------------------------------------------+
| Field |
Value |
+-------------
+------------------------------------------------------------------------------------+
| checks
| |
| | 'Checking
Controller Cluster readiness for Restore operations.' |
| | 'Check
if upgrade operation is already in progress.' |
| | 'Checking
ServiceEngineGroup has an ongoing upgrade operation.' |
| | 'Checking
if backup config version can be restored on current controller version.' |
| | 'Checking
if backup config version has FIPS enabled.' |
| | 'Checking
if all images objects from the config are present on the controller.' |
| | 'Checking
if the configuation is valid.' |
| | 'Checking
if FIPS mode of backup configuration against controller environment.' |
| | 'Checking
Controller Cluster disk space for Restore operations.' |
| | 'Checking
if active versions post restore exceeds max allowed active versions.' |
| | 'Checking
if system package exists in disk.' |
| | 'Checking
if migration across versions are required for restore operation.' |
| | 'Checking
user consent prior to restore operations.' |
| | 'Checking
if all file objects from the config are present on the controller.' |
| | 'Checking
if the Service Engines are attached to the controller.' |
| | 'Checking
the system configuration.' |

VMware by Broadcom 371

VMware Avi Load Balancer Administration Guide

Use theshow upgrade status command and the show upgrade status detail filter
controller command to check the restore progress.

[admin:ctrl-3node1]: > show upgrade status

+---------------+--------+---------------+-----------------------+-----------
+-----------------------------+-------+--------+
| Name | Tenant | Cloud | State
| Operation | Image | Patch | Reason |
+---------------+--------+---------------+-----------------------+-----------
+-----------------------------+-------+--------+
| cluster-0-1 | admin | - |
UPGRADE_FSM_COMPLETED | RESTORE | 30.2.1-9044-20240404.235217 | - | - |
| Default-Group | admin | Default-Cloud |
UPGRADE_FSM_INIT | None | 30.2.1-9044-20240404.235217 | - | - |
| 100.65.9.249 | admin | Default-Cloud |
UPGRADE_FSM_INIT | None | 30.2.1-9044-20240404.235217 | - | - |
| 100.65.9.248 | admin | Default-Cloud |
UPGRADE_FSM_INIT | None | 30.2.1-9044-20240404.235217 | - | - |
+---------------+--------+---------------+-----------------------+-----------
+-----------------------------+-------+--------+

[admin:ctrl-3node1]: > show upgrade status detail filter controller

+-----------------------

VMware by Broadcom 372

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 373

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 374

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 375

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 376

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 377

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 378

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 379

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 380

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 381

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 382

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 383

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 384

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 385

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 386

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 387

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 388

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 389

VMware Avi Load Balancer Administration Guide

Use the show serviceengine command to verify status of Service Engines after restore:

[admin:ctrl-3node1]: > show serviceengine

Note Follow the steps below to restore configuration in a cluster set-up.

1 Restore the configuration on one of the nodes

2 Reform the cluster by inviting the two new nodes to the cluster.

Restoring Avi Load Balancer Controller using API Method

API request accepts the exported configuration file and the passphrase which which the
configration file was exported as request. The followings are the API specifications to be used to
restore an Avi Load Balancer Controller.

n API: /api/configuration/restore

n Method: POST

n Body: {“passphrase”: “_passphrase_”,“skip_warnings”: true}

n Content-Type: multipart/form-data.

n Single configuration backup file shall be attached with the request.

root@ctrl-3node1:/home/admin# curl -X POST https://100.65.9.244/api/configuration/restore -k

--user admin:admin --form file='@/home/admin/config.json' -F passphrase=**** -F consent=true
--header 'X-Avi-Version: 30.2.1'
{
"status": "'Restore of Controller started. Use 'show upgrade status' to check the status.'"
}

VMware by Broadcom 390

VMware Avi Load Balancer Administration Guide

Use the following curl command to check the restore progress using the API method.

curl -X GET https://100.65.9.244/api/upgradestatusinfo -k --user admin:****

System Report Option for Upgrade Operations

Avi Load Balancer 22.1.6 supports System Reports for every upgrade operation with the
summary of the operation performed and pre-check results from the operation.

A system report is generated for every upgrade operation, the report contains summary of the
operation performed and precheck results from the operation. Unlike the pre-checks tied to
upgradestatusinfo, system report object is retained over multiple iterations.

System Report can be viewed and downloaded from Avi Load Balancer CLI. Login to Avi Load
Balancer controller and use the show systemreport # Prints Summary command.

Command: show systemreport [<report-name> [export]]

n Options:

n Default - List all available report.

n Example: show systemreport

n name:

n Show details of specified report.

n Ex: show systemreport upgrade_system_20231108-142156

n export

n Export the report to specified path. [Downloads the full report]

n Ex: show systemreport upgrade_system_20231108-142156 export

Below is a sample output for generating a system report for Avi Load Balancer Controller.

[admin:100-65-10-137]: > upgrade controller image_ref 30.2.1-8143-20240115.080016

VMware by Broadcom 391

VMware Avi Load Balancer Administration Guide

| |
| | 'Checking Controller Cluster readiness for upgrade
operations.' |
| | 'Check if upgrade operation is already in
progress.' |
| | 'Checking active versions compatibility for upgrade
operations.' |
| | 'Checking if system in Avi-ESSENTIALS/BASIC mode and allows only permitted
operations.' |
| | 'Checking if the cloud api versions are compatible after
upgrade.' |
| | 'Checking ServiceEngineGroup has an ongoing upgrade
operation.' |
| | 'Checking Controller Cluster disk space for upgrade
operations.' |
| | 'Checking image version compatibility for upgrade
operations.' |
| | 'Checking idempotent operations for upgrade
operations.' |
| | 'Checking Image state across Cluster members for upgrade
operations.' |
| | 'Checking the system
configuration.' |
| | 'Checking config
migration.' |
| | 'Checking if Gslb Maintenance Mode is enabled prior to
upgrade.' |
| | 'Checking if Gslb Feature is enabled and provides feature specific
messages.' |
| | 'Checking and inform user to take a backup prior to upgrade
operations.' |
| | 'Checking if Docker version is
compatible.' |
| | 'Checking for the mandatory patch in image
bundle.' |
| | 'Checking if configured IP type is DHCP or
STATIC.' |
| | 'Checking if se linux is enabled on controller
nodes.' |
| | 'Checking total number of alerts for upgrade
operations.' |
+-------------
+-----------------------------------------------------------------------------------------+
Starting upgrade
+--------------------
+---------------------------------------------------------------------------------------------
---+
| Field |
Value
|
+--------------------
+---------------------------------------------------------------------------------------------
---+
| status_code |
SYSERR_EVAL_UPGRADE_CONTROLLER_STARTED

VMware by Broadcom 392

VMware Avi Load Balancer Administration Guide

|
| status | 'Pre-checks for Upgrade of Controller started. Use 'show upgrade
status' to check the status.' |
| system_report_uuid | systemreport-6f499bff-fcbd-4fca-
a43a-483d1764eed5 |
+--------------------
+---------------------------------------------------------------------------------------------
---+
[admin:100-65-10-137]: > show systemreport
+--------------------------------+---------------------------------------------------
+---------------------------+-----------------------------+
| Name | UUID |
State | Image |
+--------------------------------+---------------------------------------------------
+---------------------------+-----------------------------+
| upgrade_controller_checks_f345 | systemreport-6f499bff-fcbd-4fca-a43a-483d1764eed5 |
UPGRADE_PRE_CHECK_WARNING | 30.2.1-8143-20240115.080016 |
+--------------------------------+---------------------------------------------------
+---------------------------+-----------------------------+
[admin:100-65-10-137]: > show systemreport upgrade_controller_checks_f345
+-------------------------
+----------------------------------------------------------------------------------+
| Field |
Value |
+-------------------------
+----------------------------------------------------------------------------------+
| uuid | systemreport-6f499bff-fcbd-4fca-
a43a-483d1764eed5 |
| name |
upgrade_controller_checks_f345 |
| tenant_ref |
admin |
| image_ref |
30.2.1-8143-20240115.080016 |
| archive_ref | report://
upgrade_controller_checks_f345.tar.gz |
| state
| |
| state |
UPGRADE_PRE_CHECK_WARNING |
| last_changed_time | Tue Jan 16 09:02:02 2024 ms(167792166)
UTC |
| summary
| |
| name | Prechecks for Controller
Upgrade |
| description | System evaluation report to perform Controller Upgrade from
22.1.6 to 30.2.1 |
| previews[1] | 'Checking Controller Cluster readiness for upgrade
operations.' |
| previews[2] | 'Check if upgrade operation is already in
progress.' |
| previews[3] | 'Checking ServiceEngineGroup error recovery options prior to
upgrade operations. |
| |

VMware by Broadcom 393

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 394

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 395

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 396

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 397

VMware Avi Load Balancer Administration Guide

VMware by Broadcom 398

VMware Avi Load Balancer Administration Guide

| description | 'Checking if Gslb Feature is enabled and provides feature

VMware by Broadcom 399

VMware Avi Load Balancer Administration Guide

[admin:100-65-8-89]: > show systemreport

VMware by Broadcom 400

VMware Avi Load Balancer Administration Guide

| upgrade_controller_checks_169b | SYSTEM_REPORT_ERROR | Tue Feb 27 2024 05:08:10 |

| upgrade_controller_checks_601c | SYSTEM_REPORT_ERROR | Wed Feb 28 2024 08:46:55 |
+--------------------------------+---------------------+--------------------------+
[admin:100-65-8-89]: > delete systemreport upgrade_controller_checks_8ec5
Successfully deleted upgrade_controller_checks_8ec5.
[admin:100-65-8-89]: > show systemreport
+--------------------------------+---------------------+--------------------------+
| Name | State | Timestamp |
+--------------------------------+---------------------+--------------------------+
| upgrade_controller_checks_67a0 | SYSTEM_REPORT_ERROR | Tue Feb 27 2024 04:59:28 |
| upgrade_controller_checks_d3f0 | SYSTEM_REPORT_ERROR | Tue Feb 27 2024 05:05:47 |
| upgrade_controller_checks_169b | SYSTEM_REPORT_ERROR | Tue Feb 27 2024 05:08:10 |
| upgrade_controller_checks_601c | SYSTEM_REPORT_ERROR | Wed Feb 28 2024 08:46:55 |
+--------------------------------+---------------------+--------------------------+
[admin:100-65-8-89]: >

Note A periodic cleanup is performed to keep the count within the threshold every hour by
default. The interval is configurable via system_report_cleanup_interval in Controller properties.

Must-Checks or Prechecks
This section explains the must-checks that are executed prior to the upgrade operations. Must-
checks are specific to upgrade operations. It returns the messages as warnings or errors.
Warnings can be skipped while errors cannot be overridden. Avi Load Balancer REST API/Avi
Load Balancer CLI presents the skip_warnings option to control the above behavior.

n For REST API: add /preview/ at the end of APIs described in V2 APIs to get previews for that
particular flow

n For Avi Load Balancer CLI: This is directly integrated into the normal work-flow and there is
no separate command.

Starting with Avi Load Balancer version 30.2.1, users can trigger the mandatory checks and
navigate away from the System Update or SEG Update page. They just need to be present on
the page at the time of completion of mandatory checks (called pre-checks ahead) to be able to
resume the update in case the pre-checks end with suppressible warnings.

Folllowing are the list of steps intiated as soon as an upgrade process is started:

1 Login to Avi Load Balancer UI, navigate to Administration > Controller > Software, and
select the Upload From Computer option to select the required image and start the upgrade
process.

2 As soon as the uploading of the image is completed, it is available under Administration >
Controller > System Update or Administration > Controller > SEG Update (depending on the
type of image selected).

VMware by Broadcom 401

VMware Avi Load Balancer Administration Guide

3 Navigate to Administration > Controller > System Update, select the uploaded image
available under Available Update, and use the UPGRADE option, as shown below.

4 Once the UPGRADE option is selected, the following pop-up screen appears. It shows a list
of recommended checks before starting the upgrade process and options to select an action
in case there is an upgrade failure. The three options available in case of an Sytem/SEG
update failure are - Continue, Rollback, or Suspend. In the below example, we are selecting
the Suspend option.

VMware by Broadcom 402

VMware Avi Load Balancer Administration Guide

5. Once you click UPGRADE, mandatory checks are triggered, and the status of the same is
available on the UI, as shown below. The status of prechecks or mandatory checks is available
under the IN PROGRESS section.

6. You can click on the VIEW PROGRESS option to check the status of the ongoing prechecks.
The transcript box shown below has the list of prechecks performed.

Prior to Avi Load Balancer version 30.2.1, the list of completed checks was not visible on Avi
Load BalancerUI, only the progress status was available. While the mandatory checks are in
progress, you can navigate away from the System Update page and perform any other activities
if required. Please note that you need to be present on the System Update page at the time of
completion of mandatory checks to be able to resume the update in case the pre-checks end
with supressible warnings.

VMware by Broadcom 403

VMware Avi Load Balancer Administration Guide

7. If the mandatory checks end with one or more warnings, select the Suppress option. Selecting
the Suppress option ignores the warning, restart the precheck process and update the System.

8. If you are not at the System Update page, when the prechecks are completed, you can
see the overall status, as shown below. Any warning with no action taken will stop the update
process.

9. The following message appears on the Controller UI in case mandatory checks are completed
and the upgrade is successful.

VMware by Broadcom 404

VMware Avi Load Balancer Administration Guide

Additional Options for Flexible Upgrades

This section explains the following additional options available for Flexible Upgrades:

n Rollback - Error Recovery

n Stop Cleanup

n SE Group Resume Option

Rollback – Error Recovery

n If the upgrade process hits an error, then an automated rollback error recovery is initiated to
bring the Avi Load Balancer Controller(s) into a known good state.

n If the SE group encounters an error, SE group will suspend upgrade if the

suspend_on_failure flag is used. Otherwise, it will continue to upgrade the rest of the Service
Engines in the SE group.

n If the error happens in the context of a patch, then the patch will be rollback.

n If an error is encountered during the execution of rollback mechanism, then it will be treated
as upgrade cancelled.

Stop Cleanup
Whenever the rollback operation is triggered and it fails, then the Avi Load Balancer Controller or
SE group will be moved to the abort state. In Flexible Upgrades, these states can be cleaned up
and Avi Load Balancer Controller and SE group are transitioned to a known stable state.

Using SE Group Resume Option

Note SE group resume option is supported only from Avi Load Balancer release 18.2.8.

Whenever an SE group is upgraded with suspend_on_failure enabled and an issue is

encountered, the upgrade process for that SE group is suspended. After the issue is resolved
through manual intervention, use the following options to resume the upgrade:

n Se-group-uuids — Specify the SE group that needs to be resumed.

n Ignore_failure — This field overrides the earlier suspend on failure. The upgrade will take
place in an unconditional manner and will proceed even if there is a failure in the subsequent
upgrade iteration. Default value is false.

n Skip-suspended — This field will skip the SE(s) that are suspended in the previous upgrade
iteration and proceed with the remaining SE(s) in the group. The default value is false.

[admin:controller]: >resume segroup se_group_refs <se-group-name>

[admin:controller]: >resume segroup se_group_refs seg-a

VMware by Broadcom 405

VMware Avi Load Balancer Administration Guide

The following options are available for resuming SE Group Upgrade with Options:

[admin:controller]: >resume segroup skip_suspended se_group_refs Default-Group

action_on_error continue_upgrade_ops_on_error

[admin:controller]: >resume segroup skip_suspended se_group_refs Default-Group

action_on_error suspend_upgrade_ops_on_error

Use the following API POST method to resume SE group upgrade.

API: /api/segroup/resume
POST /api/segroup/resume
JSON data:
{
"se_group_uuids": [
"serviceenginegroup-ec9c8141-844d-467d-bdc0-d7855e9d8419"
],
"skip_warnings": true
}

Note When skip_warnings": true is used, upgrade proceeds without exhibiting warnings
messages and upgrade previews.

Resume with other options:

Option: Skip suspended SE’s and continue with the upgrade, update the se_group
action_on_error with CONTINUE_UPGRADE_OPS_ON_ERROR.

{
"se_group_options": {
"action_on_error": "CONTINUE_UPGRADE_OPS_ON_ERROR",
"skip_suspended": true
},
"se_group_resume_options": {
"action_on_error": "CONTINUE_UPGRADE_OPS_ON_ERROR",
"skip_suspended": true
},
"se_group_uuids": [
"serviceenginegroup-ec9c8141-844d-467d-bdc0-d7855e9d8419"
],
"skip_warnings": true
}

Option: Skip suspended SE’s and continue with the upgrade.

{
"se_group_options": {
"action_on_error": "SUSPEND_UPGRADE_OPS_ON_ERROR",
"skip_suspended": true
},
"se_group_resume_options": {
"action_on_error": "SUSPEND_UPGRADE_OPS_ON_ERROR",
"skip_suspended": true

VMware by Broadcom 406

VMware Avi Load Balancer Administration Guide

},
"se_group_uuids": [
"serviceenginegroup-ec9c8141-844d-467d-bdc0-d7855e9d8419"
],
"skip_warnings": true
}

Creating SE Group using SE Group Template

During cloud initiation and SE Group creation, it is checked if the cloud has a Service Engine
group template. If the template is available, the base image or patch image is copied from
the SE group template. Otherwise, the image information is picked from the Avi Load Balancer
Controller.

For each cloud, there is a se_group_template_uuid. This is used to ensure that the newly created
SE Group follow the se_group_template_uuid.

Any SE group can be designated as the template. If an SE group (Seg1) is assigned as the default
SE group template, then the newly created SE group (Seg2) picks the base and patch image
from Seg as shown below.

[admin:controller]: > show upgrade status filter serviceenginegroup Seg2

The patch rollback option should be enabled as shown below.

[admin:controller]: > show upgrade status detail filter serviceenginegroup Seg2

VMware by Broadcom 407

VMware Avi Load Balancer Administration Guide

2p1 |
| patch_image_ref |
18.2.9-9000-2p1-20200430.133146 |
| state
| |
| state |
UPGRADE_FSM_INIT |
| last_changed_time |
Sat May 9 06:15:49 2020 ms(41) UTC |
| seg_status
| |
| notes[1] |
[2020-05-09 06:15:49] Init segroup(seg3 defaults to seg-template(seg1). |
| start_time |
2020-05-09 06:15:49.406502 |
| enable_rollback |
False |
| enable_patch_rollback |
True |
| progress |
0 percent |
| tenant_ref |
admin |
| obj_cloud_ref |
Default-Cloud |
+-----------------------
+-------------------------------------------------------------------------|

Upgrading SE Group
This interface is used to upgrade all or some of the SE groups.

Using UI
The following section details the steps to perform SE Group Update through the Avi Load
Balancer Controller UI.

Navigate to Administrator > Controller > SEG Update, select the required SE group, and click
UPGRADE to proceed with the update process.

Starting with Avi Load Balancer 22.1.3, SE group upgrade can be initiated in both admin and
non-admin tenants as required.

VMware by Broadcom 408

VMware Avi Load Balancer Administration Guide

Using the CLI

n Use the upgrade segroup se_group_refs Default-Group image_ref<image name>

command to upgrade an SE group to the Controller image.

[admin:-controller]: >upgrade segroup se_group_refs Default-Group image_ref

18.2.6-9134-20191101.042535

n Use the upgrade segroup se_group_refs Default-Group image_ref *lt;Controller

image> se_patch_ref <SE patch name&gt: command to upgrade an SE group to the
Controller image and the SE patch image.

[admin:-controller]: >upgrade segroup se_group_refs Default-Group image_ref

18.2.6-9134-20191101.042535 se_patch_ref 18.2.6-9134-2p1-20190806.011824

Using the REST API

SE Group UUID is obtained by the GET /api/serviceenginegroup API.

The followings are the additional options for SE group upgrade:

Disruptive

This option disables a non-disruptive mechanism to facilitate a faster upgrade. If enabled, the
SE(s) are upgraded in a disruptive manner. The default value is false.

Suspend-on-failure

VMware by Broadcom 409

VMware Avi Load Balancer Administration Guide

This option suspends the upgrade of subsequent SE(s) within a SE-group when a failure is
encountered in the SE upgrade path. The default value is false.

The followings are the different APIs for the SE group upgrade:

n Use the following API to upgrade the SE group to the Controller image.

API: /api/upgrade

Method: POST

JSON Data:

{
'image_uuid': 'image-b8adc2bd-d27f-469d-b78d-5e2bc14a14e4',
'se_group_uuids': [
'serviceenginegroup-e553b1a6-4851-4e82-ad12-cecc4bbda6c7'
]
}

n Use the following with the additional SE Group options — Disruptive and Suspend_on_failure.

API: /api/upgrade

Method: POST

JSON Data:

{
'image_uuid': 'image-b8adc2bd-d27f-469d-b78d-5e2bc14a14e4',
'se_group_uuids': [
'serviceenginegroup-e553b1a6-4851-4e82-ad12-cecc4bbda6c7'
],
'disruptive':true,
'suspend_on_failure': true
}

n Use the following API to upgrade the SE group to the Controller image and the SE patch
image.

API: /api/upgrade

Method: POST

JSON Data:

{
'image_uuid': 'image-b8adc2bd-d27f-469d-b78d-5e2bc14a14e4',
'se_patch_uuid': 'image-e3aaad68-5aaf-485a-8bd9-1db3ec562d6a',
'se_group_uuids': [
'serviceenginegroup-e553b1a6-4851-4e82-ad12-cecc4bbda6c7'
]
}

VMware by Broadcom 410

VMware Avi Load Balancer Administration Guide

Additional Options for SE Group Upgrade

The following upgrade options are available for upgrading SE group:

Option Behaviour Notes

SUSPEND_UPGRADE_OPS_ON_FAILURE This option is used to suspend the It is enabled by default.This option

upgrade operations (Upgrade/Patch) serializes the SE upgrades in the
on SE-Group if the SE(s) hit an issue SE group upgrade. It increases the
and does NOT come up during the overall upgrade time for the entire SE
upgrade operations. group. Batch size is used to decrease
the upgrade time. Even if the SEs do
not have scaled-out virtual services, it
still upgrades serially.

CONTINUE_UPGRADE_OPS_ON_FAILURE This option is used to continue the This option parallelizes the SE
upgrade or patch upgrade operations upgrade in the SE group upgrade if
on the SE group even if the SE(s) SEs do not have scaled-out virtual
hit an issue and does not come services. If SEs have scaled-out virtual
up during the upgrade operations. services, then they continue with
Service disruption can be observed. serial upgrades.

Disruptive This option is used to disable the This option is disabled by default.
non-disruptive nature of SE upgrades. All SE(s) will be upgraded in parallel,
It is used to upgrade all the SE(s) irrespective of scaled-out virtual
in the group to the next version service existence. Traffic/Service
irrespective of the traffic disruption. disruption will take place.

Check Failed Error during Service Engine Upgrade

The Avi Load Balancer SE upgrade can fail with the error Check failed. This topic covers the
troubleshooting and resolution steps for this error.

The following errors were observed in the SE log:

[127.1.0.3] Executing task 'se_upgrade_check'

cat: /host/prev/bootstrap/VERSION: No such file or directory
se_upgrade_check error exception: Error reading SSH protocol banner

The following error was observed in the secure_channel.log (in /var/log/upstart in SE CLI):

AVI-SE:/var/log/upstart
Authenticated to 10.1.1.1 ([10.1.1.1]:22).^M

connect_to localhost: unknown host (Name or service not known)^M

VMware by Broadcom 411

VMware Avi Load Balancer Administration Guide

Resolution
The errors observed in the SE log file suggest that the entry for the localhost mapping to
127.0.0.1 is missing in /etc/hosts. This issue occurs when name resolution for the local host from
SE is not successful as shown in the following snippet.

root@app-node3:/var/log/upstart# ping localhost

ping: unknown host localhost

Check /etc/hosts file by using cat command. Note that the localhost entry was missing as
shown below.

app-node3.avi-systest.local-avitag-1root@app-node3:/# cat /etc/hosts

127.0.0.1 app-node3.avi-systest.local--nameless-abc-xyz

#used by abc_servers: node1.controller.local

{127.0.0.9, 10.140.88.197}

127.0.0.9 node1.controller.local

#used by abc_servers: node2.controller.local {127.0.0.8, 10.140.88.199}

127.0.0.8 node2.controller.local

#used by abc_servers: node3.controller.local {127.0.0.7, 10.140.88.198}

127.0.0.7 node3.controller.local

root@app-node3:/#

To solve this issue, add the localhost entry 127.0.0.1 localhost to the localhost file. After
adding the localhost entry, try again upgrading the SEs.

Additional Troubleshooting Tips

Usually, /etc/hosts content on the root1 folder has this entry. From the Avi Load Balancer
Controller, run the following command:

ssh -i /etc/ssh/id_se [email protected]

If ssh is successful, no issue is observed with the localhost entry.

Run the following command:

ssh -i /etc/ssh/id_se -p 5097 [email protected]

If ssh fails, add an entry to /etc/hosts file with the localhost 127.0.0.1 and retry ssh.

If ssh is successful, retry upgrading Avi Load Balancer SEs.

VMware by Broadcom 412

VMware Avi Load Balancer Administration Guide

Patch Guide
This section discusses about the patch upgrade options.

Upgrading using Patch Release

The followings are the available options for patch upgrade:

n System — Patch upgrade for Avi Load Balancer Controller and all SE groups.

n Controller — Patch upgrade for the Avi Load Balancer Controller alone.

n SE group — Patch upgrade for some or all the SE groups.

Note The following are a few points for a patch upgrade process:
n An image with a patch can be applied.

n The image and the patch must have the same base version.

n A patch cannot be applied without applying the image.

n Compatibility checks prevent incorrect patches from getting applied to different versions.

Patch Upgrade Process for Avi Load Balancer

The patches are designed not to interrupt active services. In cases where an interruption
is expected, the patch package is released with related documents and details. To ensure
configuration integrity, changes to the configuration are locked out during a patch upgrade.

Patch Process
1 Download a patch package from the Avi Load BalancerCustomer Portal.

2 There can be as many as three packages for every patch release.

3 Use the patch shell command to apply the desired patch as follows:

a The controller_patch and the se_patch provide the administrator with an option to
patch some but not all aspects of the Avi Load Balancer.

b By applying the se_patch, one has the flexibility to upgrade just some SE groups.

c The avi_patch (system patch) applies to all the other patches.

For more information, see Patch Upgrade Options.

Preparing for the Patch

This section discusses the steps, prerequisites and restrictions while preparing for the patch.
Finding the Version
One or more patch packages may be applicable to a specific Avi Load Balancer version.
Therefore, it is essential to know the version that the Avi Load Balancer is currently on. Check the
Avi Load Balancer Controller or SE version(s) using the following commands:

n show version controller

VMware by Broadcom 413

VMware Avi Load Balancer Administration Guide

n show version serviceengine

Prerequisites and Restrictions

n Choose the required patch package based on the Controller and SE versions.

n All Controllers must be on the same (base+patch) version to form a cluster. For instance,
assume: A – is the major release and x.y – is the maintenance version. You cannot form a
cluster with one Controller on A.x.y, one on A.x.y 2p1, and one on A.x.y 2p2.

n For more information, see Clustering Patched Controllers.

n All patches from a maintenance release are incorporated into successive maintenance
releases.

n Once a Avi Load Balancer Controller is upgraded to a new maintenance release, all underlying
SE groups must be upgraded to the same release as well.

n A patch family is the one in which the leading digit is the same. For instance, 1p1, 1p2, and 1p3
are patches in the 1px family.

n Fixes accumulate within a patch family. For instance, the 1p2 patch contains new fixes unique
to it, plus all the fixes from 1p1. The 1p3 patch includes fixes from both the 1p1 and 1p2
patches. Additionally, the 2p1 patch is the first in a new patch family and does not contain 1px
fixes.

n A given fix may appear in more than one patch family.

n The following options are allowed when selecting a patch version:

n Choose any patch applicable to a particular maintenance release as the first patch to be
applied to that base version.

For example, in a patch family comprised of 1p1, 1p2 and 1p3, any one of the three can be
the first applied.

n Apply any subsequent patch, as long as it is within the same patch family. For instance,
you can apply 1p5 to 1p1.

n The following options are not advisable while choosing a patch version:

n Applying a patch from a patch family other than the one already chosen.

For instance, you cannot apply patch 2p1 once any 1px patch has been applied.

n Apply a patch that would imply an upgrade to a different Avi Load Balancer maintenance
release.

For example, it is impossible to patch upgrade from 17.2.3 to 17.2.4-1p3.

n .pkg is the same for both container and non-container.

n For Controllers on BareMetal/LSC or legacy GCP, the upgrade package is available in

docker.tgz.

VMware by Broadcom 414

VMware Avi Load Balancer Administration Guide

Upload the Patch Package

Use WinSCP or any similar tool to upload the patch package to the Controller.

The following are the ways to upload patch image to the Avi Load Balancer Controller.

n Copy the downloaded patch image to the Avi Load Balancer Controller/tmp directory and
then upload it on Avi Load Balancer Controller using image API.

n Use the curl command to upload the respective patch packages.

Note The leader Controller ensures that the follower Controllers are on the same version.
The Controller machine on the base version of Avi Load Balancer might be previously patched.
Upload patch package by using image the API /api/image/.

Images should be uploaded before starting the upgrade process.

1 Use the upload image filename <file path> command to start uploading the image.

[admin:controller]: > upload image filename /tmp/se_patch.pkg

Note Image upload is supported only on the Cluster Leader.

2 Use the show image command to view the image.

[admin:10-50-54-123]: > show image

VMware by Broadcom 415

VMware Avi Load Balancer Administration Guide

3 Log in to the Avi Load Balancer shell using admin credentials. Use the show upgrade status
and show upgrade status detail commands to check the upgrade status.

[admin:controller]: > show upgrade status

+---------------+---------------+-----------------------+-----------
+-----------------------------+-------+
| Name | Cloud | State
| Operation | Image | Patch |
+---------------+---------------+-----------------------+-----------
+-----------------------------+-------+
| cluster-0-1 | - | UPGRADE_FSM_COMPLETED
| UPGRADE | 18.2.8-9000-20200212.075158 | - |
| Default-Group | Default-Cloud | UPGRADE_FSM_COMPLETED
| UPGRADE | 18.2.8-9000-20200212.075158 | - |
| se1 | Default-Cloud | UPGRADE_FSM_COMPLETED
| UPGRADE | 18.2.8-9000-20200212.075158 | - |
+---------------+---------------+-----------------------+-----------
+-----------------------------+-------+

Patch Upgrade Options

This section discusses about the version upgrade and patch of the Avi Load Balancer.
Version Upgrade and Patch
Avi Load Balancer Controller can be upgraded to a more recent version along with the required
patch as follows:

1 Use the upgrade controller image_ref <image> controller_patch_ref <patch> as

shown below to upgrade the Avi Load Balancer Controller along with a patch.

[admin:controller]: > upgrade controller image_ref 18.2.7-5000-20200213.181331

controller_patch_ref 18.2.7-5000-2p1-20200213.182111

2 Use the upgrade segroup image_ref <image> se_patch_ref <patch> command to

upgrade a SE group with a patch.

3 Use the upgrade system image_ref <image> controller_patch_ref <patch>

se_patch_ref <se_path> command to upgrade the Avi Load Balancer System (Controller
and SE groups) to the desired patch.

VMware by Broadcom 416

VMware Avi Load Balancer Administration Guide

This ensures that the Avi Load Balancer Controller is upgraded and the desired patch is applied,
at the same instance.

Note
n The patch should be of the same version as that of the Controller upgrade.

n se_group_options and se_group_resume options are available in the Avi Load Balancer
CLI.

Additional Options for Patch Upgrade

Apart from this, the following are the four options for the patch command:

n Disruptive Patch

n Controller Patch

n SE Group Patch

n System Patch
Disruptive Patch
The disruptive patch option is set to False by default. The se_group_refs attribute governs the
scope of the upgrade. If the non-disruptive rolling upgrade of Service Engines are not required,
this flag can be set to True to go through the upgrade process quickly. This flag can be set to
True, when the require.

The below command initiates an upgrade with the disruptive flag set to True.

[admin:controller]: > patch segroup

action_on_error The error recovery action configured for a SE
Group.

disruptive Disable non-disruptive

mechanism.

se_group_refs SE Groups subjected to patch

operations.

se_patch_ref Image name for identifying SE patch

image.

skip_warnings This is flag when set as true skips few optional must checks.

Note The action_on_error option is supported for SE group upgrade.

Controller Patch

[admin:10-50-54-122]: > patch controller controller_patch_ref

18.2.12-9110-2p1-20210220.231845

VMware by Broadcom 417

VMware Avi Load Balancer Administration Guide

SE Group Patch
If the se_group_refs option is not enabled, all SE groups are upgraded. When enabled, it
identifies a specific SE group for patching. If more than one SE group require patching, each
will require a separate patch command.

[admin:controller]: > patch segroup se_group_refs Default-Group se_patch_ref

18.2.8-9000-1p2-20200219.121101

[admin:controller]: > patch segroup se_group_refs Default-Group se_group_refs Default-Group

abc-group se_patch_ref 18.2.8-9000-1p2-20200219.121101

Note If the se_group_refs option is not enabled, all SE groups are upgraded.

System Patch
Use the following command to patch the Avi Load Balancer Controller along with system patch.

[admin:controller]: > patch system controller_patch_ref 18.2.8-9000-1p3-20200219.121643

se_patch_ref 18.2.8-9000-1p3-20200219.121643

action_on_error The error recovery action configured for a SE

Group.
disruptive Disable non-disruptive
mechanism.

skip_warnings This is flag when set as true skips few optional must checks.

Note
n SEs check for the version present on the Controller. In the event of a mismatch, the SE is
rebooted and upgraded with the new patch available on the Controller.

n If a patch 18.2.6-5p1 is applied to the SE group, then all the entities in the system (SEs and
Controller) — can only be upgraded to 5p1 or some member of the 5px patch series. For
example, a different patch series 6p1 can be applied to the Avi Load Balancer Controller and
5p1 to the SE group.

Patch Upgrade for Avi Load Balancer System

Use the following CLI command for the base image upgrade with a patch image.

[admin:controller]: > upgrade system image <image-name> controller_patch <controller-patch-

name> se_patch <se-patch-name>

VMware by Broadcom 418

VMware Avi Load Balancer Administration Guide

[admin:controller]: >upgrade system image 18.2.6 controller_patch 18.2.6-1p1 se_patch

18.2.6-1p1

1 Use the upgrade system image_ref <image name > controller_patch_ref <SE
patch name> command for the Avi Load Balancer.

[admin:-controller]: upgrade system image_ref 18.2.6-9000-20191031.063017

controller_patch_ref 18.2.6-2p1-20191031.063017

2 Use the upgrade system image_ref <image name> se_patch_ref <SE patch name>
command for the Avi Load Balancer.

[admin:-controller]: upgrade system image_ref 18.2.6-9000-20191031.063017 se_patch_ref

18.2.6-2p1-20191031.063017

3 Use the upgrade system image_ref <image name> controller_patch_ref

<Controller patch image> se_patch_ref <SE patch image> command for the system
upgrade with both Controller and SE patch.

[admin:-controller]:upgrade system image_ref 18.2.6-9000-20191031.063017

controller_patch_ref 18.2.6-2p1-20191031.063017 se_patch_ref 18.2.6-2p1-20191031.063017

Patch Upgrade for Avi Load Balancer Controller

This interface is used to patch upgrades for the Avi Load Balancer Controller.

Patch Upgrade Using UI

Patch Upgrade involves uploading the patch image for the controller and service engines.

The following patch upgrade options are available:

n Patch upgrade for Controller

n Patch upgrade for Controller and Service Engines

n Patch upgrade for Service Engines

Upload the Patch Image
1 Navigate to Administration > Controller > Software. Click Upload From Computer to upload
the patch image to the Controller. Once the patch file is selected, the upload of the patch
image starts. The status of the patch image upload progress is available on the UI. In the
following example, the Controller’s patch image is uploaded.

VMware by Broadcom 419

VMware Avi Load Balancer Administration Guide

2 A software image containing patch files for the Controller and Service Engines can also be
uploaded using the same process. The following screenshots exhibit the patch upload for the
Controller and Service Engine. Patch upgrades for only Service Engines are also supported.

3 To upgrade the Service Engine group with the patch image, use the patch upgrade file for
Service Engines. The process to patch upgrade the Controller or Service Engine Groups is the
same as mentioned for the main software release. For more information, see Using the UI.

4 Navigate to Administration > Controller > SEG Update, select the required Service Engine
Group, and click UPGRADE, as shown below.

VMware by Broadcom 420

VMware Avi Load Balancer Administration Guide

5 For Service Engine Group update, select only the SE patch, as shown below.

6 Once the patch update for the SE group is completed, the UI exhibits the status as successful,
as shown below. The selected SE group is updated with the 22.1.3-2p1 patch.

VMware by Broadcom 421

VMware Avi Load Balancer Administration Guide

Using the CLI

Use the upgrade controller image_ref <image name> controller_patch_ref
<Controller patch image command to upgrade the Avi Load Balancer Controller with a patch.

[admin:-controller]: upgrade controller image_ref 18.2.6-9000-20191031.063017

controller_patch_ref 18.2.6-2p1-20191031.063017

[admin:controller]: > patch controller <patch-name>

[admin:controller]: > patch controller controller_patch 18.2.5-5p1

Using the REST API

This section describes the using Avi Load Balancer REST API.

POST api/upgrade JSON data:{‘controller_patch_uuid’: <image-uuid>}

Patch Upgrade for SE Group

SE groups can be of different versions and different versions of patches can be applied.

Use the upgrade segroup image_ref <image name> se_group_refs Default-Group

se_patch_ref <patch for the SE Group> command to upgrade specific SE groups along
with a patch.

[admin:-controller]: upgrade segroup image_ref 18.2.6-9000-20191031.063017 se_group_refs

Default-Group se_patch_ref 18.2.6-2p1-20191031.063017

Note Patch name and patch UUID is retrieved from the image service.

API - HTTP PATCH Support

This section explains the usage of HTTP PATCH method.

Avi Load Balancer supports use of PATCH to perform the following operations:

n Update scalar fields (string, bool, int32, etc.).

n Add a new entry to a list in an object.

n Update an entry in a list in an object.

n Unset scalar fields - This causes the fields to be reset to their default values, if applicable.

n Delete an entry from a list in an object - According to section 5.1.1 of RFC2616, “The Method
token indicates the method to be performed on the resource identified by the Request-URI.
The method is case-sensitive.” So spell it “PATCH” to avoid “400 bad request” errors from
Avi Load Balancer when executing the HTTP PATCH request.

VMware by Broadcom 422

VMware Avi Load Balancer Administration Guide

Editing Nested Fields

In the current release, editing nested fields using PATCH is supported only to a field depth of 1.
PATCH is not supported for editing objects with deep nesting characteristics. To accomplish an
edit to a nested field, PATCH expects the data to be in the following form:

{
"add": {

}
}

{
"replace": {

}
}

{
"delete": {

}
}

For scalar fields, add or replace are equivalent as they replace the value of the scalar field with
the value provided. In case of a list, add indicates that the specified set of entries needs to be
added to the list and replace indicates that the list itself is replaced with what is specified in the
request. Action delete is used to remove specified entries from the list.

Examples
The following examples use PATCH to update server information in a pool. The pool is identified
by its UUID.

n Add Servers to a Pool- This request to the Avi Load Balancer API adds two new servers (1.1.1.1
and 1.1.1.2) to an existing pool.

API: /api/pool/pool_uuid
Data:
{
"add": {
"servers": [
{
"ip": {
"addr": "1.1.1.1",
"type": "V4"
}
},
{

VMware by Broadcom 423

VMware Avi Load Balancer Administration Guide

"ip": {
"addr": "1.1.1.2",
"type": "V4"
}
}
]
}
}

n Update Server Information in a Pool - This request to the Avi Load Balancer API updates one
of the servers in an existing pool.

API: /api/pool/pool_uuid
Data:
{
"add": {
"servers": [
{
"ip": {
"addr": "1.1.1.1",
"type": "V4"
},
"ratio": 10
}
]
}
}

Note If a field is an array of structures, each structure is typically identified by a key

(combination of the fields within the structure). This is used to identify the element in the list
to update. In the case of pool servers, the server key is identified by ip, port.

n Replace Servers in a Pool with a new set of Servers - This request replaces the entire server
list with a new server list. The other fields of the pool remain intact.

API: /api/pool/pool_uuid
Data:
{
"replace": {
"servers": [
{
"ip": {
"addr": "3.3.3.3",
"type": "V4"
},
},
{
"ip": {
"addr": "3.3.3.4",
"type": "V4"
},
}
]

VMware by Broadcom 424

VMware Avi Load Balancer Administration Guide

}
}

n Delete a Server from a Pool- This request deletes one of the servers from an existing pool.

API: /api/pool/pool_uuid
Data:
{
"delete": {
"servers": [
{
"hostname": "www.example.com",
"ip": {
"addr": "3.3.3.3",
"type": "V4"
},
"resolve_server_by_dns": true
}
]
}
}

n Updating Scalar Fields - The examples in this section set some scalar fields. The following
request enables HTTP request queuing and sets the default server port to 8080.

API: /api/pool/pool_uuid
Data:
{
"delete": {
"default_server_port": 8080
}
}

The following request resets the default server port to the system default, by deleting its explicit
setting from the pool.

API: /api/pool/pool_uuid
Data:
{
"replace": {
"request_queue_enabled": True,
"default_server_port": 8080
}
}

Supported Objects
PATCH is officially supported for the following objects and fields:

VMware by Broadcom 425

VMware Avi Load Balancer Administration Guide

Object Fields

Cloud linuxserver.hosts

GslbService enabled

addrs

IpAddrGroup prefixes

ranges

MicroServiceGroup service_refs

health_monitor_refs
Pool
servers

kv
StringGroup
type

dns_virtualservice_refs

SystemConfiguration global_tenant_config

snmp_configuration.community

VirtualService http_policies

Asynchronous Patch
Adding pools and deleting pools at a high rate requires applying patches at a high rate. The size
of the deployment often adds more load on the Controller and increases the latency.

The Asynchronous Patch feature allows patch requests to be queued, consolidated in fixed
intervals, and updated together with minimal latency.

Note This feature is recommended only in case of large deployment size.

When async_patch_merge_period is 0 in Controller properties all PATCH calls on pool with ?

async_mode=true are converted into synchronous call and 200 response is returned.

Using the Asynchronous Patch Update

By default, this feature is disabled. To enable asynchronous patch:

1 Change the async_patch_merge_period in the Controller properties from 0 (default value) to

the required interval (30-120 seconds).

2 Pass the async_mode as a query parameter in the patch request. For example, PATCH /api/
pool/<pool-uuid>?async_mode. The response displays a unique patch_id, which can be
used to view the status of the request. The patch consolidation is initiated, thereby merging
the pending requests in the queue.

VMware by Broadcom 426

VMware Avi Load Balancer Administration Guide

3 Review the status using a GET request on /api/pool-async/status/<patch_id>. The

status is displayed as follows:

{
"uuid": <uuid of the pool>,

"status": "QUEUED | IN_PROGRESS | SUCCESS | FAIL",

"created": <timestamp of request creation>,

"error_message": <error message in case of failure>,

"status_code": <status code for server PATCH>

Caveats
n Currently, this feature is available only for the servers field (with IP4) in the pool object.

n This feature is limited to only adding or deleting servers in a pool

n All patch requests to the Controller within an async_patch_merge_period must have the
same API version (HTTP_X_AVI_VERSION)

n Patch updates are not installed until the next update cycle, as defined in the time interval. So
the patch updates are not real time.

Rollback
Rollbacks are non-disruptive. When a rollback operation is performed, the Avi Load Balancer
Controller or SEs will transition to the previous major version of the software. Selective rollback is
possible for the Controller and SE groups.

The following options are available:

n Rollback for System

n Rollback for Avi Load Balancer Controller only

n Rollback for some or all the SE groups

Note Rollback of the SE Group will be to the previous version.

Rollback for System

The rollback of the system will result in the rollback of the SE(s) followed by the rollback of the
Avi Load Balancer Controller.

Use the following CLI and REST API for performing rollback for a patch version for the Avi Load
Balancer system (Controller and SE groups).

VMware by Broadcom 427

VMware Avi Load Balancer Administration Guide

Using the CLI

[admin:controller]: > rollback system

Using the REST API

POST api/rollback JSON data:{‘system’:true}

POST api/rollback JSON data:{‘system’:true,‘rollback_type’:2}

Rollback for Avi Load Balancer Controller

This interface is used to rollback the Avi Load Balancer Controller.

Using the CLI

[admin:controller]: > rollback controller

Using the REST API

POST api/rollback

Rollback for SE Groups

This interface is used to rollback for SE groups.

Using the CLI

[admin:controller]: > rollback segroup <se-group-name>

[admin:controller]: > rollback segroup seg-a

Using the REST API

POST api/rollback JSON data:{‘se_group_uuids’: [‘seg-a-uuid’]}

Rollback - Patch
Rollback of a patch release transitions the software to a version without the specific patch. It will
not roll back to the previous major version.

Selective ability to rollback the patch on the Avi Load Balancer Controller and SE groups is
available.

This interface is used to roll back the patch and not the major version.

The followings are the available options:

n System: rollback patch for Avi Load Balancer Controller and all SE groups

n Controller: rollback patch the Avi Load Balancer Controller only.

n SE-group: rollback patch for all or some of the SE groups.

VMware by Broadcom 428

VMware Avi Load Balancer Administration Guide

Rollback Patch for System

Use the following CLI and REST API for performing rollback for Avi Load Balancer System (Avi
Load Balancer Controller and SE groups).

Using the CLI

[admin:controller]: > rollbackpatch system

Using the REST API

POST api/rollback JSON data:{‘rollback_type’:2}

Rollback Patch for Avi Load Balancer Controller

Use the following CLI and REST API for performing rollback for a patch version for an Avi Load
Balancer Controller.

Using the CLI

[admin:controller]: > rollbackpatch controller

Rollback Patch for SE Groups

Use the following CLI and REST API for performing rollback for a patch version for an SE group.

Using the CLI

[admin:controller]: > rollbackpatch segroup <se-group-name>

[admin:controller]: > rollbackpatch segroup seg-a

Using the REST API

POST api/rollback JSON data:{‘rollback_type’:2,‘se_group_uuids’: [‘seg-a-uuid’]

Note See Additional Options for Flexible Upgrades for the following additional options:
n Rollback - Error Recovery

n Abort Cleanup

n SE Group Resume Option

Show Commands
The following show commands provide software version visibility in the system:

n show version controller

n show version serviceengine

n show version serviceenginegroup

VMware by Broadcom 429

VMware Avi Load Balancer Administration Guide

The following commands provide upgrade visibility in the system:

n show upgrade status: Various filters will be implemented as per UI workflow.

n show upgrade history: This command is deprecated.

Note
n If the Controller is at the highest version, whereas the SE groups are at lower versions,
commands might not work due to the higher version of the Controller.

n Due to the API version semantics, fields might not be available as they are deprecated in the
annotation.

n Due to API endpoint deprecation, some internal commands might not work.

Alerts and Events

The following events are available to provide visibility:

n Image upload/delete events

n Upgrade-specific events

n Patch-specific events

n Rollback-specific events

n Rollback patch-specific events.

n Failures will translate into alerts.

Additional APIs
The following GET API calls are applicable:

n The following REST API provides information about all the images present in the system.

Get API: api/image/

n The following API provides information about a specific image whose UUID is passed as a
slug.

Get API: api/image/image-uuid

n Use the following API to delete the image provided if not in use.

Delete API: api/image/image-uuid

n Inventory API - api/image-inventory: This API provides the image inventory on the system.
It filters based on various options such as retrieve all packages for a version and so on.

Reset to Factory Defaults

In some scenarios, it might be useful to reset the Avi Load Balancer to its factory default
settings. With a few exceptions, factory reset is equivalent to completely reinstalling a fresh

VMware by Broadcom 430

VMware Avi Load Balancer Administration Guide

deployment. As part of the factory reset, the Controllers and SEs reboot and delete their load
balancing and proxy configurations, while still maintaining their basic networking configurations.

Following a factory reset, on logging in to the Controller through the web interface, the initial
setup wizard starts.

Configure the settings as required, including setting a new password.

Resetting to Factory Defaults

To reset to factory defaults, run the following command from the Avi Load Balancer shell.

: > | reboot clean

This is different from the reboot command in the Linux CLI.

On reboot clean, a new self-signed certificate is generated for the Controller UI. If you are logged
into the UI, the browser might not redirect the page to login, as it sees a certificate change.
Refresh the page in the browser to get back to the login page.

Note

n In the write access mode deployments, if an SE is not given a new configuration within a
configured interval, it is deleted. In the web interface, this interval is configurable on the
Advanced tab for the SE group, in the Delete Unused Service Engines After field.

n When reboot clean is issued, the SE virtual machines get picked up by the new Controller and
are put in the default cloud. You cannot use or remove them other than from the underlying
environment, e.g., VMware vCenter.

n In case of a cluster, reboot clean is applicable only from a leader node in the cluster.

n On performing a reboot clean on a leader node in the cluster, the cluster remains intact,
including the UUID. You have to reset the password from the Avi Load Balancer UI.

n In case of a no-access deployment, the SEs are moved from the no-access cloud to the
default cloud, as the cloud config is deleted.

Settings Retained After Factory Reset

The following are the settings retained after factory reset.

n SE and Controller virtual machines.

n SE and Controller management networking properties, such as IP addresses and default

gateways.

Settings Removed After Factory Reset

The following are the settings removed after factory reset.

n Licenses.

n Virtual service configurations.

VMware by Broadcom 431

VMware Avi Load Balancer Administration Guide

n Pool configurations.

n SSL certificates, including certificates that might have been used for the Controller UI.

n Backups of the configuration.

n Logs and metrics.

n RBAC, users, passwords.

Locking a Linux System to a Specific OS Version

This section explains the ways to prevent Specific OS Version from being updated.

CentOS Linux
To prevent CentOS from being updated beyond a release level, it’s necessary to appropriate
set the $releasever parameter. Create a new file, /etc/yum/vars/releasever, containing the
value of the highest point release to which an update is acceptable.

This file creation can be done in two ways:

head -n1 /etc/centos-release | awk '{print $4}' > /etc/yum/vars/releasever

echo '7.2.1511' > /etc/yum/vars/releasever

The first code automatically restricts to the release that is currently running.

Red Hat Enterprise Linux

RHEL is a bit more complicated, as there are many possible options, which are detailed here:
https://access.redhat.com/solutions/238533. To summarize all but the EUS Subscription details:

Systems not Registered to Customer Portal or Satellite

Any of the following will work.

n Modify the /etc/yum.conf file under the [main] heading:

[main] distroverpkg=7.2

n Create the var file to override $releasever:

head -n1 /etc/redhat-release | awk '{print $7}' > /etc/yum/vars/

releasever

n Alternatively you can also use:

echo '7.2' > /etc/yum/vars/releasever

Systems Registered to Customer Portal or Satellite

n See a list of possible releases:

subscription-manager release --list

VMware by Broadcom 432

VMware Avi Load Balancer Administration Guide

n Set the release:

subscription-manager release --set=7.2

n Clean your yum cache:

yum clean all

n Verify the system is set to the correct release:

subscription-manager release --show

Increasing Docker Container Size

The devicemapper basesize changes will take effect only for containers created after the change.
Existing containers will keep using the basesize that was originally present. Follow according to
the type of host on which commands are being run.

For Controllers Without Any Co-Located SE

Follow the step-by-step procedure and run the commands from follower nodes to the leader
node. Wait for the cluster to become active after each node is configured:

1 $ systemctl stop avicontroller.

2 Check if docker thinpool has more space to increase from 10GB to 30GB (more if
devicemapper volume has the capacity).

3 Edit /etc/docker/daemon.json to have basesize set to at least 30G:

"storage-driver": "devicemapper",

"storage-opts": [

"dm.thinpooldev=/dev/mapper/docker-thinpool",

"dm.use_deferred_removal=true",

"dm.use_deferred_deletion=true",

"dm.basesize=30G" # <-- Add this

4 Run the following commands:

$ sudo systemctl daemon-reload and $ reboot #, for both kernel and devicemapper
changes to take effect.

5 Verification:

VMware by Broadcom 433

VMware Avi Load Balancer Administration Guide

$ docker info # pool base device size should show as 30GB (won’t reflect inside
container yet, new container needs to be created for that).

6 Verify cluster is in active state (from controller CLI: show cluster nodes).

For Controllers With Co-Located SE

Follow the step-by-step procedure and run the commands from follower nodes to the leader
node:

1 $ systemctl stop avicontroller.

2 Check if docker thinpool has more space to increase from 10GB to 30GB (more if
devicemapper volume has the capacity).

[root@vzn-ctrl-1 ~]# lvs -a # LSize is 37G

LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync

thinpool docker twi-aot--- <37.11g 20.08 4.30

3 Edit /etc/docker/daemon.json to have basesize set to at least 30G:

"storage-driver": "devicemapper",

"storage-opts": [

"dm.thinpooldev=/dev/mapper/docker-thinpool",

"dm.use_deferred_removal=true",

"dm.use_deferred_deletion=true",

"dm.basesize=30G" # <-- Add this

4 Run the following commands for both kernel and devicemapper changes to take effect:

$ sudo systemctl daemon-reload

$ reboot #

5 Verification:

$ docker info # pool base device size should show as 30GB (wont reflect inside container
yet, new container needs to be created for that).

6 Verify cluster is in the active state (from controller CLI: show cluster nodes), SE is OPER_UP
(show servicengine </code>), VLANs if any are still present on SE, basic traffic validation.

VMware by Broadcom 434

VMware Avi Load Balancer Administration Guide

For SE Hosts
Follow the step-by-step procedure and run the commands from follower nodes to the leader
node. Wait for the SE to become OPER_UP after each host is configured:

1 $ systemctl stop avise.

2 Verify SE is OPER_UP (from controller CLI: show servicengine </code>), VLANs if any are still
present on SE, basic traffic validation.

Note
n The above changes will take effect for the Docker Container (controller/SE) on the next
upgrade.

n You must rebuild the controller container if the upgrade is not being performed. As a result,
the cluster will take effect for the above changes.

For more information on deleting and re-adding the cluster nodes, see Changing Avi Load
Balancer Controller Cluster Configuration.

VMware by Broadcom 435

Avi Conversion Tool Appliance
9
When deploying Avi Load Balancer into existing environments, it is often required to migrate
application workloads from legacy load balancers to Avi Load Balancer. Avi Load Balancer
recognizes the stringent requirements of customers and the need to maintain up-time during
a live migration. This section provides insight into the process of migrating from F5’s BIG-IP LTM
to Avi Load Balancer.

Avi load balancer creates an extremely intuitive load balancer fabric which is highly automated to
reduce operational complexity and time to deploy, learn, and manage

Avi Load Balancer provides a wide ranging breadth of functionality common in competitive
application delivery controllers, or load balancers. But it also extends the load balancer’s
capabilities and value by incorporating extensive analytics data, a centralized control plane and
modern distributed data plane architecture.

Concepts
While F5 and Avi Load Balancer provide many similar high-level features, there are important
differences architecturally, in the operations of various features, and even in the names of
features and concepts.

Concept F5 Term Avi Load Balancer Term

application proxy virtual server Virtual Service

For more information, see Virtual
Service topic in the VMware Avi Load
BalancerConfiguration Guide

group of servers pool Pool

For more information, see Server
Pools topic in the VMware Avi Load
BalancerConfiguration Guide

Data plane scripting iRules™ DataScript

For more information, see
DataScripts Guide Overview topic
in the VMware Avi Load
BalancerDataScript Guide

API iControl™ REST API

VMware by Broadcom 436

VMware Avi Load Balancer Administration Guide

Concept F5 Term Avi Load Balancer Term

Load balancer BIG-IP™ LTM™ + GTM™ Service Engine

For more information, see Service
Engine topic in the VMware Avi Load
BalancerInstallation Guide

Connection aggregation OneConnect™ Multiplexing

For more information, see Connection
Multiplexing topic in the VMware Avi
Load BalancerConfiguration Guide

Central config manager Enterprise Manager™ / BIG-IQ™ Controller

For more information, see Control
Plane topic in the VMware Avi Load
BalancerInstallation Guide

Orchestrator none Controller

For more information, see Control
Plane topic in the VMware Avi Load
BalancerInstallation Guide

Control Plane
Architecturally, Avi Load Balancer is managed by a Controller or a redundant cluster of
Controllers. Rather than logging into and managing each pair of load balancer appliances, the
Avi Load Balancer fabric is managed by the Controller cluster. A single Controller cluster may
manage hundreds of Service Engines, even if they are deployed in different clouds such as
VMware or OpenStack. You may also choose to deploy more than one Controller cluster, though
this is usually done for geographically separate data centers. One cluster can manage both
test and production environments separated by different tenants, or each could have their own
cluster created.

Data Plane
Avi Load Balancer load balancers, called Service Engines (SEs), may be deployed similar to BIG-IP
in active/standby pairs (using Avi Load Balancer's Legacy HA mode), or they may preferably
be deployed in elastic HA mode, with fully active groups. There are a number of configuration
options to carve out separate groups through tenants, VRFs, clouds, and SE groups. Each
application may have its own load balancer, or all applications may share a group of Service
Engines. When migrating from BIG-IP, the appliance-pair versus fabric choice is one of the first
architectural questions that should be answered to determine how to best consolidate and
minimize unused load balancer capacity.

VMware by Broadcom 437

VMware Avi Load Balancer Administration Guide

Migration
Migration from existing, live environments is a delicate process. Avi Load Balancer provides
migration services, with a combination of migration tools and engineers with decades of
experience in load balancing.

Automated
BIG-IP LTM configurations can be automatically imported for Avi Load Balancer using Avi
Conversion Tool. The Avi Conversion Tool imports the configured objects and rules, along with
keys and certificates, and works towards simplifying the transition process to Avi Load Balancer.
This eliminates the potential of errors when converting BIG-IP configuration files that are often
tens of thousands of lines long. The Avi Conversion Tool provides a complete output, showing
every configuration setting that has been converted.

Conversion of objects not automatically covered by the

Conversion Tool
Some functionality from BIG-IP LTM may not yet be available to go through the automatic
conversion process. For instance, even while the coverage is constantly being improved, some
F5’s iRules may not be automatically converted yet.. Avi Load Balancer’s experience is that
about 75% of all iRules can be converted to native point-and-click features. iRules that cannot be
performed as native features will be rewritten in Avi Load Balancer’s DataScript format, which is
similar in logic and function, but based on the more modern Lua language.

Only LTM configuration is migrated; other modules must still be done manually. If a feature or
functionality cannot be converted directly, a VMware engineer may work with the customer to
provide a workaround or determine the best course of action.

Cutover
Once the configuration is migrated to the Avi Load Balancer deployment, it is time to begin
testing. All virtual services are migrated and left in a disabled state so they do not cause an ARP
conflict. There are various methods for testing the configuration prior to cutting over. Most often,
the virtual service is given a new IP address and marked as enabled.

The virtual service is deployed onto a Service Engine and is available for testing. This can
involve accessing the virtual service directly, by using an alternate name in DNS, or by altering
a client host file. In this test scenario, SNAT is typically recommended to ensure no changes are
required on the servers, yet traffic will synchronously return through the Avi Load Balancers load
balancers rather than the server’s default gateway.

VMware by Broadcom 438

VMware Avi Load Balancer Administration Guide

Once the virtual service is ready to go live, the virtual service is disabled on the BIG-IP and
enabled on Avi Load Balancer. If additional IP addresses are available, Avi Load Balancer can
be configured to use a unique IP for the virtual service. The cutover is performed by changing
the IP address advertised through DNS. Traffic will gracefully bleed off the BIG-IP as new traffic
is processed through Avi Load Balancer with no disruption to live traffic. This process can be
repeated as necessary for all applications.

n Virtual Service Status

n The migration tool imports virtual services with traffic disabled state. Based on the
requirement, virtual service state can be modified by patching the JSON configuration.

n Policies

n Avi Load Balancer supports reusing polices with multiple virtual services and this is
enabled using the reuse_http_policy option.

n If the reuse_http_policy option is not used, the migration tool duplicates the polices and
applies individual policies to the appropriate virtual services.

n Application Profile

n Avi Load Balancer default behavior is to share application profile with multiple virtual
services. This is the preferred way and any change to an application profile affects all
of the attached virtual services. Use the --distinct_app_profile option during migration to
create a unique application profile for each virtual services.

Read the following topics next:

n Deploying Avi Conversion Tool Appliance

n Accessing Avi Conversion Tool

n Using Avi Conversion Tool

n Using Avi Conversion Tool in Offline Mode

n iRules Automatic Translation

Deploying Avi Conversion Tool Appliance

Before starting the migration process, make sure the following requirements are achieved:

n Legacy Load balancer Cleanup

n Complete and latest configuration files from F5 load balancer should be available to
complete the migration activities.

n F5 configuration file should be free from unused/decommissioned configuration.

n Import options – details of cloud, tenant, VRF, and SE group required during migration.

See Deploy an OVF or OVA Template for detailed instructions for deploying the OVA into a
vCenter server appliance.

VMware by Broadcom 439

VMware Avi Load Balancer Administration Guide

Accessing Avi Conversion Tool

This section discusses steps to access the Avi Conversion Tool for converting F5 configuration
objects to Avi configuration objects.

The Avi Conversion Tool is mainly targeted to be used along with by VMware Professional
Services. Please contact your VMware Professional Services engineer, or your Account Team, to
get more information about how to obtain and run the tool.

After successfully deploying the Avi Conversion Tool Appliance, the UI can be accessed by
browsing the host server (https://<Avi-Conversion-Tool-IP>/

Available Modes to Run Avi Conversion Tool

The following two modes are available to access Avi Conversion Tool:

n Online Mode

n In ths mode, a connection to an F5 appliance is mode using admin credentials to gather

configuration, keys, and certificates. .

n Offline Mode

n F5 configuration is converted into the corresponding Avi configuration using the

bigip.conf files.
In online mode, the additional option available as compared to the offline mode is the ability to
generate a report using the discovery option on the Avi Convertion Tool. The other three options
- iRule, Configuration, and Migrate are available in both the modes.

Avi Conversion Tool CLI Access

Access to the conversion tool is via SSH connection.

The following are the credentials to access the conversion tools.

n User: ubuntu

n Password: assigned when the ova was deployed.

The Appliance will ask to define a new password upon accessing via SSH for the first time.

In order to access the Avi Conversion tool CLI, after logging into the appliance, run the following
set of commands"

sudo -i
docker ps

VMware by Broadcom 440

VMware Avi Load Balancer Administration Guide

From the docker ps output, copy the container ID from the row that includes avimigrationtools
which is the first row on the example snippet (f1856fc368b2). After obtaining the container ID, run
the following command:

root@migrationtools:~# docker exec -it <container-id> bash

root@migrationTools:/server#

From the migrationTools bash, all the tools available on the appliance can be accessed.

Configuring a Static IP Address for the Avi Conversion Tool's Management

Interface
DHCP is the default method for IP addres allocation. To configure static IP address on the Avi
Conversion Tool, follow the steps mentioned in the this section.

First proceed to backup the original cloud init files.

It is highly recommended to temporarily add an IP address to the host and access it via SSH
instead of using vSphere’s web console. This procedure adds an IP address to an interface, but
the IP address will not be persistent, meaning it will disappear after a reboot. Complete the steps
in this chapter to add a persistent IP address to the interface.

To add an IP address ,execute the following command:

sudo ip addr add <IP-ADDRESS>/<NETMASK-Bits> ens192

sudo ip route add default via <GATEWAY-IP>

For more details on Temporary IP address assignment, see https://ubuntu.com/server/docs/

configuring-networks.

After adding an IP to the host, use any SSH client to access the Avi Conversion Tools VM.

ubuntu@migrationtools:~$ sudo -i
root@migrationtools:~# cp /etc/netplan/50-cloud-init.yaml /etc/netplan/50-cloud-init.yaml.old
root@migrationtools:~# echo network: {config: disabled} > /etc/cloud/cloud.cfg.d/99-disable-
network-config.cfg.test

Disable cloud-init's network configuration capabilities:

root@migrationtools:~# echo network: {config: disabled} > /etc/cloud/cloud.cfg.d/99-disable-

network-config.cfg

Edit /etc/netplan/50-cloud-init.yaml using your preferred editor. In the below example, the vim
editor is used.

Change dhcp4 from 'true' to 'false':

root@migrationtools:~# vim /etc/netplan/50-cloud-init.yaml

before: dhcp4: true
after: dhcp4: false

VMware by Broadcom 441

VMware Avi Load Balancer Administration Guide

Add IPv4 details (replace what is inside of the brackets).

addresses:
- [ip]/[netmask-bits]
routes:
- to: default
via:[next-hop]
nameservers:
addresses:
- [dns]

Below is an example file:

network:
ethernets:
ens192:
dhcp4: false
match:
macaddress: 00:50:56:8c:05:60
set-name: ens192
addresses:
- 192.168.110.51/24
routes:
- to: default
via: 192.168.110.1
nameservers:
addresses:
- 8.8.8.8
version: 2

Apply the changes

root@migrationtools:~# netplan apply

Procedure to add static IP address configuration from the UI

Use Encoded user-data field to add encoded user-data during deployment with static IP
configuration.

Steps to generate Encoded user-data field value are as follows:

Create a cloud-init userdata file for network configuration:

$ cat <<EOF >> cloud-config-static-ip.yaml

#cloud-config
write_files:
- path: /etc/cloud/cloud.cfg.d/99-custom-networking.cfg
permissions: '0644'
content: |
network: {config: disabled}
- path: /etc/netplan/00-installer-config.yaml
permissions: '0644'
content: |
network:

VMware by Broadcom 442

VMware Avi Load Balancer Administration Guide

version: 2
renderer: networkd
ethernets:
####BEGIN: Configure the network section####
ens192: # replace this interface name to match your system
dhcp4: false
addresses:
- <static-ip>/<mask> # replace with your desired static IP address and subnet
mask
nameservers:
addresses: [10.214.129.198, 10.214.129.199] # replace with your DNS server
addresses
routes:
- to: default
via: <gateway-ip> # replace with your gateway IP address
####END: Configure the network section####
runcmd:
- rm /etc/netplan/50-cloud-init.yaml
- netplan generate
- netplan apply
EOF

Encode the contents to base64 format:

$ openssl base64 -in cloud-config-static-ip.yaml -out cloud-config-static-ip-base64-

encoded.yaml

Pass the encoded contents to 'Encoded user-data' during deployment of avi-migration-tools from
vCenter UI:

$ cat cloud-config-static-ip-base64-encoded.yaml

Add above encoded content to Encoded user-data and perform remaining steps for the same.

Extracting Logs from the Avi Conversion Tool Appliance

Follow the steps mentioned in this section to extract logs from the Avi Conversion Tool appliance
for troubleshooting purpose.

Server logs can be extracted by accessing https://<Avi-Conversion-Tool-IP-IP>:8080/logs.

In order to extract the complete converter logs via SCP, the logs need to be packaged and
copied from a container running the Avi Conversion Tool to the base OS.

For complete logs extraction, CLI access to the Avi Conversion tool is required. See Avi
Conversion Tool CLI Access section, to learn how to get access to the Avi Conversion Tools's
CLI.

From migrationTools (after issuing docker exec):

root@migrationTools:/server# tar czf migration_logs.zip migration/

VMware by Broadcom 443

VMware Avi Load Balancer Administration Guide

Copying the file to the host:

Note To get the container ID, follow the steps mentioned in the Avi Conversion Tool CLI Access
section.

root@migrationtools:~# cd /home/ubuntu/
root@migrationtools:~# docker cp <container-id>:/server/migration_logs.zip migration_logs.zip

After issuing the last command, the zipfile will be available in the home directory of the ubuntu
user The file can be extracted using the SCP tool of the user’s preference (example winscp,
cyberduck).

Using Avi Conversion Tool

This section discusses the steps involved to successfully discover F5 configuration objects and
generating corresponding Avi configuration objects using Avi Conversion Tools in online and
offline mode.

Note Before proceeding with the steps to access conversion tool using the online mode, make
sure you have the following information available:
n F5 (Source) device IP address

n F5 (Source) device username and password with advanced shell

n Staging Avi Controller.

n Destination Avi Controller.

Starting with release 2.3.1, the admin username is available as the default username to log in to
Avi Conversion Tool UI. You will be asked to enter a password of your choice. Enter the email
ID address (optional) and click on the CREATE USER tab to save the provided password for the
default admin username.

VMware by Broadcom 444

VMware Avi Load Balancer Administration Guide

The option to set the password is available only at the first login. You can use the same password
for subsequent logings. You can change the password for the default admin username use
change password option available under the Setting tab of the Avi Conversion tool UI.

Discovering F5 environment
Once you login to the Avi Conversion Tool UI, you have option to either start the migrate process
or use the discovery option.

To start the discovery workflow (optional), select the Discovery option available on the left
navigation panel.

VMware by Broadcom 445

VMware Avi Load Balancer Administration Guide

Select the START option availalble on the discovery page to initiate discovery of F5 environment.

VMware by Broadcom 446

VMware Avi Load Balancer Administration Guide

Once you enter the credentials and it is successfully authorized, the tool analyzes the
configuration on F5 load balancer and starts the discovery of the configuration objects.

As part of F5 discovery process , Avi Conversion Tool fetches virtual servers , pool objects ,
configured profiles from F5 instance and perform analysis on these fetched configuration objects.

Once the discovery phase is completed, the following report will be available on the tool
dashboard as shown below:

1. Virtual Services - Virtual Services are classified into the following categories:

n L4 : F5 virtual services of type fastL4 and TCP are converted into Layer 4 virtual services on
Avi Controller.

n L7 : F5 virtual services of type http, https, fasthttp, clientssl, oneconnect are coverted into
Layer 7 virtual Service on Avi Controller.

n DNS : Number of virtual services consisting of type DNS.

n UDP : Number of virtual services consisting of type UDP.

n SSL : Number of virtual services configured with SSL services.

n WAF : Number of virtual services configured with WAF policies.

2. Pools - total number and status of pool members

3. iRules - total number of iRules available

VMware by Broadcom 447

VMware Avi Load Balancer Administration Guide

4. Tenant - total number of tenants available

The report dashboard has two options -CONTINUE TO MIGRATION or DOWNLOAD REPORT.
You can choose the DOWNLOAD REPORT option to download report in JSON format or proceed
with the conversion workflow by selecting the CONTINUE TO MIGRATION option.

Note Discovery of F5 environment is not available for the 2.2.1 version of the Avi Conversion
Tools.

Generating Avi Load Balancer Configurations

Once the discovery phase is completed, the next phase in the conversion process is to generate
Avi load balancer configuration. Select the CONTINUE TO MIGRATE option available on the Avi
Conversion Tool page or select the Migrate > START option from the tool home page to log in to
online or offline mode.

VMware by Broadcom 448

VMware Avi Load Balancer Administration Guide

Note Prerequisites:
n The Staging Controller's image version should be same as of the Destination Controller's
image version.

n Before proceeding with the discovery steps, make sure the required cloud, VRF, and tenant
are configured on the destination Controller.

This phase consists of the following steps:

1 Let's begin - Let's begin page allow you to select the online or offline method to convert F5
configuration to the corresponding Avi Configuration.

Select the Online Mode when you want to provide F5 load balancer's credentials and wants
Avi Conversion tool to fetch all the necessary files. Select the Offline Mode when you want to
manually upload F5 configuration files, certificates and key fro your local machine.

VMware by Broadcom 449

VMware Avi Load Balancer Administration Guide

2 Collect Configuration: Provide F5 credentials if you have selected the Online Mode Enter the
F5 Controller IP address and admin credentials to access F5's certificates and key. This will
start mapping configuration objects on F5 with the configuration objects available on Avi
Load Balancer. Upload the F5 configuration file (bigip.conf) and relevant certificates and key
if using the Offline Mode. The remaining steps in the workflow are same for both the modes
(online and offline). ClickNEXT to enter the Staging and Destination Controller details.

VMware by Broadcom 450

VMware Avi Load Balancer Administration Guide

3 Controller information- Provide IP address, username and password for the Staging Avi
Controller. The Staging Avi Controller is the Controller where all the Avi Load Balancer-related
configuration changes are performed.

VMware by Broadcom 451

VMware Avi Load Balancer Administration Guide

The final configuration file available on the Staging Controller is then uploaded to the
Destination Avi Controller which is discussed in the later section of this document. The
Staging Avi Controller's image version should be same as of the Destination Controller's
image version. The Destination Controller is the Controller where the ansible playbook
generated by the conversion tool will be deployed. Enter IP address, username, and
password for Destination Controller. This will help to fetch pre-populated fields and push
playbooks.

You can select the Use the same Controller as Staging option too if required.

Note While preforming discovery, only one tenant is accessible at a time. For multiple
tenants, you need to repeat the process for each tenant.

4 Map Destination - Enter the preferred tenant, Cloud, VRF, and Service Engine Group on
Avi load balancer. This will enable migration tool to map the imported configuration to the
mentioned destination. Click NEXT to move to the migration section.

VMware by Broadcom 452

VMware Avi Load Balancer Administration Guide

5 Migration Section - Starting with the 2.3.1 version of the Avi Conversion Tool, the following
two options are available for the migration:

a Full - Use this option to proceed migration with the complete set of virtual services
available on F5 load balancer.

VMware by Broadcom 453

VMware Avi Load Balancer Administration Guide

b Selected Partition - Use this option to select among the list of virtual services from the
selected partition configured on F5 load balancer. In the below example, the virtual
services are selected from the Common partition. Use the Selected Only option from the
drop-down available for the MIGRATE tab to proceed migration only with the selected
virtual services. Use the All option to select all the virtual services from a specific partition
on F5 load balancer. Click on MIGRATE to proceed further to the Finalizing Import step.

6 Finalizing Import - This step will start importing configuration objects from F5 load balancer
and Avi load balancer. The import process may take a while to complete based on the size of
F5 configuration file.

VMware by Broadcom 454

VMware Avi Load Balancer Administration Guide

The conversion tool generates the following report. It will show the list of virtual services which
require manual intervention. The status of these virtual services will exhibit as partial.

Once the fetching of configuration files is completed, the Conversion Tool has the following tabs
to analyse and execute different tasks:

n Convert iRules

n Convert Configurations

n Deploy

Convert IRules

Click on the Convert IRules > IRULE CATEGORIZATION tab to check the status of iRules. The
iRULE CATEGORIZATION section shows the following details:

n Total iRules fetched

n Number of iRules reviewed

n Number of iRules that needs review or manual check

Select the START option to initiate review of iRules with the Needs Review status.

VMware by Broadcom 455

VMware Avi Load Balancer Administration Guide

The first iRule in the list with Needs Review status will open in a subsequent screen. Select the
required options for Type and Avi Objects.

The options available under Type are - Native and Unsupported, and the Avi Objects options
available for native profile are - Application Profile and Persistence Profile.

VMware by Broadcom 456

VMware Avi Load Balancer Administration Guide

Select the SAVE option to submit the changes for the selected iRules. Once you submit the
SAVE option, the next iRule in the review list opens in the subsequent screen. Perform the
desired changes (Type and Avi Object) for the iRule as mentioned previously. Once all the
actions are completed, the status of iRule on the migration tool page will reflect correctly.

Please note the Needs Review status against iRules in the list will change to either Reviewed or
Unsupported based on the action taken in the previous step.

You may choose the MARK ALL IRULES UNDER REVIEW AS UNSUPPORTED option if it suits
the deployment/migration requirement.

To start reviewing the configuration files fetched, select the CONTINUE TO MIGRATION and click
the START option available on the UI.

It will show the list of virtual services that require manual intervention with the status Needs
Review. The virtual services with status Auto-Converted do not need any manual intervention.
Select the STARToption or Actions option to review the translated configuration file. Once
you click on START or Actions , the configuration editor provides F5 configuration and the
corresponding translated Avi configuration for the particular virtual service for easier comparison.

VMware by Broadcom 457

VMware Avi Load Balancer Administration Guide

In the screenshot shown above, persistence profile is missing for the specific policy.

Select one of the existing profiles from the drop-down available next to the Select Profile option
or use the Create Persistence Profile option to create a new persistence profile. Once a new
persistence profile is create, use the refrest option (referesh icon) to make it available under the
Select Profile list.

In the below example, System-Persistence-App-Cookie is selected as the persistence profile type

for the selected policy.

VMware by Broadcom 458

VMware Avi Load Balancer Administration Guide

Once the required persistence profile is mapped, click NEXT to perform additional changes as
required for the assoicated virtual service.

Select the NEXT option to move to the Review stage. The Review step allows you to check the
previous changes performed on the Avi configuration objects.

To submit the configuration changes, select the READY option.

VMware by Broadcom 459

VMware Avi Load Balancer Administration Guide

Once all the virtual service configuration changes have been taken care, select the CONTINUE
TO MIGRATE option to navigate to the Migrate tab.

Click the GENERATE PLAYBOOK option, provide a name of your ansible playbook, and select
the GENERATE option.

VMware by Broadcom 460

VMware Avi Load Balancer Administration Guide

Two files are generated – a configuration file to be added to Avi Load Balancer and a rollback file
to remove the required configuration.

The following options are available for the generated Ansible playbook:

n Download - Select the download option to download the configuration files to your local
machine. These configuration (.yml) files can be uploaded to the Avi Controller using Ansible.

n Push to Controller - Use this option to push Ansible playbook to the destination Avi
Controller. Once the .yml playbook file is successfully pushed, a transcript is available to
check which shows the configuration

With the latest version of the Avi Conversion tool, you can select one or more virtual services and
generate an anisble playbook for those virtual services only. Select the desired virtual services
and click the GENERATE PLAYBOOK option and choose the Selected Only option to initiate an
ansible playbook download for the selected virtual services only.

VMware by Broadcom 461

VMware Avi Load Balancer Administration Guide

Provide a name for the playbook as required and click on the GENERATE option.

In case, you encounter any error while pushing .yaml files to Avi Controller, use the View
Transcript and Edit Playbook options to modify the configuration file. The status of the .yaml
file in this case will be Error.

VMware by Broadcom 462

VMware Avi Load Balancer Administration Guide

Selecting the Edit Playbook option opens up the configuration file in a YAML editor.

Once configuration changes are completed, download a new playbook and use thePush to
Controller option again to upload the latest configuration file to the destination Avi Controller.

VMware by Broadcom 463

VMware Avi Load Balancer Administration Guide

Using Avi Conversion Tool in Offline Mode

This section discusses the steps involved to successfully discover F5 configuration objects and
generate corresponding Avi configuration objects using Avi Conversion Tools in offline mode.

In the offline mode of accessing Avi Conversion Tools, F5 configuration objects are discovered
using the bigip.conf files (F5 backup/configuration files).

Acces the Avi Conversion Tools appliance using the UI and select the IMPORT option to start the
discovering process for F5 configuration objects.

Under the upload page, select the browse file option to upload the F5 configuration file
(bigip.conf).

VMware by Broadcom 464

VMware Avi Load Balancer Administration Guide

Select the F5 configuration file from your local machine and click NEXT to provide staging Avi
Controller details.

Provide the following details of the staging Avi Controller.

n IP Address

n Username and password

VMware by Broadcom 465

VMware Avi Load Balancer Administration Guide

n Version - The staging Controller's image version should match with the destination
Controller's image version.

Click NEXT to enter the destination Avi Controller's information.

n IP Address

n Username and password

n Version

VMware by Broadcom 466

VMware Avi Load Balancer Administration Guide

Under the Map Destination option, select the required tenant, cloud, VRF, and Service Engine
group for the destination Controller. Use the drop-down available next to each field to select
preconfigued configuration objects on the destination Avi Controller.

VMware by Broadcom 467

VMware Avi Load Balancer Administration Guide

Click NEXT to navigate to the Finalizing Import option. The import may take a while to complete,
depending on the size of the F5 configuration file.

Once the import is successful, options to fetch/convert iRules, modify and deploy Avi Controller
configuration are available on the subsequent screens.

The steps to convert iRules, convert configuration, and deploy configuration are the same as
mentioned in the corresponding sections of the online Avi Conversion tool(mentioned in the
previous section).

VMware by Broadcom 468

VMware Avi Load Balancer Administration Guide

iRules Automatic Translation

This following sections cover the list of iRules automatically translated into Avi Controller policies.

Content Switching
iRules Corresponding Avi Policies

XFP Insert x_forwarded_proto_enabled: true

ltm rule /Common/insert_x-forwarded-

proto_rule {
when HTTP_REQUEST {
HTTP::header insert
X_FORWARDED_PROTO "https"
}
}

Secure Cookie secure_cookie_enabled: true

ltm rule /Common/

RDN_set_secure_bit_on_cookie {
when HTTP_RESPONSE {
set myValues [HTTP::cookie names]
foreach mycookies $myValues {
HTTP::cookie secure
$mycookies enable
}
}
}

Match URI
- rule_name: match_path_payment
type: HTTPPolicySet
ltm rule /Common/match_path_payment {
avi_config:
when HTTP_REQUEST {
name: match_path_payment
if { ([HTTP::uri] contains "/ws-test/") }{
http_request_policy:
pool payment_pool
rules:
}
- index: 1
}
enable: true
}
name: match_path_payment
match:
path:
match_case: INSENSITIVE
match_str:
- "/ws-test/"
match_criteria: CONTAINS
switching_action:
action: HTTP_SWITCHING_SELECT_POOL
status_code:
HTTP_LOCAL_RESPONSE_STATUS_CODE_200
pool_ref: "/api/pool?
name=payment_pool"
is_internal_policy: false

VMware by Broadcom 469

VMware Avi Load Balancer Administration Guide

iRules Corresponding Avi Policies

Match Path
- rule_name: strip_path
type: HTTPPolicySet
ltm rule /Common/strip_path {
avi_config:
when HTTP_REQUEST {
rules:
- enable: true
if { [HTTP::path] contains "-12345" }
index: 1
then {
match:
path:
# remove the HASHvalue from the path
match_case: INSENSITIVE
HTTP::path [string map {"/
match_criteria: CONTAINS
test/-12345/" "/test/"} [HTTP::path]]
match_str:
- "-12345"
# send changed HTTP:URI/path to
name: Rule 1
payment pool
rewrite_url_action:
pool payment_pool
path:
tokens:
}
- end_index: 0
}
start_index: 0
}
type: URI_TOKEN_TYPE_PATH
- end_index: 65535
start_index: 2
type: URI_TOKEN_TYPE_PATH
type: URI_PARAM_TYPE_TOKENIZED
query:
keep_query: true
switching_action:
action: HTTP_SWITCHING_SELECT_POOL
status_code:
HTTP_LOCAL_RESPONSE_STATUS_CODE_200
pool_ref: "/api/pool?
name=payment_pool"
is_internal_policy: false

VMware by Broadcom 470

VMware Avi Load Balancer Administration Guide

iRules Corresponding Avi Policies

Redirect
- rule_name: path_switch
type: HTTPPolicySet
ltm rule /Common/path_switch {
avi_config:
when HTTP_REQUEST {
name: path_switch
switch -glob [HTTP::path] {
http_request_policy:
"/test-credit*" {
rules:
pool Pool_credit-trade_front_1
- enable: true
}
index: 1
"/credit-kiosk*" {
match:
pool Pool_credit-kiosk_1
path:
}
match_case: INSENSITIVE
}
match_criteria: CONTAINS
}
match_str:
}
- "/test-credit"
name: path_switch_gbss_trade
switching_action:
action: HTTP_SWITCHING_SELECT_POOL
pool_ref: /api/pool?
name=Pool_test_credit_front_1"
status_code:
HTTP_LOCAL_RESPONSE_STATUS_CODE_200
- enable: true
index: 2
match:
path:
match_case: INSENSITIVE
match_criteria: CONTAINS
match_str:
- "/credit-kiosk"
name: path_switch_credit_kiosk
switching_action:
action: HTTP_SWITCHING_SELECT_POOL
pool_ref: "/api/pool?name=Pool_credit-
kiosk_1"
status_code:
HTTP_LOCAL_RESPONSE_STATUS_CODE_200
is_internal_policy: false

VMware by Broadcom 471

VMware Avi Load Balancer Administration Guide

iRules Corresponding Avi Policies

Match Host and Redirect

- rule_name: path_switch_multi
type: HTTPPolicySet
ltm rule /Common/path_switch_multi {
avi_config:
when HTTP_REQUEST {
name: path_switch_multi
switch -glob [string tolower
http_request_policy:
[HTTP::host]] {
rules:
lyncweb.corp.avinetworks.com*
- enable: true
{ pool
index: 1
avinetworks.lync_reverse_proxy_front_end_444
match:
3_pool }
host_hdr:
# { pool
match_case: INSENSITIVE
yp.lync_reverse_proxy_front_end_4443_pool }
match_criteria: HDR_BEGINS_WITH
meet.avinetworks.com* { pool
value:
avinetworks.lync_reverse_proxy_front_end_444
- lyncweb.corp.avinetworks.com
3_pool }
name: path_switch_multi_1
dialin.avinetworks.com* { pool
switching_action:
avinetworks.lync_reverse_proxy_front_end_444
action: HTTP_SWITCHING_SELECT_POOL
3_pool }
pool_ref: /api/pool?
lyncdiscover.avinetworks.com*
name=avinetworks.lync_reverse_proxy_front_en
{ pool
d_4443_pool
avinetworks.lync_reverse_proxy_front_end_444
status_code:
3_pool }
HTTP_LOCAL_RESPONSE_STATUS_CODE_200
}
- enable: true
}
index: 2
}
match:
host_hdr:
match_case: INSENSITIVE
match_criteria: HDR_BEGINS_WITH
value:
- meet.avinetworks.com
name: path_switch_multi_2
switching_action:
action: HTTP_SWITCHING_SELECT_POOL
pool_ref: /api/pool?
name=avinetworks.lync_reverse_proxy_front_en
d_4443_pool
status_code:
HTTP_LOCAL_RESPONSE_STATUS_CODE_200
- enable: true
index: 3
match:
host_hdr:
match_case: INSENSITIVE
match_criteria: HDR_BEGINS_WITH
value:
- dialin.avinetworks.com
name: path_switch_multi_3
switching_action:
action: HTTP_SWITCHING_SELECT_POOL
pool_ref: /api/pool?
name=avinetworks.lync_reverse_proxy_front_en
d_4443_pool
status_code:
HTTP_LOCAL_RESPONSE_STATUS_CODE_200
- enable: true
index: 4
match:
host_hdr:
match_case: INSENSITIVE
match_criteria: HDR_BEGINS_WITH
value:
- lyncdiscover.avinetworks.com

VMware by Broadcom 472

VMware Avi Load Balancer Administration Guide

iRules Corresponding Avi Policies

name: path_switch_multi_4
switching_action:
action: HTTP_SWITCHING_SELECT_POOL
pool_ref: /api/pool?
name=avinetworks.lync_reverse_proxy_front_en
d_4443_pool
status_code:
HTTP_LOCAL_RESPONSE_STATUS_CODE_200
is_internal_policy: false

Multiple Branch with If

- avi_config:
http_request_policy:
when HTTP_REQUEST { if{ [HTTP::host]
rules:
ends_with ".vmware.com"}
- enable: true
{ HTTP::redirect
index: 1
https://[HTTP::host][HTTP::uri] }
match:
else{ HTTP::redirect https://
host_hdr:
[HTTP::host].vmware.com[HTTP::uri] }}
match_case: INSENSITIVE
match_criteria: HDR_ENDS_WITH
value:
- .vmware.com
name: test-endswith
redirect_action:
keep_query: true
port: 443
protocol: HTTPS
status_code:
HTTP_REDIRECT_STATUS_CODE_302
- enable: true
index: 2
name: test-endswith-catchall
redirect_action:
host:
tokens:
- end_index: 65535
start_index: 0
type: URI_TOKEN_TYPE_HOST
- str_value: vmware.com
type: URI_TOKEN_TYPE_STRING
type: URI_PARAM_TYPE_TOKENIZED
keep_query: true
port: 443
protocol: HTTPS
status_code:
HTTP_REDIRECT_STATUS_CODE_302
name: test-endswith
rule_name: test-endswith
type: HTTPPolicySet

VMware by Broadcom 473

VMware Avi Load Balancer Administration Guide

HTTP to HTTPS Redirect

iRule Example Avi Controller Configuration HTTP Policy

With status code 302

"name": "System-Secure- - avi_config:
HTTP", http_request_policy:
ltm rule /Common/
"type": rules:
https_redirect {
"APPLICATION_PROFILE_TYPE_H - enable: true
when HTTP_REQUEST {
TTP", index: 1
HTTP::redirect "https://
"http_profile": { name:
[HTTP::host][HTTP::uri]"
"http_to_https": true, https_redirect
}
redirect_action:
}
keep_query: true
port: 443
protocol: HTTPS
status_code:
HTTP_REDIRECT_STATUS_CODE_3
02
name: https_redirect
rule_name: https_redirect
type: HTTPPolicySet

With status code 301

"name": "System-Secure- - avi_config:
HTTP","type": http_request_policy:
ltm rule /Common/
"APPLICATION_PROFILE_TYPE_H rules:
https_301_redirect.irule {
TTP","http_profile": - enable: true
when HTTP_REQUEST {
{ "http_to_https": true, index: 1
HTTP::respond
name:
301 Location "https://
https_301_redirect.irule
[HTTP::host][HTTP::uri]"
redirect_action:
}
keep_query: true
}
port: 443
protocol: HTTPS
status_code:
HTTP_REDIRECT_STATUS_CODE_3
01
name:
https_301_redirect.irule
rule_name:
https_301_redirect.irule
type: HTTPPolicySet

HTTP to HTTPS Redirect "name": "System-Secure- - avi_config:

ltm rule /Common/ HTTP","type": http_request_policy:
https_redirect_host_only.ir "APPLICATION_PROFILE_TYPE_H rules:
ule { TTP","http_profile": - enable: true
when HTTP_REQUEST { { "http_to_https": true, index: 1
set host name:
[HTTP::host] https_redirect_host_only.ir
HTTP::respond 302 ule
Location "https://$host/" redirect_action:
} keep_query: true
} port: 443
protocol: HTTPS
status_code:
HTTP_REDIRECT_STATUS_CODE_3
02
name:
https_redirect_host_only.ir
ule

VMware by Broadcom 474

VMware Avi Load Balancer Administration Guide

iRule Example Avi Controller Configuration HTTP Policy

rule_name:
https_redirect_host_only.ir
ule
type: HTTPPolicySet

Redirection
IRule Examples Avi HTTP Policies

Redirect Match Host

- avi_config:
http_request_policy:
when HTTP_REQUEST {
rules:
- enable: true
if { [HTTP::host] equals
index: 1
"test5.test7.com"} {
match:
host_hdr:
HTTP::redirect "https://
match_case: INSENSITIVE
test5.test7.com[HTTP::uri]"
match_criteria: HDR_EQUALS
value:
}
- test7sps3-staging.scene7.com
name: staging_https
}
redirect_action:
}
host:
tokens:
- str_value: test7sps3-
staging.test7.com
type: URI_TOKEN_TYPE_STRING
type: URI_PARAM_TYPE_TOKENIZED
keep_query: true
port: 443
protocol: HTTPS
status_code:
HTTP_REDIRECT_STATUS_CODE_302
name: staging_https
rule_name: staging_https
type: HTTPPolicySet

Redirect Match Host

- avi_config:
http_request_policy:
ltm rule /Common/foo.example_https {
rules:
when HTTP_REQUEST {
- enable: true
index: 1
if { [HTTP::host] ends_with
match:
"foo.example.com"} {
host_hdr:
match_case: INSENSITIVE
HTTP::redirect "https://[HTTP::host]
match_criteria: HDR_ENDS_WITH
[HTTP::uri]"
value:
- foo.example.com
}
name: f00.example.com
redirect_action:
}
keep_query: true
}
port: 443
protocol: HTTPS
status_code:
HTTP_REDIRECT_STATUS_CODE_302
name: foo.example.com
rule_name: foo.example.com
type: HTTPPolicySet

VMware by Broadcom 475

VMware Avi Load Balancer Administration Guide

IRule Examples Avi HTTP Policies

Redirect URI
- avi_config:
http_request_policy:
ltm rule /Common/L7TetsScriptsHttpsRedirect
rules:
{
- enable: true
when HTTP_REQUEST {
index: 1
if { [string tolower [HTTP::uri]]
match:
starts_with "/supportscripts" } {
path:
HTTP::redirect "https://[HTTP::host]
match_case: INSENSITIVE
[HTTP::uri]"
match_criteria: BEGINS_WITH
}
match_str:
}
- /Testscripts
}
name: L7TestScriptsHttpsRedirect
redirect_action:
keep_query: true
port: 443
protocol: HTTPS
status_code:
HTTP_REDIRECT_STATUS_CODE_302
name: L7TestScriptsHttpsRedirect
rule_name: L7TestScriptsHttpsRedirect
type: HTTPPolicySet

Redirect: Match Host and Path

ltm rule /v1-vpnlive30/crc_redirect_IR {
when HTTP_REQUEST {
ltm rule /v1-vpnlive30/crc_redirect_IR {
if { [HTTP::host] equals
when HTTP_REQUEST {
"example.test7.com" and [HTTP::uri]
if { [HTTP::host] equals
contains "/is-foo/" } {
"example.test7.com" and [HTTP::uri]
HTTP::redirect "https://
contains "/foo/" } {
marketing.foo.com/resources/help/en_US/s7/
HTTP::redirect "https://
foo_api/image_rendering_api_ref.html"
marketing.foo.com/resources/help/en_US/s7/
}
foo_api/image_rendering_api_ref.html"
}
}
}
}
}

- avi_config:
http_request_policy:
rules:
- enable: true
index: 1
match:
host_hdr:
match_case: INSENSITIVE
match_criteria: HDR_EQUALS
value:
- example.test7.com
path:
match_case: INSENSITIVE
match_criteria: CONTAINS
match_str:
- /is-foo
name: crc_redirect_IR
redirect_action:
host:
tokens:
- str_value: marketing.foo.com
type: URI_TOKEN_TYPE_STRING
type: URI_PARAM_TYPE_TOKENIZED
keep_query: true
path:
tokens:

VMware by Broadcom 476

VMware Avi Load Balancer Administration Guide

IRule Examples Avi HTTP Policies

-
str_value: resources/help/en_US/s7/foo_api/
image_rendering_api_ref.html
type: URI_TOKEN_TYPE_STRING
type: URI_PARAM_TYPE_TOKENIZED
port: 443
protocol: HTTPS
status_code:
HTTP_REDIRECT_STATUS_CODE_302
name: crc_redirect_IR
rule_name: crc_redirect_IR
type: HTTPPolicySet

VMware by Broadcom 477

Testing Avi Load Balancer with
Load Generators 10
Validating performance characteristics of Avi Vantage or any device under test (DUT) with load
generation tools is a common practice. There are many load generators, and operating them is
often more of an art than a science.

Avi Load Balancer has no specific preference for which load generators should be used. Our
intent and goal is for any published HTTP performance numbers to be fully repeatable via the
industry standard ApacheBench (AB), which is freely available for most Linux distributions.

Avi Load Balancer provides an Ubuntu virtual machine, downloadable from our Portal. This VM
includes a default web server and AB, which can be used to quickly spin up clients and servers to
test Avi Load Balancer load balancing and performance.

The most common issue when testing with a load generator is the client or server running out
of capacity. When this happens, responses become slow, packets are dropped, and the load
generator will report the device under test did not perform well. Load generators make no
distinction in results when the bottleneck is the client, the device under test, or the server. If the
performance doubles when adding a second load generator or a second server, then its a safe
assumption that the Service Engine was not the bottleneck.

This section is not a definitive guide to load generators or their optimization. VMware strongly
recommends working with one of our engineers to ensure best results.

Load Generators Versus Real Clients

Testing with a load generator is an important step in establishing trust with a load balancer. It is
helpful to understand the behavior when a Service Engine will reach maximum capacity and the
impact to connections or latency. However, load generators are not real clients, which behave
very differently. For instance, clients connecting over a network will have some latency, while
load generators often are configured for near zero latency via immediate adjacency to the device
under test.

Free load generators generally are software that generate traffic and validate the timing and
success of the responses. Commercial load generators, such as Ixia or Spirent provide both the
clients and servers. With systems that include the servers, often the servers do not start until the
test starts. Service Engines health checking the servers will find them down for a few seconds
after clients begin sending requests.

VMware by Broadcom 478

VMware Avi Load Balancer Administration Guide

Service Engine Optimization

Ensure the Service Engines to be tested have been configured with enough hardware resources
to achieve the desired goals. This includes hardware such as CPU, memory, NICs etc. Every
virtualized environment has different performance maximums, particularly in the packets per
second able to be sent to a single VM. For instance, ESX 5.5 is about 500,000/s, ESX is about
950,000/s.

For more information on general guidelines on performance, see Sizing Service Engines topic in
the VMware Avi Load BalancerConfiguration Guide.

Avi Load Balancer Configuration Recommendations

The following configuration changes should be made to Avi Load Balancer:

n Health monitors: Disable active and passive health monitors from within the pool. This
ensures there is no lag between the servers being turned on, clients sending traffic, and
Avi Load Balancer waiting for a successful health monitor response.

n Slow Ramp: Set the pool’s Slow Ramp time to 0, effectively disabling the ramp. This ensures
Avi Load Balancer sends connections as fast as it receives them. The assumption in a load
test is the clients and servers will not be the bottleneck.

n Logs: Disable Non-Significant client logs if they have been enabled. It is never recommended
to attempt to log hundreds of thousands of requests per second, as there is a performance
penalty for this task. Significant logs (errors) will still be performed, which is the default state
of a virtual service.

When deploying SEs in virtual environments such as ESX:

n Reserve CPU

n Reserve Memory

VMware by Broadcom 479

Maintaining Avi Load Balancer
Controller Single Node Cluster
Availability using VMware
11
It is recommended to deploy Avi Load Balancer Controller in a three-node cluster. Owing to the
limited availability of resources, some businesses may not find this recommendation preferable.
In such cases, Avi Load Balancer Controller can be deployed as a standalone node. In a
standalone node deployment, the availability of Avi Load Balancer Controller is maintained by
the underlying infrastructure rather than the clustering mechanism.

By deploying and hosting Avi Load Balancer Controller on VMware, VMware’s native solutions
can be utilized for maintaining high availability for Avi Load Balancer Controller. This section
focuses on how a single node Avi Load Balancer Controller cluster is being hosted within
VMware and how using native solutions we can maintain and restore Controller availability
both proactively and reactively to an infrastructure impact.It focuses on how to use the
VMware environment in conjunction with a deployed Controller. However, setting up the VMware
environment is not within the scope of this section.

Note
n It is highly recommended to maintain up-to-date configuration archives. A single node cluster
implies there is only one host maintaining the configuration. During disaster recovery, for
example, after a storage failure, Avi Load Balancer Controller will have to be restored from
the configuration archive.

n For more information on VMware HA/ DRS/ vMotion on Controller cluster and Service
Engines, see NSX Advanced Load Balancer High Availability in VMware vCenter Environment
topic in the VMware Avi Load BalancerInstallation Guide.

VMware Solutions
VMware provides native mechanisms for maintaining availability of critical applications and virtual
machines. vMotion and VMware High Availability (VMware HA) were tested, and any impact
to availability of a single node Avi Load Balancer Controller was measured and data integrity
validated.

VMware by Broadcom 480

VMware Avi Load Balancer Administration Guide

vMotion
vMotion allows live migration of a virtual machine from one host to another with no downtime.
This is a proactive approach to maintaining availability of a virtual machine’s services, migrating
virtual machines prior to server maintenance, or move virtual machines off a degraded or failing
server.

For more information, see vMotion Overview and Best Practices for Configuring Resources for
vMotion.

VMware High Availability (VMware HA)

VMware high availability provides failure protection for applications and virtual machines. In case
of server failure, VMware HA allows virtual machines to be dynamically migrated and started up
on another server restoring the application’s availability.

For more information, see About VMware HA and Best practices for Configuring Resources for
VMware.

Validation Scenarios
The following scenarios were tested for validating, restoring, and maintaining the availability of a
single node Avi Load Balancer Controller running on VMware:

n vMotion live migration

n Server failure

Setup Specifications
The following setup specifications were used when validating the scenarios:

n VMware vSphere 6.0 update 3

n Avi Load Balancer 17.2.7

n NFS Shared Datastore

Use Cases
vMotion Live Migration

During the vMotion live migration, Avi Load Balancer Controller was migrated from one server
to another, all the while being powered on.

Expectation

Availability maintained throughout with instances of increased API latency.

Result

VMware by Broadcom 481

VMware Avi Load Balancer Administration Guide

During the process, Avi Load Balancer Controller remained available with intermittent
occurrences of increased latency to the API. After migration, all services are restored and
the Controller is back to functioning normally. There was no data path impact (load balanced
applications) for the duration of the test scenario.

vmotion

NSX Advanced NSX Advanced

Load Balancer Controller Load Balancer Controller

ESXi 1 ESXi 2

Server Failure

The ESXi server hosting the Avi Load Balancer Controller was made to fail.

Expectation

Unavailability of Controller for 3 – 5 minutes, with additional 2-3 minutes before the controller
services are up and running. VMWare high availability restores the availability of the Avi Load
Balancer Controller.

migrate

Avi Controller Avi Controller

ESXi 1 ESXi 2

Result

On ESXi host failure, it took between 3-5 minutes for vCenter to detect the host failure,
migrate the Avi Load Balancer Controller and power on.In the next 2-3 minutes, Controller
services were up and the APIs were available.There was no data path impact (load balanced
applications) for the duration of the test scenario.

Avi Load Balancer Deployment Recommendations

There are some best deployment recommendations to be considered when deploying using a
single Avi Load Balancer Controller. These recommendations are applicable when the Avi Load
Balancer Controller and Service Engines are hosted on the same VMware cluster. Owing to the
potential fate sharing of the Service Engines and Avi Load Balancer Controller being hosted on
the same cluster, both the control plane availability (Avi Load Balancer Controller) and the data
plane (Service Engine) availability are taken into consideration. These recommendations may
vary based on the:

VMware by Broadcom 482

VMware Avi Load Balancer Administration Guide

For more information on configured SE group, see Elastic HA for NSX Advanced Load Balancer
Service Engines topic in the VMware Avi Load BalancerInstallation Guide or Legacy HA for NSX
Advanced Load Balancer Service Engines topic in the VMware Avi Load BalancerConfiguration
Guide.
For more information on VMware integration, see Installing NSX Advanced Load Balancer in
VMware vSphere Environments topic in the VMware Avi Load BalancerInstallation Guide.

Note These recommendations are applicable when the Avi Load Balancer Controller and Avi
Load Balancer Service Engines are hosted on the same VMware server cluster.

Elastic HA - Write Access

When deploying Service Engines in an Elastic HA mode, it is recommended to host Avi Load
Balancer Controller on different servers than the Service Engines. If the Service Engine and
Controller are hosted on the same server, in a failure scenario, the virtual services hosted on that
Service Engine will be impacted until the controller services are completely restored.

Note vMotion and DRS are not recommended for Service Engines.

Configurations in Avi Load Balancer

Modify the Service Engine group to specify which hosts/clusters the Service Engines can/can
not be created on. By doing so, you can specify hosts that the Controller will not be hosted
on. For more information, see Performing an Include-Exclude Operation on a Cluster and Host
topic in the VMware Avi Load BalancerConfiguration Guide.

Configurations in VMWare

For more information, see VMware VM Affinity and VMware VM Affinity Setup.

Elastic HA - No Access
When deploying Service Engines in an elastic HA mode, it is recommended to host Avi Load
Balancer Controller on different servers than the Service Engines. If the Service Engine and
Avi Load Balancer Controller are hosted on the same server, then in a failure scenario, the
virtual services hosted on that Service Engine will be impacted until the controller services are
completely restored.

Note vMotion and DRS are not recommended for Service Engines.

Configurations in Avi Load Balancer

VMware by Broadcom 483

VMware Avi Load Balancer Administration Guide

None required.

Configurations in VMWare

To keep the Avi Load Balancer Controller separate from the Service Engines, we will be using
VM Affinity rules within VMWare. VM Affinity rules are configured such that the controller
runs on servers different from the servers hosting the Service Engines. We will use one rule
for defining which hosts are eligible for the controller and another for servers for which the
Service Engines are eligible for. By doing so, you can specify different hosts for Avi Load
Balancer Controller than those that are specified for the Service Engines. These rules should
keep the Service Engines separated from the Avi Load Balancer Controllers, thus reducing
the risk to data plane traffic in the event of a host failure.

For more information, see VMware VM Affinity and VMware VM Affinity Setup.

Legacy HA - write access

When deploying Service Engines in the Legacy HA mode, it is recommended that Avi Load
Balancer Controller is hosted on different servers than the Service Engines. If the primary
Service Engine and Controller are hosted on the same server, in a failure scenario, the virtual
services hosted on that Service Engine will be impacted until the controller services are
completely restored.

Note vMotion and DRS are not recommended for Service Engines.

Configurations in Avi Load Balancer

Modify the Service Engine group to specify which hosts or clusters in which the Service
Engines can/cannot be created on. By doing so, you can specify hosts that Avi Load Balancer
Controller will not be hosted on.

For more information, see Performing an Include-Exclude Operation on a Cluster and Host
topic in the VMware Avi Load BalancerConfiguration Guide.

Configurations in Avi Load Balancer

To keep the Avi Load Balancer Controller separate from the Service Engines, we will be
using VM Affinity rules within VMWare. VM Affinity rules are configured such that Avi Load
Balancer Controller runs on servers different from the servers hosting the Service Engines.

For more information, see VMware VM Affinity and VMware VM Affinity Setup.

VMware by Broadcom 484

VMware Avi Load Balancer Administration Guide

Legacy HA - No Access
When deploying Service Engines in an elastic HA mode, it is recommended to host Avi Load
Balancer Controller on different servers than the Service Engines. If the Service Engine and Avi
Load Balancer Controller are hosted on the same server, in a failure scenario, the virtual services
hosted on that Service Engine will be impacted until the Avi Load Balancer Controller services are
completely restored.

Note vMotion and DRS are not recommended for Service Engines.

If the server resources are not sufficient to keep the Avi Load Balancer Controller separate from
all the Service Engines, it is recommended to not select Distribute Load within the Legacy HA
configuration. This will result in only a single Service Engine being primary for all virtual services.

Configurations in Avi Load Balancer

If there are not enough compute resources available preventing the controller from being
hosted separately from all Service Engines, do not configure Distribute Load within the
Legacy HA setup.

For more information, see Legacy HA for NSX Advanced Load Balancer Service Engines topic
in the VMware Avi Load BalancerConfiguration Guide.

Configuration in VMware

To host the Avi Load Balancer Controller separately we will be using the VM Affinity rules
within VMware. We will use one rule for defining which hosts are eligible for the controller
and another for servers for which the Service Engines are eligible for. By doing so, you
specify different hosts for the controller than those specified for the Service Engines. These
rules should keep the Service Engines separated from the Controllers, thus reducing the risk
to data plane traffic in the event of a host failure.

If there are not enough server resources available to keep Avi Load Balancer Controller
hosted separately from all the Service Engines, setup the Affinity rules to keep the controller
separate from the primary Service Engine. This will prevent impact to the data plane traffic in
the event of the server hosting Avi Load Balancer Controller fails.

For more information, see VMware VM Affinity and VMware VM Affinity Setup.

In conclusion, VMware’s native solutions can be relied upon to provide High Availability for
Avi Load Balancer Controller when a business decides that a single node Controller suits them
the best.

VMware by Broadcom 485

Avi Load Balancer SaaS
12
The Avi Load Balancer SaaS complements the flagship Avi Load Balancer Platform, as the cloud-
hosted option to deliver application services including distributed load balancing, web application
firewall, global server load balancing (GSLB), network and application performance management
across a multi-cloud environment. It helps ensure fast time-to-value, operational simplicity, and
deployment flexibility in a highly secure manner.

Avi Load Balancer Architecture

The Avi Load Balancer architecture comprises two key components:

n The Controller

The Avi Load Balancer Controller is the control plane for the Avi Load Balancer solution.
It provides a single point of management and operations, including configuration, visibility,
analytics, and metrics. It uses big data analytics to analyze the data and present actionable
insights to administrators on intuitive dashboards.

n The Service Engines

The Avi Load Balancer Service Engines (SEs) are the data-plane component. SEs collect
real-time application analytics from traffic flows between end users and applications. The SEs
are high-performance data-plane entities which perform the actual load balancing (LB) and
web application firewall (WAF) functions. All configurations for a given SE are managed by
the respective Controller Cluster.

VMware by Broadcom 486

VMware Avi Load Balancer Administration Guide

AVI CONSOLE

REST API

AVI CONTROLLER

AVI SERVICE ENGINES

For more information on the Avi Load Balancer Architecture, see the Load Balancing topic in the
VMware Avi Load Balancer Configuration Guide.

Introduction to Avi Load Balancer SaaS

Prior to the introduction of Avi Load Balancer SaaS, Controllers and Service Engines would be
deployed in the customer environment. With the advent of Avi Load Balancer SaaS, an additional
operational model is offered, control plane as a service. With Avi Load Balancer SaaS, the Avi
Load Balancer hosts and operates the control plane. Service Engines (the data plane) continue
to reside in the customer environment (on-premises or in a public cloud). The following diagram
depicts a Controller managed by Avi Load Balancer.

Software Intelligent Elastic

Load Balancer WAF Service Mesh

Avi-Managed Customer-Managed

Data Center Hybrid Cloud

ON- PREMISES ON- PREMISES + CLOUD PUBLIC OR PRIVATE CLOUD

VMware by Broadcom 487

VMware Avi Load Balancer Administration Guide

The salient features are listed below:

n Avi Load Balancer-managed control plane, available as a service.

n Instantaneous on-boarding: Avi Load Balancer Controller deployed in minutes.

n The Avi Load Balancer provides continuous monitoring, configuration backups, software
upgrades and patches, security alerts, and performance tune-ups

n Proactive support

n Multi-cloud solutions — data plane can be on-premises or in a public cloud.

Requirements for Connectivity to Avi Load Balancer SaaS

The following considerations are important while connecting your environment to Avi Load
Balancer SaaS.

Case 1: No additional connectivity requirements from Avi Load Balancer SaaS to customer
environment

In this scenario, Avi Load Balancer SaaS communicates with public API endpoints for
respective Infrastructure as a service (IaaS) cloud infrastructure where the customer
workloads are provisioned. This works automatically, with no additional configuration or
access policy requirement. The Avi Load Balancer SaaS supports the following cloud types in
this mode:

n Amazon Web Services

n Microsoft Azure

n OpenStack with public URLs (for example, Platform 9)

In addition, the Avi Load Balancer SaaS can work with no-orchestrator clouds. In this case,
SE provisioning is done by the customer manually (or with the help of automation tools such
as Ansible or Terraform). Avi Load Balancer SaaS does not require inbound access to the
customer virtualization infrastructure. The cloud types supported in this mode are:

n Linux Server Cloud

n No-orchestrator clouds (for example, VMware read access)

Case 2: Avi Load Balancer SaaS requires connectivity to customer’s orchestration environment

In this scenario, the Avi Load Balancer SaaS communicates with the infrastructure typically
residing in a private network. Once Avi Load Balancer SaaS is allowed to communicate
with the virtualization infrastructure, it can communicate continuously with the virtualization
orchestrator and provide fully automated provisioning. The Avi Load Balancer SaaS supports
the following cloud types in this mode:

n VMware write access

n OpenStack hosted on-premises

VMware by Broadcom 488

VMware Avi Load Balancer Administration Guide

n VMware read and write access

Note
n In both Case 1 and 2 above, the Service Engines need to communicate with Avi Load
Balancer SaaS. This might require allowing communication between the Avi Load Balancer
Service Engines and the Controller IPs, over secure communication channels such as
HTTPS and SSH.

n Linux Server Cloud can function in either of the modes above.

Deploying Avi Load Balancer SaaS

This section lists the process for deploying the Avi Load Balancer Platform with the Avi Load
Balancer SaaS operational model. The full deployment process is divided into three steps:

n Registering for Avi Load Balancer SaaS

n Connecting Avi Load Balancer SaaS to your cloud

n Configuring application delivery features

Registering for Avi Load Balancer SaaS

1 To sign up for Avi Load Balancer SaaS, navigate to https://info.avinetworks.com/saas and
register.

VMware by Broadcom 489

VMware Avi Load Balancer Administration Guide

2 An Avi Load Balancer representative will contact you and provide a URL through which to
connect. Log in to create an Avi Load Balancer account.

Once the previous step completed, you will be provided with a login ID and the URL with
which you can access Avi Load Balancer SaaS.

Connecting Your Cloud to Avi Load Balancer SaaS

In this section, Avi Load Balancer SaaS is configured to integrate with the network infrastructure
where your applications are hosted (and where the Avi Load Balancer Service Engines will be
provisioned).

Depending on the cloud type, please refer to the respective document mentioned below:

VMware by Broadcom 490

VMware Avi Load Balancer Administration Guide

Cloud Type Documentation

Microsoft Azure See Installing Avi Load Balancer in Microsoft Azure topic
in the VMware Avi Load BalancerInstallation Guide.

Amazon Web Service See Installing Avi Load Balancer in Amazon Web Services
topic in the VMware Avi Load BalancerInstallation Guide.

Linux server cloud See Considerations for Deploying Avi Load Balancer
in Linux Server Cloud topic in the VMware Avi Load
BalancerInstallation Guide.

Configuring Application Delivery Features

Once the cloud is integrated with Avi Load Balancer SaaS, you can get started with creating your
first virtual service. For more information, see Create a Virtual Service topic in the VMware Avi
Load Balancer Configuration Guide.
Read the following topics next:

n Configuring Avi Load Balancer SaaS Controller Using Service Account

n Accessing the Avi Load Balancer SaaS REST API

n Accessing the Avi Load Balancer SaaS CLI Using SSH

Configuring Avi Load Balancer SaaS Controller Using

Service Account
The Avi Load Balancer supports single sign-on (SSO) to the Avi Load Balancer Controller UI
using Security Assertion Markup Language (SAML). However, during debugging or even normal
day-to-day operations, there is often a need to access the Controller’s CLI using SSH. SAML
credentials cannot be used to login to the CLI.

To access the Controller through SSH, a registered user must have a valid token. Once a token
has been created, one can initiate an SSH connection to the Controller using cli as the SSH user.
A CLI shell will be created. Once the shell has been created, a login prompt will be presented.
Provide the required username and the token as the password. This topic explains the process
needed to configure a service account for use on an Avi Load Balancer SaaS Controller.

Generate the Authorization Token

1 Log in to the Avi Load Balancer UI.

2 Click the three dots in the dashboard.

VMware by Broadcom 491

VMware Avi Load Balancer Administration Guide

3 Click Generate Token.

A pop-up screen appears as shown below:

4 Enter the Lifetime for the token’s validity in hours.

Note
a To generate a single use token, enter 0.

b The maximum value that can be entered in this field is 87600 hours.

c In case another token is generated before the first one expires, the first token still remains
valid.

5 Click Generate. The token is generated for your Service Account.

VMware by Broadcom 492

VMware Avi Load Balancer Administration Guide

6 Save this token for your automation or API usage.

7 To test your credentials use the following Python code using the Requests library.

import requests
import urllib3

urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

data = {
'username': '<service account name>',
'password': '<your token that was generated in step 5>
}
login = requests.post('https://<your controller name>.saas.avinetworks.com/login',
verify=False, data=data)
print(login.status_code)

The status code 200 is returned for a successful query, and the status code 401 is returned
for the failed query.

Accessing the Avi Load Balancer SaaS REST API

To access an Avi Load Balancer SaaS instance through the REST API, a registered user must
have a valid token. Once that token has been created, the user can authenticate to the Avi Load
Balancer Controller REST API using the token as the password.

Generate the Authorization Token

1 Login to the Avi Load Balancer UI.

2 Click the three dots in the dashboard.

3 Click Generate Token.

VMware by Broadcom 493

VMware Avi Load Balancer Administration Guide

A pop-up screen appears as shown below:

4 Enter the Lifetime for the token’s validity in hours.

Note
a To generate a single use token, enter 0.

b The maximum value that can be entered in this field is 87600 hours.

c In case another token is generated before the first one expires, the first token still remains
valid.

5 Click Generate. The token is generated and displayed as shown below:

6 Copy the token.

VMware by Broadcom 494

VMware Avi Load Balancer Administration Guide

Login to Avi Load Balancer SaaS REST API

As shown in the SDK example below, access the Controller API using the token as your
password.

from avi.sdk.avi_api import ApiSession

api = ApiSession.get_session("avikb.saas.avinetworks.com", "[email protected]",

"499e4c833c183312f3eeab2c9f5e8bd47c48d440", tenant="kb")

#----- retrieve virtualservices

resp = api.get('virtualservice')

Accessing the Avi Load Balancer SaaS CLI Using SSH

In an Avi Load Balancer SaaS environment, SAML authenticates the user to the Avi Load Balancer
Controller UI. However, during debugging or even normal day-to-day operations, it can be useful
to access the Controller’s CLI using SSH instead.

For more information on Accessing the Avi Load Balancer SaaS CLI, see Avi Load Balancer CLI
Access to a SAML-Authenticated Avi Load Balancer Controller topic in the VMware Avi Load
BalancerAdministration Guide.

VMware by Broadcom 495

SAML Support for Avi Load
Balancer SDK 13
Avi Load Balancer supports SDKs to use IdP credentials for it and a REST API login. The Python
SDK needs a SAML authentication profile set up on the Controller to establish a connection and
access resources.

Note
n Logging into the Avi Load Balancer CLI using IdP credentials is not yet supported.

n SAML-based authentication using the Python SDK is supported for Okta and OneLogin.

n The service provider (SP) never directly interacts with the identity provider. A browser or the
Python SDK acts as the agent to carry out all redirections.

n The service provider needs to know to which identity provider to redirect before it has any
idea who the user is.

n The service provider does not know who the user is until the SAML assertion comes back
from the identity provider.

n The process of SAML authentication is asynchronous. Whether the IdP will finish the entire
flow is unknown to the service provider. As a result, the service provider does not keep
records of the status of any generated authentication requests. When an IdP responds to the
service provider, the response must contain all necessary information.

For more information, see SAML Authentication for Single Sign-On topic.

SAML Python SDK

Under the SDK, a file named saml_avi_api.py contains the IdP class definition for each supported
IdP. IdP-specific classes are inherited from the ApiSession base class. An IdP-specific class
definition has its own authentication method to be called to authenticate a given user. URL
redirection and SAML assertion are handled in this class. This class returns the Controller session
after successful authentication from the given IdP.

Example of Okta
In this collection of code snippets, the OktaSAMLApiSession class is used to authenticate a user for
Okta IdP, get the Controller session, and create the virtual service.

VMware by Broadcom 496

VMware Avi Load Balancer Administration Guide

from avi.sdk.saml_avi_api import OktaSAMLApiSession

Create Avi Load Balancer API Session

api = OktaSAMLApiSession("10.10.10.42", "okta_username", "okta_password")

api = ApiSession.get_session("controller_ip", username="foo", password="foo",

idp=OktaSAMLApiSession)

Create VS Using Pool sample_pool

pool_obj = api.get_object_by_name('pool', 'sample_pool')

pool_ref = api.get_obj_ref(pool_obj)
services_obj = [{'port': 80, 'enable_ssl': False}]
vs_obj = {'name': 'sample_vs', 'ip_address': {'addr': '11.11.11.42', 'type': 'V4'},
'services': services_obj, 'pool_ref': pool_ref}
resp = api.post('virtualservice', data=vs_obj)

Print List of all Virtual Services

resp = api.get('virtualservice')
for vs in resp.json()['results']:
print vs['name']

Delete a Virtual Service

resp = api.delete_by_name('virtualservice', 'sample_vs')

Example of OneLogin
In this collection of code snippets, the OneloginSAMLApiSession class is used to authenticate a
user for OneLogin IdP, get the Controller session, and create the virtual service.

from avi.sdk.saml_avi_api import OneloginSAMLApiSession

Create Avi Load Balancer API Session

api = OneloginSAMLApiSession("10.10.10.42", "onelogin_username",

"onelogin_password")

VMware by Broadcom 497

VMware Avi Load Balancer Administration Guide

api = ApiSession.get_session("10.10.10.42", "ws1_username", "ws1_password",

idp_class=WS1loginSAMLApiSession)

Create VS Using Pool sample_pool

pool_obj = api.get_object_by_name('pool', 'sample_pool')

Print List of all Virtual Services

resp = api.get('virtualservice')
for vs in resp.json()['results']:
print vs['name']

Delete a Virtual Service

resp = api.delete_by_name('virtualservice', 'sample_vs')

Example of Workspace ONE Access

In this collection of code snippets, the WS1loginSAMLApiSession class is used to authenticate a
user for WS1 IdP, get the Controller session, and create the virtual service.

from avi.sdk.saml_avi_api import WS1loginSAMLApiSession

Create Avi Load Balancer API Session

session = WS1loginSAMLApiSession("10.10.10.42", "ws1_username", "ws1_password")

api = ApiSession.get_session("10.10.10.42", "ws1_username", "ws1_password",

idp_class=WS1loginSAMLApiSession)

Create Pool with One Server

pool_obj = {'name': 'sample_pool', 'servers': [{'ip': {'addr': '192.0.0.1', 'type':

'V4'}}]}
pool_resp = session.post('pool', data=pool_obj)
print(pool_resp.json())

Create vsvip

vsvip_obj = {'name': 'sample_vsvip', 'vip': [{'vip_id': '1',

'ip_address': {'addr': '11.11.11.42', 'type':
'V4'}}]}

VMware by Broadcom 498

VMware Avi Load Balancer Administration Guide

vsvip_resp = session.post('vsvip', data=vsvip_obj)

print(vsvip_resp.json())

Create Virtual Service using sample_vsvip and sample_pool

pool_ref = '/api/pool?name={}'.format(pool_obj.get('name'))
vsvip_ref = '/api/vsvip?name={}'.format(vsvip_obj.get('name'))
services_obj = [{'port': 80, 'enable_ssl': False}]
vs_obj = {'name': 'sample_vs', 'services': services_obj, 'vsvip_ref': vsvip_ref,
'pool_ref': pool_ref}
resp = session.post('virtualservice', data=vs_obj)

Print List of all Virtual Services

resp = session.get('virtualservice')
for vs in resp.json()['results']:
print vs['name']

Delete a Virtual Service

resp = session.delete_by_name('virtualservice', 'sample_vs')

VMware by Broadcom 499

Avi Load Balancer Go SDK
Support 14
This section explains how to install, configure, and use the Avi Load Balancer Go SDK package
for the various automation requirements.

Avi Load Balancer Go SDK is a Go Package that provides other APIs to communicate with the
Avi Load Balancer REST APIs. Avi Load Balancer GO SDK package also provides utilities, which
can be used to simplify and automate load balancing configurations on an Avi Load Balancer
Controller. The following are the essential features for the Avi Load Balancer GO SDK package:

n Uses Avi Load Balancer session class and provides utilities to simplify integration with the Avi
Load Balancer Controller.

n Helps in session authentication and keeps a cache of sessions to avoid multiple connections
and tear downs across the different API sessions invocation.

n Automatically updates session cookies and CSRF Tokens from the Avi Load Balancer
Controllers and provides helper APIs and templates for Avi Load Balancer Objects.

n X-AVI-TENANT (tenant) header handling and provides the sample source code for common
load balancing examples.

SDK Directories
Avi Load Balancer Go SDK package is the source for the Go SDK for Avi Load Balancer Controller
configuration. It is not required to download packages. Go will take care of package installations.

The following are the important Avi Load Balancer Go SDK directories:

1 examples: Go samples are in the go/examples directory. create_vs.go provides automation

samples for the following configuration on Avi Load Balancer Controller:

n Create session

n Generic client to Avi Load Balancer Controller

n Create a pool with one server, create an SSL virtual service, and fetch an object by name

2 clients: Contains Avi Load Balancer clients to establish a connection between the Go SDK and
Avi Load Balancer Controller using the Avi Load Balancer session class. Each resource has its
client to establish the connection.

VMware by Broadcom 500

VMware Avi Load Balancer Administration Guide

3 session: Contains all the generic codes of the REST API calls for the Avi Load Balancer session
and helper routines from REST API calls. It creates and maintains a session for the given
resource.

4 models: Contains all models required to capture the API response. These are the structures to
grab and store data of corresponding Avi Load Balancer REST API calls.

Prerequisites
Go Lang package must be installed before installing the Avi Load Balancer Go SDK package. To
install the Go Lang package, see Go Lang Installation.

For more information on the Go programming language, see Go Lang Documentation.

Installing Avi Load Balancer Go SDK Package

The following is the syntax for installing Avi Load Balancer Go SDK package:

$ mkdir -p src/github.com/avinetworks/
$ cd src/github.com/avinetworks/
$ git clone https://github.com/avinetworks/sdk.git
#GOPATH will be path till src dir.
$ export GOPATH=~/src

Usage Examples
To create a session, a pool, and a basic virtual service (my-test-vs), execute create_vs.go
file. Before executing this script, specify Avi Load Balancer Controller IP address, username,
password, and tenant in create_vs.go file.

Importing Avi Load Balancer Sessions, Models, and Clients

package main

import (
"github.com/avinetworks/sdk/go/clients"
"github.com/avinetworks/sdk/go/models"
"github.com/avinetworks/sdk/go/session"
)

Creating Avi Load Balancer API Session

The sample syntax mentioned below creates a session with the following attributes:

n Client session IP address: 10.10.25.25

n Tenant: admin

VMware by Broadcom 501

VMware Avi Load Balancer Administration Guide

n Session Type: Not secure

aviClient, err := clients.NewAviClient("10.10.25.25", "admin",

session.SetPassword("something"),
session.SetTenant("admin"),
session.SetInsecure)

Creating a Pool

The syntax mentioned below creates a pool with the following attributes:

n Pool Name: my-test-pool

n Server IP Address: 10.90.20.12

pobj := models.Pool{}
pobj.Name = "my-test-pool"
serverobj := models.Server{}
serverobj.Enabled = true
serverobj.IP = &models.IPAddr{Type: "V4", Addr: "10.90.20.12"}
pobj.Servers = append(pobj.Servers, &serverobj)
npobj, err := aviClient.Pool.Create(&pobj)
if err != nil {
fmt.Println("Pool creation failed: ", err)
return
}

Creating a Virtual Service

The syntax mentioned below created a virtual service with the following attributes:

n Name: my-test-vs

n IP Address: 10.90.20.51

n Port Number: 80

n The pool associated with the virtual service: my-test-pool

vsobj := models.VirtualService{}
vsobj.Name = "my-test-vs"
vipip := models.IPAddr{Type: "V4", Addr: "10.90.20.51"}
vsobj.Vip = append(vsobj.Vip, &models.Vip{VipID: "myvip", IPAddress: &vipip})
vsobj.PoolRef = npobj.UUID
vsobj.Services = append(vsobj.Services, &models.Service{Port: 80})

nvsobj, err := aviClient.VirtualService.Create(&vsobj)

if err != nil {
fmt.Println("VS creation failed: ", err)
return
}
fmt.Printf("VS obj: %+v", *nvsobj)

Fetching an Object by the Name

VMware by Broadcom 502

VMware Avi Load Balancer Administration Guide

The sample syntax mentioned below fetches the virtual service details with the name
my-test-vs and provides the following output: Cloud UUID: f39f950a-e6ca-442d-b546-
fc31520991bb.

var obj interface{}

err = aviClient.AviSession.GetObjectByName("virtualservice", "my-test-vs", &obj)
fmt.Printf("VS obj: %v\n", obj)

err = aviClient.AviSession.GetObject(
"virtualservice", session.SetName("my-test-vs"), session.SetResult(&obj),
session.SetCloudUUID("cloud-f39f950a-e6ca-442d-b546-fc31520991bb"))
fmt.Printf("VS with CLOUD_UUID obj: %v", obj)

Deleting a Virtual Service

The syntax mentioned below delete the desired virtual service using the Object ID for the
virtual service.

aviClient.VirtualService.Delete(nvsobj.UUID)

Deleting a Pool

Use the following syntax to delete the desired pool.

aviClient.Pool.Delete(npobj.UUID)

Creating a Basic Virtual Service (my-test-vs) using create_vs.go

$ go run create_vs.go

Metric and Inventory API Example

package main

import (
//"flag"
"fmt"
"github.com/avinetworks/sdk/go/clients"
"github.com/avinetworks/sdk/go/session"
)

type MetricRequest struct {

Step int `json:"step"`
Limit int `json:"limit"`
EntityUUID string `json:"entity_uuid"`
MetricID string `json:"metric_id"`
IncludeName string `json:"include_name"`
IncludeRefs string `json:"include_refs"`
PadMissingData string `json:"pad_missing_data"`
}

type Metrics struct {

VMware by Broadcom 503

VMware Avi Load Balancer Administration Guide

MetricRequests []MetricRequest `json:"metric_requests"`

}

func main() {
// Create a session and a generic client to Avi Controller
aviClient, err := clients.NewAviClient("10.10.25.42", "admin",
session.SetPassword(""),
session.SetTenant("admin"),
session.SetInsecure)
if err != nil {
fmt.Println("Couldn't create session: ", err)
return
}
mr := MetricRequest{Step: 1, Limit: 1, EntityUUID: "*", MetricID:
"l7_server.max_concurrent_sessions", IncludeName: "True", IncludeRefs: "True",
PadMissingData: "False"}
sr := []MetricRequest{}
sr = append(sr, mr)
req := Metrics{MetricRequests: sr}
path := "/api/analytics/metrics/collection"
var rsp interface{}
aviClient.AviSession.Post(path, req, &rsp)
fmt.Printf("response %v\n", rsp)
}

Compilation
Use the following syntax to compile the Avi Load Balancer Go SDK package.

$ go build -o /usr/bin/create_vs create_vs.go

Including Go SDK package in a third party code

The Avi Load Balancer Go SDK package can also be integrated with any third-party Go code
package. The following is an example of importing the Avi Load Balancer Go SDK package into a
third-party Go code package.

import
(
"github.com/avinetworks/sdk/go/clients"
"github.com/avinetworks/sdk/go/session"

The following is an example entry of vendor.json file for the Terraform provider:

"package": [ {
"path": "github.com/avinetworks/sdk/go/clients",
"revision": "796ddcccdc37a5a9771bfbb716b159ae5c9b4b11",
"revisionTime": "2018-04-06T16:51:27.185773",
"version": "18.1.3",
"versionExact": "18.1.3"
},
{
"path": "github.com/avinetworks/sdk/go/session",

VMware by Broadcom 504

VMware Avi Load Balancer Administration Guide

"revision": "796ddcccdc37a5a9771bfbb716b159ae5c9b4b11",
"revisionTime": "2018-04-06T16:51:27.185773",
"version": "18.1.3",
"versionExact": "18.1.3"
} ]

Additional Information
The location on GitHub for Avi Load Balancer Go SDK package: https://github.com/vmware/alb-
sdk/tree/eng/go

VMware by Broadcom 505

JNCIE
100% (3)
JNCIE
389 pages
C79000-G8976-1477 ROX-II v2.14 RX1500 ConfigurationManual CLI
No ratings yet
C79000-G8976-1477 ROX-II v2.14 RX1500 ConfigurationManual CLI
1,014 pages
Bug Fix Appendix-8.1.4.7
No ratings yet
Bug Fix Appendix-8.1.4.7
172 pages
VMware NSX Advanced Load Balancer Admin Guide
No ratings yet
VMware NSX Advanced Load Balancer Admin Guide
393 pages
VMware Avi Load Balancer VSphere Deployment Guide
No ratings yet
VMware Avi Load Balancer VSphere Deployment Guide
54 pages
VMware Avi Load Balancer Vsphere Design Guide
No ratings yet
VMware Avi Load Balancer Vsphere Design Guide
21 pages
HyperV Admin AOS v58
100% (1)
HyperV Admin AOS v58
40 pages
AHV Admin Guide v51
No ratings yet
AHV Admin Guide v51
61 pages
Integrated Openstack 41 Administration Guide
No ratings yet
Integrated Openstack 41 Administration Guide
122 pages
AHV Admin Guide v5 16
No ratings yet
AHV Admin Guide v5 16
74 pages
VMware_Cloud_Director_Tenant_Guide
No ratings yet
VMware_Cloud_Director_Tenant_Guide
527 pages
Integrated Openstack 41 Administration Guide
No ratings yet
Integrated Openstack 41 Administration Guide
134 pages
Ent Cloud Platform Admin 50 Courseware Labmanual Ahv
100% (1)
Ent Cloud Platform Admin 50 Courseware Labmanual Ahv
31 pages
VCD - 97 - Vcloud Director Tenant Portal Guide
No ratings yet
VCD - 97 - Vcloud Director Tenant Portal Guide
248 pages
vmware-powercli-133-user-guide
No ratings yet
vmware-powercli-133-user-guide
201 pages
Vmware Powercli 131 User Guide
No ratings yet
Vmware Powercli 131 User Guide
185 pages
Vsphere Esxi Vcenter Server 703 Host Management Guide
No ratings yet
Vsphere Esxi Vcenter Server 703 Host Management Guide
219 pages
Elastic Load Balancing: User Guide
No ratings yet
Elastic Load Balancing: User Guide
32 pages
Vmware Powercli 114 User Guide
No ratings yet
Vmware Powercli 114 User Guide
123 pages
VMware Cloud Director Tenant Portal Guide
No ratings yet
VMware Cloud Director Tenant Portal Guide
332 pages
VMware PowerCLI User's Guide
No ratings yet
VMware PowerCLI User's Guide
150 pages
Vmware Powercli 124 User Guide
No ratings yet
Vmware Powercli 124 User Guide
146 pages
Microsoft Remote Desktop Services Deployment Guide
No ratings yet
Microsoft Remote Desktop Services Deployment Guide
62 pages
Vcenter Server and Host Management VMware Vsphere6.5 - ESXi6.5 - Vcenter Server6.5 PDF
No ratings yet
Vcenter Server and Host Management VMware Vsphere6.5 - ESXi6.5 - Vcenter Server6.5 PDF
184 pages
NS TrafficMgmt Guide
100% (1)
NS TrafficMgmt Guide
798 pages
NSX 60 Api
No ratings yet
NSX 60 Api
360 pages
Vsphere Esxi Vcenter 802 Management Guide
No ratings yet
Vsphere Esxi Vcenter 802 Management Guide
216 pages
Vsphere Esxi Vcenter 803 Management Guide
No ratings yet
Vsphere Esxi Vcenter 803 Management Guide
207 pages
Vrealize Automation 76 Programming Guide
No ratings yet
Vrealize Automation 76 Programming Guide
413 pages
Vsphere Esxi Vcenter Server 652 Host Management Guide PDF
No ratings yet
Vsphere Esxi Vcenter Server 652 Host Management Guide PDF
190 pages
IEPM2018 3users
No ratings yet
IEPM2018 3users
1,223 pages
Vmware Powercli 127 User Guide
No ratings yet
Vmware Powercli 127 User Guide
152 pages
Vsphere Esxi Vcenter Server 601 Host Management Guide
No ratings yet
Vsphere Esxi Vcenter Server 601 Host Management Guide
182 pages
vsphere-esxi-vcenter-server-703-networking-guide
No ratings yet
vsphere-esxi-vcenter-server-703-networking-guide
297 pages
Vsphere Esxi Vcenter Server 602 Host Management Guide
No ratings yet
Vsphere Esxi Vcenter Server 602 Host Management Guide
200 pages
AHV Admin Guide v5 - 17
No ratings yet
AHV Admin Guide v5 - 17
69 pages
Double-Take Availability User's Guide
No ratings yet
Double-Take Availability User's Guide
656 pages
Vsphere Basic System Administration
No ratings yet
Vsphere Basic System Administration
368 pages
VCD 100 Tenantportal
No ratings yet
VCD 100 Tenantportal
263 pages
docu81578
No ratings yet
docu81578
142 pages
Load Balancer Administration en US
No ratings yet
Load Balancer Administration en US
68 pages
HCL Interview VM
100% (1)
HCL Interview VM
50 pages
Vsphere Esxi Vcenter Server 652 Networking Guide
No ratings yet
Vsphere Esxi Vcenter Server 652 Networking Guide
261 pages
Vsphere Basic System Administration
No ratings yet
Vsphere Basic System Administration
370 pages
9.1 VCenter Server and Host Management
No ratings yet
9.1 VCenter Server and Host Management
198 pages
Asm Guide 114
No ratings yet
Asm Guide 114
278 pages
(Ebook) VMware Cookbook: A Real-World Guide to Effective VMware Use by Ryan Troy ISBN 9780596801748, 0596801742download
100% (3)
(Ebook) VMware Cookbook: A Real-World Guide to Effective VMware Use by Ryan Troy ISBN 9780596801748, 0596801742download
55 pages
VMware Admin Guide
100% (1)
VMware Admin Guide
368 pages
XenServer-6.5.0 WLB Administrators Guide
No ratings yet
XenServer-6.5.0 WLB Administrators Guide
79 pages
Vsphere Esxi Vcenter 80 Management Guide
No ratings yet
Vsphere Esxi Vcenter 80 Management Guide
229 pages
Vra 75 Prepare Use
No ratings yet
Vra 75 Prepare Use
1,040 pages
Vsphere Esxi Vcenter 802 Networking Guide
No ratings yet
Vsphere Esxi Vcenter 802 Networking Guide
310 pages
Configuration Guide VMware
No ratings yet
Configuration Guide VMware
778 pages
VSOS51 Lecture V1 PR PDF
No ratings yet
VSOS51 Lecture V1 PR PDF
360 pages
VSOS51 Lecture V1 PDF
No ratings yet
VSOS51 Lecture V1 PDF
360 pages
Red Hat Enterprise Linux-6-Load Balancer Administration-En-US
No ratings yet
Red Hat Enterprise Linux-6-Load Balancer Administration-En-US
60 pages
NS TrafficMgmt Guide PDF
No ratings yet
NS TrafficMgmt Guide PDF
770 pages
Lenovo RD550 BMC User Guide
No ratings yet
Lenovo RD550 BMC User Guide
94 pages
Barracuda Load Balancer Administrator S Guide
No ratings yet
Barracuda Load Balancer Administrator S Guide
108 pages
VMware Certification Training Course Content
No ratings yet
VMware Certification Training Course Content
3 pages
Virtual Infrastructure Web Access Administrator'S Guide: Esx Server 3.0.1 and Virtualcenter Server 2.0.1
No ratings yet
Virtual Infrastructure Web Access Administrator'S Guide: Esx Server 3.0.1 and Virtualcenter Server 2.0.1
110 pages
Silo - Tips Barracuda Load Balancer Administrator S Guide
No ratings yet
Silo - Tips Barracuda Load Balancer Administrator S Guide
76 pages
Linux DevOps Tools Engineer (701) Practice Tests: 400 Questions to Ace Your Certification
From Everand
Linux DevOps Tools Engineer (701) Practice Tests: 400 Questions to Ace Your Certification
Steve Brown
No ratings yet
RAX711-L (A) Configuration Guide (Rel - 14)
No ratings yet
RAX711-L (A) Configuration Guide (Rel - 14)
407 pages
6GK60230AS1..... Datasheet en
No ratings yet
6GK60230AS1..... Datasheet en
5 pages
Discovering and Managing Brownfield Deployment With Cisco Catalyst Center (Formerly Cisco DNA Center)
No ratings yet
Discovering and Managing Brownfield Deployment With Cisco Catalyst Center (Formerly Cisco DNA Center)
88 pages
CYGNUS-843RE-Bridging-Terminal-Server
No ratings yet
CYGNUS-843RE-Bridging-Terminal-Server
1 page
Technologies Cheat Sheet
No ratings yet
Technologies Cheat Sheet
28 pages
Pasolink NMS
No ratings yet
Pasolink NMS
12 pages
OS9000 Datasheet
No ratings yet
OS9000 Datasheet
8 pages
Verax NMS & APM - Unified Network, Application and Infrastructure Management
No ratings yet
Verax NMS & APM - Unified Network, Application and Infrastructure Management
2 pages
SLES PSP Installation & Online Log Collection.
No ratings yet
SLES PSP Installation & Online Log Collection.
75 pages
Me in Berg
No ratings yet
Me in Berg
5 pages
Application Layer: Network Security
No ratings yet
Application Layer: Network Security
16 pages
OceanStor V500R007 MIB Interface Notes
No ratings yet
OceanStor V500R007 MIB Interface Notes
137 pages
EAP 3660 UsersManual V1 1
No ratings yet
EAP 3660 UsersManual V1 1
33 pages
A6V10316241 NK8237 Installation
No ratings yet
A6V10316241 NK8237 Installation
96 pages
Vendor: Juniper Exam Code: JN0-103 Exam Name: Junos, Associate (JNCIA-Junos) Version: 20.061
No ratings yet
Vendor: Juniper Exam Code: JN0-103 Exam Name: Junos, Associate (JNCIA-Junos) Version: 20.061
12 pages
OmniSwitch 6860 Datasheet
No ratings yet
OmniSwitch 6860 Datasheet
11 pages
ZTE ToB ElasticNet_ UME R88 Product Description
No ratings yet
ZTE ToB ElasticNet_ UME R88 Product Description
26 pages
VCL TP Teleprotection Equipment Easy To Use GUI Guide: Valiant Communications Limited - 2018 1
No ratings yet
VCL TP Teleprotection Equipment Easy To Use GUI Guide: Valiant Communications Limited - 2018 1
58 pages
Installation and Quick Start: Rack Automatic Transfer Switch
No ratings yet
Installation and Quick Start: Rack Automatic Transfer Switch
24 pages
Cisco.200-301.v2021-04-16.q222
No ratings yet
Cisco.200-301.v2021-04-16.q222
104 pages
Os8 Specs Reva
No ratings yet
Os8 Specs Reva
77 pages
PM5560 PM5563 PublicFirmwareRevisionHistory v2.1.0 PDF
No ratings yet
PM5560 PM5563 PublicFirmwareRevisionHistory v2.1.0 PDF
3 pages
Powervault-Tl4000 User's Guide En-Us
No ratings yet
Powervault-Tl4000 User's Guide En-Us
304 pages
Option - Fieldbus - Instruction - For Ac Drives and Tsa - 01 3698 01r11.en
No ratings yet
Option - Fieldbus - Instruction - For Ac Drives and Tsa - 01 3698 01r11.en
178 pages
CompTIA Security+ SY0-301 Practice Test
No ratings yet
CompTIA Security+ SY0-301 Practice Test
28 pages
DD Assignment 2
No ratings yet
DD Assignment 2
24 pages
Imagerunner Series Redefined: Transform The Way You Work
No ratings yet
Imagerunner Series Redefined: Transform The Way You Work
4 pages