CMG_Cloud_Native_Function_Installation_Guide
CMG_Cloud_Native_Function_Installation_Guide
Release 22.8.R1
©2022 Nokia. Nokia Confidential Information. Use subject to agreed restrictions on disclosure and use.
Nokia is committed to diversity and inclusion. We are continuously reviewing our customer documentation and consulting with
standards bodies to ensure that terminology is inclusive and aligned with the industry. Our future customer documentation will be
updated accordingly.
This document includes Nokia proprietary and confidential information, which may not be distributed or disclosed to any third parties
without the prior written consent of Nokia.
This document is intended for use by Nokia's customers ("You"/"Your") in connection with a product purchased or licensed from any
company within Nokia Group of Companies. Use this document as agreed. You agree to notify Nokia of any errors you may find in
this document; however, should you elect to use this document for any purpose(s) for which it is not intended, You understand and
warrant that any determinations You may make or actions You may take will be based upon Your independent judgment and analysis of
the content of this document.
Nokia reserves the right to make changes to this document without notice. At all times, the controlling version is the one available on
Nokia’s site.
NO WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY OF AVAILABILITY,
ACCURACY, RELIABILITY, TITLE, NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, IS MADE IN
RELATION TO THE CONTENT OF THIS DOCUMENT. IN NO EVENT WILL NOKIA BE LIABLE FOR ANY DAMAGES, INCLUDING BUT NOT
LIMITED TO SPECIAL, DIRECT, INDIRECT, INCIDENTAL OR CONSEQUENTIAL OR ANY LOSSES, SUCH AS BUT NOT LIMITED TO LOSS OF
PROFIT, REVENUE, BUSINESS INTERRUPTION, BUSINESS OPPORTUNITY OR DATA THAT MAY ARISE FROM THE USE OF THIS DOCUMENT
OR THE INFORMATION IN IT, EVEN IN THE CASE OF ERRORS IN OR OMISSIONS FROM THIS DOCUMENT OR ITS CONTENT.
Copyright and trademark: Nokia is a registered trademark of Nokia Corporation. Other product names mentioned in this document
may be trademarks of their respective owners.
©2022 Nokia.
©2022 Nokia. Nokia Confidential Information. Use subject to agreed restrictions on disclosure and use.
CMG CLOUD NATIVE FUNCTION INSTALLATION GUIDE Table of Contents
Table of Contents
List of Figures.............................................................................................................................................. 5
List of Tables................................................................................................................................................6
1 Getting started.............................................................................................................................................7
2 What’s new....................................................................................................................................................9
4 CMG as CNF................................................................................................................................................15
4.1 CMG and containers............................................................................................................................... 15
4.1.1 Supported CMG CNF functions...................................................................................................... 16
4.1.1.1 CMG CNF as CP....................................................................................................................... 16
4.1.1.2 CMG CNF as UP....................................................................................................................... 17
4.1.1.3 CMG CNF as ePDG...................................................................................................................18
4.1.2 CMG CNF architecture.....................................................................................................................19
4.2 CMG CNF deployment options...............................................................................................................20
4.3 Platform requirements........................................................................................................................... 21
4.3.1 Infrastructure and host deployment requirements...................................................................... 21
4.3.2 AWS requirements...........................................................................................................................25
4.4 K8s open source components............................................................................................................... 25
5 Lifecycle management............................................................................................................................... 28
5.1 Lifecycle management actions.............................................................................................................. 28
5.2 Container probes.................................................................................................................................... 28
5.3 CMG deployment requirements............................................................................................................. 29
7 Helm charts................................................................................................................................................ 52
7.1 CMG Helm charts.................................................................................................................................... 52
7.2 CMG attributes in the values.yaml file............................................................................................. 55
7.3 CDB Helm chart...................................................................................................................................... 73
7.4 CDB values.yaml file............................................................................................................................... 76
Appendix C: References...........................................................................................................................137
List of Figures
Figure 1: Nokia CNF deployed with the CNF-provided PaaS....................................................................... 13
Figure 2: Nokia CNF deployed with operator-provided CaaS/PaaS on bare metal.................................... 14
Figure 3: Nokia CNF deployed with operator-provided CaaS/PaaS on VM/hypervisor.............................. 14
Figure 4: K8s Cluster CMG CNF (CP).............................................................................................................17
Figure 5: K8s Cluster CMG CNF (UP)............................................................................................................ 18
Figure 6: K8s Cluster CMG ePDG CNF (CP and UP)..................................................................................... 19
Figure 7: CMG CNF deployment on an operator-provided CaaS/PaaS....................................................... 21
Figure 8: CMG container lifecycle management.......................................................................................... 28
Figure 9: CMG CNF networking (CP)............................................................................................................. 35
Figure 10: CMG CNF networking (UP)............................................................................................................. 36
Figure 11: ePDG as CNF networking (CP and UP)..........................................................................................37
List of Tables
Table 1: What’s new in release 22.8.R1..........................................................................................................9
Table 2: CMG cluster type requirements..................................................................................................... 25
Table 3: CMG cluster service requirements (operator-provided CaaS/PaaS)............................................. 26
Table 4: Docker images for deploying CMG CNF components................................................................... 29
Table 5: SMF requirements........................................................................................................................... 31
Table 6: SMF with sidecars requirements.................................................................................................... 31
Table 7: UPF requirements............................................................................................................................32
Table 8: UPF with sidecars requirements.....................................................................................................32
Table 9: Functional testing (no TPS or packet rate expected).................................................................... 33
Table 10: Labs and small traffic (10 Gb/s with DPI and a maximum of 1000 sessions).............................. 33
Table 11: Vault configuration parameters (values.yaml file)......................................................................... 47
Table 12: Log streaming options.................................................................................................................... 50
Table 13: Files of the CMG Helm chart.......................................................................................................... 53
Table 14: Parameters of the values.yaml file.................................................................................................56
Table 15: Files of the CDB Helm chart...........................................................................................................75
Table 16: Parameters of the charts/cdb/values.yaml file............................................................................. 77
Table 17: Acronym definitions and term expansions.................................................................................... 81
Table 18: Support NICs for CMG CNF SR-IOV..............................................................................................135
1 Getting started
Get general information about this guide.
The CMG CNF supports mobile gateway functionality that can be deployed on a generic compute using a
CNF application management platform such as Kubernetes (K8s). The CMG CP CNF supports the SMF and
PGW-C/GGSN-C/SGW-C CP gateway functions. The CMG UP CNF supports the UPF and PGW-U/GGSN-U/
SGW-U UP gateway functions.
Topics include:
Note:
• Configuration outputs shown in this guide are examples and actual displays may differ
depending on the user configuration.
• This guide covers content for the release specified on the title page of the guide, and may
also contain content that will be released in later maintenance loads. Refer to the applicable
7750 SR MG and CMG Release Notes for information about features supported in each load
of the release software.
Audience
This guide is intended for network administrators who are responsible for configuring CMG CP/UP functions
and containerized deployments. It is assumed that the network administrators have an understanding of
the following topics:
Refer also to the guides listed in the CMG Guide to Documentation for information about the software
configuration and the CLI that is used to configure network parameters and services. The CMG Guide to
Documentation includes the 7750 SR configuration guides, which describe SR OS service features that are
supported by the CMG and are mostly used without modification. For the complete list of CMG technical
publications, refer to the CMG Guide to Documentation.
2 What’s new
Discover the new features and enhancements that have been documented in this guide since the
previous publication.
Refer to the 7750 SR MG and CMG Release Notes for additional information about features and
enhancements in the specific releases.
CMG CNF as ePDG Added descriptions of the CMG e CMG CNF as ePDG
PDG CNF and its networking
CMG CNF ePDG networking
NGINX is deployed as a Linux process or a container on nodes with external connectivity. In some
CaaS solutions (like NCS), NGINX may not be deployed on worker nodes but on nodes with external
connectivity assigned role (edge nodes).
• database pods and optional plug-in pods (Grafana, Prometheus, and so on)
• all containers can communicate with each other without using NAT
• all nodes can communicate with all containers (and the other way around) without using NAT
• a container sees itself as the same IP address that other containers see it as
These requirements mean that all pods are able to freely communicate with any other pods in the cluster,
even when they reside in different hosts. A pod identifies another pod using its IP address, because the
underlying host does not exist. The host is also able to communicate with any pod using its own IP address,
without using address translation.
The service mesh is a dedicated infrastructure layer for handling service-to-service communication. It
also allows you to configure how your service instances perform critical actions such as service discovery,
load balancing, data encryption, authentication, and authorization. This is implemented by providing
Istio ingress gateway pods for load balancing across the NRD pods for each service instance, as well as
sidecars within application pods. Sidecars handle inter-service communications, monitoring, security-
related concerns, and anything that can be abstracted away from the individual services.
Istio service mesh is logically split into a DP and a CP. The DP consists of sidecar proxies (such as Envoy),
which mediate and control all the network communication between microservices. The CP consists of the
following components, which are responsible for:
Note: CMG is only supported on an operator-provided CaaS/PaaS that can be deployed on VMs/
hypervisor, or natively without an NFVI layer.
For more information about CMG models, refer to the 7750 SR MG and CMG Release Notes.
4 CMG as CNF
CMG is provided as a CNF and can be deployed over the K8s cluster.
CMG as CNF supports only deployments using an operator-provided CaaS/PaaS that can be deployed on
VMs/hypervisor, or natively without an NFVI layer. For more information about the supported CMG models,
refer to the 7750 SR MG and CMG Release Notes.
Note: Contact a Nokia representative for information about CMG CNF support with other CNF
infrastructure platforms that are built on top of K8s.
• SMF/CP function
• UPF/UP function
• ePDG
Note: Contact a Nokia representative for information about gateway functions that are supported
with the CMG CNF.
The most common method of deploying an application component or instance on a K8s cluster is using
pods. A pod is the most granular method to identify a component of a cloud native application.
A CMG CNF instance consists of multiple pods running on a K8s cluster. Each pod that participates in the
CMG CNF instance is dedicated for a specific function, currently including, OAM, MG, LB, DB proxy, and
Redis DB pods. The specific function for which each CMG pod is dedicated can be replicated across many
similar pods.
A group of pods can operate in synchronization with other similar pod groups in the instance to support a
network function and is represented as a single instance of CMG CNF. The ability to add multiple pods for
each function allows the CMG CNF to scale horizontally to support a range of a few thousand to several
million subscriber devices.
A K8s cluster can be deployed on a generic computing infrastructure on a private or public cloud. The
cluster can be on bare metal (no hypervisor) or inside a VM that could be managed by OpenStack.
Note: Contact a Nokia representative for requirements to deploy CMG CNF on a K8s cluster
running on bare metal (no hypervisor) or inside a VM.
The CMG CNF CP operates in LB mode for the majority of cases. In this mode, all external traffic flows
through the LB function and is distributed to the MG function over the internal network. The CP can also
operate in LB-less mode using GTP-C redirection. The CMG CNF UP operates in either LB-mode or LB-less
mode. CMG CNF deployed as ePDG is supported only in LB-mode.
For more information about the supported deployment models, refer to the 7750 SR MG and CMG Release
Notes.
Note:
• Multus and SR-IOV are used for the DSF and the external network connectivity. DPDK
acceleration is performed on the application level using DPDK poll mode drivers.
• The primary and secondary DSF networks must be configured on different physical ports to
provide redundancy on the link- and switch-level. The same rule applies for the primary and
secondary external networks.
• DB proxy and Redis Pods communication is established through the primary Calico interface.
• The OAM pods require Multus and IPVLAN to use on a secondary interface for management
access (SSH, SFTP, and SNMP).
Note:
• Multus and SR-IOV are used for the DSF and the external network connectivity. DPDK
acceleration is performed on the application level using DPDK poll mode drivers.
• The primary and secondary DSF networks must be configured on different physical ports to
provide redundancy on the link- and switch-level. The same rule applies for the primary and
secondary external networks.
• DB proxy and Redis Pods communication is established through the primary Calico interface.
Note:
• Multus and SR-IOV are used for the DSF and the external network connectivity. DPDK
acceleration is performed on the application level using DPDK poll mode drivers.
• The primary and secondary DSF networks must be configured on different physical ports to
provide redundancy on the link- and switch-level. The same rule applies for the primary and
secondary external networks.
Apart from the OAM container, the OAM pod can also include
the NASC sidecar and the logging containers
DB proxy The DB proxy pod acts as the proxy interface between MG and
DB pods
Note:
• A minimum of two worker nodes are required to deploy a minimum CMG instance.
• SR-IOV CNI (or host device CNI in CN-A) and DPDK are available on the nodes where MG and LB pods
will be deployed
The CMG pods are placed on the appropriate nodes by specifying the nodeSelector which must match the
labeled K8s nodes.
• tunnel support
– net.ipv4*
– net.ipv6*
– net.core*
• net.ipv4.udp_rmem_min = 1048576
• net.ipv4.udp_wmem_min = 1048576
• net.ipv6.conf.all.forwarding = 1
• net.core.rmem_max=4194304
• net.core.wmem_max=4194304
• net.core.rmem_default = 1048576
• net.core.wmem_default = 1048576
– net.core.rmem_max = 1048576
– net.core.wmem_max = 1048576
– net.core.rmem_default = 1048576
– net.core.wmem_default = 1048576
– net.ipv4.udp_rmem_min = 1048576
– net.ipv4.udp_wmem_min = 1048576
• the following setting must be set in the Calico configuration to prevent performance impact on the
CSF network:
• in the case where the OAM pod of CMG needs to be collocated with the CMM NECC pod, add
LimitMSGQUEUE=infinity to the /etc/systemd/ system/containerd.service file; you
must restart the service afterwards
• extra tuning may be required on the underlying CaaS for enabling IPv6 and/or dual stack to enable and
configure IPv6 on the application level
when CMG is deployed on the NCS and IPv6 is required on the application networking, the
Ipv4_dualstack parameter must be enabled on the NCS cluster; refer to the CaaS documentation
for instructions about enabling use of dual stack and IPv6
• in NCS deployments with Mellanox NICs, the SR-IOV offload setting must be set to true to enable the
DPDK functionality
• Nokia recommends setting the kernel core pattern parameter as follows for debugging purposes:
kernel.core_pattern=/var/crash/core.%p
CMG can be deployed on an operator-provided CaaS/PaaS that is running natively or inside a VM. There are
several infrastructure and host requirements, both for VMs carrying pods, as well as for the pods.
A cluster deployed on VMs carrying the pods must meet the following requirements:
• HA must be enabled either by setting availability zones or anti-affinity groups on the VMs
• the cpuManagerPolicy flag in the kubelet configuration must be set to static to enable CPU
pinning on the pod level
• the --reserved-cpus option must be set to reserve cores for system processes
• Hugepages must be enabled for CMG pods using DPDK (Ηugepages1G must be enabled)
• the Kubernetes Topology Manager must be enabled and set to single-numa-node policy to deploy the
CMG pods; this ensures that all CPU cores and NIC resources are allocated from the same NUMA
CMG pods can be deployed either in privileged or restricted mode. If security requires the use of the
restricted mode, specific settings must be defined in the PSP which are aligned with the Pod Security
context settings. The PSP can be created by the CaaS administrator (using the CMG requirements set in the
default PSP file in the Helm charts) or during pod deployment using the default PSP included in the Helm
charts.
The containers included in the CMG pods, can be instantiated running as root or non-root user. If they are
instantiated running as a non-root user, a hard-coded ID is used for the CMG pods.
Networking requirements
Multus manages the multiple network interfaces required on OAM, LB, and MG pods. The CMG has been
tested with the following:
• Calico for native K8s networking, which also provides the network for communication between CMG
pods
Note: Only use a management interface through the IPVLAN interface with Multus. CNF
deployments do not require static route configuration in the BOF. The management network
is configured in the values.yaml file, on the Multus and IPVLAN interface of the OAM.
The MTU value for the interfaces that run K8s Calico CNI traffic or SMF/UPF Multus traffic must be set to
9000.
For pods that require SR-IOV, the network redundancy is handled at the CMG pod-level. The SR-IOV VFs
associated with two redundant ports/PFs are requested for each pod (for each network type).
NUMA alignment is mandatory for pods that require SR-IOV connectivity. When a node has SR-IOV NICs
configured in both NUMAs, the dual MG statefulset feature must be used to place the MG pods in both
NUMAs of the node.
For non SR-IOV-CNI-based interface access (such as Calico, IPVLAN, and so on), the infrastructure must
ensure the redundancy of the physical network being deployed. For example, the IPVLAN interface required
for the OAM management must be based on Linux bonding incorporating two underlying redundant
interfaces.
• SRI-OV VF configured on the host must operate in pass-through mode and attach to the VM hosting
worker node
Helm charts
For CMG deployed on an operator-provided CaaS/PaaS, Nokia provides Helm charts for the supported CNF
functions. The values.yaml files included in the Helm charts must be edited to include custom variables
prior to the CMG installation.
If the CaaS/PaaS is deployed on VMs, the Heat templates must be provided and managed by the
infrastructure administrator.
Related information
Lifecycle management
Helm charts
Nokia provides the necessary Helm charts for managing the CMG CNF deployment. The Helm charts are located
under the CMG package available for download.
DSF networking
For AWS deployments, the subnet information passed via the values.yaml file is ignored. To set up the
DSF network, you must update the values.yaml file as follows.
aws:
enable: 1
region: us-east-2
aws.enable
When set to 1, the CMG application obtains the IP address allocated to that ENI interface
which maps to the DSF port from the AWS API server.
aws.region
The default AWS region
k8s.gcr.io/etcd No dependency
Helm Mandatory
prom/prometheus Optional
CNF provides metric endpoint
Calico Optional
Version 3.9.2 is required for IPv6 addresses
Multus Mandatory
fluentd Optional
jaegertracing/all-in-one Optional
grafana/grafana Optional
k8s.gcr.io/elasticsearch Optional
Used for Elasticsearch
CNF independent of version
docker.elastic.co/kibana/kibana-oss Optional
Used for Elasticsearch
CNF independent of version
GlusterFS Optional
Used for volumes and persistent volumes Version
6.7 is required for IPv6 addresses
k8s.gcr.io/metrics-server-amd64 Optional
Used by K8s HPA and K8s dashboard No CNF
dependency
5 Lifecycle management
• deleting deployments
• healing pods
Note: In the current release, software upgrade and rollback of CMG pods are not supported.
Startup probe Detects when the application within the container has started.
Once configured, all other probes are disabled until the startup
probe succeeds. If no startup probe is provided, the default
state is "Success".
Note: The values included in the configuration provided are the Nokia-recommended values
and are tuned and verified to bring up the pod quickly and monitor it appropriately based on the
application code.
1 The LMG, LOAM, LLB, and logging containers use a common container image (lmg).
efficient
recovery of
session
Note: Contact your local Nokia representative for information about VM and pod resource
requirements for VM-based containerized CMG in the current release.
OAM 6 8
MG 8 64
LB 6 16
OAM loam 6 8 Gi
nasc 2 1 Gi
MG — 8 64 Gi
LB — 6 16 Gi
OAM 6 8
MG 8 64
LB 6 16
OAM loam 6 8 Gi
nasc 2 1 Gi
MG — 8 64 Gi
LB — 6 16 Gi
OAM 4 16
LB 4 16
MG 8 32
DB proxy 4 8
DB pod 2 4
Table 10: Labs and small traffic (10 Gb/s with DPI and a maximum of 1000 sessions)
OAM 4 16
LB 4 16
MG 16 48
Table 10: Labs and small traffic (10 Gb/s with DPI and a maximum of 1000 sessions) (continued)
DB proxy 4 8
DB pod 2 4
Management network (OAM only) The management network provides external network
connectivity for CMG management; a dedicated interface is
needed
CSF network The CSF network provides the internal network for CMG CNF
control messaging (discovery, configurations, and status)
among the CMG CNF pods
DSF internal network The DSF network provides the user signaling and user traffic
between pods and is connected to all MG and LB pods
The CMG CNF requires independent interfaces for internal and external networks; therefore, multiple
interface support is mandatory. By default, K8s assigns a single interface per pod; however CNI plug-ins
such as Multus support deployment of multiple interfaces per pod.
CMG CNF networking (CP) shows the internal and external network connections on the CMG CNF CP. The
internal network for OAM, MG, LB, and DB is enabled over a K8s network. The external network connection
requires multiple interfaces on the LB function. These interfaces can be configured using the K8s CNI plug-
in Multus, which is supported for managing multiple interfaces that manage multiple K8s CNIs.
For redundancy, dual DSFs are supported from the application side.
Nokia recommends using pod-level VLAN tagging (instead of host-level) to reduce the number of required
interfaces on the pod-level and avoid complexity when assigning interfaces.
To ensure that dual DSF have SR-IOV interfaces allocated from different physical NIC ports and dual
external interfaces also have SR-IOV interfaces allocated from different physical NIC ports, a new section
is added to the DPDK section of the Helm charts. This section (portOrder), allows you to map each pod
interface to a specific NIC interface.
CMG CNF networking (UP) shows the internal and external network connections on a CMG CNF UP. The
internal network for the OAM, MG, LB, and DB functions is enabled over a network provided by K8s. The
external network connection requires multiple interfaces on the LB function and MG function (for LB-less
deployment). These can be configured using the Kubernetes CNI plug-in Multus, which is supported for
managing multiple interfaces that manage multiple K8s CNIs.
The same restrictions for the DSF and external network interface assignments described in CMG CNF
networking (CP) also apply to CNF UP networking.
ePDG as CNF networking (CP and UP) shows the internal and external network connections on the CMG
ePDG CNF (CP and UP). The internal network for OAM, MG, LB, and DB is enabled over a K8s network. The
external network connection requires multiple interfaces on the LB function. These interfaces can be
configured using the K8s CNI plug-in Multus, which is supported for managing multiple interfaces that
manage multiple K8s CNIs.
For redundancy, dual DSFs are supported from the application side.
Nokia recommends using pod-level VLAN tagging (instead of host-level) to reduce the number of required
interfaces on the pod-level and avoid complexity when assigning interfaces.
To ensure that dual DSF have SR-IOV interfaces allocated from different physical NIC ports and dual
external interfaces also have SR-IOV interfaces allocated from different physical NIC ports, a new section
is added to the DPDK section of the Helm charts. This section (portOrder), allows you to map each pod
interface to a specific NIC interface.
The object specifications for the different K8s objects are defined in the manifest in the CMG Helm charts.
The objects can be customized with deployment-specific values using the charts/cmg/values.yaml file
which is common to CMG CNF as CP and UP, or using CP- and UP-specific values defined in the charts/
cmg/smf_values/ and charts/cmg/upf_values/ YAML files respectively.
Related information
Helm charts
Nokia provides the necessary Helm charts for managing the CMG CNF deployment. The Helm charts are located
under the CMG package available for download.
Product Support Portal
CAUTION: Nokia recommends using any of the offered K8s volume types, apart from the
hostPath and local storage volume types. hostPath volumes pose security risks while local storage
exposes the risk that data can be lost if the CMG pod re-spawns for any reason on another node.
• If CMG pods are expected to run in restricted mode, a proper pod security policy must be configured
in CaaS and claimed through the Helm charts
• a network policy is a specification of how groups of pods are allowed to communicate with each other
and other network endpoints
If the network policy is configured on the CaaS to provide network isolation, a specific network policy
must be configured using appropriate labels in namespaces to allow communication between the CMG
and CMG-DB.
Note:
• If you want to deploy MG pods in both NUMAs of a server that has SR-IOV capable NICs
in both NUMAs, use the dualMGstatefulesets attribute in the values.yaml file.
This option splits the MG pods into two groups; one deployed in NUMA-0 and the other in
NUMA-1.
Procedure
Sub-steps
Create different namespaces for SMF and the UPF. If network policies must be defined, you must
add labels to the namespaces to be used in the network policy.
Step example
K8s
Step example
OpenShift
oc new-project smf
oc new-project upf
For the SMF namespace, create the Role and RoleBindings to allow privileged pods to run. This is
done as part of the helm install command by setting the openshift.enable parameter to
true.
Note: For the UPF namespace, similar Role and RoleBindings must be created.
b) Create the namespaces to deploy the DB resources for the SMF and, or UPF CNFs.
Step example
2. Extract the CMG tar file and upload the Docker images.
Sub-steps
Tag the image with the host name or IP address and the port of the registry.
Step example
Step example
For OpenShift deployments, upload the SMF container image (for example, lmg_12.0_R1.tar)
to OpenShift. Then tag it and push it to the image registry.
Step example
Note: Perform the same steps, if required, for the UPF container image.
Depending your deployment, you can install any of the SMF and, or UPF Helm charts.
Sub-steps
Use the helm install command to deploy the SMF. The Helm chart must be available in the
charts directory and all the commands in this step must be executed in the same directory.
Note: Nokia recommends including all the necessary changes in the values.yaml file,
to avoid errors when using the helm install command.
Step example
If the container images are stored in a private registry and, or repository, you must configure the
secret to access the repository and pull the images. To configure the secret, add the following
argument in the helm install command:
Note: In OpenShift deployments, the helm install command is similar. Use the
appropriate image registry, image tag, namespace and, or project, and so on.
Use the helm install command to deploy the UPF. The Helm chart must be available in the
charts directory and all the commands in this step must be executed in the same directory.
Step example
STATUS: deployed
REVISION: 1
TEST SUITE: None
If the container images are stored in a private registry and, or repository, you must configure the
secret to access the repository and pull the images. To configure the secret, add the following
argument in the helm install command:
Note: In OpenShift deployments, the helm install command is similar. Use the
appropriate image registry, image tag, namespace and, or project, and so on.
Sub-steps
Note: A unique NodePort must be used when using the same CDB Helm charts for SMF
and UPF.
Step example
helm list -A
helm list -n <namespace>
Step example
Step example
Step example
Note: A unique NodePort must be used when using the same CDB Helm charts for CP
and UP.
Step example
helm list -A
helm list -n <namespace>
Step example
Step example
Sub-steps
Verify the SMF DB resources in the smf-cdb namepace or in the namespace that was created to
deploy the DB resources.
Step example
Verify the UPF DB resources in the upf-cdb namepace or the namespace that was created to
deploy the DB resources.
Step example
Related information
For information about CMG troubleshooting, refer to the CMG and CMG-a Troubleshooting Guide.
To use Vault, CMG CNF must be configured during deployment with the following information in the
values.yaml file (within the CMG Helm chart):
vault:
enable: 1
name: <k8s service name|domain name>
port: <port>
basePath: <absolute path in vault>
adminKeyRpath: <relative path in vault>
tlsCaCert: </path/to/ca-certificate.cert>
For more information about Vault and what information is stored to Vault, refer to the section Secure
Storage in the 7750 SR MG and CMG Configuration Guide.
If it is left empty, no CA
certificate is used for validating
Authentication to Vault
The CMG CNF instance authentication to Vault is handled automatically using the K8s authentication
method. K8s provides CMG with a service account and a token to be used toward Vault for authentication.
K8s also provides the TokenReview API which Vault uses to authenticate a client connecting to it.
For more information about the use of Prometheus, refer to the section Prometheus Metrics in the
7750 SR MG and CMG KPI-KCI Counters Guide.
6.2 Logging
CMG CNF supports log streaming to external PaaS components like FluentD. The logging container is
used to redirect the logging data to stdout, so that they are stored and processed by the standard K8s
logging architecture, or other PaaS components like FluentD.
Alternatively, FluentBit and LogSplitter can be deployed as a sidecar container to the OAM pod. This is used
for log filtering and streaming to different components like FluentD, Kafka broker, and so on.
Alternatively, the logging functionality can be configured to tag a subset of logging data as FM data.
The ‘_fm_’ string is prefixed to log records that are configured to be tagged as FM data. The tagging
configuration is performed by using the cmg_alarms.csv file into the configmap of the logsplitter
container. Each line in the CSV file identifies a log event from the log events listed in the 7750 SR MG and
CMG Log Events Guide, that is tagged as an FM event.
application-name;severity;event-id;paylod-match
where:
paylod-match An arbitrary string which must match the message of the event
log, as defined in the message format string
For the application name, severity, event ID, and message format string of the supported event logs, refer
to the 7750 SR MG and CMG Log Events Guide.
1. application name
The CMG CNF package includes a default configuration file. When configuring a custom set of logs to be
tagged as FM data, make sure that both the raising event and at least one clearing event of an alarm are
included in the CSV file.
3. Apply the configmap to the running pod or to the pod specifications prior to deployment.
All event-logs to stdout with FM logs being tagged LogSplitter container with the following:
All event-logs streamed out from FluentBit with FM • Deploy LogSplitter container with the
logs streamed to different destination following:
For example, streaming FM to kafka and the rest to – OUTPUT_DIR variable set
FluentD, which is the default configmap for the Log
– shared-logs volumeMount configured
Splitter container
in the container specifications in the
deployment.yaml file
BGP;;2032;
CHASSIS;;2016;1.3.6.1.4.1.6527.3.1.3.2.1.0.7
MOBILE_GATEWAY;;2001;Peer State: pathRestart
NTP;critical;;
7 Helm charts
Nokia provides the necessary Helm charts for managing the CMG CNF deployment. The Helm charts are
located under the CMG package available for download.
The CMG CNF deployment includes the following Helm charts:
The following sections describe the folder structure, contents, and guidelines to modify the Helm chart
templates using the values.yaml file.
Related information
Folder structure
| CMG <release-tag>
|-- <helm charts folder>
+-- cmg
|-- Chart.yaml
|-- values.yaml
|-- license.txt
|-- templates
|-- AWS_ConfigMap.yaml
|-- Card_ConfigMap.yaml
|-- ClusterRole.yaml
|-- ClusterRoleBinding.yaml
|-- CmgAlarms_ConfigMap.yaml
|-- Connectivity_Service.yaml
|-- Deployment.yaml
|-- Dut_ConfigMap.yaml
|-- Endpoints.yaml
|-- FluentBit_ConfigMap.yaml
|-- GlusterFS_Service.yaml
|-- HorizontalPodAutoscaler.yaml
|-- Internal_Service.yaml
|-- License_ConfigMap.yaml
|-- Nasc_ConfigMap.yaml
|-- NetworkAttachmentDefinition.yaml
|-- PersistentVolume.yaml
|-- PersistentVolumeClaim.yaml
|-- PriortyClass.yaml
|-- PodSecurityPolicy.yaml
|-- Role.yaml
|-- RoleBinding.yaml
|-- SecurityContextConstraints.yaml
|-- ServiceAccount.yaml
|-- SriovNetwork.yaml
|-- StatefulSet.yaml
|-- Vault_Tls_cert.yaml
Contents
2 Applies only when the logsplit sidecar container is enabled in the LOAM pods.
Note: For more information about the K8s-related content, refer to the official K8s
documentation.
5 Apllies only when the priorityclass parameter is defined in the values.yaml file.
6 Applies only to an OpenShift cluster and is executed by setting the openshift.enable=true
parameter during the CMG Helm installation process.
7 Applies only to an OpenShift cluster and is executed by setting the openshift.enable=true
and openshift.sriovOperatorEnable=true parameters during the CMG Helm installation
process.
The parameters marked as optional may be missing from default the values.yaml file. You can add these
parameters manually if needed.
Parameter Description
baseSlotNum.SetTwo
resources.setOne.multus.resourcename
8 The minReplicas and maxReplicas parameters must be configured with the same value because
in the current release the HorizontalPodAutoscaler parameter for scale-out and scale-in of the
MG pods is not supported.
Parameter Description
resources.setOne.multus.numDevices
resources.setTwo.multus.resourcename
resources.setTwo.multus.numDevices
setOneScale.minReplicas
setOneScale.maxReplicas
setOneScale.targetCPUUtilization
Percentage
setTwoScale.minReplicas
setTwoScale.maxReplicas
setTwoScale.targetCPUUtilization
Percentage
service.loamA.console.targetPort
service.loamB.console.nodePort
service.loamB.console.port
service.loamB.console.targetPort
service.lmg.console.nodePort
service.lmg.console.port
Parameter Description
service.lmg.console.targetPort
service.llb.console.nodePort
service.llb.console.port
service.llb.console.targetport
Default: 2222
image.name
image.tag
image.pullPolicy
imagePullSecrets Optional
Example:
imagePullSecrets:
- name: privateRegSecret
9 OAM and MG use the same image as specified by the image.name attribute.
Parameter Description
nasc.scrapeInterval.loam.kpiInfo
nasc.scrapeInterval.lmg.kciInfo
nasc.scrapeInterval.lmg.kpiInfo
nasc.externalLabels
logging.imageTag
logsplit.imagePullPolicy
logsplit.imageRepository
Parameter Description
logsplit.imageName
logsplit.imageTag
logsplit.imagePullPolicy
fluentbit.imageRepository
fluentbit.imageName
fluentbit.imageTag
fluentbit.imagePullPolicy
kafka.brokerEndpoint
kafka.topic
fluentbit.host
fluentbit.port
fluentbit.tag
multus.loam.gw
Parameter Description
multus.loam.hostInterface
multus.loam.cniVersion
multus.llb.netNames • lmg.reosurceName
• lmgenvName
multus.llb.resourceName
multus.llb.envName
Parameter Description
• lmg.netNames
multus.dpdk.configVlan
• lmg.reosurceName
• lmgenvName
Values:
• smf
• upf
lmgScale.targetCPUUtilization • lmg.netNames
Percentage
• lmg.reosurceName
• lmgenvName
10 The minReplicas and maxReplicas parameter values must be equal because the HorizontalPod
Autoscalar parameter for scale-out and scale-in of the MG pods is not fully supported.
Parameter Description
llbScale.maxReplicas
llbScale.targetCPUUtilization
Percentage
resources.loam.cpu CPU and memory requests and limits for OAM, MG,
and LB containers
resources.lmg.memory
resources.lmg.memory.hugepages1Gi
resources.lmg.multus
resources.llb.cpu
resources.llb.memory
resources.llb.memory.hugepages1Gi
resources.llb.multus
resources.nasc.cpu CPU and memory requests and limits for the NASC
container
Parameter Description
resources.logsplit.cpu CPU and memory requests and limits for the log
splitting and fluentBit containers
resources.fluentBit.cpu
resources.fluentBit.memory
storage.pvStorageClass
storage.pvLogsClaimName
storage.pvSize
storage.cfSize
storage.cfAInfo
storage.cfBInfo
Parameter Description
podsecuritypolicy.privileged Optional
Default: false
podsecuritypolicy.runAsNonRoot Optional
Default: true
Parameter Description
AntiAffinity.external Optional
Example:
external:
type: hard
lmg:
- label:
key: name
value: SBC
namespace:SBC
Parameter Description
Antiaffinity.dualMgStatefulsets Optional
Values:
bootstring.lmg.cpcores Optional
Parameter Description
bootstring.lmg.cfp Optional
bootstring.llb.cpcores Optional
bootstring.llb.cfp Optional
13 Optional parameter.
Parameter Description
peers.cdbx.port
peers.cdbx.interface
peers.nrf.port
peers.nrf.interface
peers.nrf.uuid
peers.upf.interface
peers.gx.interface
peers.rf.interface
Parameter Description
slice
uuid
network.interface
network.staticRoute
network.bgp
apn
uepool
nodeSelector.lmg
nodeSelector:
loamA:
nodeSelector.llb - key: key1
value: value1
loamB:
- key: key2
value: value2
lmg:
- key: key3
value: value3
llb:
- key: key4
Parameter Description
value: value4
tolerations.lmg
tolerations:
#pod type#:
tolerations.llb - key: key1
value: value1
operator: operator1
effect: effect1
tolerationSeconds: toleration
Seconds1
Optional
Default: 9000
Optional parameter
Default: 3
Optional parameter
Default: 1
k8DualStack Optional
Parameter Description
Enables the K8s dual stack within the CMG CNF and
must be set to true to use IPv6 addressing
Optional parameter
Default: 1
Optional parameter
Default: 17
Default: aws.enable=1
aws.region
For more information, see AWS requirements
Parameter Description
Default: false
bootstring.msmStatsPoll Optional
Note:
– antiAffinity.loam=hard
– antiAffinity.lmg=hard
– antiAffinity.llb=hard
– antiAffinity.loamLmg=hard
Setting the MG anti-affinity to hard can lead to high number of hardware resources
in some cases. Alternatively, soft anti-affinity can be used, assuming impact is
acknowledged.
• The values_xdp.yaml file must not be used for live deployments. Only DPDK mode is
supported.
NAME: smf-cdb
LAST DEPLOYED: Wed May 19 15:29:10 2021
NAMESPACE: smf-cdb
STATUS: deployed
REVISION: 1
TEST SUITE: None
Note: A unique NodePort must be used when using the same CDB Helm charts for CP and UP.
Folder structure
| CMG <release-tag>
|-- charts
+-- cdb
|-- Chart.yaml
|-- values.yaml
|-- values_multus.yaml
|-- values_ncs.yaml
|-- templates
|-- ClusterRole.yaml
|-- ClusterRoleBinding.yaml
|-- Connectivity_Service.yaml
|-- Dbproxy_ConfigMap.yaml
|-- Deployment.yaml
|-- HorizontalPodAutoscaler.yaml
|-- Internal_Service.yaml
|-- NetworkAttachmentDefinition.yaml
|-- PodSecurityPolicy.yaml
|-- RoleBinding.yaml
|-- Role.yaml
|-- SecurityContextConstraints.yaml
|-- SrioNetwork.yaml
|-- StatefulSet.yaml
Contents
15 These resources are created only when the Values.podsecuritypolicy.create attribute is set
to true.
Note:
• For more information about the K8s-related content, refer to the official K8s documentation.
• Nokia recommends editing the sample YAML files. Refer to the parameters set in those files
and make the necessary changes to the main values.yaml file.
Parameter Description
service.dbproxy.Port
service.dbproxy.targetPort
service.redis.port Sets the port for the K8s service and containerPort
for the Redis DB container
image.dbproxy.name
image.dbproxy.tag
image.dbproxy.pullPolicy
image.redis.name
image.redis.tag
image.redis.pullPolicy
Parameter Description
redisScale.targetCPUUtilization
Percentage
resources.dbproxy.memory
resources.redis.memory
Example:
multus.dbproxy.networkInfo
networkInfo:
multus.dbproxy.attachDef ip: 192.168.1.104
mask: 24
multus.groFlag mtu: 9000
vlan: 1095
Parameter Description
AntiAffinity.external Optional
Example:
external:
type: hard
dbproxy:
- label:
key: name
value: SBC
namespace:SBC
podsecuritypolicy.privileged Optional
Default: false
podsecuritypolicy.runAsNonRoot Optional
Parameter Description
Default: true
imagePullSecrets Optional
Example:
imagePullSecrets:
- name: privateRegSecret
Numbers
5GS 5G System
AA Application Assurance
AC Access Concentrator
Air Conditioned
ACK Acknowledge
AF Application Function
AN Access Network
AS Access Stratum
BD Billing Domain
BP Branching Point
CA Certification Authority
CC Charging Characteristics
Content of Communication
CF Compact Flash
CN Core Network
Change of Authorization
CP Control Plane
CT Call Trace
DB Database
DB-VM Database VM
DF Delivery Function
DID Domain ID
DL Downlink
DM Disconnect Message
DN Data Network
DP Data Plane
EPS Bearer ID
EH Extension Headers
EP Entry Point
FH Failure Handling
FM Fault Management
FP Fast Path
GBR QoS Flow A QoS flow using the GBR resource type or the
delay-critical GBR resource type and requiring
GFBR
HA High Availability
Home Agent
HR Home-Routed
HW Hardware
IA Identity Association
IE Information Element
IP Multimedia Subsystem
I/O Input/Output
IP Internet Protocol
IP-CAN IP-Connectivity AN
KVM Kernel-based VM
LA Location Area
LI Lawful Interception
ME Mobile Equipment
MEI ME Identity
MF Matched Filter
MG Mobile Gateway
See MSCP
MO Mobile Originated
MP Multi-Path
MR Mobile Router
MS Mobile Station
MT Mobile Terminated
NA Neighbor Advertisement
Non-temporary Address
NBR Neighbor
ND Neighbor Discovery
Next Generation RAN A RAN that supports one or more of the following
options with the common characteristic that it
connects to 5GC:
• Standalone NR
• Standalone E-UTRA
NF Network Function
NI Network Indicator/Identity
Non-GBR QoS Flow A QoS flow using the non-GBR resource type and
not requiring guaranteed flow bit rate
NR New Radio
NS Neighbor Solicitation
Network Slice
OI Operator Identifier/Interface/Integration
OS Operating System
OTT Over-The-Top
PD Prefix Delegation
PDU Session Type The type of PDU session, which can be IPv4, IPv6,
IPv4v6, Ethernet, or unstructured
PF Physical Function
PM Performance Management
PTT Push-To-Talk
PV Persistent Volume
RA Router Advertisement
RF Rating Function
RG Rating Group
Residential Gateway
RS Router Solicitation
RU Rack Unit
Rx Receive
SD Service Data
SM Session Management
SR Service Router
TA Tracking Area
TC Traffic Class
TE Terminal Equipment
TF Triggering Function
Tx Transmit
UE User Equipment
UL Uplink
UL CL Uplink Classifier
UP User Plane
UPF Service Area The area within which a PDU session associated
with the UPF can be served by (R)AN nodes via
an N3 interface between the (R)AN and the UPF,
without requiring to add a new UPF in between, or
to remove or re-allocate the UPF
VF Virtual Function
VIP Virtual IP
VM Virtual Machine
A MANO component responsible for managing the
NFV infrastructure including compute, storage, and
network resources
WT Worker Task
Supported controllers Supported speed (Gb/s) Support status in CMG Minimum tested host
CNF release version
Version: 5.1.0-k-rh7.5
Firmware version:
0x61c10001
Version: 2.1.14-k
Version: 5.0-0
Version: 5.0-0
Version: 5.0-0
Note:
• CMG CNF qualification for the listed controllers for SR-IOV is performed using RHEL/CentOS
as the host OS and with the corresponding inbox SR-IOV drivers.
• All Mellanox NICs of the same family (NICs that share the same vendor and device ID) are
expected to use the same software and firmware drivers, providing SR-IOV compatibility and
support for CMG CNF deployments.
• CMG CNF does not restrict the firmware and, or software version of a supported NIC. The
minimum versions are listed to avoid using old firmware and software drivers, but newer
versions are expected to work, unless stated otherwise in the guide or release notes.
Appendix C: References
1. Calico: https://www.projectcalico.org
2. Ceph: https://ceph.io
3. etcd: https://etcd.io
4. Grafana: https://grafana.com
5. Helm: https://helm.sh
6. Istio: https://opensource.google.com/projects/istio
7. Kubernetes: https://kubernetes.io
8. NGINX: https://www.nginx.com
9. MariaDB: https://www.mariadb.org