An Analysis of Container-based Platforms for NFV

Sriram Natarajan, Deutsche Telekom Inc.

Ramki Krishnan, Dell Inc.
Anoop Ghanwani, Dell Inc.
Dilip Krishnaswamy, IBM Research
Peter Willis, BT Plc
Ashay Chaudhary, Verizon

1
 Virtual Machine vs. Container Stack
KVM Container-stack
VNF

Libraries VNF

Guest-OS Libraries
Hypervisor Container Engine
Host-OS Host-OS

• Lightweight footprint: Very small

• Deployment time:
images with API-based control to
Rapidly deploy
automate the management of services Pod
Container B (container
applications with minimal
Container A
(Application (Application group) A run-time requirements
• Resource Overhead: Lower use of + + (Application
system resources (CPU, memory, etc.) Libraries) Libraries) +
Libraries)
• Updates: Depending on
by eliminating hypervisor & guest OS
requirements, updates,
overhead
Container Engine failures or scaling apps
can be achieved by
scaling containers
Kernel Functions and Modules: up/down
Namespaces, cgroups, capabilities, chroot, SELinux
Host-OS
2
VM based Network Functions
Key Challenges

3
 Service Agility/Performance
• Provisioning time: VNF VNF VNF
– Hypervisor configuration Libraries Libraries Libraries
– Spin-up guest OS
Guest-OS Guest-OS Guest-OS
– Align dependencies between Guest-OS
& VNFs Hypervisor

Host-OS

• Runtime performance overhead:

– Performance proportional to resource allocated to individual VMs (throughput,
line rate, concurrent sessions, etc.)
– Overhead stems from components other than VNF process (e.g. guest OS)
– Need for inter-VM networking solution
– Meeting SLAs requires dynamic fine tuning or instantiation of additive features,
which is complex in a VM environment

4
 Portability/ Elasticity/Scalability
• Porting VNFs require:
– Identifying suitable nodes for new VNF
instances (or re-locating existing
instances). For example, resource types, VNF VNF
available capacity, guest OS images,
Libraries Libraries
hypervisor configs, HW/SW accelerators,
etc.) Guest-OS
Same
Guest-OS
– Allocating required resources for new
Hypervisor
instances Hypervisor
Re-config
– Provisioning configs to components in the Host-OS (vCPU, RAM,
guest OS, libraries and VNF Host-OS
SSL accelerator)

• Elastic scalability needs are driven by

workloads on the VNF instances, and
stateful VNFs increase the latency to
spin up new instances to fully
working state.
5
 Security/Isolation

VNF
VNF VNF VNF ✗ If VNF is compromised
Securely recover
with minimal or no Libraries Libraries Libraries (misconfiguration,
downtime etc.), how to securely
Guest-OS Guest-OS Guest-OS
(reschedule VNF) quarantine the VNF,
Hypervisor
but ensure continuity
of other VNFs?
Host-OS

Guarantee complete isolation across Resource hungry VNF can starve the
resource entities (hardware units, shared resources (noisy neighbor
hypervisor, protection of shared effect) that are allocated to other VNFs;
resource, isolation of virtual networks, Need to monitor and cut-off hungry
L3 cache, QPI, etc.) VNF usage

6
 Containerized Network Functions
Key Benefits, Challenges and Potential Solutions

7
Service Agility/Performance/Isolation (1)

Key Benefits:
VNF VNF VNF
C B A - Containers can provide better
service agility (e.g. dynamically
Container Engine provision VNFs for offering on-
demand services), and performance
Host-OS as it allows us to run the VNF process
directly in the host environment
Cluster
Management VNF
- Inter-VNF communication latency
VNF
E depends on inter-process
Tool D
communication option (when hosted
Scheduler
Container Engine in the same host)
Host-OS

8
Service Agility/Performance/Isolation (2)
Key Challenges:
- Isolation: Containers create a slice of
VNF VNF VNF the underlying host using techniques
C B A like namespaces, cgroups, chroot etc.;
Container Engine
several other kernel features that are
not completely isolated.
Host-OS - Resource Mgmt: Containers do not
provide a mechanism to quota manage
the resources and hence susceptible to
Cluster the “noisy neighbor” challenge.
Management VNF VNF
Tool D E Potential Solutions:
Container Engine - Kernel Security Modules: SElinux,
Scheduler AppArmor
Host-OS - Resource Mgmt: Kubernetes
- Platform Awareness: ClearLinux

9
 Elasticity & Resilience
Key Benefits:
VNF VNF VNF
Pod Pod Pod
- Auto-scaling VNFs or achieving
Container Engine service elasticity in runtime can be
simplified by the use of container
Replication Host-OS
based VNFs due to the lightweight
Controller
resource usage of containers (e.g.
Cluster Mesosphere/Kubernetes)
Management
Tool
VNF
Pod
VNF
Pod
VNF
Pod - Container management solutions
(e.g. Kubernetes) provide self-healing
Scheduler
Container Engine features such as auto-placement,
restart, and replacement by using
Host-OS service discovery and continuous
monitoring

10
 Operations & Management

VNF VNF VNF

Service
Pod Pod Pod Key Challenges:
Discovery Container Engine
- Containers are supported in
selective operating systems such as
Replication Host-OS Linux, Windows and Solaris
Controller - In the current range of VNFs, many
Cluster don’t support Linux OS or other OSes
such as Windows and Solaris
Management
VNF VNF VNF
Tool Pod Pod Pod
Potential Solutions:
Scheduler Container Engine - Hybrid deployment with VMs and
containers can be envisioned, e.g.
Host-OS leverage ideas from Aptible
Security technology currently used for
applications

11
Conclusion and Future Work

12
 Conclusion and Future Work
• Use of containers for VNFs appears to have significant
advantages compared to using VMs and hypervisors,
especially for efficiency and performance
– “Virtual Customer CPE Container Performance White Paper,”
http://info.ixiacom.com/rs/098-FRB-840/images/Calsoft-Labs-CaseStudy2015.pdf
• Test Setup:
– COTS server with Intel Xeon E5-2680 v2 processor
– Virtual CPE VNFs (Firewall etc.) fast path optimized using Intel DPDK
– Measured L2-L3 TCP traffic throughput per core
• VM (KVM) environment with SRIOV -- 5.8Gbps
• Containers (LXC) environment -- 7.2Gbps
– ~25% PERFROMANCE IMPROVEMENT OVER VMs
• Opportunistic areas for future work
– Distributed micro-service network functions
– VNF Controller discovery/management/etc. standardization
– etc.

13
 Call for Action
• Address aforementioned challenges
• Further research to identify currently unknown challenges
• Vendors to consider developing container based solutions –
especially to support proof of concepts and field trials
• Reach consensus on a common framework for use of
containers for NFV
• Field trial container-based VNFs

14