Flexible 4G/5G Testbed Setup For Mobile Edge Computing Using O A I and O S Mano
Flexible 4G/5G Testbed Setup For Mobile Edge Computing Using O A I and O S Mano
Thomas Dreibholz
Simula Metropolitan Centre for Digital Engineering
c/o OsloMet – storbyuniversitetet
Pilestredet 52, 0167 Oslo, Norway
[email protected]
1 Introduction
Clearly, the fifth generation of mobile broadband networks (5G) is the next ma-
jor step in the evolution of mobile communications. In the very near future, 5G
networks will be the base of all kinds of advanced applications, from high-speed
mobile media streaming over low-latency vehicular communication and control of
Unmanned Aerial Vehicles (UAV) to energy-efficient Internet of Things (IoT) com-
munication and reliability-critical emergency networks. Currently, a significant ef-
fort by both, industry and academic research, is made to design and develop all
the components of 5G networks. However, the current Long-Term Evolution (LTE,
4G), which is the basis of the developing 5G standards, already has a high complex-
ity. Even setting up a small testbed setup already requires a significant amount of
software to be deployed, and expensive proprietary hardware to be purchased and
installed. Therefore, the goal of the O PENA IR I NTERFACE consortium is the create
an open source software and hardware base to realise a full 4G setup. Based on this,
development towards 5G is made.
O PENA IR I NTERFACE is a large consortium, with many very active members
– from industry and academia – contributing to the enhancements of its software.
Therefore, it clearly reflects the state-of-the-art in 5G development. Also, this makes
it a very good foundation to develop further extensions on. Particularly, this means to
combine it with extensions for Mobile Edge Computing (MEC) services: having an
own 4G/5G mobile network testbed setup, it is possible to develop, deploy and test
own MEC services for research and evaluation purposes. Since O PENA IR I NTER -
FACE is free, open source and running on of-the-shelf hardware, it allows people
to get involved into MEC development without high investments into proprietary
equipment and software.
However, setting up O PENA IR I NTERFACE [10] is not a simple and easy task.
Even the most basic 4G/5G testbed setup requires a significant complexity of de-
ploying, configuring and monitoring the underlying software components. Keeping
track with the ongoing fast development pace of O PENA IR I NTERFACE makes this
even more difficult. Therefore, for simplifying and automating such testbed setups,
we have developed an open source Virtual Network Function (VNF) for an Evolved
Packet Core (EPC) based on O PENA IR I NTERFACE. This EPC VNF can be deployed
easily by O PEN S OURCE MANO (OSM) in a cloud setup (e.g. based on O PEN -
S TACK). Using our VNF as network basis, it is therefore straightforward to plug
it together with further functionalities for MEC, in order to build an advanced test
network for MEC-based services and applications. In this paper, we introduce our
VNF and show a basic proof-of-concept evaluation.
2 Software
Before we introduce our EPC VNF, we briefly explain the underlying software.
Since EPC, O PENA IR I NTERFACE and OSM make heavy usage of abbreviations,
we will shortly introduce the basic ones here. This will particularly also be helpful
for the reader to understand the referenced detailed documentations.
Fig. 2 Virtual Network Function Descriptor (VNFD) and Network Service Descriptor (NSD)
In the following, we describe our EPC VNF, denoted as SimulaMet OAI EPC, its
configuration with J UJU, and its usage as part of an NS.
The VNFD of the SimulaMet OAI EPC VNF is illustrated in Figure 3. Clearly, it
consists of a VDU for each of the four OAI EPC parts introduced in Subsection 2.1
and Figure 1: HSS, MME, SPGW-C and SPGW-U. The interfaces S6a, S11 and
SXab are realised by VLs. S1-C, S1-U and SGI are attached to external CPs. In
4 J UJU: https://jaas.ai.
an NSD, they can be attached to e.g. physical interfaces or VLs connecting them
to other VNFs (e.g. to add MEC components). Our VNF uses an U BUNTU 18.04
“Bionic Beaver” operating system image; the interface names ens3, . . . , ens6 are
the interface names in the virtualised Linux systems.
Instantiating our VNF requires to define an NSD (in the simplest case: just con-
sisting of the VNF and attaching it to physical interfaces and a management net-
work for the J UJU Charms) and provide the parameters listed in Table 1. In par-
ticular, these parameters provide the information about the mobile network to be
created, i.e. realm, Mobile Country Code (MCC), Mobile Network Code (MNC),
Operator Code (OP), Subscriber Key (K), as well as the first user’s International
Mobile Subscriber Identity (IMSI) and Mobile Station International Subscriber Di-
rectory Number (MSISDN). IMSI and MSISDN are incremented for further users.
The J UJU Charms of the VNF will create the given number of users and subscrip-
tions according to these details; simcards for the new mobile network have to match
these settings. Further parameters include the addressing of the components and the
addresses of the Domain Name Servers (DNS) to be used.
OAI is under heavy development, and working with branches and forks of the
OAI sources is an essential part of development and tests with OAI. Therefore, the
J UJU Charms of our VNF not just deploy pre-built variants of HSS, MME, SPGW-
C and SPGW-U. Instead, the parametrisation allows to specify the corresponding
G IT repository and tag for each of the four components. During day-1 configu-
ration, each component is therefore directly compiled from its sources inside the
corresponding VDU. For this purpose, the operating system image already has de-
velopment packages, e.g. compilers, libraries, etc., preinstalled. A script is provided
to create this image from scratch, by downloading and automatically installing a
preseeded U BUNTU Linux. Since the VDUs need Internet access for the build (e.g.
to fetch the sources and dependencies), it is necessary to allow Internet access over
the management network.
While the management network provides Internet access for package and source
downloads during day-1 and day-2 configuration, it is not desirable that the PDN
access of the mobile network goes over the management network as well. Therefore,
the SPGW-U Charm also deploys a routing rule setup for the SPGW-U VDU: the
OAI SPGW-U configures a dummy device pdn. Network/port address translation
is then used for forwarding all PDN traffic via the public IPv4 address of the SGi
interface ens6. Our SPGW-U Charm configures a new routing rule for all traffic
from pdn: all user traffic is routed via the SGi interface ens6 to the PDN, without
routes for the internal networks. So, UE is only able to access the SGi network and
the PDN, but unable to interfere with the internal networks.
As last step of the day-1 configuration, the J UJU Charm of each component sets
up a SYSTEMD unit configuration for the corresponding service. This service is then
automatically started, i.e. once the NS is fully configured, all components should be
running, and the new mobile network is ready for attaching eNodeBs.
The SimulaMet OAI EPC VNFD sources, together with example NSDs, build
tool-chain and test scripts, operating system image build script as well as docu-
mentation is available as open source in form of a public G IT repository under
https://github.com/simula/5gvinni-oai-ns.
4 Testbed Setup
For our proof-of-concept evaluation, we deployed an O PEN S TACK setup [12] based
on O PEN S TACK “Stein” on U BUNTU 19.04 “Disco Dingo”. A management network
with Internet access was configured, in addition to a further network with Internet
access for the PDN. Furthermore, we deployed the latest OSM version “Release
SEVEN” on U BUNTU 18.04 “Bionic Beaver”. Another dedicated U BUNTU 18.04
server running in a network of our site provided a peer instance for the N ET-
P ERF M ETER5 [1, Section 6.3] [2,4] transport protocol performance evaluation tool.
Our VNF was then instantiated by OSM into O PEN S TACK, with the SGi inter-
face configured into our O PEN S TACK PDN network, as well as S1-U and S1-C
connected to the eNodeB networks. The VNF instance used the “2019.w45” tags
of the HSS and MME sources6 , as well as the “2019.w47” tags of the SPGW-
U and SPGW-C sources7 . Furthermore, we deployed an OAI eNodeB using tag
“2019.w44” of the eNodeB sources8 on a dedicated U BUNTU 18.04 “Bionic Beaver”
laptop with E TTUS USRP B210 SDR board attached over USB 3.0. As UE, we used
an U BUNTU 20.04 “Focal Fossa” laptop with H UAWEI E392 USB modem.
For our experiment, we examined three different scenarios: (1) download from
server to UE, (2) upload from UE to server, and (3) bidirectional communication be-
tween UE and server. We tested TCP and SCTP [1, Chapter 3] as transport protocols.
For SCTP, we used messages of 1400 bytes (i.e. resulting in one packet per message
without need for segmentation). The parameters for send and receiver buffer sizes,
as well as the congestion control had been left at the Linux kernels’ default settings.
Explicit Congestion Notification (ECN) [6] support had been enabled. The duration
of each transmission run was 20 s. All runs had been repeated at least 10 times; the
results show average values together with their 95% confidence intervals.
5 Results
Payload Throughput [Mbit/s]
Download Upload
11.3 4.5
3.9
10
SCTP SCTP
5
TCP TCP
0.3
0
0
SCTP
SCTP
TCP
TCP
Figure 4 presents the application payload reception rate results for the “Down-
load” scenario (at the UE, Subfigure 4(a)) and the “Upload” scenario (at the server,
Subfigure 4(b)) for both transport protocols in Mbit/s. For using SCTP, the results
are as expected in both directions, with around 11.3 Mbit/s for download to the UE,
and 3.9 Mbit/s for upload to the server.
For TCP, we would have expected slightly higher results, since the mobile net-
work should not interfere with higher-level protocols. However, the download speed
5 N ET P ERF M ETER: https://www.uni-due.de/∼be0001/netperfmeter/.
6 OAI HSS and MME: https://github.com/OPENAIRINTERFACE/openair-cn.git.
7 OAI SPGW-U and SPGW-C: https://github.com/OPENAIRINTERFACE/openair-cn-cups.git.
8 OAI eNodeB: https://gitlab.eurecom.fr/oai/openairinterface5g.git.
with TCP just reaches around 0.3 Mbit/s, while the upload speed reaches 4.5 Mbit/s.
Since the overhead for TCP is smaller than for SCTP (TCP can utilise the full MTU
size, while SCTP messages of 1400 bytes generate one packet per message), the
higher TCP upload rate is as expected. As part of ongoing work, we are currently
investigating the reason for this significant download speed difference, in order to
identify performance problems and/or bugs in OAI.
Payload Throughput [Mbit/s]
5
4
0.6
0.5
4
3.5
0.4
3
SCTP SCTP
2
TCP TCP
0.2
1
0.1
0.0
0
SCTP
SCTP
TCP
TCP
Transport Protocol Transport Protocol
(a) “Bidirectional”: Reception Rate at UE (b) “Bidirectional”: Reception Rate at Serv.
Fig. 5 Application Payload Throughput for Reception in the “Bidirectional” Scenario
The results for the “Bidirectional” scenario are presented in Figure 5, with the
application payload reception rate at the UE in Subfigure 5(a) and at the server in
Subfigure 5(b). As expected for the server side, the SCTP reception rate is slightly
lower than in the unidirectional “Upload” scenario: SCTP’s selective acknowledge-
ments [1, Chapter 3] for received chunks cause additional overhead. The reception
rate at the UE side is significantly lower than for the unidirectional “Download”
scenario for both, SCTP (ca. 0.5 Mbit/s vs. 11.3 Mbit/s) and TCP (ca. 0.1 Mbit/s
vs. 0.3 Mbit/s). Particularly the performance drop for SCTP cannot be explained by
additional overhead for the acknowledgements. As part of ongoing work, it is there-
fore necessary to further investigate the OAI software, in order to identify and solve
performance problems and/or bugs.
In summary, we have shown that our setup is working, with some likely bugs
and performance issues in the experimental OAI software used. However, since the
goal of our VNF is to easily instantiate mobile network testbeds with different ver-
sions and variants of the underlying software, our system can be very helpful in
supporting the OAI developers to identify problems as well as to evaluate bugfixes
and enhancements, in order to provide the users an easy-to-use deployment base for
their ongoing research and development work on 5G systems.
References
1. Dreibholz, T.: Evaluation and Optimisation of Multi-Path Transport using the Stream Con-
trol Transmission Protocol. Habilitation treatise, University of Duisburg-Essen, Faculty of
Economics, Institute for Computer Science and Business Information Systems (2012)
2. Dreibholz, T.: NetPerfMeter: A Network Performance Metering Tool. Multipath TCP Blog
(2015)
3. Dreibholz, T.: MELODIC at Hainan University: An Introduction to the MELODIC Project.
Keynote Talk at Hainan University, College of Information Science and Technology (CIST)
(2019)
4. Dreibholz, T., Becke, M., Adhari, H., Rathgeb, E.P.: Evaluation of A New Multipath Conges-
tion Control Scheme using the NetPerfMeter Tool-Chain. In: Proceedings of the 19th IEEE
International Conference on Software, Telecommunications and Computer Networks (Soft-
COM), pp. 1–6. Hvar, Dalmacija/Croatia (2011). ISBN 978-953-290-027-9
5. Dreibholz, T., Mazumdar, S., Zahid, F., Taherkordi, A., Gran, E.G.: Mobile Edge as Part of
the Multi-Cloud Ecosystem: A Performance Study. In: Proceedings of the 27th Euromicro
International Conference on Parallel, Distributed and Network-Based Processing (PDP), pp.
59–66. Pavia, Lombardia/Italy (2019). DOI 10.1109/EMPDP.2019.8671599. ISBN 978-1-
7281-1644-0
6. Fairhurst, G., Welzl, M.: The Benefits of using Explicit Congestion Notification (ECN). Inter-
net Draft draft-ietf-aqm-ecn-benefits-08, IETF (2015)
7. Hong, J., Dreibholz, T., Schenkel, J.A., Hu, J.A.: An Overview of Multi-Cloud Computing.
In: Proceedings of the 1st International Workshop on Recent Advances for Multi-Clouds and
Mobile Edge Computing (M2EC) in conjunction with the 33rd International Conference on
Advanced Information Networking and Applications (AINA), pp. 1055–1068. Matsue, Shi-
mane/Japan (2019). DOI 10.1007/978-3-030-15035-8 103. ISBN 978-3-030-15034-1
8. Lavado, G.: OSM VNF Onboarding Guidelines. White paper, ETSI (2019)
9. Luo, Y., Zhou, X., Dreibholz, T., Kuang, H.: A Real-Time Video Streaming System over
IPv6+MPTCP Technology. In: Proceedings of the 1st International Workshop on Recent
Advances for Multi-Clouds and Mobile Edge Computing (M2EC) in conjunction with the 33rd
International Conference on Advanced Information Networking and Applications (AINA),
pp. 1007–1019. Matsue, Shimane/Japan (2019). DOI 10.1007/978-3-030-15035-8 99. ISBN
978-3-030-15034-1
10. OpenAirInterface: OpenAirInterface Software Installation Support (2019)
11. OpenStack: OpenStack Architecture Design Guide (2019)
12. OpenStack: OpenStack Installation Guide (2019)
13. OpenStack: OpenStack Operations Guide (2019)
14. Reid, A., González, A., Armengol, A.E., de Blas, G.G., Xie, M., Grønsund, P., Willis, P.,
Eardley, P., Salguero, F.J.R.: OSM Scope, Functionality, Operation and Integration Guidelines.
White paper, ETSI (2019)