This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2021.3089549, IEEE
Transactions on Network and Service Management
1

The Impact of Networking Protocols on Massive

M2M Communication in the Industrial IoT
Cenk Gündoğan, Peter Kietzmann, Martine S. Lenders, Hauke Petersen, Michael Frey,
Thomas C. Schmidt, Felix Shzu-Juraschek, and Matthias Wählisch

Abstract—Common use cases in the Industrial Internet of creation of sub-networks for specific services and users that
Things (IIoT) deploy massive amounts of sensors and actuators can have specific 5G network parameters such as end-to-end
that communicate with each other or to a remote cloud. While latency, maximum throughput, and traffic density. It allows
they form too large and too volatile networks to run on ultra-
reliable, time-synchronized low-latency channels, participants companies to deploy ultra-reliable low-latency networks for
still require reliability and latency guaranties. We elaborate this critical infrastructure by exploiting time-slotted wireless link
for safety-critical use cases. This paper focuses on the effects technologies. It also supports massive machine type commu-
of networking protocols for industrial communication services. nication (mMTC) services to integrate billions of things using
It analyzes and compares the traditional Message Queuing contention-based wireless access, which is in the focus of this
Telemetry Transport for Sensor Networks (MQTT-SN) with the
Constrained Application Protocol (CoAP) as a current IETF article.
recommendation, and also with emerging Information-centric This new class of connected devices cannot be integrated
Networking (ICN) approaches, which are ready for deployment. into today’s Internet infrastructure without technologies that
Our findings indicate a rather diverse picture with a large bridge the scale. The IETF has designed a suite of protocols
dependence on deployment: Publish-subscribe protocols are more for successfully serving the needs of a constrained IoT. IPv6
versatile, whereas ICN protocols are more robust in multi-
hop environments. MQTT-SN competitively claims resources on adaptation layers such as 6LoWPAN [3] enable a deployment
congested links, while CoAP politely coexists on the price of its on constrained links (e.g., IEEE 802.15.4 [4]), which the Rout-
performance. ing Protocol for Low-Power and Lossy Networks (RPL) [5]
Index Terms—Industrial Internet of Things, 5G, constrained arranges in a multi-hop topology. The Constrained Applica-
environment, MQTT, CoAP, NDN, performance evaluation tion Protocol (CoAP) [6] offers a lightweight alternative to
HTTP while running over UDP, or DTLS [7] for session
security. This set of solutions extends the host-centric end-
I. I NTRODUCTION
to-end paradigm of the Internet to the embedded world and
The Internet of Things (IoT) is evolving by an increas- puts IPv6 in place for loosely joining the things.
ing number of controllers in the field that is augmented Doubts arose whether host-to-host sessions are the appro-
with network interfaces which speak IP. Emerging industrial priate approach in these disruption-prone environments of
IoT (IIoT) deployments are often stimulated by adding online (wireless) things, and the data-centric nature at the Internet
services to already existing systems for the sake of additional edge called for rethinking the current IoT architecture [8].
features and benefits. Such devices usually connect to power, ICN networks [9] have been identified as promising candi-
use common broadband links, and adopt the old Message date networks for a future IoT. Name-based routing and in-
Queuing Telemetry Transport (MQTT) protocol [1] for pub- network caching as contributed by Named Data Network-
lishing IoT data to a remote cloud. The prevalent use case ing (NDN) [10] bear the potential to increase robustness
forecast for the IoT, though, consists of billions of constrained of application scenarios in regimes of low reliability and
sensors and actuators that are mainly not cabled to power, but reduced infrastructure (e.g., without DNS). The quest for the
mobile and connected via low power wireless links. The key best solution remains open. Rather little is known about the
target of the IoT will be data generated from massive amounts differences and commonalities when deploying the varying
of tiny, cheap things that are severely challenging the current protocols in the wild. This surprisingly unsatisfying state of the
way of connecting to the Internet. art motivates us to implement, deploy, and thoroughly analyze
A number of approaches allow the creation of networks the different protocols in typical use cases and scenarios for
that can tackle these challenges, some of which are part of the the constrained IIoT.
recent 5G [2] efforts. 5G allows companies to create their own The main contributions of this paper shed light on a sys-
private networks on site. Companies can make use of a key tematic and comprehensive understanding of protocol design
5G concept called network slicing. Network slicing enables the for the IIoT. In detail:
1) We characterize important industrial use cases and sum-
C. Gündoğan, P. Kietzmann, and T.C. Schmidt are with Hamburg University
of Applied Sciences, Germany (e-mail: {cenk.guendogan, peter.kietzmann, marize requirements, backed up by field experiences of
t.schmidt}@haw-hamburg). the safety-critical industry. These requirements serve our
M. Lenders, H. Petersen, M. Wählisch are with Freie Universität Berlin, evaluations and may guide future analyses.
Germany, (e-mail: {m.lenders, hauke.petersen, m.waehlisch}@fu-berlin.de).
M. Frey, F. Shzu-Juraschek are with Safety io, Germany, (e- 2) We perform a thorough comparative analysis based
mail:{Michael.Frey, Felix.Juraschek}@safetyio.com). on extensive real-world experiments, including dense

1932-4537 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:09:20 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2021.3089549, IEEE
Transactions on Network and Service Management
2

scenarios of 50 constrained nodes. We consider three

common protocol families, CoAP, MQTT1 , and NDN,
3) We make our implementations publicly available and
thus provide an evaluation framework for protocol as-
Cloud Service
sessment in the IIoT. report
Gateway Gateway
control

This paper extends our previous work from ICN 2018 [11]
by refocusing on the industrial use cases and by adding many
Sensors & Actuators
experimental analyses tailored to the industrial requirements.
Our analysis revealed significant differences in protocol be- monitor

havior without an overall winner: The challenging multi- alarm

hop domain is best mastered by the ICN protocols, while

MQTT-SN proved most resistant against cross-traffic from Monitor & Report Alarm & Control
coexisting networks on the price of bursty occupation of
Fig. 1. Communication flows in IIoT environments.
network resources. Lightweight publish-subscribe protocols
such as CoAP observe and MQTT-SN operate fastest and most
versatile under relaxed wireless conditions. the control components, because here advanced communica-
The remainder of this paper is structured as follows. Sec- tion interfaces do not only improve manufacturing processes
tion II introduces safety-critical use cases and derives require- but may also help to save lives. Concrete examples are
ments for networking. The following Section III summarizes industrial processing plants or refineries. Typically, a mix of
the related work on key protocol concepts along with a qual- personal mobile and fixed gas detectors are used to sense the
itative comparison. Section IV explains our implementations environment for possible leaks of hazardous or combustible
and experimental setup. We present our measurements with gases. Often multiple teams of workers perform maintenance
a special focus on the impact of single- versus multi-hop tasks in designated areas, in which every worker is equipped
topologies and uncontrolled side traffic in Sections V and VI, with a gas detector. In addition, fixed gas detectors are
respectively. In Section VII, we revisit the related work on deployed on critical infrastructure, which support the main-
protocol performance and conclude in Section VIII. tenance tasks of workers. Detection of a dangerous level of
gas switches the gas detectors to alarm mode. In a networked
II. I NDUSTRIAL I OT U SE C ASES scenario, each detector sends a message to a centralized safety
The Industrial Internet of Things (IIoT) revolutionizes how monitoring application that runs either locally or in the (edge)
processes in industrial environments are controlled. It makes cloud. Based on new alarm information, the safety manager
use of local data aggregation, processing on the edge, and decides whether to preemptively evacuate close-by areas.
cloud computing to refine and optimize process controls. Industrial Control Systems. Control systems are widely
Here, infrastructure such as sensors and actuators are inter- deployed in industrial process automation, where they con-
connected. Applications range from aggregating locally stored tinuously monitor flows, and in factory automation, where
log data (reporting) to sensing and raising alarms if thresholds they mainly deal with discrete on/off signals of machines like
are undercut or exceeded (monitoring) and even intervening robots. Continuous monitoring periodically transmits process
processes with regulating actuators (controlling) [12]. The values and directly adjusts control of actuators such as valves
aggregation of log data has to be reliable and secure. Reliable or pumps in a closed loop. In contrast, discrete control
in a sense that all existing data needs to be transferred to a signals are event driven (e.g., generated from a relay after
designated device, secure within respect to the integrity and a robot action) and require individual reactions, which are not
authenticity of the data. While the sensing and propagation of stabilized by corrective periodic updates. Control events may
non-critical data in-network has modest timing requirements, be critical and consequently more sensitive to signal delays or
critical data such as alarms and control commands need to losses.
be forwarded and disseminated to relevant parties with low Deployment of sensors and actuators in industrial produc-
latency. tion environments is likewise harsh. Plants often undergo
Among others, use cases include monitoring and report- unpredictable variations in the environment (e.g., temperature,
ing of environmental and vital data of workers in harsh humidity, vibrations), in the radio regime (e.g., cross traffic,
environments (e.g., early responders), process regulation in reflections from moving metal objects, steam emittance from
manufacturing industries (e.g., chemicals, gas, oil, minerals), machines), and energy-wise. Many field devices operate on
and even factory automation using with control of robots batteries and may need to survive periods from days to months
(e.g., assembly lines) [13]. An overview of the communication between recharging and general overhauls.
flows as typical for industrial environments is visualized in
Requirements. From the networking perspective, the
Figure 1.
following requirements have to be satisfied to fully support
Safety-critical Environments. Safety-critical environments these industrial safety use cases. The most important
clearly benefit from the Industry 4.0 paradigm, i.e., networking requirement is latency for alarm messages from the detector
1 We use MQTT-SN, the UDP-adapted version in the MQTT family, because to the safety application and for evacuation requests in
TCP is inappropriate for the constrained IIoT the reverse direction. Being able to react quickly to safety

1932-4537 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:09:20 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2021.3089549, IEEE
Transactions on Network and Service Management
3

relevant incidents is crucial to contain and resolve dangerous Having a promising network access architecture such as 5G
conditions. The ANSI/ISA-100.11a-2011 standard [14] in place still requires efficient protocols on top. The current
defines latency requirements for three traffic categories in IIoT ecosystem proposes several competing solutions. These
industrial process automation applications: protocols require careful evaluation with respect to resource
allocation, convergence problems, and coexistence scenarios,
in particular in the context of a safety-critical Industry 4.0.
1) Safety traffic indicates emergency and requires a maxi-
mum of 10 ms delay in a deterministic fashion.
2) Control traffic is often but not always critical and III. N ETWORKING P ROTOCOLS FOR I NDUSTRY 4.0
depends on its application context, latencies between 10 Domain-specific protocols in the IIoT include Zigbee,
and 100 ms are sufficient. ISA100.11a, and WirelessHART [13], [17], all of which spec-
3) Monitoring traffic is used for maintenance and should ify a full protocol stack which can be configured to application
deliver messages within 100 ms on average. requirements. This is done by a centralized instance, usually
called a network manager. The network- and transport layers
Additionally, lost messages may lead to undetected alarms deal with IP connectivity on a backbone router whereas routing
in the safety monitoring software and, hence, a high reliability between constrained devices is implemented in a proprietary
is crucial. fashion directly on top of the MAC layer.
When not in alarm mode, detectors log their sensor readings Standard IoT networking protocols to handle massive vol-
on the device and send their status once per minute. This umes of heterogeneous data flows are CoAP and MQTT on
frequency increases when a detector changes to alarm state, the application layer in the current Internet, and information-
since regulations stipulate that the sensor readings need to centric (or data-centric) networking for the next generation
be logged at least once per second. Those logs are required IIoT. The latter provides higher layer services known from
for any investigation following up the particular gas alarm the application layer, such as naming and caching, directly
incident. Thus, it is desirable to send data at very low on top of the data link layer. In this section, we briefly give
frequency to the centralized safety application. technical background to common link technologies in the IIoT,
From an operational perspective, the network architecture and provide a qualitative comparison of the core protocols
should allow for the deployment of a flexible ecosystem, which CoAP, MQTT and ICN.
enables private as well as open networks.
Challenges. Meeting these requirements is challenging A. Common Link Layers for the IIoT
in harsh industrial environments, where time-slotting traffic Industrial protocols to handle data flows of sensors and
schedules are difficult to deploy. Workers are constantly mov- actuators heavily rely on the MAC at its link layer, which
ing, and path-loss and shadowing effects appear due to the we briefly discuss here. The popular 802.15.4 family is a
massive amounts of steel used in processing plants. In addition, characteristic example of lossy local area wireless transmission
there may be uncoordinated side channel traffic initiated by at minimal energy. We base our experimental work on 802.15.4
co-located systems from different manufacturers, which is with non-slotted media access to provide robust transmissions
particularly harmful for synchronized communication channels and neutral performance impact, for the absence of time
as defined in IEEE 802.15.4e (TSCH) [4] deployments [15], schedules.
[16]. In case of larger incidents, in which several hundred
IEEE 802.15.4-based technologies.
or thousand detectors send alarm notifications, coping with
Many short range wireless solutions in the IIoT are built
network traffic is even more challenging. And finally, some
on IEEE 802.15.4, which specifies low-power and low-rate
industrial areas are so remote that network coverage provided
physical layers and media access control. Prominent examples
by technologies such as cellular is very poor or non-existing.
are Zigbee, ISA100.11a, ore WirelessHART. The PHY in
On the upside, monitoring the complete gas detector status most deployments operates on the 2.4 GHz band and applies a
typically fits in less than one kilobyte of data. Thus, the simple O-QPSK (Quadrature Phase Shift Keying) modulation.
required available data rate is very low. Symbols are spread in the code domain to operate on a direct-
Potentials of 5G. A key building block for a successful sequence spread spectrum (DSSS). This increases resistance
IIoT is 5G [2]. Massive machine type communication (mMTC) against narrow-band interference.
provides a narrowband Internet access for sensing, actuating, We distinguish two classes of media access with this tech-
and monitoring devices. The ultra-reliable low latency commu- nology: (i) time-slotted and (ii) non-slotted multiple access.
nication (URLLC) in 5G will provide sub-millisecond latency The former reduces energy consumption, though, its perfor-
communication, which is essential for dedicated devices in mance is heavily affected by the scheduling logic upfront.
process control. Additionally, allowing industrial customers to Furthermore, network synchronization is susceptible to inter-
operate their private 5G-based networks provides the chance to ference. In contrast, non-slotted access omits scheduling and
close coverage gaps in remote areas. These private networks exploits carrier sensing to avoid collisions.
can then be inter-connected with a mobile carriers network. Wireless media is susceptible to eavesdropping, and security
Finally, 5G opens the scene for a data-centric network core, between neighbored nodes is provided by the 802.15.4 MAC.
which may help to increase reliability in constrained and lossy Hence, higher layer security is still required to achieve security
environments. on data domain. 802.15.4 specifies eight levels of protection

1932-4537 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:09:20 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2021.3089549, IEEE
Transactions on Network and Service Management
4

which reflect increasing security strengths to achieve data communication of unscheduled state changes, CoAP was ex-
privacy, integrity, and authenticity. Data encryption and mes- tended to support pushing new events to its peers. Still, this
sage integrity codes utilize AES with 128 Bit keys, though, does not allow for publish-subscribe scenarios when producer
provisioning of keys between peers needs to be handled by and consumer are decoupled in time and data is not yet
the upper layer, or manually during deployment. In addition, available at the request. The support for delayed data delivery
access control lists exclude frames that are received from un- in publish-subscribe was specified in CoAP observe [25]. Here,
trusted nodes and hence, could be malicious. It is noteworthy, clients can signal interest in observing data, which implies that
though, that bare 802.15.4 is still vulnerable to a number of a CoAP server delivers data as soon as available and maintains
attacks [18], [19]. state until clients explicitly unsubscribe. The default approach
All three standards mentioned above utilize the time-slotted to reinforce communication channels in CoAP deployments
channel hopping mode of the IEEE 802.15.4e specification is to use (datagram) transport layer security (D)TLS [7], [26].
to guarantee link resources. This type of time- and frequency OSCORE [27] is a recent addendum to the CoAP specification
multiplex requires coordination among nodes to synchronize and allows for securing content objects on the application
to a schedule, and to grant resource access. Hence, it adds layer, in addition to any transport protection.
signaling overhead, especially for sporadic and asynchronous CoAP is the IETF standard for implementing data transfer
data. The slot mode, however, enables device sleep cycles on the application layer in the future Industrial Internet of
to save energy. The IETF adopted 6TiSCH [20], [21] as an Things. Currently, several implementations exist, as well as
open standard solution that bases on the above mentioned early adoption in a few selected products and deployments.
protocols and enables IPv6 connectivity on constrained nodes The well-deployed solution, MQTT. The Message Queue
themselves. Due to central coordination and susceptibility to Telemetry Transport (MQTT) [1] was designed as a publish-
side-channel interference [15], [22], however, TiSCH-type link subscribe messaging protocol between clients and brokers.
layers do not meet the use cases of uncoordinated deploy- Clients can publish content, subscribe to content, or both.
ment in harsh industrial environments. We therefore base our Servers (commonly called broker) distribute messages between
experimental work on the contention-based and grant-free publishing and subscribing clients. It is worth noting that the
CSMA/CA mode of IEEE802.15.4 and concentrate on the protocol is symmetric: Clients as well as brokers can be sender
performance impacts of the higher layers. and receiver when MQTT delivers application messages.
Novel, non-orthogonal technologies. Orthogonal access MQTT is considered a lightweight protocol for two reasons.
schemes like 802.15.4 as presented above, are key to cur- First, it provides a lean header structure, which reduces packet
rent wireless systems, however, the orthogonality criterion parsing and makes it suitable for constrained devices with
limits the number of users. Consequently, mMTC platforms low energy resources. Second, it is easy to implement. In its
advance in modulation and multiplexing by introducing non- simplest form, MQTT offloads reliability support completely
orthogonal schemes to the space, time, frequency, or code onto TCP.
domain [23], [24]. This allows for resource overloading to To provide flexible Quality of Service on top of the un-
extend the number of simultaneous users but also increases derlying transport, MQTT defines three QoS levels. QoS 0
receiver complexity. Sparse code multiple access (SCMA) is implements unacknowledged data transfer. An MQTT receiver
a core technique in 5G systems which operates in the code gets a message at most once, depending on the capabilities
domain to enable overloading. SCMA maps data-streams to of the underlying network, as there is no retransmission on
non-orthogonal code streams. Codewords of multiple SCMA- the application layer. QoS 1 guarantees that a message is
layers are combined and transmitted over OFDMA (orthogonal delivered at least once. Based on timeouts, an MQTT sender
frequency-division multiple access), a multi-carrier technique will retransmit application messages when an acknowledgment
with time slotted access. Space division is achieved by tradi- is missing. QoS 2 ensures that a message is received exactly
tional cell clustering and advanced with antenna beamforming once, to avoid packet loss or processing of duplicates at the
to reduce cell overlap, and thus, to increase resource re- MQTT receiver side. This requires a two-step acknowledgment
utilization. Hence, 5G extends media access in four dimen- process and more state at both sides.
sions: code, frequency, time, and space. To adapt MQTT to constrained networks which are based
on low data rates and very small packet lengths such as in
802.15.4, MQTT-SN [28] is specified. Header complexity is
B. Common Application Layer Protocols for the IIoT
reduced by replacing topic strings with topic IDs, to identify
The IETF solution, CoAP. The Constrained Application content. In contrast to MQTT, MQTT-SN is able to run on top
Protocol (CoAP) [6] aims for replacing HTTP to enable of UDP. It still supports all QoS levels but does not inherit
M2M communication between constrained nodes. In contrast any reliability property from the transport layer.
to HTTP, CoAP is able to run on top of UDP and introduces MQTT provides optional header fields during the establish-
a lean transactional messaging layer to compensate for the ment of connections to authenticate with a broker, but most
connectionless transport. CoAP provides a more compact other responsibilities, such as encrypting and authenticating
header structure than HTTP. It currently supports three com- published data, are relayed to the application. The transport
munication primitives: (i) pull, (ii) push, and (iii) observe. is commonly protected using transport layer security (TLS)
Pull implements the common request response communication for the TCP-based MQTT, and the datagram variant DTLS
pattern. However, as IoT scenarios also include the pro-active for MQTT-SN. The specification provides implementation

1932-4537 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:09:20 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2021.3089549, IEEE
Transactions on Network and Service Management
5

TABLE I
C OMPARISON OF C OAP, MQTT, AND ICN PROTOCOLS . C OAP AND MQTT SUPPORT RELIABILITY ONLY IN CONFIRMABLE MODE ( C ) AND
Q O S LEVELS 1 AND 2 (Q1, Q2).

Current IoT Protocols ICN Protocols

CoAP
MQTT MQTT-SN NDN HoPP
PUT GET Observe
Transport UDP UDP UDP TCP UDP n/a n/a
Pub/Sub 8 8 3 3 3 8 3
Push 3 8 3 3 3 8 8
Pull 8 3 8 8 8 3 3
Flow Control 8 8 8 3 8 3 3
Reliability (c) (c) 8 (Q1, Q2) (Q1, Q2) 3 3
transport / transport / transport /
Security Mechanism content object content object content object transport transport content object content object
End-to-end Protection (3) (3) (3) 8 8 3 3

notes and guidance for a secured deployment in the protocol instead; (ii) it does not expect an immediate reply, but issues
specification [1, Section 5]. Interests with extended lifetimes. HoPP enables rapid commu-
nication of unscheduled data events. It operates at a similar
C. Upcoming Data-centric Networking Layers timescale as push protocols without actually pushing data.
Information-centric networking (ICN) implements the vi-
sion of a native data-centric Internet. The most active approach
is named-data networking (NDN). The core NDN proto- D. Qualitative Comparison of Protocols
col [10] combines name-based routing with stateful forwarding
to deploy a request response scheme on the network layer. Key properties of the three protocol families CoAP, MQTT,
Any consumer can request named data using so-called Interest and NDN and its variants are compared in TABLE I. Special-
messages, which are forwarded towards publishers. Data is ized properties of the different approaches become apparent:
subsequently delivered along a trail of reverse path forwarding Every protocol variant features distinct capabilities. Notably
states, starting either from the original publisher or the first in the IoT, where TCP (aka generic MQTT) is unavailable,
in-network cache that can provide the requested data. As an the pull-based NDN and NDN-HoPP are the only protocols
important feature, data will only be delivered to those who admitting flow control and reliability as a generic service.
requested the data. This means that data must be (individually) Further, low-power deployments show a growing demand for
named at the Interest request and that yet unavailable data application gateways to perform protocol conversions and
requires repeating Interests until the application receives the changing the transport, e.g., from UDP to TCP. These opera-
data. Due to the comprehensive use of on-path caches and tions naturally break the end-to-end principle [33] at gateways,
the stateful forwarding fabric, the concept of endpoints be- terminate any transport security, and therefore render the
comes negligible for NDN deployments. Thus, these regimes communication between constrained IoT devices and cloud
allow for an orthogonal approach of delivering autonomously services vulnerable to interception attacks [34], [35]. The NDN
verifiable content objects independently of location and com- family of protocols and CoAP with OSCORE protection can
munication endpoints. guarantee security properties to remain intact beyond protocol
Several publish-subscribe extensions have been proposed for conversions [36]. In addition, content object security enables
NDN [29]–[31] to provide further decoupling of consumers multicast and multi-homing capabilities for the IoT, while
and data sources. HoP and Pull (HoPP) [31] is a lightweight these are hardly feasible to deploy with transport protection
variant we previously developed to provide a publish-subscribe due to the tight endpoint binding. This especially affects
system for constrained IIoT deployments based on ICN/NDN setups that experience device mobility and frequent network
principles. A constrained publisher announces a name towards disruptions.
a content proxy to trigger content requests and to replicate the To give a first estimate of the different protocol complex-
data towards a content proxy (or broker). Forwarding nodes ities, we compare the sizes of the message types for each
on the path between publisher and content proxy hop-wise protocol in Figure 2. Most of the protocols need nearly the
request content for this name by using common Interest and same amount of data. CoAP observe (CoAP OBS) exhibits
data messages. A content subscriber in HoPP behaves almost the lowest complexity but does not acknowledge. A single
like any content requester in NDN and issues a regular Interest registration is sufficient to receive subsequent data published
request towards the content proxy. However, in contrast to under the same name. HoPP, on the other hand, introduces
NDN (i) a subscriber cannot extract content names from its overall the largest packet size as it introduces name advertising
forwarding information base (FIB), since FIBs only contain on the data plane. We elaborate the scenario and give a
default routes [32], but uses application-specific topic tables comprehensive performance evaluation in the next sections.

1932-4537 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:09:20 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2021.3089549, IEEE
Transactions on Network and Service Management
6

Packet Size [Bytes]

300 choose two arbitrary nodes and run all single-hop experi-
Request Response Advertise Publish ACK
ments on them. One node is a content producer, the other
200
node acts as consumer (gateway/broker).
100
Multi-hop topology The Grenoble site consists of approxi-
mately 350 nodes spread evenly in the Inria Grenoble
0 building. We choose 50 M3 nodes (low-end device)
CoAP GET CoAP OBS MQTT-SN NDN HoPP
and one A8 node (gateway/broker) arbitrarily and run
Fig. 2. Packet sizes in bytes for each protocol. all multi-hop experiments on them. All low-end devices
operate as content producers. In our CoAP and MQTT ex-
periments, we use RPL to build and maintain the routing
IV. A N E NVIRONMENT FOR A SSESSING I NDUSTRIAL topology across all nodes. For NDN-based experiments
M2M N ETWORKS we build analogous tree topologies. Typical path lengths
Common deployments in the IIoT consist of stub net- are four to six hops.
works that are single-hop in areas of dense infrastructure, Two-hop topology with cross-traffic We choose three M3
but may also be multi-hop in widespread facilities such as nodes that are arranged in a line topology within the
oil refineries or platforms. Traffic flows from or to the edge Grenoble site. One node acts as a consumer, another node
nodes in three patterns: (i) scheduled periodic sensor readings, serves as a producer and the last node is a forwarder in
(ii) unscheduled and uncoordinated data updates, or (iii) on between. Additionally, we deploy a fourth node acting as
demand notifications or alerting. It is worth noting that the a cross-traffic generator in the vicinity of our forwarder.
different protocol properties (e.g., pub-sub versus request-
response) meet these alternating demands differently well. In C. Scenarios and Parameters
the following, we present a testing environment consisting
We align all experiments with respect to the configura-
of software, a real-world testbed, and relevant scenarios that
tions of retransmissions and timeouts to ensure comparability
approximates the characteristics of massive M2M networks for
among protocols. All protocols employ the same retransmis-
embedded devices.
sion strategy: In case of failures, each node waits 2 seconds
before retransmitting the original application or control data.
A. Software Platforms For NDN and HoPP, retransmissions are performed hop-by-
On the constrained nodes, all of our experiments are hop, while CoAP and MQTT retransmit from end to end.
based on the RIOT operating system [37] version 2018.01. At most 4 retransmissions will occur for each data item.
To analyze CoAP, MQTT-SN, and NDN we use gCoAP, Interest lifetimes are configured to 10 seconds for NDN based
Asymcute, and CCN-lite respectively. All three protocol protocols to limit PIT memory consumption. We repeat each
implementations are part of the common RIOT release and experiment 1,000 times.
thus reflect typical software components used in low-end IoT To accommodate all 50 nodes in the routing topology, the
scenarios. FIB sizes have been adjusted accordingly on each constrained
Brokers or gateways are deployed on Linux systems within node. For CoAP and MQTT, this translates in our IPv6
the testbed infrastructure. To support an MQTT-SN broker and scenario to a FIB size of 50 entries with roughly 32 bytes
a CoAP observe client, we used aiocoap version 0.3 and each. In our NDN scenarios, each node owns a unique prefix
mosquitto.rsmb version 1.3.0.2. Both are popular open of the form /ρi with a length of 24 bytes. The next-hop
source implementations in this context. face of each FIB entry points to the 8-byte IEEE 802.15.4
link-layer address. In total, this setup yields comparable size
B. Testbed requirements for all scenarios.
In the NDN scenarios, we use unique content names pre-
We conduct our experiments in the FIT IoT-LAB2 testbed.
fixed by /ρi with incremental local packet counters. CoAP
The hardware platform consists of typical class 2 devices [38]
works without unique names but uses common URIs. The
and features an ARM Cortex-M3 MCU with 64 kB of RAM
MQTT-SN protocols register a common topic name, similar
and 512 kB of ROM. Each device is equipped with an
to CoAP, and publish under a unique topic ID thereafter. In
Atmel AT86RF231 transceiver to operate an IEEE 802.15.4
all scenarios, the data is of the same JSON format consisting
radio. The gateway runs on a Cortex-A8 node, which is more
of a unique identifier and a sensor value attribute. These short
powerful than the M3 edge nodes. Every node in the testbed
messages can be accommodated by the link layer and do not
is monitored by a control node which allows for parallel radio
require fragmentation. It is noteworthy that we neither apply
sniffing without misusing transceivers of M3 devices.
header compression in the IP [39] nor in the NDN world [40].
The testbed provides access to several sites with varying
properties. We perform our experiments on different sites, to
V. T HE I MPACT OF T OPOLOGY IN M ASSIVE D EPLOYMENT
analyze single-hop as well as multi-hop scenarios.
Single-hop topology The Paris site consists of approximately The objective of this work is to quantify the impact of
70 nodes, which are within the same radio range. We network protocols on IIoT communication systems. With this
goal in mind, we deploy the different publish-subscribe and
2 http://www.iot-lab.info/ request-response protocols in the same physical environment

1932-4537 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:09:20 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2021.3089549, IEEE
Transactions on Network and Service Management
7

1.0 1.0
MQTT-SN (Q1)
0.8 0.8 CoAP OBS
CoAP GET (c)
NDN
0.6 0.6
HoPP
CDF

CDF
MQTT-SN (Q1)
0.4 0.4
CoAP OBS
CoAP GET (c)
0.2 NDN
0.2
5 10 15 20
HoPP [ms]

5 10 15 20 25 30 0 2 4 6 8 10
Time to Completion [ms] Time to Completion [s]
(a) Scheduled publishing at 1 s interval (b) Unscheduled random publishing in 1s . . . 3s

Fig. 3. Time to content arrival in a single-hop topology.

and compare their operational properties as well as their range of 10 ms without additional protocol operations – the
performance results. Evaluation metrics focus on reliability unsurprising outcome of content triggers. CoAP requests con-
and timeliness of the data delivery, which are critical in the tent using a common name with the result of likely duplicate
low power lossy environment of these systems. Additionally, content transmissions. On average, CoAP needs two requests
we study link stress and resource efficiency of the constrained to retrieve fresh content with the expected average delay of
data flows. We start our analysis by comparing single- versus ≈ 2 s and a corresponding polling overhead of 200 %, see
multi-hop topologies. Fig. 3(b). In contrast, NDN admits lower overhead, as Interests
are locally managed at the PIT and only retransmitted after
A. Single-Hop Topology state timeout. Issuing Interests at a higher rate than content
Protocol performances are first evaluated in a single-hop arrival, however, leads to an accumulation of open states in the
topology at the Paris testbed of IoT-LAB. In agreement with PIT. As resources on the constrained nodes are tightly bound,
the requirements of our industrial use case, we perform a the PIT limits are quickly reached and can be only met by
periodically scheduled publishing at every second, and a ran- either discarding newly arriving Interests, or by overwriting
domized, unscheduled publishing. We measure the time until pending Interest state. Both countermeasures delay content
content arrives at the consumer. The results are summarized delivery, as can be seen in Fig. 3(b).
in CDFs as functions of packet transfer time, see Fig. 3.
In the case of scheduled traffic, all protocols successfully B. Multi-Hop Topology
deliver data packets within short, similar times as shown in We now consider the more challenging use case of mixed
Fig. 3(a). Lightly visible steps in the CDFs indicate retrans- communication in multi-hop topologies: 50 nodes exchange
missions on layer 2, which occur on the same timescale of content that is published every 5 or 30 seconds in an unco-
milliseconds. Naturally, the protocols that push data (MQTT, ordinated manner. Repeated experiments were performed on
CoAP OBS) react quicker than request-response schemes. As a the Grenoble testbed with tree topologies of routing depths
pull-based publish-subscribe scheme, HoPP performs slowest, varying from four to six hops.
as it initiates hop-wise data transfers on request. First, we examine the temporal distributions from content
Our second evaluation addresses the common IoT use publishing to arrival in analogy to the single-hop cases.
case of publishing data at irregular intervals. This is the Fig. 4 combines the results for all protocols, as well as both
typical pattern for observing third party actions (e.g., alarms), publishing rates. The overall results reveal a much slower
or largely uncoordinated sensing environments. The publish- and less reliable protocol behavior than could be expected
subscribe protocols naturally serve these application needs. We from the single-hop values in Fig. 3. Graphs reflect the
quantify the behavior of the request-based protocols in practice common experience in low power multi-hop environments that
and chose the moderate setting of publishing content every two interferences and individual error probabilities accumulate in
seconds on average. Publishing is uniformly distributed in the a destructive manner.
interval of [1 s,. . . 3 s]. The protocols CoAP and NDN request The IP-based protocols, which operate in an end-to-end
the content periodically every second so that updates are not paradigm, now all fail in delivering data, the publish-subscribe
lost. protocols CoAP OBS and MQTT-SN representing the lower
Fig. 3(b) visualizes content delivery times for unscheduled end. Widespread temporal distributions indicate repeated re-
publishing and reveal a diverse picture. CoAP GET and NDN transmissions on the network layer that operate on the scale
now operate on a timescale of seconds, while the publish- of many seconds and still cannot compensate losses. In con-
subscribe protocols continues to complete in the unaltered trast, the hop-by-hop nature of the ICN protocols enfolds its

1932-4537 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:09:20 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2021.3089549, IEEE
Transactions on Network and Service Management
8

1.0 1.0

0.8 0.8

0.6 0.6
CDF

CDF
0.4 0.4

0.2 NDN MQTT-SN (Q1) 0.2 NDN MQTT-SN (Q1)

HoPP CoAP OBS HoPP CoAP OBS
0.0 CoAP GET (c) 0.0 CoAP GET (c)
0 5 10 0 5 10
Time to Completion [s] Time to Completion [s]
(a) 5 s publishing interval (b) 30 s publishing interval

Fig. 4. Time to content arrival in multi-hop topologies of 50 nodes for publish-subscribe and request-response protocols at different publishing intervals.

robustness in these harsh environments. The publish-subscribe sity increases toward the gateway, which serves as the root
protocol HoPP quickly reaches 100 % success in data transfer of the routing tree. Here packets accumulate on the last hops,
– 80 % (Fig. 4(a)) resp. 95 % (Fig. 4(b)) of data units why link exhaustion, collisions, and buffer drops increase. The
arrive within milliseconds and without any network layer effective success rate of packet traversal is largely influenced
retransmission. The performance of the plain NDN also shows by the flow properties (i.e., bursts versus balanced flows)
decent results both in promptness and reliability, even though as shaped by the networking protocol. In this, the protocol
5 % of data chunks remain lost in the fast publishing scenario behaviors largely differ and lead to diverging results. The ICN
of 5 s. protocols NDN and HoPP in Fig. 6 show a more random
Second, we focus on the link utilization. We measure all distribution of small losses, which is typical for wireless
individual paths that each unique data packet traveled on interference and can be compensated by local retransmissions.
its destination from source to sink, and contrast the results In contrast, the IP-based protocols all suffer from more intense
with the corresponding shortest possible path. Results are losses close to the gateway—loss of IP packets exceeds ICN
visualized as scatterplots in Fig. 5. Each dot represents one loss by factors between 10 and 100. Only CoAP OBS looses
or several events, the dot size is drawn proportionally to event moderately, because CoAP retransmissions are not active in
multiplicities. Solid lines indicate the shortest paths, while this protocol variant and the total number of packets remains
events left of the line represent failures (traversal shorter than lower.
the shortest path). Right of the solid diagonal retransmissions Compared to the confirmable CoAP GET configuration,
are counted. MQTT-SN exhibits less loss events on the links farther away
The ideal protocol performance is situated on the diagonal from the source, because of its more compact packet encoding.
line with all data traversing each link only once on the shortest Extreme loss values show up at the source for MQTT-SN,
path. This ideal behavior is most closely approximated by the however, due to its uncoordinated, bursty retransmissions.
NDN core and the NDN-HoPP protocols. A largely contrasting These effects amplify in the multi-hop tree topology as the
performance can be seen from the reliable IP protocols MQTT- total network traffic accumulates towards the few links that
SN (Q1), which admits huge numbers of retransmissions. directly connect to the gateway node. This explains the details
These retransmissions stress an exhausted link even further behind the large transmission numbers seen in Fig. 5.
and stimulate cascading failures. The CoAP protocol variants Next, we comparatively examine the nodal energy consump-
behave more network friendly, thereby accumulating loss in a tion as a function of time throughout an experiment duration
polite fashion. of ≈60 minutes for each deployment in our protocol selection.
We further question the details of packet loss and count In the typical IoT scenario of acquiring and distributing sensor
the transmission failures on each link during the experiment. readings, energy expenditures due to computational efforts
Fig. 6 displays the number of packets lost in one minute as a usually remain within tolerable limits. Radio activities, on the
function of time and hop distance from the gateway. Note that other hand, dominantly drive energy demands when receiving
in this analysis every packet lost on some link is counted, no and transmitting data over the air. To concentrate on power
matter whether the retransmission mechanisms on the different expenses based on protocol characteristics, such as packet
layers can compensate this loss. An overall successful packet sizes and the quality of corrective actions, we only measure
transfer in this analysis can thus account for many loss events energy levels for actual radio operations. Consequently, we
on intermediate links. Frequent losses indicate a less effective disregard expenses due to actively listening on the radio in
link utilization by the network protocol. our calculations, since this part of the equation decreases
It is common for this convergecast scenario that loss inten- substantially in proper deployments with correct duty cycling

1932-4537 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:09:20 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2021.3089549, IEEE
Transactions on Network and Service Management
9

6 Loss Shortest Path

5
Hop Count

4
3 Retransmissions
2
1 NDN HoPP CoAP GET (c) CoAP OBS MQTT-SN (Q1)
0 10 20 0 10 20 0 10 20 0 10 20 0 10 20
Data Packets per Publish [#]

Fig. 5. Link traversal vs. shortest path for a 30 s publishing interval. The scatterplots reveal the link stress with dot sizes proportional to event multiplicity.

6
4

NDN
2
Hop Distance from Gateway [#]

0
6

HoPP
4 102
2

Loss Events [#]

0
6

GET (c)
CoAP
4
2
0
6 101

CoAP
4

OBS
2
0
6

MQTT-SN
4

(Q1)
2
0 100
0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180
Experiment Duration [min]

Fig. 6. Loss count at links as a function of experiment time and hop distance. Cells show the loss intensity per minute for a 30 s publishing interval.

and utilizing low-power modes. We obtain the power con- minimal number of packets in the network. Shortened re-
sumption levels for transmitting and receiving from the Atmel transmission paths with on-path caching are the key protocol
AT86RF231 [41] data sheet and convert nodal packet statistics features of NDN to reduce overall energy expenditures down to
into appropriate energy expenditures. an average of ≈99 mJ and still maintain distinct success rates.
TABLE II compiles the statistical key properties for the Since HoPP counts a link-local signaling overhead for each
nodal energy expenses of the 50 nodes in our multi-hop setup published data, the total power consumption slightly elevate.
with a publishing interval of 30 s. Principally, maximum Last, we dive deeper into the flow balance of the different
values represent energy levels of gateway nodes, since packets protocols and evaluate its effective data goodputs during
naturally accumulate there due to the convergecast setup. The various content publishing experiments. Fig. 7 summarizes the
25% (Q1) and 75% (Q3) quartiles roughly illustrate the energy results. We display the distribution of goodput from the dif-
distributions. We note that nodes closer to the gateway are ferent experiments in box plots and compare to the theoretical
much more engaged with forwarding duties and experience optimum (lines). Time series of data goodput further reveal
additional radio activities when compared to leaf nodes. Thus, the flow behavior as displayed in the lower row of the figure.
these nodes generally position towards the higher end of the Clearly, HoPP admits the most evenly balanced flows and
distribution. shows nearly optimal goodput values, closely followed by
The average consumption for a single node greatly varies NDN. All other flow performances fluctuate with some ten-
between the selected configurations, but agrees with our pre- dency of instability when approaching its full transmission
vious conclusions in Fig. 5 and Fig. 6. CoAP OBS displays speed. Some IP-based flows in MQTT-SN and CoAP drop
the lowest average expense with ≈55 mJ per node, which is to lower delivery rates which is dominantly caused by slow
expected due its push-based nature and the lack of retrans- repeated end-to-end retransmission. Multi-hop retransmissions
missions. MQTT-SN presents another extreme: the excessive in this error-prone regime tend to cause additional interfer-
amount of corrective actions, especially at the gateway—see ences and accumulate transmission errors. As a consequence,
the elevated maximum in TABLE II—leads to an average protocols operate at reduced efficiency – for CoAP OBS
expense that is fourfold. CoAP GET situates between both protocol performance drops down to 50 %. The overall results
configurations with an average of ≈152 mJ. NDN operates show that the absence of flow control as in UDP/IP–based
reliably throughout the experiment (see Fig. 4(b)) with a protocols make protocols fragile. Hop-wise retransmission

1932-4537 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:09:20 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2021.3089549, IEEE
Transactions on Network and Service Management
10

Protocol µ [mJ] σ [mJ] min [mJ] Q1 [mJ] median [mJ] Q3 [mJ] max [mJ] sum [mJ]
NDN 98.99 213.96 23.66 23.66 23.88 70.98 1,243.54 4,949.50
HoPP 167.33 271.50 34.69 37.55 44.87 158.37 1,494.29 8,534.27
CoAP GET (c) 151.61 293.72 25.62 27.94 29.54 82.26 1,411.53 7,732.13
CoAP OBS 55.78 89.66 10.59 12.88 20.17 42.92 371.84 2,844.80
MQTT-SN (Q1) 245.66 394.63 65.61 68.66 74.91 183.10 1,915.61 12,529.12

TABLE II
S TATISTICAL KEY PROPERTIES OF NODAL ENERGY EXPENDITIURES W. R . T. RADIO TRANSCEIVER OPERATIONS , i.e., ACTIVELY SENDING AND RECEIVING .
VALUES CALCULATE OVER THE EXPERIMENT DURATION FOR OUR PROTOCOL SELECTION CONFIGURED WITH A 30 S PUBLISHING INTERVAL . Q1 AND
Q3 REPRESENT THE FIRST (25%) AND THIRD (75%) QUARTILE , RESPECTIVELY.

NDN HoPP CoAP GET (c) CoAP OBS MQTT-SN (Q1)

20
theoretical theoretical theoretical theoretical theoretical
goodput goodput goodput goodput goodput
Goodput [KiB / minutes]

10

0
5 10 15 20 25 30 5 10 15 20 25 30 5 10 15 20 25 30 5 10 15 20 25 30 5 10 15 20 25 30
Content Publishing Intervall [seconds]
20
05s 20s 05s 20s 05s 20s 05s 20s 05s 20s
10s 25s 10s 25s 10s 25s 10s 25s 10s 25s
10 15s 30s 15s 30s 15s 30s 15s 30s 15s 30s

0
20 40 20 40 20 40 20 40 20 40
Experiment Duration [minutes]

Fig. 7. Goodput summary and flow evolution for all protocols at different publishing intervals.

management as applicable in NDN and HoPP re-balances Hidden Terminal

flows and explicitly demonstrates its benefits for the IIoT Cross-traffic
instead. CT
radio range
VI. T HE I MPACT OF C OEXISTING W IRELESS N ODES ns Collisio
Collisio ns
We continue our protocol analysis by investigating the case
of uncontrolled disturbances. In unshielded environments, a F
C P
frequent source of wireless degradation is caused by uncoor-
dinated concurrent networks or by radiating appliances that Effective
Collision
interfere in the utilized frequency range. Such alien sources Avoidance
of disturbance are emulated by cross-traffic from a hidden
terminal in our experiments.
Fig. 8. Experiment setup for measuring protocol resilience under cross-traffic.

A. Setup of Cross-traffic at Intermediate Hop

We examine the robustness of the networking protocols To scale the effect of cross-traffic at different stress lev-
under cross-traffic using a two-hop topology between a content els, we configure the traffic generator in two dimensions as
producer (P) and a consumer (C). Cross-traffic is injected illustrated in Fig. 9.
towards an intermediate forwarder (F) as illustrated in Fig. 8.
We center (C) and (P) at (F) and verify in preceding measure- Burst Size reflects the number of consecutive packets sent
ments that both links perform comparably in both directions. to a third party link-layer unicast address. Each burst
By ensuring symmetry, we prevent a measurement bias with consists of a series of packets with a payload of 100 bytes.
respect to antisymmetric sequencing of the different protocols. Inter Burst Time denotes the pause in which our cross-
(CT) is our cross-traffic generator and placed next to (F), so traffic generator keeps the radio silent.
that (F) overhears all transmissions, while (CT) remains hidden With varying cross-traffic patterns in place, we measure the
for (C) and (P). Hence, CSMA/CA fails for (C) and (P) and error rates, data load, and time to completion for each protocol.
we expect an increased packet loss due to collisions for these We apply the periodic traffic pattern of one data unit every
nodes. 1 s advised by our use case. Our first measurement validates

1932-4537 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:09:20 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2021.3089549, IEEE
Transactions on Network and Service Management
11

the experimental environment. Fig. 10 displays the times to

content arrival in the absence of cross traffic. All protocols 1.0
perform perfectly as expected.
0.8
B. Results
Turning on the cross-traffic generator changes the picture. 0.6

CDF
Fig. 11 presents an overview of the protocol behaviors under MQTT-SN (Q1)
25 different scenarios of competing traffic. The color in each 0.4 CoAP OBS
block visualizes the relative packet loss, while the numbers CoAP GET (c)
denote the relative redundancy of data packets on the links. NDN
0.2
A regular, undisturbed data packet traverses each link only HoPP
once. Numbers higher than 1 indicate duplicate data packets,
lower numbers indicate loss on the paths of data or request 0 20 40 60 80 100
messages. Time to Completion [ms]
Decreasing the pauses between increasing bursts pushes the
performance of all protocols below 50 % success rate. Still, the
results are quite diverse. While the request-response protocols Fig. 10. Time to content arrival in a two-hop topology at 1 s interval without
cross-traffic.
quickly degrade to error rates above 80 %, those protocols that
push data (MQTT-SN and CoAP OBS) show a much higher
chance of successfully transmitting data. It should be noted VII. R ELATED W ORK ON P ROTOCOL E VALUATION
here that the CoAP OBS is unreliable and does not repeat data.
A. Data dissemination in the Industrial IoT
Hence, its success rate turns lower than MQTT-SN, while its
data rate on the air also drops. On average, only ≈ 60 % Wireless communication plays an important role for con-
of the data packets traverse both links, many of which only necting sensors and actuators in the IIoT and its heterogeneous
make the first hop. In contrast, MQTT-SN pushes packets via systems. We have discussed current wireless link layers in
UDP until an acknowledgment arrives. This leads to a very Section III, which are all error prone in the often harsh
high redundancy, which almost triples the data rates on the industrial deployments. Networking protocols on its upper
links. By pushing data intensely, though, MQTT-SN manages layers may procure for high reliability as well as security
to attain superior performance among all protocols. needed for application scenarios such as control loops or
CoAP GET and the ICN protocols transmit data only on safety related alerting. Challenges and requirements for typical
request. Since the cross-traffic jamming repeatedly destroys IIoT scenarios have been investigated in [42]. Bernieri et
these requests, data is often not even transmitted. In conse- al. [43] monitor factory automation systems and identify traffic
quence, data only sparsely appears on links even though these anomalies in a hybrid system of traditional Modbus/TCP [44]
reliable protocols retransmit. The results are slightly better for as well as CoAP communication. Experimental evaluations of
the ICN protocols, since they transfer packets hop-wise with a distributed IoT data plane were recently presented in [45]
caching in place at the forwarding node. and [46]. While the first work aims at optimizing the overall
This harsh, highly disruptive experimental regime reveals a network throughout on edge nodes, the second introduces
significant heterogeneity among the protocols and their ability a lightweight messaging middleware to minimize resource
to co-exist on stressed links. While MQTT-SN accesses wire- consumption on low-end devices for edge computing.
less resources rather aggressively – possibly on the expense Eggert [47] demonstrates on real IoT hardware the feasibil-
of concurrent communication, the request-response protocols ity of using QUIC [48] for constrained devices. As a transport
politely retreat from flooding data onto the congested link. It based on UDP, it provides a lightweight replacement for TCP
is noteworthy that data arrives in about equal shares among with flow-controlled and multiplexed streams, a low-latency
the four protocol retransmissions, so that about 25 % reaches connection establishment, and built-in security features, which
the consumer only after 8 s. Reducing the retransmission are valuable additions for safety-critical infrastructures. Ex-
timeouts would further enhance the link utilization and lower tensions, such as Multipath-QUIC [49] and QUIC-FEC [50],
the chances of a successful data transfer. bring an improved resiliency to connectivity failures. While
comparative evaluations [51], [52] were mainly conducted for
general purpose hardware, they yield promising results for
collision collision deployments using multiple interfaces with high loss proba-
avoided bilities. Multiple interfaces on the same network hierarchy,
however, are uncommon in industrial IoT deployments.

B. Performance evaluation of CoAP and MQTT

Burst Size [#] Inter Burst Time [ms] Burst Size [#] The performances of CoAP and MQTT have been studied
from several perspectives over the last years [53]–[56]. Very
Fig. 9. Burst size and inter burst time for our generated cross-traffic. early work analyzed the interoperability of specific CoAP

1932-4537 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:09:20 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2021.3089549, IEEE
Transactions on Network and Service Management
12

NDN HoPP CoAP GET (c) CoAP OBS MQTT-SN (Q1)

1.0
Burst Size [Packets]

5 1.3 1.3 1.4 1.9 2.0 1.2 1.3 1.5 1.7 1.9 1.3 1.3 1.5 1.6 1.2 0.9 0.9 0.9 0.8 0.7 1.5 1.7 2.1 2.7 2.9 80

Error Rate [%]

0.8
10 1.3 1.4 1.6 1.8 1.6 1.2 1.2 1.2 1.4 1.4 1.1 1.1 1.1 1.0 0.7 0.8 0.8 0.8 0.7 0.6 2.0 2.2 2.5 3.1 2.9
60
0.6
15 1.3 1.1 1.5 1.7 1.3 1.1 1.1 1.1 1.3 1.0 1.0 1.0 0.9 0.7 0.5 0.7 0.7 0.7 0.6 0.6 2.2 2.3 2.7 3.0 2.8
0.4 40
20 1.2 1.2 1.3 1.5 1.3 0.9 0.9 0.9 1.0 0.9 0.9 0.8 0.7 0.6 0.4 0.7 0.6 0.6 0.6 0.6 2.3 2.4 2.6 2.9 2.7
0.2 20
25 1.2 1.1 1.3 1.2 1.1 0.9 0.8 0.8 1.0 0.7 0.7 0.7 0.6 0.5 0.3 0.7 0.6 0.6 0.6 0.6 2.3 2.5 2.7 2.8 2.7
0.0
0.0 30
0
25 20 15 10 0.2 30 25 20 15 10 0.4 30 25 20 15 10 0.6 30 25 20 15 10 0.8 30 25 20 15 10 1.0
Inter Burst Time [ms]

Fig. 11. Error rate vs. data redundancy for a 1 s publishing interval. Colors encode errors and numbers tell the effective ratio of data packets sent over
uniquely published items.

implementations [57], [58] without performance evaluation. VIII. C ONCLUSIONS AND O UTLOOK
Later, CoAP implementations have been assessed in com- This paper discussed and analyzed current networking solu-
parison to HTTP [59] or on different hardware architec- tions for the constrained Industrial Internet of Things. Starting
tures [60]. MQTT was evaluated in [61] and compared to from the challenging use case of safety-critical sensors and
HTTP in [62]. Rodrı́guez et al. [63] analyze MQTT and HTTP industrial control systems, we derived key requirements for
using TCP/IPv4 as a transport. Thangavel et al. [64] proposed the protocol behavior in a target deployment. Facing these
a common middleware to abstract from CoAP and MQTT. requirements, we deployed and evaluated the three protocol
Based on this middleware, CoAP and MQTT were evaluated families MQTT-SN, CoAP, and ICN in real-world experiments
in a single-hop wired setup. In emulation, MQTT and CoAP with settings characteristic for the IIoT.
have been studied in the context of medical application sce-
Our analysis revealed that the choice of protocol largely
nario [65]. Experimental analyses of MQTT and CoAP run-
impacts the application performance. On the overall, lean and
ning on a hardware simulator (Cooja) have been presented by
simple publish-subscribe protocols such as MQTT-SN and
Martı́ et al. [66], and Proos et al. [67] perform measurements
CoAP Observe are versatile and operate efficiently in relaxed
on a Raspberry Pi. The authors in [68] evaluate implications of
environments with low error rates. Request-response schemes
the radio technology on higher layers protocols. They focus on
hardly meet latency constraints of unscheduled alerts. Even
the cellular 4G technology Narrowband IoT (NB-IoT). Still, a
though reliable, MQTT-SN and CoAP quickly fail in massive
holistic analysis of these protocols in a consistent experimental
multi-hop scenarios, in which NDN and NDN-HoPP can both
setting including many real low-end devices with low-power
enfold strength of hop-wise transfer and reliably deliver data
wireless short range radio technologies is missing.
without the need for significant retransmission rates. MQTT-
SN best withstands degradation from cross-traffic of coexisting
wireless users—at the price of straining the overall resources
C. ICN and the IoT by bursty (re-)transmissions.
With these results, we hope to shed light on the role of
The benefits of ICN/NDN in the IoT have been analyzed networking and to strengthen deployment in the constrained
mainly from three angles. (i) design aspects [69], [70], (ii) IIoT. Our future work will concentrate on progressing dis-
architecture work [8], [71], [72], and (iii) use cases [73]–[75]. tributed IoT applications—facilitated by a robust and versatile
By stacking CoAP on ICN, Islam et al. [71] introduced CoAP Data-centric Web of Things.
as a convergence layer for applications that can run over both Acknowledgment. This work was supported in part by the
networking worlds. Another approach [76] constructs a pure German Federal Ministry for Education and Research (BMBF)
CoAP deployment option that replicates information-centric within the projects I3: Information Centric Networking for the
properties to gain the beneficial effects of ICN and still sustain Industrial Internet and PIVOT: Privacy-Integrated design and
protocol compliance with the CoAP specification [6]. Exper- Validation in the constrained IoT.
imental evaluations are supported by several implementations A Note on Reproducibility. We explicitly support re-
that have become publicly available, including CCN-lite [77] producible research [83], [84]. Our experiments have been
on RIOT [37] and on Contiki [78], and NDN on RIOT [79]. conducted in an open testbed. The source code of our im-
The evaluation of NDN protocol properties in the wild plementations (including scripts to set up the experiments,
includes the exploitation of NDN communication patterns to RIOT measurement apps etc.) will be available on Github at
improve wireless resource management [80] as well as data https://github.com/5G-I3/Impact-Industrial-IoT-2020.
delivery on the network layer [81], [82], which are to a larger
extent reproducible with data-centric CoAP deployments [76]. R EFERENCES
We performed a first comparison with common IoT network
[1] A. Banks and R. G. (Eds.), “MQTT Version 3.1.1,” OASIS, OASIS
stacks in [11]. This paper extends our previous work and Standard, October 2014. [Online]. Available: http://docs.oasis-open.org/
deepens the analyses in the context of the IIoT. mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html

1932-4537 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:09:20 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2021.3089549, IEEE
Transactions on Network and Service Management
13

[2] M. Shafi, A. F. Molisch, P. J. Smith, T. Haustein, P. Zhu, P. De Silva, [24] Y. Cai, Z. Qin, F. Cui, G. Y. Li, and J. A. McCann, “Modulation
F. Tufvesson, A. Benjebbour, and G. Wunder, “5G : A Tutorial Overview and Multiple Access for 5G Networks,” IEEE Communications Surveys
of Standards, Trials, Challenges, Deployment, and Practice,” IEEE Tutorials, vol. 20, no. 1, pp. 629–646, 2018.
Journal on Selected Areas in Communications, vol. 35, no. 6, pp. 1201– [25] K. Hartke, “Observing Resources in the Constrained Application Proto-
1221, 06 2017. col (CoAP),” IETF, RFC 7641, September 2015.
[3] G. Montenegro, N. Kushalnagar, J. Hui, and D. Culler, “Transmission [26] E. Rescorla, “The Transport Layer Security (TLS) Protocol Version 1.3,”
of IPv6 Packets over IEEE 802.15.4 Networks,” IETF, RFC 4944, IETF, RFC 8446, August 2018.
September 2007. [27] G. Selander, J. Mattsson, F. Palombini, and L. Seitz, “Object Security
[4] IEEE 802.15 Working Group, “IEEE Standard for Low-Rate Wire- for Constrained RESTful Environments (OSCORE),” IETF, RFC 8613,
less Networks,” IEEE, New York, NY, USA, Tech. Rep. IEEE Std July 2019.
802.15.4™–2015 (Revision of IEEE Std 802.15.4-2011), 2016. [28] A. Stanford-Clark and H. L. Truong, “MQTT For Sensor Networks
[5] T. Winter, P. Thubert, A. Brandt, J. Hui, R. Kelsey, P. Levis, K. Pister, (MQTT-SN) Version 1.2,” IBM, Protocol Specification, November
R. Struik, J. Vasseur, and R. Alexander, “RPL: IPv6 Routing Protocol 2013. [Online]. Available: http://mqtt.org/new/wp-content/uploads/2009/
for Low-Power and Lossy Networks,” IETF, RFC 6550, March 2012. 06/MQTT-SN\ spec\ v1.2.pdf
[6] Z. Shelby, K. Hartke, and C. Bormann, “The Constrained Application [29] J. Chen, M. Arumaithurai, L. Jiao, X. Fu, and K. Ramakrishnan,
Protocol (CoAP),” IETF, RFC 7252, June 2014. “COPSS: An Efficient Content Oriented Publish/Subscribe System,” in
[7] E. Rescorla and N. Modadugu, “Datagram Transport Layer Security ACM/IEEE Symposium on Architectures for Networking and Communi-
Version 1.2,” IETF, RFC 6347, January 2012. cations Systems (ANCS’11). Los Alamitos, CA, USA: IEEE Computer
[8] E. M. Schooler, D. Zage, J. Sedayao, H. Moustafa, A. Brown, and Society, Oct. 2011, pp. 99–110.
M. Ambrosin, “An Architectural Vision for a Data-Centric IoT: Re- [30] M. Zhang, V. Lehman, and L. Wang, “Scalable name-based data
thinking Things, Trust and Clouds,” in IEEE 37th Intern. Conference synchronization for named data networking,” in IEEE INFOCOM’17,
on Distributed Computing Systems (ICDCS). Piscataway, NJ, USA: ser. INFOCOM ’17. Los Alamitos, CA, USA: IEEE Computer Society,
IEEE, June 2017, pp. 1717–1728. 2017, pp. 1–9.
[9] B. Ahlgren, C. Dannewitz, C. Imbrenda, D. Kutscher, and B. Ohlman, [31] C. Gündogan, P. Kietzmann, T. C. Schmidt, and M. Wählisch,
“A Survey of Information-Centric Networking,” IEEE Communications “HoPP: Robust and Resilient Publish-Subscribe for an Information-
Magazine, vol. 50, no. 7, pp. 26–36, July 2012. Centric Internet of Things,” in Proc. of the 43rd IEEE Conference
[10] L. Zhang, A. Afanasyev, J. Burke, V. Jacobson, k. claffy, P. Crowley, on Local Computer Networks (LCN). Piscataway, NJ, USA:
C. Papadopoulos, L. Wang, and B. Zhang, “Named Data Networking,” IEEE Press, Oct. 2018, pp. 331–334. [Online]. Available: http:
SIGCOMM Comput. Commun. Rev., vol. 44, no. 3, pp. 66–73, 2014. //doi.org/10.1109/LCN.2018.8638030
[11] C. Gündogan, P. Kietzmann, M. Lenders, H. Petersen, T. C. [32] T. C. Schmidt, S. Wölke, N. Berg, and M. Wählisch, “Let’s Collect
Schmidt, and M. Wählisch, “NDN, CoAP, and MQTT: A Comparative Names: How PANINI Limits FIB Tables in Name Based Routing,” in
Measurement Study in the IoT,” in Proc. of 5th ACM Conference Proc. of 15th IFIP Networking Conference. Piscataway, NJ, USA: IEEE
on Information-Centric Networking (ICN). New York, NY, USA: Press, May 2016, pp. 458–466.
ACM, September 2018, pp. 159–171. [Online]. Available: https: [33] J. H. Saltzer, D. P. Reed, and D. D. Clark, “End-to-End Arguments in
//doi.org/10.1145/3267955.3267967 System Design,” ACM Trans. Comput. Syst., vol. 2, no. 4, pp. 277–288,
[12] J. Chen, X. Cao, P. Cheng, Y. Xiao, and Y. Sun, “Distributed Col- Nov 1984.
laborative Control for Industrial Automation With Wireless Sensor [34] X. de Carnavalet and M. Mannan, “Killed by Proxy: Analyzing Client-
and Actuator Networks,” IEEE Transactions on Industrial Electronics, end TLS Interception Software,” in Network and Distributed System
vol. 57, no. 12, pp. 4219–4230, 2010. Security Symposium (NDSS). Internet Society, 2016.
[13] Q. Wang and J. Jiang, “Comparative Examination on Architecture [35] R. Holz, T. Riedmaier, N. Kammenhuber, and G. Carle, “X.509 Foren-
and Protocol of Industrial Wireless Sensor Network Standards,” IEEE sics: Detecting and Localising the SSL/TLS Men-in-the-Middle,” in
Communications Surveys Tutorials, vol. 18, no. 3, pp. 2197–2219, 2016. Computer Security – ESORICS 2012. Berlin, Heidelberg: Springer
[14] International Society of Automation, “Wireless Systems for Industrial Berlin Heidelberg, 2012, pp. 217–234.
Automation: Process Control and Related Applications,” ISA, Tech. Rep. [36] C. Gündogan, C. Amsüss, T. C. Schmidt, and M. Wählisch,
Standard ISA-100.11a-2011, 2011. “IoT Content Object Security with OSCORE and NDN: A First
[15] L. M. Feeney, M. Frey, V. Fodor, and M. Günes, “Modes of inter- Experimental Comparison,” in Proc. of 19th IFIP Networking
network interaction in beacon-enabled IEEE 802.15.4 networks,” in 14th Conference. Piscataway, NJ, USA: IEEE Press, June 2020, pp. 19–27.
Mediterranean Ad Hoc Networking Workshop, MED-HOC-NET. IEEE, [Online]. Available: https://ieeexplore.ieee.org/document/9142731
June 2015, pp. 1–8. [37] E. Baccelli, C. Gündogan, O. Hahm, P. Kietzmann, M. Lenders,
[16] S. B. Yaala, F. Théoleyre, and R. Bouallegue, “Cooperative resynchro- H. Petersen, K. Schleiser, T. C. Schmidt, and M. Wählisch,
nization to improve the reliability of colocated IEEE 802.15.4 -TSCH “RIOT: an Open Source Operating System for Low-end Embedded
networks in dense deployments,” Ad Hoc Networks, vol. 64, pp. 112 – Devices in the IoT,” IEEE Internet of Things Journal, vol. 5,
126, 2017. no. 6, pp. 4428–4440, December 2018. [Online]. Available: http:
[17] A. A. Kumar S., K. Ovsthus, and L. M. Kristensen., “An Industrial //dx.doi.org/10.1109/JIOT.2018.2815038
Perspective on Wireless Sensor Networks - A Survey of Requirements, [38] C. Bormann, M. Ersue, and A. Keranen, “Terminology for Constrained-
Protocols, and Challenges,” IEEE Communications Surveys Tutorials, Node Networks,” IETF, RFC 7228, May 2014.
vol. 16, no. 3, pp. 1391–1412, 2014. [39] C. Bormann, “6LoWPAN-GHC: Generic Header Compression for IPv6
[18] N. Sastry and D. Wagner, “Security Considerations for IEEE 802.15.4 over Low-Power Wireless Personal Area Networks (6LoWPANs),” IETF,
Networks,” in Proc. of the 3rd ACM Workshop on Wireless Security RFC 7400, November 2014.
(WiSe ’04). New York, NY, USA: ACM, 2004, pp. 32–42. [40] C. Gündogan, T. C. Schmidt, M. Wählisch, C. Scherb, C. Marxer, and
[19] S. M. Sajjad and M. Yousaf, “Security analysis of IEEE 802.15.4 MAC C. Tschudin, “ICN Adaptation to LowPAN Networks (ICN LoWPAN),”
in the context of Internet of Things (IoT),” in Conference on Information IRTF, IRTF Internet Draft – work in progress 10, February 2021.
Assurance and Cyber Security (CIACS ’14). IEEE, 2014, pp. 9–14. [Online]. Available: https://tools.ietf.org/html/draft-irtf-icnrg-icnlowpan
[20] T. Watteyne, M. Palattella, and L. Grieco, “Using IEEE 802.15.4e [41] Atmel, Low Power 2.4 GHz Transceiver for ZigBee, IEEE 802.15.4,
Time-Slotted Channel Hopping (TSCH) in the Internet of Things (IoT): 6LoWPAN, RF4CE, SP100, WirelessHART, and ISM Applications,
Problem Statement,” IETF, RFC 7554, May 2015. Atmel Corporation, September 2009. [Online]. Available: http:
[21] X. Vilajosana, K. Pister, and T. Watteyne, “Minimal IPv6 over the TSCH //www.atmel.com/images/doc8111.pdf
Mode of IEEE 802.15.4e (6TiSCH) Configuration,” IETF, RFC 8180, [42] G. Hansch, P. Schneider, and G. S. Brost, “Deriving Impact-Driven
May 2017. Security Requirements and Monitoring Measures for Industrial IoT,”
[22] L. M. Feeney and P. Gunningberg, “Avoiding an IoT ’Tragedy of the in Proc. of the 5th on Cyber-Physical System Security Workshop
Commons’,” in Proc. of the 16th International Conference on Mobile (CPSS’19). New York: ACM, 2019, pp. 37–45.
Systems, Applications, and Services (MobiSys ’18). New York, NY, [43] G. Bernieri, M. Conti, and G. Pozzan, “AMON: An Automaton MONitor
USA: ACM, 2018, pp. 495–497. for Industrial Cyber-Physical Security,” in Proc. of the 14th International
[23] C. Bockelmann, N. Pratas, H. Nikopour, K. Au, T. Svensson, C. Ste- Conference on Availability, Reliability and Security (ARES ’19). New
fanovic, P. Popovski, and A. Dekorsy, “Massive machine-type commu- York: ACM, 2019, pp. 1–10.
nications in 5g: physical and MAC-layer solutions,” IEEE Communica- [44] Modbus–IDA, “Modbus application protocol specification v1. 1b,”
tions Magazine, vol. 54, no. 9, pp. 59–65, 2016. Modbus–IDA, Tech. Rep., 2006.

1932-4537 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:09:20 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2021.3089549, IEEE
Transactions on Network and Service Management
14

[45] M. Nolan, M. J. McGrath, M. Spoczynski, and D. Healy, “Adaptive Selected Topics in Mobile & Wireless Networking (MoWNeT). Piscat-
Industrial IOT/CPS Messaging Strategies for Improved Edge Compute away, NJ, USA: IEEE, 2016, pp. 1–7.
Utility,” in Proc. of the Workshop on Fog Computing and the IoT (IoT- [66] M. Martı́, C. Garcia-Rubio, and C. Campo, “Performance Evaluation of
Fog ’19). New York: ACM, 2019, pp. 16–20. CoAP and MQTT-SN in an IoT Environment,” in 13th Conference on
[46] B. Chun, B. Oh, C. Cho, and D. Lee, “Design and Implementation of Ubiquitous Computing and Ambient Intelligence (UCAmI’19), vol. 31,
Lightweight Messaging Middleware for Edge Computing,” in Proceed- 2019, p. 49.
ings of the 6th International Conference on Control, Mechatronics and [67] D. P. Proos and N. Carlsson, “Performance Comparison of Messaging
Automation (ICCMA ’18). New York: ACM, 2018, pp. 170–174. Protocols and Serialization Formats for Digital Twins in IoV,” in Proc.
[47] L. Eggert, “Towards Securing the Internet of Things with QUIC,” in of 19th IFIP Networking Conference. Piscataway, NJ, USA: IEEE
Proc. of 3rd NDSS Workshop on Decentralized IoT Systems and Security Press, June 2020, pp. 10–18.
(DISS). Internet Society (ISOC), 2020. [68] A. Larmo, A. Ratilainen, and J. Saarinen, “Impact of CoAP and MQTT
[48] J. Iyengar and M. Thomson, “QUIC: A UDP-Based Multiplexed and on NB-IoT system performance,” Sensors, vol. 19, p. 7, 2018.
Secure Transport,” IETF, Internet-Draft – work in progress 34, January [69] W. Shang, Y. Yu, T. Liang, B. Zhang, , and L. Zhang, “NDN-ACE:
2021. Access Control for Constrained Environments over Named Data Net-
[49] Q. D. Coninck and O. Bonaventure, “Multipath Extensions for QUIC working,” NDN, Technical Report NDN-0036, December 2015.
(MP-QUIC),” IETF, Internet-Draft – work in progress 07, May 2021. [70] B. Mathieu, C. Westphal, and P. Truong, “Towards the usage of ccn for
[50] I. Swett, M.-J. Montpetit, V. Roca, and F. Michel, “Coding for QUIC,” iot networks,” in Internet of Things (IoT) in 5G Mobile Technologies.
IETF, Internet-Draft – work in progress 04, March 2020. Cham, Switzerland: Springer, 2016, pp. 3–24.
[51] Q. D. Coninck and O. Bonaventure, “Multipath QUIC: Design and [71] H. M. A. Islam, D. Lagutin, A. Ylä-Jääski, N. Fotiou, and A. V. Gurtov,
Evaluation,” in Proc. of CoNEXT ’17. New York, NY, USA: ACM, “Transparent CoAP Services to IoT Endpoints through ICN Operator
Dec. 2017, pp. 160–166. Networks,” Sensors, vol. 19, no. 6, p. 1339, 2019.
[52] F. Michel, Q. D. Coninck, and O. Bonaventure, “QUIC-FEC: Bringing [72] A. L. R. Madureira, F. R. C. Araújo, G. B. Araújo, and L. N. Sampaio,
the benefits of Forward Erasure Correction to QUIC,” in Proc. of 19th “NDN Fabric: Where the Software-Defined Networking Meets the
IFIP Networking Conference. Piscataway, NJ, USA: IEEE Press, May Content-Centric Model,” IEEE Transactions on Network and Service
2019, pp. 1–9. Management, 2020.
[53] M. Iglesias-Urkia, A. Orive, and A. Urbieta, “Analysis of CoAP Im- [73] J. Burke, P. Gasti, N. Nathan, and G. Tsudik, “Securing Instrumented
plementations for Industrial Internet of Things: A Survey,” Procedia Environments over Content-Centric Networking: the Case of Lighting
Computer Science, vol. 109, pp. 188–195, 2017. Control and NDN,” in Computer Communications Workshops (INFO-
[54] J. Dizdarevic, F. Carpio, A. Jukan, and X. Masip-Bruin, “Survey of COM WKSHPS), 2013 IEEE Conference on. Piscataway, NJ, USA:
Communication Protocols for Internet-of-Things and Related Challenges IEEE, 2013, pp. 394–398.
of Fog and Cloud Computing Integration,” ACM Comput. Surv., vol. 51, [74] M. Amadeo, C. Campolo, A. Iera, and A. Molinaro, “Information
no. 6, pp. 116–1 – 116–29, Jan. 2019. Centric Networking in IoT scenarios: The case of a smart home,” in
[55] D. Koutras, G. Stergiopoulos, T. Dasaklis, P. Kotzanikolaou, D. Glynos, Proc. of IEEE International Conference on Communications (ICC).
and C. Douligeris, “Security in IoMT Communications: A Survey,” Piscataway, NJ, USA: IEEE, June 2015, pp. 648–653.
Sensors, vol. 20, no. 17, p. 4828, 2020. [75] M. Frey, C. Gündogan, P. Kietzmann, M. Lenders, H. Petersen,
[56] N. F. Syed, Z. Baig, A. Ibrahim, and C. Valli, “Denial of service attack T. C. Schmidt, F. Shzu-Juraschek, and M. Wählisch, “Security for the
detection through machine learning for the IoT,” Journal of Information Industrial IoT: The Case for Information-Centric Networking,” in 2019
and Telecommunication, vol. 4, no. 4, pp. 482–503, 2020. IEEE 5th World Forum on Internet of Things (WF-IoT) (WF-IoT 2019).
[57] C. Lerche, K. Hartke, and M. Kovatsch, “Industry adoption of the Piscataway, NJ, USA: IEEE Press, April 2019, pp. 424–429. [Online].
Internet of Things: A constrained application protocol survey,” in Proc. Available: http://dx.doi.org/10.1109/WF-IoT.2019.8767183
17th IEEE International Conf on Emerging Technologies & Factory [76] C. Gündogan, C. Amsüss, T. C. Schmidt, and M. Wählisch, “Toward a
Automation (ETFA). Piscataway, NJ, USA: IEEE, 2012, pp. 1–6. RESTful Information-Centric Web of Things: A Deeper Look at Data
[58] B. C. Villaverde, D. Pesch, R. de Paz Alberola, S. Fedor, and Orientation in CoAP,” in Proc. of 7th ACM Conference on Information-
M. Boubekeur, “Constrained Application Protocol for Low Power Centric Networking (ICN). New York: ACM, September 2020, pp.
Embedded Networks: A Survey,” in Proc. of 6th International Conf 77–88. [Online]. Available: https://doi.org/10.1145/3405656.3418718
on Innovative Mobile and Internet Services in Ubiquitous Computing [77] C. Tschudin, C. Scherb et al., “CCN Lite: Lightweight implementation
(IMIS). Washington, DC, USA: IEEE Computer Society, 2012, pp. of the Content Centric Networking protocol,” 2018. [Online]. Available:
702–707. http://ccn-lite.net
[59] A. Ludovici, P. Moreno, and A. Calveras, “TinyCoAP: A Novel Con- [78] A. Dunkels, B. Grönvall, and T. Voigt, “Contiki - A Lightweight and
strained Application Protocol (CoAP) Implementation for Embedding Flexible Operating System for Tiny Networked Sensors.” in Proc. of
RESTful Web Services in Wireless Sensor Networks Based on TinyOS,” IEEE Local Computer Networks (LCN). Los Alamitos, CA, USA:
J. Sensor and Actuator Networks, vol. 2, no. 2, pp. 288–315, 2013. IEEE Computer Society, 2004, pp. 455–462.
[60] C. P. Kruger and G. P. Hancke, “Benchmarking Internet of things [79] W. Shang, A. Afanasyev, and L. Zhang, “The Design and Implemen-
devices,” in Proc. of 12th IEEE International Conf on Industrial In- tation of the NDN Protocol Stack for RIOT-OS,” in Proc. of IEEE
formatics (INDIN). Piscataway, NJ, USA: IEEE, 2014, pp. 611–616. GLOBECOM 2016. Washington, DC, USA: IEEE, 2016, pp. 1–6.
[61] A. Elmangoush, R. Steinke, T. Magedanz, A. A. Corici, A. Bourreau, and [80] P. Kietzmann, C. Gündogan, T. C. Schmidt, O. Hahm, and M. Wählisch,
A. Al-Hezmi, “Application-derived communication protocol selection in “The Need for a Name to MAC Address Mapping in NDN: Towards
M2M platforms for smart cities,” in Proc. of 18th International Confer- Quantifying the Resource Gain,” in Proc. of 4th ACM Conference on
ence on Intelligence in Next Generation Networks (ICIN). Piscataway, Information-Centric Networking (ICN). New York, NY, USA: ACM,
NJ, USA: IEEE, 2015, pp. 76–82. September 2017, pp. 36–42.
[62] D. Mishra, R. S. Yadav, K. K. Agrawal, and A. Abbas, “Study of Ap- [81] E. Baccelli, C. Mehlis, O. Hahm, T. C. Schmidt, and M. Wählisch,
plication Layer Protocol for Real-Time Monitoring and Maneuvering,” “Information Centric Networking in the IoT: Experiments with NDN
in International Conference on Innovative Computing and Communi- in the Wild,” in Proc. of 1st ACM Conf. on Information-Centric
cations, A. Khanna, D. Gupta, S. Bhattacharyya, V. Snasel, J. Platos, Networking (ICN-2014). New York: ACM, September 2014, pp.
and A. E. Hassanien, Eds. Singapore: Springer Singapore, 2020, pp. 77–86. [Online]. Available: http://dx.doi.org/10.1145/2660129.2660144
439–449. [82] C. Gündogan, J. Pfender, P. Kietzmann, T. C. Schmidt, and M. Wählisch,
[63] J. J. R. Rodrı́guez, J. F. C. Garcı́a, and E. J. A. üello Prada, “Toward “On the Impact of QoS Management in an Information-centric Internet
Automatic and Remote Monitoring of the Pain Experience: An Internet of Things,” Computer Communications, vol. 154, pp. 160–172, March
of Things (IoT) Approach,” in Applied Technologies, M. Botto-Tobar, 2020. [Online]. Available: https://doi.org/10.1016/j.comcom.2020.02.
M. Z. Vizuete, P. Torres-Carrión, S. M. León, G. P. Vásquez, and 046
B. Durakovic, Eds. Cham: Springer International Publishing, 2020, [83] Q. Scheitle, M. Wählisch, O. Gasser, T. C. Schmidt, and G. Carle,
pp. 194–206. “Towards an Ecosystem for Reproducible Research in Computer Net-
[64] D. Thangavel, X. Ma, A. Valera, H.-X. Tan, and C. K.-Y. Tan, “Perfor- working,” in Proc. of ACM SIGCOMM Reproducibility Workshop. New
mance evaluation of MQTT and CoAP via a common middleware,” in York, NY, USA: ACM, August 2017, pp. 5–8.
Proc. of ISSNIP. Piscataway, NJ, USA: IEEE, 2014, pp. 1–6. [84] ACM, “Result and Artifact Review and Badging,” http://acm.org/
[65] Y. Chen and T. Kunz, “Performance evaluation of IoT protocols under publications/policies/artifact-review-badging, Jan., 2017.
a constrained wireless access network,” in International Conference on

1932-4537 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:09:20 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2021.3089549, IEEE
Transactions on Network and Service Management
15

AUTHOR B IOGRAPHY Michael Frey received his M.Sc. in Computer Sci-

ence at RheinMain University of Applied Sciences,
Wiesbaden, Germany in 2010. He is senior software
engineer at Safety IO where he focuses on tackling
challenges bringing ICN to safety-critical environ-
ments. His research interests include routing and
network management in ICN-based IIoT networks.
Cenk Gündoğan received the M.Sc. degree in com-
puter science from the Institut für Informatik, Freie
Universität Berlin, Germany, in 2016. Currently,
he is pursuing the Ph.D. degree with the Internet
Technologies Group, Hamburg University of Ap-
plied Sciences, Germany, and explored within the I3
project—Information Centric Networking (ICN) for
the Industrial Internet—routing, QoS, and resilience
Thomas C. Schmidt is professor of Computer Net-
in ICN-based and IoT tailored networks. Recently,
works and Internet Technologies at Hamburg Univer-
Cenk Gündoğan put focus on a data-centric Web of
sity of Applied Sciences (HAW), where he heads the
Things deployment option by applying ICN prin-
Internet Technologies research group (iNET). Prior
ciples to the IETF envisioned IoT network stack. He is one of the core
to moving to Hamburg, he was director of a scientific
developers and maintainer of RIOT.
computer centre in Berlin. He studied mathematics,
physics and German literature at Freie Universitaet
Berlin and University of Maryland, and received
his Ph.D. from FU Berlin in 1993. Since then he
has continuously conducted numerous national and
international research projects. He was the principal
Peter Kietzmann received the M.Eng. degree in investigator in a number of EU, nationally funded and industrial projects as
information technology from the Hamburg Univer- well as visiting professor at the University of Reading, U.K.. His continued
sity of Applied Sciences, Hamburg, Germany, where interests lie in the development, measurement, and analysis of large-scale
he is currently pursuing the Ph.D. degree with the distributed systems like the Internet. He serves as co-editor and technical
Internet Technologies Research Group. His partic- expert in many occasions and is actively involved in the work of IETF
ular research interest includes low-power radios, and IRTF. Together with his group he pioneered work on an information-
and IoT protocols, many of which he analyzed centric Industrial IoT and the emerging data-centric Web of Things. Thomas
and transformed into code of RIOT. In the German is a co-founder of several large open source projects and coordinator of the
research project I3 (ICN for the Industrial Internet) community developing the RIOT operating system - the friendly OS for the
he explores IoT-based technologies for information Internet of Things.
centric networks and security components.

Felix Shzu-Juraschek is an IoT system architect

for safety products that protect people and facility
Martine S. Lenders is a researcher at the of the infrastructures at Safety io, a subsidiary of MSA The
Internet Technologies research group at Freie Uni- Safety Company. Felix heads MSA’s global connec-
versität Berlin. She obtained both her B.Sc. (2011) tivity steering team to accelerate the connection of
and M.Sc. (2016) in Computer Science at Freie IoT devices and SaaS applications for safety-critical
Universität Berlin with focus on the development environments, such as the fire service and other
and comparison of several IP-based network stacks markets. He graduated from Freie Universität Berlin
for embedded devices. Her research interests include after studying Computer Science and Psychology.
the Internet of Things, Information Centric Network- He received his Ph.D. in Computer Science from
ing, and API design. Accompanying her work as the Humboldt Universität Berlin in 2015.
researcher she is currently pursuing her PhD in those
fields. Martine Sophie Lenders is also one of core
developers of the operating system RIOT and maintains large parts of its
networking infrastructure.
Matthias Wählisch is an Assistant Professor of
Computer Science at Freie Universität Berlin where
he heads the Internet Technologies Research Lab.
He received his Ph.D. in computer science with
highest honors from Freie Universität Berlin. His
research and teaching focus on efficient, reliable,
Hauke Petersen is a research associate at Freie and secure Internet communication. This includes
Universität Berlin, where he is pursuing his Ph.D. the design and evaluation of networking protocols
in the fields of operating systems and networking and architectures, as well as Internet measurements
for constrained devices in the Internet of Things. and analysis. His efforts are driven by improving
After obtaining a B.Sc. in Computer Science from Internet communication based on sound research..
Hochschule Darmstadt and a M.Sc. in Computer Matthias is the PI of several national and international projects, supported by
Engineering from the Technical University of Den- overall 4.7M EUR grant money. He published more than 150 peer-reviewed
mark, he worked as a Systems and Safety engineer papers (e.g., at ACM HotNets, ACM IMC, The Web Conference). Since 2005,
for the German automotive industry, before join- Matthias is active within IETF/IRTF, including eight RFCs and several Internet
ing Freie Universität Berlin. Particularly passion- drafts. His research results have been distinguished multiple times. Amongst
ate about combining real-time control systems with others, he received the Young Talents Award of Leibniz-Kolleg Potsdam for
Internet connectivity and implementing this on very constrained hardware outstanding achievements in advancing the Internet, as well as the Excellent
platforms, Hauke Petersen is one of the core developers and maintainers Young Scientists Award (10,000 EUR) for his contributions to the Internet
of RIOT OS, and published articles on such topics in high-quality venues of Things and their prospective entrepreneurial practice. He co-founded some
including ACM and IEEE conferences. successful open source projects such as RIOT, where he is still responsible
for the strategic development.

1932-4537 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:09:20 UTC from IEEE Xplore. Restrictions apply.