IoT Architecture

Special Topic: Internet of Things

Rudra Dutta
Computer Science, NCSU
 IoT
• A system of intercommunicating computing, data stores, and
devices embedded in the physical world (with some non-ICT
purpose but capable of some degree of
sensing/actuating/processing/storage), fulfilling some
purpose that could not be achieved by some subset of system
• Architecture
– What components in an IoT system?
– How do they interact?
• Technology / protocols
– What is an IoT technology?
– What is an IoT protocol?
– What is an IoT platform?
– What is an IoT application?

Copyright Spring 2017, Rudra Dutta, NCSU 2

 Sample Usecase
• Consider the following usecase
– Bicyclists and pedestrians with GPS, connected
heart-rate/bp/blood-oxy meters, augmented reality
glasses; stationary and mobile connected air quality
monitors
– When they turn to look in some direction, AR display
provides detailed/digested information about personalized
risk by smart integration of data from sensors, historical
data of health incidents of users correlated with historical
sensor data
• What does it take to get this done?

Copyright Spring 2017, Rudra Dutta, NCSU 3

Architecture – What is it
• Underlying structure that bridges intent and design
– What something needs to do and how to do it
– Actual design is distinct from architecture
• Typically same architecture can be realized by many designs
• Entities and interactions
– Potentially at multiple levels
• Entities are separated by jurisdiction/policy
• Interactions define entity capabilities/functions
– Must be standardized 🡪 protocols
– Define what, not how

Copyright Fall 2016, Rudra

 Networking
• A network is a distributed system
• Main complexity / variety comes from “boxes”
– Routers, switches, bridges, gateways, …
– Links are communications devices rather than
networking
• A network “box” is a special purpose
computer
– Use: forward data packets
– But need to configure, and possibly program
– May be static
Copyright Spring 2017, Rudra Dutta, NCSU 5
Network Operation
• Much of network operation comes from what routers
(forwarding engines) do
– “They forward packets”
• What routers are asked to do
– Architectural question: affects many entities
• How routers do what they do
– Implementation question: vendor-specific smarts, black-box solutions
• To have an impact on performance, must change either the
How or the What
– And trying to change the What will require
changing/re-doing/re-examining the How also

Copyright Fall, 2016, Rudra Dutta, NCSU

Ethernet – Example of Why/How
• Originally a wrapper around CSMA/CD
• A station does not transmit on busy carrier
– Uninterrupted packet transmissions can occur,
even with high traffic arrival rate
• A station stops transmission on sensing
collision
– Time wasted in contention is reduced

Copyright Fall, 2016, Rudra Dutta, NCSU

Ethernet
10101010 (7 times) EtherType/
Dest. address Length FCS
Preamble
... ... ... ...

SFD Source address Payload

46 ≤ ≤ 1500
10101011

• Classic Ethernet
• Type/Length duality
– Value ≤ 1500 🡪 payload length (bytes)
– Value ≥ 1536 🡪 protocol encapsulated
• Organizationally Unique Identifier in addresses (first 3 octets)
– For globally unique addresses (bitflag in OUI part)
• Maximum and minimum wire lengths
• Binary exponential backoff
• Reception by noting destination address
• 12 octet interframe gap
Copyright Fall, 2016, Rudra Dutta, NCSU
Ethernet Hub

• Instead of long distributed medium and short tap-offs, a

short localized “medium”-in-a-box, and long “tap-off”s
– Convenience in installation
– But a major philosophical shift
• Standard defines how stations send and transmit on
medium, not how “medium” behaves
– Assumption: like a wire – but not required for compliance
Copyright Fall, 2016, Rudra Dutta, NCSU
Switched Ethernet

• “Hub” contracted long wire into a box

• “Switch” replaces short wire in box with a
switchable network
– No collisions – no MAC!
• Supports simultaneous transmissions
– But needs buffering Copyright Fall, 2016, Rudra Dutta, NCSU
Infinite Diversity Allowed
• “… while SIP typically is used over UDP or TCP, it could, without technical
changes, be run over IPX, or carrier pigeons, ATM AAL5 or X.25, in rough
order of desirability” (RFC 2458)
• RFC 1149: A Standard for the Transmission of IP Datagrams on Avian
Carriers
– “Avian carriers can provide high delay, low throughput, and low altitude
service”

Architectural change?
Copyright Fall 2016, Rudra
Architectural Change
• When the scope of the system, or entity within
system changes
• Either way, requires new protocols
– Old protocols may exist unchanged, require change, or
become unnecessary
• Usual drivers
– System scope change
• The need/desire for more automation
• In turn driven by the need for speed (or other desirable metric), or
reducing ennui
– Entity separation or re-factoring
• Economics
Copyright Fall 2016, Rudra
Example: IP Address Configuration
• Originally static, and manual
– Independently at each host that needs to connect
• Desire to automate
– Reducing ennui for admin (and ensuing errors)
– Diskless workstations
– Scale
• BUT: different authority for configuring different sets of hosts
– Solution must be scoped for that authority
– Possibly replicated, possibly with variations, in other scope
• RARP 🡪 BOOTP 🡪 DHCP

Copyright Fall 2016, Rudra

 DHCP: Dynamic Host Configuration Protocol
DHCP overview:
– host broadcasts “DHCP discover”
Goal: dynamically obtain an IP address msg
from network server
– DHCP server responds with “DHCP
• Allows reuse of addresses offer” msg
• Support for mobile users – host requests IP address: “DHCP
• Critical for ISPs request” msg
– DHCP server sends address:
Relay agent on “DHCP ack” msg
every LAN

DHCP
request
Copyright Fall 2016, Rudra Dutta, CSC, NCSU
Architectural Evolution
• Interactions represent constraints on entities
– Entities must present correct interface for other
entities to be able to inter-operate
• “Constrain to de-constrain” (John Doyle,
Caltech)
– Defining what an entity must do allows freedom in
how to do it
– Can create new entities and interactions that
would have had no place in earlier architecture

Copyright Fall 2016, Rudra

 IoT Architecture
• “Physical” components / capabilities
– Sensors / Actuators
– Compute, store, communicate data
• Additional “logical” components
– Security and dependability composition
– Time bound composition
– Cross-ownership service composition
– Policy negotiation and governance
– Federated orchestration
• Objectives
– Predictable scalability, stability, correctness, time-to-complete

Copyright Spring 2017, Rudra Dutta, NCSU 16

 IoT as Confluence

* Reproduced from Luigi Atzori, Antonio Iera, Giacomo Morabito, "The Internet of Things: A survey”.
Computer
Copyright Networks,
Rudra Dutta, CSC, NCSU, Volume
Spring 201654, Issue 15, October 2010, Pages 2787 - 2805. 17
 IoT as Confluence
• What examples do the authors give to show
that each vision cannot individually, or in
pairs, capture the real IoT vision completely ?
• Can running example (air quality health
advising) be seen as adequately covered by
the confluence of the three visions?
• Can you think of an example that is not
sufficiently covered even by the confluence of
all three?

Copyright Spring 2017, Rudra Dutta, NCSU 18

 Cyber-Physical Systems
• “Cyber-physical systems integrate sensing, computation,
control and networking into physical objects and
infrastructure, connecting them to the Internet and to each
other.” – NSF
• “The next generation of engineered systems that require tight
integration of computing, communication, and control
technologies to achieve stability, performance, reliability,
robustness, and efficiency in dealing with physical systems of
many application domains.” [KimKumar 2012]
• So… same as IoT, then?
– Wait, “control”…

Kyoung-Dae Kim and P. R. Kumar, "Cyber-Physical Systems: A Perspective at the

Centennial". Proceedings of the IEEE, Vol.100, Special Centennial Issue, Pages 1287-1308, May 2012.

Copyright Rudra Dutta, CSC, NCSU, Spring 2016 19

 Control System
• A “plant” evolves according to applicable
physical laws or phenomena, and some
“input”
• A “governor” / controller ensures some target
characteristic of the plant state and output, by
observing output, and modifying input
• Digital/hybrid control system: controller is
wholly or partly a computer/algorithm
• Networked control system: controlling
computer is remote
Copyright Rudra Dutta, CSC, NCSU, Spring 2016 20
Control System

Plant

Governor

Copyright Spring 2017, Rudra Dutta, NCSU 21

 Classical Control
• Feedback control
– Controller may have a model of plant
– Even in absence of model uncertainty, setpoints can be
achieved
– Stable systems can be made from underlying inherently
less stable (or unstable) plants
• Transfer functions
– What plant (system) does under given reference inputs
– If system is linear, can superpose
– If system is time-invariant, can dispense with history
– Laplace transforms as an easier way to analyze and design

Copyright Spring 2017, Rudra Dutta, NCSU 22

 PID Controllers
Plant

P
I
D

• P, I, and D of error function superposed to create

correction input 🡪 general class of controllers, tuning
consists of picking three constants
– Pure (especially) D may not be realizable or desirable 🡪
introduce smoothing / lag

Copyright Spring 2017, Rudra Dutta, NCSU 23

 Generalization
• Non-linear system control
• Analysis: Laplace and Z transforms, Bode plots,
Nyquist plots, Lyapunov stabilty criteria, …
• Digital control, hybrid control, hierarchical control
• Main concepts relevant to context
– Observability, controllablity
– Model identification, adaptation, robustness
– Decentralized control
– Delay (lag), stability, relation thereof

Copyright Spring 2017, Rudra Dutta, NCSU 24

 Back to IoT / CPS
• A controller that “closes the loop” in the
physical world by using computation in
cyberspace
• Plant may be (is likely) distributed
• Controller may be (is likely) distributed
• Perforce, must be real-time (for some
definition of real-time)

Copyright Rudra Dutta, CSC, NCSU, Spring 2016 25

 CPS Example
• HVAC example previously discussed
• Autonomous cars?
• Running example: air quality?

• Observation: arbitrary time-bounds arise from

desired performance, less arbitrary
time-bounds arise from stability criteria

Copyright Spring 2017, Rudra Dutta, NCSU 26

 IoT
• A system of intercommunicating computing, data stores, and
devices embedded in the physical world (with some non-ICT
purpose but capable of some degree of
sensing/actuating/processing/storage), fulfilling some
purpose that could not be achieved by some subset of system
• Architecture
– What components in an IoT system?
– How do they interact?
• Technology / protocols
– What is an IoT technology?
– What is an IoT protocol?
– What is an IoT platform?
– What is an IoT application?

Copyright Spring 2017, Rudra Dutta, NCSU 27

 IoT
• A system of intercommunicating computing, data stores, and
devices embedded in the physical world (with some non-ICT
purpose but capable of some degree of
sensing/actuating/processing/storage), fulfilling some
purpose that could not be achieved by some subset of system
– With multiple owners, interests
– With dynamic membership
– With “real-time” bounds
• What is an IoT platform?
• What is an IoT application?

Copyright Spring 2017, Rudra Dutta, NCSU 28

 Back to IoT Architecture
• Remember: entities and interactions
– How to enable some interactions can give rise to other entities (example of
DNS)
– Expansion/refactoring 🡪 architectural change
• Entities: Users/Pro-sumers (agents)
– Represent sense/actuate/transmit/transform/store capabilities
– Represent goals, translating to need for same
– Represent intent/policy to collaborate to achieve goal
• Interactions (need interfaces/protocols):
– Rendezvous
• Declare intents, capabilities
• Declare trust basis if any
– Exchange and negotiate requests for specific service
• Agree on channels for service exchange
– Perform functionality/service
– Sign-off
Copyright Spring 2017, Rudra Dutta, NCSU 29
 Entity Aggregation
• Entities represent roles that different stakeholders (with
different interests) can play
– Inter-entity interactions requires interface that both (all) agree on 🡪
standardized protocol
• If one entity performs multiple (standardized) roles, it must
realize every standardized interface at “external” edges, but
may ignore standardized interfaces at “internal” edges

Copyright Spring 2017, Rudra Dutta, NCSU 30

 Entity Aggregation
• Entities represent roles that different stakeholders (with
different interests) can play
– Inter-entity interactions requires interface that both (all) agree on 🡪
standardized protocol
• If one entity performs multiple (standardized) roles, it must
realize every standardized interface at “external” edges, but
may ignore standardized interfaces at “internal” edges

Copyright Spring 2017, Rudra Dutta, NCSU 31

 Open Interfaces vs. Closed Silos

• An entity may attempt to maintain external standard

interfaces, while speaking proprietary value-added interfaces
– Faster/better growth of efficient proprietary interfaces 🡪 silo’d
solutions
• The previous architectural discussions pertain to an open,
collaborative, multi-interest IoT vision
Copyright Spring 2017, Rudra Dutta, NCSU 32
 Open Interfaces vs. Closed Silos

• An entity may attempt to maintain external standard

interfaces, while speaking proprietary value-added interfaces
– Faster/better growth of efficient proprietary interfaces 🡪 silo’d
solutions
• The previous architectural discussions pertain to an open,
collaborative, multi-interest IoT vision
Copyright Spring 2017, Rudra Dutta, NCSU 33
 Open Interfaces vs. Closed Silos

• An entity may attempt to maintain external standard

interfaces, while speaking proprietary value-added interfaces
– Faster/better growth of efficient proprietary interfaces 🡪 silo’d
solutions
• The previous architectural discussions pertain to an open,
collaborative, multi-interest IoT vision
Copyright Spring 2017, Rudra Dutta, NCSU 34
 Platforms and Applications
• Traditional computing models: applications execute on
platforms
– Platform is “resource” for application to utilize
• Early computing: applications (programs) ran on “whole
computer” (with time-sharing)
• OS 🡪 fine-grain sharing (multi-processing illusion) of
processor and storage
– “System program” is part of platform for “application program” 🡪
Program-as-a-Platform
– Later: virtualization 🡪 multiple levels
• Client-server
– Distributed application program
– Later: P2P 🡪 multiple end-points
– Still only at network endpoints, still specific underlying hardware
Copyright Spring 2017, Rudra Dutta, NCSU 35
 Platforms and Applications
• AFS/NFS (storage), map-reduce (compute)
– Seamless distribution over underlying hardware
– Distribution capability now commoditized, part of platform
• SFC/NFV approaches
– Multi-point “within the network” compute storage
available to application as platform
• Platform vs. application
– Platform: general, multiuser, multi-mission
– Application: user-, mission-, and situation-specific
– Application still has the connotation of only software, but
real distinction is provider-consumer boundary

Copyright Spring 2017, Rudra Dutta, NCSU 36

 Data
• Originally, internal to programs
– Systems program (OS) kept its own data
– Application program kept and crunched its own data
– Exchange of data: solely as archival/retrieval (use of
storage platform): “meaning” only for one program
(internal)
• Increasingly, data can be seen as resource
– Metadata allows “meaning” to be integrated into data
– Producer of data may be considered owner
– But owner can trade usage right, not just use
– Right-to-use is not same as right-to-resell

Copyright Spring 2017, Rudra Dutta, NCSU 37

 Missions and Coalitions
• Concept of applications and platforms 🡪 less defined
in dynamic multi-owner multi-technology context
• Consider alternate concepts
• Coalition
– Transient grouping of users and capabilities
– Characterized by (and formed for) mission
• Mission
– Functional intent of a coalition
– Triggered and defined by some user (agent)
• Mission executes on coalition

Copyright Spring 2017, Rudra Dutta, NCSU 38

 IoT System Mini-Project

Architecture, Application, Implementation

Copyright Spring 2017, Rudra Dutta, NCSU 39

 “Okay, people, listen up…”
• Achieve
– Coalition formation (and dissolution)
– Mission negotiation and execution (and verification)
• Using
– Semantics
– Sensing/Actuating capabilities
– Analytics
– Networking/Orchestration
• Achieving
– Real-time bounds
– Multi-ownership (governance and policy, including trust)
– Security

https://youtu.be/ry55--J4_VQ?t=38s
Copyright Spring 2017, Rudra Dutta, NCSU 40
 Expertise Groups
• Semantics
– Decide: what concepts need to be represented
• Anything that needs to be understood by more than one entity
– What reasoning rules to be shared (standardized)
– “Data model” only, not platform (technology embedding)
– Implement in technology picked by orchestration group
• Sensing/Actuating capabilities
– What sensors, actuators (pick actual hardware/software)
– Implement capabilities to the edge of entity (using orchestration tools)
• Analytics
– Decide analytics methods to be applied, core/edge decomposition
– Implement in technology picked by orchestration group
• Networking/Orchestration
– Decide platform for edge/core cloud analytics, semantics
• With advice from expertise groups
– Decide and implement communication/networking
Copyright Spring 2017, Rudra Dutta, NCSU 41
 Mini-project Definition
• Running example of cooperative air-quality
based health advisory for bicyclists
• MUST ACTUALLY WORK
– We will change elements as necessary for
feasibility
– Consider testing/verification feasibility as well as
implementation/execution feasibility
• (Can you change air pollution anywhere on order?)
• Keep within (Centennial) campus
• All open technology/interfaces
Copyright Spring 2017, Rudra Dutta, NCSU 42
 Picking Expertise Groups
• Fill out Google form to indicate your ranked choice of
groups you want to be part of
• For your first (and optionally second) choice, provide
reasoning for why you are suitable to contribute well
to that group
• For your last choice, provide reasoning for why you
are not suitable to contribute well to that group
• Balance prior strength and learning objectives
• Remember: both design and implmentation aspects

Copyright Spring 2017, Rudra Dutta, NCSU 43

 Architectural Decisions
• Reminder: “How to enable some interactions can
give rise to other entities (example of DNS)”
– Need to make some architectural decisions up front to
ensure pieces will fit together
• Some key choices center around user’s agents
– Whether partitioned/replicated
– If partitioned, whether referral is iterative or recursive (or
either, or both)
• If any “directory services” (NEW ENTITY)
– Whether local/global

Copyright Spring 2017, Rudra Dutta, NCSU 44

 Go
https://youtu.be/f6F6MzMT2g8?t=2s

• Questions welcome
• Seed thoughts/questions for each group

Copyright Spring 2017, Rudra Dutta, NCSU 45