Week 9 Lecture Material PDF
Week 9 Lecture Material PDF
EL
PT Dr. Sudip Misra
N Associate Professor
Department of Computer Science and Technology
IIT Kharagpur
Email: [email protected]
Website: http://cse.iitkgp.ac.in/~smisra/
Introduction to Internet of Things 1
Contents
Introduction to Openstack
Components
Installation
EL
Creating a key‐pair and manage security group
PT
Launce Instances
Creating an image N
Accessing and Communicating with instances
Introduction to Internet of Things 2
Introduction to Openstack
A software to create a cloud unfrastructure
Launched as a joint project of Rackspace Hosting and NASA in 2010
Opensource
EL
Presently many companies are contributing to openstack
Eg. IBM, CISCO, HP, Dell, Vmware, Redhat, suse, Rackspace hosting
PT
It has a very large community
Can be used to develop private cloud or public cloud
N
Versions:
Austin, Bexar, Cactus, Diablo, Essex, Folsom, Grizzly, Havana, Icehouse, Juno,
Kilo, Liberty, Mitaka, Newton, Ocata (Latest)
Introduction to Internet of Things 3
Components
Horizon
Dashboard
EL
Nova Glance Swift
PT
Neutron Cinder Heat Ceilometer Keystone
N
Introduction to Internet of Things 4
Components contd.
Keystone
Identity service
Provides authentication and authorization
EL
PT
Horizon
Dashboard N
GUI of the software
Provides overview of the other components
Introduction to Internet of Things 5
Components contd.
Nova
Compute service
Where you launce your instances
EL
PT
Glance
Image service N
Discovering, registering, retrieving the VM
Snapshots
Introduction to Internet of Things 6
Components contd.
Swift
Object storage
Helps in storing data safely, cheaply and efficiently
EL
PT
Neutron
Provides networking service
N
Enables the other services to communicate with each other
Make your own network
Introduction to Internet of Things 7
Components contd.
Cinder
Block storage
Virtualizes the management of block service
EL
Heat
PT
Orchestration
Ceilometer N
Billing
What service you are using
How long are you using
Introduction to Internet of Things 8
Installation
Can be installed manually or using scripts like Devstack
We will use devstack
EL
Steps:
Install git ( sudo apt‐get install git )
PT
Clone devstack ( git clone https://git.openstack.org/openstack‐
N
dev/devstack )
Go to devstack directory ( cd devstack )
Introduction to Internet of Things 9
Installation contd.
Open local.conf file and paste the following and save the file
ADMIN_PASSWORD=<YOUR PASSWORD>
DATABASE_PASSWORD =<YOUR PASSWORD>
EL
RABBIT_PASSWORD =<YOUR PASSWORD>
SERVICE_PASSWORD =<YOUR PASSWORD>
PT
HOST_IP=<the IP of your PC>
N
Run the stack.sh file ( ./stack.sh)
For uninstallation, go to devstack directory and run unstack.sh file
Introduction to Internet of Things 10
References
1. https://www.openstack.org/
https://docs.openstack.org/developer/devstack/
EL
PT
N
Introduction to Internet of Things 11
EL
PT
N
Introduction to Internet of Things 12
EL
Sensor-Cloud-Part I
Sensor-as-a-Service
PT
Dr. Sudip Misra
Associate Professor
N
Department of Computer Science and Engineering
IIT Kharagpur
Email: [email protected]
Website: http://cse.iitkgp.ac.in/~smisra/
Introduction to Internet of Things 1
Introduction
It is not mere integration of sensors and cloud computing
It is not only “dumping the sensor data into cloud”
EL
PT Cloud
N
Cloud
Introduction to Internet of Things 2
Wireless Sensor Networks (WSNs): Recap
Contain sensor nodes which sense some physical phenomena from the
environment
Transmit the sensed data (through wireless communication) to a centralized
EL
unit, commonly known as Sink node
The communication between Sink node and other sensor nodes in the
PT
network may be single/multi‐hop
Sink node further process data
N
Introduction to Internet of Things 3
Wireless Sensor Networks (WSNs): Recap
Sink Applications
Target Tracking
Sensing unit Wildlife Monitoring
Healthcare
EL
Processing unit Industrial Applications
Communication unit Smart Home
PT
Smart City
Major Components of a Agriculture
Sensor Node
N …
Wireless Sensor Networks
Introduction to Internet of Things 4
Cloud Computing: Recap
An architecture which provides on‐demand computing resources
Advantages
Elasticity: Scaling up/down
EL
Pay‐per‐use: Payment for the resource as per requirement
Self Service: Resource can be accessed by self
PT
N
Introduction to Internet of Things 5
Cloud Computing: Services
Cloud‐Clients
App, Web browser, terminal
Software‐as‐a‐Service (SaaS)
EL
Platform‐as‐a‐Service (PaaS)
PT
N
Infrastructure‐as‐a‐Service (IaaS)
Introduction to Internet of Things 6
Cloud Computing: Services
Software‐as‐a‐Service (SaaS)
A third party provides a host application over internet
Example: Microsoft Office 365
EL
Platform‐as‐a‐Service (PaaS)
Provide a platform to develop and run applications
PT
Example: Windows Azure
Infrastructure‐as‐a‐Service (IaaS)
N
Provide computing resources
Example: Storage space
Introduction to Internet of Things 7
Virtualization Concept
One computer host appears as many computers‐concept of Virtual Machine
(VM)
Improve IT throughput and costs by using physical resources as a pool from
EL
which virtual resources can be allocated.
Benefit
PT
Sharing of resources: Same resource can be shared, in turn cost reduction
Encapsulation: A complete computing environment
N
Independence: Runs independently of underlying hardware
Portability: VM Migration
Introduction to Internet of Things 8
Limitations of WSNs
Change of Requirement
Procurement An example
Price
Right vendor Today Tomorrow
EL
Types of sensor integrated with it
Deployment
PT
Right way of deployment
Right place of deployment Smart Home
N Agriculture
Maintenance
Result: Change in Sensor type, deployment
Post deployment maintenance area, topology design, and many more….
Battery lifetime
Introduction to Internet of Things 9
Sensor-Cloud: Introduction
Not only the mere integration of cloud computing and sensor networks, but
sensor‐cloud is more than that
Concept of virtualization of sensor node
EL
Pay‐per‐use
One sensor node/network appears as many
PT
A stratum between sensor nodes and end‐users
N
Introduction to Internet of Things 10
Difference with WSN
Multiple applications/
WSN user users
Sensor‐cloud
Aggregated data Virtualization
EL
infrastructure
Dedicated to a
PT Serves multiple
single user
N
applications
WSN Sensor‐Cloud
Source: S. Misra; S. Chatterjee; M. S. Obaidat, "On Theoretical Modeling of Sensor Cloud: A Paradigm Shift From Wireless Sensor Network," in IEEE
Systems Journal , vol.PP, no.99, pp.1-10
Introduction to Internet of Things 11
Difference with WSN (Contd.)
Actors and Roles
Attributes WSN Sensor Cloud
Ownership WSN‐user Sensor‐owner
EL
Deployment WSN‐user Sensor‐owner
Redeployment WSN‐user SCSP
PT
Maintenances N WSN‐user SCSP
Overhead WSN‐user SCSP
Usage WSN‐user End‐user
Source: S. Misra; S. Chatterjee; M. S. Obaidat, "On Theoretical Modeling of Sensor Cloud: A Paradigm Shift From Wireless Sensor Network," in IEEE
Systems Journal , vol.PP, no.99, pp.1-10
Introduction to Internet of Things 12
Actors in Sensor-cloud
End‐users
Enjoy Se‐aaS through applications as per the requirements.
Unknown about what and which physical sensor is/are allocated to serve the
EL
application
Sensor‐owner
PT
Plays a role from business perspective.
They purchase physical sensor devices, deployed over different geographical
N
locations, and lend these devices to the sensor‐cloud
Sensor‐Cloud Service Provider (SCSP)
A business actor.
SCSP charges price from the end‐users as per their usage of Se‐aaS.
Introduction to Internet of Things 13
Sensor-cloud: Architecture
End‐users: Registered themselves, selects
templates, and request for application(s)
Sensor‐owner: Deploy heterogeneous/
EL
homogeneous physical sensor nodes over
different geographical location
PT
SCSP: Plays managerial role
N
Introduction to Internet of Things 14
Sensor-cloud: View
User Login
Xml
interpretation
specification
Interaction with
Application 1
Template
Dynamic Scaling physical sensor
On‐demand physical
Data feed Template sensor scheduling
Web portal
display
EL
Vast data storage and Heterogeneous
User Browser specialized processing pool of physical
organization Interface sensors
Sensed Energy
Sensed data
PT
Data feed management, QoS
information
Application
Application 2 specific real‐time
On‐demand
sensor data
N data aggregation
Template specification
User organization view Real View
Source: S. Misra; S. Chatterjee; M. S. Obaidat, "On Theoretical Modeling of Sensor Cloud: A Paradigm Shift From Wireless Sensor Network," in IEEE
Systems Journal , vol.PP, no.99, pp.1-10
Introduction to Internet of Things 15
Work Flow of Sensor-Cloud Source: S. Misra; S. Chatterjee; M. S.
Obaidat, "On Theoretical Modeling of
Sensor Cloud: A Paradigm Shift From
User SensorML Virtual Sensor Virtual Sensor Resource Wireless Sensor Network," in IEEE
organization interpretor Manager Controller Manger Systems Journal , vol.PP, no.99, pp.1-10
Operations
request Create virtual
sensor instance Manage
operations
Response
EL
Response
Data request
XML template Decode
Compatible
PT
sensor Sensor Physical sensor definition,
scheduling, resource pool Virtual sensor
allocation, (WSN) Group definition
N deallocation Client information
Data Data retrieval Metadata
provisioning Data
Templates
aggregation
Delete Release
virtual sensor instance Release
resource
Introduction to Internet of Things 16
Case Study: Target Tracking
EL
intrusion within a particular zone has to deploy its own WSN. This leads to a
long‐term investment due to costly network setup and maintenance
PT
overheads. However, in a sensor‐cloud environment, the same organization
can use the same tracking application and still get the service without actually
N
owning the WSN”
Source: S. Misra; S. Chatterjee; M. S. Obaidat, "On Theoretical Modeling of Sensor Cloud: A Paradigm Shift From Wireless Sensor Network," in IEEE
Systems Journal , vol.PP, no.99, pp.1-10
Introduction to Internet of Things 17
EL
PT
N
Introduction to Internet of Things 18
EL
Sensor-Cloud-Part II
Sensor-as-a-Service
PT
Dr. Sudip Misra
Associate Professor
N
Department of Computer Science and Engineering
IIT Kharagpur
Email: [email protected]
Website: http://cse.iitkgp.ac.in/~smisra/
Optimal Composition of virtual sensor nodes
Data Caching
Optimal Pricing
EL
PT
N
PT
N
Source: S. Chatterjee and S. Misra, “Dynamic Optimal Composition of a Virtual Sensor for Efficient Virtualization Within Sensor‐cloud”,
IEEE ICC 2015.
Efficient virtualization of the physical sensor nodes
An optimal composition of VSs
Consider same geographic region: CoV‐I
EL
Spanning across multiple regions: CoV‐II
PT
N
Source: S. Chatterjee and S. Misra, “Dynamic Optimal Composition of a Virtual Sensor for Efficient Virtualization Within Sensor‐cloud”,
IEEE ICC 2015.
EL
PT
N
Source: S. Chatterjee and S. Misra, “Dynamic Optimal Composition of a Virtual Sensor for Efficient Virtualization Within Sensor‐cloud”,
IEEE ICC 2015.
EL
within same geographical
boundary
PT
N
Source: S. Chatterjee and S. Misra, "Optimal composition of a virtual sensor
for efficient virtualization within sensor‐cloud," 2015 IEEE International
Conference on Communications (ICC), London, 2015, pp. 448‐453
EL
VS2 VS3
nodes across different
geographical locations
PT
N
Source: S. Chatterjee and S. Misra, "Optimal composition of a virtual sensor
for efficient virtualization within sensor‐cloud," 2015 IEEE International
Conference on Communications (ICC), London, 2015, pp. 448‐453
EL
PT
N
Source: S. Chatterjee and S. Misra, "Optimal composition of a virtual sensor for efficient virtualization within sensor‐cloud," 2015 IEEE International Conference on
Communications (ICC), London, 2015, pp. 448‐453
PT
N
Source: S. Chatterjee, S. Misra, “Dynamic and Adaptive Data Caching Mechanism for Virtualization within Sensor‐Cloud”, IEEE ANTS 2014.
Introduces internal and external caching mechanisms
Ensures efficiency in resource utilization
Flexible with the varied rate of change of the physical environment
EL
PT
N
Source: S. Chatterjee, S. Misra, “Dynamic and Adaptive Data Caching Mechanism for Virtualization within Sensor‐Cloud”, IEEE ANTS 2014.
EL
PT
N
Source: S. Chatterjee, S. Misra, “Dynamic and Adaptive Data Caching Mechanism for Virtualization within Sensor‐Cloud”, IEEE ANTS 2014.
EL
unaltered
In such a situation, unnecessary sensing causes energy consumption
PT
N
Source: S. Chatterjee, S. Misra, “Dynamic and Adaptive Data Caching Mechanism for Virtualization within Sensor‐Cloud”, IEEE ANTS 2014.
EL
user or is it required to re‐cache the data from external cache
External Cache (EC)
PT
After every certain interval data are required to re‐cache
Initially, few data are used to be transmitted to IC
N
Source: S. Chatterjee, S. Misra, “Dynamic and Adaptive Data Caching Mechanism for Virtualization within Sensor‐Cloud”, IEEE ANTS 2014.
Sensor‐Cloud
Sensor‐Cloud IC
EL
Resource
EC
PT
pooling
N
Existing Architecture Cache‐enabled Architecture
Source: S. Chatterjee, S. Misra, “Dynamic and Adaptive Data Caching Mechanism for Virtualization within Sensor‐Cloud”, IEEE ANTS 2014.
EL
PT
N
Source: S. Chatterjee, S. Misra, “Dynamic and Adaptive Data Caching Mechanism for Virtualization within Sensor‐Cloud”, IEEE ANTS 2014.
Existing schemes consider homogeneity of service (e.g. for IaaS, SaaS)
No scheme for SeaaS.
The proposed pricing scheme comprises of two components:
EL
Pricing attributed to hardware (pH)
Pricing attributed to Infrastructure (pI)
PT
Goal of the proposed pricing scheme:
Maximizing profit of SCSP
N
Maximizing profit of sensor owner
End users’ satisfaction
Web Portal
EL
PT
Base
N Station Sensor‐Cloud
Pricing and negotiation
Maximizing the profit made by SCSP
Optimal pricing to the end‐users
End users satisfaction
EL
Pricing attributed to hardware (pH)
Deals with usage of physical sensor nodes
PT
Pricing attribute to infrastructure (pI)
Deals with the price associated with infrastructure of sensor‐cloud
N
EL
S. Chatterjee, R. Ladia and S. Misra, "Dynamic Optimal Pricing for Heterogeneous Service‐Oriented
Architecture of Sensor‐Cloud Infrastructure," in IEEE Transactions on Services Computing, vol. 10, no. 2, pp.
PT
203‐216, 2017
S. Chatterjee and S. Misra, "Optimal composition of a virtual sensor for efficient virtualization within sensor‐
cloud," 2015 IEEE International Conference on Communications (ICC), London, 2015, pp. 448‐453
N
S. Misra; S. Chatterjee; M. S. Obaidat, "On Theoretical Modeling of Sensor Cloud: A Paradigm Shift From
Wireless Sensor Network," in IEEE Systems Journal , vol.PP, no.99, pp.1‐10
PT
Dr. Sudip Misra
Associate Professor
N
Department of Computer Science and Technology
IIT Kharagpur
Email: [email protected]
Website: http://cse.iitkgp.ac.in/~smisra/
EL
The primary aim: solve the problems faced by cloud computing during IoT
data processing.
PT
an intermediate layer between cloud and devices.
N
Cloud
EL
Fog
PT
N Device
Fig. Fog as intermediate layer between cloud and device
EL
years.
2.5 quintillion bytes of data is generated per day.
PT
total expenditure on IoT devices will be $1.7 Trillion by 2020
N
EL
The amount of data generated by IoT devices is simply huge.
PT
N
EL
Volume
PT
Latency
Bandwidth
N
Cloud
EL
Sends data Sends back
PT
for analysis command or
and storage action
required
N
Devices
Fig.1: Present day cloud model
EL
Presently billions of devices produce exabytes of data everyday.
Device density is still increasing everyday.
PT
Current cloud model is unable to process this amount of data.
N
Private firms, Factories,
airplane companies produces Cloud
EL
colossus amount of data
Storing data
everyday
PT
Current cloud model cannot
store all these data
Data need to be filtered
N
Private firms Factories
Airplane firms
EL
If edge devices send time sensitive data to cloud for analysis and wait
for the cloud to give a proper action, then it can lead to many
PT
unwanted results.
While handling time sensitive data, a millisecond can make a huge
N
differences.
EL
+
where T = Time
PT
Sending time
Latency will be increased sensitive data for Appropriate
analysis action
When the action reaches the device,
N
accident may have already
occured
EL
If all the data generated by IoT devices are sent to cloud for storage
and analysis, then, the traffic generated by these devices will be
PT
simply gigantic.
consumes almost all the bandwidths.
N
Handling this kind of traffic will be simply a very hard task.
Billions of devices consuming bandwidth Cloud
If all the devices become online even IPv6
EL
will not be able to provide facility to all
the devices
PT
Data may be confidential which the firms Sending data for
analysis and Appropriate
do not want to share online
N storage action
EL
A minute delay while taking a decision makes a huge difference
Latency can be reduced by analyzing the data close to the data source
PT
N
EL
Data are required to be monitored 24x7
An appropriate action should be taken before the attack causes major
PT
harm to the network
N
EL
problem
Integrity and availability of the data must be guaranteed
PT
Unavailability and tampering of data can be hazardous
N
EL
less time sensitive data
data which are not time sensitive
PT
Extremely time sensitive data should be analyzed very near to the data
source
N
Data which are not time sensitive will be analyzed in the cloud.
EL
geographical region
E.g. monitoring the railway track of a country or a state
PT
the devices are exposed to the harsh environments condition
N
EL
If the devices are separated by a large geographical distance
If the devices are needed to be subjected to extreme conditions
PT
N
PT
Dr. Sudip Misra
Associate Professor
N
Department of Computer Science and Technology
IIT Kharagpur
Email: [email protected]
Website: http://cse.iitkgp.ac.in/~smisra/
EL
many fog nodes can be present
Sensor data are processed in the fog before it is sent to the cloud
PT
Reduces latency, save bandwidth and save the storage of the cloud
N
EL
PT
N
EL
Computing facility
‐ To process the data before it is sent to cloud
PT
‐ To take quick decisions
Network connectivity ‐ To connect with IoT devices, other fog nodes
N
and cloud
EL
Each fog nodes have their aggregate fog node.
PT
N
EL
Less time‐sensitive data
Data which are not time‐sensitive
PT
Fog nodes works according to the type of data they receive.
An IoT application should be installed to each fog nodes
N
Cloud
Sends the summary for
historical analysis and
Non‐time‐sensitive
EL
storage
data
Nearest Sends the summary for
PT
Fog Node Less time‐sensitive historical analysis and
data storage
If time‐sensitive
Ingest data
N
data then take Aggregate fog
immediate action node
Devices
Action
Fig : Working of fog
EL
Data which should be analyzed within fraction of a second
Analyze at the nearest node itself
PT
Sends the decision or action to the devices
Sends and stores the summary to cloud for future analysis
N
EL
Are sent to the aggregate node for analysis
After analysis, the aggregate node send the decision or action to the
PT
device through the nearest node
The aggregate node sends the summary to cloud for storage and
N
future analysis.
EL
Sent to cloud for storage and future analysis.
Those summaries from fog nodes can be considered as less time
PT
sensitive data.
N
EL
devices
PT
IoT data storage Transient Hour, days Months to years
duration
N
Geographical Very local Wider Global
coverage
EL
Fog nodes can use the same security policy
Low operation cost
PT
Data are processed in the fog nodes before sending to cloud
Reduces the bandwidth consumption
N
EL
Quick decision making
Better privacy
PT
Every industry can analyze their data locally
Store confidential data in their local servers
N
Send only those data which can be shared to the cloud
EL
Can be deployed anywhere we need
Can be programed according to the customer’s need
PT
Support mobility
Nodes can be mobile
N
Nodes can join and leave the network anytime
EL
Can be subjected to harsh environmental conditions
Under sea, railway tracks, vehicles, factory floor etc
PT
Better data handling
Can operate with less bandwidth
N
Data can be analyzed locally
Reduce the risk of latency
EL
Stroke patients
Analyze the data real time
PT
During emergency, alerts the respective doctors immediately
Historical data analysis can predict future dangers of the patient
N
EL
Reports detail power consumption report everyday
Suggest economical power usage plan
PT
N
EL
Real time monitoring of the track conditions
For high speed train, sending the data in cloud for analysis is inefficient
PT
Fog nodes provide fast data analysis
Improve safety and reliability
N
EL
Real time monitoring of pressure, flow, compressor is necessary
Terabytes of data are created
PT
Sending all this data to cloud for analysis and storage is not efficient
Network latency is not acceptable
N
Fog is a solution
EL
Data can be monitored real time
PT
N
EL
Data Security
Data generating nodes are distributed
PT
Providing authentication and authorization system for the whole nodes is
not an easy task N
Reliability
Maintaining data integrity and availability for millions of nodes is difficult
failure of a node cannot affect the network
EL
Individual failure should not affect the whole scenario
Real time analysis
PT
Real time analysis is a primary requirement for minimizing latency
Dynamic analysis and decision making reduces danger and increase output
N
Monitor huge number of nodes is not easy
EL
Nodes can connect and leave the network when necessary
Many data processing frameworks are statically configured
PT
These frameworks cannot provide proper scalability and flexibility
N
EL
Benefits extends from an individual person to huge firms
Provides real time analysis and monitoring
PT
N
EL
R System white paper, Fog Computing for Big Data analytics, October 2016
Redowan Mahmud,Rajkumar Buyya, Fog Computing: A Taxonomy, Survey and future
PT
Directions,Cornell University Library, November 2016
http://www.businessinsider.in/THE‐INTERNET‐OF‐EVERYTHING‐2015‐SLIDE‐
DECK/articleshow/45695215.cms
N
http://www.gartner.com/newsroom/id/2970017
https://www.uschamberfoundation.org/bhq/big‐data‐and‐what‐it‐means
https://www.afcea.org/content/?q=node/12239