![Gartner: “IoT + Big Data” 2020
projection
“In 2009 there were 0.9 billion sensors and 1.6 billion personal devices …”
“[That’s] roughly 2.5 billion things that were connected. By 2020, that will
grow to become 30 billion things.
This uber-connectedness combined with the analytics capabilities promised by the
big data technology trend will create new value for all organizations, spearheading
an economic boom
“Gartner projects that
the total economic
value-add for the
Internet of things will be
$1.9 trillion dollars by
2020”
http://www.datanami.com/datanami/2013-10-09/gartner:_internet_of_things_plus_big_data_transforming_the_world.html](https://image.slidesharecdn.com/2013-iswc-webofthings-131021152049-phpapp01/85/ISWC-2013-Tutorial-on-the-Web-of-Things-31-320.jpg)
![Example: Stream summarization by incremental hierarchical clustering
The goal is to maintain summary of data from stream in a form of a
taxonomy of prototype clusters – each new events updates the taxonomy
In the example below, events are
Reuters news articles coming in time,
Colors represent topics
[Blaz Novak, 2008]](https://image.slidesharecdn.com/2013-iswc-webofthings-131021152049-phpapp01/85/ISWC-2013-Tutorial-on-the-Web-of-Things-104-320.jpg)
![Example: Stream processing on
sliding window
Database
11.3 9.0
6.0
• Indexing of window data segments
40
12.3 10.0 11.0
20
40
Extract a
Signal
2.3
Fourier
Transform
0
20
0
9.0
-20
-20
1.0
-40
-40
-60
-60
-80
-80
0
0
500
1000
1500
2000
2500
3000
• Query/Template Pattern Preprocessing
500
1000
40
20
1500
2000
2500
3000
3500
4000
Fourier
Transform
4500
3500
Matching with one
of distance
metrics – if data
and query are similar,
trigger an alert
4000
4500
5000
5000
10.0 9.5
60
0
-20
[Faloutsos, 1994]
Slide taken from [Pramod Vemulapalli: Pattern Matching with Acceleration Data]](https://image.slidesharecdn.com/2013-iswc-webofthings-131021152049-phpapp01/85/ISWC-2013-Tutorial-on-the-Web-of-Things-105-320.jpg)
![Public lighting control: greener
lights + control
Power consumption vs. dimming level
40.0
35.0
30.0
25.0
20.0
15.0
10.0
5.0
0.0
SGA LSL 30 main characteristics
• the number of LED: 30*1w
• consumption: 35W (at full power)
• colour of the light: 4200K
• light current: 2700lm
• life-expactancy: min. 60.000h
• IP66
• NET mass: 4,8k
0.3
10.7
21.2
31.7
42.2
52.8
63.3
73.9
84.4
94.9
100
118
Dimming
Power
level
consumption
[%]
[W]
0,3
7,4
10,7
8,4
21,2
13,8
31,7
17,9
42,2
21,6
52,8
25,2
63,3
28,5
73,9
31,5
84,4
34,2
94,9
36,4
100
37,2](https://image.slidesharecdn.com/2013-iswc-webofthings-131021152049-phpapp01/85/ISWC-2013-Tutorial-on-the-Web-of-Things-118-320.jpg)
![Example queries
Easy and fast detection of
events on current state data
(very simple rules)
Fog forming example
If
(humidity[AVG,1h] < 90%) &
(humidity[AVG,10m] > 95%)
Then
trigger fog forming risk event.
Simple validation of more
complex event queries (using
current state and previous
aggregates)
Road Icing example
If
(precipitation[SUM,12h,6h ago] > X) &
(temparature[MAX,12h,6h ago] > 0) &
(temperature[MIN, 6h]) < 0)
Then
trigger road icing risk event.
Can handle time queries
Time example
If
(temperature[AVG,1w,3d ago] <-5) &
(temperature[AVG,24h,2d ago] < 5) &
(temperature[AVG,24h,1d ago] < 5) &
Then
trigger lake still frozen event.](https://image.slidesharecdn.com/2013-iswc-webofthings-131021152049-phpapp01/85/ISWC-2013-Tutorial-on-the-Web-of-Things-167-320.jpg)
ISWC 2013 Tutorial on the Web of Things
- 1. The web of things CAROLINA FORTUNA AND MARKO GROBELNIK [email protected] [email protected] JOŽEF STEFAN INSTITUTE, LJUBLJANA, SLOVENIA HTTP://SENSORLAB.IJS.SI
- 2. L’Internet des Objets, SVJ 268 2 The WoT explained to children
- 3. Use case: Connected toys (1/2) • A start-up produces a device that can be used to remotely: • • • • 3 watch your pet using the embedded camera talk to your cat using the embedded speakers hear your pet using the embedded microphone play with your pet using the embedded laser pointer • record and share pictures, etc. • The remote interaction is enabled through a smartphone and uses web 2.0 and internet technologies
- 4. Images from http://petcube.net/ 4 Use case: Connected toys (2/2)
- 5. Use case: light control • Control the lights in your home by • Installing new switches • Installing the control app on the smart phone • The switched and the phone use wireless to communicate 5 Images from http://www.pluggxlabs.com/
- 6. Use case: disaster monitoring (1/2) http://en.wikipedia.org/wiki/2011_Tōhoku_earthquake_and_tsunami http://wikileaksreputationcrisis.wordpress.com/2011/03/13/fukushima-nuclear-crisis-media-coverage-evolution/ 6 Facts: • March 11, 2011: Tōhoku earthquake and tsunami in Japan • Nuclear reactors were affected: explosions and radioactive pollution • Confusing information about the levels of radioactivity from authorities • Radiation level maps based on Geiger counter data started to appear
- 7. http://blog.pachube.com/2011/03/real-time-radiation-monitoring-in-japan.html 7 Use case: disaster monitoring (2/2) – the Radiation level map
- 8. Disclaimer This is by no means an exhaustive list of use cases … The main purpose of the list was to show that now, unlike few years ago when we started this tutorial, using Web of Things technologies had been made simple by innovative products. 8 A nice visualization of the use cases is available at: http://www.beechamresearch.com/article.aspx?id=4
- 9. Outline Part I. Motivation & background Part II. Technology and tools for exploiting the WoT 9 Part III. Demos, tools & research directions
- 10. Part I. Motivation & background outline 10 Web Of Things • What is it? What problems can it solve? Architectural considerations • How it looks like? What are its components? The “Things” • What are the ingredients? The “Glue” • How do things stick together? Applications and services • What can be built on top of it? Quick start recipes • How does the “Hello World!” look like?
- 11. What is the Web of Things? In order to answer the question we will look at the following differences: • The Internet vs the Web • The Internet of Things vs the Internet • Web of Things vs the Web 11 • Web of Things vs the Internet of Things
- 12. Quick history: the “old Internet” IPv4 IPv4 IPv4 IPv4 IPv4 IPv4 IPv4 IPv4
- 13. Quick history: The “contemporary Internet” (mobile) IPv4 (mobile) IPv4 IPv4 (IPv6) IPv4 (IPv6) IPv4 (IPv6) IPv4 (IPv6) (mobile) IPv4 (mobile) IPv4
- 14. The Internet as we know it www.facebook.com www.amazon.com ….. This is actually not the Internet….this is the Web. It’s HTTP (over IP).
- 15. What is the Web of Things? In order to answer the question we will look at the following differences: • The Internet vs the Web • The Internet of Things vs the Internet • Web of Things vs the Web 15 • Web of Things vs the Internet of Things
- 16. The “next Internet”: IoT/WoT (mobile) IPv4/IPv6 IPv4/IPv6 (mobile) IP IPv4/IPv6 IPv4/IPv6 IPv4/IPv6 (mobile) IPv4/IPv6 (mobile) IPv
- 17. What is the Web of Things? In order to answer the question we will look at the following differences: • The Internet vs the Web • The Internet of Things vs the Internet • Web of Things vs the Web 17 • Web of Things vs the Internet of Things
- 18. Transition towards machine generated information Past: “manual input of information by 500 million or a billion users”1 Future: 1Pete Hartwell, How a Physically Aware Internet Will Change the World, Mashable, October 13, 2010. 18 “new information can be created automatically without human data entry… the next generation of sensor networks can monitor our environment and deliver relevant information – automatically.1
- 19. How Web-of-things fits on the map? Description Technologies Static HTML pages (web as we first learned it) HTML, HTTP Web 1.5 Dynamic HTML content (web as we know it) Client side (JavaScript, DHTML, Flash, …), server side (CGI, PHP, Perl, ASP/.NET, JSP, …) Web 2.0 Participatory information sharing, interoperability, usercentered design, and collaboration on the World Wide Web (web of people) weblogs, social bookmarking, social tagging, wikis, podcasts, RSS feeds, many-to-many publishing, web services, … URI, XML, RDF, OWL, SparQL, … Web 3.0 …definitions vary a lot – from Full Semantic Web to AI (web as we would need it) http://en.wikipedia.org/wiki/Web_3.0# Web_3.0 Web of Things Everyday devices and objects are connected by fully integrating them to the Web. (web as we would like it) Well-accepted and understood standards and blueprints (such as URI, HTTP, REST, Atom, etc.) http://en.wikipedia.org/wiki/Web_of_T hings 19 Web 1.0
- 20. What is the Web of Things? In order to answer the question we will look at the following differences: • The Internet vs the Web • The Internet of Things vs the Internet • Web of Things vs the Web 20 • Web of Things vs the Internet of Things
- 21. Web of Things vs Internet of Things: what is the difference? Internet = Interconnected networks • They are interconnected via IP (Internet Protocol) • There are IP addresses in the internet, no domain names such as wikipedia.org • Started around 1950 in a effort to make two computers talk to each other Web = Linked documents and resources • Uses HTTP • The web needs the Internet underneath to function 21 • Started around 1980 in an effort to help people share data over the Internet
- 22. The web as an enabler • the web enabled, among others, large scale behavioral, communication, social and linguistic studies, 22 • the things joining the web enable studying the world on far larger scale that ever before
- 23. Fortuna et al., Towards Building a Global Oracle: a Physical Mashup Using Artificial Intelligence Technology, International Workshop on the Web of Things, 2012. 23 Analogy humans - WoT
- 24. Market Research overview • Many consultancy and technology companies offer a range of market research material on “Internet/Web of Things/Everything” 24 • …in the following slides we collected some of the highlights from a number of reports
- 25. Goggle Trends: popularity of “Internet of Things” keyword www.google.com/trends/explore?q=internet+of+things
- 26. McKinsey: Data from “Internet of Things”
- 27. SRI: IoT Technology Roadmap http://en.wikipedia.org/wiki/Internet_of_Things
- 29. BOSH: Projecting IoT growth http://www.bosch-si.com/media/bosch_software_innovations/documents/iot_2/Infographics_IoT.pdf
- 31. Gartner: “IoT + Big Data” 2020 projection “In 2009 there were 0.9 billion sensors and 1.6 billion personal devices …” “[That’s] roughly 2.5 billion things that were connected. By 2020, that will grow to become 30 billion things. This uber-connectedness combined with the analytics capabilities promised by the big data technology trend will create new value for all organizations, spearheading an economic boom “Gartner projects that the total economic value-add for the Internet of things will be $1.9 trillion dollars by 2020” http://www.datanami.com/datanami/2013-10-09/gartner:_internet_of_things_plus_big_data_transforming_the_world.html
- 32. CISCO: IoT Birth & Growth
- 33. CISCO: IoE (Internet of Everything) Index http://www.cisco.com/web/about/ac79/docs/innov/IoE-Value-Index_External.pdf
- 34. Ericsson: More than 50 billion connected devices by 2020 Development of the networked world is progressing in three major waves Some high-level, macro-economic trends and statistics. As a few examples, by 2020 there will be: • 3 billion subscribers with sufficient means to buy information on a 24-hour basis to enhance their lifestyles and improve personal security. • in mature markets, these customers will typically possess between 5-10 connected devices each. • 1.5 billion vehicles globally, not counting trams and railways. • 3 billion utility meters (electricity, water and gas). • A cumulative 100 billion processors shipped, each capable of processing information and communicating http://www.ericsson.com/res/docs/whitepapers/wp-50-billions.pdf
- 36. Part I. Motivation & background outline 36 Web Of Things • What is it? What problems can it solve? Architectural considerations • How it looks like? What are its components? The “Things” • What are the ingredients? The “Glue” • How do things stick together? Applications and services • What can be built on top of it? Quick start recipes • How does the “Hello World!” look lilke?
- 37. Architectural considerations Traffic today app Web publishing RSS feed Data processing Real time event detection software Data storage Utility company data center Data collection Utility company server Embedded software Proprietary firmware Communication technology 6LoWPAN Embedded device Microcontroller with sensors Physical object Public light pole 37 Applications
- 38. Architectural considerations MyHome status page Web publishing Html Data collection Home computer Embedded software RTOS Communication technology Bluetooth Device Microcontroller with sensors 38 Applications
- 39. Main Components of a vertical RSS feed Real time event detection software Utility company data center Data collection Utility company server Proprietary firmware Communication technology 6LoWPAN Embedded device Microcontroller with sensors Physical object Public light pole 39 Web publishing Embedded software “Things” Traffic today app Data storage “Glue” Applications Data processing Apps and Services
- 40. From raw measurements to meaningful information: iDiary (1/3) Apps and Services iDiary: Searchable diary Summary of visited places Summary of activities Text to activities using Latent Semantic Analysis on Yelp reviews Reverse geo-coding from GPS coordinates to text using external web services “Glue” Trajectory clustering on coresets Stream data compression (coresets) Data transmission GPS coord acquisition application D. Feldmann, A. Sugaya, C. Sung, D. Rus, “iDiary: From GPS Signals to a Text-Searchable Diary”, SenSys 2013, Rome, Italy. 40 “Things”
- 41. D. Feldmann, A. Sugaya, C. Sung, D. Rus, “iDiary: From GPS Signals to a Text-Searchable Diary”, SenSys 2013, Rome, Italy. 41 From raw measurements to meaningful information: iDiary (2/3)
- 42. D. Feldmann, A. Sugaya, C. Sung, D. Rus, “iDiary: From GPS Signals to a Text-Searchable Diary”, SenSys 2013, Rome, Italy. 42 From raw measurements to meaningful information: iDiary (3/3)
- 43. From raw measurements to meaningful information: FixtureFinder (1/3) fixture discovery system that automatically infers the existence of electrical and water fixtures in the home V. Srinivasan, J. Stankovic, K. Whitehoue, FixtureFinder: Discovering the Existence of Electrical and Water Fixtures, IPSN April 2013. 43 “Things”
- 44. From raw measurements to meaningful information: FixtureFinder (2/3) The goal of the FixtureFinder algorithm is to combine smart meters with in-home sensors to form a fused data stream, and to discover frequently repeating patterns within that stream. • it will detect when a 5 liter/minute water flow repeatedly cooccurs with activity in a particular motion sensor Step IV - the usage events are clustered into groups that represent the fixtures that have been discovered. Step III, the edge pairs are matched in rising/falling sequences called usage events Step II - data streams are fused by finding events in multiple streams That frequently co-occur in time, and combining them to creating edge pairs. Step I - uses edge detection to compute a sequence of timestamped rising and falling edges in each data stream. V. Srinivasan, J. Stankovic, K. Whitehoue, FixtureFinder: Discovering the Existence of Electrical and Water Fixtures, IPSN April 2013. 44 “Glue”
- 45. From raw measurements to meaningful information: FixtureFinder (3/3) Apps and Services While being an extremely interesting system, presenting the final user with easy to understand information seems to have been outside the scope of the paper. Alternatively, events could be manually annotated using for instance Linked Data. These annotations can be used, similar as in iDiary for searching, logging, etc. 45 Since in the case of electricity and water, the space of possible events is rather limited, perhaps manual templates reporting on the events would be sufficient for an app.
- 46. Part I. Motivation & background outline 46 Web Of Things • What is it? What problems can it solve? Architectural considerations • How it looks like? What are its components? The “Things” • What are the ingredients? The “Glue” • How do things stick together? Applications and services • What can be built on top of it? Quick start recipes • How does the “Hello World!” look lilke?
- 47. The “things” Microcontroller with sensors Physical object “Things” Embedded device Public light pole = embedded device + physical object (smart public light pole) = sensor/actuator node = mobile phone 47 = a set of sensor nodes and/or embedded device + physical things which are abstracted as one “thing” (large water tank + set of sensor nodes monitoring water level, temperature and purity)
- 48. Definitions of components related to things physical object • An object built for fulfilling other tasks than computing • Coffee mug, show, light pole, washing machine, electric oven, fruit press, water tank sensor • a material or passive device which changes its (conductive) properties according to a physical stimulus 48 • Thermo couple (temp->voltage), photo resistor (light>resistance variations), etc.
- 49. Sensor nodes and their structure Power Source = Sensors + Microcontroller + Communication Module + Power Source Comm. Module CPU & Memory Sensor (Actuator) Classification: • adapted/augmented general-purpose computers • system on chip (SoC) solutions 49 • embedded sensor modules
- 50. Other types of sensors photodiode speaker seismic motion 1% 1% button switch 1% 2% pressure LCD 2% 2% 2% irDA 2% ECG camera 2% N axis 4% magnetometer 4% ultrasonic vibration sound 1% 1% temperature 21% N axis accelerometer 13% RGB LED 8% light 10% Monitoring valuable assets: Vibration, humidity …endless combinations 50 microphone 4% acoustic / sound GPS 5% 5% humidity 6% Activity recognition: N-axis accelerometer, microphone, camera
- 51. Existing solutions for sensor nodes Solutions developed in research community or by groups of enthusiasts. • • • Combine HW components from different produces (for radio, it seems that TI chips are used in vast majority of ‚products‘). open-source experimental software such as Contiki OS, TinyOS (& NesC), NanoRK, FreakZ stack (except for Arduino/Libelium where OEM radio is used whilst crowdsourcing is happening on the level of easy microcontroller programming. open source development tools are usually used. Commercial solutions from particular producers (TI, Atmel, Microchip,…) • • composed of components sold by produces themselves. development kits can usually be used with proprietary integrated development environments and allow compiling of certified stacks (most often Zigbee). Modules assembled by companies trying to sell software solutions Sun is in this case promoting the use of Java for sensor networks Sensinode is selling one of the 6LoWPAN ports. 51 • •
- 52. Examples of the three categories of solutions • FreakLabs Chibi • Memsic (ex. Crossbow) MICAz/ MICA2, IRIS, TelosB, eKo kit • CMU FireFly • GINA • Arduino/Libelium (XBee) TI eZ430-RF2500 Microchip PICDEM Z Atmel RZ600 Ember InSight Jennic JN5148 • SunSPOT • Sensinode • NanoSensor 52 • • • • •
- 53. Why are sensor nodes different than other computing devices? • Most SNs are application specific. • Asymmetric, highly directional information flow (data fusion). • Energy is highly constrained. • Networks of SNs may have huge amount of nodes. 53 • Application run-time is extremely long.
- 54. Sensor nodes vs computing devices Diminishing maintenance costs: Personal devices Sensor nodes 54 • Integrating sensors into personal computing devices such as phones/laptops • Efficient remote configuration and management • Disposable
- 55. Part I. Motivation & background outline 55 Web Of Things • What is it? What problems can it solve? Architectural considerations • How it looks like? What are its components? The “Things” • What are the ingredients? The “Glue” • How do things stick together? Applications and services • What can be built on top of it? Quick start recipes • How does the “Hello World!” look lilke?
- 56. The “glue” The communication • The communication medium • The network Node centric programming • operating system • virtual machine System level programming (macro-programming) • distributed/centralized storage and retrieval • content management Data storage “Glue” Real time event detection software Utility company data center Data collection Utility company server Embedded software Proprietary firmware Communication technology 6LoWPAN 56 Data processing
- 57. Communication technology Communication medium • Wireless and/or Wired point-to-point or point-tomultipoint • Several open and proprietary standards exist, operating in different frequency bands with various rate and range performance 57 • ZigBee, Bluetooth, RFID, WiFi, etc.
- 59. Communication technology Network • only the devices and the links between them can be seen, the communication medium is hidden • it has a hierarchical structure • The network layer protocols are less standardized as for the internet: 59 • IPv4, IPv6 can be used but not available in most commercial products • proprietary, • other..
- 60. http://www.freescale.com/files/32bit/doc/white_paper/INTOTHNGSWP.pdf 60 The most common sense-processcommunicate hierarchy
- 61. Making sense of the data Some examples presented already 61 Explained in the second part of the tutorial
- 62. Part I. Motivation & background outline 62 Web Of Things • What is it? What problems can it solve? Architectural considerations • How it looks like? What are its components? The “Things” • What are the ingredients? The “Glue” • How do things stick together? Applications and services • What can be built on top of it? Quick start recipes • How does the “Hello World!” look lilke?
- 63. Apps and services Apps and Services Applications Traffic today app Web publishing RSS feed Combine data, presentation or functionality from several sources (mash-up) to create new services. Things generate only part of the data sources 63 Selected demos shown throughout the presentation.
- 64. Part I. Motivation & background outline 64 Web Of Things • What is it? What problems can it solve? Architectural considerations • How it looks like? What are its components? The “Things” • What are the ingredients? The “Glue” • How do things stick together? Applications and services • What can be built on top of it? Quick start recipes • How does the “Hello World!” look like?
- 65. Programing the “things” 65 Complex and time consuming process • tool chain: IDE, compiler, debugger All this programmed and executes the • microcontroller isis not necessary anymore unless code you want to achieve something rather • radio chip is not programmed, but controlled by specific. microcontroller, usually via SPI which sets/reads registers Web programmer (rather than embedded • compiled code is loaded to the microcontroller using programmer) products and do it yourself bootloader or JTAG kits exist. • protocol stack may be precompiled and available through API or available as library • operating system (not needed for simple tasks) • virtual machine (optional)
- 66. Some developer friendly hardware solutions for WoT compared Pro Con Cost Arduino big community, open-source hw, documentation Computational power 25-90 € Nanode built-in web connectivity, opensource hw, Arduino compatible Computational power, documentation ~ 35 € openPICUS built-in web connectivity, good documentation, support Computational power, development windows oriented ~ 70 € Netduino open-source hw, documentation, arduino compatible, .NET programming Computational power, development windows oriented 25-90 € libelium documentation, solid, radio boards, sensors boards, over the air programming, libelium support. Computational power ~ 150 € Hacking the Internet of Things, Hardware solutions compared, Lelylan Chief, Mar 09, 2012. 66 Solution
- 67. Decision process 67 Before starting, the following questions should be answered: What is the scope or application? • Monitoring measurements? What is the scenario? • A thing with embedded web service? • A set of things connected through a gateway? What programming language? • Options: C, nesC, Java, C# or Javascript What is the publishing infrastructure? • None, custom, third party.
- 68. How to start? If you want to get a flavor of what is possible today with existing technologies, a good way to start is: Ninjablocks , Twine and Sen.se 68 A good idea is to also have a look at kickstarter for any new gadgets and platforms
- 69. Summary In part one we • Provided an explanation on the WoT by comparing it to IoT, Internet and Web • A market study of the area • Provided an overview of possible verticals of different complexity 69 • Inspiring examples of what problems can be solved with these verticals
- 70. Outline Part I. Motivation & background Part II. Technology and tools for exploiting the WoT 70 Part III. Demos, Tools & Research directions
- 71. Outline Part II. Technology and tools for WoT data Information infrastructure for “Web of Things” Conceptualization of sensors domain Stream Data Processing Stream Mining Complex Event Processing 71 Anomaly Detection
- 72. INFORMATION INFRASTRUCTURE FOR “WEB OF THINGS”
- 73. Why we need WoT? …the key objective is to make decision maker more efficient by understanding observed environment Decision maker 73 Sensor network
- 74. Why we need WoT? …the key objective is to make decision maker more efficient by understanding observed environment To achieve this, we need to introduce several information layers between sensor setup and decision maker: Conceptualization Streaming (ontology) Storage Stream Mining; Decision maker Complex Events; Anomaly Detection 74 Sensor network
- 75. Outline of this part of the talk In this part we will review approaches on • How to conceptualize sensor domain? • How to process streaming data? 75 • How to detect complex events? • How to perform mining on streaming data? • How to detect anomalies?
- 77. http://lists.w3.org/Archives/Public/public-xg-ssn/2009Aug/att-0037/SSN-XG_StateOfArt.pdf 77 Semantic Sensor Network architecture
- 78. Sensor ontologies Several ontologies are covering sensor domain • …most of them only parts W3C Semantic Sensor Network (SSN) Ontology (next slide) is an attempt to cover complete domain http://lists.w3.org/Archives/Public/public-xg-ssn/2009Aug/att-0037/SSN-XG_StateOfArt.pdf
- 79. http://www.w3.org/2005/Incubator/ssn/wiki/Main_Page 79 W3C Semantic Sensor Network (SSN) ontology structure
- 80. So, how does a value look like? Having all the semantic infrastructure in place, how an observed value is encoded in SSN? http://www.w3.org/2005/Incubator/ssn/wiki/Main_Page 80 The observed value
- 81. Spatio-temporal Reasoning for Traffic Scene Understanding Brehar et al, Spatio-temporal reasoning for traffic scene understanding. 2011 IEEE 7th International Conference on Intelligent Computer Communication and Processing. 81 System architecture
- 82. Traffic Scene Understanding and the WoT vertical Apps and Services Enable traffic scene queries and natural language statement generation Mapping the classes to an ontological model provided by Cyc Object classification “Glue” Object tracking using Kalman filter Object detection Pre-processing: apply undistortion, scaling and rectification “Things” Stereo image acquisition by cameras on a vehicle Brehar et al, Spatio-temporal reasoning for traffic scene understanding. 2011 IEEE 7th International Conference on Intelligent Computer Communication and Processing.
- 83. Querying traffic scenes QUESTION: Image depicts Person? QUESTION: Image depicts Person? ANSWER: PEDESTRIAN2A000282 is a person. ANSWER: PEDESTRIAN2A000283 is a person. QUESTION: Image depicts UtilityPole? QUESTION: Image depicts UtilityPole? ANSWER: UNCLASSIFIED0A000282 is a utility pole. POLE4A000282 is a utility pole. UNCLASSIFIED6A000282 is a utility pole. UNCLASSIFIED7A000282 is a utility pole. UNCLASSIFIED8A000282 is a utility pole. UNCLASSIFIED9A000282 is a utility pole. UNCLASSIFIED10A000282 is a utility pole. ANSWER: UNCLASSIFIED0A000283 is a utility pole. POLE4A000283 is a utility pole. UNCLASSIFIED6A000283 is a utility pole. UNCLASSIFIED7A000283 is a utility pole. UNCLASSIFIED8A000283 is a utility pole. POLE9A000283 is a utility pole. QUESTION: Image depicts Automobile? QUESTION: Image depicts Automobile? ANSWER: UNCLASSIFIED3A000282 is a car. ANSWER: UNCLASSIFIED3A000282 is a car.
- 84. Natural language traffic scene description generation QUESTION: Image depicts ObjectWithUse? ANSWER: UNCLASSIFIED0A000283 is a utility pole, every utility pole is a post, every post is a shaft, every shaft is a rod, every rod is an implement, every implement is a device, every device is an object with uses. CAR3A000283 is a car, every car is a device that is not a weapon, every device that is not a weapon is a device, every device is an object with uses. POLE4A000283 is a utility pole, every utility pole is a post, every post is a shaft, every shaft is a rod, every rod is an implement, every implement is a device, every device is an object with uses. Brehar et al, Spatio-temporal reasoning for traffic scene understanding. 2011 IEEE 7th International Conference on Intelligent Computer Communication and Processing.
- 86. Stream data processing Applications that require real-time processing of highvolume data steams are pushing the limits of traditional data processing infrastructures In the following slides we present the requirements of that system… 86 • …based on the paper “The 8 Requirements of Real-Time Stream Processing” by Stonebraker, Çetintemel, Zdonik; ACM SIGMOD Record Volume 34 Issue 4
- 87. Eight rules for stream processing (1/2) Rule 1: Keep the Data Moving • Processing messages “in-stream”, without requirements to store them; ideally the system should also use an active (i.e., non-polling) Rule 2: Query using SQL on Streams • High-level SQL like language with built-in extensible stream oriented primitives and operators Rule 3: Handle Stream Imperfections http://www.complexevents.com/2006/06/30/the-eight-rules-of-real-time-stream-processing/ 87 • Dealing with stream “imperfections”, including missing and out-of-order data, which are commonly present in real-world data streams Rule 4: Generate Predictable Outcomes
- 88. Eight rules for stream processing (2/2) Rule 5: Integrate Stored and Streaming Data • Combining stored with live streaming data Rule 6: Guarantee Data Safety and Availability • Integrity of the data maintained at all times, despite failures Rule 7: Partition and Scale Applications Automatically • Distribute its processing across multiple processors and machines to achieve incremental scalability Rule 8: Process and Respond Instantaneously http://www.complexevents.com/2006/06/30/the-eight-rules-of-real-time-stream-processing/ 88 • Minimal-overhead execution engine to deliver real-time response
- 89. http://www.complexevents.com/2006/06/30/the-eight-rules-of-real-time-stream-processing/ 89 “Straight-through” processing of messages with optional storage
- 90. Basic architectures for stream processing databases Rule engine Traditional DBMS system http://www.complexevents.com/2006/06/30/the-eight-rules-of-real-time-stream-processing/ 90 Stream processing engine
- 91. http://www.complexevents.com/2006/06/30/the-eight-rules-of-real-time-stream-processing/ 91 The capabilities of various systems software
- 94. Steps in processing events Data filtering • Low level filtering • Semantic filtering Data transformation and aggregation • Database updates • Creating relationships among objects Complex event definition • Event constructors specifying the constituent events (nontemporal and temporal) Processing of non-spontaneous events • Pseudo-events as objects containing temporal constraints Wang, F., S. Liu, and P. Liu, Complex RFID event processing. The VLDB Journal, 2009.
- 95. Good practices in distributed CEP Data centric storage – data mapped to different locations • Data lookup – multi-level hashing • Data robustness – replication Data caching – multiple copies of the most requested data • Consistency – response time trade-off Group management - cooperation among group of nodes • Provide higher reliability • Anomaly detection Publish/subscribe for event subscription • Loose coupling Li, S., et al., Event Detection Services Using Data Service Middleware in Distributed Sensor Networks. Telecommunication Systems, 2004.
- 96. Complex Event Processing Application Development 1. Defining event sources and event targets (sinks) 2. Creating an input adapter to read the events from the source into the CEP server 3. Creating an output adapter to consume the processed events for submission to the event targets 4. Creating the query logic required to meet your business objectives binding the query to the adapters at runtime, and to instantiate the query in the CEP server http://msdn.microsoft.com/en-us/sqlserver/ee476990 96 1. 2.
- 97. Examples of Queries in Microsoft StreamInsight (1/2) http://msdn.microsoft.com/en-us/sqlserver/ee476990 97 Filtering of events • from e in inputStream where e.value < 10 select e; Calculations to introduce additional event properties • from e in InputStream select new MeterWattage {wattage=(double)e.Consumption / 10}; Grouping events • from v in inputStream group v by v.i % 4 into eachGroup from window in eachGroup.Snapshot() select new { avgNumber = window.Avg(e => e.number) }; Aggregation • from w in inputStream.Snapshot() select new { sum = w.Sum(e => e.i), avg = w.Avg(e => e.f), count = w.Count() };
- 98. Examples of Queries in Microsoft StreamInsight (2/2) http://msdn.microsoft.com/en-us/sqlserver/ee476990 98 Identifying top N candidates • (from window in inputStream.Snapshot() from e in window orderby e.f ascending, e.i descending select e).Take(5); Matching events from different streams • from e1 in stream1 join e2 in stream2 on e1.i equals e2.i select new { e1.i, e1.j, e2.j }; Combining events from different streams in one • stream1.Union(stream2); User defined functions • from e in stream where e.value < MyFunctions.valThreshold(e.Id) select e;
- 99. Event Models in Microsoft StreamInsight Interval model Start Time End Time INSERT 2009-07-15 09:13:33.317 2009-07-15 09:14:09.270 2009-07-15 09:14:22.255 2009-07-15 09:14:09.270 2009-07-15 09:14:22.253 2009-07-15 09:15:04.987 • Event has predefined duration Point model INSERT Event Kind Start Time End Time • Event is occurrence in a point in time Edge model INSERT 2009-07-15 09:13:33.317 INSERT 2009-07-15 09:14:09.270 INSERT 2009-07-15 09:14:22.255 2009-07-15 09:13:33.31 7 2009-07-15 09:14:09.27 0 2009-07-15 09:14:22.25 5 • Only start time known upon arrival to server; end-time is updated later INSERT Event Kind INSERT INSERT INSERT INSERT INSERT Edge Type Start End Start End Start Start Time t0 t0 t1 t1 t3 http://msdn.microsoft.com/en-us/sqlserver/ee476990 Payload (Power Consumption) 100 200 100 Payload (Consumption) 100 200 100 End Time ∞ t1 ∞ t3 ∞ Payload a a b b c 99 Event Kind
- 100. CEP and Sensor Networks CEP role is to discovering meaningful information from sensor data • observations – raw outputs of sensors • event – detected and of interest for the application • centralized vs. distributed processing
- 101. STREAM MINING
- 102. Typical stream mining architecture Streams can include different types of data • We can prepare system ahead of time for “Standing Queries” • We can prepare only for certain class of “Adhoc Queries” http://infolab.stanford.edu/~ullman/mmds.html 102 •
- 103. How we mine streams? • Maintain summaries of the streams, sufficient to answer the expected queries about the data • …operations on a sliding window mimic more traditional database/mining operations 103 • …summaries can be in various forms: clusters (flat or hierarchic, statistical aggregates, …) • Maintain a sliding window of the most recently arrived data
- 104. Example: Stream summarization by incremental hierarchical clustering The goal is to maintain summary of data from stream in a form of a taxonomy of prototype clusters – each new events updates the taxonomy In the example below, events are Reuters news articles coming in time, Colors represent topics [Blaz Novak, 2008]
- 105. Example: Stream processing on sliding window Database 11.3 9.0 6.0 • Indexing of window data segments 40 12.3 10.0 11.0 20 40 Extract a Signal 2.3 Fourier Transform 0 20 0 9.0 -20 -20 1.0 -40 -40 -60 -60 -80 -80 0 0 500 1000 1500 2000 2500 3000 • Query/Template Pattern Preprocessing 500 1000 40 20 1500 2000 2500 3000 3500 4000 Fourier Transform 4500 3500 Matching with one of distance metrics – if data and query are similar, trigger an alert 4000 4500 5000 5000 10.0 9.5 60 0 -20 [Faloutsos, 1994] Slide taken from [Pramod Vemulapalli: Pattern Matching with Acceleration Data]
- 106. Data reduction stream mining tasks • Sampling • …challenge is to obtain representative data sample (i.e., enabling to perform correctly required operations on data) • Filtering • …simple filters are easy to implement (e.g. simple conditions like “x<10”) • …filtering by a membership of a set which doesn’t fit in the main memory requires more sophisticated algorithms (e.g. Bloom filtering) 106 • (example of set membership: list of spam URLs)
- 107. ANOMALY DETECTION
- 109. Anomaly detection http://www.dtc.umn.edu/publications/reports/2008_16.pdf 109 Key components associated with an anomaly detection technique
- 110. Techniques to detect anomalies Classification based • A pre-trained classifier can distinguish between normal and anomalous classes Clustering based • Normal data instances belong to large and dense clusters, while anomalies either belong to small or sparse clusters Nearest neighbor approaches • Normal data instances occur in dense neighborhoods, while anomalies occur far from their closest neighbors Statistical approaches • Normal data instances occur in high probability regions of a stochastic model, while anomalies occur in the low probability regions Information theoretic approaches • Anomalies in data induce irregularities in the information content of the data set • Normal instances appear in a lower dimensional subspace, anomalies in the rest (noise) http://www.dtc.umn.edu/publications/reports/2008_16.pdf 110 Spectral methods
- 111. The key to a successful anomaly detection is proper feature engineering! Contextual anomaly t2 in a temperature time series. Note that the temperature at time t1 is same as that at time t2 but occurs in a different context and hence is not considered as an anomaly. http://www.dtc.umn.edu/publications/reports/2008_16.pdf 111 Anomalies are detectable if data instances are represented in an informative feature space
- 112. Application: Telecommunication Network Monitoring Alarms Server Telecom Network (~25 000 devices) Alarms ~10-100/sec Live feed of data Alarms Explorer Server Alarms Explorer Server implements three real-time scenarios on the alarms stream: 1. Root-Cause-Analysis – finding which device is responsible for occasional “flood” of alarms 2. Short-Term Fault Prediction – predict which device will fail in next 15mins 3. Long-Term Anomaly Detection – detect unusual trends in the network Operator Big board display
- 113. Outline Part I. Motivation & background Part II. Technology and tools for exploiting the WoT 113 Part III. Demos, Tools & Research directions
- 114. Outline Part III. Demos, Tools & Research directions Use cases • What systems and prototypes exist? Open problems • Are there unsolved problems? Summary • What was this tutorial about? List of sources for further studies 114 • Where to start digging?
- 115. 1Ludwig Siegele, A special report on smart systems, The Economist, Nov. 4 2010. 115 Web of things use cases according to Beecham Research
- 116. Web of things use cases mapped to the Beecham Research classification 116 A case for Energy: dynamic power pricing using smart grid technology
- 117. Commercial use case: Power grids1 “If the power grid in America alone were just 5% more efficient, it would save greenhouse emissions equivalent to 53m cars (IBM).“ Solutions: • demand pricing – 10-15% peak hour demand cut • Energy consumption monitoring with smart meters encourage shifting consumption to off-peak hours through personalized price plans • demand response – extra 10-15% cut 1Ludwig Save energy by sensing and actuation: smart meters + actuators turn off air-conditioning systems when demand for electricity is high Siegele, A special report on smart systems, The Economist, Nov. 4 2010. 117 •
- 118. Public lighting control: greener lights + control Power consumption vs. dimming level 40.0 35.0 30.0 25.0 20.0 15.0 10.0 5.0 0.0 SGA LSL 30 main characteristics • the number of LED: 30*1w • consumption: 35W (at full power) • colour of the light: 4200K • light current: 2700lm • life-expactancy: min. 60.000h • IP66 • NET mass: 4,8k 0.3 10.7 21.2 31.7 42.2 52.8 63.3 73.9 84.4 94.9 100 118 Dimming Power level consumption [%] [W] 0,3 7,4 10,7 8,4 21,2 13,8 31,7 17,9 42,2 21,6 52,8 25,2 63,3 28,5 73,9 31,5 84,4 34,2 94,9 36,4 100 37,2
- 119. Public lighting control: lights dimming Lights dimmed to 75% luminosity between 23:00 and 5:00 with smooth 15-minute linear transitions. 𝑃 𝑛𝑜𝑡 𝑑𝑖𝑚𝑚𝑒𝑑 = 37.2𝑊 𝑃 𝑑𝑖𝑚𝑚𝑒𝑑 = 31.8𝑊 Electricity consumption per night: 𝐴 𝑒 𝑛𝑜 𝑑𝑖𝑚𝑚𝑖𝑛𝑔 = 0.372𝑘𝑊ℎ 𝐴 𝑒 𝑑𝑖𝑚𝑚𝑖𝑛𝑔 = 0.341𝑘𝑊ℎ Dimming and power in time. Red line represents light poles with no dimming. 119 Reduced by ~ 8,3%.
- 120. Web of things use cases mapped to the Beecham Research classification 120 A case for transportation systems: dynamic pricing for congestion, automatic parking finding assistats
- 121. “In 2007 its congested roads cost the country 4.2 billion working hours and 10.6 billion litres of wasted petrol (Texas Transportation Institute)” 1 Solutions: • Charging for city centers and busy roads • London, Stockholm, Singapore, etc. • Green wave • Adjustment of traffic lights to suit the flow of vehicles • Automatic parking guidance • Singapore is developing a parking-guidance system (cars looking for somewhere to park are now a big cause of congestion). • Real-time dynamic pricing • Singapore 1Ludwig Siegele, A special report on smart systems, The Economist, Nov. 4 2010. 121 Commercial use case: Transport systems1
- 122. Web of things use cases mapped to the Beecham Research classification 122 A case for infrastructures: automated fixture findings and intervention management
- 123. Utilities around the world lose between 25% and 50% of treated water to leaks (Lux Research). Solutions: • Renew infrastructure • London, UK, Thames Water was losing daily nearly 900m litres of treated water and had to fix 240 leaks due to aging infrastructure1. • Install sensors for monitoring the pipe system • Automatically detect leaks fast (instead of customers calling and reporting leaks). London, Singapore, etc. • Automate the management and maintenance process • Automatic scheduling of work crews and automatic alerts (i.e. text messages to affected customers) 1Ludwig Siegele, A special report on smart systems, The Economist, Nov. 4 2010. 123 Commercial use case: Water distribution1
- 124. Water distribution WaterWiSe in Singapore • develop generic wireless sensor network capabilities to enable real time monitoring of a water distribution network. • three main applications: http://aqueduct.nus.edu.sg/waterwise/ 124 • On-line monitoring of hydraulic parameters within a large urban water distribution system. • Integrated monitoring of hydraulic and water quality parameters. • Remote detection of leaks and prediction of pipe burst events.
- 126. Web of things use cases mapped to the Beecham Research classification 126 A case for logistic systems: automated asset tracking and supply chain management
- 127. Commercial use case: Logistics Cargo loss due to theft or damage is significant, estimates that the global financial impact of cargo loss exceeds $50 billion annually (The National Cargo Security Council)1. The cost is eventually passed to the customers. Solutions: • Automatic track and trace • Tag and trace their wares all along the supply chain (RFIDs and sensors) - and consumers to check where they come from (i.e. FoodLogiQ, SenseAware)2 • Event detection and mitigation 1 Tom Hayes, The Full Cost of Cargo Losses 2Ludwig Siegele, A special report on smart systems, The Economist, Nov. 4 2010. 127 • Detect events that affect the cargo (i.e. delay, inappropriate transport conditions) and minimize damage (i.e. re-route)
- 128. Logistics - SenseAware http://www.senseaware.com/ 128 - temperature readings - shipment’s exact location - shipment is opened or if the contents have been exposed to light - real-time alerts and analytics between trusted parties regarding the above vital signs of a shipment
- 130. Web of things use cases mapped to the Beecham Research classification 130 A case for industrial automation: automatic event detection and reporting
- 131. Commercial use case: Industrial automation The integration gap between the production and business processes comes at a high cost, especially in multi-site enterprises. Solutions: • Automatic monitoring of the production process • • Monitor the devices on the production floor (i.e. robotic arm overheating)1 Automatic event detection and notification • • Process the measurements, detect anomalies and notify the business process (i.e. production at site interrupted, relocate) Productivity comparison • • 5% increase in paper production by automatically adjusting the shape and intensity of the flames that heat the kilns for the lime used to coat paper2 1SOCRADES 2Ludwig project, http://www.socrades.eu/ Siegele, A special report on smart systems, The Economist, Nov. 4 2010. 131 • Machines equipped with sensors allow productivity comparison based on sensed data (i.e. Heidelberger Druckmaschinen)2 Dynamic production optimization
- 132. Process integration • SunSpot on Robotic ARM, exposing measurements as Web service • SunSpot GW connected to Windows machine, then to the Enterprise Network or Internet • Failure, production interruption alarm – moving to alternative production site Enterprise Resource Planning SIA server Thing (DPWS) Thing (REST) 1SOCRADES 2D. project, http://www.socrades.eu Guinard, V. Trifa, S. Karnouskos, P. Spiess, D. Savio, Interacting with the SOA-based Internet of Things: Discovery, Query, Selection and On-Demand Provisioning of Web Services, IEEE Transactions on Services Computing, Vol. 3, July-Sept 2010. 132 http://www.youtube.com/watch?v=K8OtFD6RLMM
- 133. Discover things Automatic context data collection Device Profile for Web Services (DPWS) • Subset of Web Service standards (WSDL and SOAP) • Successor of Universal Plug and Play (UPnP) Representational State Transfer (REST) • Lightweight, suitable for less complex services 1SOCRADES 2D. Thing (DPWS) Thing (REST) Thing (REST) project, http://www.socrades.eu Guinard, V. Trifa, S. Karnouskos, P. Spiess, D. Savio, Interacting with the SOA-based Internet of Things: Discovery, Query, Selection and On-Demand Provisioning of Web Services, IEEE Transactions on Services Computing, Vol. 3, July-Sept 2010. 133 Thing (DPWS)
- 134. Query embedded services - Insert search keywords, perform query enrichment (augmentation) - Tested 2 strategies: Wikipedia and Yahoo! Search Manually tune the augmented query by adding/deleting keywords Search services in the store and rank them according to some criteria (i.e. QoS) Search Keywords Query Enrichment Wikipedia, Yahoo! Search Manual tuning Service query Service Ranking Service Instances Service testing Device and service store Thing (DPWS) Thing (DPWS) Thing (REST) Thing (REST) 134 -
- 135. Web of things use cases mapped to the Beecham Research classification 135 A case for security and public safety: environmental monitoring
- 136. Commercial use case: Environmental intelligence Data from large number of sensors deployed in infrastructure (such as roads) or over other area of interest (such as agriculture fields) can give decision makers a realtime awareness on the observed phenomena and events. Solutions: • Remote monitoring of cultures, soil moisture, insect infestations or disease infections • Irrigation and pesticide spraying in precision agriculture 136 • Livestock monitoring for maximizing production (meat, milk, eggs) and achieve higher reproduction rates
- 137. Hyperthermia detection in stables 137 • The goal is to measure temperature and humidity inside and outside stables in order to detect the danger of hyperthermia at cows and issue an early warning.
- 138. 138 Hyperthermia detection in stables http://sensorlab.ijs.si/sl/demos.html
- 139. Remote observation of sportfishing conditions http://sensorlab.ijs.si/sl/demos.html 139 By using WSN technology information on water level, picture of a fishing spot and water and outside temperature can be provided
- 140. http://sensorlab.ijs.si/sl/demos.html 140 Remote observation of sportfishing conditions
- 141. Beehive local climate conditions The purpose of this testbed is to monitor climate conditions inside (temperature and humidity) and outside (temperature, humidity, air pressure, wind direction and speed) of the beehives. 141 Through bee counting sensor presence of pesticides in the vicinity can be detected. For the test purposes also sound monitoring is possible
- 142. Multispectral imaging and data harvesting over Unmanned Aerial Vehicle (UAV) Data harvesting over large areas where deployed WSNs have no Internet connection can be a very time consuming and expensive task. 142 Our solution uses Unmanned Aerial Vehicles (UAV) equipped with a gateway sensor node. In addition, UAV is used to collect multispectral images with a Tetracam ADC camera.
- 143. Experimental infrastructures for IoT/WoT All results have to be based on real data collected from real infrastructures (see iDiary, FixtureFinder) • experimental infrastructure enabling such research is mandatory For experimental research by application domain experts • smart cities, smart grids • sensorial infrastructure for experimental research 143 For experimental research on the sensor network infrastructure and services • FIRE (EU) and GENI (USA) initiatives (also PPP and Living Labs) • experimental sensorial infrastructure
- 144. The LOG-a-TEC experimental facility LOG-a-TEC joint name for several specific testbeds The main testbed situated in Logatec (SI) Based on wireless sensor network Focus on • low-cost spectrum sensing hardware • wireless sensor network applications (AQ) Sensor nodes are (primarily) installed on public light poles 144 Infrastructure rewiring ensures 24/7 power supply
- 145. Experimental sensor network LOG-a-TEC green – UHF, blue - ISM 868 MHz, red - ISM 2400 MHz, yellow - reserve locations 145 50+ VESNA sensor nodes City center & industrial zone ZigBee based management network @ 868 MHz (& OTAP) Coordinating node provides gateway to the internet
- 146. VESNA plaform Modular platform for WSN (VESNA=SNC+SNR+SNE) • • • • High processing power and low energy consumption Sensor node & gateway (multi-tier / IP) capability Battery, solar or external power supply Multiple communication technologies • IEEE 802.15.4, ZigBee, 6LoWPAN, Wireless M-BUS, Bluetooth, Wi-Fi, GSM/GPRS, Ethernet • Extensive portfolio of sensors and actuators • Temperature, Humidity, Luminance, Color, Reflectance, Pressure/Force, Camera, GPS, Microphone, Accelerometer, Gas (O2, CO2, CO), Hall effect, Motion/presence/range (IR, ultrasonic, Doppler), Capacitive/inductive touch, Gyroscope, Compass, … • Analog, Digital, PWM, LED, LCD, Relay, Motor, … Open HW / SW / dev. tools 146 Contiki OS port (6LoWPAN)
- 147. VESNA assembling SNE-ISMTV 2.4 GHz TRX CC2500 868 MHz TRX 868 MHz TRX TDA18219HN CC1101 TV UHF RX AT86RF212 SPI, GPIO SNC v1.0 SNR-MOD v1.0 custom code ATZB-900-B0 147 Contiki + custom code SPI / UART or
- 149. Spectrum sensing and cognitive radio 3 clusters ZigBee mesh Ethernet VESNA • ISMTV • IEEE 802.15.4 • SD card other devices • USRP, SA, ... HTTPlike custom protocol 149 SSL server
- 150. Environmental monitoring and lights control testbed • Location: Slovenia, Europe (August 2010 – Sept 2012) • The “things”: public light poles + VESNA sensor nodes • Sensors: temperature, humidity, pressure, illuminance, etc. 150 • Actuator: dim the intensity of the light (pulse width modulation)
- 151. 151 Videk: mashed sources of data http://sensors.ijs.si/
- 153. Sensors, knowledge modeling and transliteration Knowledge base Sensors Fortuna et al., Web of Things, 2012. Mash-up 153 Device Identification Protocol Transliteration
- 154. Moraru et al, Exposing Real World Information for the Web of Things, IIWeb (WWW2011), Hyderabad, India. 154 Environmental intelligence: SemSense system architecture
- 155. Environmental intelligence: SemSense implementation details Scenario • architecture for collecting real world data from a physical system of sensors and publishing it on the Web Implementation: • VESNA Sensor Nodes platform are the “things” • Self-Identification Protocol • • Custom protocol for collecting meta-data and data MySQL database for storage of data and meta-data • Meta-data semantic enrichment component • • RDF representation Semantic Sensor Network (SSN) ontology, Basic GeoWGS84 Vocabulary, GeoNames and FOAF as vocabulary Linking to Linked Opened Data Cloud D2R for mapping the database schema A. Moraru, M. Vucnik, M. Porcius, C. Fortuna, M. Mohorcic, D. Mladenic, Exposing Real World Information for the Web of Things, IIWeb (WWW2011), Hyderabad, India. 155 • •
- 156. Browse at: http://sensors.ijs.si:2020/ 156 Environmental intelligence: browse the semantic representation
- 158. Photovoltaics system monitoring 158 Motivation • Systematically investigate the pros and cons of different PV technologies, effect of panels deployment (orientation) and impact environment (weather) conditions Sensorics • Light intensity in different spectrum (UV/VIS/IR) • Solar panel U/I characteristic • Performance of inverter MPP tracker • Temperature of a PN junction • Environment conditions (context) Measurements stored and post processed in the web platform
- 159. Photovoltaics system monitoring 5 sets of PV panels • S, E, W orientation • Amorphous & crystalline silicon 7 VESNA sensor nodes • Weather • Temp. + current • Reference solar cell 1 VESNA GW ZigBee sensor network @ 868 MHz 159 • 3G radio module
- 160. Air quality monitoring 160 FP7 CITI-SENSE- Development of sensor-based Citizens' Observatory Community for improving quality of life in cities Indoor/outdoor air quality, weather, • Urban quality radiation, noise level … • Public spaces • Schools indor VESNA • Gas sensors - CO2, CO, Nox, VOC • Environmental sensors – air pressure, temperature, humidity, luminence, weather • Noise sensor LOG-a-TEC testbed • Trial and validation of VESNA based prototype system • Final pilot system deployed in Ljubljana (+ 8 EU cities)
- 161. Smart Mobile Home Global environment Local environment Micro environment Sensing: • Car to Car • Road conditions • Twitter on Events • Weather • Geography • Tourism • Parking Communication: • Social network among camper drivers • Car service & advice The goal is to help a driver to manage complexity of all the data sources …typical age of a driver is over 60 years not being to skilled in driving a camper 161 Sensing: • Gas • Electricity • Petrol • Fridge • …
- 162. EnStreaM Rule generation and validation on environmental data • The supporting architecture is capable of handling large amounts of data • The system is meant for domain experts to generate and validate rules that describe specific events • Applied in environmental scenarios: landslides, oil spills and river floods, where main source of data comes from sensors Also a ESWC 2012 demo: Supporting Rule Generation and Validation on Environmental Data in EnStreaM
- 163. System Overview • Input: sensor data and event data – e.g.: volume of rainfall for a given geographical location and landslides that occurred • Output: rules and the related dataset, semantically annotated
- 164. Data Layer Schema Node • Name, URN • GPS Latitude, Longitude, Altitude, Sattelite No., Speed • Description, Geolocation (string), Owner Sensor • SensorNodeId • SensorTypeId • CurrentState fields (current aggregates) • LastMeasurementId Measurement Events • Name, URN • Timestamp • GPS latitude, longitude • SensorId • Value • Timestamp Type • Name, Measured Phenomena, URN • Sampling information, Reporting interval, feature of interest, unit of measurement Aggregate • SensorId • Value • Aggregate type • Timespan • Timestamp
- 165. Aggregates Saves aggregates after transition into a new time window: Count, Average, Sum, Min, Max, Standard deviation Primary aggregates Calculated from raw measurements, fine grained. Secondary aggregates Calculated from other aggregates (only possible to use with on-line type). <?xml version="1.0" encoding="utf-8"?> <configuration> <timespans> <timespan id="1" timewindow="3600" /> <timespan id="2" pid="1" timewindow="24" interval="1"/> <timespan id="3" pid="2" timewindow="7" interval="1"/> <timespan id="3" pid="2" timewindow="30" interval="1"/> <timespan id="4" pid="2" timewindow="365" interval="1"/> </timespans> <aggregates> <aggregate type="MAX"/> <aggregate type="MIN"> <timespan id="1" timewindow="3600"> <timespan id="2" pid="1" timewindow="48" interval="24"/> </aggregate> <aggregate type="AVG"/> <aggregate type="SUM"/> <aggregate type="STD"/> <aggregate type="MED"/> <aggregate type="1QU"/> <aggregate type="3QU"/> <aggregate type="CNT"/> </aggregates> <sensortypes> <sensortype id="1"> <aggregate type="MAX"/> <aggregate type="SUM"/> </sensortype> </sensortypes> <sensors> <sensor id="1"> <aggregate type="MAX"/> <aggregate type="SUM"/> </sensor> </sensors> </configuration>
- 166. Time windows & intervals • • • <timespan id="1" timewindow="3600" /> Time windows of aggregates can overlap Overlapping interval is set in configuration file (interval) For example: • M <timespan id="2" pid="1" timewindow="24" interval="1"/> <timespan id="3" pid="2" timewindow="7" interval="1"/> Weekly aggregates can be calculated from Monday to Monday, from Tuesday to Tuesday, etc. T W T F Overlap/update interval is 1 day 7-day time window S S M T W time
- 167. Example queries Easy and fast detection of events on current state data (very simple rules) Fog forming example If (humidity[AVG,1h] < 90%) & (humidity[AVG,10m] > 95%) Then trigger fog forming risk event. Simple validation of more complex event queries (using current state and previous aggregates) Road Icing example If (precipitation[SUM,12h,6h ago] > X) & (temparature[MAX,12h,6h ago] > 0) & (temperature[MIN, 6h]) < 0) Then trigger road icing risk event. Can handle time queries Time example If (temperature[AVG,1w,3d ago] <-5) & (temperature[AVG,24h,2d ago] < 5) & (temperature[AVG,24h,1d ago] < 5) & Then trigger lake still frozen event.
- 168. System GUI
- 169. Use cases Environmental intelligence Others • Intelligent buildings • Smart cities • Smart infrastructures 169 • …
- 170. Intelligent building Berkley: Motescope* - Soda Hall, the Computer Science building - Permanent testbeds for research, development and testing *According to web site visited on Oct 2010. 170 - 78 Mica2DOT nodes
- 171. University campus CMU: SensorAndrew* - campus-wide testbed - Firefly nodes * According to web site on Oct 2010 and tech report from 2008. 171 - Unknown scale
- 172. Smart city MIT: Senseable City Lab* - Sensor nodes built into the wheels of bikes *Neil Savage, Cycling through Data, Communications of the ACM, Sept 2010. 172 - Unknown number
- 173. Smart infrastructure Harward, BBN: CitySense* - 100 wireless sensors deployed across a city - Sensor nodes are embedded PC, 802.11a/b/g interface, and various sensors for monitoring weather conditions and air pollutants * According to web site visited on Oct 2010, last modified in 2008. 173 - open testbed
- 174. Outline Part III. Demos, Tools & Research directions Use cases • What systems and prototypes exist? Open problems • Are there unsolved problems? Summary • What was this tutorial about? List of sources for further studies 174 • Where to start digging?
- 175. Current state and open problems with respect to Sensor Nodes WSN is • Well developed field with many degrees of freedom • Complex, large-scale, resource constrained systems • Focus is on intra network communications Efficient management and maintenance of the “things” Remote reconfiguration of parameters • Remote software updates • Real implementations solving real problems, particularly large scale (see next slide) 175 •
- 176. Myths & lessons regarding Sensor Networks Myth #1: Nodes are deployed randomly. Myth #2: Sensor nodes are cheap and tiny. Myth #3: The network is dense. Lesson #1: It’s all about the data. Lesson #2: Computer scientists and domain scientists need common ground. M. Welsh, Sensor Networks for the Sciences, Communications of the ACM, Nov. 2010. 176 Lesson #3: Don’t forget about the base station!
- 177. Challenges with respect to conceptualization WoT covers a long pipeline of technologies from sensors to high level services • ...current ontologies are covering just parts of the space and are yet to be interlinked 177 • ...ideally, sensor network domain should be linked to general common-sense ontologies and further to domain specific service ontologies
- 178. Challenges with respect to analytics & CEP • Traditional mining and analytic techniques are not ready for the scale and complexity coming from large sensor setups • connection to background knowledge (ontologies) for enrichment of sensor data for expressive feature representations needed for analytic techniques • "complex events" are in the context of WoT much more complex compared to traditional "complex events" research • real-time response on complex events appearing in WoT setups 178 • ...in particular:
- 179. Outline Part III. Demos, Tools & Research directions Use cases • What systems and prototypes exist? Open problems • Are there unsolved problems? Summary • What was this tutorial about? List of sources for further studies 179 • Where to start digging?
- 180. Summary The tutorial had 3 parts: 1. Motivation & background • • 2. Problems that the Web of Things can solve Components and complexity of the system, from “Things” to Apps and Services • Quick start Technology and tools for exploiting the WoT 3. • Semantic aspects • Analytic aspects • Services Demos, Tools & Research directions • Overview of existing setups and tools used for their implementation Research directions 180 •
- 181. Outline Part III. Demos, Tools & Research directions Use cases • What systems and prototypes exist? Open problems • Are there unsolved problems? Summary • What was this tutorial about? List of sources for further studies 181 • Where to start digging?
- 182. Relevant Conferences WWW - International World Wide Web Conferences • ICML – International Conference of Machine Learning • NIPS – Neural Information Processing Systems • KDD – ACM Knowledge Discovery in Databases • ICWS - IEEE International Conference on Web Services • ISWC – International Semantic Web Conference • IPSN – Information Processing in Sensor Networks • Percom - IEEE Pervasive Computing and Communication • SenSys - ACM Conference on Embedded Networked Sensor Systems • MobiSys - International Conference on Mobile Systems, Applications, and Services • INSS – International Conference on Networked Sensing Systems • DCOSS - International Conference on Distributed Computing in Sensor Systems • iThings - IEEE International Conference on Internet of Things Apps and Services “Glue” 182 •
- 183. Relevant Workshops • WebOfThings - International Workshop on the Web of Things • SensorKDD - International Workshop on Knowledge Discovery from Sensor Data • PURBA - Workshop on Pervasive Urban Applications • Urban-IOT – the Urban Internet of Things Workshop • Web Enabled Objects International Workshop on WebEnabled Objects 183 • ….
- 184. Books on data streams
- 185. Books on event processing
- 186. 186 Books on sensor networks
- 187. Relevant blogs • Web of Things Blog • Wireless Sensor Network Blog • The Internet of Things • Dust Networks – In the News 187 • ReadWriteWeb
- 188. Related Wikipedia Links Data Stream Mining: http://en.wikipedia.org/wiki/Data_stream_mining Complex Event Processing: http://en.wikipedia.org/wiki/Complex_Event_Processing Real Time Computing: time_computing http://en.wikipedia.org/wiki/Real- Online Algorithms: http://en.wikipedia.org/wiki/Online_algorithms Worst Case Analysis: http://en.wikipedia.org/wiki/Worst-case_execution_time
- 189. Related Wikipedia Links Web of Things: http://en.wikipedia.org/wiki/Web_of_Things Internet of Things: http://en.wikipedia.org/wiki/Internet_of_Things Wireless Sensor Networks: http://en.wikipedia.org/wiki/Wireless_Sensor_Networks Major Appliance: http://en.wikipedia.org/wiki/Household_appliances RFID – Radio Frequency Identification: http://en.wikipedia.org/wiki/RFID
- 190. Video Tutorials State of the Art in Data Stream Mining: Gama, University of Porto Joao • http://videolectures.net/ecml07_gama_sad/ Data stream management and mining: Georges Hebrail, Ecole Normale Superieure • http://videolectures.net/mmdss07_hebrail_dsmm/
- 191. Outline Part III. Demos, Tools & Research directions Use cases • What systems and prototypes exist? Open problems • Are there unsolved problems? Summary • What was this tutorial about? List of sources for further studies 191 • Where to start digging?
- 193. Help us improve the tutorial! Send comments and relevant info to [email protected] (…and irrelevant to [email protected] :) Acknowledgements 193 We would like to thank Miha Smolnikar, Kemal Alic, Klemen Kenda and Miha Mohorcic for contributing some slides, and the SensorLab team for their support.