Building IoT Middleware with Microservices
Download as PPTX, PDF0 likes460 views
The document outlines the Symbiote project, which aims to enhance interoperability among Internet of Things (IoT) platforms by integrating various smart objects across different environments. It describes the architecture, development processes, and challenges of the IoT system while emphasizing the importance of microservices and the need for automated deployment. Key takeaways include the necessity of effective communication in distributed teams and the significance of API versioning and testing automation for successful project implementation.
Building IoT Middleware with Microservices
- 1. © 2019 – The symbIoTe Consortium Building IoT Middleware with Microservices Mario Kušek, University of Zagreb, FER Javantura v6, 23 Feb 2019, Zagreb Grant Agreement No 688156
- 2. © 2019 – The symbIoTe Consortium2 Who am I? @MarioKusek [email protected] University of Zagreb Faculty of Electrical Engineering and Computing Department of Telecommunications IoT Laboratory
- 3. © 2019 – The symbIoTe Consortium3 • What is a ”thing”? – Object from physical world (physical object with build in sensors and/or actuators) or virtual object – Internet Connected Object (ICO) • Has unique identifier and is connected to the Internet • Communicates and generate data (reading from environment) • Can receive data/commands from network • Can execute commands – actuate (electrical or mechanical) • Can receive data from other ICO, process them and send for processing to cloud Internet of Things - IoT
- 4. © 2019 – The symbIoTe Consortium4 • IoT platforms integrate „things” and continuously acquire data – Large distributed system – Processing large amount of data (often in real time) – Integrates and saves data from different sources – For application developers offer: • Searching for things (sensors/actuators) • Access to data • There are more then 400 platforms – Mostly specialised for one area How to integrate „things” and provide user applications?
- 5. © 2019 – The symbIoTe Consortium5 Architecture of IoT System User IoT Platform Gateway IoT Application Smart spaceCloudUser
- 6. © 2019 – The symbIoTe Consortium6 • Can not easily create cross domain applications! • Interoperability: – Data structures are different – Different measurement units – Vertical domain specific platforms • Project: symbIoTe – Symbiosis of smart objects across IoT environments – H2020 project: 2016-2018 (3 years) – 15 EU partners (universities, institutes, SMEs) Some Challenges
- 7. © 2019 – The symbIoTe Consortium7 The symbIoTe approach Core Services IoT Platform A IoT Platform B AdaptorAdaptor Domain Enabler Application Adaptor IoT Platform A IoT Platform C Adaptor Adaptor IoT Platform D Adaptor IoT Platform B Adaptor Application Agent IoT Gateway A IoT Gateway B Smart Device Smart Space Gateway Agent Agent L1 L2 L3L4 roaming Smart Space A Core Services Smart Device Smart Space B Smart Space GatewaySmart Space Gateway Smart Device
- 8. © 2019 – The symbIoTe Consortium8 • Development – Agile process (iterations) – planned 4 week releases – Weekly meetings of all developers (video conference) – One or two weeks for internal component releases – Programming language is Java – Unit testing: >70% code coverage • Tools: – Attlasian Jira as the feature planning tool – github.com as code repository tool – Git as versioning and code revisioning tool – Travis CI as Continuous Integration server Project organisation
- 9. © 2019 – The symbIoTe Consortium9 High-level Architecture
- 10. © 2019 – The symbIoTe Consortium10 Putting it all together: L1 compliance Components for semantic and syntactic interoperability + security
- 11. © 2019 – The symbIoTe Consortium11 • Each component is one microservice – Has a component owner – One microservice – mostly implemented by one organisation – One repository • Frameworks: – SpringBoot, Spring Cloud – Communication between microservices: RabbitMQ Microservices – Decisions
- 12. © 2019 – The symbIoTe Consortium12 • Spring Cloud Config – configuration • Zipkin – distributed tracing system • Spring Cloud Gateway – did not exist then – We used NGINX • Spring Cloud Netflix: – Eureka – service registration – Ribbon – REST client that uses Eureka – Feign – declarative HTTP client • For accessing core services Spring Cloud
- 13. © 2019 – The symbIoTe Consortium13 • Microservices were not divided by domains • Frequent changing of messages – Hard integration testing, lots of communication between teams • RabbitMQ – Most of communication is request/response • Design not appropriate for messaging – No message (API) versioning – Late arrival of Spring Cloud Contract (v1.0.0 Sep 2016) • Not so good support for messages (documentation and examples) Problems
- 14. © 2019 – The symbIoTe Consortium14 • Lots of microservices need to implement the same data objects: – Put data objects in library: SymbIoTeLibraries • Put all libraries in Jitpack (https://jitpack.io) • Problems with versions of library: – Start using semantic versioning (https://semver.org) • All components need to contact AAM (Authentication Authorization Manager) component – Created SymbIoTeSecurity library • SymbIoTeLibraries has dependency to it Problems/decisions
- 15. © 2019 – The symbIoTe Consortium15 • Each IoT platform needs to deploy cloud components – No automatic deployment • 3 stages/ways of deployment: – 1. From sources: • Support web page (github wiki) • Problems: – Complex automatization – Slow automatization – Hard to configure configuration generator (web page similar to https://start.spring.io) Deployment (1)
- 16. © 2019 – The symbIoTe Consortium16 • 2. From jars: – Created a script for downloading, configuring, starting and stopping – Problems: • Microservices have startup dependencies – Solved by putting small class in main method to wait for some service (host:port) • If it doesn’t work first time you need to dive into details – Similar to SpringBoot • Bash script - problem on windows if someone just want to try it Deployment (2)
- 17. © 2019 – The symbIoTe Consortium17 • 3. From dockers – Easy startup – Starting with docker-compose or docker swarm – Config git repo is in docker image • Problem of custom configuration – Putting config in volume – Different documentation for using docker directly (linux) or in docker machine • Problems with port mapping Deployment (3)
- 18. © 2019 – The symbIoTe Consortium18 • For testing, hackathons you run components on machine that is in local network • Cloud components need to have public IP, DNS, and certificate (for https) – Workaround is ngrok tunnelling • Different NGINX configuration Cloud components on dev machine
- 19. © 2019 – The symbIoTe Consortium19 Memory Consumption – Problem • 1.5GB on startup • 2-3GB when working • Future work: – Java 11: new GC, String internals, netty, ACDS, … – Problem: late release of SpringBoot 2.1 (Oct 2018) and Spring Cloud (Greenwich)(Jan 2019)
- 20. © 2019 – The symbIoTe Consortium20 • Distributed teams require lots of communication • Be careful in choosing technologies • Divide features to microservices that are independent and scalable (up/down) • Versioning of API is important • Dedicate time for building/testing automation – Do it in the beginning of the project and maintain Conclusions
- 21. © 2019 – The symbIoTe Consortium www.symbiote-h2020.eu middleware.symbiote-h2020.eu [email protected] @symbiote_h2020 H2020 symbIoTe github.com/symbiote-h2020 Member of Thank you! Questions? @MarioKusek [email protected]