Javantura v6 - Building IoT Middleware with Microservices - Mario Kusek
0 likes244 views
In H2020 EU project symbIoTe (symbiosis of smart objects across IoT environments) we have been building IoT middleware based on microservices programmed in Java with Spring Boot and Spring Cloud components. Here I will present our experiences in developing such services in distributed team across EU and employed by 15 organizations. I will present organizational and technical advantages and drawbacks as well as our choices in building such system.
Ad
More Related Content
Similar to Javantura v6 - Building IoT Middleware with Microservices - Mario Kusek (20)
More from HUJAK - Hrvatska udruga Java korisnika / Croatian Java User Association (20)
Ad
Recently uploaded (20)
Ad
Javantura v6 - Building IoT Middleware with Microservices - Mario Kusek
- 1. © 2019 – The symbIoTe Consortium Building IoT Middleware with Microservices Mario Kušek, University of Zagreb, FER Javantura v6, 23 Feb 2019, Zagreb Grant Agreement No 688156
- 2. © 2019 – The symbIoTe Consortium2 Who am I? @MarioKusek [email protected] University of Zagreb Faculty of Electrical Engineering and Computing Department of Telecommunications IoT Laboratory
- 3. © 2019 – The symbIoTe Consortium3 • What is ”thing”? – Object from physical world (physical object with build in sensors and/or actuators) or virtual object – Internet Connected Object (ICO) • Has unique identifier and is connected to the Internet • Communicates and generate data (reading from environment) • Can receive data/commands from network • Can execute commands – actuate (electrical or mechanical) • Can receive data from other ICO, process them and send for processing to cloud Internet of Things - IoT
- 4. © 2019 – The symbIoTe Consortium4 • IoT platforms integrates „things” and continuously acquire data – Large distributed system – Processing large amount of data (often in real time) – Integrates and saves data from different sources – For application developers offer: • Searching for things (sensors/actuators) • Access to data • There are more then 400 platforms – Mostly specialised for one area How to integrate „things” and provide user applications?
- 5. © 2019 – The symbIoTe Consortium5 Architecture of IoT System User IoT Platform Gateway IoT Application Smart spaceCloudUser
- 6. © 2019 – The symbIoTe Consortium6 • Can not easy create cross domain applications? • Interoperability: – Data structures are different – Different measurement units – Platforms vertically oriented • Project: symbIoTe – Symbiosis of smart objects across IoT environments – H2020 project: 2016-2018 (3 years) – 15 EU partners (universities, institutes, SMEs) Some Challenges
- 7. © 2019 – The symbIoTe Consortium7 The symbIoTe approach Core Services IoT Platform A IoT Platform B AdaptorAdaptor Domain Enabler Application Adaptor IoT Platform A IoT Platform C Adaptor Adaptor IoT Platform D Adaptor IoT Platform B Adaptor Application Agent IoT Gateway A IoT Gateway B Smart Device Smart Space Gateway Agent Agent L1 L2 L3L4 roaming Smart Space A Core Services Smart Device Smart Space B Smart Space GatewaySmart Space Gateway Smart Device
- 8. © 2019 – The symbIoTe Consortium8 • Development – Agile process (iterations) – planned 4 week releases – Weekly meetings of all developers (video conference) – One or two weeks for internal component releases – Programming language is Java – Unit testing: >70% code coverage • Tools: – Attlasian Jira as the feature planning tool – github.com as code repository tool – Git as versioning and code revisioning tool – Travis CI as Continuous Integration server Project organisation
- 9. © 2019 – The symbIoTe Consortium9 High-level Architecture
- 10. © 2019 – The symbIoTe Consortium10 Putting it all together: L1 compliance Components for semantic and syntactic interoperability + security
- 11. © 2019 – The symbIoTe Consortium11 • Each component – Have component owner – One microservice (mostly in one organisation) – One repository • Frameworks: – SpringBoot, Spring Cloud – Communication between microservices: RabbitMQ Microservices – Decisions
- 12. © 2019 – The symbIoTe Consortium12 • Spring Cloud Config – configuration • Zipkin – distributed tracing system • Spring Cloud Gateway – did not exist then – We used NGINX • Spring Cloud Netflix: – Eureka – service registration – Ribbon – REST client that uses Eureka – Feign – declarative HTTP client • For accessing core services Spring Cloud
- 13. © 2019 – The symbIoTe Consortium13 • Microservices were not divided by domains • Frequent changing of messages – Hard integration testing, lots of communication between teams • RabbitMQ – Most of communication is request/response • Architecture not designed for messaging – No messages (API) versioning – Late knowledge about Spring Cloud Contract (v1.0.0 Sep 2016) • Not so good support for messages (documentation and examples) Problems
- 14. © 2019 – The symbIoTe Consortium14 • Lots of microservices need to implement the same data objects: – Put data objects in library: SymbIoTeLibraries • Put all libraries in Jitpack (https://jitpack.io) • Problems with versions of library: – Start using semantic versioning (https://semver.org) • All components need to contact AAM (Authentication Authorization Manager) component – Created SymbIoTeSecurity library which is dependency of SymbIoTeLibraries Problems/decisions
- 15. © 2019 – The symbIoTe Consortium15 • Each IoT platform needs to deploy cloud components – No automatic deployment • 3 stages/ways of deployment: – 1. From sources: • Support web page (github wiki) • Problems: – Complex → automatization – Slow → automatization – hard to configure → configuration generator (web page similar to https://start.spring.io) Deployment (1)
- 16. © 2019 – The symbIoTe Consortium16 • 2. From jars: – Created script for downloading, configuring, starting and stopping – Problems: • Microservices have startup dependencies – Solved by putting small class in main method to wait for some service (host:port) • If doesn’t work first time you need to dive into details – Similar to SpringBoot • Bash script - problem on windows if someone just want to try it Deployment (2)
- 17. © 2019 – The symbIoTe Consortium17 • 3. From dockers – Easy startup – Starting with docker-compose or docker swarm – Config git repo is in docker image • Problem of custom configuration – Putting config in volume – Different documentation for using docker directly (linux) or in docker machine • Problems with port mapping Deployment (3)
- 18. © 2019 – The symbIoTe Consortium18 • For testing, hackathons you run components on machine that is in local network • Cloud components need to have public IP, DNS, and certificate (for https) – Workaround is ngrok tunnelling • Different NGINX configuration Cloud components on dev machine
- 19. © 2019 – The symbIoTe Consortium19 Memory Consumption – Problem • 1.5GB on startup • 2-3GB when working • Solution: – Java 11: new GC, String internals, netty, … – Problem: late release of SpringBoot 2.1 (Oct 2018) and Spring Cloud (Greenwich)(Jan 2019)
- 20. © 2019 – The symbIoTe Consortium20 • Distributed teams require lots of communication • Be careful in choosing technologies • Divide to microservices that are independent and scalable (up/down) • Versioning of API is important • Dedicate time for building/testing automation – To it in the beginning of the project and maintain Conclusions
- 21. © 2019 – The symbIoTe Consortium www.symbiote-h2020.eu middleware.symbiote-h2020.eu [email protected] @symbiote_h2020 H2020 symbIoTe github.com/symbiote-h2020 Member of Thank you! Questions? @MarioKusek [email protected]