FIWARE Tech Summit - Docker Swarm Secrets for Creating Great FIWARE Platforms
0 likes198 views
Presentation by Federico Facca Head of Martel Lab, Martel Innovate FIWARE Tech Summit 28-29 November, 2017 Malaga, Spain
![19
Context Broker: How to implement that in Docker Swarm?
§ The Load Balancer
• It is the easy part: Docker Swarm implements
a simple Load Balancing mechanism
§ Context Broker API HA
• Context Broker is stateless, we don’t have to
worry about data
• We create a service (using replica mode to
scale it up and down)
• We leverage on health checks to evaluate
single instance health
§ MongoDB
• Now things get complex... Recall CAPs
Theorem
version: '3'
services:
orion:
image: fiware/orion:${ORION_VERSION:-1.7.0}
ports:
- "1026:1026”
command: -logLevel DEBUG -dbhost
${MONGO_SERVICE_URI:-"mongo-rs_mongo"} -rplSet
${REPLICASET_NAME:-rs} -dbTimeout 10000
deploy:
replicas: 2
healthcheck:
test: ["CMD", "curl", "-f",
"http://0.0.0.0:1026/version"]
interval: 1m
timeout: 10s
retries: 3
networks:
...](https://image.slidesharecdn.com/2-171207110835/85/FIWARE-Tech-Summit-Docker-Swarm-Secrets-for-Creating-Great-FIWARE-Platforms-20-320.jpg)
![21
Context Broker: How to implement that in Docker Swarm?
§ MongoDB
• We create a service for mongo (using
global, and volumes if we want persistency)
• We create a service for the sidecar
microservice
• We leverage on health checks to evaluate
single instance health
§ Why global?
• If you want to leverage on volume for data
persistency, you need to deal with the fact
that there can be only 1 volume with a
given name per swarm node.
• How can I scale up / down then?
□ Using placement constraints!
version: '3.2'
mongo:
image: mongo:${MONGO_VERSION:-3.2}
entrypoint: [ "/usr/bin/mongod", "--replSet",
"${REPLICASET_NAME:-rs}", "--journal", "--smallfiles"]
volumes:
- mongodata:/data/db
secrets:
- mongo-healthcheck
healthcheck:
test: ["CMD", "bash", "/run/secrets/mongo-healthcheck"]
interval: 1m
timeout: 10s
retries: 3
deploy:
mode: global
...
controller:
image: martel/mongo-replica-ctrl:latest
volumes:
- /var/run/docker.sock:/var/run/docker.sock
deploy:
mode: replicated
replicas: 1
placement:
constraints: [node.role==manager]
...](https://image.slidesharecdn.com/2-171207110835/85/FIWARE-Tech-Summit-Docker-Swarm-Secrets-for-Creating-Great-FIWARE-Platforms-22-320.jpg)
![22
Scaling Up and Down
§ docker service scale orion_orion=3
§ docker service scale orion_orion=2
§ Global Mode does not support scale up /
down. Using Global Mode you can have as
many mongo as cluster nodes.
§ Add a placement constraint to the mongo
service
• placement:
constraints: [node.labels.mongo == yes]
§ Add/remove label to nodes to be (not) used
for MongoDB
• docker node update --label-add
mongo=yes NODE
Context Broker MongoDB](https://image.slidesharecdn.com/2-171207110835/85/FIWARE-Tech-Summit-Docker-Swarm-Secrets-for-Creating-Great-FIWARE-Platforms-23-320.jpg)
Ad
More Related Content
What's hot (20)
Similar to FIWARE Tech Summit - Docker Swarm Secrets for Creating Great FIWARE Platforms (20)
Ad
More from FIWARE (20)
Ad
Recently uploaded (20)
FIWARE Tech Summit - Docker Swarm Secrets for Creating Great FIWARE Platforms
- 1. Docker Swarm secrets for creating great FIWARE platforms Federico M. Facca email: [email protected] twitter: @chicco785
- 2. 1 What you will learn? § How to deploy enablers using cloud architecture patterns § How to apply cloud patterns using Docker § How to deploy on multiple hardware architectures § Where to find examples for some enablers
- 3. 2 In case you wanna run the 5 demo at the end of the Talk § If internet is good enough... § Install VirtualBox • https://www.virtualbox.org/wiki/Downloads § Install Docker • https://docs.docker.com/engine/installation/ § Install Docker Machine • https://docs.docker.com/machine/install-machine/ § Create a Swarm Cluster (https://github.com/aelsabbahy/miniswarm) • curl -sSL https://raw.githubusercontent.com/aelsabbahy/miniswarm/master/miniswarm -o /usr/local/bin/miniswarm • chmod +rx /usr/local/bin/miniswarm # As root • miniswarm start 3 # 1 manager 2 workers § Clone the recipes • git clone https://github.com/smartsdk/smartsdk-recipes
- 4. 3 Does this work only with Docker Swarm? § The code you will find in the repository is for Docker Swarm § The principles are generic and can be applied on different containerized (or not) platforms
- 5. 4 Why Docker Swarm instead of Kubernetes? § K8S is more production ready • More advanced features (e.g. autoscaling). § Swarm is simpler • It is included in Docker (N.B. K8S will be soon) • It is more suitable for “educational” purposes • It runs better on a RasperryPI 0 J
- 6. Learn and understand the basics
- 7. 6 What are cloud patterns? Why it is important to master them? Virtualization <> Cloudification
- 8. 7 Cattle vs Pets Cloud Native Applications Legacy Applications
- 9. 8 Monolith vs Modern SOA (aka microservices) § Monolith architectures run all their services in a single process § Monolith architectures may scale by replicating all the “monolith” on different servers § Microservice architectures each functionality in a separate (possibly stateless) process § Microservices scale individually by distributing on different servers
- 10. 9 Cloud Architecture Patterns are the path to § Move from Pets to Cattle § ... i.e. achieve • Service resiliency • Flexible scalability • Lower latency
- 12. 11 Scalability Horizontal Scaling Vertical Scaling
- 13. 12 Multisite
- 14. 13 Queue Centric Workflow Message Queue Producers Consumers
- 15. 14 Stateless vs Stateful services § Stateless services • The output of the service depends only on the input • Easy to scale and distribute § Stateful • The output of the service depends on the input and on a set of information stored by the service itself • Not so easy to scale and distribute (maintaining a consistent state)
- 16. 15 CAP Theorem § The CAP theorem states that it is impossible for a distributed computer system to simultaneously provide all three of the following guarantees: • Consistency: Every read receives the most recent write or an error • Availability: Every request receives a response, without guarantee that it contains the most recent version of the information • Partition tolerance: The system continues to operate despite an arbitrary number of messages being dropped by the network between nodes § I.e. when you implement HA in a stateful service, you can choose of being CA, AP, CP. In general you strive to AP and eventually consistent.
- 17. From concepts to practise 16
- 18. 17 Context Broker § Context Broker is perhaps the most used GE J § It includes to components: • The API • The Backend § The API is HTTP based § The Backend in based on MongoDB § How to make it high available? • An easy crossover mechanism for HTTP APIs are Load Balancers • MongoDB has its proprietary HA mechanism (replica set) Context Broker MongoDB
- 19. 18 Context Broker: Target architecture Context Broker MongoDB Context Broker MongoDB Context Broker MongoDB LB LB LB MongoDB replica set Virtual IP 1. Provide high available and partition tolerant distributed data 2. Eventually consistent 3. MongoDB HA solutions use quora mechanism for evaluate consistency, so O as to be an odd number (max actually is 7) 1. Provides the reliable cross over (i.e. transparent access to different instances) 2. Provides the transparent detection failure 3. Relies on virtual IP mechanism 1. N-instances of context broker, removing single point of failure
- 20. 19 Context Broker: How to implement that in Docker Swarm? § The Load Balancer • It is the easy part: Docker Swarm implements a simple Load Balancing mechanism § Context Broker API HA • Context Broker is stateless, we don’t have to worry about data • We create a service (using replica mode to scale it up and down) • We leverage on health checks to evaluate single instance health § MongoDB • Now things get complex... Recall CAPs Theorem version: '3' services: orion: image: fiware/orion:${ORION_VERSION:-1.7.0} ports: - "1026:1026” command: -logLevel DEBUG -dbhost ${MONGO_SERVICE_URI:-"mongo-rs_mongo"} -rplSet ${REPLICASET_NAME:-rs} -dbTimeout 10000 deploy: replicas: 2 healthcheck: test: ["CMD", "curl", "-f", "http://0.0.0.0:1026/version"] interval: 1m timeout: 10s retries: 3 networks: ...
- 21. 20 Data Layer HA Management § Your distributed data layer has some level of self discovery • You can relay on it to automatically create the “data service cluster”. • In some cases, you need pass service names... Luckily you can leverage on tricks (e.g. DNSRR mode of Docker Swarm – being VIP the default) • E.g. elasticsearch / hadoop § Your distributed data layer has no self discovery • You need a sidecar service that implements the data cluster management logic. • E.g. mongodb / mysql MongoDB MongoDBMongoDB MongoDB replica set ReplicaSet Controller Docker Swarm MongoDB replica set MongoDB replica set
- 22. 21 Context Broker: How to implement that in Docker Swarm? § MongoDB • We create a service for mongo (using global, and volumes if we want persistency) • We create a service for the sidecar microservice • We leverage on health checks to evaluate single instance health § Why global? • If you want to leverage on volume for data persistency, you need to deal with the fact that there can be only 1 volume with a given name per swarm node. • How can I scale up / down then? □ Using placement constraints! version: '3.2' mongo: image: mongo:${MONGO_VERSION:-3.2} entrypoint: [ "/usr/bin/mongod", "--replSet", "${REPLICASET_NAME:-rs}", "--journal", "--smallfiles"] volumes: - mongodata:/data/db secrets: - mongo-healthcheck healthcheck: test: ["CMD", "bash", "/run/secrets/mongo-healthcheck"] interval: 1m timeout: 10s retries: 3 deploy: mode: global ... controller: image: martel/mongo-replica-ctrl:latest volumes: - /var/run/docker.sock:/var/run/docker.sock deploy: mode: replicated replicas: 1 placement: constraints: [node.role==manager] ...
- 23. 22 Scaling Up and Down § docker service scale orion_orion=3 § docker service scale orion_orion=2 § Global Mode does not support scale up / down. Using Global Mode you can have as many mongo as cluster nodes. § Add a placement constraint to the mongo service • placement: constraints: [node.labels.mongo == yes] § Add/remove label to nodes to be (not) used for MongoDB • docker node update --label-add mongo=yes NODE Context Broker MongoDB
- 24. 23 Multi Site (for replicated mode services) § In each site, have at least a Docker Swarm master. • The number of master should be always odd. § Add a “site” label to all the nodes part of a given site. • docker node update --label-add region=us NODE • docker node update --label-add region=eu NODE § Add a placement preference to the service (not supported in compose files!) • docker service update --placement-pref-add 'spread=node.labels.region’ SERVICE
- 25. Quick Demo 24
- 26. 25 § cd tools/ § sh create_networks.sh § cd ../data-management/context-broker/ha/ § sh deploy_back.sh § docker service ls § docker service logs -f orion-backend_controller § sh deploy_front.sh § docker service ls § curl http://192.168.99.101:1026/version
- 28. 27 Multi Site and Edge § Edge devices may not have a Public IP § Can we create a cluster connecting such devices? § OpenVPN is your friend! • Configure OpenVPN server on all the master nodes in the cloud using a multipoint configuration. • Configure OpenVPN clients on all the edge nodes. • Unfortunately, due to the fact that docker service does not support privileged mode, you cannot run OpenVPN as a container to create a Docker Swarm cluster § What if my edge nodes are based on a different architecture (e.g. ARM)? • Develop image manifests that implements v2.2 spec, this allows to redirect an image version to specific version per hardware platform. image: myprivreg:5000/someimage:latest manifests: - image: myprivreg:5000/someimage:ppc64le platform: architecture: ppc64le os: linux - image: myprivreg:5000/someimage:amd64 platform: architecture: amd64 features: - sse os: linux
- 29. Træfik: advanced load balancing § Docker Swarm proxy is not configurable, for example it does not support sticky sessions § Traefik listens to backend /orchestrator API’s and detects any changes, applying it § Routes are dynamically managed § You can create / update / destroy routes at any time § Traefik reads service metadata on Docker / Kubernetes / etcd / etc • Hosts, ports, load balancing algorithm etc § You can configure SSL certifications • Let’s Encrypt integration requires a key-value storage • Let’s Encrypt integration requires public IP
- 30. 29 Testing your dockerized platform § Learn from the GURU’s of micro service architectures! § Chaos Monkey § https://github.com/gaia-adm/pumba Netflix picture or /logo
- 31. On going and future activities in FIWARE 30
- 32. Did it look complex? I hope not J 31
- 33. 32 Smart Security • Common architecture patterns: e.g. scalability pattern • Common generic enablers: e.g. orion context-broker • Common data models: e.g. geo-location • Specific architecture patterns: e.g. secured data access pattern • Specific and customised generic enablers: e.g. security risk detection filters for kurento media server • Specific data models: e.g. security’s events Smart Security Application “recipe” 1. Analyse HA architectures for the different Data and IoT Management enablers 2. Creating Docker compose recipes to allow easy deployment of HA enablers 3. Making them available in FIWARE Lab to experimenters
- 34. Do you have questions? Do you want to contribute? 33 Contact Us www.martel-innovate.com Federico M. Facca Head of Martel Lab [email protected] Dorfstrasse 73 – 3073 Gümligen (Switzerland) 0041 78 807 58 38
- 35. Thank you! http://fiware.org Follow @FIWARE on Twitter 34
- 37. 36