[NYC Meetup] Docker at Nuxeo
11 likes3,494 views
In this talk, Damien describes the infrastructure Nuxeo has built around Docker containers, which is mainly based on CoreOS and Docker, and how it provides a way to generically run applications not only on a single host, but across a whole cluster of hosts. The resulting architecture can be used to implement a PaaS approach for any application.
![Application startup unit
!
[Unit]!
Description=Sample startup script!
!
[Service]!
ExecStartPre= Set starting status in etcd!
ExecStart= Launch docker container!
ExecStartPost= Wait container to be launched and set app location in etcd!
ExecStop= Stop docker container!
ExecStopPost= Set stopped status in etcd!](https://image.slidesharecdn.com/dockermeetupnyc-140605110024-phpapp01/85/NYC-Meetup-Docker-at-Nuxeo-40-320.jpg)
![[오픈소스컨설팅] 쿠버네티스와 쿠버네티스 on 오픈스택 비교 및 구축 방법](https://cdn.slidesharecdn.com/ss_thumbnails/osck8svsk8sonopenstackkhoj-210310051504-thumbnail.jpg?width=560&fit=bounds)
![[2018.10.19] Andrew Kong - Tunnel without tunnel (Seminar at OpenStack Korea ...](https://cdn.slidesharecdn.com/ss_thumbnails/tunnelwithouttunnel-181022130534-thumbnail.jpg?width=560&fit=bounds)
![[오픈소스컨설팅] EFK Stack 소개와 설치 방법](https://cdn.slidesharecdn.com/ss_thumbnails/elasticstack-210712042246-thumbnail.jpg?width=560&fit=bounds)
Ad
More Related Content
What's hot (20)
Viewers also liked (10)
Ad
Similar to [NYC Meetup] Docker at Nuxeo (20)
Ad
More from Nuxeo (20)
Recently uploaded (20)
[NYC Meetup] Docker at Nuxeo
- 2. Who we are
- 3. What is Nuxeo platform • A platform to develop you own content apps • Highly pluggable thru extension points • Model you content with Nuxeo Studio • Point and click UI to define your business model • with hot-reload
- 4. Nuxeo tech. stack • Java stack : Seam + JSF for Nuxeo DM and DAM • Rest API to build your JS Apps • Nuxeo Studio (Saas) to model your business • Thru data model (doc types, schemas) • Thru Automation chains
- 5. Nuxeo in the cloud ?
- 6. Goals • Customers ask to host their app • Provide quick access to test instances (onboarding) • Provide onDemand dev or CI instances
- 7. Goals • Customers ask to host their app • Provide quick access to test instances (onboarding) • Provide onDemand dev or CI instances This is the nuxeo.io promise
- 9. Manual OnDemand • Based on AWS CloudFormation • Manually deployed • Works but : • not scalable • manual admin & monitoring • difficult upgrade process
- 10. Second attempt
- 11. AWS Automation • Provisionning may be automated • Working mockup (python, celery, boto) but… • Strong adherence to AWS • difficult to test • slow provisionning (one EC2 instance per Nuxeo instance) • upgrade process still difficult • scaling out is difficult • sensible to failures
- 12. This leads to our challenge
- 13. Current challenge • Build a scalable infrastructure • Quick provisioning (less than a minute) • Easy setup (no admin needed) • Easy upgrade process • Automated healing • Easy to test
- 14. How Docker solves some of these issues ?
- 15. Quick provisioning • We don’t have to start a server • No VM overhead • Start time is Nuxeo start time
- 16. Data free containers • Docker container doesn’t hold data • Upgrading is just using a new version of the image • Healing means killing and restarting Nuxeo v5.9.3 S3Postgres
- 17. Data free containers • Docker container doesn’t hold data • Upgrading is just using a new version of the image • Healing means killing and restarting S3Postgres
- 18. Data free containers • Docker container doesn’t hold data • Upgrading is just using a new version of the image • Healing means killing and restarting S3Postgres Nuxeo v5.9.4
- 19. Several containers per server • Scaling is just a matter of starting several containers • But means infinite CPU / RAM and IO Nuxeo.io server io container NXIO-0001 io container NXIO-0002 io container NXIO-0003 io container NXIO-XXXX …
- 20. Challenge status • Build a scalable infrastructure • Quick provisioning (less than a minute) • Easy setup (no admin needed) • Easy upgrade process • Automated healing • Easy to test
- 21. How to solve the other issues ? … with a little help from my friends
- 22. The infinite resource computer…
- 23. … is a cluster
- 24. Docker is not sufficient • Docker handles processes at the OS level • On the cluster level we need a companion • To load balance the processes • To check the statuses of the processes • We need a docker ps at the cluster level
- 25. Introducing CoreOS • A minimal Linux distribution (small footprint) • Designed to handle cluster • Providing cluster tooling • Docker • etcd • fleet https://coreos.com/
- 26. etcd : service discovery • etcd is • small distributed key value store • keys may have TTL • using Raft protocol to elect cluster leader • suitable for : • shared configuration • service discovery
- 27. fleet : clustered init system • fleet is : • an init system at the cluster level • compatible with systemd unit files • suitable for : • launching docker containers • distributing load across cluster • get the status of the jobs
- 28. fleet demo • Deploy our nuxeo container in the cluster
- 29. Challenge status • Build a scalable infrastructure • Quick provisioning (less than a minute) • Easy setup (no admin needed) • Easy upgrade process • Automated healing • Easy to test
- 30. What are the admin tasks ? • Launch the fleetctl command • Setup DNS and reverse proxies • Link container to external services (S3, Postgres etc…)
- 31. DNS and reverse proxy • docker start containers and link host port to container port • The cluster has to redirect calls to mycustomDNS.io to the running container meaning knowing • the ip of the host • the port on the host
- 32. Introducing Gogeta ! /services/NXIO-0001/location:{host:172.31.51.1,port:49153}! /status/current:started! /expected:started! !! ! ! ! ! /alive:1! /domains/mydomain.nuxeo.io/type:iocontainer! /value:NXIO-0001 Given this etcd configuration
- 33. Cluster CoreOS 172.31.51.20 etcd io container NXIO-0098 systemd CoreOS 172.31.51.1 etcd io container NXIO-0001 Gogeta systemd … … … … Fleet … Gogeta … Introducing Gogeta Round robin load Balancer etcd ! /services/NXIO-0098/location:{host:172.31.51.20,port:49153}! /status/current:started! /expected:started! !! ! ! ! ! /alive:1! /domains/mydomain.nuxeo.io/type:iocontainer! /value:NXIO-0001
- 34. Cluster CoreOS 172.31.51.20 etcd io container NXIO-0098 systemd CoreOS 172.31.51.1 etcd io container NXIO-0001 Gogeta systemd … … … … Fleet … Gogeta … Introducing Gogeta Round robin load Balancer etcd ! /services/NXIO-0098/location:{host:172.31.51.20,port:49153}! /status/current:started! /expected:started! !! ! ! ! ! /alive:1! /domains/mydomain.nuxeo.io/type:iocontainer! /value:NXIO-0001
- 35. Cluster CoreOS 172.31.51.20 etcd io container NXIO-0098 systemd CoreOS 172.31.51.1 etcd io container NXIO-0001 Gogeta systemd … … … … Fleet … Gogeta … Introducing Gogeta Round robin load Balancer etcd ! /services/NXIO-0098/location:{host:172.31.51.20,port:49153}! /status/current:started! /expected:started! !! ! ! ! ! /alive:1! /domains/mydomain.nuxeo.io/type:iocontainer! /value:NXIO-0001
- 36. Cluster CoreOS 172.31.51.20 etcd io container NXIO-0098 systemd CoreOS 172.31.51.1 etcd io container NXIO-0001 Gogeta systemd … … … … Fleet … Gogeta … Introducing Gogeta Round robin load Balancer etcd ! /services/NXIO-0098/location:{host:172.31.51.20,port:49153}! /status/current:started! /expected:started! !! ! ! ! ! /alive:1! /domains/mydomain.nuxeo.io/type:iocontainer! /value:NXIO-0001
- 37. Cluster CoreOS 172.31.51.20 etcd io container NXIO-0098 systemd CoreOS 172.31.51.1 etcd io container NXIO-0001 Gogeta systemd … … … … Fleet … Gogeta … Introducing Gogeta Round robin load Balancer etcd ! /services/NXIO-0098/location:{host:172.31.51.20,port:49153}! /status/current:started! /expected:started! !! ! ! ! ! /alive:1! /domains/mydomain.nuxeo.io/type:iocontainer! /value:NXIO-0001
- 38. Cluster CoreOS 172.31.51.20 etcd io container NXIO-0098 systemd CoreOS 172.31.51.1 etcd io container NXIO-0001 Gogeta systemd … … … … Fleet … Gogeta … Introducing Gogeta Round robin load Balancer etcd ! /services/NXIO-0098/location:{host:172.31.51.20,port:49153}! /status/current:started! /expected:started! !! ! ! ! ! /alive:1! /domains/mydomain.nuxeo.io/type:iocontainer! /value:NXIO-0001
- 39. Introducing Gogeta • Easy to start as a docker container ! ! • Written in Go • Available at : http://github.com/nuxeo/gogeta /usr/bin/docker run --rm --name gogeta -p 7777:7777 nuxeo/gogeta
- 40. Application startup unit ! [Unit]! Description=Sample startup script! ! [Service]! ExecStartPre= Set starting status in etcd! ExecStart= Launch docker container! ExecStartPost= Wait container to be launched and set app location in etcd! ExecStop= Stop docker container! ExecStopPost= Set stopped status in etcd!
- 41. Linking services • Docker recommend to use ambassadors to link services • Service may be hosted on other servers or outside of the cluster • We’ve implemented dynamic ambassadors • http://coreos.com/blog/docker-dynamic-ambassador- powered-by-etcd/
- 42. Cluster CoreOS 172.31.51.20 etcd Postgres systemd CoreOS 172.31.51.1 etcd io-container NXIO-0001 systemd … … … … Fleet postgres-amb Dynamic ambassador etcd ! /services/postgres/location:{host:172.31.51.20,port:49159}! https://github.com/nuxeo/etcd-netfw
- 43. Cluster CoreOS 172.31.51.20 etcd Postgres systemd CoreOS 172.31.51.1 etcd io-container NXIO-0001 systemd … … … … Fleet postgres-amb Dynamic ambassador etcd ! /services/postgres/location:{host:172.31.51.20,port:49159}! https://github.com/nuxeo/etcd-netfw
- 44. Cluster CoreOS 172.31.51.20 etcd Postgres systemd CoreOS 172.31.51.1 etcd io-container NXIO-0001 systemd … … … … Fleet postgres-amb Dynamic ambassador etcd ! /services/postgres/location:{host:172.31.51.20,port:49159}! https://github.com/nuxeo/etcd-netfw
- 45. Dynamic ambassador • Very easy to start ! • Uses https://github.com/nuxeo/etcd-netfw docker run --rm --name postgres-amb -P nuxeo/service-amb postgres-service
- 46. Launching commands • A manager web app now launches commands : • sets up basic etcd values (like domain to service resolution) • launches fleet commands (start & stop apps) • polls statuses
- 47. Challenge status • Build a scalable infrastructure • Quick provisioning (less than a minute) • Easy setup (no admin needed) • Easy upgrade process • Automated healing • Easy to test
- 48. Let’s resume
- 49. From nuxeo.io… • We’ve built a cluster infrastructure that : • knows how to deploy Docker containers (with fleet) • Expose those containers thru domain name resolution (with gogeta) • Link dependent services to the container (with etcd-netfw)
- 50. From nuxeo.io… • Given that a container should represent an application • Running this apps means : • Launch it with runtime parameters • Link it to its dependent services • Then we can build a metamodel around it
- 51. … name: io-container! version: 1.0! params:! - env_tech_id: ! - description: The id of the environment to run! - type:string! - clid:! - description: a registered connect client ID! - type:string! ! buildimage:! path: "./docker"! ! dependencies:! - postgres.service! - s3.service! !
- 52. … to arken • to be able to run any application ! • it will take care of : • Submitting the model to the cluster • Building the image if needed • Launching the container • Registering everything in etcd arkenctl start iocontainer.ark —domain myapp.mycluster.io
- 53. Arken status • We are extracting things from http://github.com/nuxeo/nuxeo.io • http://github.com/arkenio • home of gogeta and etcd • provide cluster init instruction • arken and arkenctl tools needs to be done • new opensource projects (not in nuxeo repo) • pull requests are welcome
- 54. Conclusion • Docker helped us to enter the micro containers world • Lots of new features : multi-repo, automatic /etc/hosts etc… • CoreOS provided us cluster deployment • Arken wants to provide a higher deployment model