Use Docker to Deliver Cognitive Services Running Cross Platform and Multi Cloud Environments by Susan Diamond
3 likes5,186 views
Watson developer cloud delivers Watson Cognitive services as micro services on the cloud that are being used by many IBM Watson customers. The micro services were packaged in ova at the first release. There were some drawbacks in ova deployment in the cloud. We gradually switched to use docker. As a result, the service deployment time and start up time are significantly improved. It also greatly simplified our continuous delivery process since our services run on both Intel and Power platform and we have offerings on our public cloud, dedicated cloud as well as customers’ on premise cloud. With minimal deployment time and quick startup time, Docker makes our dynamic creation of service instance on the fly per customer request possible.
![6
DevOps
[Imaginator, Artifactory,
Asgard]
Data Services
[Zookeeper,Cassandra
WS3,Priam]
Remote Log Archival/
Analytics
[ELK,Cloudsight]
Dynamic Routing
[DataPower,Zuul]
Service Registry/
Discovery
[Eureka]
DevOps Config
Management
[Archaius]
Operational Visibility
[Grafana, Graphite]
Testing
[Jenkins,Chaos]
Metrics and Event
Sources
[Servo,Crawler]
Internal Service Load
Balancing
[Ribbon,Litelinks,Eureka]
Service Health Checks
[Eureka]
Service Discovery/
Registration
[Eureka, Karyon,]Microservice
Architecture
Failure Resilient
Doocker Repo
Container Management
[Mesos,Marathon]
Image Management Baremetal
VM
Availability Zones/Pods
NetworkingDNS
ObjectStorage
Provisioning
[CSB]
Authorization
Authentication
[LDAP,Apache Shiro]
Alerting
[Seyren,Uptime,
PagerDuty]
Metering
Watson
Services
Patterns
Watson
Foundation
Services
Softlayer
IaaS
Watson
Security
ID Management
[USAM]
Vulnerability Scanning
[AppScan,Nesus]
Security Logging
[QRadar]
Watson Developer Cloud Architecture
* Netflix Components](https://image.slidesharecdn.com/susandiamonddockertodelivercoginitiveservices-160711163330/85/Use-Docker-to-Deliver-Cognitive-Services-Running-Cross-Platform-and-Multi-Cloud-Environments-by-Susan-Diamond-7-320.jpg)
![Learning the Alphabet: A/B, CD and [E-Z] in the Docker Datacenter by Brett Ti...](https://cdn.slidesharecdn.com/ss_thumbnails/alphabet-160624232958-thumbnail.jpg?width=560&fit=bounds)
Ad
More Related Content
What's hot (20)
Viewers also liked (20)
Ad
Similar to Use Docker to Deliver Cognitive Services Running Cross Platform and Multi Cloud Environments by Susan Diamond (20)
Ad
More from Docker, Inc. (20)
Recently uploaded (20)
Use Docker to Deliver Cognitive Services Running Cross Platform and Multi Cloud Environments by Susan Diamond
- 2. Use Docker to Deliver Cognitive Services in Multi Cloud Environments - The Watson Developer Cloud Use Case Susan Diamond Senior Software Engineer/Continuous Delivery Leader 6/20/2016
- 3. Agenda • Watson/Watson Services/Watson Developer Cloud (WDC) Introduction • WDC Transition from VM to Docker • WDC Continuous Delivery Process • Things to Consider to Run Private Docker Registries • Watson Service Demo
- 4. What do we do …
- 6. © 2015 International Business Machines Corporation Watson Developer Cloud Architecture Goals • Deliver open platform to host WDC services and provide operational excellence ➢ Cloud Native Enablement ➢ High Availability / Auto Recovery ➢ Microservices Architecture ➢ Continuous Delivery ➢ Elasticity ➢ Operational Visibility ➢ Security / Compliance ➢ Productivity Increase for Service Developers ▪ Quick onboarding, common components, tools, process improvements 5
- 7. 6 DevOps [Imaginator, Artifactory, Asgard] Data Services [Zookeeper,Cassandra WS3,Priam] Remote Log Archival/ Analytics [ELK,Cloudsight] Dynamic Routing [DataPower,Zuul] Service Registry/ Discovery [Eureka] DevOps Config Management [Archaius] Operational Visibility [Grafana, Graphite] Testing [Jenkins,Chaos] Metrics and Event Sources [Servo,Crawler] Internal Service Load Balancing [Ribbon,Litelinks,Eureka] Service Health Checks [Eureka] Service Discovery/ Registration [Eureka, Karyon,]Microservice Architecture Failure Resilient Doocker Repo Container Management [Mesos,Marathon] Image Management Baremetal VM Availability Zones/Pods NetworkingDNS ObjectStorage Provisioning [CSB] Authorization Authentication [LDAP,Apache Shiro] Alerting [Seyren,Uptime, PagerDuty] Metering Watson Services Patterns Watson Foundation Services Softlayer IaaS Watson Security ID Management [USAM] Vulnerability Scanning [AppScan,Nesus] Security Logging [QRadar] Watson Developer Cloud Architecture * Netflix Components
- 8. Watson Developer Cloud and Docker • Started investigate Docker in mid 2014 • Lack of monitoring/management tooling • Picked up Docker again in beginning of 2015 and use docker in production environment since mid 2015 Some characteristics of Cognitive services: • Self-learning • Training data/models • Data security • One service instance per customer Dock Deployment • Deployment time significantly improved • Start up time significantly improved • Enable dynamic service instance creation on the fly • Enable seamless DevOps experience across multi-cloud environments in multi-GEOs VM deployment Pain Points: • Long deployment time • Slow start up time • Problematic in dynamic deployment automation
- 9. Transitioning from VM to Docker 1. VM/docker image building, bake the image once and use for all environments. 2. Service deployment 3. Service registration 4. Service request routing 5. Full stack in private network except DataPower
- 10. Canary & Red/Black Deployments/Updates ▪Design Philosophy: No destructive or in-place upgrades ▪New ASG deployed alongside existing ASG ▪#instances determine load (point in time) ▪Disable old ASG after sufficient confidence in new – Can flip to old with a couple of clicks in case of an issue
- 11. Ubuntu VM Docker engine Docker Registry on Docker Container Ubuntu VM Docker engine Docker Registry on Docker Container Docker container 1. Java 7 2. Tomcat 7 3. Side-car 4. Eureka 5. Zookeeper 6. Zuul … Imaginator Container 1 base image deb files 2. Services debfiles 3. Dreamfiles 4. dockerfiles … SL object storage Docker AAS SL image template repository v3 VM Image template v1 Deb files + dream file d1 Deb files + dock file d2 Store/retrieve deb files v2 Store/retrieve deb files d3 Docker image SL firewall Jenkins Git Enterprise Ubuntu VM Haproxy source code Imaginator server/Ubuntu VM Docker engine Imaginator VM AAS • Design Philosophy: Every change should be done via images • Base images available to service teams ➢ Ubuntu 14.04 LTS + security hardening + agents (logging, metrics) + JRE + Tomcat + Sidecar for non-Java services • Service teams build their image on top of base image ➢ “Dream” or Docker file describes dependencies and build targets ➢ Gradle fpm plugin for Debian package generation • Output is Softlayer CCI image or Docker image for deployment Continuous Delivery - Image Baking Process
- 12. © 2015 International Business Machines Corporation VM DevOps Console: Asgard • From Images to Running Servers – Continuous Delivery • Few button clicks to deploy one or more service instances into Auto Scaling Group (ASG) 11 • Console allows you to: ➢ Provide parameters for HA and instance recovery ➢ Deploy one or more service versions at a time ➢ Canary Testing / Red-Black Deployments ➢ Increase/Decrease number of instances in ASG ➢ Configure auto scaling policies
- 13. Docker DevOps: Active Deploy (Investigating)
- 14. Service build Server 9.x Docker deamon SL object storage Docker images Docker Registry 9.x.x.x:5000 SL object storage Docker images Dev Staging Private Docker Registries in Public Cloud Docker Registry 9.x.x.x:5001 haproxy haproxy Docker deamon Softlayer IBM intranet Docker deamon push push pull pull dockerrepo-v2-01:5000 dockerrepo-v2-02:5000 dockerregistry1-v2:5000 dockerregistry2-v2:5000 SL object storage Docker images Production haproxy Docker deamon pull dockerregistry1-v2:5000 dockerregistry2-v2:5000
- 15. • Security • Proprietary source code needs to be stored in a private registry • Customer’s security requirement: fully isolated stack • Network accessible to all docker agent in the environment. • docker agents are in private network that is not accessible to public network • Docker registries need to available in each GEO for good performance. • Maintenance • Maintaining the multiple private docker registries up to date and operational needs resources. Things to Consider to Run Private Docker Registries