Docker & aPaaS: Enterprise Innovation and Trends for 2015
Download as PPTX, PDF26 likes5,975 views
The document discusses enterprise innovation trends in 2015, emphasizing the significance of custom applications and container technologies like Docker in enhancing agility and decision-making within organizations. It highlights the challenges of implementing enterprise applications, the importance of microservices architecture, and the evolving landscape of application delivery and orchestration. The presentation suggests that a robust Docker management platform is crucial for effectively leveraging its capabilities to optimize app workloads and achieve substantial cost savings.
Docker & aPaaS: Enterprise Innovation and Trends for 2015
- 1. Docker & aPaaS: Enterprise Innovation and Trends for 2015 Samir Adams Ghosh, CEO Anurag Parashar, Technical Architect WaveMaker, Inc.
- 2. Presenters 2 Technical Architect Samir Ghosh CEO Anurag Parashar WaveMaker, Inc. • Headquartered in Mountain View, CA • Enterprise Software Platform for Custom App Delivery
- 3. Enterprises Facing Increasing Challenges • Global Competition • Scarce, Expensive Resources • Complex Supply Chains • Social-media Empowered Customers • Demanding Consumerized Employees • Disruptive Business Models • Governance and Compliance
- 4. Successful Enterprises Enable Agility Democratize Decision Making Foster Innovation IBM CIO Study – Successful Companies… • Build a workplace strategy and roadmap that drives innovation and productivity • Enable secure anytime, anywhere access instead of restricting usage
- 5. Custom Applications are Key Enablers Making information more broadly available Capturing and refining competitive and innovative processes Gather input and interaction from broader constituents Improved accessibility = faster decision making Automation for consistency and speed Improved customer service
- 6. Competitive advantage: Get the top-tier right 6 High Volume of Change. Experimentation. Systems of Innovation Incremental Change Systems of Differentiation Data. Low change apps. Systems of Record ”Pace Layering” - Gartner
- 7. Why new Applications are important? 7 Software Is Eating the World Mark Andreesen, August 2011* All New Business Strategy is powered by New Software *Read more at http://online.wsj.com/news/articles/SB10001424053111903480904576512250915629460
- 8. But there is a problem… 8
- 9. Enterprise App Gaps Build Run Business Pro Developer IT Ops 9 Long-tail app / Shadow IT problem Pro dev doesn’t want Self-service vs. Control/Legacy lock-in to RAD tools
- 10. Trends in Enterprise App Delivery Development Integration Deployment 10 Rapid API Application Development (RAAD) App Optimized Cloud (Docker containers) Microservices Architecture
- 11. Trends in Enterprise App Delivery Development Integration Deployment 11 Rapid API Application Development (RAAD) App Optimized Cloud (Docker containers) Microservices Architecture
- 12. Rapid Developers Business Expert Pro Developer • Non-programmer • Needs app quickly • Cannot wait for IT • Time constrained • Limited front end experience • Needs to extend
- 13. Modern RAAD Visual Development •WYSIWYG, Drag & Drop, Live Data Standards Based Code • AngularJS, Java/JEE, Spring, Hibernate Auto Device Responsiveness Easy Services Consumption Re-usable Components Avoid Vendor Lock-in •Open Source, Support Extendable and Maintainable • Eclipse, Maven Data Visualization SaaS & Desktop • Evaluate using data behind firewall Social Team Development •Multi-dev, feeds, follow Version Control • Git, BitBucket, Apache SVN 1-Click Deployment
- 14. Poll How should business users create custom apps for their needs? • They shouldn’t • Let them choose the tools of their choice • Provide company approved tool(s) 14
- 15. Trends in Enterprise App Delivery Development Integration Deployment 15 Rapid API Application Development (RAAD) App Optimized Cloud (Docker containers) Microservices Architecture
- 16. APIs are becoming Ubiquitous 16 11,927 APIs Sept 2014
- 17. API Success 17
- 18. Poll How easy is it to leverage APIs within your organization (or, if you’re a vendor, in the average enterprises you work with)? • Impossible • Difficult even for the Pros • Fine if you’re a Professional Developer • Possible for even non-Professional Developers • Super easy 18
- 19. Trends in Enterprise App Delivery Development Integration Deployment 19 Rapid API Application Development (RAAD) App Optimized Cloud (Docker containers) Microservices Architecture
- 20. Challenges Implementing Docker for enterprise application delivery
- 21. Problem – A lot of scripts http://blog.docker.com/2013/08/paas-present-and-future/
- 22. Problem – A lot of scripts • Combination of technologies (Java, Node, Ruby …), – Layers (DB, Web app, LB …), – Targets (dev machine, staging, production …) – Platforms (linux, windows, cloud …) • What?! Microsoft will also support Docker? • Consistent delivery is challenging, – Even with best devops tools: Puppet, Chef, Ansible ...
- 23. Ideal solution = run exactly the same image everywhere
- 24. VM vs Container. • Containers are “orders of magnitude better” than virtual machines. • Shared OS • “Multi-tenancy at OS level” • Building VMs take mins to hrs • Launching VMs takes minutes vs sub-seconds for container. 24 Phil Wainewright, Diginomica 2-Jul-2014
- 25. Virtualization Approaches Compared. Hypervisor Vs. Container 25 More Juice Same Glass Hypervisor-based VM Docker on bare metal Multiple-OS-layers
- 26. Virtualization Approaches Compared. Application density 26 Reserve Capacity App#1 App Stack Reserve space could hold other apps App#2 App could grow using available capacity App#1 Shared App Stack Hypervisor-based VM Docker on bare metal
- 27. What is Container? • Not a new concept – Solaris Zones – BSD Jails – Warden containers for CloudFoundry – LXC • Isolated OS process with its own process space, network interface, user space • Share kernel with host
- 28. What is Container? • Isolation with namespaces for – pid, mnt, net, uts , ipc, user • Control with cgroups for – memory, cpu, blkio, devices
- 29. Why Docker? • Commoditize containers – Usable tools and APIs • Copy-on-write and layers for quick provisioning • Create and share the images • Standard format for container • Reproducible images with DockerFile
- 30. Use Case Dockerfile Source Code Repository Docker Engine Docker Registry Build Docker Host 2 OS (Linux) Container A Container B Container C Container A Push Search Pull Run Host 1 OS (Linux)
- 31. What does Docker provide? • Compose application stack with Docker file • Ability to run more than one Docker container on VM • Lightweight and provision new containers in seconds • Portable and can be run on any hardware • Complete isolation for app and its stack • Keep the application footprint on the disk minimal – By separating out read-only O/S & software stack directories from writeable app-specific directories • Docker Registry public / private (portability) 31
- 32. Is Docker a cloud platform? 32 No, Docker only provides necessary tooling to build your own private cloud It is up-to the Enterprise or the Platform Vendor to get the solution right for the IT and Developer needs, bridging that gap.
- 33. What does it take to use Docker to build your cloud platform?
- 34. Orchestration – Why? • Micro Services and integration – Easy upgrades and fine grained control • Optimized Utilization • Fault Tolerance • Scalability
- 35. Optimized Utilization 35 Hibernate container, freeing the resources utilized App #3 App #3 App #2 App #1 Stack v1.0 App #4 App #1 Stack v1.0 Activate new app container from saved app data 1 Passivate app 2 contents to disk for later activation 3
- 36. Fault Tolerance 36 App #3 App #2 App #1 Stack v1.0 App #3 App #4 App #1 Stack v1.0 App #4 App #2 App #1 Stack v1.0 Should be resilient to app & stack failures, as well as VM or hardware problems
- 37. Horizontal Scaling & Replication 37 App #2 App #1 Stack v1.0 App #2 App #1 Stack v1.0 Load balance app requests across containers
- 38. Orchestration tools • CoreOS/Fleet • Kubernetes from Google • Apache Mesos / Marathon • Consul from Hashicorp • Geard from Redhat • Helios • Centurian • Fig • Shipper • … Docker libswarm - Standardized APIs for Docker Orchestration (and more) - Adaptors for actual implementations - Far from finalized
- 39. Orchestration - Bootstrapping Docker • Setting up and managing Docker – Boot2docker for developer machine – Evolving lightweight OS like CoreOS provides Docker by default. Can be initialized with PXE boot. – On other infrastructure like public clouds, OpenStack, VMWare or a single VM setup using Vagrant/Virtual Box • Separate set of instructions. – “docker hosts” command proposal • https://github.com/docker/docker/issues/8681 • Plugins/adaptors for different providers
- 40. Orchestration - Communication • Create and manage a multi container service – Docker acquired Fig, plans to include it in Docker product. But multi-host support still evolving. • Inter container communication – Docker provides port mapping for configuring container port to the external port on the machine/VM (-P or –p options) – Docker also has container linking for local containers (link based on container name and environment properties). Static links as of now. – Ambassador pattern for solving static nature of links and providing fault tolerance. – Missing multi host inter container communication with docker container linking. – Docker Networking Proposals • https://github.com/docker/docker/issues/8951 - Multi host networking • https://github.com/docker/docker/issues/8952 - Network Drivers • https://github.com/docker/docker/issues/7468 - Dynamic links • https://github.com/docker/docker/issues/7467 - Upgrade network model • Enable communication among containers running on different machines. – Link containers using Open vSwitch.
- 41. Orchestration - Cluster Management • Scheduled containers on a bunch of distributed resources – Constraints (MySQL master and slave not running on the same node) – Scaling resources • Distributed configuration management • Service Discovery • Distributed init system • Logging / Monitoring / Alerting
- 42. Orchestration -Schedule containers • Optimized container placement Enterprise have different needs – Heterogeneous workloads – Limit h/w resources • Constraints (e.g. MySQL master / slave containers should not run on same host) • Docker Clustering Proposal https://github.com/aluzzardi/docker/blob/clustering-proposal/ docs/sources/userguide/cluster.md • Relevant products – CoreOS Fleet – Kubernetes – Mesos (can write custom scheduler)
- 43. Orchestration - Distributed configuration management Consistent configuration across cluster, distributed locking and consensus. • Master Election – RAFT / Paxos • Relevant products – etcd – Consul – Zookeeper
- 44. Orchestration - Service Discovery • Env variables – static • Mounted configuration – still need to reload files • DNS – how will restart with new IP work? • Ambassador Pattern – Still using env variables, but more dynamic and a proxy container can be restarted. • Other products – etcd (CoreOS and Kubernetes use this) – Consul – HAProxy(Marathon) – confd (watcher for etcd, env variable, consul) • Service Events
- 45. Orchestration - Cluster Service Manager • For fault tolerance and scalability • Like systemd for cluster • Relevant products – Fleet for CoreOS – Marathon for Mesos – Kubernetes
- 46. 46 Managing and migrating data volumes Container Platform Stack Host VM App Data resides in an attached device/volume Volume Layer Container Platform Stack Host VM Move & swap containers 1. Portable containers enable seamless Application Stack upgrades separating Application data from Stack & libraries 2. Containers restored quickly in the event of Host VM failures 3. Allows replication of Containers for scalability Failed VM & Container
- 47. Recovery and Backup • File systems which makes snapshotting easier – AUFS file based snapshots – BTRFS other block snapshotting file systems – OverlayFS • Still need to create integration for backup infrastructure and recovery semantics
- 48. 48 Recovery and Backup App #3 App #3 App #2 App #1 Stack v1.0 App #2 App #1 Stack v1.0 Keep data transfers for snapshots minimal and hence a faster recovery time Incremental snapshots at time intervals 1 2 OverlayFS/AUFS/BTRFS provides support for incremental snapshots
- 49. Upgrades • Support for rolling upgrades for 0-downtime upgrades in redundant distributed deployments
- 50. Application Stack Upgrades 50 Create a container with new version of application stack App #2 App #1 Stack v1.0 App #2 App #1 Stack v2.0
- 51. Application versions & migration 51 App #1 version2 App #1 Stack v1.0 Swap out older version after successful migration Use same stack for guaranteed stability
- 52. Logging / Monitoring / Alerting • cAdvisor for Kubernetes • libswarm (API for aggregation)
- 53. Application Stack creation and management • Address complexity of creation of application stack – Developers need not be aware of docker image creation and management – Multi-image orchestrated application stacks need to be made reusable. – Panamax enabling application templates sharing through a central repository. • Better abstraction is needed for fine tuning existing stacks.
- 54. Security • Implementation needs to ensure end to end security. • Not giving host root access to containers. • Docker support – SELinux – AppArmor – Seccomp • Application Isolation, disable inter container communication. • Multi-tenancy
- 55. Docker Related Infrastructure Technologies
- 56. Mesos • Popular open source cluster manager from apache • Enables building resource-efficient distributed systems • Efficient dynamic resource isolation and sharing across multiple distributed applications e.g. Hadoop, Spark, Memcache • Production Tested Application Scaling to 10,000+ nodes
- 57. Mesos http://mesos.apache.org/documentation/latest/mesos-architecture/
- 58. Mesos Data Center or cloud With Mesos and related technologies • Apache Mesos • Apache ZooKeeper • Mesosphere Marathon • Chronos • HAProxy
- 59. Kubernetes Builds upon Google’s more than decade experience with GAE • Manages containers across multiple hosts • Provides mechanisms for deployment, maintenance and scaling application • Multi-cloud support • Pluggable architecture • Self-healing – auto restart, auto-placement, auto-replication
- 61. CoreOS • CoreOS Linux – Minimum Linux OS – Automatic clustering – Automatic updates for security 61 • fleet – cluster-level init system – etcd + systemd https://coreos.com/using-coreos/clustering/
- 62. Service Discovery with etcd • etcd provides a service registry for containers • Service metadata can be looked up from etcd • etcd cluster allows for service lookup across a cluster of machines • Services can listen on dependent service lifecycle events, e.g.: Start, stop, update etc. 62 https://coreos.com/using-coreos/
- 63. Flocker • Flocker is a data volume manager • Flocker can run multiple containers on multiple nodes. • Connect to any node on a Flocker cluster and traffic is routed to the node hosting the appropriate container (based on port). • Flocker - Application State – Flocker manages ZFS filesystem as Docker volumes. It attaches them to your containers. – Flocker provides tools for copying those volumes between nodes. – If an application container is moved from one node to another, Flocker automatically moves the volume with it.
- 65. Technology Landscape - Summary • Address evolving problems • Fast moving technology space • No standardization PaaS can provide abstraction to handle this change
- 66. Docker is not a ready-made solution For Enterprise IT or DevOps to manage their application infrastructure using Docker, they need to understand a complex set of things: – Linux images – Read-Write & Layered file systems – Disk snapshots and other recovery mechanisms – Managing firewall and port bridges – Network configuration for containers – Setting up monitoring – Managing software stacks & their configurations – Automating deployment processes 66
- 67. A good Docker PaaS should simplify above complexities, e.g. Simplified Docker Container Management • Console on top of Docker • Provision and Manage Containers and App Stacks within Containers Effective Scheduling and Resource Utilization • Automatic Container Passivation, Hibernation, and Activation Comprehensive Monitoring • Container, Application, Deployment, Instance, Usage Role-based Access Control • For container management operations Data and Incremental Snapshots • Efficient backup support & faster recovery … 67
- 68. Poll How likely are you to explore Docker and/or Docker-architected PaaS solutions? • Neither • Docker yes, but probably don’t need Docker PaaS • Docker PaaS, yes. 68
- 69. Our Findings • WaveMaker RAAD Studio: millions of downloads • In Beta, we hosted thousands of custom Java apps across 90+ countries via our Docker PaaS on AWS. • We found: – 30x greater VM density – 50-100x performance (e.g., 2 sec vs. 5 min app start in VM) • E.g., Enables hibernation – Literally, 80% cost savings! 69
- 70. Summary • Custom apps are key to enabling agility and innovation in the enterprise. • Platform should leverage RAD + Microservices + Docker. • Docker is ideal for optimizing custom app workloads. • A well-architected Docker management platform is needed to manage complexity and extract value out of Docker. 70
- 71. Q&A Docker aPaaS trial: WaveMakerOnline.com Thank You Contact us: [email protected] WaveMaker.com