Play Framework + Docker + CircleCI + AWS + EC2 Container Service
16 likes13,159 views
The document discusses using Play Framework, Docker, CircleCI, and AWS together to create an automated microservices build pipeline. Key aspects include using GitHub for source control, CircleCI for continuous integration to build Docker images, pushing images to Docker Hub, and deploying to AWS using ECS for container orchestration. The author demonstrates setting up each part of the pipeline live.
Play Framework + Docker + CircleCI + AWS + EC2 Container Service
- 1. Play Framework + Docker + CircleCI + AWS = An Automated Microservice Build Pipeline Josh Padnick Wednesday, November 11, 2015 [email protected] @OhMyGoshJosh
- 2. What do I want out of a Java-based microservices infrastructure?
- 3. Java-Based • Java-based (or modern hipster JVM language) • No Java EE • Reload without compile (e.g. refresh the browser) • Native support for JSON, REST, and Websockets • Supports “reactive” mindset (async, non-blocking, etc.)
- 4. Microservices Infrastructure • A Universal unit of deployment (i.e. Docker) • Continuous integration • Continuous deployment • Ability to run multiple containerized services on the same VM • Simple setup • Long-term scalability • Minimal “undifferentiated heavy lifting”
- 5. Does this mythical beast exist?
- 6. Actually, it’s increasingly less mythical.
- 7. Padnick Josh • Full-stack engineer for 12+ years • Professional AWS & DevOps Guy via Phoenix DevOps • Experienced Java programmer Lover of Scala Favorite Web Framework is Play Framework • [email protected] @OhMyGoshJosh
- 8. DevOps & AWS • I wrote a 12,000+ word article on building scalable web apps on AWS at https://goo.gl/ aD6gNC • See JoshPadnick.com for prior DevOps & AWS presentations. • Interested in getting in touch? Contact me via PhoenixDevOps.com.
- 9. Today’s talk is about putting together a quick but scalable solution for this problem.
- 10. First we’ll cover the big picture concepts. Then we’ll show it working. We’ll end by talking about how it could be even better.
- 11. Let’s start with the world’s most generic build pipeline
- 12. VCS Developer commits code to Version Control System.
- 13. VCS VCS notifies Build Server we have a new build. Build Server
- 14. VCS Build Server - build/compile - run the “fast” automated tests - prepare a deployment artifact Build Server
- 15. VCS Build Server Build server pushes deployment artifact to artifact repository. Artifact Repository
- 16. VCS Build Server We’d like to do Continuous Deployment. So let’s assume this was a deployable commit. We immediately deploy the artifact. Artifact Repository
- 17. VCS Build Server Artifact Repository Deploy to infrastructure.
- 18. Now let’s pick our technologies.
- 19. Docker Hub
- 20. Developer commits code to GitHub.
- 21. Options • GitHub De facto source control system. • BitBucket Hosted but more enterprisey. Theoretical tighter integration with other Atlasssian tools. • AWS CodeCommit No fancy UI but fully hosted git repo in AWS.
- 23. GitHub uses web hooks to automatically kick off a build in CircleCI.
- 24. Options • CircleCI Hosted build tool. Awesome UI. Get up and running in an hour or less. But no first-class support for Docker. • Travis Hosted build tool. Built on Jenkins behind the scenes. Comparable to Circle. More expensive. • Shippable First-class Docker support, but clunky UI. Fast and customizable. Use your own Docker container for your build environment! • Jenkins The self-hosted stalwart. Medium overhead in exchange for maximum customizability.
- 26. Docker Hub Circle will: - build/compile - run automated tests - build a docker image - push image to Docker Hub
- 27. Options • Docker Hub The “official” place to house Docker registries. Free for public repos; paid for private. Poor UI, sometimes goes down. Easiest integration with rest of Docker ecosystem, but easy to switch to another repo. • Amazon EC2 Container Registry (ECR) AWS’s private container registry service. Looks like a winner. Coming out by end of year. Unless Amazon really screws up, obvious alternative to Docker Hub. • Google Cloud Registry (GCR) Mature, solid solution. Lowest pull latencies with Google Cloud Engine, but usable anywhere. • Quay Early docker registry upstart with superior UX. Acquired by CoreOS. Solid solution, but probably not as compelling as AWS ECR.
- 28. Docker Hub
- 29. Docker Hub Deploy to AWS.
- 30. Options • Let’s just assume all AWS for now.
- 31. Options within AWS • AWS EC2 Container Service (ECS) Amazon’s solution for running multiple services on a single VM in docker. Not perfect, but does an excellent job of being easy to setup and start using right away. • AWS Elastic Beanstalk AWS’s equivalent of Platform-as-a-Service. Works great when using one Docker container per VM, and meant to be scalable, but eventually you’ll want more control over your infrastructure. • Roll Your Own Use a custom method to get containers deployed on your VMs. • Container Framework Use a framework like CoreOS+Fleet, Swarm, Mesos, Kubernetes or Nomad. • Container Framework PaaS Use a pre-baked solution like Deis or Flynn. Or a tool like Empire that sits on top of ECS.
- 32. What about our app code?
- 33. Let’s talk about it.
- 34. • Re-architected the web framework from scratch. • Nice dev workflow • Young enough to be hipster; mature enough to be stable • Solid IDE support (IntelliJ) • Non-blocking / async • Outstanding performance • Designed for RESTful APIs
- 35. Live Demo
- 36. Now let’s build up our build pipeline live.
- 37. Step #1: Create a base docker image
- 38. • We may have many different microservices using Docker. • A common base image = standardization • See my base docker image at: https://github.com/PhoenixDevOps/phxjug-ctr-base
- 39. • # BUILD THE BASE CONTAINER cd /repos/phxdevops/phxjug-ctr-base docker build -t "phxdevops/phxjug-ctr-base:3.2" . docker push “phxdevops/phxjug-ctr-base:3.2" • NOTE: You won’t have rights to push to my repo. So replace this with your own Docker Hub repo.
- 40. Step #2: Create a base Play Framework docker image
- 41. • We may have many different microservices using Play. • Also, one of Play’s downsides is that Activator (which is really just a wrapper around SBT) uses Ivy for dependencies, and it is painfully slow on initial downloads. • If we create a Docker image with all our dependencies pre- downloaded, our docker build times will be MUCH faster. • Even if some of our dependencies are off, it’s not a big deal. The point is that we’ll get most of them here. • See my base docker image at: https://github.com/PhoenixDevOps/phxjug-ctr-base-play
- 42. • # BUILD THE BASE PLAY CONTAINER cd /repos/phxdevops/phxjug-ctr-base-play docker build -t "phxdevops/phxjug-ctr-base-play:2.4.3" . docker push "phxdevops/phxjug-ctr-base-play:2.4.3" • NOTE: You won’t have rights to push to my repo. So replace this with your own Docker Hub repo.
- 43. Step #3: Take our Play app and build a Docker image out of it.
- 44. • SBT includes a “dist” plugin that will create an executable binary for our entire Play app! • We’ll run that and make that the process around which the Docker container executes. • See my image at: https://github.com/PhoenixDevOps/phxjug-play- framework-demo • Note that this is a standard Play app with a Dockerfile in the root directory. “docker build” takes care of the rest.
- 45. • # BUILD A PLAY APP IN A CONTAINER cd /repos/phxdevops/phxjug-play-framework-demo docker build -t "phxdevops/phxjug-play-framework-demo:demo" docker push "phxdevops/phxjug-play-framework-demo:demo" • NOTE: You won’t have rights to push to my repo. So replace this with your own Docker Hub repo.
- 46. Step #4: Define our ECS Infrastructure in AWS.
- 47. Options • Point and click around the AWS Web Console Good for learning. Bad for long-term maintainability • AWS CloudFormation AWS’s official “infrastructure as code” tool. Pretty stable and mature, but painfully slow to work with, and JSON format gets too verbose. • Terraform A brilliant achievement of infrastructure as code tooling! But still suffers from some bugs. You can work around them once you get the hang of it, or with guidance from experienced hands. • Ansible Offers similar tool, but doesn’t compare in sophistication to CloudFormation or Terraform.
- 48. Our Choice • We’ll use terraform. • To save time, I’ve already provisioned the infrastructure for today. • But you can see the entire set of Terraform templates I used to create my ECS cluster at https://github.com/PhoenixDevOps/phxjug-ecs- cluster
- 49. Wait, how does ECS work?
- 50. Key ECS Concepts • Cluster • Container Instances
- 51. Cluster
- 53. Key Concepts • Task Definitions
- 54. Task Definitions • JSON object • Describes how 1 or more containers should be run and possibly links Container A to Container B. • You can also use Docker Compose yml files as an alternative to the proprietary ECS JSON format.
- 55. Components of a Task Definition • Task Family Name (e.g. “MyApp”) • 1 or more container definitions: • docker run command + args • Resource requirements (CPU, Memory)
- 56. Deploying new versions of your app • All your app’s versions are individual “Task Definitions” within a “Task Definition Family” • Each time you need to deploy a new version of your app, you’ll need a new Docker image with a new tag. Then just create a new Task Definition that points to the new Docker image. • ECS handles deployment for you, but there are some pitfalls here.
- 58. Key Concepts • Task Definitions • Task Definition Families • Tasks • Services
- 60. Tasks • An “instance” of a Task Definition is a Task. • Really, this just means a single Docker container (or a “group” of Docker containers if the Task Definition specified more than one Docker image).
- 61. Tasks as Services • Should your task always remain running? • Should it be auto-restarted if it fails? • Might it need an ELB? • Then you want to run your Task Definition as a Service!
- 62. Tasks and Services • Note that the same Task Definition… • …can be used to run as a one-time Task • …or a long-running Service. • That’s because Task Definitions are really just definitions of Docker containers and how they should run. It doesn’t “know” anything else about the container itself.
- 64. ECS Pro’s • Very little to manage. • Built-in service discovery, cluster state management, and container scheduler. • Allows for resource-aware container placement. • Container scheduling is pluggable. • Fully baked GUI that allows you to learn/do most anything. • Tolerable learning curve. • Supported by Amazon. • Feel free to build your own service discovery!
- 65. ECS Con’s • Default Service Discovery: One ELB per service = $18/service per month —> potentially expensive • Less flexible on deployments than you’d like. • Lacks the power of a more general purpose “data center operating system” such as Mesos or Kubernetes.
- 67. Step #5: Configure our Play app for circle.yml
- 69. Now let’s see it all live in action!
- 70. Q&A