Zero To Cloud (OSCon 2014)
2 likes1,474 views
This tutorial provides a detailed hands-on experience to bring up the necessary components to run the @NetflixOSS stack. This includes priming your Amazon account (IAM Profiles, Security Groups, etc) and setting up Asgard and Aminator. Together they can be used, time permitting, to launch many more @NetflixOSS services, like Edda, Eureka and Ice.
Ad
More Related Content
What's hot (20)
Viewers also liked (10)
Ad
Similar to Zero To Cloud (OSCon 2014) (20)
Ad
Recently uploaded (20)
Zero To Cloud (OSCon 2014)
- 1. Zero to Cloud with @NetflixOSS Justin Ryan (@quidryan)
- 2. Who We Are • Justin Ryan • Michael Tripoli • Various other Netflix Employees in the Audience #netflixoss
- 3. #netflixoss Netflix is the world's leading Internet television network with over 48 million members in more than 40 countries enjoying more than one billion hours of TV shows and movies per month, including original series. For a low monthly price, Netflix members can watch as much as they want, anytime, anywhere, on nearly any Internet-connected screen. Members can play, pause and resume watching, all without commercials or commitments. (Credit: ir.netflix.com)
- 4. Tutorial Format • 30 Minutes of Lecture • 30 Minutes on Console • 30 Minutes on Jumphost • 30 Minutes deploying Asgard • 30 Minutes Edda and Eureka • 30 Minutes Karyon #netflixoss
- 5. Tutorial Purpose • Help adoption of @NetflixOSS components • Answer questions along the way • Less trail blazing for us #netflixoss
- 6. Assumptions • Stay Local (using us-west-2) • Non-destructive operations • Follow instructions closely • Security through obscurity • Not a “developing for the cloud” tutorial • This will cost money #netflixoss
- 8. #netflixoss Step 1 - Sign Up for AWS
- 10. #netflixoss Step 2 - AWS Console
- 11. #netflixoss Step 3 - Key Pair
- 12. #netflixoss Step 4 - Create Jumphost
- 13. #netflixoss Step 5 - Create Role
- 14. #netflixoss Step 6 - Create User
- 15. Cloud Deployment Model #netflixoss Auto Scaling Group Elastic Load Balancer Launch Configuration Security Group Amazon Machine Image Configuration Runtime InstanceInstanceInstances
- 16. #netflixoss Step 7 - Security Group
- 17. Immutable Infrastructure • Simplifies Operations • Baking with Aminator #netflixoss Foundation Base AMI Application
- 18. #netflixoss Step 8 - Foundation AMI
- 19. #netflixoss Step 9 - Setup Jumphost
- 20. #netflixoss Step 10 - Credentials
- 21. Building • Bake a system package (RPM, DEB) • Use ospackage plugin to generate #netflixoss Foundation Base AMI Application WAR DEB Java Python
- 22. #netflixoss Step 11 - Base AMI
- 25. Who • Platform Infrastructure • Cloud Database Engineering • Cloud Solutions Engineering Tools • Cloud and Platform Engineering • (Anyone with code that’s not a competitive advantage) #netflixoss
- 26. What • Infrastructure • Caching • Database • Cloud • Building • Deployment ! #netflixoss • Testing • Monitoring • Networking • Robustness • Security • Configuration
- 27. Not • Streaming • Encoding • Merchandising • Movie metadata • Recommendations • 10’ UI #netflixoss
- 28. Open source infrastructure • Github • Cloudbees for Builds • http://techblog.netflix.com/ • http://twitter.com/netflixoss • Apache 2.0 license (when possible) • Meetups - Season 2 going on right now! #netflixoss
- 29. #netflixoss Step 12 - Bake Asgard
- 30. Islands of Applications #netflixoss Search Auto-complete Ratings Signup API Streaming Starts
- 31. Islands of Applications #netflixoss Search Auto-complete Ratings Signup API Streaming Starts Application Application Application Application Application Application
- 32. Inventing the Application • Problem - Application is not an Amazon concept • Solution - Create an Application domain in SimpleDB - Enforce naming conventions on Amazon objects #netflixoss
- 35. Inventing the Cluster • Problem - Two ASGs on purpose - Different names • Solution - Maintain same prefix - Append version - Derive Cluster from ASG names #netflixoss
- 36. Asgard vs Console • Asgard - Provide Application and Cluster - Simplifies object construction - Introduce workflow - Tightly couple with internal systems • Console - Access to all objects #netflixoss
- 37. #netflixoss Step 13 - Asgard/Asgard
- 38. #netflixoss Step 14 - Deploy Edda
- 39. Mid-tier Calling #netflixoss Eureka Data Plane Control Plane InstanceInstanceApp A InstanceInstanceApp B
- 40. #netflixoss Step 14 - Deploy Eureka
Editor's Notes
- #3: Mike and myself work in Engineering Tools. Which is a team that focuses on Build and Deploy tooling for the company.
- #5: Step by step guide, with informative interludes during delays Please do try to follow along, as nice as my voice is, you’ll get a lot more out of it.
- #6: Goal - Bring up the necessary components to run the @NetflixOSS stack. Lot of moving pieces in AWS, so we’re showing some defaults that work Be a smaller fish. As people use our techniques more, we’re blazing less trail.
- #7: Working in US West (Oregon) aka us-west-2. We’re performing non destructive operations, so if you have an existing AWS account setup, that will be fine and they won’t conflict. But it might be easier to find instances, etc if using a new region. In the case of existing infrastructure, like keys, please follow the instructions closely and do not re-use existing provisioned items. I wouldn’t want to be responsible for opening up a security hole in existing infrastructure. There are plenty of opportunities to lock down these applications at the network layer or the application layer. Or restrict what the instances can do. References will be made to additional security precautions, but they have not all been integrated into this tutorial. This is not a developing for the cloud tutorial, that makes for a great followup. Given enough time, we can talk about it. This will cost money. Not much, but some.
- #8: Using steps on http://nflx.it/zerotocloud Only URL you have to type, all others should be Copy & Paste from that document
- #16: ASG, LB, LC, AMI, SG, Instances, Oh My! API For a DC
- #18: Greatly simplifies operations, NoOps. A bulk of the size of the image is the Operating System. We layer our favorite tools in the Base AMI. And finally install the application.
- #22: Greatly simplifies operations, NoOps Baking Foundation Base AMI Aminator
- #25: Libraries: RxJava, RxNetty, Hystrix, Archaius, Governator, Astyanax, Curator In Big Data: Genie, Aegisthus
- #36: This also means that ASGs serve up a single version.
- #44: Feel free to ask questions over twitter. This version of the presentation will be up on SlideShare shortly.