Meetup #3: Migrate a fast scale system to AWS
Download as PPTX, PDF0 likes245 views
The document discusses migrating a fast scaling system from physical servers to AWS. It outlines the agenda which includes an overview of physical systems and virtualization, the Topica Edumall system design, applying DevOps practices, and a Q&A. The Topica Edumall hybrid system is described as using VPS, physical servers, and AWS for core video streaming. Benefits of migrating the auto-scaling system to Docker and Kubernetes on AWS include high availability, easy scaling, auto-healing, and saving 40% costs with full utilization of AWS services. DevOps practices like CI/CD workflows are also discussed.
Ad
More Related Content
What's hot (18)
Similar to Meetup #3: Migrate a fast scale system to AWS (20)
Ad
More from AWS Vietnam Community (18)
Ad
Recently uploaded (20)
Meetup #3: Migrate a fast scale system to AWS
- 1. @Wayarmy – [email protected] Topica Edumall SysAdmin Migrate a fast scale system to AWS Sep 22 2016
- 2. About me • @wayarmy • Interested: Cloud Computing, AWS, Docker, IaaS, SaaS, PaaS, Apache Stack, Elastic Stack, Hashi Stack, Ruby on Rails, Linux, Unix… • Github: http://github.com/wayarmy • Sysadmin, SysOps, DevOps, HumanOps
- 3. 1. Physical System – Virtualization – Cloud Computing 2. Topica Edumall System Design 3. Apply with DevOps 4. Q&A Meetup Agenda
- 5. Advantage • Easy to control local system • Not depend on hosting provider • Do whatever if you want • … Disadvantage • So hard to scale • Security • Depend on network, energy power • Expensive • …
- 6. • Ops aren’t immutable, they’re not available 99.99% nor 24/7 they’re not machines. • The goal of HumanOps is to improve and maintain the good health of your team: easing communication reducing fatigue and reducing stress » David Mytton – CEO Server Density
- 7. Cloud Computing • The simple definition: It’s a style of computing based on shared, elastic resources delivered to users in a self-service, metered manner using web technologies. Yet, if you ask five people “what is cloud computing?” you can expect five different answers. Why? Because what matters to them is not what cloud computing is, but what it does for them. » Oracle Cloud.
- 8. Virtualization And Cloud Computing • Easy to Scale • Don’t care about insfrastructure design • Don’t care about network design • Don’t depend on everything, depend on only “your wallet” • Cloud computing is the easiest way to deliver your services over the Internet.
- 9. 1. Physical System – Virtualization – Cloud Computing 2. Topica Edumall System Design 3. Apply with DevOps 4. Q&A
- 10. Topica Edumall hybrid system design • 3 important element of system design – Security – High Availability – Scalable • => need a system with HA, Scalable, Security
- 12. Element of Edumall Insfrastructure • VPS < DigitalOcean > (~40%) • Physical Server and Network (Viettel, VDC, FPT) • Cloud Computing ( AWS ) ( ~60%) • Core system: Video Streaming
- 13. Edumall – Microservices on AWS • IaaS – VPC – Network – Firewall – Compute – EC2 – Storage: S3, Glacier ( EFS - Future) • SaaS – DBMS: DynamoDB – Caching – CDN – WAF – Datanalyst Service (Near Future) – Message Bus Application – API Gateway – ETS – WorkSpaces • ServerLess Architecture (Near Future)
- 26. Benefit • High availability system • Easy with scale and everything will be automated • Auto healing • Save much money ( save about 40% with 100% running application on AWS )
- 27. Edumall auto-scale system with Docker and Kubernestes on AWS
- 28. Docker • Docker containers wrap a piece of software in a complete filesystem that contains everything needed to run: code, runtime, system tools, system libraries – anything that can be installed on a server. This guarantees that the software will always run the same, regardless of its environment. • Build – Ship – Run • Build once, run any where
- 29. Kubernetes • Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. • “Write once, run forever” • Build Auto-scale and self-healing system
- 30. Why? • Easy to use • Fast to scale • Auto scale infrastructure and auto scale apps • Auto create ELB • Auto healing, warm-up
- 32. Some Number? (6 months ago) • Video content transfer: 90GB (/1 day) • Static Content (image, js, html, css): 1GB • Total Request: ~ 32k • Total Ram: ~ 128GB • Total CPU: ~32vCpus • Unique Visitor: ~4k6 • Total User: ~500k • Maximum Unique Visitors: ~300
- 33. Some number? (present) • Video content transfer: 1,5 TB (/1 day) • Static Content (image, js, html, css): 100GB • Total Request: ~ 5 mil • Total Ram: ~ 2TB • Total CPU: ~400vCpus • Unique Visitor: ~50k • Total User: ~20m • Maximum Unique Visitors: ~5k • EC2 Start: ~ 5 (M4 2xLarge)
- 34. 1. Physical System – Virtualization – Cloud Computing 2. Topica Edumall System Design 3. Apply with DevOps 4. Q&A
- 35. With Dev • Easy to use with SaaS • Easy with authentication • Easy with test environment • Easy with code, change writing apps to reading docs of AWS SaaS, and apply AWS SaaS to Our Microservice System • Document as Code • Save 40% code-time with SaaS
- 36. With Ops • Don’t need think about the insfrastructure, about physical servers, ISP, Network, … • Easy with Deployment • Don’t need a monitoring system, Cloudwatch will take care all of them. • Less time for build, deploy a new insfrastructure, platform
- 37. CI-CD workflow Developer push a new commit to gitlab Gitlab Trigger Gitlab Runner fetch and pull repo Gitlab-runner build Docker image from Dockerfile Excute test on all target With all environment Delivery images to Docker hub Or Private registry Deploy to K8s Cluster
- 38. Disadvantages - While moving system on AWS, need to keep alive two system ( Old and New ). So that, need much money. - So hard to motivate all team to use AWS. Someone can’t change their mind. - High latency from Vietnam to AWS when AAG is on maintainance.
- 39. Future • Data analyst Tool on AWS • Serverless Architecture • IOT • Mobile Hub
- 40. Goals • System Design with High Availability, Security and Scalabililty • Easy Operations -> HumanOps • Save much money, and more benefit .
- 41. Q&A
- 42. Meetup: www.meetup.com/aws-vn FB: www.facebook.com/groups/amazonwebservicevietnam Join Slack: https://aws-vn.herokuapp.com/