MySQL in the Cloud
10 likes4,820 views
The document provides an overview of MySQL as a Database as a Service (DBaaS), detailing its deployment, management, and various service offerings. It discusses cost considerations, versions and access methods, as well as features and limitations across different cloud providers such as AWS, Google Cloud, and Rackspace. The document emphasizes the importance of choosing the right service for specific needs, including scalability, availability, and operational ease.
MySQL in the Cloud
- 1. MySQL in the Cloud Colin Charles, MariaDB/SkySQL Ab [email protected] | [email protected] http://mariadb.com/ | http://mariadb.org/ http://bytebot.net/blog/ | @bytebot on Twitter LinuxCon/CloudOpen, New Orleans, Louisiana, USA 17 September 2013 1
- 2. whoami • Chief Evangelist, MariaDB at Monty Program SkySQL • Formerly MySQL AB/Sun Microsystems • Using/developing/hacking on MySQL since 2000 • Previously on FESCO for The Fedora Project, and worked on OpenOffice.org (2000-2005) 2
- 3. Agenda • MySQL as a service offering (DBaaS) • Choices • Considerations • MySQL versions & access • Costs • Deeper into RDS • Should you run this on EC2 or an equivalent? • Conclusion 3
- 4. MySQL as a service • Database as a Service (DBaaS) • MySQL available on-demand, without any installation/configuration of hardware/ software • Pay-per-usage based • Provider maintains MySQL, you don’t maintain, upgrade, or administer the database 4
- 5. New way of deployment • Enter a credit card number • call API (or use the GUI) ec2-run-instances ami- xxx -k ${EC2_KEYPAIR} -t m1.large nova boot --image centos6-x86_64 --flavor m1.large db1 5 credit: http://www.flickr.com/photos/68751915@N05/6280507539/
- 6. Why DBaaS? • “Couldn’t we just have a few more servers to handle the traffic spike during the elections?” • Don’t have a lot of DBAs, optimise for operational ease • Rapid deployment & scale-out 6
- 7. Your choices today • Amazon Web Services Relational Database Service (RDS) • Rackspace Cloud Databases • Google Cloud SQL • HP Cloud Relational Database 7
- 8. There are more • Jelastic - PaaS offering MySQL, MariaDB • ClearDB - MySQL partnered with heroku, appfog,Azure clouds • Joyent - Image offers Percona MySQL • Xeround - 2 weeks notice... 8
- 9. Whom we won’t be covering • GenieDB - globally distributed MySQL as a service, master-master replication, works on EC2, Rackspace, Google Compute Engine, HP Cloud • ScaleDB - promises write scaling, HA clustering, etc. replacing InnoDB/MyISAM 9
- 10. Regions & Availability Zones • Region: a data centre location, containing multiple Availability Zones • Availability Zone (AZ): isolated from failures from other AZs + low- latency network connectivity to other zones in same region 10
- 11. Location, location, location • AWS RDS: US East (N.Virginia), US West (Oregon), US West (California), EU (Ireland), APAC (Singapore),APAC (Tokyo),APAC (Sydney), South America (São Paulo), GovCloud • Rackspace: USA (Dallas DFW, Chicago ORD, N.Virginia IAD),APAC (Sydney), EU (London)* • Google Cloud SQL: US, EU • HP Cloud: US-East (Virginia), US-West 11
- 12. Service Level Agreements (SLA) • AWS - 99.95% in a calendar month • Rackspace - 99.9% in a calendar month • Google - 99.95% in a calendar month • HP Cloud - no SLA yet, services in beta • SLAs exclude “scheduled maintenance” • AWS is 30 minutes/week, so really 99.65% 12
- 13. Support • AWS - forums; $49/mo gets email; $100+ phone # • Rackspace - live chat, phone #, forums • Google - forums; $150/mo gets support portal; $400+ for phone # • HP Cloud - phone #, chat, customer forum 13
- 14. Who manages this? • AWS: self-management, Enterprise ($15k+) • Rackspace: $100 + 0.04 cents/hr over regular pricing • Google: self-management • HP Cloud: self-management 14
- 15. MySQL versions • AWS: MySQL Community 5.1.71, 5.5.33, 5.6.13 • Rackspace: MySQL Community 5.1 • Google: MySQL Community 5.5 • HP Cloud: Percona Server 5.5.28 15
- 16. Access methods • AWS - within Amazon, externally via mysql client,API access. • Rackspace - private hostname within Rackspace network, standard mysql client,API access. • Google - within AppEngine, or a command line Java tool (gcutil) • HP Cloud - within HP Cloud, externally via client (trove-cli, reddwarf) or API access. 16
- 17. Can you configure MySQL? • You don’t access my.cnf naturally • In AWS you have parameter groups which allow configuration of MySQL 17 source: http://www.mysqlperformanceblog.com/2013/08/21/amazon-rds-with-mysql-5-6-configuration-variables/
- 18. Cost • Subscribe to relevant newsletters of your services • Cost changes rapidly, plus you get new instance types and new features (IOPS) • Don’t forget network access costs • Monitor your costs daily, hourly if possible (EC2 instances can have spot pricing) • https://github.com/ronaldbradford/aws 18
- 19. Costs:AWS • AWS prices vary between regions • http://aws.amazon.com/rds/pricing/ 19
- 20. Costs:AWS II • Medium instances (3.75GB) useful for testing ($2,411/yr) • Large instance (7.5GB) production ready ($4,777/ yr) • XL instance (15GB, 8ECUs) ($9,555/yr) • m2.2XL instance (34GB, 13ECUs, 500Mbps PIOPS) ($16,568/yr) • m2.4XL instance (68GB, 26ECUs, 1000Mbps PIOPS) ($33,048/yr) 20
- 21. Costs: Rackspace • Option to have regular Cloud Database or Managed Instances • 4GB instance (testing) is $3,504/yr • 8GB instance (production) is $6,658/yr • Consider looking at I/O priority, and the actual TPS you get 21
- 22. Costs: Google • You must enable billing before you create Cloud SQL instances • https://developers.google.com/cloud-sql/docs/billing • Testing (D8 - 4GB RAM) - ($4,274.15) • XL equivalent for production (D16 - 8GB RAM) - ($8,548.30) • Packages billing plans are cheaper than per-use billing plans 22
- 23. Costs: HP Cloud • Its currently in beta • Its free... 23
- 24. Where do you host your application? • Typically within the compute clusters of the service you’re running the DBaaS in • This also means your language choices are limited based on what the platform offers (eg.AppEngine only recently started offering PHP) 24
- 25. RDS: Multi-AZ • Provides enhanced durability (synchronous data replication) • Increased availability (automatic failover) • Warning: can be slow (1-10 mins+) • Easy GUI administration • Doesn’t give you another usable “read- replica” though 25
- 26. External replication • MySQL 5.6 you can do RDS -> Non-RDS • enable backup retention, you now have binlog access • You still can’t replicate INTO RDS • use Tungsten Replicator • also supports going from RDS to Rackspace/etc. 26
- 27. MySQL 5.6, MariaDB 10 • MySQL 5.6 in RDS provides crash-safe slaves, the InnoDB memcached interface, online schema changes, full-text InnoDB indexes, optimizer improvements, INFORMATION_SCHEMA enhancements, scalability/replication improvements, PERFORMANCE_SCHEMA enhancements • MariaDB 10 has that, plus multi-source replication, GTIDs that don’t require full restarts, threadpool, audit plugin and more 27
- 28. Getting started • Importing data into the cloud? • RDS: mysqldump is a good choice • Google Cloud SQL is only via existing Google Cloud Storage • Upgrading from RDS 5.5 to RDS 5.6? • mysqldump! 28
- 29. Handling backups • You don’t get to use xtrabackup! • Google Cloud SQL automates backups (has a backup window) • Amazon has automated backups (with point-in-time recovery), with full daily snapshots (has a backup window). 29
- 30. Monitoring • Options are limited,AWS has the best options currently available • Today you have CloudWatch • Google has basic read/write graphs 30
- 31. Storage Engines • MySQL (/MariaDB) has many • cool ones include TokuDB, SPIDER, CONNECT, CassandraSE • You basically use InnoDB and MyISAM with cloud solutions • MyISAM on RDS won’t guarantee point- in-time recovery, snapshot restore 31
- 32. High Availability • Plan for node failures • Don’t assume node provisioning is quick • Backup, backup, backup! • “Bad” nodes exist • HA is not equal across options - RDS wins so far 32
- 33. Unsupported features • AWS: GTIDs, InnoDB Cache Warming, InnoDB transportable tablespaces, authentication plugins, semi-sync replication • Google: UDFs, replication, LOAD DATA INFILE, INSTALL PLUGIN, SELECT ... INTO OUTFILE 33
- 34. Provisioned IOPS • Only available on Amazon • Faster, predictable, consistent I/O performance with low latencies • Good throughput, RAID on backed • EBS is more reliable 34
- 35. More on RDS • log access via API • no SUPER access to skip replication errors easily • sync_binlog=0 not available • no OS access (sar, ps, swap?, tcpdump) • https://github.com/boto/boto 35
- 36. Warning: automatic upgrades • Regressions happen even with a minor version upgrade in the MySQL world • InnoDB update that modifies rows PK triggers recursive behaviour until all disk space is exceeded? 5.5.24->5.5.25 (fixed: 5.5.25a) • Using query cache for partitioned tables? Disabled since 5.5.22->5.5.23! 36
- 37. Benchmarking for use • sysbench • OLTP test, use tables with 20M rows and 20M transactions, check 1-128 threads/run (run this on RDS, Rackspace) • June 2013, tps, performance per dollar, Rackspace delivers more performance across all flavours except 512MB instance • Yahoo! Cloud Serving Benchmark • https://github.com/brianfrankcooper/YCSB 37
- 38. Roadmaps? • There don’t seem to be public roadmaps. You find out when there’s a change! • Expect more to move to MySQL 5.6 • except Google, who will probably move to MariaDB 10.0 • maybe Rackspace is up in the air? 38
- 39. Running MySQL in EC2 • Can do multiple geographic regions via replication • Run just one Percona Server/ MariaDB server/instance • Use additional EBS volumes for InnoDB tablespaces • RAID EBS volumes (RAID1) • Warm up data partitions, mount partitions with noatime, nodirtime • Vertical scaling with SSD-backed storage • Monitoring with nagios • Snapshot backups and save to S3 • Can use Elastic Load Balancer • Can use spot instances • Can use tools like MHA to provide automatic failover • Can use MariaDB Galera Cluster/ Percona XtraDB Cluster 39
- 40. Some closing thoughts • Hardware varies per region • Sometimes, software manageability varies per region • Beware cost on your credit card! 40
- 41. Q&A.ThankYou. [email protected] | [email protected] http://bytebot.net/blog/ | @bytebot on Twitter Download MariaDB and give it a try: http://mariadb.org/ 41