Improve aws withproxysql
Download as PPTX, PDF1 like457 views
Marco Tusa gave a presentation on using ProxySQL to improve performance with Amazon Aurora databases. He explained what ProxySQL and AWS Aurora are and how ProxySQL can be configured to work with Aurora to distribute read queries across replicas for load balancing. Metrics were shown that read latency is lower with ProxySQL compared to directly connecting to Aurora. The native Aurora connection endpoints are limited while ProxySQL allows more granular control over query routing and additional features like caching.
Improve aws withproxysql
- 1. Budapest meetup | May 9th, 2018 Improve AWS/Aurora with ProxySQL
- 2. 2 Marco “The Grinch” • Open source enthusiast • In love with ProxySQL • Consulting team leader • Percona Consultant About me
- 3. 3 Hello, Attendees! Agenda • What… • How… • Why… • Why … explain
- 4. 4 What is ProxySQL (in 1 slide) • ProxySQL has an advanced multi-core architecture. • It's built from the ground up to support hundreds of thousands of concurrent connections, multiplexed to potentially hundreds of backend servers. • Query filtering by design • Query caching • Embedded configuration distribution (cluster) • Design to scale (the largest ProxySQL deployment spans several hundred proxies). • … and more
- 5. 5 What is AWS Aurora (in 1 slide) • Amazon Aurora is a MySQL and PostgreSQL compatible relational database built for the cloud • Features a distributed, fault-tolerant, self-healing storage system that auto-scales up to 64TB per database instance • Delivers high performance and availability with up to 15 low-latency read replicas, point-in-time recovery, continuous backup to Amazon S3, and replication across three Availability Zones • fully managed by Amazon Relational Database Service (RDS)
- 6. 6 The problem ProxySQL deal with backend servers using: • Replication Hostgroup • Async replication • Scheduler • PXC, NDB etc AWS Aurora do not use READ_ONLY but INNODB_READ_ONLY https://dev.mysql.com/doc/refman/5.7/en/innodb-read-only-instance.html
- 7. 7 Solution October 2017, this issue was opened (https://github.com/sysown/proxysql/issues/1195 ) MYHGM_MYSQL_REPLICATION_HOSTGROUPS "CREATE TABLE mysql_replication_hostgroups (writer_hostgroup INT CHECK (writer_hostgroup>=0) NOT NULL PRIMARY KEY , reader_hostgroup INT NOT NULL CHECK (reader_hostgroup<>writer_hostgroup AND reader_hostgroup>=0) , check_type VARCHAR CHECK (LOWER(check_type) IN ('read_only','innodb_read_only','super_read_only')) NOT NULL DEFAULT 'read_only' , comment VARCHAR NOT NULL DEFAULT '' , UNIQUE (reader_hostgroup))” mysql> select * from mysql_replication_hostgroups; +------------------+------------------+------------------+------------+ | writer_hostgroup | reader_hostgroup | check_type | comment | +------------------+------------------+------------------+------------+ | 70 | 71 | innodb_read_only | aws-aurora | +------------------+------------------+------------------+------------+ 1 row in set (0.00 sec)
- 8. 8 How to implement First rollout your Aurora setup • Identify the Endpoint for EACH instance • aws rds describe-db-instances • Web interface INSERT INTO mysql_servers (hostname,hostgroup_id,port,weight,max_connections) VALUES ('proxysqltestdb.eu-central-1',70,3306,1000,2000); VALUES ('proxysqltestdb.eu-central-1',71,3306,1000,2000); VALUES ('proxysqltestdb2.eu-central-1',71,3306,1000,2000); VALUES ('proxysqltestdb-eu-central-1b.eu-central.1',71,3306,1,2000); INSERT INTO mysql_replication_hostgroups(writer_hostgroup,reader_hostgroup,comment,check_type) VALUES (70,71,'aws-aurora’, 'innodb_read_only’); LOAD MYSQL SERVERS TO RUNTIME; SAVE MYSQL SERVERS TO DISK;
- 11. 11 Why Latency while running the tests Left Aurora Right ProxySQL
- 12. 12 Why Only connection latency When using Java connector Single pointer Values in nanoseconds
- 13. 13 Why it happens The Cluster endpoint is an endpoint for an Aurora DB cluster that connects to the current primary instance for that DB cluster. Each Aurora DB cluster has a cluster endpoint and one primary instance. That endpoint receives the read and write request and sends them to the same instance. The main use for it is to perform failover if needed. Each Aurora DB cluster has a reader endpoint. If there is more than one Aurora Replica, the reader endpoint directs each connection request to one of the Aurora Replicas. The reader endpoint only load balances connections to available Aurora Replicas in an Aurora DB cluster. It does not load balance specific queries. If you want to load balance queries to distribute the read workload for a DB cluster, you need to manage that in your application and use instance endpoints to connect directly to Aurora Replicas to balance the load.
- 14. 14 Why it happens ProxySQL can redirect the queries as you like and to the instance you want. How do we read this graph? From left to right: • read_only test with an Aurora cluster endpoint • read_only test with ProxySQL • write_only with an Aurora cluster endpoint • write_only with ProxySQL • read and write with an Aurora cluster endpoint • read and write with ProxySQL
- 15. 15 Conclusions • Native AWS Cluster endpoints and Reader endpoints are limited in what they offer • With ProxySQL you can very granularly choose how to use each instance, without the need to have the application modify how it works • Using ProxySQL will allow the use of additional elements like • Query Cache • Query rewrite • Blocking/firewalling
- 16. 16
- 17. 17 Contacts To contact Me [email protected] [email protected] To follow me http://www.tusacentral.net/ http://www.percona.com/blog/ https://www.facebook.com/marco.tusa.94 @marcotusa http://it.linkedin.com/in/marcotusa/ “Consulting = No mission refused!”