MariaDB - a MySQL Replacement #SELF2014
9 likes1,477 views
MariaDB - a MySQL replacement at South East Linux Fest 2014 - SELF2014. Learn about features that are not in MySQL 5.6, some that are only just coming in MySQL 5.7, and some that just don't exist.
![New KILL syntax
• HARD | SOFT & USER USERNAME are MariaDB-specific (5.3.2)
• KILL QUERY ID query_id (10.0.5) - kill by query id, rather than thread id
• SOFT ensures things that may leave a table in an inconsistent state
aren’t interrupted (like REPAIR or INDEX creation for MyISAM or Aria)
KILL [HARD | SOFT] [CONNECTION | QUERY] [thread_id |
USER user_name]
MariaDB 5.3+](https://image.slidesharecdn.com/mariadbself-140620220732-phpapp02/85/MariaDB-a-MySQL-Replacement-SELF2014-34-320.jpg)
![Feedback plugin
• feedback=on in my.cnf [mysql]](https://image.slidesharecdn.com/mariadbself-140620220732-phpapp02/85/MariaDB-a-MySQL-Replacement-SELF2014-40-320.jpg)
![[B14] A MySQL Replacement by Colin Charles](https://cdn.slidesharecdn.com/ss_thumbnails/b14mariadbcolin-140630004007-phpapp01-thumbnail.jpg?width=560&fit=bounds)
![[db tech showcase Tokyo 2014] B15: Scalability with MariaDB and MaxScale by ...](https://cdn.slidesharecdn.com/ss_thumbnails/dbtstokyo2014b15mariadbcorporationcolincharlesscalabilitywithmariadbandmaxscale-141208020836-conversion-gate01-thumbnail.jpg?width=560&fit=bounds)
Ad
More Related Content
What's hot (20)
Similar to MariaDB - a MySQL Replacement #SELF2014 (20)
Ad
More from Colin Charles (15)
Ad
Recently uploaded (19)
MariaDB - a MySQL Replacement #SELF2014
- 1. MariaDB - a MySQL replacement Colin Charles, Team MariaDB, SkySQL Ab [email protected] | http://mariadb.org/ http://bytebot.net/blog/ | @bytebot on Twitter South East Linux Fest, Charlotte, NC, USA 20 June 2014
- 2. whoami • Work on MariaDB at SkySQL Ab • Merged with Monty Program Ab, makers of MariaDB • Formerly MySQL AB (exit: Sun Microsystems) • Past lives include Fedora Project (FESCO), OpenOffice.org
- 3. Who are you? • Developer? • Operator? (DBA, sysadmin) • A bit of both?
- 4. Global Top 20 sites 1.Google 2.Facebook 3.YouTube 4.Yahoo! 5.Baidu 6.Wikipedia 7.QQ 8.Taobao 9.Twitter 10.Live 11.LinkedIn 12.Sina 13.Amazon 14.hao123.com 15.google.co.in 16.blogspot 17.weibo.com 18.wordpress.com 19.360.cn 20.yandex.ru
- 5. 5W1H is MariaDB • Drop-in compatible MySQL replacement • Community developed, Foundation backed, feature enhanced, backwards compatible, GPLv2 licensed • Steady stream of releases in 4 years 4 months: 5.1, 5.2, 5.3, 5.5, 10.0, MariaDB Galera Cluster 5.5, MariaDB with TokuDB 5.5 • Enterprise features open: PAM authentication plugin, threadpool, audit plugin • Default in Red Hat Enterprise Linux, Fedora, openSUSE, etc.
- 7. Microseconds • TIME, DATETIME, TIMESTAMP, temporal functions, CAST, dynamic columns CREATE TABLE microsec( column_microsec DATETIME(6), column_millisec TIME(3) ); SELECT CURTIME(6); MariaDB 5.3+
- 8. Microseconds & 5.6 • TIME_TO_SEC(), UNIX_TIMESTAMP() preserve microseconds of the argument MariaDB 10.0 MySQL 5.6 SELECT TIME_TO_SEC('10:10:10.12345'); +-------------------------------+ | TIME_TO_SEC('10:10:10.12345') | +-------------------------------+ | 36610.12345 | +-------------------------------+ 1 row in set (0.01 sec) SELECT TIME_TO_SEC('10:10:10.12345'); +-------------------------------+ | TIME_TO_SEC('10:10:10.12345') | +-------------------------------+ | 36610 | +-------------------------------+ 1 row in set (0.00 sec)
- 9. Virtual Columns • A column in a table that has its value automatically calculated either with a pre-calculated/deterministic expression or values of other fields in the table • VIRTUAL - computed on the fly when data is queried (like a VIEW) • PERSISTENT - computed when data is inserted and stored in a table MariaDB 5.2+
- 10. PCRE Regular Expressions • Powerful REGEXP/RLIKE operator • New operators: • REGEXP_REPLACE(sub,pattern,replace) • REGEXP_INSTR(sub,pattern) • REGEXP_SUBSTR(sub,pattern) • Works with multi-byte character sets that MariaDB supports, including East-Asian sets MariaDB 10.0+
- 11. GIS • MariaDB implements a subset of SQL with Geometry Types • No longer just minimum bounding rectangles (MBR) - shapes considered CREATE TABLE geom (g GEOMETRY NOT NULL, SPATIAL INDEX(g)) ENGINE=MyISAM; • ST_ prefix - as per OpenGIS requirements MariaDB 5.3+
- 12. Sample use cases • Import OpenStreetMap data into MariaDB: http://www.slideshare.net/ hholzgra/fosdem-2014mariadbgis • Use the OpenStreetMap dataset: https://mariadb.com/kb/en/ openstreetmap-dataset/ • Screencast: https://blog.mariadb.org/screencast-mariadb-gis-demo/ • node.js example use case for mapping GPX data: https:// blog.mariadb.org/node-js-mariadb-and-gis/ & jQuery usage: https:// blog.mariadb.org/jquery-and-gis-distance-in-mariadb/
- 13. Dynamic columns • Allows you to create virtual columns with dynamic content for each row in table. Store different attributes for each item (like a web store). • Basically a BLOB with handling functions: COLUMN_CREATE, COLUMN_ADD, COLUMN_GET, COLUMN_DELETE, COLUMN_EXISTS, COLUMN_LIST, COLUMN_CHECK, COLUMN_JSON • In MariaDB 10.0: name support (instead of referring to columns by numbers, name it), convert all dynamic column content to JSON array, interface with Cassandra INSERT INTO tbl SET dyncol_blob=COLUMN_CREATE("column_name", "value"); MariaDB 5.3+
- 14. Full-text search via SphinxSE mysql> INSTALL PLUGIN sphinx SONAME 'ha_sphinx.so'; Query OK, 0 rows affected (0.01 sec) MariaDB 5.2+
- 15. What is SphinxSE? • SphinxSE is just the storage engine that still depends on the Sphinx daemon • It doesn’t store any data itself • Its just a built-in client to allow MariaDB to talk to Sphinx searchd, run queries, obtain results • Indexing, searching is performed on Sphinx
- 16. Sphinx search table CREATE TABLE t1 ( id INTEGER UNSIGNED NOT NULL, weight INTEGER NOT NULL, query VARCHAR(3072) NOT NULL, group_id INTEGER, INDEX(query) ) ENGINE=SPHINX CONNECTION="sphinx://localhost:9312/test"; ! SELECT * FROM t1 WHERE query='test it;mode=any';
- 17. Sphinx search tables • 1st column: INTEGER UNSIGNED or BIGINT (document ID) • 2nd column: match weight • 3rd column: VARCHAR or TEXT (your query) • Query column needs indexing, no other column needs to be
- 18. Query Cassandra • Data is mapped: rowkey, static columns, dynamic columns • super columns aren’t supported • No 1-1 direct map for data types • Write to Cassandra from SQL (SELECT, INSERT, UPDATE, DELETE) MariaDB 10.0+
- 19. Cassandra II pk varchar(36) primary key, data1 varchar(60), data2 bigint ) engine=cassandra keyspace='ks1' column_family='cf1' • Table must have a primary key • name/type must match Cassandra’s rowkey • Columns map to Cassandra’s static columns • name must be same as in Cassandra, datatypes must match, can be subset of CF’s columns
- 20. Mapping • Datatype mapping - complete table at KB • Data mapping is safe - engine will refuse incorrect mappings • Command mapping: INSERT overwrites rows, UPDATE reads then writes, DELETE reads then writes
- 21. Typical use cases • Web page hits collection, streaming data • Sensor data • Reads served with a lookup • Want an auto-replicated, fault-tolerant table?
- 22. CONNECT • Target: ETL for BI or analytics • Import data from CSV, XML, ODBC, MS Access, etc. • WHERE conditions pushed to ODBC source • DROP TABLE just removes the stored definition, not data itself • “Virtual” tables cannot be indexed MariaDB 10.0+
- 23. SPIDER • Horizontal partitioning, built on top of PARTITIONs • Associates a partition with a remote server • Transparent to user, easy to expand • Has index condition pushdown support enabled MariaDB 10.0+
- 24. TokuDB • Opensource - separate MariaDB 5.5+TokuDB/integrated in 10.0.5 • Improved insert (10-20x faster) & query speed, compression (up to 90% space reduction), replication performance and online schema flexibility • Uses Fractal Tree Indexes instead of B-Tree • Tests & builds of TokuDB on multiple platforms
- 25. Engines, etc • Plan for backups - TokuDB can be cool for your uses as an example • Galera: study your workload patterns, your application, etc. • SPIDER (built-in sharding capabilities, partitioning & XA transaction capable with multiple backends including Oracle) • its not going to be straightforward to “just start” - need to know right tables to implement, etc.
- 26. Threadpool • Modified from 5.1 (libevent based), great for CPU bound loads and short running queries • Windows (threadpool), Linux (epoll), Solaris (event ports), FreeBSD/OSX (kevents) • No minimization of concurrent transactions with dynamic pool size • thread_handling=pool-of-threads • https://mariadb.com/kb/en/thread-pool-in-mariadb-55/ MariaDB 5.5+
- 27. PAM Authentication • Authentication using /etc/shadow • Authentication using LDAP, SSH pass phrases, password expiration, username mapping, logging every login attempt, etc. • INSTALL PLUGIN pam SONAME ‘auth_pam.so’; • CREATE USER foo@host IDENTIFIED via pam • Remember to configure PAM (/etc/pam.d or /etc/pam.conf) • http://www.mysqlperformanceblog.com/2013/02/24/using-two-factor- authentication-with-percona-server/ MariaDB 5.2+
- 28. Non-blocking client library • start operation, do work in thread, operation processed, result travels back • use cases: multiple queries against single server (utilize more CPUs); queries against multiple servers (SHOW STATUS on many machines) • https://mariadb.com/kb/en/about-non-blocking-operation-in-the- client-library/ • fast node.js driver available: mariasql MariaDB 5.5+
- 29. LIMIT ROWS EXAMINED • The purpose of this optimization is to provide the means to terminate the execution of SELECTstatements which examine too many rows, and thus use too many resources. • SELECT * from t1, t2 LIMIT 10 ROWS EXAMINED 1000; • https://mariadb.com/kb/en/limit-rows-examined/ MariaDB 5.5+
- 30. SQL Error Logging Plugin • Log errors sent to clients in a log file that can be analysed later. Log file can be rotated (recommended) • a MYSQL_AUDIT_PLUGIN install plugin SQL_ERROR_LOG soname 'sql_errlog.so'; MariaDB 5.5+
- 31. Audit Plugin • Log server activity - who connects to the server, what queries run, what tables touched - rotating log file or syslogd • a MYSQL_AUDIT_PLUGIN INSTALL PLUGIN server_audit SONAME ‘server_audit.so’; MariaDB 10.0+
- 32. Replication made better • Selective skipping of replication events (session-based or on master or slave) • Dynamic control of replication variables (no restarts!) • Using row-based replication? Annotate the binary log with SQL statements • Slaves perform checksums on binary log events • Slaves crash-safe (data stored inside transaction tables) MariaDB 5.3+
- 33. Replication made better II • Group commit in the binary log - finally, sync_binlog=1, innodb_flush_log_at_trx_commit=1 performs • START TRANSACTION WITH CONSISTENT SNAPSHOT • mysqldump —single-transaction —master-data - full non- blocking backup • Parallel replication • Multi-source replication - (real-time) analytics, shard provisioning, backups, etc.
- 34. New KILL syntax • HARD | SOFT & USER USERNAME are MariaDB-specific (5.3.2) • KILL QUERY ID query_id (10.0.5) - kill by query id, rather than thread id • SOFT ensures things that may leave a table in an inconsistent state aren’t interrupted (like REPAIR or INDEX creation for MyISAM or Aria) KILL [HARD | SOFT] [CONNECTION | QUERY] [thread_id | USER user_name] MariaDB 5.3+
- 35. Statistics • Understand server activity better to understand database loads • SET GLOBAL userstat=1; • SHOW CLIENT_STATISTICS; SHOW USER_STATISTICS; • # of connections, CPU usage, bytes received/sent, row statistics • SHOW INDEX_STATISTICS; SHOW TABLE_STATISTICS; • # rows read, changed, indexes • INFORMATION_SCHEMA.PROCESSLIST has MEMORY_USAGE, EXAMINED_ROWS (similar with SHOW STATUS output) MariaDB 5.2+ MariaDB 10.0+
- 36. EXPLAIN enhanced • Explain analyser: https://mariadb.org/explain_analyzer/analyze/ • SHOW EXPLAIN for <thread_id> • EXPLAIN output in the slow query log • EXPLAIN not just for SELECT but INSERT/UPDATE/DELETE MariaDB 10.0+
- 37. Roles • Bundles users together, with similar privileges - follows the SQL standard CREATE ROLE audit_bean_counters; GRANT SELECT ON accounts.* to audit_bean_counters; GRANT audit_bean_counters to ceo; MariaDB 10.0+
- 38. FusionIO • If you have nvmfs (formerly DirectFS), you can disable the innodb_doublewrite buffer • page level compression in background threads (reduces I/O, saves the life of your device) MariaDB 10.0+
- 39. What else is there • Engines: Aria, OQGRAPH, FederatedX • Progress reporting for ALTER/LOAD DATA INFILE • Table Elimination • HandlerSocket • SHUTDOWN functionality • And a lot more….
- 40. Feedback plugin • feedback=on in my.cnf [mysql]
- 41. Connectors • The MariaDB project provides LGPL connectors (client libraries) for: • C • Java • ODBC • Embedding a connector? Makes sense to use these LGPL licensed ones…
- 42. Optimizer MariaDB 10 MySQL 5.6 index_merge=on index_merge_union=on index_merge_sort_union=on index_merge_intersection=on index_merge_sort_intersection=off engine_condition_pushdown=off index_condition_pushdown=on derived_merge=on derived_with_keys=on firstmatch=on loosescan=on materialization=on in_to_exists=on semijoin=on partial_match_rowid_merge=on partial_match_table_scan=on subquery_cache=on mrr=off mrr_cost_based=off mrr_sort_keys=off outer_join_with_cache=on semijoin_with_cache=on join_cache_incremental=on join_cache_hashed=on join_cache_bka=on optimize_join_buffer_size=off table_elimination=on extended_keys=on exists_to_in=off index_merge=on index_merge_union=on index_merge_sort_union=on index_merge_intersection=on engine_condition_pushdown=o n index_condition_pushdown=on mrr=on mrr_cost_based=on block_nested_loop=on batched_key_access=off materialization=on semijoin=on loosescan=on firstmatch=on subquery_materialization_cost_ based=on use_index_extensions=on MariaDB 5.3+
- 43. MariaDB Galera Cluster • MariaDB Galera Cluster is made for today’s cloud based environments. It is fully read-write scalable, comes with synchronous replication, allows multi-master topologies, and guarantees no lag or lost transactions. • Currently 5.5-based • 10.0 is in beta (almost ready for release)
- 44. Trusted by many • Google • Wikipedia • Tumblr • SpamExperts • Limelight Networks • KakaoTalk • Paybox Services
- 45. Quality matters • [email protected] is now commonly on CC when it comes to MySQL bugs • Selective (not blind) merging • Tests (mysql-test/) • MySQL 5.5: 2,466 • MySQL 5.6: 3,603 • MariaDB 10.0: 3,812
- 46. Going forward • column level & block level encryption (Eperi, Google - InnoDB, Aria) • Kerberos authentication plugin • Query timeouts • More Google Summer of Code features (4 students 2014; 3 students 2013) • Full 5.6 compatibility + 5.7 features (so syntax will match for duplicated functionality)
- 48. Resources • We moved to github! https://github.com/MariaDB/server • We’re still on launchpad for older branches: https://launchpad.net/maria • [email protected] • [email protected] • #maria on freenode • facebook.com/MariaDB.dbms • @mariadb / +MariaDB
- 49. Q&A [email protected] | [email protected] http://skysql.com/ | http://mariadb.org/ twitter: @bytebot | url: http://bytebot.net/blog/