M|18 Writing Stored Procedures in the Real World
Download as PPTX, PDF0 likes346 views
This document discusses using MariaDB stored procedures and parallel processing to optimize the "Wordament" word game. It presents solutions to run the game using: 1) A single thread on one node. 2) Multiple threads on one node using MariaDB events. 3) Multiple threads across multiple nodes using MariaDB replication. It concludes that MariaDB supports parallelism through events and replication, but could benefit from a thread API to more easily develop multithreaded stored procedure solutions.
M|18 Writing Stored Procedures in the Real World
- 1. Hacking with MariaDB for fun and not so much profit. Piotr Jarmuż, Allegro.pl Maria DB M|18 - Feb 26/27 2018, New York
- 2. 2 • Introduction about me and my company • MariaDB stored procedures current state and history • The "Wordament" game • Game solution - Single thread • Inherent parallelism in game and MariaDB events • Game solution - Multiple threads • Scaling out – Another node • Game solution - Multiple threads on multiple nodes • Can we wrap it all up in a nice thread API? - Not now, but wait! There is UDF • Conclusions Agenda
- 3. 3 • MariaDB, as any other RDBMS, has stored procedures • in MySQL core since 5.0 beta in 2005 - loosely based on Sybase Transact SQL • procedures, functions, triggers and events • low level "C" UDFs – plugins • resources – oficial documentation - https://mariadb.com/kb/en/library/stored- procedures/ • tutorials - https://mariadb.com/kb/en/library/programming-customizing-mariadb/ MariaDB stored procedures current state and history
- 4. 4 • original from Microsoft • popular as mobile app • displays a board 4x4 with random letters • goal is find as many words as long as possible • 120 sec for solution • 3 letter minimum length • no reuse of board tiles in current run • great for learning new words :) Wordament game CART
- 5. 5 • data structures current word being built - local to the session board representing state of the game - temporary table engine memory solution table for found words - temporary table engine memory dictionary for checking valid words - regular MariaDB table • algorithm used – depth first search tree with dynamic decision prunning • unicode support for many languages $> (echo "start transaction;"; aspell -d ru dump master | aspell -l ru expand | sed 's/ /n/g' | (sed -r 's/(.*)/U1/g’| sort | uniq -i | sed = | sed 'N;s/n/t/'";s/'/''/g"| sed -r "s/(.*)t(.*)/insert into words_ru values (1,'2');/g"; echo "commit;" )) | mysql -D wordament Solution - single thread
- 6. 6 • let’s play! • mysql> call play (‘xtoe evrc aean ygas’); Solution - single thread
- 7. 7 mysql> call play (‘xtoe evrc aean ygas’); | CARNAGE | | CAVERNS | | CORNAGE | | CRANAGE | | EXTRAVAGANCE | +-------------------------+ 275 rows in set (1.40 sec) Solution - single thread
- 8. 8 • up to 16 independent search trees can be run in parallel • leveraging MariaDB event scheduler • in Mysql core since 5.1 • complementary objects events • DDL create, drop, alter • schedule, one-time, recurring • event states ENABLED, DISABLED Inherent parallelism in game and MariaDB events
- 9. 9 • refactoring code a bit • range root search for distributing load • replicating game state • expanding data structures - new table gsolution • auxillary tables and support procedures • it scales up! Solution - multiple threads
- 10. 10 • another node set-up in multi-master replication • event behavior in replicated environment • another event state SLAVESIDE_DISABLED • flip-flopping state - there can only be one • transfering execution via replication Scaling out – Another node
- 11. 11 • no refactoring this time - we are already parallel • create all events on one node • selectively enable events on 2 hosts - load distribution • gsolution table converges results via bidirectional replication • it scales out! Solution - multiple threads, multiple nodes
- 12. 12 Solution - multiple threads, multiple nodes
- 13. 13 • there is a "thread" infrastructure in MariaDB - a bit awkward • we would need a nice posix thread like API • create_thread(), start_thread(), join_thread(), destroy_thread() • sadly event management are DDL - no-go in stored procedures • why not write our own udf_execute_immediate with DDL support • "C" level UDF to the rescue • back to the game - putting it all together with thread API Can we wrap it all up in a nice thread API?
- 14. 14 • MariaDB is inherently parallel environment • needs a little user support in parallelisation • scaling up and out - events scheduler + replication and/or sharding • new exciting features in future Conclusions
- 15. Thank you for your attention. Questions?