jacobs_tuuri_performance

InnoDB Designing Applications and Configuring for Best Performance Heikki Tuuri CEO Innobase Oy Vice President, Development Oracle Corporation 1 InnoDB

Today’s Topics Application Design and Performance Basic InnoDB Configuration Maximizing CPU Efficiency Maximizing Disk I/O Efficiency Maximizing Transaction Throughput Some Operating System Tips Speeding Bulk Data Operations

Application Design and Performance Create appropriate indexes, so that SELECTs, UPDATEs and DELETEs do not require table scans to find the rows index the most-often referenced columns in your tables try different column orderings in composite (multi-column) keys Note: too many indexes can slow INSERT, UPDATE, DELETE Use EXPLAIN SELECT ... to check that query plans look sensible and there are good indexes for them Consider “fat indexes” (a secondary index including extra columns that queries need), to avoid clustered index lookup Note: InnoDB's secondary index records always contain the clustered index columns (usually the table’s PRIMARY KEY); consider storage implications of long keys! Proper application design is the most important part of performance tuning

Application Design and Performance Use SQL statements that process sets of rows at a time, rather than just one row-at-a-time Enforce referential integrity within the server, not at the application level Use transactions to group operations Avoid excessive commits (e.g., avoid autocommit) But don’t make transactions to large, or too long-lasting Proper application design is the most important part of performance tuning

Basic InnoDB Configuration [mysqld] # You can write your other MySQL server options here # ... # Data files must be able to hold your data and indexes. # Make sure that you have enough free disk space. innodb_data_file_path = ibdata1:10M:autoextend # # Set buffer pool size to 50-80% of your computer's memory innodb_buffer_pool_size=70M innodb_additional_mem_pool_size=10M # # Set the log file size to about 25% of the buffer pool size innodb_log_file_size=20M innodb_log_buffer_size=8M # innodb_flush_log_at_trx_commit=1

Analyzing InnoDB Performance Problems Print several consecutive SHOW INNODB STATUS\G outputs during high workload Print SHOW STATUS Enable the MySQL 'slow query log' In Unix/Linux, use 'top', vmstat, iostat In Windows use the Task Manager

CPU or Disk-Bound Workload? Database throughput can be determined by the processor speed or by the disk access time Both CPU and disk use need to be optimized first tune the more limited resource, the one limiting throughput Unix/Linux shell command ‘ top ’ and Windows Task Manager show Total CPU usage CPU usage < 70 % suggests a disk-bound workload

Save CPU Time Use multi-row inserts: INSERT INTO t VALUES (1, 2), (2, 2), (3, 2); Use stored procedures Instead of many small SELECT queries, try to use one bigger query Use the MySQL query cache: query_cache_type=ON query_cache_size=100M Save CPU time by minimizing communications between the client and the server

Reduce Disk Access Bottleneck Keep your hot data set small; delete obsolete data Run OPTIMIZE TABLE ... Rebuilds the table to eliminate fragmentation -> smaller table! Requires taking the database offline Create 'fat' secondary indexes with extra columns Use a large buffer pool to cache more data set innodb_buffer_pool_size up to 80% of computer's RAM Use sufficiently large log files set innodb_log_file_size to approx 25% of the buffer pool (assumes 2 log files with combined size not more than 4 GB)

Make Your InnoDB Tables Smaller Use new (default) COMPACT InnoDB table format in MySQL 5.0 typically saves ~20 % of space vs. old REDUNDANT table format Use a short PRIMARY KEY , or surrogate key InnoDB stores the primary key in every secondary index record InnoDB creates 6-byte internal ROW_ID for tables with no primary key or UNIQUE, NOT NULL index Use VARCHAR instead of CHAR(n) where possible InnoDB always reserves n bytes for fixed-size CHAR( n ) Note: UPDATEs to VARCHAR columns can cause fragmentation Smaller tables use fewer disk blocks, thus requiring less i/o

Reduce Transaction Commit Disk I/O InnoDB must flush its log to a durable medium at a COMMIT Log flush is not normally a bottleneck with a battery-backed disk controller and write-back cache enabled But … log flush to a physical disk can be a very serious bottleneck Log flush typically takes 4 milliseconds on a fast disk Minimize log flushes by wrapping several individual INSERT s, UPDATE s, DELETE s in one transaction OR, allow transactions to commit without flush Set innodb_flush_log_at_trx_commit=2 in my.cnf WARNING : can lose 1 second’s worth of transactions that occurred prior to an operating system crash

Spread Disk I/O to Several Disks Two InnoDB modes for storing tables in files: one file/table ( innodb_file_per_table in my.cnf ) multiple tables per ibdata file With one InnoDB file/table on Unix/Linux Symlink MySQL database directories to separate disk drives Also can symlink .ibd (data) files on different drives Note: ALTER TABLE and CREATE INDEX relocate the . ibd file because the table is recreated With InnoDB data in multiple ibdata files, create the files on different disk drives InnoDB fills ibdata files linearly, starting from first listed in the my.cnf Though maybe the easiest solution is to buy a RAID disk array! Store InnoDB data on multiple physical disk drives

Avoid O/S Double Buffering InnoDB itself buffers database pages. Using the operating system file cache is a waste of memory and CPU time the o/s can page out parts of the InnoDB buffer pool use ~80% of physical RAM for InnoDB, and turn off o/s file caching Linux : set innodb_flush_method=O_DIRECT in my.cnf to advise o/s not to buffer InnoDB files Solaris : use a direct i/o UFS filesystem use mount option forcedirectio ; see mount_ufs(1M) with Veritas file system VxFS, use the mount option convosync=direct Windows : InnoDB always uses unbuffered async i/o. No action required Devote physical RAM to InnoDB, not the o/s file cache!

Avoid Slow O/S File Flushing Linux/Unix use fsync() to flush data to disk fsync() is extremely slow on some old Linux and Unix versions NetBSD was mentioned by a user as one such platform Setting innodb_flush_method=O_DSYNC in my.cnf may help

Avoid “Thread Thrashing” in High Concurrency Environments With too many threads (on Linux), performance can degrade badly SHOW INNODB STATUS\G shows many threads waiting for a semaphore queries pile up, and little work gets done throughput can drop to 1/1000 of the normal Workaround: set innodb_thread_concurrency to 8 , or even to 1 in my.cnf We are working on removing this problem MySQL-5.1.9 behaves somewhat better in this respect Limit the number of threads executing inside InnoDB at once

Avoid Transaction Deadlocks Use short transactions, which are less prone to collide and deadlock Appropriate indexes reduce table scans and reduce the number of locks taken Access tables and rows in the same predefined order throughout your application Design and configure to reduce the likelihood of deadlocks, which lower throughput if they occur frequently

Avoid Transaction Deadlocks (2) With MySQL V5.0 or before, use the my.cnf option innodb_locks_unsafe_for_binlog to minimize next-key locking This is safe only if you do not use binlogging for replication, OR, if your application is not prone to 'phantom row' problems Beginning with MySQL-5.1.xx , if you use row-based replication, you can safely reduce next-key locking by … setting innodb_locks_unsafe_for_binlog use TRANSACTION ISOLATION LEVEL READ COMMITTED Design and configure to reduce the likelihood of deadlocks, which lower throughput if they occur frequently

Avoid Transaction Deadlocks (3) Last resort: use table locks to serialize transactions table locks eliminate deadlocks, but reduce throughput You have to SET AUTOCOMMIT=0 to make them work properly Design and configure to reduce the likelihood of deadlocks, which lower throughput if they occur frequently SET AUTOCOMMIT=0; LOCK TABLES t1 WRITE, t2 READ, ...; <INSERTs, UPDATEs, DELETEs> …; UNLOCK TABLES; COMMIT;

Speed up Table Imports to InnoDB Execute many INSERT s per transaction, not one transaction per row! Before loading, sort the rows in primary key order to reduce random disk seeks For tables with foreign key constraints, turn off foreign key checks with SET FOREIGN_KEY_CHECKS=0 Do this only if you know your data conforms to the constraints Use only for the duration of the import session For tables with unique secondary indexes, , turn off uniqueness checking with SET UNIQUE_CHECKS=0 Do this only if the data is unique in required columns Use only for the duration of the import session Pre-process your data and know whether it is valid before loading

Avoid Big Deletes or Rollbacks Use TRUNCATE TABLE to empty a table, rather than DELETE all rows Beware of big rollbacks … use smaller transactions! InnoDB speeds INSERT s with an insert buffer, but … Rollback occurs row-by-row – can be 30 times slower than inserts If you end up with a runaway rollback, drop the table if you can afford losing the data

Summary: InnoDB Performance Tuning Design SQL and Indexes with care Follow buffer pool, log & data file size guidelines Use performance problem diagnostic tools Minimize client-server, MySQL-InnoDB traffic Reduce i/o w/ small data, large buffers, good indexes Spread i/o to multiple physical disk drives Avoid o/s double buffering, slow fsync(), “thread thrashing” Reduce or eliminate transaction deadlocks Pre-process your data and know its validity Avoid big deletes or rollbacks

A Q & Q U E S T I O N S A N S W E R S

jacobs_tuuri_performance

More Related Content

What's hot (20)

Similar to jacobs_tuuri_performance (20)

More from Hiroshi Ono (20)

Recently uploaded (20)

jacobs_tuuri_performance