MySQL crash course by moshe kaplan

The VP R&D Open Seminar

MySQL Crash Course
From basics to experts
mokplan@gmail.com
http://top-performance.blogspot.com


WHY WE CHOSE MYSQL?


Who is Using MySQL?

3

Who is Behind MySQL


What MySQL is Made of?

5
http://dev.mysql.com/tech-resources/articles/mysql_5.0_psea1.html

MySQL Limitations

50-150M Records/Table
50-5000 SELECT Statements/Second

6

Why Do I Care?
From 0 to 100 (US mass adaptation)
Phone: 100 yrs
Radio: 40 yrs
TV: 30 yrs
Mobile: 20 yrs
Internet: 10 yrs
Facebook: 2 yrs

7

100K New Users/Week

8


WHAT MYSQL IS MADE OF?


MySQL Structure
Daemon
/var/lib/mysql
ibdata1
ib_logfile0, Ib_logfile1
/mysql
/database
/performance_schema
/etc/my.cnf


What to Install
wget mysql_package
yum –y localinstall mysql_package

The Packages:
Devel+Client+Server
Relevant platform (x86/x64)
Relevant operating system


MySQL Installation



DATABASE LOGICAL DESIGN
(OR ERD)


Instances, Schemas and inside information


1:Many

schoolId: schoolId:
Primary Key Foreign Key


Many:Many

?: ?:
Primary Key Foreign Key


Many:Many in Practice

testId: testId, studentId:
Primary Key studentId: Primary Key
Foreign Key


Design a Student Information System ERD



FROM ERD TO A DATABASE


Connect to the Database


Create a Schema/Database


Create a Table


Implement the a Student Information System ERD



SQL: DDL


Databases Manipulation
SHOW DATABASES;
CREATE DATABASE mydb;
DROP DATABASE mydb;
USE mydb;


Tables Manipulation
SHOW TABLES;
CREATE TABLE users (
id int,
INDEX users_id (id)
);
DROP TABLE users;
ALTER TABLE users ADD INDEX IX_id (id);
ALTER TABLE users DROP INDEX IX_id;



SQL: DML


SELECT
SELECT table.field, table.*
FROM table
WHERE table.field = 123
ORDER BY field1, field2 DESC


SELECT + Join
SELECT c.*, u.*
FROM classes c
INNER JOIN users u
ON c.teacherId = u.id


LEFT JOIN


GROUP BY
SELECT gender, count(id)
FROM users
WHERE gender = ‘M’
GROUP BY gender
HAVING count(id) > 10


UNION
SELECT id
FROM users
WHERE gender = ‘M’
UNION ALL
SELECT id
FROM users
WHERE gender = ‘F’
ORDER BY id;


UPDATE
UPDATE users
SET gender = ‘M’
WHERE gender IS NULL;


INSERT
INSERT INTO users (gender)
VALUES (‘M’);

INSERT INTO users (gender)
SELECT gender
FROM users
WHERE gender = ‘M’;


DELETE
DELETE
FROM users
WHERE gender = ‘M’;



MySQL Authentication and Authorization

SECURITY


Root User
First Time
mysqladmin -u root password NEWPASSWORD
Nth Time
mysqladmin -u root -p'oldpassword' password
newpass
How to enter
Mysql –uuser_name –ppassword –D
database_name
-h host_name/ip -A


Users Table
mysql> SELECT * FROM mysql.user limit 1G
*************************** 1. row ***************************
Host: localhost
User: root
Password: *4FB5C598BC3FF56E262F863A715F8CB2976A9C69
Select_priv: Y
Insert_priv: Y
Update_priv: Y
Delete_priv: Y
Create_priv: Y
Drop_priv: Y
….
1 row in set (0.00 sec)


Create a Users and Provide Permissions
-- Create a user w/ password
CREATE USER 'readonly'@'%' IDENTIFIED BY
‘sakexkas';
-- Grant Permissions
GRANT select ON my_db.* to 'readonly'@'%';
GRANT ALL PRIVILEGES ON *.* to
‘master_user’@’%’ WITH GRANT OPTION;
FLUSH PRIVILEGES;

-- Beware of too complex passwords



MYSQL AND LINUX ISSUES


Built in Linux Security Measures
IP TABLES
sudo /etc/init.d/iptables stop
sudo yum –y remove iptables
Configure the relevant rules
SELinux
sudo echo 0 > /selinux/enforce
vi /etc/selinux/config: SELINUX=enforcing  SELINUX=disabled
Files Limit (Table Partitioning)
/etc/security/limits.conf
mysql soft nofile 8192
mysql hard nofile 8192
my.cnf:
open_files_limit = 8192


Disk Resizing
http://top-
performance.blogspot.com/2012/02/do-you-
need-more-disk-space-for-mysql.html


Create a Master/Slave configuration



HIGH AVAILABILITY
ARCHITECTURES


Master/Slave


Master/Master

?

Show Status (1)
mysql> show slave statusG
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 10.12.12.167
Master_User: repl
Master_Port: 3306
Connect_Retry: 60

Master_Log_File: master.000094
Read_Master_Log_Pos: 167250867
Relay_Log_File: mysqld-relay-bin.000137
Relay_Log_Pos: 167251010

Relay_Master_Log_File: master.000094
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table: mysql.%,information_schema.%


Show Status (2)
Last_Errno: 0
Last_Error:
Skip_Counter: 0

Exec_Master_Log_Pos: 167250867
Relay_Log_Space: 167251310
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 1


Skip
DON’T DO THAT unless you have to!
The Process:
STOP SLAVE;
SET GLOBAL SQL_SLAVE_SKIP_COUNTER =
1;
START SLAVE;
SHOW SLAVE STATUSG


How to Implement?
Stop a Master and record position
Copy a Master (instance/disk) to a Slave
Start the Master
Change slave server-id
SET MASTER HOST …
START SLAVE
Verify
Or…
http://top-performance.blogspot.co.il/2012/03/how-to-
setup-mysql-replication-in-11.html



SCALE-ABLE ARCHITECTURES


Do You Remember?

53

Strategy A: Sharding

54

Strategy B: In Memory Databases

55

700 Inserts/Sec

In Memory Engine

3000 Inserts/Sec

Amazon AWS

Standard InnoDB Engine

Large Instance 700 Inserts/Sec

56

IMDB - Implementation

57

Lessons
Cache: MySQL IMDB
HA: MySQL Replication
Persistence: Periodic 5min Batch
Scalability: Sharding

4 Bill Req/Day using 25 machines (2000/s)

58

Strategy C: MapReduce

59

Strategy D: NoSQL
insert
get
multiget
remove
truncate

<Key, Value>
http://wiki.apache.org/cassandra/API

60

When Should I Choose NoSQL?

Eventually Consistent
Document Store
Key Value

http://guyharrison.squarespace.com/blog/tag/nosql

61

Sharding Again

62

Vertical Sharding

63

Horizontal Sharding
Static Hashing
Complex growth Mod 10 = 0

Mod 10 = 1
Simple
Mod 10 = 2

Mod 10 = 3

Mod 10 = 4

Mod 10 = 5

Mod 10 = 6

Mod 10 = 7

Mod 10 = 8

Mod 10 = 9

64

Horizontal Sharding
Key locations are defined in a directory
Simple growth
Directory is SPOF

65

Horizontal Sharding
Static Hashing with Directory Mapping
Simple Growth
Small Directory still SPOF

Mod 1000 = 4

66

Horizontal Sharding
Each key signed by DB#
generated on creation
Simple growth
New key generation is SPOF

67

Limitations: Reporting

68

Best Practices
$connection =
new_db_connection("customer://1234");
$statement =
$connection->prepare(
$sql_statement,
$params
);
$result =
$statement->execute();

69


BACKUP AND RESTORE


Options
mysqldump
Physical disk copy
3rd Party solution: XtraDB, amanda


Backup Your Server



HANDLING PROBLEMS


Error Log
Get the Location:
[/]$ sudo more /etc/my.cnf | grep -i error
log-error=/var/lib/mysql/log.err
Look for issues
[/]$ sudo tail -n 20 /var/lib/mysql/log.err
121025 13:24:23 InnoDB: Initializing buffer pool, size = 128.0M
121025 13:24:23 InnoDB: Completed initialization of buffer pool
121025 13:24:24 InnoDB: highest supported file format is
Barracuda.
121025 13:24:25 InnoDB: Waiting for the background threads to
start



DEVOPS MYSQL


-e
mysql
-uuser
-ppassword
-Ddatabase
-hserver
-e”SELECT * FROM table;”


Import Data
mysql
-uuser
-ppassword
-Ddatabase
-hserver
< /dir/to/filename.sql


Export Data
mysqldump
-uuser
-ppassword
--databases database_name
--single-transaction --master-data=1 >
/path/to/backup/file.sql



MYSQL TUNING


my.cnf


mysqltuner


Slow Query
log-slow-queries = /var/log/mysql/mysql-
slow.log
long_query_time = 1
log-queries-not-using-indexes


Profiling
http://top-
performance.blogspot.co.il/2011/07/mysql-
statement-profiling.html


RAM Usage Optimization
Preferred: SizeOf(database) < SizeOf(RAM)
innodb_buffer_pool_size
50-70% of your RAM

85

Do We Utilize the Cache?
SHOW STATUS LIKE 'Qcache%';

Qcache_free_blocks 718
Qcache_free_memory 13004008
Qcache_hits 780759
Qcache_inserts 56292
Qcache_lowmem_prunes 0
Qcache_not_cached 3711
Qcache_queries_in_cache 1715
Qcache_total_blocks 4344

86

Cache Queries
query_cache_type = 1
query_cache_limit = 1M
query_cache_size = 16M

87

Buffer Optimization
key_reads/key_read_requests < 0.01
If not
Increase Key Buffer

88

CPU Optimization
thread_concurrency = 2 X #(CPU Cores)

89

Tune Your Server


Summary


What happens when mysql fails to start…
Create my.cnf
touch /etc/my.cnf
Create the error log
vi /etc/my.cnf
[mysqld]
log-error=/var/lib/mysql/log.err
If the my.cnf is already configured
more /etc/my.cnf | grep –i error
Restart mysql
Find the error
tail –n 50 /var/lib/mysql/log.err
In this case we kill all the mysql proceses
ps –aux | grep –i mysql
sudo kill process_id


The Bottom Line: Grow ∞

Thank you!
and Keep Performing!

Moshe Kaplan

93

MySQL crash course by moshe kaplan

Recommended

More Related Content

What's hot (20)

Viewers also liked (13)

Similar to MySQL crash course by moshe kaplan (20)

More from Moshe Kaplan (20)

Recently uploaded (20)

MySQL crash course by moshe kaplan

Editor's Notes