Caching and tuning fun for high scalability @ FOSDEM 2012
7 likes11,264 views
Caching has been a 'hot' topic for a few years. But caching takes more than merely taking data and putting it in a cache : the right caching techniques can improve performance and reduce load significantly. But we'll also look at some major pitfalls, showing that caching the wrong way can bring down your site. If you're looking for a clear explanation about various caching techniques and tools like Memcached, Nginx and Varnish, as well as ways to deploy them in an efficient way, this talk is for you.
Ad
More Related Content
What's hot (20)
Viewers also liked (19)
Ad
Similar to Caching and tuning fun for high scalability @ FOSDEM 2012 (20)
Ad
More from Wim Godden (20)
Recently uploaded (20)
Caching and tuning fun for high scalability @ FOSDEM 2012
- 1. Caching and tuning fun for high scalability Wim Godden Cu.be Solutions
- 2. Who am I ? Wim Godden (@wimgtr)
- 3. Owner of Cu.be Solutions (http://cu.be)
- 4. Open Source developer since 1997
- 7. Zend Framework Certified Engineer
- 9. Who are you ? Developers ?
- 11. Managers ?
- 13. Caching and tuning fun for high scalability Wim Godden Cu.be Solutions
- 14. Goals of this tutorial Everything about caching and tuning
- 15. A few techniques How-to
- 16. How-NOT-to -> Increase reliability, performance and scalability
- 17. 5 visitors/day -> 5 million visitors/day
- 18. (Don't expect miracle cure !)
- 19. LAMP
- 20. Architecture
- 21. Our base benchmark Apachebench = useful enough
- 22. Result ? Single webserver Proxy Static PHP Static PHP Apache + PHP 3900 17.5 6700 17.5 Limit : CPU, network or disk Limit : database
- 23. Caching
- 24. What is caching ?
- 25. What is caching ? select * from article join user on article.user_id = user.id order by created desc limit 10
- 26. Theory of caching if ($data == false) DB
- 27. Theory of caching DB
- 28. Caching techniques #1 : Store entire pages #2 : Store part of a page (block) #3 : Store data retrieval (SQL ?) #4 : Store complex processing result #? : Your call ! When you have data, think : Creating time ?
- 31. How to find cacheable data New projects : start from 'cache everything'
- 32. Existing projects : Look at MySQL slow query log
- 33. Make a complete query log (don't forget to turn it off !)
- 34. Check page loading times
- 35. Caching storage - Disk Data with few updates : good
- 36. Caching SQL queries : preferably not
- 37. DON'T use NFS or other network file systems especially for sessions
- 38. locking issues !
- 39. high latency
- 40. Caching storage - Disk / ramdisk Local 5 Webservers -> 5 local caches
- 41. -> Hard to scale
- 42. How will you keep them synchronized ? -> Don't say NFS or rsync !
- 43. Caching storage - Memcache(d) Facebook, Twitter, YouTube, … -> need we say more ?
- 44. Distributed memory caching system
- 45. Multiple machines ↔ 1 big memory-based hash-table
- 46. Key-value storage system Keys - max. 250bytes
- 47. Values - max. 1Mbyte
- 48. Caching storage - Memcache(d) Facebook, Twitter, YouTube, … -> need we say more ?
- 49. Distributed memory caching system
- 50. Multiple machines ↔ 1 big memory-based hash-table
- 51. Key-value storage system Keys - max. 250bytes
- 52. Values - max. 1Mbyte Extremely fast... non-blocking, UDP (!)
- 53. Memcache - where to install
- 54. Memcache - where to install
- 55. Memcache - installation & running it Installation Distribution package
- 56. PECL
- 57. Windows : binaries Running No config-files
- 58. memcached -d -m <mem> -l <ip> -p <port>
- 59. ex. : memcached -d -m 2048 -l 172.16.1.91 -p 11211
- 60. Caching storage - Memcache - some notes Not fault-tolerant It's a cache !
- 62. Lose shopping cart data
- 63. ...
- 64. Caching storage - Memcache - some notes Not fault-tolerant It's a cache !
- 66. Lose shopping cart data
- 67. … Firewall your Memcache port !
- 68. Memcache in code <?php $memcache = new Memcache(); $memcache->addServer( '172.16.0.1' , 11211); $memcache->addServer( '172.16.0.2' , 11211); $myData = $memcache->get( 'myKey' ); if ($myData === false ) { $myData = GetMyDataFromDB(); // Put it in Memcache as 'myKey', without compression, with no expiration $memcache->set( 'myKey' , $myData, false , 0); } echo $myData;
- 69. Where's the data ? Memcache client decides (!)
- 70. 2 hashing algorithms : Traditional Server failure -> all data must be rehashed Consistent Server failure -> 1/x of data must be rehashed (x = # of servers)
- 71. Benchmark with Memcache Single webserver Proxy Static PHP Static PHP Apache + PHP 3900 17.5 6700 17.5 Apache + PHP + MC 3900 55 6700 108
- 72. Memcache slabs (or why Memcache says it's full when it's not) Multiple slabs of different sizes : Slab 1 : 400 bytes
- 73. Slab 2 : 480 bytes (400 * 1.2)
- 74. Slab 3 : 576 bytes (480 * 1.2) (and so on...) Multiplier (1.2 here) can be configured
- 75. Store a lot of very large objects
- 76. -> Large slabs : full
- 77. -> Rest : free
- 78. -> Eviction of data !
- 79. Memcache - Is it working ? Connect to it using telnet "stats" command ->
- 80. Use Cacti or other monitoring tools STAT pid 2941 STAT uptime 10878 STAT time 1296074240 STAT version 1.4.5 STAT pointer_size 64 STAT rusage_user 20.089945 STAT rusage_system 58.499106 STAT curr_connections 16 STAT total_connections 276950 STAT connection_structures 96 STAT cmd_get 276931 STAT cmd_set 584148 STAT cmd_flush 0 STAT get_hits 211106 STAT get_misses 65825 STAT delete_misses 101 STAT delete_hits 276829 STAT incr_misses 0 STAT incr_hits 0 STAT decr_misses 0 STAT decr_hits 0 STAT cas_misses 0 STAT cas_hits 0 STAT cas_badval 0 STAT auth_cmds 0 STAT auth_errors 0 STAT bytes_read 613193860 STAT bytes_written 553991373 STAT limit_maxbytes 268435456 STAT accepting_conns 1 STAT listen_disabled_num 0 STAT threads 4 STAT conn_yields 0 STAT bytes 20418140 STAT curr_items 65826 STAT total_items 553856 STAT evictions 0 STAT reclaimed 0
- 81. Memcache - backing up
- 82. Memcache - tip Page with multiple blocks ? -> use Memcached::getMulti() But : what if you get some hits and some misses ?
- 83. Updating data
- 85. Adding/updating data $memcache->delete( 'LCD_Popular_Product_List' );
- 87. Adding/updating data - Why it crashed
- 88. Adding/updating data - Why it crashed
- 89. Adding/updating data - Why it crashed
- 90. Cache stampeding
- 91. Cache stampeding
- 92. Memcache code ? DB
- 93. Cache warmup scripts Used to fill your cache when it's empty
- 94. Run it before starting Webserver !
- 95. 2 ways : Visit all URLs Error-prone
- 96. Hard to maintain Call all cache-updating methods Make sure you have a warmup script !
- 97. Cache stampeding - what about locking ? Seems like a nice idea, but... While lock in place
- 98. What if the process that created the lock fails ?
- 99. LAMP... -> LAMMP -> LNMMP
- 100. Nginx Web server
- 101. Reverse proxy
- 102. Lightweight, fast
- 103. 9.6% of all Websites
- 104. Nginx No threads, event-driven
- 105. Uses epoll / kqueue
- 106. Low memory footprint
- 107. 10000 active connections = normal
- 108. Nginx - a true alternative to Apache ? Not all Apache modules mod_auth_*
- 109. mod_dav*
- 110. … Basic modules are available
- 111. Some 3 rd party modules (needs recompilation !)
- 112. Nginx - Configuration server { listen 80; server_name www.domain.ext *.domain.ext; index index.html; root /home/domain.ext/www; } server { listen 80; server_name photo.domain.ext; index index.html; root /home/domain.ext/photo; }
- 113. Nginx + PHP-FPM - performance ? Single webserver Proxy Static PHP Static PHP Apache + PHP 3900 17.5 6700 17.5 Apache + PHP + MC 3900 55 6700 108 Nginx + PHP-FPM + MC 11700 57 11200 112
- 115. Varnish Not just a load balancer
- 116. Reverse proxy cache / http accelerator / …
- 117. Caches (parts of) pages in memory
- 118. Careful : uses threads (like Apache)
- 119. Nginx usually scales better (but doesn't have VCL)
- 120. Varnish - backends + load balancing backend server1 { .host = "192.168.0.10"; } backend server2{ .host = "192.168.0.11"; } director example_director round-robin { { .backend = server1; } { .backend = server2; } }
- 121. Varnish - VCL Varnish Configuration Language
- 122. DSL (Domain Specific Language) -> compiled to C Hooks into each request
- 123. Defines : Backends (web servers)
- 124. ACLs
- 125. Load balancing strategy Can be reloaded while running
- 126. Varnish - whatever you want Real-time statistics (varnishtop, varnishhist, ...)
- 127. ESI
- 128. Varnish - ESI Perfect for caching pages In your article page output : <esi:include src="/news"/> In your Varnish config : sub vcl_fetch { if (req.url == "/news") { esi; /* Do ESI processing */ set obj.ttl = 2m; } elseif (req.url == "/nav") { esi; set obj.ttl = 1m; } elseif …. … . }
- 129. Varnish with ESI - hold on tight ! Single webserver Proxy Static PHP Static PHP Apache + PHP 3900 17.5 6700 17.5 Apache + PHP + MC 3900 55 6700 108 Nginx + PHP-FPM + MC 11700 57 11200 112 Varnish - - 11200 4200
- 130. Varnish - what can/can't be cached ? Can : Static pages
- 131. Images, js, css
- 132. Pages or parts of pages that don't change often (ESI) Can't : POST requests
- 133. Very large files (it's not a file server !)
- 136. ESI -> no caching on user-specific content ? Logged in as : Wim Godden 5 messages TTL = 5min TTL=1h TTL = 0s ?
- 137. Coming soon... Based on Nginx
- 138. Reduces load by 50 – 95% Requires code changes !
- 139. Well-built project -> few changes
- 140. Effect on webservers and database servers
- 141. What's the result ?
- 142. What's the result ?
- 143. Figures First customer : No. of web servers : 18 -> 4
- 144. No. of db servers : 6 -> 2
- 145. Total : 24 -> 6 (75% reduction !) Second customer (already using Nginx + Memcache) : No. of web servers : 72 -> 8
- 146. No. of db servers : 15 -> 4
- 147. Total : 87 -> 12 (86% reduction !)
- 148. Availability Stable at 2 customers
- 149. Still under heavy development
- 150. Final release : July 2012
- 151. Released as ????
- 152. Tuning
- 153. PHP speed - some tips Upgrade PHP - every minor release has 5-15% speed gain !
- 154. Use an opcode cache
- 155. Caching storage - Opcode caching
- 156. DB speed - some tips Use same types for joins i.e. don't join decimal with int RAND() is evil !
- 157. count(*) is evil in InnoDB without a where clause !
- 158. Persistent connect is sort-of evil
- 159. Caching & Tuning @ frontend http://www.websiteoptimization.com/speed/tweak/average-web-page/
- 160. Frontend tuning 1. You optimize backend 2. Frontend engineers messes up -> havoc on backend 3. Don't forget : frontend sends requests to backend ! SO... Care about frontend
- 161. Test frontend
- 162. Check what requests frontend sends to backend
- 163. Tuning frontend Minimize requests Combine CSS/JavaScript files
- 164. Tuning frontend Minimize requests Combine CSS/JavaScript files
- 165. Use CSS Sprites
- 166. CSS Sprites
- 167. Tuning content - CSS sprites
- 168. Tuning content - CSS sprites 11 images 11 HTTP requests 24KByte 1 image 1 HTTP requests 14KByte
- 169. Tuning frontend Minimize requests Combine CSS/JavaScript files
- 170. Use CSS Sprites (horizontally if possible) Put CSS at top
- 171. Put JavaScript at bottom Max. no connections
- 172. Especially if JavaScript does Ajax (advertising-scripts, …) ! Avoid iFrames Again : max no. of connections Don't scale images in HTML
- 173. Have a favicon.ico (don't 404 it !) -> see my blog
- 174. What else can kill your site ? Redirect loops Multiple requests More load on Webserver
- 175. More code to process Additional latency for visitor
- 176. Try to avoid redirects anyway Watch your logs, but equally important...
- 177. Watch the logging process ->
- 178. Logging = disk I/O -> can kill your server !
- 179. Slashdot effect
- 180. Above all else... be prepared ! Have a monitoring system
- 181. Use a cache abstraction layer (disk -> Memcache)
- 182. Don't install for the worst -> prepare for the worst
- 183. Have a test-setup
- 184. Have fallbacks -> Turn off non-critical functionality
- 185. Questions ?
- 186. Questions ?
- 187. Contact Twitter @wimgtr
- 191. Thanks !