Planning for-high-performance-web-application
Download as PPT, PDF1 like1,854 views
Planning for performance is a document about optimizing web applications for performance. It discusses basic practices like using source code management, continuous integration tools, and issue trackers. It also covers hardware platforms, software platforms, system essentials like monitoring and security tools, and optimizations like caching, SQL tuning, and scaling through load balancing, databases, file systems, and more. The overall document provides an overview of many aspects of planning for and ensuring high performance of web applications.
![[245] presto 내부구조 파헤치기](https://cdn.slidesharecdn.com/ss_thumbnails/245presto-150915054242-lva1-app6891-thumbnail.jpg?width=560&fit=bounds)
![[NYC Meetup] Docker at Nuxeo](https://cdn.slidesharecdn.com/ss_thumbnails/dockermeetupnyc-140605110024-phpapp01-thumbnail.jpg?width=560&fit=bounds)
Ad
More Related Content
What's hot (18)
Viewers also liked (9)
Ad
Similar to Planning for-high-performance-web-application (20)
Ad
More from Nguyễn Duy Nhân (20)
Recently uploaded (20)
Planning for-high-performance-web-application
- 1. Planning for performance For web developer, open discussion Tin@Beijing Open Party
- 2. Teachers do not have to be stronger than themselves? Already do not have as good as teacher
- 3. Agenda Basic programming practice Hardware platform Software platform System essentials Optimizations Load Balancing
- 4. Basic practices Use proper SCM CVS SVN Mercurial Git
- 5. Basic practices Use a auto-build system Shell scrips Make Ant, Nant Rake
- 6. Basic practices Use a Continues Integration tool So first you need a lot of tests Add auto test, compile job as daily task Use CI tools to monitor health of your code base CruiseControl, Luntbuild, Continnum, Hudson Cruise, Teamcity, Banboo Use cc-tray, cc-menu desktop widget
- 7. Basic practices Use a issue tracker Trac (only svn) Bugzilla, Mantis Bug Tracker Jira Mingle BugFree
- 8. Voice from twitter Be sure to test! Be sure to earlier test! Be sure to earlier test! Otherwise, you're dead. Any part test. Performance testing to be handed over to the user to do. As makes sense. So to do well log.
- 9. Basic practices Lifecycle control Develop -> Test -> Deploy Release management Trunk, Branch, Tag Milestone, Release candicate
- 10. Basic practices Use Agile methodologies XP practices TDD Pair programming Scrum Hybrid agile
- 11. Hardware platform Use economical hardware CPU and Memory Disk and disk I/O (Raid) NIC Power and fan 1U 2U 3U 4U ?
- 12. Hardware platform Brand Dell, IBM, HP, Lenovo, Asus? Service quality Hardware redundancy Part redundancy Availability and Lead Time (critical parts) Capacity redundancy Future plan?
- 13. Network & hosting VPS, 虚拟主机 Co-Located Hardware (colo), 主机托管 Bandwidth, Duel lines, air-condition Geo-location Self-Hosting How to choose network hardware (switch/router)? Cisco, Huaway, Foundry
- 14. Software platform Use pre-compiled OS and software Choose a OS CentOS, Redhat, Suse Freebsd Solaris no ubuntu server (from nicholas ding)
- 15. Software platform Choose a language (scriptiing language is better) PHP Python Perl Ruby Java Many many many... but not c...
- 16. Software platform Choose a database ( or data provider) Mysql Posgresql Big table implementation?
- 17. Now, let’s go
- 18. System essentials Web server Apache Lighthttpd Nginx Tux, Cherokee, Lightspeed Tomcat, Jetty Mongrel, Thin
- 19. System essentials Different deployment style (python/ruby) Apache + mod_python (mod_rails, passenger) Fastcgi, SCGI, CGI Proxy (Load balancing) + Multi-server instance thread? process?
- 20. System essentials Monitoring your system web server logs Webalizer, Report Magic Beacon (seperate static file server tracker) error log analysis AWStats & Google Analytics
- 21. System essentials Monitoring your system Monit (RubyWorks use runit) Monitoring process status Auto restart your important process Better than cron for monitoring Munin & Nagios Distributed monitoring all of your system Administrator’s eyes, developers friends
- 22. System essentials Munin & Nagios continues Munin has server and nodes, it generate sites to report the statistics of your server (in interval) Munin and Nagios and integrate Mem usage, CPU, process, disk usage Service: HTTP, SMTP, POP3, NNTP, Ping Hardware temperature and other datas Network statistics Custom scrips (plugins): db related, user number
- 23. Systemsystem ( Protect your essentialsManagement is important than tools) SSH brute attack protection ssh key login blockhost (scripts + pf/iptables) Audit: SELinux... Firewall (port block and audit) Use safe OS? (Netbsd, freebsd) Network safety (but no hardware firewall for websites)
- 24. System essentials SNA (Share Nothing Architecture) (This is relative term) All static file and rsync Database centric SNA Memcached + db-persistence Server hash, cluster, partition Amazon/Blogger/Cragslist/Facebook/Google /LiveJournal/Slashdot/Wikipedia/Yahoo/YouT ube Session sticky
- 25. System essentials Make your modules independent Layers, packages Easy to replace module Easy to deploy Easy to profile and make improves
- 26. Optimizations dynamic content Split your static content and server Use lightweight web server to server static contents Use different domain to different server Caching Memcached Query result, domain objects, sessions Page tiles, template tiles Everything that you need
- 27. Optimizations Caching Optimize your code (lazy evaluate, cache result) Cache and asynchronous update (cron update) Target, the hit rate of 90% or more ！ Target 90%+ But cache invalidation is a critical problem! Asynchronous messaging make sure cache validate No blocking!
- 28. Optimizations Caching Better client side caching Use expired header: max-age, expired E-tag? (Not recommended, IE doesn’t support it) Use HEAD method and 301 to detect changes (for squid or other proxy scenarios) Compress (contact js, css)
- 29. Optimizations SQL optimizations Add index (especially the column in where closure) De-normalized SQL Useful redundancy (use duplication avoid join) Don’t relay on ORM. No matter Data- mapper/Active Record/Unit Of Work Don’t use full-text search Use seperate search engine module (lucene)
- 30. Optimizations Choose proper database store engine Mysql: MyISAM? InnoDB? BDB? Heap? Accelerator PHP: APC, Zend Optimizer, XCache, eAccelerator, ionCube PHP Accelerator, Turck MMCache Python: psyco Ruby: Joyent accelerator
- 31. But most important thing: Find out the bottle neck before you start to optimize your application.
- 32. Next, Scaling, If time is enough
- 33. What is scaling? Three basics, 简单特性 : 能够使用率的提高 , Useable capacity increasing 能够容纳数据集提高， Data capacity increasing 系统可维护， Maintainable
- 34. Scaling, 2 ways Vertical Scaling Upgrade your hardware system More CPU, memory .... Horizontal Scaling Buy more same hardware, deploy more server instance Distributed your system But this way need you modify your code (generally)
- 35. Scaling-Load Balancing DNS-GSLB Use DNS’s round-robin algorithm randomize IP result xBayDNS Can’t deal with failure (TTL) Hard to do accurate management CDN content delivery network transparent service provide by some company expansive, and not suitable for dynamic content
- 36. Scaling-Load Balancing Hardware LB Citrix: Netscalers, Foundry: ServerIron, F5 (4- 7) Expensive Software LB Perlbal (4), Pound (7) LVS (4)
- 37. Scaling-Load Balancing Layer2, Layer4 and Layer7 LB Layer 2: Link aggregation, provide redundancy and fault tolerance, improve access speed Layer 4: round-robin on TCP (with port info) Layer 7 Session sticky enalbed Easy to write complicate hash logic Good for Squid (Squid cluster enabled)
- 38. Scaling-Load Balancing Huge Scale LB GSLB -> DNS round robin Virtual IP -> L4 or L7 LB (SNAT) Example Level 1 LB use GSLB give geo-located DNS result VIP is dispatched by F5 F5 -> Squid, reverse proxy Squid delegate real dynamic or static server
- 39. Scaling-Proxy Cache Reverse proxy Squid Use http head method to validate content Use memory to cache content - light speed Mature, fast, industry standard
- 40. Scaling-Database Scaling MySQL MySQL replication/duplication (Failure, Lag) Master/Slave Tree replication Data partition MySQL proxy Data shard
- 41. Scaling-File System Single Disk (Array) Raid 1, Raid 0, Raid5 Partition table type (GPT, MBR) Partition Format (ext2, ext3, resierfs, XFS, ZFS) Cluster Single Disk has limitation, but Cluster has no limit NetApp Filer (NAS - Network-attached storage) Many many choices
- 42. Scaling-File System Sharing Hardware based sharing NAS (previous page) NFS - most simple way to share FS Samba - almost same with NFS, nice to try MogileFS (for web, no cursor based random access) GFS, Hadoop FS (chunk based)
- 43. We are coming a long way, baby
- 44. Thanks!