Ad
More Related Content
What's hot (20)
Similar to Scaling real world applications using gevent (20)
![[오픈소스컨설팅] 프로메테우스 모니터링 살펴보고 구성하기](https://cdn.slidesharecdn.com/ss_thumbnails/oscprometheus-190422050231-thumbnail.jpg?width=560&fit=bounds)
Ad
Recently uploaded (20)
Ad
Scaling real world applications using gevent
- 1. Concurrency & Gevent Scaling Real World Applications
- 2. Concurrency handling a number of things at the same time Examples: (Incoming) WebServers, Database Servers (Outgoing) SSH Mux
- 3. SSH Mux ● Execute a command on a remote SSH server ● Handle concurrent SSH Clients ● Command execution time varies from seconds to days ● Command execution happens on remote servers, SSH mux is I/O bound
- 4. SSH Client 1. Init session 2. Authenticate 3. Get a channel 4. Issue command 5. Read output
- 5. Need Concurrency? ● Process blocks on read() ● No new connections can be inititated ● Need ability to handle multiple clients at the same time
- 6. Multiprocessing ● One process is the master ● Master can spawn workers ● Each worker handles one request at a time ● Pre-forked pool of workers
- 8. SSH Mux Memory Usage ssh mux memory usage 600 500 400 Memory (MB) 300 Processes 200 100 0 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 # Concurrent Reqs
- 9. SSH Mux Performace ssh mux performance sheet 160 140 120 100 Time(s) 80 Processes 60 40 20 0 20 40 80 150 200 400 800 1200 1500 1800 2100 # Concurrent Reqs
- 10. Multiprocessing yay ● Easy to get started ● OS guaranteed process isolation & fairness ● Covers up for misbehaving workers ● Add more concurrency by adding more workers ● Convenient when numbers are smaller numbers
- 11. Multiprocessing nay ● Concurrency limited by number of processes ● Memory heavy ● Implicit scheduling ● Synchronization is not trivial
- 12. More Concurrency? ● Command execution is happening on remote servers, we are mostly blocked on I/O ● Handle multiple I/O in a single process?
- 13. Gevent gevent is a coroutine-based Python networking library that uses greenlet to provide a high-level synchronous API on top of the libevent event loop.
- 14. Greenlets ● Lightweight 'threads' - not OS threads ● Explicit scheduling - Cooperative ● Minimal stack ● Application decides execution flow ● Easy to synchronize/ Avoid locks ● All run inside one process
- 15. Libevent ● Use fastest mechanism to poll (portable) ● Fast Event loop ● In Gevent, event loop runs in a greenlet (event hub) ● Instead of blocking, greenlets switch to event hub ● It's all done behind the scene
- 16. Monkey Patching Monkey patching Modifies behaviour of blocking calls such as select, sleep to non-blocking Patches the python standard socket library
- 17. Gevent ● Greenlet 1 is running ● Greenlet 2 and 3 are ready
- 18. Gevent ● Greenlet 1 has to wait for read ● Greenlet 1 switches to Event hub
- 19. Gevent ● Event hub switches to Greenlet 3
- 20. Gevent ● Greenlet 2 runs
- 21. Gevent ● Greenlet 2 wants to sleep ● Greenlet 2 switches to Event hub
- 22. Gevent ● Greenlet 1 data has come, moved to ready state ● Eventhub switches to Greenlet 3
- 23. Gevent ● Greenlet 3 runs
- 24. Gevent ● When Greenlet 1 resumes, its from next instruction ● It's as if it were a blocking call
- 25. Green SSH Client 1. Init session 2. Authenticate 3. Get a channel 4. Issue command 5. Read output
- 26. A closer look
- 27. Going Concurrent Use pre-forked processes to use all cores
- 28. Memory usage ssh mux memory usage 45 40 35 30 Memory(MB) 25 Gevent+Processes 20 15 10 5 0 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 # Concurrent Reqs
- 29. SSH Mux Performace ssh mux performace chart 70 60 50 40 Time(s) Gevent+Processes 30 20 10 0 20 40 80 150 200 400 800 1200 1500 1800 2100 # Concurrent Reqs
- 30. SSH Mux memory usage ssh mux memory usage 600 500 400 Memory(MB) Processes 300 Gevent 200 100 0 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 # Concurrent Reqs
- 31. SSH Mux Performance ssh mux performance sheet 160 140 120 100 Processes Time(s) 80 Gevent+Processes 60 40 20 0 20 40 80 150 200 400 800 1200 1500 1800 2100 # Concurrent Reqs
- 32. Gevent yay! ● Untwist – write linear non blocking code ● Explicit scheduling, dictate the execution flow ● Timeouts ● Events, AsyncResults for Synchronization ● gevent.wsgi ● Pre-spawned pool of greenlets
- 33. Gevent beware of ● No multicore support ● Not great for CPU bound applications ● Third party libs must be green (non blocking) ● Misbehaving workers can be lethal ● No fairness when it comes to scheduling
- 34. Take Away ● Gevent lets you write asynchronous code in a synchronous manner ● No multicore support, still need multiprocessing ● Not so great for CPU bound applications ● Split your application into CPU bound and IO bound parts ● Be willing to contribute patches ● Code available at [email protected]:aaloksood/pyexamples.git
- 35. Thank you That's all folks!
- 36. Countdown Timer ● Count down from 200000000 ● Split work among workers
- 37. Threads Multithreading wonder 25 20 15 1 Core Time(s) 4 cores 10 5 0 1 2 3 4 5 6 7 8 9 10 # Workers
- 38. One core Execution time One Core 14.5 14 13.5 13 Processes 1 Core Time (s) 12.5 Gevent_1 12 Gevent_4 11.5 11 10.5 1 2 3 4 5 6 7 8 9 10 # Workers
- 39. Four cores Execution time 4 cores 25 20 Process 15 Threads Time(s) Gevent_1 10 Gevent_4 5 0 1 2 3 4 5 6 7 8 9 10 # Workers