Do more than one thing at the same time, the Python way

5 likes6,227 views

The document discusses doing more than one thing at a time in Python using threads and processes. It describes how to create threads using the threading module and processes using the multiprocessing module. While threads are easier to use, the Global Interpreter Lock (GIL) in Python prevents true parallelism. Processes can better utilize multiple CPUs but require more work for communication. Asynchronous programming is recommended for I/O-bound tasks while processes are better for CPU-bound work. The talk cautions that threading should be used carefully in Python due to the GIL.

Technology

DO MORE THAN ONE
THING AT THE TIME
the Python way!

Jaime Buelta

SLICE A PROBLEM TO
SOLVE IT USING MORE
RESOURCES

THREADS IN PYTHON

module threading

module thread

THREAD EXAMPLE
import threading
ITERATIONS = 1000000

class Example1(threading.Thread):

def __init__(self, num):
self.num = num
super(Example1, self).__init__()

def run(self):

for i in xrange(ITERATIONS):
pass

def main():
for j in xrange(10):
t = Example1(j)
t.start()

if __name__ == '__main__':
main()

TIMERS
from threading import Timer
DELAYED_TIME = 10.5

def delayed():
print 'This call is delayed'

t = Timer(10.5, delayed)
t.start()

t.cancel() # Cancels the execution

MULTIPROCESS MODULE
import multiprocessing
ITERATIONS = 1000000

class Example1(multiprocessing.Process):

def __init__(self, num):
self.num = num
super(Example1, self).__init__()

def run(self):

for i in xrange(ITERATIONS):
pass

def main():
for j in xrange(10):
t = Example1(j)
t.start()

if __name__ == '__main__':
main()

PROCESS COMMUNICATION
NEEDS TO BE STRUCTURED

but that is not necessarily a bad thing

Do more than one thing at the same time, the Python way

Task 1 Task 2 Task 3

The task will release
waiting control once they are
ready! blocked waiting for an
done! input from IO
waiting
ready! Callback

done!
done!

EVENTLET
NUM_URLS = 1000
URL = 'http://www.some_address.com/'

urls = [URL] * NUM_URLS

import eventlet
from eventlet.green import urllib2

def fetch(url):
return urllib2.urlopen(url).read()

pool = eventlet.GreenPool()
for body in pool.imap(fetch, urls):
do_something_with_result(body)

Asynchronous
programing is great
when the tasks are
IO - Bound

So the CPU is
basically waiting...

Asynchronous
programing is not
good when tasks are
CPU - Bound

If one tasks enters on
an inﬁnite loop, the
whole system is
blocked

YESTERDAY THERE WAS A
TALK ABOUT ASYNC PYTHON
PROGRAMMING
Hope you attended, I did.
If you don’t, you can watch it online later

It doesn’t allow to run two threads at the same time, even if the
OS will do it.

Only one thread run. The rest will be blocked.

100,000,000 iterations

40 s

30 s

20 s

10 s

0s
1 10 100 1000 10000

4 core machine

I WANT TO WATCH THIS YOUTUBE

BUT I’M ALREADY LISTENING TO MUSIC
AT THE SAME TIME

EEH, NOT AS BIG AS IT LOOKS

GIL MAKES CONCURRENT
PROGRAMMING MUCH EASIER

And the problems are quite limited in practice

BUT MAYBE YOUR PROGRAM
IS ONE OF THE FEW

Avoid problems not using threads, but processes

threading multiprocess sequential

100,000ms

10,000ms

1,000ms

100ms

10ms

1ms
1000 10000 100000 1000000 10000000

Great, detailed talk
“Understanding GIL”
by David Beazly
http://www.dabeaz.com/python/UnderstandingGIL.pdf

ALL PYTHON OPERATIONS
ARE ATOMIC
Hey, that’s what the GIL is for

LOCKING TO
BE USED WITH
EXTREME
CAUTION
If you need to set exclusive
sections, you are probably
doing it wrong

BUT WHEN I
DO, I USE WITH

from threading import Lock
my_lock = Lock()

def some_function(args):

with my_lock:
protected_section()

Main periodic thread

task A

task B

task C

task D

Main periodic thread

Main periodic thread

but that’s probably not the best use of Python

QUESTIONS?
THANKS
FOR
YOUR
ATTENTION
@jaimebuelta
wrongsideofmemphis.wordpress.com

Do more than one thing at the same time, the Python way

1. DO MORE THAN ONE THING AT THE TIME the Python way! Jaime Buelta

2. WHY?

3. SLICE A PROBLEM TO SOLVE IT USING MORE RESOURCES

4. SAME THING FOR DIFFERENT ACTORS

5. DIFFERENT THINGS FOR THE SAME ACTOR

6. DOING MORE THAN ONE THING IS TOUGH

7. CHOOSE WISELY

8. THREADS

9. THREADS IN PYTHON module threading module thread

10. THREAD EXAMPLE import threading ITERATIONS = 1000000 class Example1(threading.Thread): def __init__(self, num): self.num = num super(Example1, self).__init__() def run(self): for i in xrange(ITERATIONS): pass def main(): for j in xrange(10): t = Example1(j) t.start() if __name__ == '__main__': main()

11. TIMERS from threading import Timer DELAYED_TIME = 10.5 def delayed(): print 'This call is delayed' t = Timer(10.5, delayed) t.start() t.cancel() # Cancels the execution

12. PROCESSES

13. YE OLDE FORK

14. MULTIPROCESS MODULE import multiprocessing ITERATIONS = 1000000 class Example1(multiprocessing.Process): def __init__(self, num): self.num = num super(Example1, self).__init__() def run(self): for i in xrange(ITERATIONS): pass def main(): for j in xrange(10): t = Example1(j) t.start() if __name__ == '__main__': main()

15. OS ARE GREAT AT MULTITASKING

16. PROCESS COMMUNICATION NEEDS TO BE STRUCTURED but that is not necessarily a bad thing

17. ASYNCHRONOUS PROGRAMMING

18. Thread 1 Thread 2

20. Task 1 Task 2 Task 3 The task will release waiting control once they are ready! blocked waiting for an done! input from IO waiting ready! Callback done! done!

21. death by callback

22. EVENTLET NUM_URLS = 1000 URL = 'http://www.some_address.com/' urls = [URL] * NUM_URLS import eventlet from eventlet.green import urllib2 def fetch(url): return urllib2.urlopen(url).read() pool = eventlet.GreenPool() for body in pool.imap(fetch, urls): do_something_with_result(body)

23. Asynchronous programing is great when the tasks are IO - Bound So the CPU is basically waiting...

24. Asynchronous programing is not good when tasks are CPU - Bound If one tasks enters on an inﬁnite loop, the whole system is blocked

25. YESTERDAY THERE WAS A TALK ABOUT ASYNC PYTHON PROGRAMMING Hope you attended, I did. If you don’t, you can watch it online later

26. THE INFAMOUS GIL

27. It doesn’t allow to run two threads at the same time, even if the OS will do it. Only one thread run. The rest will be blocked.

28. 2 core machine Thread A Thread B

29. 100,000,000 iterations 40 s 30 s 20 s 10 s 0s 1 10 100 1000 10000 4 core machine

30. IS IT REALLY A PROBLEM?

31. I WANT TO WATCH THIS YOUTUBE BUT I’M ALREADY LISTENING TO MUSIC AT THE SAME TIME EEH, NOT AS BIG AS IT LOOKS

32. GIL MAKES CONCURRENT PROGRAMMING MUCH EASIER And the problems are quite limited in practice

33. BUT MAYBE YOUR PROGRAM IS ONE OF THE FEW Avoid problems not using threads, but processes

34. threading multiprocess sequential 100,000ms 10,000ms 1,000ms 100ms 10ms 1ms 1000 10000 100000 1000000 10000000

35. Great, detailed talk “Understanding GIL” by David Beazly http://www.dabeaz.com/python/UnderstandingGIL.pdf

36. AVOID PROBLEMS

37. SIMPLE ARCHITECTURE

38. ALL PYTHON OPERATIONS ARE ATOMIC Hey, that’s what the GIL is for

39. LOCKING TO BE USED WITH EXTREME CAUTION If you need to set exclusive sections, you are probably doing it wrong

40. BUT WHEN I DO, I USE WITH from threading import Lock my_lock = Lock() def some_function(args): with my_lock: protected_section()

41. USE QUEUES (AND PIPES)

42. PROCESS THE TASK WITH WORKERS

43. LIMIT THE NUMBERS!!!

44. THREAD COORDINATION IS HELL

45. Main periodic thread task A task B task C task D Main periodic thread Main periodic thread

46. but that’s probably not the best use of Python

47. QUESTIONS? THANKS FOR YOUR ATTENTION @jaimebuelta wrongsideofmemphis.wordpress.com

Editor's Notes

#2: \n
#3: The first point is, why complicate the code trying to do more than one thing at the same time. The answer is to better use the resources. We need spare resources to start with\n\nThe typical applications that need these kind of techniques are three (divide a big tasks into several computers/cores, do the same thing for different actors, do different things for the same actor)\n
#4: The typical example is crunching numbers. Render a movie, data mining, etc. Usually can be treated as the next case (do the same thing more than once) with an algorithm that works with parts of the data.\n
#5: Do the same thing, for different actors\n Web server\n Different tabs on a web browser\n
#6: Do different things for the same actor (usually some coordination is needed, more later):\n Game (AI of each enemy, render image, sound)\n
#7: Debug can be very tricky, you&#x2019;ve been advised. If coordination is needed, that can be tricky. If only one thing is do (several times), the complexity is highly reduced due help from the Operative System\n
#8: There are several ways of execute code concurrently, we&#x2019;ll discuss threads, processes and asynchronous programming\n
#9: \n
#10: module thread\n Low level, in general is better avoid it\n module threading\n Higher level. More functionality\n
#11: \n
#12: \n
#13: \n
#14: A long, long time ago, that was the way of dealing with multiprocessing\nOnly available on Unix machines\nYou can fork using os.fork() in Python!\nFork is not in common use anymore due problems (fork bomb, etc)\n
#15: \n
#16: OS are very good at multitasking (and have been for some time). Communication can be achieved through ports, files, pipes, external queues, etc...\nBe radical, use more than one program\n
#17: Threads share all the memory, so they can access whatever. That is good (no communication overhead), but can lead to abuse and instability. \n
#18: it is the new cool technology, and there has been some recent uses, like NodeJS\n
#19: On a thread model, you have a supervisor (OS) that will stop one thread and execute the other, transparently for the program\n
#20: On asynchronous programing, every task is waiting for the others to end their dance to get on the dance floor\n
#21: Or voluntarily releasing control (sleep, yield, etc). The typical way of calling the next block is to add callbacks to the code (when this result is available, use it as input for this code).\n
#22: Callbacks make the code hard to follow and debug\n
#23: No threads! All the fetch functions start, there is no need for callbacks, the urlopen call will yield and resume at the same point\n
#24: The typical example is a web app, which normally takes time in get data from a DB and spend a small amount of time in composing the response.\nYou can keep a huge number of connections, like in chat servers, etc.\n
#25: Number crunching, 3d calculations, etc... In a threaded model, the rest of the thread will be executed from time to time.\n
#26: Use of Twisted and Tornado a some Node.js\n
#27: \n
#28: This will only happen on multicore processors.There is some overhead on the blocking. It also make UNIX signals to act weird (CTRL + C example)\n
#29: The OS could (and probably do, depending on the load) set a thread to run on each core.\n
#30: Num iterations is constant, with different threads. Increasing the number of threads magnifies the problem, as adds more overhead. Adding threads adds MORE time instead of reducing it.\n
#31: \n
#32: it is a very &#x201C;first world problem&#x201D;. It is only causing problems on a subset of programs, in general the effects are limited.\n
#33: Only if CPU-bound, multithread program is running on a multicore machine, you need to worry.\nEven one CPU bound thread can have an effect due overhead and could be more efficient to use sequential programming\n\n
#34: \n
#35: time vs iterations for 10 threads/processes\n
#36: He has a couple more articles on his web page www.dabeaz.com\n
#37: In general, for concurrent programming\n
#38: probably true for all software development\n\nkeep the tasks separated and reduce communication between elements\n\n\n
#39: That also includes C extensions. You don&#x2019;t need a lock for reads, only for writes that are not a single python operation.\n
#40: Python should be enough for most of the situations. Think carefully if you need a lock, mutex or semaphore\n\n\n
#41: \n
#42: There are different kind of Queues, like LIFO, FIFO, Priority, etc. multiprocess has also queues.\nExternal queues can also be useful (RabbitMQ, AMQP, etc)\nA pipe is a queue with one input and one output.\n
#43: \n
#44: Limit the number of workers. Unlimited workers is unsafe and can block the system.\n\nA lot of threads can be worse than less threads. Do some tests to find the sweet spo\n
#45: Thread coordination is very difficult. Try to create and destroy short-lived threads instead.\n
#46: This structure is typical of games (and real time systems)\n\nEach interval (1 second), you evaluate the needs and throw the needed threads (or execute sequentially)\nThe main periodic thread can cancel next time threads that haven&#x2019;t finished their task.\n\nKill unused threads\n
#47: So consider using other languages...\n
#48: \n

Do more than one thing at the same time, the Python way

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Do more than one thing at the same time, the Python way (20)

Recently uploaded (20)

Do more than one thing at the same time, the Python way

Editor's Notes