Python High Performance 2nd Edition Gabriele Lanaro - Read the ebook online or download it to own the full content
Python High Performance 2nd Edition Gabriele Lanaro - Read the ebook online or download it to own the full content
com
https://ebookname.com/product/python-high-performance-2nd-
edition-gabriele-lanaro/
OR CLICK HERE
DOWLOAD EBOOK
https://ebookname.com/product/python-high-performance-
programming-boost-the-performance-of-your-python-programs-using-
advanced-techniques-1st-edition-gabriele-lanaro/
https://ebookname.com/product/high-performance-pigments-2nd-
edition-faulkner/
https://ebookname.com/product/typescript-high-performance-1st-
edition-ajinkya-kher/
https://ebookname.com/product/wavelet-and-wave-analysis-as-
applied-to-materials-with-micro-or-nanostructure-carlo-cattani/
Science and Scepticism John W.N. Watkins
https://ebookname.com/product/science-and-scepticism-john-w-n-
watkins/
https://ebookname.com/product/body-transformation-manual-the-
ultimate-12-week-workout-plan-suitable-for-women-and-men-sean-
lerwill/
https://ebookname.com/product/dandelion-fire-n-d-wilson/
https://ebookname.com/product/a-history-of-the-old-icelandic-
commonwealth-islendinga-saga-u-of-m-icelandic-series-jon-
johannesson/
https://ebookname.com/product/physiology-a-clinical-core-text-of-
human-physiology-with-self-assessment-3rd-edition-j-g-mcgeown/
Python High Performance
Second Edition
Gabriele Lanaro
BIRMINGHAM - MUMBAI
Python High Performance
Second Edition
Copyright © 2017 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or
transmitted in any form or by any means, without the prior written permission of the
publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the
information presented. However, the information contained in this book is sold without
warranty, either express or implied. Neither the author, nor Packt Publishing, and its
dealers and distributors will be held liable for any damages caused or alleged to be caused
directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the
companies and products mentioned in this book by the appropriate use of capitals.
However, Packt Publishing cannot guarantee the accuracy of this information.
I'd like to acknowledge the support from Packt editors, including Vikas Tiwari. I would also
like to thank my girlfriend, Harani, who had to tolerate the way-too-long writing nights,
and friends who provided company and support throughout. Also, as always, I’d love to
thank my parents for giving me the opportunity to pursue my ambitions.
Lastly, I would like to thank Blenz coffee for powering the execution engine of this book
through electricity and caffeine.
About the Reviewer
Will Brennan is a C++/Python developer based in London with previous experience in
writing molecular dynamics simulations. He is currently working on high-performance
image processing and machine learning applications. You can refer to his repositories at htt
ps://github.com/WillBrennan.
www.PacktPub.com
For support files and downloads related to your book, please visit www.PacktPub.com.
Did you know that Packt offers eBook versions of every book published, with PDF and
ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a
print book customer, you are entitled to a discount on the eBook copy. Get in touch with us
at [email protected] for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a
range of free newsletters and receive exclusive discounts and offers on Packt books and
eBooks.
https://www.packtpub.com/mapt
Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt
books and video courses, as well as industry-leading tools to help you plan your personal
development and advance your career.
Why subscribe?
Fully searchable across every book published by Packt
Copy and paste, print, and bookmark content
On demand and accessible via a web browser
Customer Feedback
Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial
process. To help us improve, please leave us an honest review on this book's Amazon page
at https://www.amazon.com/dp/1787282899.
If you'd like to join our team of regular reviewers, you can e-mail us at
[email protected]. We award our regular reviewers with free eBooks and
videos in exchange for their valuable feedback. Help us be relentless in improving our
products!
Table of Contents
Preface 1
Chapter 1: Benchmarking and Profiling 7
Designing your application 8
Writing tests and benchmarks 14
Timing your benchmark 16
Better tests and benchmarks with pytest-benchmark 19
Finding bottlenecks with cProfile 22
Profile line by line with line_profiler 27
Optimizing our code 29
The dis module 31
Profiling memory usage with memory_profiler 32
Summary 35
Chapter 2: Pure Python Optimizations 36
Useful algorithms and data structures 37
Lists and deques 38
Dictionaries 40
Building an in-memory search index using a hash map 42
Sets 44
Heaps 45
Tries 47
Caching and memoization 49
Joblib 52
Comprehensions and generators 53
Summary 55
Chapter 3: Fast Array Operations with NumPy and Pandas 56
Getting started with NumPy 57
Creating arrays 57
Accessing arrays 59
Broadcasting 64
Mathematical operations 67
Calculating the norm 68
Rewriting the particle simulator in NumPy 68
Reaching optimal performance with numexpr 72
Pandas 74
Pandas fundamentals 74
Indexing Series and DataFrame objects 76
Database-style operations with Pandas 78
Mapping 79
Grouping, aggregations, and transforms 81
Joining 83
Summary 84
Chapter 4: C Performance with Cython 86
Compiling Cython extensions 86
Adding static types 89
Variables 89
Functions 91
Classes 92
Sharing declarations 94
Working with arrays 95
C arrays and pointers 95
NumPy arrays 98
Typed memoryviews 99
Particle simulator in Cython 101
Profiling Cython 105
Using Cython with Jupyter 109
Summary 112
Chapter 5: Exploring Compilers 114
Numba 115
First steps with Numba 115
Type specializations 117
Object mode versus native mode 118
Numba and NumPy 121
Universal functions with Numba 121
Generalized universal functions 123
JIT classes 126
Limitations in Numba 129
The PyPy project 130
Setting up PyPy 131
Running a particle simulator in PyPy 132
Other interesting projects 133
Summary 134
Chapter 6: Implementing Concurrency 135
[ ii ]
Asynchronous programming 136
Waiting for I/O 136
Concurrency 137
Callbacks 139
Futures 142
Event loops 144
The asyncio framework 147
Coroutines 148
Converting blocking code into non-blocking code 152
Reactive programming 154
Observables 154
Useful operators 157
Hot and cold observables 161
Building a CPU monitor 164
Summary 167
Chapter 7: Parallel Processing 168
Introduction to parallel programming 169
Graphic processing units 171
Using multiple processes 172
The Process and Pool classes 173
The Executor interface 175
Monte Carlo approximation of pi 176
Synchronization and locks 179
Parallel Cython with OpenMP 182
Automatic parallelism 184
Getting started with Theano 185
Profiling Theano 190
Tensorflow 192
Running code on a GPU 194
Summary 198
Chapter 8: Distributed Processing 199
Introduction to distributed computing 199
An introduction to MapReduce 200
Dask 203
Directed Acyclic Graphs 204
Dask arrays 205
Dask Bag and DataFrame 208
Dask distributed 212
Manual cluster setup 214
[ iii ]
Using PySpark 216
Setting up Spark and PySpark 217
Spark architecture 218
Resilient Distributed Datasets 220
Spark DataFrame 225
Scientific computing with mpi4py 227
Summary 230
Chapter 9: Designing for High Performance 231
Choosing a suitable strategy 231
Generic applications 232
Numerical code 233
Big data 235
Organizing your source code 236
Isolation, virtual environments, and containers 238
Using conda environments 238
Virtualization and Containers 240
Creating docker images 243
Continuous integration 245
Summary 246
Index 248
[ iv ]
Preface
The Python programming language has seen a huge surge in popularity in recent years,
thanks to its intuitive, fun syntax, and its vast array of top-quality third-party libraries.
Python has been the language of choice for many introductory and advanced university
courses as well as for numerically intense fields, such as the sciences and engineering. Its
primary applications also lies in machine learning, system scripting, and web applications.
Over the years, many strategies have been developed to overcome CPython's performance
shortcomings. This book aims to fill this gap and will teach how to consistently achieve
strong performance out of your Python programs.
This book will appeal to a broad audience as it covers both the optimization of numerical
and scientific codes as well as strategies to improve the response times of web services and
applications.
The book can be read cover-to-cover ; however, chapters are designed to be self-contained
so that you can skip to a section of interest if you are already familiar with the previous
topics.
Chapter 2, Pure Python Optimizations, discusses how to improve your running times by
order of magnitudes using the efficient data structures and algorithms available in the
Python standard library and pure-Python third-party modules.
Chapter 3, Fast Array Operations with NumPy and Pandas, is a guide to the NumPy and
Pandas packages. Mastery of these packages will allow you to implement fast numerical
algorithms with an expressive, concise interface.
Chapter 5, Exploring Compilers, covers tools that can be used to compile Python to efficient
machine code. The chapter will teach you how to use Numba, an optimizing compiler for
Python functions, and PyPy, an alternative interpreter that can execute and optimize
Python programs on the fly.
Chapter 8, Distributed Processing, extends the content of the preceding chapter by focusing
on running parallel algorithms on distributed systems for large-scale problems and big
data. This chapter will cover the Dask, PySpark, and mpi4py libraries.
Chapter 9, Designing for High Performance, discusses general optimization strategies and
best practices to develop, test, and deploy your high-performance Python applications.
The recommended way to install Python and the associated libraries is through the
Anaconda distribution, which can be downloaded from https://www.continuum.io/downl
oads, for Linux, Windows, and Mac OS X.
[2]
Preface
Conventions
In this book, you will find a number of text styles that distinguish between different kinds
of information. Here are some examples of these styles and an explanation of their meaning.
Code words in text, database table names, folder names, filenames, file extensions,
pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "To
summarize, we will implement a method called ParticleSimulator.evolve_numpy and
benchmark it against the pure Python version, renamed as
ParticleSimulator.evolve_python"
inputs = [0, 1, 2, 3, 4]
outputs = pool.map(square, inputs)
When we wish to draw your attention to a particular part of a code block, the relevant lines
or items are set in bold:
def square(x):
return x * x
inputs = [0, 1, 2, 3, 4]
outputs = pool.map(square, inputs)
New terms and important words are shown in bold. Words that you see on the screen, for
example, in menus or dialog boxes, appear in the text like this: "On the right, clicking on the
tab Callee Map will display a diagram of the function costs."
[3]
Other documents randomly have
different content
En supposant les appointements à 500 francs par mois c'est 25
francs que l'agent touche tous les mois.
Aussitôt l'engagement signé l'artiste touche un mois d'avance par
l'entremise de l'agent qui retient sa commission.
L'artiste part, débute, est sifflé, il revient chez le même agent qui
l'engage pour une autre ville toujours moyennant la même
commission.
Il y a des farceurs qui se font ainsi 6,000 francs de rentes en se
faisant siffler partout. Quand ils ont fini en France ils vont se faire
siffler à l'étranger, c'est plus difficile, mais ils y mettent tant de
bonne volonté!
Le côté des dames n'est pas beaucoup plus favorisé, mais les
femmes ont une manière à elles de porter la pauvreté qui enlève à
ce vice une grande partie de l'horreur qu'il inspire aux mauvais
cœurs.
L'ancienne comédienne aux airs évaporés, la bonne fille qui allait
jadis demander à Toulouse ou à Bordeaux les bravos que Paris lui
refusait, n'existe plus.
Le théâtre en province est alimenté régulièrement.
Les étoiles vieillies au boulevard n'ont que deux partis à prendre,
devenir duègnes à Paris ou aller en province jouir d'un printemps
éternel. Il est rare qu'elles ne prennent pas ce dernier parti.
Quelques jeunes filles du Conservatoire ou d'ailleurs vont faire
assez volontiers une saison dans une grande ville, afin de s'habituer
à la scène, et d'acquérir le pied marin.
Elles reviennent sans avoir acquis autre chose que les mauvaises
habitudes passées à l'état de tradition.
Pour le reste il est à peu près inutile d'en parler. Ce reste se
compose de choristes ou de coryphées des théâtres de la capitale,
braves filles dévorées du désir de devenir aussi des étoiles.
Elles ont chanté deux cents fois les chœurs de la Grande-
Duchesse ou de la Timbale d'argent et elles arrivent à imiter
madame Schneider ou Judic avec une perfection bien capable
d'illusionner Castelnaudary ou Lons-le-saunier.
Où leur embarras commence, c'est lorsqu'il faut créer un nouveau
rôle, Castelnaudary ne rit plus.
Pourtant on a vu quelques-unes de ces échappées de la troupe de
fer-blanc gagner quelque talent et devenir passables.
Après elles, il n'y a plus que des pauvres filles qui sont là comme
elles seraient ailleurs, parce que c'est leur destinée.
Pendant que toutes les autres rêvent de revenir à Paris sur un vrai
théâtre, pour un vrai rôle, celles-ci rêvent le théâtre d'Alger, parce
qu'en Afrique les officiers sont nombreux et forment un très bon
public.
L'ÉDUCATION D'UN VICOMTE
Un pauvre diable de licencié se présente chez un gentilhomme fort
riche qui a demandé, par la voie de la publicité, un précepteur pour
son fils, âgé de douze ans.
—Mon gaillard est un peu en retard, dit le gentilhomme.
—Nous rattraperons vite le temps perdu, monsieur le comte,
surtout si le sujet est intelligent.
—Pourquoi ne serait-il pas intelligent? s'écrie le comte en se
redressant.
—C'est ce que je me demandais, fait humblement le précepteur.
—Qu'est-ce que vous allez apprendre à mon drôle?
—Mais, monsieur le comte, cela dépend de vos intentions.
—Je n'en ai pas.
—Vous désirez sans doute que M. votre fils soit bachelier ès
lettres?
—Oh! mon Dieu, pas absolument.
—Bachelier ès-sciences?
—Ah! du tout! Je veux que mon fils sache tout simplement ce qui
est nécessaire à un homme du monde qui a un beau nom et qui
aura un jour trois cent mille francs de rentes.
—Avec trois cent mille francs de rentes, on peut se passer de bien
des choses, monsieur le comte.
—C'est assez mon avis.
—Un peu de latin?
—Beaucoup de latin; le Saint-Père aime notre famille.
—Un peu de grec?
—Beaucoup de grec; j'ai un oncle à succession qui est helléniste
en diable.
—Des langues vivantes?
—Toutes; la comtesse veut que son fils traverse les légations.
—La littérature me paraît d'une nécessité absolue.
—Dites les littératures.
—Quant aux mathématiques.....
—Cela va sans dire; un homme du monde qui ne sait pas compter
est un bien triste sire, monsieur le professeur.
—C'est bien mon avis.
—Il serait possible, d'ailleurs, que mon gaillard ait un jour l'envie
de passer par Saint-Cyr, c'est une maladie de famille.
—En ce cas, il faudrait soigner la géométrie et l'algèbre.
—Naturellement.
—On pourrait effleurer la chimie, la physique et l'astronomie?
—Vous oubliez le dessin.
—Je le réservais.
—Vous n'aurez à vous occuper ni de la musique, ni de la danse, ni
de l'escrime.
—C'est heureux, car je vous avoue, monsieur le comte, que je suis
assez peu entendu dans ces matières.
—A propos, savez-vous la gymnastique?
—Théoriquement.
—Ça ne suffit pas; mais peu importe: je passerai là-dessus parce
que vous me convenez beaucoup.
—Monsieur le comte me comble.
—Vous connaissez les conditions?
—Votre intendant m'en a parlé.
—Elles vous conviennent?
—Mon Dieu, oui.
FIGURES CONTEMPORAINES
LOUIS PHILIPPE ET MARIE AMÉLIE
Le roi Louis-Philippe arrivait avec une tout autre politique que celle
du droit divin. Il pensa, non sans raison, qu'il deviendrait populaire
en se faisant bourgeois, et, pour ce faire, il n'hésita pas à couvrir sa
majesté d'une redingote à la propriétaire.
Tout s'enchaîne; le salut et la discrétion respectueuse se
changèrent en poignées de mains.
—Bonjour, monsieur le roi, comment vous portez-vous?
Et le roi répondait en pressant toutes les mains prolétaires qui se
tendaient vers lui.
—Bien, mes bons amis, très bien.
Et il causait avec Dubois, Durand ou Lefèvre, de pair à
compagnon, s'informant de leur famille, et de leurs affaires et de
leurs affections.
Pauvre roi! prince vertueux, comme il fut bien payé de tant de
bonne grâce par ces bourgeois si fiers de lui toucher la main!
Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.
ebookname.com