Learning Concurrency in Python: Build highly efficient, robust, and concurrent applications

352 Pages·2017·2.356 MB·English

by  Elliot Forbes

Learning Concurrency in
Python
(cid:52)(cid:81)(cid:70)(cid:70)(cid:69)(cid:2)(cid:86)(cid:81)(cid:2)(cid:90)(cid:80)(cid:86)(cid:83)(cid:2)(cid:49)(cid:90)(cid:85)(cid:73)(cid:80)(cid:79)(cid:2)(cid:68)(cid:80)(cid:69)(cid:70)(cid:2)(cid:88)(cid:74)(cid:85)(cid:73)(cid:2)(cid:68)(cid:77)(cid:70)(cid:66)(cid:79)(cid:14)(cid:2)(cid:83)(cid:70)(cid:66)(cid:69)(cid:66)(cid:67)(cid:77)(cid:70)(cid:14)(cid:2)(cid:66)(cid:79)(cid:69)
(cid:66)(cid:69)(cid:87)(cid:66)(cid:79)(cid:68)(cid:70)(cid:69)(cid:2)(cid:68)(cid:80)(cid:79)(cid:68)(cid:86)(cid:83)(cid:83)(cid:70)(cid:79)(cid:68)(cid:90)(cid:2)(cid:85)(cid:70)(cid:68)(cid:73)(cid:79)(cid:74)(cid:82)(cid:86)(cid:70)(cid:84)
Elliot Forbes
BIRMINGHAM - MUMBAI
Learning Concurrency in Python
Copyright © 2017 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or
transmitted in any form or by any means, without the prior written permission of the
publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the
information presented. However, the information contained in this book is sold without
warranty, either express or implied. Neither the author, nor Packt Publishing, and its
dealers and distributors will be held liable for any damages caused or alleged to be caused
directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the
companies and products mentioned in this book by the appropriate use of capitals.
However, Packt Publishing cannot guarantee the accuracy of this information.
First published: August 2017
Production reference: 1140817
(cid:49)(cid:86)(cid:67)(cid:77)(cid:74)(cid:84)(cid:73)(cid:70)(cid:69)(cid:2)(cid:67)(cid:90)(cid:2)(cid:49)(cid:66)(cid:68)(cid:76)(cid:85)(cid:2)(cid:49)(cid:86)(cid:67)(cid:77)(cid:74)(cid:84)(cid:73)(cid:74)(cid:79)(cid:72)(cid:2)(cid:45)(cid:85)(cid:69)(cid:16)
(cid:45)(cid:74)(cid:87)(cid:70)(cid:83)(cid:90)(cid:2)(cid:49)(cid:77)(cid:66)(cid:68)(cid:70)
(cid:21)(cid:23)(cid:2)(cid:45)(cid:74)(cid:87)(cid:70)(cid:83)(cid:90)(cid:2)(cid:52)(cid:85)(cid:83)(cid:70)(cid:70)(cid:85)
(cid:35)(cid:74)(cid:83)(cid:78)(cid:74)(cid:79)(cid:72)(cid:73)(cid:66)(cid:78)
(cid:35)(cid:21)(cid:2)(cid:20)(cid:49)(cid:35)(cid:14)(cid:2)(cid:54)(cid:44)(cid:16)
ISBN 978-1-78728-537-8
(cid:88)(cid:88)(cid:88)(cid:16)(cid:81)(cid:66)(cid:68)(cid:76)(cid:85)(cid:81)(cid:86)(cid:67)(cid:16)(cid:68)(cid:80)(cid:78)
Credits
Author Copy Editor
Elliot Forbes Sonia Mathur
Reviewer Project Coordinator
Nikolaus Gradwohl Vaidehi Sawant
Commissioning Editor Proofreader
Merint Mathew Safis Editing
Acquisition Editor Indexer
Chaitanya Nair Francy Puthiry
Content Development Editor Graphics
Rohit Kumar Singh Abhinash Sahu
Technical Editors Production Coordinator
Ketan Kamble Nilesh Mohite
About the Author
Elliot Forbes he worked as a full-time software engineer at JPMorgan Chase for the last two
years. He graduated from the University of Strathclyde in Scotland in the spring of 2015
and worked as a freelancer developing web solutions while studying there.
He has worked on numerous different technologies such as GoLang and NodeJS and plain
old Java, and he has spent years working on concurrent enterprise systems. It is with this
experience that he was able to write this book.
Elliot has even worked at Barclays Investment Bank for a summer internship in London and
has maintained a couple of software development websites for the last three years.
About the Reviewer
Nikolaus Gradwohl was born 1976 in Vienna, Austria and always wanted to become an
inventor like Gyro Gearloose. When he got his first Atari, he figured out that being a
computer programmer is the closest he could get to that dream. For a living, he wrote
programs for nearly anything that can be programmed, ranging from an 8-bit
microcontroller to mainframes. In his free time, he likes to master on programming
languages and operating systems.
Nikolaus authored the Processing 2: Creative Coding Hotshot book, and you can see some of
his work on his blog at (cid:73)(cid:85)(cid:85)(cid:81)(cid:28)(cid:17)(cid:17)(cid:88)(cid:88)(cid:88)(cid:16)(cid:77)(cid:80)(cid:68)(cid:66)(cid:77)(cid:15)(cid:72)(cid:86)(cid:83)(cid:86)(cid:16)(cid:79)(cid:70)(cid:85)(cid:17).
(cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2)
www.PacktPub.com
For support files and downloads related to your book, please visit (cid:88)(cid:88)(cid:88)(cid:16)(cid:49)(cid:66)(cid:68)(cid:76)(cid:85)(cid:49)(cid:86)(cid:67)(cid:16)(cid:68)(cid:80)(cid:78).
Did you know that Packt offers eBook versions of every book published, with PDF and
ePub files available? You can upgrade to the eBook version at (cid:88)(cid:88)(cid:88)(cid:16)(cid:49)(cid:66)(cid:68)(cid:76)(cid:85)(cid:49)(cid:86)(cid:67)(cid:16)(cid:68)(cid:80)(cid:78) and as a
print book customer, you are entitled to a discount on the eBook copy. Get in touch with us
at (cid:84)(cid:70)(cid:83)(cid:87)(cid:74)(cid:68)(cid:70)(cid:33)(cid:81)(cid:66)(cid:68)(cid:76)(cid:85)(cid:81)(cid:86)(cid:67)(cid:16)(cid:68)(cid:80)(cid:78) for more details.
At (cid:88)(cid:88)(cid:88)(cid:16)(cid:49)(cid:66)(cid:68)(cid:76)(cid:85)(cid:49)(cid:86)(cid:67)(cid:16)(cid:68)(cid:80)(cid:78), you can also read a collection of free technical articles, sign up for a
range of free newsletters and receive exclusive discounts and offers on Packt books and
eBooks.
(cid:73)(cid:85)(cid:85)(cid:81)(cid:84)(cid:28)(cid:17)(cid:17)(cid:88)(cid:88)(cid:88)(cid:16)(cid:81)(cid:66)(cid:68)(cid:76)(cid:85)(cid:81)(cid:86)(cid:67)(cid:16)(cid:68)(cid:80)(cid:78)(cid:17)(cid:78)(cid:66)(cid:81)(cid:85)
(cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2)
Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt
books and video courses, as well as industry-leading tools to help you plan your personal
development and advance your career.
Why subscribe?
Fully searchable across every book published by Packt
Copy and paste, print, and bookmark content
On demand and accessible via a web browser
Customer Feedback
Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial
process. To help us improve, please leave us an honest review on this book's Amazon page
at (cid:73)(cid:85)(cid:85)(cid:81)(cid:84)(cid:28)(cid:17)(cid:17)(cid:88)(cid:88)(cid:88)(cid:16)(cid:66)(cid:78)(cid:66)(cid:91)(cid:80)(cid:79)(cid:16)(cid:68)(cid:80)(cid:78)(cid:17)(cid:69)(cid:81)(cid:17)(cid:19)(cid:25)(cid:26)(cid:25)(cid:20)(cid:26)(cid:23)(cid:21)(cid:25)(cid:23).
(cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2) (cid:2)
If you'd like to join our team of regular reviewers, you can e-mail us at
(cid:68)(cid:86)(cid:84)(cid:85)(cid:80)(cid:78)(cid:70)(cid:83)(cid:83)(cid:70)(cid:87)(cid:74)(cid:70)(cid:88)(cid:84)(cid:33)(cid:81)(cid:66)(cid:68)(cid:76)(cid:85)(cid:81)(cid:86)(cid:67)(cid:16)(cid:68)(cid:80)(cid:78). We award our regular reviewers with free eBooks and
videos in exchange for their valuable feedback. Help us be relentless in improving our
products!
Table of Contents
Preface
1
Chapter 1: Speed It Up!
6
History of concurrency 7
Threads and multithreading 8
What is a thread? 8
Types of threads 9
What is multithreading? 9
Processes 10
Properties of processes 11
Multiprocessing 12
Event-driven programming 13
Turtle 14
Breaking it down 15
Reactive programming 16
ReactiveX - RxPy 16
Breaking it down 18
GPU programming 19
PyCUDA 20
OpenCL 20
Theano 21
The limitations of Python 21
Jython 22
IronPython 23
Why should we use Python? 23
Concurrent image download 23
Sequential download 24
Breaking it down 24
Concurrent download 25
Breaking it down 26
Improving number crunching with multiprocessing 26
Sequential prime factorization 27
Breaking it down 27
Concurrent prime factorization 28
Breaking it down 29
Summary 30
Chapter 2: Parallelize It
31
Understanding concurrency 32
Properties of concurrent systems 32
I/O bottlenecks 33
Understanding parallelism 35
CPU-bound bottlenecks 36
How do they work on a CPU? 36
Single-core CPUs 37
Clock rate 37
Martelli model of scalability 38
Time-sharing - the task scheduler 39
Multi-core processors 40
System architecture styles 41
SISD 41
SIMD 42
MISD 44
MIMD 44
Computer memory architecture styles 45
UMA 45
NUMA 46
Summary 48
Chapter 3: Life of a Thread
49
Threads in Python 49
Thread state 50
State flow chart 51
Python example of thread state 51
Breaking it down 52
Different types of threads 52
POSIX threads 53
Windows threads 53
The ways to start a thread 53
Starting a thread 53
Inheriting from the thread class 54
Breaking it down 54
Forking 55
Example 55
Breaking it down 55
Daemonizing a thread 56
Example 57
Breaking it down 57
[ ii ]