Python in a Nutshell. A Desktop Quick Reference

767 Pages·2017·6.49 MB·English

by  Alex Martelli

C 3
o r
ve d
r
with s PytEdit
3. h i
6 o o
hi n n
ghli2.7
ghts& 3.
5
Python

in a Nutshell
THE DEFINITIVE REFERENCE
Alex Martelli,
Anna Ravenscroft & Steve Holden

PYTHON
IN A NUTSHELL
Third Edition
Alex Martelli, Anna Ravenscroft,
and Steve Holden
Python in a Nutshell
by Alex Martelli, Anna Ravenscroft, and Steve Holden
Copyright © 2017 Alex Martelli, Anna Martelli Ravenscroft, and Steve Holden. All rights
reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales promotional use. Online
editions are also available for most titles (http://oreilly.com/safari). For more information, con‐
tact our corporate/institutional sales department: 800-998-9938 or [email protected].
Editor: Dawn Schanafelt Indexer: Judith McConville
Production Editor: Kristen Brown Interior Designer: David Futato
Copyeditor: Kim Cofer Cover Designer: Karen Montgomery
Proofreader: Rachel Monaghan Illustrator: Rebecca Demarest
April 2017: Third Edition
Revision History for the Third Edition
2017-04-06: First Release
See http://oreilly.com/catalog/errata.csp?isbn=9781449392925 for release details.
The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Python in a Nutshell, the
cover image, and related trade dress are trademarks of O’Reilly Media, Inc.
While the publisher and the authors have used good faith efforts to ensure that the informa‐
tion and instructions contained in this work are accurate, the publisher and the authors dis‐
claim all responsibility for errors or omissions, including without limitation responsibility for
damages resulting from the use of or reliance on this work. Use of the information and
instructions contained in this work is at your own risk. If any code samples or other technol‐
ogy this work contains or describes is subject to open source licenses or the intellectual prop‐
erty rights of others, it is your responsibility to ensure that your use thereof complies with
such licenses and/or rights.
978-1-449-39292-5
[LSI]
Table of Contents
Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Part I. Getting Started with Python
1. Introduction to Python. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
The Python Language 3
The Python Standard Library and Extension Modules 5
Python Implementations 5
Python Development and Versions 12
Python Resources 14
Installation 17
Installing Python from Binaries 18
Installing Python from Source Code 19
Installing Jython 21
Installing IronPython 22
Installing PyPy 22
2. The Python Interpreter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
The python Program 23
Python Development Environments 28
Running Python Programs 30
The jython Interpreter 31
The IronPython Interpreter 31
The PyPy Interpreter 32
iii
Part II. Core Python Language and Built-ins
3. The Python Language. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Lexical Structure 35
Data Types 41
Variables and Other References 50
Expressions and Operators 55
Numeric Operations 57
Sequence Operations 59
Set Operations 65
Dictionary Operations 67
Control Flow Statements 70
Functions 80
4. Object-Oriented Python. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Classes and Instances 99
Special Methods 123
Decorators 138
Metaclasses 140
5. Exceptions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
The try Statement 147
The with Statement and Context Managers 151
Exception Propagation 153
The raise Statement 155
Exception Objects 156
Custom Exception Classes 161
Error-Checking Strategies 163
The assert Statement 168
6. Modules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
Module Objects 172
Module Loading 176
Packages 182
Distribution Utilities (distutils) and setuptools 184
Python Environments 186
7. Core Built-ins and Standard Library Modules. . . . . . . . . . . . . . . . . . . . . . . . . 195
Built-in Types 196
Built-in Functions 199
The sys Module 210
iv | Table of Contents
The copy Module 214
The collections Module 215
The functools Module 220
The heapq Module 222
The argparse Module 225
The itertools Module 226
8. Strings and Things. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
Methods of String and Bytes Objects 231
The string Module 237
String Formatting 238
Text Wrapping and Filling 247
The pprint Module 247
The reprlib Module 248
Unicode 248
9. Regular Expressions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
Regular Expressions and the re Module 251
Part III. Python Library and Extension Modules
10. File and Text Operations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
Other Chapters That Also Deal with Files 269
Organization of This Chapter 269
The io Module 271
Auxiliary Modules for File I/O 279
In-Memory “Files”: io.StringIO and io.BytesIO 284
Compressed Files 285
The os Module 295
Filesystem Operations 297
Text Input and Output 315
Richer-Text I/O 317
Interactive Command Sessions 323
Internationalization 327
11. Persistence and Databases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
Serialization 336
DBM Modules 346
Berkeley DB Interfacing 349
The Python Database API (DBAPI) 2.0 349
Table of Contents | v
12. Time Operations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
The time Module 361
The datetime Module 366
The pytz Module 372
The dateutil Module 373
The sched Module 375
The calendar Module 377
13. Controlling Execution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379
Site and User Customization 379
Termination Functions 381
Dynamic Execution and exec 381
Internal Types 385
Garbage Collection 386
14. Threads and Processes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393
Threads in Python 394
The threading Module 395
The queue Module 404
The multiprocessing Module 406
The concurrent.futures Module 415
Threaded Program Architecture 418
Process Environment 421
Running Other Programs 422
The mmap Module 427
15. Numeric Processing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431
The math and cmath Modules 431
The operator Module 436
Random and Pseudorandom Numbers 439
The fractions Module 441
The decimal Module 442
The gmpy2 Module 443
Array Processing 444
The array Module 444
Extensions for Numeric Array Computation 446
16. Testing, Debugging, and Optimizing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453
Testing 454
Debugging 465
The warnings Module 474
Optimization 477
vi | Table of Contents
Part IV. Network and Web Programming
17. Networking Basics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 495
Networking Principles 496
The Berkeley Socket Interface 497
Transport Layer Security (TLS, AKA SSL) 512
18. Asynchronous Alternatives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515
Coroutine-Based Async Architectures 516
The asyncio Module (v3 Only) 517
The selectors Module 540
19. Client-Side Network Protocol Modules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 547
Email Protocols 548
HTTP and URL Clients 551
Other Network Protocols 565
20. Serving HTTP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 567
WSGI 568
Python Web Frameworks 569
21. Email, MIME, and Other Network Encodings. . . . . . . . . . . . . . . . . . . . . . . . . . 591
MIME and Email Format Handling 591
Encoding Binary Data as ASCII Text 601
22. Structured Text: HTML. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 605
The html.entities (v2: htmlentitydefs) Module 606
Generating HTML 617
23. Structured Text: XML. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 625
ElementTree 626
Part V. Extending, Distributing, v2/v3 Migration
24. Extending and Embedding Classic Python. . . . . . . . . . . . . . . . . . . . . . . . . . . . 641
Extending Python with Python’s C API 642
Extending Python Without Python’s C API 677
Cython 678
Embedding Python 683
Table of Contents | vii
25. Distributing Extensions and Programs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 687
setuptools 688
Distributing Your Package 698
26. v2/v3 Migration and Coexistence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 705
Preparing for Python 3 706
Minimizing Syntax Differences 707
Choosing Your Support Strategy 711
Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 723
viii | Table of Contents