Table Of Content[ 1 ]
Mastering Python for
Data Science
Explore the world of data science through Python and
learn how to make sense of data
Samir Madhavan
BIRMINGHAM - MUMBAI
Mastering Python for Data Science
Copyright © 2015 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval
system, or transmitted in any form or by any means, without the prior written
permission of the publisher, except in the case of brief quotations embedded in
critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy
of the information presented. However, the information contained in this book is
sold without warranty, either express or implied. Neither the author, nor Packt
Publishing, and its dealers and distributors will be held liable for any damages
caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the
companies and products mentioned in this book by the appropriate use of capitals.
However, Packt Publishing cannot guarantee the accuracy of this information.
First published: August 2015
Production reference: 1260815
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham B3 2PB, UK.
ISBN 978-1-78439-015-0
www.packtpub.com
Credits
Author Project Coordinator
Samir Madhavan Neha Bhatnagar
Reviewers Proofreader
Sébastien Celles Safis Editing
Robert Dempsey
Maurice HT Ling Indexer
Monica Ajmera Mehta
Ratanlal Mahanta
Yingssu Tsai
Graphics
Disha Haria
Commissioning Editor
Jason Monteiro
Pramila Balan
Production Coordinator
Acquisition Editor
Arvindkumar Gupta
Sonali Vernekar
Cover Work
Content Development Editor
Arvindkumar Gupta
Arun Nadar
Technical Editor
Chinmay S. Puranik
Copy Editor
Sonia Michelle Cheema
About the Author
Samir Madhavan has been working in the field of data science since 2010.
He is an industry expert on machine learning and big data. He has also reviewed
R Machine Learning Essentials by Packt Publishing. He was part of the ubiquitous
Aadhar project of the Unique Identification Authority of India, which is in the
process of helping every Indian get a unique number that is similar to a social
security number in the United States. He was also the first employee of Flutura
Decision Sciences and Analytics and is a part of the core team that has helped scale
the number of employees in the company to 50. His company is now recognized
as one of the most promising Internet of Things—Decision Sciences companies
in the world.
I would like to thank my mom, Rajasree Madhavan, and dad,
P Madhavan, for all their support. I would also like to thank
Srikanth Muralidhara, Krishnan Raman, and Derick Jose, who gave
me the opportunity to start my career in the world of data science.
About the Reviewers
Sébastien Celles is a professor of applied physics at Universite de Poitiers (working
in the thermal science department). He has used Python for numerical simulations,
data plotting, data predictions, and various other tasks since the early 2000s. He is a
member of PyData and was granted commit rights to the pandas DataReader project.
He is also involved in several open source projects in the scientific Python ecosystem.
Sebastien is also the author of some Python packages available on PyPi, which are
as follows:
• openweathermap_requests: This is a package used to fetch data from
OpenWeatherMap.org using Requests and Requests-cache and to get pandas
DataFrame with weather history
• pandas_degreedays: This is a package used to calculate degree days
(a measure of heating or cooling) from the pandas time series of temperature
• pandas_confusion: This is a package used to manage confusion matrices, plot
and binarize them, and calculate overall and class statistics
• There are some other packages authored by him, such as pyade,
pandas_datareaders_unofficial, and more
He also has a personal interest in data mining, machine learning techniques,
forecasting, and so on. You can find more information about him at http://www.
celles.net/wiki/Contact or https://www.linkedin.com/in/sebastiencelles.
Robert Dempsey is a leader and technology professional, specializing in
delivering solutions and products to solve tough business challenges. His experience
of forming and leading agile teams combined with more than 15 years of technology
experience enables him to solve complex problems while always keeping the bottom
line in mind.
Robert founded and built three start-ups in the tech and marketing fields, developed
and sold two online applications, consulted for Fortune 500 and Inc. 500 companies,
and has spoken nationally and internationally on software development and agile
project management.
He's the founder of Data Wranglers DC, a group dedicated to improving the
craft of data wrangling, as well as a board member of Data Community DC.
He is currently the team leader of data operations at ARPC, an econometrics
firm based in Washington, DC.
In addition to spending time with his growing family, Robert geeks out on Raspberry
Pi's, Arduinos, and automating more of his life through hardware and software.
Maurice HT Ling has been programming in Python since 2003. Having completed
his PhD in bioinformatics and BSc (Hons) in molecular and cell biology from The
University of Melbourne, he is currently a research fellow at Nanyang Technological
University, Singapore. He is also an honorary fellow of The University of Melbourne,
Australia. Maurice is the chief editor of Computational and Mathematical Biology
and coeditor of The Python Papers. Recently, he cofounded the first synthetic
biology start-up in Singapore, called AdvanceSyn Pte. Ltd., as the director and chief
technology officer. His research interests lie in life itself, such as biological life and
artificial life, and artificial intelligence, which use computer science and statistics as
tools to understand life and its numerous aspects. In his free time, Maurice likes to
read, enjoy a cup of coffee, write his personal journal, or philosophize on various
aspects of life. His website and LinkedIn profile are http://maurice.vodien.com
and http://www.linkedin.com/in/mauriceling, respectively.
Ratanlal Mahanta is a senior quantitative analyst. He holds an MSc degree in
computational finance and is currently working at GPSK Investment Group as a
senior quantitative analyst. He has 4 years of experience in quantitative trading and
strategy development for sell-side and risk consultation firms. He is an expert in high
frequency and algorithmic trading.
He has expertise in the following areas:
• Quantitative trading: This includes FX, equities, futures, options, and
engineering on derivatives
• Algorithms: This includes Partial Differential Equations, Stochastic
Differential Equations, Finite Difference Method, Monte-Carlo,
and Machine Learning
• Code: This includes R Programming, C++, Python, MATLAB, HPC, and
scientific computing
• Data analysis: This includes big data analytics (EOD to TBT), Bloomberg,
Quandl, and Quantopian
• Strategies: This includes Vol Arbitrage, Vanilla and Exotic Options Modeling,
trend following, Mean reversion, Co-integration, Monte-Carlo Simulations,
ValueatRisk, Stress Testing, Buy side trading strategies with high Sharpe
ratio, Credit Risk Modeling, and Credit Rating
He has already reviewed Mastering Scientific Computing with R, Mastering R for
Quantitative Finance, and Machine Learning with R Cookbook, all by Packt Publishing.
You can find out more about him at https://twitter.com/mahantaratan.
Yingssu Tsai is a data scientist. She holds degrees from the University of
California, Berkeley, and the University of California, Los Angeles.
www.PacktPub.com
Support files, eBooks, discount offers, and more
For support files and downloads related to your book, please visit www.PacktPub.com.
Did you know that Packt offers eBook versions of every book published, with PDF
and ePub files available? You can upgrade to the eBook version at www.PacktPub.com
and as a print book customer, you are entitled to a discount on the eBook copy. Get in
touch with us at service@packtpub.com for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign
up for a range of free newsletters and receive exclusive discounts and offers on Packt
books and eBooks.
TM
https://www2.packtpub.com/books/subscription/packtlib
Do you need instant solutions to your IT questions? PacktLib is Packt's online digital
book library. Here, you can search, access, and read Packt's entire library of books.
Why subscribe?
• Fully searchable across every book published by Packt
• Copy and paste, print, and bookmark content
• On demand and accessible via a web browser
Free access for Packt account holders
If you have an account with Packt at www.PacktPub.com, you can use this to access
PacktLib today and view 9 entirely free books. Simply use your login credentials for
immediate access.
Table of Contents
Preface vii
Chapter 1: Getting Started with Raw Data 1
The world of arrays with NumPy 2
Creating an array 2
Mathematical operations 3
Array subtraction 4
Squaring an array 4
A trigonometric function performed on the array 4
Conditional operations 4
Matrix multiplication 5
Indexing and slicing 5
Shape manipulation 6
Empowering data analysis with pandas 7
The data structure of pandas 7
Series 7
DataFrame 8
Panel 9
Inserting and exporting data 10
CSV 11
XLS 11
JSON 12
Database 12
Data cleansing 12
Checking the missing data 13
Filling the missing data 14
String operations 16
Merging data 19
Data operations 20
Aggregation operations 20
[ i ]
