Deep Learning By Example

442 Pages·08.687 MB·English

by  Menshawy

WOW! eBook
www.wowebook.org
Deep Learning By Example
(cid:34)(cid:2)(cid:73)(cid:66)(cid:79)(cid:69)(cid:84)(cid:15)(cid:80)(cid:79)(cid:2)(cid:72)(cid:86)(cid:74)(cid:69)(cid:70)(cid:2)(cid:85)(cid:80)(cid:2)(cid:74)(cid:78)(cid:81)(cid:77)(cid:70)(cid:78)(cid:70)(cid:79)(cid:85)(cid:74)(cid:79)(cid:72)(cid:2)(cid:66)(cid:69)(cid:87)(cid:66)(cid:79)(cid:68)(cid:70)(cid:69)(cid:2)(cid:78)(cid:66)(cid:68)(cid:73)(cid:74)(cid:79)(cid:70)
(cid:77)(cid:70)(cid:66)(cid:83)(cid:79)(cid:74)(cid:79)(cid:72)(cid:2)(cid:66)(cid:77)(cid:72)(cid:80)(cid:83)(cid:74)(cid:85)(cid:73)(cid:78)(cid:84)(cid:2)(cid:66)(cid:79)(cid:69)(cid:2)(cid:79)(cid:70)(cid:86)(cid:83)(cid:66)(cid:77)(cid:2)(cid:79)(cid:70)(cid:85)(cid:88)(cid:80)(cid:83)(cid:76)(cid:84)
Ahmed Menshawy
BIRMINGHAM - MUMBAI
WOW! eBook
www.wowebook.org
Deep Learning By Example
Copyright (cid:97) 2018 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form
or by any means, without the prior written permission of the publisher, except in the case of brief quotations
embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented.
However, the information contained in this book is sold without warranty, either express or implied. Neither the
author(s), nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged
to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products
mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy
of this information.
Commissioning Editor: Vedika Naik
Acquisition Editor: Tushar Gupta
Content Development Editor: Aishwarya Pandere
Technical Editor: Sagar Sawant
Copy Editor: Vikrant Phadke, Safis Editing
Project Coordinator: Nidhi Joshi
Proofreader: Safis Editing
Indexer: Mariammal Chettiyar
Graphics: Tania Dutta
Production Coordinator: Aparna Bhagat
First published: February 2018
Production reference: 1260218
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham
B3 2PB, UK.
ISBN 978-1-78839-990-6
(cid:88)(cid:88)(cid:88)(cid:16)(cid:81)(cid:66)(cid:68)(cid:76)(cid:85)(cid:81)(cid:86)(cid:67)(cid:16)(cid:68)(cid:80)(cid:78)
WOW! eBook
www.wowebook.org
(cid:78)(cid:66)(cid:81)(cid:85)(cid:16)(cid:74)(cid:80)
Mapt is an online digital library that gives you full access to over 5,000 books and videos, as
well as industry leading tools to help you plan your personal development and advance
your career. For more information, please visit our website.
Why subscribe?
Spend less time learning and more time coding with practical eBooks and Videos
from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Mapt is fully searchable
Copy and paste, print, and bookmark content
PacktPub.com
Did you know that Packt offers eBook versions of every book published, with PDF and
ePub files available? You can upgrade to the eBook version at (cid:88)(cid:88)(cid:88)(cid:16)(cid:49)(cid:66)(cid:68)(cid:76)(cid:85)(cid:49)(cid:86)(cid:67)(cid:16)(cid:68)(cid:80)(cid:78) and as a
print book customer, you are entitled to a discount on the eBook copy. Get in touch with us
at (cid:84)(cid:70)(cid:83)(cid:87)(cid:74)(cid:68)(cid:70)(cid:33)(cid:81)(cid:66)(cid:68)(cid:76)(cid:85)(cid:81)(cid:86)(cid:67)(cid:16)(cid:68)(cid:80)(cid:78) for more details.
At (cid:88)(cid:88)(cid:88)(cid:16)(cid:49)(cid:66)(cid:68)(cid:76)(cid:85)(cid:49)(cid:86)(cid:67)(cid:16)(cid:68)(cid:80)(cid:78), you can also read a collection of free technical articles, sign up for a
range of free newsletters, and receive exclusive discounts and offers on Packt books and
eBooks.
WOW! eBook
www.wowebook.org
Contributors
About the author
Ahmed Menshawy is a Research Engineer at the Trinity College Dublin, Ireland. He has
more than 5 years of working experience in the area of ML and NLP. He holds an MSc in
Advanced Computer Science. He started his Career as a Teaching Assistant at the
Department of Computer Science, Helwan University, Cairo, Egypt. He taught several
advanced ML and NLP courses such as ML, Image Processing, and so on. He was involved
in implementing the state-of-the-art system for Arabic Text to Speech. He was the main ML
specialist at the Industrial research and development lab at IST Networks, based in Egypt.
I want to thank the people who have been close to me and supported me, especially my wife
Sara and my parents
WOW! eBook
www.wowebook.org
About the reviewers
Md. Rezaul Karim is a Research Scientist at Fraunhofer FIT, Germany. He is also a PhD
candidate at RWTH Aachen University, Germany. Before joining FIT, he worked as a
Researcher at the Insight Centre for Data Analytics, Ireland. Earlier, he worked as a Lead
Engineer at Samsung Electronics, Korea.
He has 9 years of R&D experience with C++, Java, R, Scala, and Python. He has published
research papers concerning bioinformatics, big data, and deep learning. He has practical
working experience with Spark, Zeppelin, Hadoop, Keras, Scikit-Learn, TensorFlow,
DeepLearning4j, MXNet, and H2O.
Doug Ortiz is an experienced Enterprise Cloud, Big Data, Data Analytics and Solutions
Architect who has architected, designed, developed, re-engineered and integrated
enterprise solutions. Other expertise: Amazon Web Services, Azure, Google Cloud, Business
Intelligence, Hadoop, Spark, NoSQL Databases, SharePoint to mention a few.
Is the founder of Illustris, LLC reachable at: (cid:69)(cid:80)(cid:86)(cid:72)(cid:80)(cid:83)(cid:85)(cid:74)(cid:91)(cid:33)(cid:74)(cid:77)(cid:77)(cid:86)(cid:84)(cid:85)(cid:83)(cid:74)(cid:84)(cid:16)(cid:80)(cid:83)(cid:72)
Huge thanks to my wonderful wife Milla, Maria, Nikolay and our children for all their
support.
Packt is searching for authors like you
If you're interested in becoming an author for Packt, please visit (cid:66)(cid:86)(cid:85)(cid:73)(cid:80)(cid:83)(cid:84)(cid:16)(cid:81)(cid:66)(cid:68)(cid:76)(cid:85)(cid:81)(cid:86)(cid:67)(cid:16)(cid:68)(cid:80)(cid:78) and
apply today. We have worked with thousands of developers and tech professionals, just
like you, to help them share their insight with the global tech community. You can make a
general application, apply for a specific hot topic that we are recruiting an author for, or
submit your own idea.
WOW! eBook
www.wowebook.org
Table of Contents
Preface 1
Chapter 1: Data Science - A Birds' Eye View 9
Understanding data science by an example 10
Design procedure of data science algorithms 17
Data pre-processing 18
Data cleaning 18
Data pre-processing 19
Feature selection 19
Model selection 19
Learning process 20
Evaluating your model 20
Getting to learn 21
Challenges of learning 21
Feature extraction – feature engineering 21
Noise 21
Overfitting 21
Selection of a machine learning algorithm 22
Prior knowledge 22
Missing values 22
Implementing the fish recognition/detection model 23
Knowledge base/dataset 24
Data analysis pre-processing 25
Model building 28
Model training and testing 33
Fish recognition – all together 33
Different learning types 33
Supervised learning 34
Unsupervised learning 35
Semi-supervised learning 36
Reinforcement learning 37
Data size and industry needs 37
WOW! eBook
www.wowebook.org
Table of Contents
Summary 38
Chapter 2: Data Modeling in Action - The Titanic Example 39
Linear models for regression 39
Motivation 40
Advertising – a financial example 40
Dependencies 41
Importing data with pandas 41
Understanding the advertising data 43
Data analysis and visualization 43
Simple regression model 45
Learning model coefficients 45
Interpreting model coefficients 47
Using the model for prediction 47
Linear models for classification 49
Classification and logistic regression 51
Titanic example – model building and training 53
Data handling and visualization 54
Data analysis – supervised machine learning 60
Different types of errors 63
Apparent (training set) error 64
Generalization/true error 65
Summary 65
Chapter 3: Feature Engineering and Model Complexity – The Titanic
Example Revisited 66
Feature engineering 67
Types of feature engineering 67
Feature selection 68
Dimensionality reduction 68
Feature construction 68
Titanic example revisited 69
Missing values 70
Removing any sample with missing values in it 70
Missing value inputting 70
Assigning an average value 70
Using a regression or another simple model to predict the values of missing
variables 71
Feature transformations 72
[ ii ]
WOW! eBook
www.wowebook.org
Table of Contents
Dummy features 72
Factorizing 73
Scaling 73
Binning 74
Derived features 74
Name 75
Cabin 76
Ticket 77
Interaction features 78
The curse of dimensionality 80
Avoiding the curse of dimensionality 82
Titanic example revisited – all together 83
Bias-variance decomposition 95
Learning visibility 98
Breaking the rule of thumb 98
Summary 98
Chapter 4: Get Up and Running with TensorFlow 100
TensorFlow installation 100
TensorFlow GPU installation for Ubuntu 16.04 101
Installing NVIDIA drivers and CUDA 8 101
Installing TensorFlow 104
TensorFlow CPU installation for Ubuntu 16.04 105
TensorFlow CPU installation for macOS X 106
TensorFlow GPU/CPU installation for Windows 108
The TensorFlow environment 108
Computational graphs 110
TensorFlow data types, variables, and placeholders 111
Variables 112
Placeholders 113
Mathematical operations 114
Getting output from TensorFlow 115
TensorBoard – visualizing learning 118
Summary 123
Chapter 5: TensorFlow in Action - Some Basic Examples 124
Capacity of a single neuron 125
Biological motivation and connections 125
[ iii ]
WOW! eBook
www.wowebook.org
Table of Contents
Activation functions 127
Sigmoid 128
Tanh 128
ReLU 128
Feed-forward neural network 129
The need for multilayer networks 130
Training our MLP – the backpropagation algorithm 132
Step 1 – forward propagation 133
Step 2 – backpropagation and weight updation 134
TensorFlow terminologies – recap 135
Defining multidimensional arrays using TensorFlow 137
Why tensors? 139
Variables 140
Placeholders 141
Operations 143
Linear regression model – building and training 144
Linear regression with TensorFlow 145
Logistic regression model – building and training 149
Utilizing logistic regression in TensorFlow 149
Why use placeholders? 150
Set model weights and bias 151
Logistic regression model 151
Training 152
Cost function 152
Summary 157
Chapter 6: Deep Feed-forward Neural Networks - Implementing Digit
Classification 158
Hidden units and architecture design 158
MNIST dataset analysis 160
The MNIST data 161
Digit classification – model building and training 162
Data analysis 166
Building the model 169
Model training 174
Summary 179
[ iv ]
WOW! eBook
www.wowebook.org