Deep Learning Made Easy with R

251 Pages·2016·5.52 MB·English

DEEP LEARNING
MADE EASY
WITH R
A Gentle Introduction for Data Science.
Dr. N.D. Lewis
Copyright © 2016 by N.D. Lewis
All rights reserved. No part of this publication may be reproduced, dis-
tributed, or transmitted in any form or by any means, including photo-
copying, recording, or other electronic or mechanical methods, without
thepriorwrittenpermissionoftheauthor,exceptinthecaseofbriefquo-
tationsembodiedincriticalreviewsandcertainothernoncommercialuses
permitted by copyright law. For permission requests, contact the author
at: www.AusCov.com.
Disclaimer: Although the author and publisher have made every effort to
ensure that the information in this book was correct at press time, the
author and publisher do not assume and hereby disclaim any liability to
anypartyforanyloss,damage,ordisruptioncausedbyerrorsoromissions,
whether such errors or omissions result from negligence, accident, or any
other cause.
Ordering Information: Quantity sales. Special discounts are available on
quantity purchases by corporations, associations, and others. For details,
email: [email protected]
Image photography by Deanna Lewis
ISBN: 978-1519514219
ISBN: 1519514212
Contents
Acknowledgements iii
Preface viii
How to Get the Most from this Book 1
1 Introduction 5
WhatisDeepLearning? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
WhatProblemsCanDeepLearningSolve? . . . . . . . . . . . . . . . . . . . 8
WhoUsesDeepLearning? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
APrimeronNeuralNetworks . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2 Deep Neural Networks 31
TheAmazinglySimpleAnatomyoftheDNN . . . . . . . . . . . . . . . . . . 32
HowtoExplainaDNNin60SecondsorLess . . . . . . . . . . . . . . . . . . 33
ThreeBrilliantWaystoUseDeepNeuralNetworks. . . . . . . . . . . . . . . 34
HowtoImmediatelyApproximateAnyFunction . . . . . . . . . . . . . . . . 42
TheAnswertoHowManyNeuronstoInclude. . . . . . . . . . . . . . . . . . 48
TheABCsofChoosingtheOptimalNumberofLayers . . . . . . . . . . . . . 49
ThreeIdeastoSuperchargeDNNPerformance . . . . . . . . . . . . . . . . . 51
IncrediblySimpleWaystoBuildDNNswithR . . . . . . . . . . . . . . . . . 57
Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
3 Elman Neural Networks 87
WhatisanElmanNeuralNetwork? . . . . . . . . . . . . . . . . . . . . . . . 88
WhatistheRoleofContextLayerNeurons? . . . . . . . . . . . . . . . . . . 88
HowtoUnderstandtheInformationFlow . . . . . . . . . . . . . . . . . . . . 89
HowtoUseElmanNetworkstoBoostYourResult’s . . . . . . . . . . . . . . 91
FourSmartWaystouseElmanNeuralNetworks . . . . . . . . . . . . . . . . 92
TheEasyWaytoBuildElmanNetworks . . . . . . . . . . . . . . . . . . . . 95
HereisHowtoLoadtheBestPackages . . . . . . . . . . . . . . . . . . . . . 95
WhyViewingDataistheNewScience . . . . . . . . . . . . . . . . . . . . . . 96
TheSecrettoTransformingData . . . . . . . . . . . . . . . . . . . . . . . . . 100
HowtoEstimateanInterestingModel . . . . . . . . . . . . . . . . . . . . . . 102
CreatingtheIdealPrediction . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
i
4 Jordan Neural Networks 107
ThreeProblemsJordanNeuralNetworksCanSolve . . . . . . . . . . . . . . 108
EssentialElementsforEffectiveJordanModelsinR . . . . . . . . . . . . . . 110
WhicharetheAppropriatePackages? . . . . . . . . . . . . . . . . . . . . . . 110
ADamnGoodWaytoTransformData . . . . . . . . . . . . . . . . . . . . . 112
HereisHowtoSelecttheTrainingSample. . . . . . . . . . . . . . . . . . . . 114
UseThisTiptoEstimateYourModel . . . . . . . . . . . . . . . . . . . . . . 115
Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
5 The Secret to the Autoencoder 119
AJediMindTrick . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
TheSecretRevealed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
APracticalDefinitionYouCanRunWith . . . . . . . . . . . . . . . . . . . . 124
SavingtheBrazilianCerrado . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
TheEssentialIngredientYouNeedtoKnow . . . . . . . . . . . . . . . . . . . 125
ThePowerfulBenefitoftheSparseAutoencoder . . . . . . . . . . . . . . . . 126
UnderstandingKullback-LeiblerDivergence . . . . . . . . . . . . . . . . . . . 126
ThreeTimelessLessonsfromtheSparseAutoencoder . . . . . . . . . . . . . 128
MixingHollywood,Biometrics&SparseAutoencoders . . . . . . . . . . . . . 128
HowtoImmediatelyusetheAutoencoderinR . . . . . . . . . . . . . . . . . 131
AnIdeaforYourOwnDataScienceProjectswithR . . . . . . . . . . . . . . 137
Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
6 The Stacked Autoencoder in a Nutshell 147
TheDeepLearningGuru’sSecretSauceforTraining . . . . . . . . . . . . . 148
HowMuchSleepDoYouNeed? . . . . . . . . . . . . . . . . . . . . . . . . . 149
BuildaStackedAutoencoderinLessThan5Minutes . . . . . . . . . . . . . 153
WhatisaDenoisingAutoencoder? . . . . . . . . . . . . . . . . . . . . . . . . 155
TheSaltandPepperofRandomMasking . . . . . . . . . . . . . . . . . . . . 156
TheTwoEssentialTasksofaDenoisingAutoencoder . . . . . . . . . . . . . 156
HowtoUnderstandStackedDenoisingAutoencoders . . . . . . . . . . . . . . 157
AStunninglyPracticalApplication . . . . . . . . . . . . . . . . . . . . . . . . 158
TheFastPathtoaDenoisingAutoencoderinR . . . . . . . . . . . . . . . . 166
Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
7 Restricted Boltzmann Machines 177
TheFourStepstoKnowledge . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
TheRoleofEnergyandProbability . . . . . . . . . . . . . . . . . . . . . . . 179
AMagnificentWaytoThink . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
TheGoalofModelLearning . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
TrainingTricksthatWorkLikeMagic . . . . . . . . . . . . . . . . . . . . . . 183
TheKeyCriticismofDeepLearning . . . . . . . . . . . . . . . . . . . . . . . 187
TwoIdeasthatcanChangetheWorld . . . . . . . . . . . . . . . . . . . . . . 188
SecretstotheRestrictedBoltzmannMachineinR . . . . . . . . . . . . . . . 194
Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
8 Deep Belief Networks 205
HowtoTrainaDeepBeliefNetwork . . . . . . . . . . . . . . . . . . . . . . . 206
HowtoDeliveraBetterCallWaitingExperience . . . . . . . . . . . . . . . . 207
AWorldFirstIdeathatYouCanEasilyEmulate . . . . . . . . . . . . . . . . 209
StepstoBuildingaDeepBeliefNetworkinR . . . . . . . . . . . . . . . . . . 212
Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
Index 222
Dedicated to Angela, wife, friend and mother
extraordinaire.

Acknowledgments
A special thank you to:
My wife Angela, for her patience and constant encouragement.
My daughter Deanna, for taking hundreds of photographs for
this book and my website.
And the readers of my earlier books who contacted me with
questions and suggestions.
iii

Master Deep Learning with this fun,
practical, hands on guide.
With the explosion of big data deep learning is now on the
radar. Large companies such as Google, Microsoft, and Face-
book have taken notice, and are actively growing in-house deep
learning teams. Other large corporations are quickly building
out their own teams. If you want to join the ranks of today’s
top data scientists take advantage of this valuable book. It will
help you get started. It reveals how deep learning models work,
and takes you under the hood with an easy to follow process
showing you how to build them faster than you imagined pos-
sible using the powerful, free R predictive analytics package.
NO EXPERIENCE REQUIRED - Bestselling decision sci-
entist Dr. N.D Lewis builds deep learning models for fun. Now
he shows you the shortcut up the steep steps to the very top.
It’s easier than you think. Through a simple to follow process
you will learn how to build the most successful deep learning
models used for learning from data. Once you have mastered
the process, it will be easy for you to translate your knowledge
into your own powerful applications.
If you want to accelerate your progress, discover the best in
deep learning and act on what you have learned, this book is
the place to get started.
YOU’LL LEARN HOW TO:
• Develop Recurrent Neural Networks
• Build Elman Neural Networks
• Deploy Jordan Neural Networks
• Create Cascade Correlation Neural Networks
• Understand Deep Neural Networks
Deep Learning Made Easy with R
• Use Autoencoders
• Unleash the power of Stacked Autoencoders
• Leverage the Restricted Boltzmann Machine
• Master Deep Belief Networks
Once people have a chance to learn how deep learning can
impact their data analysis efforts, they want to get hands on
withthetools. Thisbookwillhelpyoutostartbuildingsmarter
applications today using R. Everything you need to get started
is contained within this book. It is your detailed, practical,
tactical hands on guide - the ultimate cheat sheet for deep
learning mastery.
A book for everyone interested in machine learning, predic-
tive analytic techniques, neural networks and decision science.
Buy the book today. Your next big breakthrough using deep
learning is only a page away!
vi