MATLAB. Deep Learning Toolbox™ Getting Started Guide

132 Pages·2022·2.267 MB·English

by  Mark Hudson, Beale Martin T. Hagan, Howard B. Demuth

Deep Learning Toolbox™
Getting Started Guide
Mark Hudson Beale
Martin T. Hagan
Howard B. Demuth
R2022b
How to Contact MathWorks
Latest news: www.mathworks.com
Sales and services: www.mathworks.com/sales_and_services
User community: www.mathworks.com/matlabcentral
Technical support: www.mathworks.com/support/contact_us
Phone: [phone]
The MathWorks, Inc.
1 Apple Hill Drive
Natick, MA 01760-2098
Deep Learning Toolbox™ Getting Started Guide
© COPYRIGHT 1992–2022 by The MathWorks, Inc.
The software described in this document is furnished under a license agreement. The software may be used or copied
only under the terms of the license agreement. No part of this manual may be photocopied or reproduced in any form
without prior written consent from The MathWorks, Inc.
FEDERAL ACQUISITION: This provision applies to all acquisitions of the Program and Documentation by, for, or through
the federal government of the United States. By accepting delivery of the Program or Documentation, the government
hereby agrees that this software or documentation qualifies as commercial computer software or commercial computer
software documentation as such terms are used or defined in FAR 12.212, DFARS Part 227.72, and DFARS 252.227-7014.
Accordingly, the terms and conditions of this Agreement and only those rights specified in this Agreement, shall pertain
to and govern the use, modification, reproduction, release, performance, display, and disclosure of the Program and
Documentation by the federal government (or other entity acquiring for or through the federal government) and shall
supersede any conflicting contractual terms or conditions. If this License fails to meet the government's needs or is
inconsistent in any respect with federal procurement law, the government agrees to return the Program and
Documentation, unused, to The MathWorks, Inc.
Trademarks
MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See
www.mathworks.com/trademarks for a list of additional trademarks. Other product or brand names may be
trademarks or registered trademarks of their respective holders.
Patents
MathWorks products are protected by one or more U.S. patents. Please see www.mathworks.com/patents for
more information.
Revision History
June 1992 First printing
April 1993 Second printing
January 1997 Third printing
July 1997 Fourth printing
January 1998 Fifth printing Revised for Version 3 (Release 11)
September 2000 Sixth printing Revised for Version 4 (Release 12)
June 2001 Seventh printing Minor revisions (Release 12.1)
July 2002 Online only Minor revisions (Release 13)
January 2003 Online only Minor revisions (Release 13SP1)
June 2004 Online only Revised for Version 4.0.3 (Release 14)
October 2004 Online only Revised for Version 4.0.4 (Release 14SP1)
October 2004 Eighth printing Revised for Version 4.0.4
March 2005 Online only Revised for Version 4.0.5 (Release 14SP2)
March 2006 Online only Revised for Version 5.0 (Release 2006a)
September 2006 Ninth printing Minor revisions (Release 2006b)
March 2007 Online only Minor revisions (Release 2007a)
September 2007 Online only Revised for Version 5.1 (Release 2007b)
March 2008 Online only Revised for Version 6.0 (Release 2008a)
October 2008 Online only Revised for Version 6.0.1 (Release 2008b)
March 2009 Online only Revised for Version 6.0.2 (Release 2009a)
September 2009 Online only Revised for Version 6.0.3 (Release 2009b)
March 2010 Online only Revised for Version 6.0.4 (Release 2010a)
September 2010 Tenth printing Revised for Version 7.0 (Release 2010b)
April 2011 Online only Revised for Version 7.0.1 (Release 2011a)
September 2011 Online only Revised for Version 7.0.2 (Release 2011b)
March 2012 Online only Revised for Version 7.0.3 (Release 2012a)
September 2012 Online only Revised for Version 8.0 (Release 2012b)
March 2013 Online only Revised for Version 8.0.1 (Release 2013a)
September 2013 Online only Revised for Version 8.1 (Release 2013b)
March 2014 Online only Revised for Version 8.2 (Release 2014a)
October 2014 Online only Revised for Version 8.2.1 (Release 2014b)
March 2015 Online only Revised for Version 8.3 (Release 2015a)
September 2015 Online only Revised for Version 8.4 (Release 2015b)
March 2016 Online only Revised for Version 9.0 (Release 2016a)
September 2016 Online only Revised for Version 9.1 (Release 2016b)
March 2017 Online only Revised for Version 10.0 (Release 2017a)
September 2017 Online only Revised for Version 11.0 (Release 2017b)
March 2018 Online only Revised for Version 11.1 (Release 2018a)
September 2018 Online only Revised for Version 12.0 (Release 2018b)
March 2019 Online only Revised for Version 12.1 (Release 2019a)
September 2019 Online only Revised for Version 13 (Release 2019b)
March 2020 Online only Revised for Version 14 (Release 2020a)
September 2020 Online only Revised for Version 14.1 (Release 2020b)
March 2021 Online only Revised for Version 14.2 (Release 2021a)
September 2021 Online only Revised for Version 14.3 (Release 2021b)
March 2022 Online only Revised for Version 14.4 (Release 2022a)
September 2022 Online only Revised for Version 14.5 (Release 2022b)

Contents
Getting Started
1
Deep Learning Toolbox Product Description . . . . . . . . . . . . . . . . . . . . . . . . 1-2
Get Started with Deep Network Designer . . . . . . . . . . . . . . . . . . . . . . . . . . 1-3
Try Deep Learning in 10 Lines of MATLAB Code . . . . . . . . . . . . . . . . . . . 1-12
Classify Image Using Pretrained Network . . . . . . . . . . . . . . . . . . . . . . . . . 1-14
Get Started with Transfer Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-16
Create Simple Image Classification Network . . . . . . . . . . . . . . . . . . . . . . 1-26
Create Simple Image Classification Network Using Deep Network
Designer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-29
Create Simple Sequence Classification Network Using Deep Network
Designer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-34
Shallow Networks for Pattern Recognition, Clustering and Time Series
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-41
Shallow Network Apps and Functions in Deep Learning Toolbox . . . . . . . 1-41
Deep Learning Toolbox Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-42
Shallow Neural Network Design Steps . . . . . . . . . . . . . . . . . . . . . . . . . . 1-43
Fit Data with a Shallow Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . 1-45
Defining a Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-45
Fit Data Using the Neural Net Fitting App . . . . . . . . . . . . . . . . . . . . . . . 1-45
Fit Data Using Command-Line Functions . . . . . . . . . . . . . . . . . . . . . . . . 1-54
Classify Patterns with a Shallow Neural Network . . . . . . . . . . . . . . . . . . . 1-63
Defining a Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-63
Classify Patterns Using the Neural Net Pattern Recognition App . . . . . . . 1-64
Classify Patterns Using Command-Line Functions . . . . . . . . . . . . . . . . . . 1-70
Cluster Data with a Self-Organizing Map . . . . . . . . . . . . . . . . . . . . . . . . . 1-77
Defining a Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-77
Cluster Data Using the Neural Net Clustering App . . . . . . . . . . . . . . . . . 1-77
Cluster Data Using Command-Line Functions . . . . . . . . . . . . . . . . . . . . . 1-83
Shallow Neural Network Time-Series Prediction and Modeling . . . . . . . 1-89
Time Series Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-89
Defining a Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-90
Fit Time Series Data Using the Neural Net Time Series App . . . . . . . . . . 1-90
v
Fit Time Series Data Using Command-Line Functions . . . . . . . . . . . . . . . 1-98
Train Shallow Networks on CPUs and GPUs . . . . . . . . . . . . . . . . . . . . . . 1-110
Parallel Computing Toolbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-110
Parallel CPU Workers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-110
GPU Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-111
Multiple GPU/CPU Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-111
Cluster Computing with MATLAB Parallel Server . . . . . . . . . . . . . . . . . 1-111
Load Balancing, Large Problems, and Beyond . . . . . . . . . . . . . . . . . . . . 1-112
Sample Data Sets for Shallow Neural Networks . . . . . . . . . . . . . . . . . . . 1-113
Shallow Neural Networks Glossary

vi Contents
1
Getting Started
• “Deep Learning Toolbox Product Description” on page 1-2
• “Get Started with Deep Network Designer” on page 1-3
• “Try Deep Learning in 10 Lines of MATLAB Code” on page 1-12
• “Classify Image Using Pretrained Network” on page 1-14
• “Get Started with Transfer Learning” on page 1-16
• “Create Simple Image Classification Network” on page 1-26
• “Create Simple Image Classification Network Using Deep Network Designer” on page 1-29
• “Create Simple Sequence Classification Network Using Deep Network Designer” on page 1-34
• “Shallow Networks for Pattern Recognition, Clustering and Time Series” on page 1-41
• “Fit Data with a Shallow Neural Network” on page 1-45
• “Classify Patterns with a Shallow Neural Network” on page 1-63
• “Cluster Data with a Self-Organizing Map” on page 1-77
• “Shallow Neural Network Time-Series Prediction and Modeling” on page 1-89
• “Train Shallow Networks on CPUs and GPUs” on page 1-110
• “Sample Data Sets for Shallow Neural Networks” on page 1-113
1
Getting Started
Deep Learning Toolbox Product Description
Design, train, and analyze deep learning networks
Deep Learning Toolbox provides a framework for designing and implementing deep neural networks
with algorithms, pretrained models, and apps. You can use convolutional neural networks (ConvNets,
CNNs) and long short-term memory (LSTM) networks to perform classification and regression on
image, time-series, and text data. You can build network architectures such as generative adversarial
networks (GANs) and Siamese networks using automatic differentiation, custom training loops, and
shared weights. With the Deep Network Designer app, you can design, analyze, and train networks
graphically. The Experiment Manager app helps you manage multiple deep learning experiments,
keep track of training parameters, analyze results, and compare code from different experiments. You
can visualize layer activations and graphically monitor training progress.
You can import networks and layer graphics from TensorFlow™ 2, TensorFlow-Keras, and PyTorch®,
the ONNX™ (Open Neural Network Exchange) model format, and Caffe. You can also export Deep
Learning Toolbox networks and layer graphs to TensorFlow 2 and the ONNX model format. The
toolbox supports transfer learning with DarkNet-53, ResNet-50, NASNet, SqueezeNet and many
other pretrained models.
You can speed up training on a single- or multiple-GPU workstation (with Parallel Computing
Toolbox™), or scale up to clusters and clouds, including NVIDIA® GPU Cloud and Amazon EC2® GPU
instances (with MATLAB® Parallel Server™).
1-2
Get Started with Deep Network Designer
Get Started with Deep Network Designer
This example shows how to use Deep Network Designer to adapt a pretrained GoogLeNet network to
classify a new collection of images. This process is called transfer learning and is usually much faster
and easier than training a new network, because you can apply learned features to a new task using a
smaller number of training images. To prepare a network for transfer learning interactively, use Deep
Network Designer.
Extract Data for Training
In the workspace, unzip the data.
unzip('MerchData.zip');
Select a Pretrained Network
Open Deep Network Designer.
deepNetworkDesigner
Load a pretrained GoogLeNet network by selecting it from the Deep Network Designer Start Page. If
you need to download the network, then click Install to open the Add-On Explorer.
Deep Network Designer displays a zoomed-out view of the whole network. Explore the network plot.
To zoom in with the mouse, use Ctrl+scroll wheel.
1-3
1
Getting Started
Load Data Set
To load the data into Deep Network Designer, on the Data tab, click Import Data > Import Image
Data. The Import Image Data dialog box opens.
In the Data source list, select Folder. Click Browse and select the extracted MerchData folder.
The dialog box also allows you to split the validation data from within the app. Divide the data into
70% training data and 30% validation data.
Specify augmentation operations to perform on the training images. For this example, apply a random
reflection in the x-axis, a random rotation from the range [-90,90] degrees, and a random rescaling
from the range [1,2].
1-4