Table Of ContentThe
Applied AI and
Natural Language
Processing
Workshop
Explore practical ways to transform your simple
projects into powerful intelligent applications
Krishna Sankar, Jeffrey Jackovich, and Ruze Richards
The Applied AI and Natural Language Processing Workshop
Copyright © 2020 Packt Publishing
All rights reserved. No part of this course may be reproduced, stored in a retrieval
system, or transmitted in any form or by any means, without the prior written
permission of the publisher, except in the case of brief quotations embedded in
critical articles or reviews.
Every effort has been made in the preparation of this course to ensure the accuracy
of the information presented. However, the information contained in this course
is sold without warranty, either express or implied. Neither the authors, nor Packt
Publishing, and its dealers and distributors will be held liable for any damages caused
or alleged to be caused directly or indirectly by this course.
Packt Publishing has endeavored to provide trademark information about all of the
companies and products mentioned in this course by the appropriate use of capitals.
However, Packt Publishing cannot guarantee the accuracy of this information.
Authors: Krishna Sankar, Jeffrey Jackovich, and Ruze Richards
Reviewers: Ridhima Garg, Sasikant Kotti, Ankit Malik, Sagnik Pal, Robert Ridley,
and Dr. Priyanka Singh
Managing Editor: Pournami Jois
Acquisitions Editors: Sneha Shinde, Anindya Sil, Archie Vankar, and Karan Wadekar
Production Editor: Shantanu Zagade
Editorial Board: Megan Carlisle, Mahesh Dhyani, Heather Gopsill, Manasa Kumar,
Alex Mazonowicz, Bridget Neale, Dominic Pereira, Shiny Poojary, Abhishek Rane,
Brendan Rodrigues, Erol Staveley, Ankita Thakur, Nitesh Thakur, and Jonathan Wray
First published: July 2020
Production reference: 1240720
ISBN: 978-1-80020-874-2
Published by Packt Publishing Ltd.
Livery Place, 35 Livery Street
Birmingham B3 2PB, UK
Table of Contents
Preface i
Chapter 1: An Introduction to AWS 1
Introduction .............................................................................................. 2
How Is AWS Special? ................................................................................ 3
What Is ML? ................................................................................................... 3
What Is AI? .................................................................................................... 4
What Is Amazon S3? ................................................................................. 4
Why Use S3? .................................................................................................. 5
The Basics of Working on AWS with S3 ..................................................... 5
AWS Free-Tier Account ................................................................................ 5
AWS Account Setup and Navigation ...................................................................6
Downloading the Support Materials for This Book ................................. 6
A Word about Jupyter Notebooks .............................................................. 7
Importing and Exporting Data into S3 ....................................................... 7
How S3 Differs from a Filesystem .............................................................. 8
Core S3 Concepts ..................................................................................... 8
S3 Operations ............................................................................................. 10
Data Replication .................................................................................... 10
The REST Interface ..................................................................................... 11
Exercise 1.01: Using the AWS Management Console
to Create an S3 Bucket .............................................................................. 11
Exercise 1.02: Importing and Exporting the File
with Your S3 Bucket ................................................................................... 16
The AWS CLI ........................................................................................... 21
Exercise 1.03: Configuring the CLI ............................................................ 22
CLI Usage ................................................................................................ 26
Recursion and Parameters .................................................................. 27
Activity 1.01: Putting the Data into S3 with the CLI ............................... 28
Using the AWS Console to Identify ML Services ................................ 29
Exercise 1.04: Navigating the AWS Management Console .................... 29
Exercise 1.05: Testing the Amazon Comprehend API Features ............ 31
The Utility of the AWS Console Interface to AI Services ........................ 39
Summary ................................................................................................ 39
Chapter 2: Analyzing Documents and Text
with Natural Language Processing 41
Introduction ........................................................................................... 42
Serverless Computing .......................................................................... 46
Amazon Lambda and Function as a Service ........................................... 47
Serverless Computing as an Approach ................................................... 48
Amazon Comprehend .......................................................................... 48
What Is an NLP Service? ....................................................................... 50
Using Amazon Comprehend to Inspect Text
and Determine the Primary Language ............................................... 51
Exercise 2.01: Detecting the Dominant Language
in a Text Document Using the Command-Line Interface ..................... 54
Exercise 2.02: Detecting the Dominant Language
in Multiple Documents by Using the CLI ................................................. 57
Extracting Information from a Set of Documents ............................ 60
Detecting Named Entities—AWS SDK for Python (boto3) ..................... 60
DetectEntities – Input and Output ........................................................... 61
Exercise 2.03: Determining the Named Entities
in a Document (the DetectEntities method) ........................................... 62
Exercise 2.04: Detecting Entities in a Set
of Documents (Text Files) ......................................................................... 64
Detecting Key Phrases ............................................................................... 66
Exercise 2.05: Detecting Key Phrases ...................................................... 67
Detecting Sentiments ................................................................................ 69
Exercise 2.06: Conducting Sentiment Analysis ....................................... 69
Setting Up a Lambda Function and Analyzing
Imported Text Using Comprehend ..................................................... 71
Integrating Comprehend and AWS Lambda for responsive NLP ......... 71
What Is AWS Lambda? ............................................................................... 71
What Does AWS Lambda Do? ................................................................... 71
Lambda Function Anatomy ...................................................................... 72
Exercise 2.07: Setting Up a Lambda Function for S3 ............................. 73
Exercise 2.08: Assigning Policies to S3_trigger
to Access Comprehend .............................................................................. 92
Activity 2.01: Integrating Lambda with Amazon
Comprehend to Perform Text Analysis ................................................... 96
Amazon Textract ................................................................................... 97
Exercise 2.09: Extracting Tax Information Using Amazon Textract ..... 98
Summary .............................................................................................. 103
Chapter 3: Topic Modeling and Theme Extraction 105
Introduction ......................................................................................... 106
Topic Modeling with Latent Dirichlet Allocation (LDA) .................. 106
Basic LDA Example ................................................................................... 106
Why Use LDA? ........................................................................................... 108
Amazon Comprehend—Topic Modeling Guidelines ....................... 108
Exercise 3.01: Using Amazon Comprehend to Perform
Topic Modeling on Two Documents with Known Topics .................... 111
Exercise 3.02: Performing Known Structure
Analysis Programmatically ..................................................................... 135
Activity 3.01: Performing Topic Modeling on a Set
of Documents with Unknown Topics .................................................... 141
Summary .............................................................................................. 143
Chapter 4: Conversational Artificial Intelligence 145
Introduction to Conversational AI .................................................... 146
Interaction Types ..................................................................................... 147
Omnichannel ............................................................................................ 147
What Is a Chatbot? .............................................................................. 147
What Is Natural Language Understanding? .................................... 148
Core Concepts in a Nutshell ................................................................... 148
Chatbot ..............................................................................................................148
Utterances .........................................................................................................148
Intent ..................................................................................................................149
Prompts .............................................................................................................149
Slot ......................................................................................................................150
Fulfillment..........................................................................................................151
Best Practices for Designing Conversational AI .............................. 152
Creating a Custom Chatbot ............................................................... 154
A Bot That Recognizes an Intent and Filling a Slot ......................... 156
Exercise 4.01: Creating a Bot That Will Recognize
an Intent and Fill a Slot .......................................................................... 156
Natural Language Understanding Engine ............................................. 170
Lambda Function – Implementing Business Logic ......................... 172
Exercise 4.02: Creating a Lambda Function
to Handle Chatbot Fulfillment ................................................................ 172
Implementing the Lambda Function ..................................................... 175
Input Parameter Structure ..................................................................... 176
Implementing the High-Level Handler Function .................................. 177
Implementing the Function to Retrieve the Market Quote ................ 178
Returning the Information to the Calling App (the Chatbot) ............. 180
Connecting to the Chatbot ..................................................................... 181
Debugging Tips ......................................................................................... 183
Summary .............................................................................................. 186
Chapter 5: Using Speech with the Chatbot 189
Amazon Connect Basics ..................................................................... 190
Free Tier Information .............................................................................. 191
Interacting with the Chatbot ............................................................. 191
Talking to Your Chatbot through a Call Center
Using Amazon Connect ...................................................................... 192
Exercise 5.01: Creating a Personal Call Center ..................................... 193
Exercise 5.02: Obtaining a Free Phone Number
for Your Call Center ................................................................................. 199
Using Amazon Lex Chatbots with Amazon Connect ...................... 201
Understanding Contact Flows ................................................................ 202
Contact Flow Templates .......................................................................... 202
Exercise 5.03: Connecting the Call Center to Your Lex Chatbot ......... 205
Activity 5.01: Creating a Custom Bot and Connecting
the Bot with Amazon Connect ................................................................ 218
Summary .............................................................................................. 220
Chapter 6: Computer Vision and Image Processing 223
Introduction ......................................................................................... 224
Amazon Rekognition Basics ............................................................... 224
Free Tier Information on Amazon Rekognition .................................... 225
Rekognition and Deep Learning ....................................................... 226
Detecting Objects and Scenes in Images .............................................. 226
Exercise 6.01: Detecting Objects and Scenes
Using Your Images ................................................................................... 232
Image Moderation .............................................................................. 236
Exercise 6.02: Detecting Objectionable Content in Images ................ 239
Facial Analysis ..................................................................................... 241
Exercise 6.03: Analyzing Faces with Your Own Images ....................... 246
Celebrity Recognition ......................................................................... 252
Exercise 6.04: Recognizing Celebrities in Your Images ........................ 256
Face Comparison ................................................................................ 260
Activity 6.01: Creating and Analyzing Different
Faces in Rekognition ................................................................................ 263
Text in Images ..................................................................................... 266
Exercise 6.05: Extracting Text from Your Own Images ........................ 268
Summary .............................................................................................. 272
Appendix 275
Index 353
