Artificial Intelligence for .NET: Speech, Language, and Search

278 Pages·2017·8.4 MB·English

by  Nishith Pathak

Artificial
Intelligence for .NET:
Speech, Language,
and Search
Building Smart Applications with Microsoft
Cognitive Services APIs
—
Nishith Pathak
Artificial Intelligence

for .NET: Speech,

Language, and Search

Building Smart Applications with Microsoft

Cognitive Services APIs

Nishith Pathak
With Contributing Author as Anurag Bhandari
Artificial Intelligence for .NET: Speech, Language, and Search
Nishith Pathak
Kotdwara, Dist. Pauri Garhwal, India
ISBN-13 (pbk): 978-1-4842-2948-4 ISBN-13 (electronic): 978-1-4842-2949-1
DOI 10.1007/978-1-4842-2949-1
Library of Congress Control Number: 2017951713
Copyright © 2017 by Nishith Pathak
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the
material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage
and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or
hereafter developed.
Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with
every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an
editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are
not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to
proprietary rights.
While the advice and information in this book are believed to be true and accurate at the date of publication,
neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or
omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material
contained herein.
Cover image designed by Freepik
Managing Director: Welmoed Spahr
Editorial Director: Todd Green
Acquisitions Editor: Gwenan Spearing
Development Editor: Laura Berendson
Technical Reviewer: Fabio Claudio Ferracchiati
Coordinating Editor: Nancy Chen
Copy Editor: Mary Behr
Artist : SPi Global
Distributed to the book trade worldwide by Springer Science+Business Media New York,

233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax [phone], e-mail

[email protected], or visit www.springeronline.com. Apress Media, LLC is a California LLC

and the sole member (owner) is Springer Science + Business Media Finance Inc (SSBM Finance Inc).

SSBM Finance Inc is a Delaware corporation.

For information on translations, please e-mail [email protected], or visit www.apress.com/

rights-permissions.

Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBook versions

and licenses are also available for most titles. For more information, reference our Print and eBook Bulk

Sales web page at www.apress.com/bulk-sales.

Any source code or other supplementar y material referenced by the author in this book is available to

readers on GitHub via the book’s product page, located at www.apress.com/9781484229484. For more

detailed information, please visit www.apress.com/source-code.

Printed on acid-free paper
To the most important person in my life, my mother, late Bina Pathak, for her guidance,

sacrifices, prayers, and blessings, which made me What I am today. I miss her each day. To my

father, Pankaj Pathak, for teaching me to do what I believe in. You are and will always be my role

model and my hero for my entire life. To my Sadh-Gurudev, who has been an eternal guiding

force and entirely changed my life. To my grandfather, the late Mahesh Chandra Pathak,

for his blessings and moral values.

To my wife, Surabhi, for bearing with me, sacrificing her splendid career for our family, and

always staying by my side through all the ups and downs. Getting married to you is

the most beautiful thing in my life. You have given me the most precious diamond of my life,

Shikhar, whom I love more than anyone else. I know this book has taken a lot of me and I

haven’t been able to spend enough time with you, Papa and Shikhar for the past year since

I’ve been working tirelessly to give this pleasant surprise. Surabhi and Shikhar, this book would not

have been possible without all your sacrifices.

To my lovely sister, Tanwi, and my niece, Aadhya—your smiling faces give me a lot of strength

and inspiration to do better each day. To my Guruji, JP Kukreti, SS Tyagi, and Rajesh Tripathi,

who have been there for me countless times and always provide me with comfort,

understanding, spiritual beliefs, and lots of motivation.

Lastly, I thank God for blessing me with such wonderful people in my life.
Contents at a Glance

About the Author .....................................................................................................xv

About the Contributing Author ...............................................................................xvii

About the Technical Reviewer ................................................................................xix

Acknowledgments ..................................................................................................xxi

Introduction ..........................................................................................................xxiii

■ Chapter 1: Getting Started with AI Basics ............................................................. 1

■ Chapter 2: Creating an AI-Based Application in Visual Studio ............................ 23

■ Chapter 3: Building a Conversational User Interface with

Microsoft Technologies........................................................................................ 45

■ Chapter 4: Using Natural Language Understanding ............................................. 71

■ Chapter 5: Exploring a Cognitive Language Model .............................................. 93

■ Chapter 6: Consuming and Applying LUIS ......................................................... 131

■ Chapter 7: Interacting with the Speech API ...................................................... 161

■ Chapter 8: Applying Search Offerings ............................................................... 193

■ Chapter 9: Working with Recommendations ..................................................... 221

■ Chapter 10: The Future of AI .............................................................................. 247

Index ..................................................................................................................... 261

v
Contents

About the Author .....................................................................................................xv

About the Contributing Author ...............................................................................xvii

About the Technical Reviewer ................................................................................xix

Acknowledgments ..................................................................................................xxi

Introduction ..........................................................................................................xxiii

■ Chapter 1: Getting Started with AI Basics ............................................................. 1

Truth vs. Fiction ................................................................................................................ 2

History and Evolution ....................................................................................................... 3

The Current State of Affairs .................................................................................................................... 4

Commoditization of AI ............................................................................................................................. 4

Microsoft and AI ............................................................................................................... 5

Basic Concepts ................................................................................................................. 6

Machine Learning ................................................................................................................................... 9

Language .............................................................................................................................................. 12

Speech .................................................................................................................................................. 13

Computer Vision .................................................................................................................................... 14

Microsoft’s Cognitive Services ....................................................................................... 17

Vision .................................................................................................................................................... 18

Speech .................................................................................................................................................. 18

Language .............................................................................................................................................. 19

Knowledge ............................................................................................................................................ 19

Search .................................................................................................................................................. 20

Recap ............................................................................................................................. 21

vii
■ CONTENTS
■ Chapter 2: Creating an AI-Based Application in Visual Studio ............................ 23

Prerequisites for Using Cognitive Services .................................................................... 24

Setting Up the Development Environment ..................................................................... 24

Getting an Azure Subscription Key for Cognitive Services ............................................. 24

Step 1: Set Up an Azure Account .......................................................................................................... 25

Step 2: Create a New Cognitive Services Account ................................................................................ 27

Step 3: Get the Subscription Key(s) ...................................................................................................... 29

Testing the API ................................................................................................................ 30

What You Want To Do ............................................................................................................................ 31

How To Do It .......................................................................................................................................... 31

Creating Your First AI-based Application ........................................................................ 33

The Code ............................................................................................................................................... 34

The Walkthrough ................................................................................................................................... 36

The Result ............................................................................................................................................. 38

Making Your Application More Interesting ...................................................................... 39

Extracting Text Out of Images ............................................................................................................... 39

The Code ............................................................................................................................................... 39

The Walkthrough ................................................................................................................................... 41

The Result ............................................................................................................................................. 42

Recap ............................................................................................................................. 43

■ Chapter 3: Building a Conversational User Interface with

Microsoft Technologies........................................................................................ 45

What Is a Conversational User Interface? ...................................................................... 47

A Brief History ................................................................................................................ 47

In the Very Beginning: the Command-Line Interface (CLI) ................................................................... 47

And Then Came the Graphical User Interface ....................................................................................... 49

And UI Evolved Yet Again: Conversational User Interface ..................................................................... 51

AI’s Role in CUI ...................................................................................................................................... 52

Pitfalls of CUI ........................................................................................................................................ 53

A Hybrid UI (CUI+GUI) May Be the Future ............................................................................................. 55

viii
■ CONTENTS
Design Principles ............................................................................................................ 57

Microsoft Bot Framework ............................................................................................... 58

Creating a CUI Application Using Bot Framework .......................................................... 58

Step 0: Prep Your Development Environment ....................................................................................... 59

Step 1: Create a New Bot Application Project ....................................................................................... 60

Step 2: First and Default Messages ...................................................................................................... 60

Step 3: Running, Testing, and Debugging Your Bot ............................................................................... 62

Step 3: Appointment Scheduling .......................................................................................................... 65

Step 4: Handling System Messages ..................................................................................................... 68

Next Steps ............................................................................................................................................ 68

Recap ............................................................................................................................. 69

■ Chapter 4: Using Natural Language Understanding ............................................. 71

What Is NLU? .................................................................................................................. 72

History of Natural Language Understanding .................................................................. 74

Why Natural Language Is Difficult for Machines to Understand ..................................... 77

Complexities in Natural Language ........................................................................................................ 77

Statistical Models as a Solution Are Insufficient .................................................................................. 79

A Promising Future ............................................................................................................................... 80

Language Understanding Intelligent Service (LUIS) ....................................................... 80

Architecture of a LUIS-Based Software Application ............................................................................. 81

Behind the Scenes ................................................................................................................................ 84

Extensive Training Is the Key ................................................................................................................ 85

Getting an Azure Subscription for LUIS .......................................................................... 86

Getting Subscription Keys on Azure Portal ........................................................................................... 87

Applying Subscription Keys in LUIS ...................................................................................................... 88

Demo: Definition App ...................................................................................................... 89

Notes .................................................................................................................................................... 91

Recap ............................................................................................................................. 92

ix
■ CONTENTS
■ Chapter 5: Exploring a Cognitive Language Model .............................................. 93

The Bing Spell Check API ............................................................................................... 93

What Is It? ............................................................................................................................................. 95

How To Use It ........................................................................................................................................ 96

Integration with LUIS ............................................................................................................................ 99

The Text Analytics API ................................................................................................... 101

Language Detection ............................................................................................................................ 102

Key Phrase Extraction ......................................................................................................................... 105

Sentiment Analysis ............................................................................................................................. 108

Topic Detection ................................................................................................................................... 110

Usage Ideas ........................................................................................................................................ 113

The Web Language Model (WebLM) API ....................................................................... 114

Word Breaking .................................................................................................................................... 116

Joint Probability .................................................................................................................................. 117

Conditional Probability ........................................................................................................................ 119

Next Word Prediction .......................................................................................................................... 120

The Linguistic Analysis API ........................................................................................... 121

Sentence Separation and Tokenization ............................................................................................... 122

Part-of-Speech Tagging ...................................................................................................................... 125

Constituency Parsing .......................................................................................................................... 127

Recap ........................................................................................................................... 130

■ Chapter 6: Consuming and Applying LUIS ......................................................... 131

Planning Your App ........................................................................................................ 131

What Should the Bot Be Able to Do? ................................................................................................... 132

What Information Does the Bot Need from the User? ......................................................................... 132

What Should Be Done in LUIS? ........................................................................................................... 132

What Should Be Done in the Bot Framework? .................................................................................... 134

Creating a LUIS App ...................................................................................................... 134

Adding Intents .............................................................................................................. 135

Adding/Labeling Utterances ......................................................................................... 135

x
■ CONTENTS
Publishing Your App ...................................................................................................... 137

Adding Entities ............................................................................................................. 139

Simple Entities .................................................................................................................................... 141

Composite Entities .............................................................................................................................. 142

Hierachical Entities ............................................................................................................................. 145

Prebuilt Entities .................................................................................................................................. 147

Adding a Phrase List .................................................................................................... 149

Suggested Next Steps .................................................................................................. 150

Active Learning Through Suggested Utterances ................................................................................ 150

Using LUIS Programmatic API for Automation .................................................................................... 151

Integrating LUIS with the Bot Framework .................................................................... 151

Creating a Project in Visual Studio...................................................................................................... 151

Handling an Entity-less Intent ............................................................................................................ 152

Setting Up Your Bot to Use HealthCheckupDialog ............................................................................... 153

Testing the Bot in an Emulator ........................................................................................................... 153

Handling an Entity-Full Intent ............................................................................................................. 154

Handling an Intent with Composite Entities ........................................................................................ 156

Handling the None Intent .................................................................................................................... 158

Adding Your Bot to Skype ............................................................................................. 158

Publishing Your Bot ............................................................................................................................. 158

Registering Your Bot ........................................................................................................................... 159

Recap ........................................................................................................................... 160

■ Chapter 7: Interacting with the Speech API ...................................................... 161

Ways to Interact with Speech ....................................................................................... 162

The Cognitive Search API .................................................................................................................... 163

Speech Recognition ............................................................................................................................ 164

Getting Started ............................................................................................................. 164

Getting the JSON Web Token First ................................................................................ 164

The Consume Speech API ............................................................................................. 166

xi