Just Enough R!: An Interactive Approach to Machine Learning and Analytics

365 Pages·2020·9.565 MB·English

by  Richard J. Roiger

Just Enough R!
.
Taylor & Francis
(
Taylor & Francis Group
http://taylorandfrancis.com
Just Enough R!
An Interactive Approach to Machine
Learning and Analytics
Richard J. Roiger
Professor Emeritus, Computer Information Science Department,
Minnesota State University, Mankato
First edition published 2020
by CRC Press
6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742
and by CRC Press
2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN
© 2020 Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, LLC
Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot
assume responsibility for the validity of all materials or the consequences of their use. The authors and p ublishers
have attempted to trace the copyright holders of all material reproduced in this publication and apologize to
copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been
acknowledged please write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or
utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including
photocopying, microfilming, and recording, or in any information storage or retrieval system, without written
permission from the publishers.
For permission to photocopy or use material electronically from this work, access www.copyright.com or contact the
Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. For works that are
not available on CCC please contact [email protected]
Trademark notice: Product or corporate names may be trademarks or registered trademarks, and are used only for
identification and explanation without intent to infringe.
Library of Congress Cataloging-in-Publication Data
Names: Roiger, Richard J., author.
Title: Just enough R! : an interactive approach to machine learning and
analytics / Richard J. Roiger, Professor Emeritus, Computer Information
Science Department, Minnesota State University, Mankato.
Description: Boca Raton : CRC Press, 2020. | Includes bibliographical
references and index.
Identifiers: LCCN 2020009226 | ISBN 9780367439149 (paperback) |
ISBN 9780367443207 (hardback) | ISBN 9781003006695 (ebook)
Subjects: LCSH: R (Computer program language) | Machine learning. |
Data structures (Computer science) | Mathematical statistics—Data processing.
Classification: LCC QA276.45.R3 R65 2020 | DDC 005.13/3—dc23
LC record available at https://lccn.loc.gov/2020009226
ISBN: 978-0-367-44320-7 (hbk)
ISBN: 978-0-367-43914-9 (pbk)
ISBN: 978-1-003-00669-5 (ebk)
Typeset in Minion
by codeMantra
Contents
Preface, xiii
Acknowledgment, xv
Author, xvii
Chapter 1 ◾ Introduction to Machine Learning 1
1.1 MACHINE LEARNING, STATISTICAL ANALYSIS, AND DATA SCIENCE 2
1.2 MACHINE LEARNING: A FIRST EXAMPLE 3
1.2.1 Attribute-Value Format 3
1.2.2 A Decision Tree for Diagnosing Illness 4
1.3 MACHINE LEARNING STRATEGIES 6
1.3.1 Classification 7
1.3.2 Estimation 7
1.3.3 Prediction 8
1.3.4 Unsupervised Clustering 11
1.3.5 Market Basket Analysis 12
1.4 EVALUATING PERFORMANCE 12
1.4.1 Evaluating Supervised Models 12
1.4.2 Two-Class Error Analysis 13
1.4.3 Evaluating Numeric Output 14
1.4.4 Comparing Models by Measuring Lift 15
1.4.5 Unsupervised Model Evaluation 17
1.5 ETHICAL ISSUES 17
1.6 CHAPTER SUMMARY 18
1.7 KEY TERMS 18
EXERCISES 20
v
vi ◾ Contents
Chapter 2 ◾ Introduction to R 25
2.1 INTRODUCING R AND RSTUDIO 25
2.1.1 Features of R 26
2.1.2 Installing R 26
2.1.3 Installing RStudio 28
2.2 NAVIGATING RSTUDIO 28
2.2.1 The Console 28
2.2.2 The Source Panel 30
2.2.3 The Global Environment 32
2.2.4 Packages 37
2.3 WHERE’S THE DATA? 38
2.4 OBTAINING HELP AND ADDITIONAL INFORMATION 38
2.5 SUMMARY 39
EXERCISES 39
Chapter 3 ◾ Data Structures and Manipulation 41
3.1 DATA TYPES 41
3.1.1 Character Data and Factors 42
3.2 SINGLE-MODE DATA STRUCTURES 44
3.2.1 Vectors 44
3.2.2 Matrices and Arrays 46
3.3 MULTIMODE DATA STRUCTURES 47
3.3.1 Lists 47
3.3.2 Data Frames 48
3.4 WRITING YOUR OWN FUNCTIONS 50
3.4.1 Writing a Simple Function 50
3.4.2 Conditional Statements 52
3.4.3 Iteration 53
3.4.4 Recursive Programming 57
3.5 SUMMARY 58
3.6 KEY TERMS 58
EXERCISES 59
Chapter 4 ◾ Preparing the Data 61
4.1 A PROCESS MODEL FOR KNOWLEDGE DISCOVERY 61
4.2 CREATING A TARGET DATASET 62
Contents ◾ vii
4.2.1 Interfacing R with the Relational Model 64
4.2.2 Additional Sources for Target Data 66
4.3 DATA PREPROCESSING 66
4.3.1 Noisy Data 66
4.3.2 Preprocessing with R 67
4.3.3 Detecting Outliers 69
4.3.4 Missing Data 69
4.4 DATA TRANSFORMATION 70
4.4.1 Data Normalization 70
4.4.2 Data Type Conversion 72
4.4.3 Attribute and Instance Selection 72
4.4.4 Creating Training and Test Set Data 74
4.4.5 Cross Validation and Bootstrapping 74
4.4.6 Large-Sized Data 75
4.5 CHAPTER SUMMARY 75
4.6 KEY TERMS 76
EXERCISES 77
Chapter 5 ◾ Supervised Statistical Techniques 79
5.1 SIMPLE LINEAR REGRESSION 79
5.2 MULTIPLE LINEAR REGRESSION 85
5.2.1 Multiple Linear Regression: An Example 85
5.2.2 Evaluating Numeric Output 88
5.2.3 Training/Test Set Evaluation 89
5.2.4 Using Cross Validation 91
5.2.5 Linear Regression with Categorical Data 93
5.3 LOGISTIC REGRESSION 99
5.3.1 Transforming the Linear Regression Model 100
5.3.2 The Logistic Regression Model 100
5.3.3 Logistic Regression with R 101
5.3.4 Creating a Confusion Matrix 104
5.3.5 Receiver Operating Characteristics (ROC) Curves 104
5.3.6 The Area under an ROC Curve 108
5.4 NAÏVE BAYES CLASSIFIER 109
5.4.1 Bayes Classifier: An Example 109
5.4.2 Zero-Valued Attribute Counts 112
viii ◾ Contents
5.4.3 Missing Data 112
5.4.4 Numeric Data 113
5.4.5 Experimenting with Naïve Bayes 115
5.5 CHAPTER SUMMARY 120
5.6 KEY TERMS 120
EXERCISES 122
Chapter 6 ◾ Tree-Based Methods 127
6.1 A DECISION TREE ALGORITHM 127
6.1.1 An Algorithm for Building Decision Trees 128
6.1.2 C4.5 Attribute Selection 128
6.1.3 Other Methods for Building Decision Trees 133
6.2 BUILDING DECISION TREES: C5.0 133
6.2.1 A Decision Tree for Credit Card Promotions 134
6.2.2 Data for Simulating Customer Churn 135
6.2.3 Predicting Customer Churn with C5.0 136
6.3 BUILDING DECISION TREES: RPART 137
6.3.1 An rpart Decision Tree for Credit Card Promotions 139
6.3.2 Train and Test rpart: Churn Data 141
6.3.3 Cross Validation rpart: Churn Data 143
6.4 BUILDING DECISION TREES: J48 147
6.5 ENSEMBLE TECHNIQUES FOR IMPROVING PERFORMANCE 149
6.5.1 B agging 149
6.5.2 B oosting 150
6.5.3 B oosting: An Example with C5.0 150
6.5.4 Random Forests 151
6.6 R EGRESSION TREES 154
6.7 CHAPTER SUMMARY 156
6.8 KEY TERMS 157
EXERCISES 157
Chapter 7 ◾ Rule-Based Techniques 161
7.1 FROM TREES TO RULES 161
7.1.1 The Spam Email Dataset 162
7.1.2 Spam Email Classification: C5.0 163
7.2 A BASIC COVERING RULE ALGORITHM 165
Contents ◾ ix
7.2.1 Generating Covering Rules with JRip 166
7.3 GENERATING ASSOCIATION RULES 169
7.3.1 Confidence and Support 169
7.3.2 Mining Association Rules: An Example 170
7.3.3 General Considerations 173
7.3.4 Rweka’s Apriori Function 173
7.4 SHAKE, RATTLE, AND ROLL 177
7.5 CHAPTER SUMMARY 184
7.6 KEY TERMS 184
EXERCISES 185
Chapter 8 ◾ Neural Networks 189
8.1 FEED-FORWARD NEURAL NETWORKS 190
8.1.1 Neural Network Input Format 190
8.1.2 Neural Network Output Format 192
8.1.3 The Sigmoid Evaluation Function 193
8.2 NEURAL NETWORK TRAINING: A CONCEPTUAL VIEW 194
8.2.1 Supervised Learning with Feed-Forward Networks 194
8.2.2 Unsupervised Clustering with Self-Organizing Maps 195
8.3 NEURAL NETWORK EXPLANATION 196
8.4 GENERAL CONSIDERATIONS 197
8.4.1 Strengths 197
8.4.2 Weaknesses 198
8.5 NEURAL NETWORK TRAINING: A DETAILED VIEW 198
8.5.1 The Backpropagation Algorithm: An Example 198
8.5.2 Kohonen Self-Organizing Maps: An Example 202
8.6 BUILDING NEURAL NETWORKS WITH R 203
8.6.1 The Exclusive-OR Function 204
8.6.2 Modeling Exclusive-OR with MLP: Numeric Output 206
8.6.3 Modeling Exclusive-OR with MLP: Categorical Output 210
8.6.4 Modeling Exclusive-OR with neuralnet: Numeric Output 212
8.6.5 Modeling Exclusive-OR with neuralnet: Categorical Output 214
8.6.6 Classifying Satellite Image Data 216
8.6.7 Testing for Diabetes 220
8.7 NEURAL NET CLUSTERING FOR ATTRIBUTE EVALUATION 223
8.8 TIMES SERIES ANALYSIS 227