Fundamentals of Machine Learning for Predictive Data Analytics

691 Pages·2015·14.29 MB·English

FUNDAMENTALS OF MACHINE LEARNING FOR PREDICTIVE
DATA ANALYTICS
FUNDAMENTALS OF MACHINE LEARNING FOR PREDICTIVE
DATA ANALYTICS
Algorithms, Worked Examples, and Case Studies
John D. Kelleher
Brian Mac Namee
Aoife D’Arcy
The MIT Press
Cambridge, Massachusetts
London, England
© 2015 Massachusetts Institute of Technology
All rights reserved. No part of this book may be reproduced in any form or by any
electronic or mechanical means (including photocopying, recording, or information
storage and retrieval) without permission in writing from the publisher.
MIT Press books may be purchased at special quantity discounts for business or sales
promotional use. For information, please email [email protected]
This book was set in the programming language by the author. Printed and bound in
the United States of America.
Library of Congress Cataloging-in-Publication Data
Kelleher, John D., 1974
Fundamentals of machine learning for predictive data analytics : algorithms, worked
examples, and case studies / John D. Kelleher, Brian Mac Namee, and Aoife DArcy.
pages cm
Includes bibliographical references and index.
ISBN 978-0-262-02944-5 (hardcover : alk. paper) 1. Machine learning. 2. Data mining. 3.
Prediction theory. I. Mac Namee, Brian, 1978 II. DArcy, Aoife, 1978 III. Title.
Q325.5.K455 2015
006.3’1—dc23
2014046123
10 9 8 7 6 5 4 3 2 1
To my wife and family,
thank you for your love, support, and patience.
John
To my family.
Brian
To Grandad D’Arcy, for the inspiration.
Aoife
Contents
Copyright
Preface
Structure of the Book
How to Use this Book
Online Resources
Acknowledgements
Notation
Notational Conventions
Notational Conventions for Probabilities
1 Machine Learning for Predictive Data Analytics
1.1 What Is Predictive Data Analytics?
1.2 What Is Machine Learning?
1.3 How Does Machine Learning Work?
1.4 What Can Go Wrong with Machine Learning?
1.5 The Predictive Data Analytics Project Lifecycle: CRISP-DM
1.6 Predictive Data Analytics Tools
1.7 The Road Ahead
1.8 Exercises
2 Data to Insights to Decisions
2.1 Converting Business Problems into Analytics Solutions
2.1.1 Case Study: Motor Insurance Fraud
2.2 Assessing Feasibility
2.2.1 Case Study: Motor Insurance Fraud
2.3 Designing the Analytics Base Table
2.3.1 Case Study: Motor Insurance Fraud
2.4 Designing and Implementing Features
2.4.1 Different Types of Data
2.4.2 Different Types of Features
2.4.3 Handling Time
2.4.4 Legal Issues
2.4.5 Implementing Features
2.4.6 Case Study: Motor Insurance Fraud
2.5 Summary
2.6 Further Reading
2.7 Exercises
3 Data Exploration
3.1 The Data Quality Report
3.1.1 Case Study: Motor Insurance Fraud
3.2 Getting to Know the Data
3.2.1 The Normal Distribution
3.2.2 Case Study: Motor Insurance Fraud
3.3 Identifying Data Quality Issues
3.3.1 Missing Values
3.3.2 Irregular Cardinality
3.3.3 Outliers
3.3.4 Case Study: Motor Insurance Fraud
3.4 Handling Data Quality Issues
3.4.1 Handling Missing Values
3.4.2 Handling Outliers
3.4.3 Case Study: Motor Insurance Fraud
3.5 Advanced Data Exploration
3.5.1 Visualizing Relationships Between Features
3.5.2 Measuring Covariance and Correlation
3.6 Data Preparation
3.6.1 Normalization
3.6.2 Binning
3.6.3 Sampling
3.7 Summary
3.8 Further Reading
3.9 Exercises
4 Information-based Learning
4.1 Big Idea
4.2 Fundamentals
4.2.1 Decision Trees
4.2.2 Shannon’s Entropy Model
4.2.3 Information Gain
4.3 Standard Approach: The ID3 Algorithm
4.3.1 A Worked Example: Predicting Vegetation Distributions
4.4 Extensions and Variations
4.4.1 Alternative Feature Selection and Impurity Metrics
4.4.2 Handling Continuous Descriptive Features
4.4.3 Predicting Continuous Targets
4.4.4 Tree Pruning
4.4.5 Model Ensembles
4.5 Summary
4.6 Further Reading
4.7 Exercises
5 Similarity-based Learning
5.1 Big Idea
5.2 Fundamentals
5.2.1 Feature Space
5.2.2 Measuring Similarity Using Distance Metrics
5.3 Standard Approach: The Nearest Neighbor Algorithm
5.3.1 A Worked Example
5.4 Extensions and Variations
5.4.1 Handling Noisy Data
5.4.2 Efficient Memory Search
5.4.3 Data Normalization
5.4.4 Predicting Continuous Targets
5.4.5 Other Measures of Similarity
5.4.6 Feature Selection
5.5 Summary
5.6 Further Reading
5.7 Epilogue
5.8 Exercises
6 Probability-based Learning
6.1 Big Idea
6.2 Fundamentals
6.2.1 Bayes’ Theorem
6.2.2 Bayesian Prediction
6.2.3 Conditional Independence and Factorization
6.3 Standard Approach: The Naive Bayes Model
6.3.1 A Worked Example
6.4 Extensions and Variations
6.4.1 Smoothing
6.4.2 Continuous Features: Probability Density Functions
6.4.3 Continuous Features: Binning
6.4.4 Bayesian Networks
6.5 Summary
6.6 Further Reading
6.7 Exercises
7 Error-based Learning
7.1 Big Idea
7.2 Fundamentals
7.2.1 Simple Linear Regression
7.2.2 Measuring Error
7.2.3 Error Surfaces
7.3 Standard Approach: Multivariable Linear Regression with Gradient Descent
7.3.1 Multivariable Linear Regression
7.3.2 Gradient Descent
7.3.3 Choosing Learning Rates and Initial Weights
7.3.4 A Worked Example
7.4 Extensions and Variations
7.4.1 Interpreting Multivariable Linear Regression Models
7.4.2 Setting the Learning Rate Using Weight Decay
7.4.3 Handling Categorical Descriptive Features
7.4.4 Handling Categorical Target Features: Logistic Regression
7.4.5 Modeling Non-linear Relationships
7.4.6 Multinomial Logistic Regression
7.4.7 Support Vector Machines
7.5 Summary
7.6 Further Reading
7.7 Exercises
8 Evaluation
8.1 Big Idea
8.2 Fundamentals
8.3 Standard Approach: Misclassification Rate on a Hold-out Test Set
8.4 Extensions and Variations
8.4.1 Designing Evaluation Experiments
8.4.2 Performance Measures: Categorical Targets
8.4.3 Performance Measures: Prediction Scores
8.4.4 Performance Measures: Multinomial Targets
8.4.5 Performance Measures: Continuous Targets
8.4.6 Evaluating Models after Deployment
8.5 Summary
8.6 Further Reading
8.7 Exercises
9 Case Study: Customer Churn
9.1 Business Understanding
9.2 Data Understanding
9.3 Data Preparation