Table Of ContentInterpretable
Machine Learning
with Python
Learn to build interpretable high-performance
models with hands-on real-world examples
Serg Masís
BIRMINGHAM—MUMBAI
Interpretable Machine Learning with Python
Copyright © 2021 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or
transmitted in any form or by any means, without the prior written permission of the publisher,
except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the
information presented. However, the information contained in this book is sold without warranty,
either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors,
will be held liable for any damages caused or alleged to have been caused directly or indirectly by
this book.
Packt Publishing has endeavored to provide trademark information about all of the companies
and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing
cannot guarantee the accuracy of this information.
Group Product Manager: Kunal Parikh
Publishing Product Manager: Sunith Shetty
Acquisition Editor: Reshma Raman
Senior Editor: Roshan Kumar
Content Development Editors: Sean Lobo and Joseph Sunil
Technical Editor: Sonam Pandey
Copy Editor: Safis Editing
Project Coordinator: Aishwarya Mohan
Proofreader: Safis Editing
Indexer: Priyanka Dhadke
Production Designer: Roshan Kawale
First published: March 2021
Production reference: 1250321
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham
B3 2PB, UK.
ISBN 978-1-80020-390-7
www.packt.com
Contributors
About the author
Serg Masís has been at the confluence of the internet, application development, and
analytics for the last two decades. Currently, he's a climate and agronomic data scientist
at Syngenta, a leading agribusiness company with a mission to improve global food
security. Before that role, he co-founded a start-up, incubated by Harvard Innovation
Labs, that combined the power of cloud computing and machine learning with principles
in decision-making science to expose users to new places and events. Whether it pertains
to leisure activities, plant diseases, or customer lifetime value, Serg is passionate about
providing the often-missing link between data and decision-making—and machine
learning interpretation helps bridge this gap robustly.
About the reviewers
Shailendra Kadre is a seasoned ML, DL, Product Development, and Digital
Transformation professional with more than 20 years of industry experience with
global IT products and services companies. Currently, Shailendra is Worldwide Product
Analytics Lead with HP Inc, Bangalore. He is active as an advisor to universities and
corporates on AI. He has authored two books on AI, published by Springer and McGraw-
Hill, USA. He is also a professional photography enthusiast. He has held leadership
positions in Machine Learning, Product Analytics, and Digital Transformation space with
HP Inc., Citi Group/ Oracle, Satyam, and TCS. Shailendra Kadre holds a master's degree
in Design Engineering from the Indian Institute of Technology (IIT), Delhi.
James Le is a Data Advocate for Superb AI, a Y Combinator-backed company that
radically improves AI production workflows with an advanced training data management
platform. Before that, he completed his M.S. in Computer Science from RIT, doing
research at the intersection of deep learning and recommendation systems.
Muhammad Rehman Zafar is a Ph.D. candidate at Ryerson University, Toronto, Canada
and affiliated with Ryerson Multimedia Research Laboratory (RML). His research focuses
on designing and developing interpretable models by combining interactive visualizations
and machine learning approaches that help end-users to understand the decisions made
by complex models in an effective way. Rehman is also part of the Toronto-based team of
Aggregate Intellect as a machine learning interpretability stream owner.
Ali El-Sharif lives in Windsor, Ontario, Canada with his wife, three children, mother, and
a cat. Ali is a Ph.D. candidate at Nova Southeastern University, conducting research in
Machine Learning interpretability. Ali is also part of the Toronto-based team of Aggregate
Intellect as a Machine Learning Interpretability Stream Owner. Ali holds a bachelor's
degree in computer engineering and an MBA from Wright State University in Dayton,
Ohio. He also holds a master's degree in information security from Nova Southeastern
University. Ali is currently an instructor teaching Data Analytics and Information
Security as part of the Zekelman School of Business & IT at St. Clair College in Windsor,
ON, Canada.
Federico Riveroll is an expert in the fields of ML and Data Science. He is the co-founder
of OpenBlender.io and has 11 years of experience working with global companies like AB
InBev. Federico has also worked with startups and universities on varied AI projects.
Packt is searching for authors like you
If you're interested in becoming an author for Packt, please visit authors.
packtpub.com and apply today. We have worked with thousands of developers and
tech professionals, just like you, to help them share their insight with the global tech
community. You can make a general application, apply for a specific hot topic that we are
recruiting an author for, or submit your own idea.
Table of Contents
Preface
Section 1: Introduction to Machine
Learning Interpretation
1
Interpretation, Interpretability, and Explainability;
and Why Does It All Matter?
Technical requirements 4 A business case for
What is machine learning interpretability 18
interpretation? 5 Better decisions 18
Understanding a simple weight More trusted brands 20
prediction model 5 More ethical 23
More profitable 26
Understanding the difference
between interpretability and Summary 26
explainability 12 Image sources 26
What is interpretability? 12 Further reading 26
What is explainability? 16
2
Key Concepts of Interpretability
Technical requirements 29 Preparations 31
The mission 30 Loading the libraries 32
Details about CVD 30 Understanding and preparing the data 32
The approach 31 Learning about interpretation
method types and scopes 36
ii Table of Contents
Model interpretability method types 41 Non-linearity 50
Model interpretability scopes 41 Interactivity 53
Interpreting individual predictions Non-monotonicity 54
with logistic regression 42
Mission accomplished 56
Appreciating what hinders
Summary 57
machine learning interpretability 48
Further reading 57
3
Interpretation Challenges
Technical requirements 60 Generalized Linear Models (GLMs) 91
The mission 60 Decision trees 108
RuleFit 115
The approach 61
Nearest neighbors 116
The preparations 61
Naïve Bayes 120
Loading the libraries 62
Recognizing the trade-off
Understanding and preparing the data 62
between performance and
Reviewing traditional model interpretability 122
interpretation methods 69
Special model properties 122
Predicting minutes delayed with
Assessing performance 123
various regression methods 69
Classifying flights as delayed or not Discovering newer
delayed with various classification interpretable
methods 75 (glass-box) models 126
Visualizing delayed flights with Explainable Boosting Machine (EBM) 126
dimensionality reduction methods 83
Skoped Rules 130
Understanding limitations
Mission accomplished 133
of traditional model
Summary 134
interpretation methods 90
Dataset sources 135
Studying intrinsically
Further reading 135
interpretable
(white-box) models 91
Table of Contents iii
Section 2: Mastering
Interpretation Methods
4
Fundamentals of Feature Importance and Impact
Technical requirements 140 Feature importance for the
Multi-layer Perceptron 159
The mission 140
Personality and birth order 141 Practicing PFI 160
Disadvantages of PFI 163
The approach 141
The preparations 142 Interpreting PDPs 164
Loading the libraries 142 Interaction PDPs 170
Understanding and preparing the data 143 Disadvantages of PDP 174
Measuring the impact of Explaining ICE plots 174
a feature on the outcome 145
Disadvantages of ICE 179
Feature importance for tree-based
Mission accomplished 180
models 150
Feature importance for Logistic Summary 180
Regression 153 Dataset sources 180
Feature importance for LDA 156
Further reading 181
5
Global Model-Agnostic Interpretation Methods
Technical requirements 184 Understanding interactions 209
The mission 184 SHAP dependence plots 213
SHAP force plots 224
The approach 185
The preparations 186 Accumulated Local
Effects (ALE) plots 226
Loading the libraries 186
Understanding and preparing the data 187 Global surrogates 230
Mission accomplished 235
Learning about Shapley values 199
Summary 235
Interpreting SHAP summary
and dependence plots 201 Further reading 236
Generating SHAP summary plots 206
iv Table of Contents
6
Local Model-Agnostic Interpretation Methods
Technical requirements 238 values 248
The mission 238 Employing LIME 260
The approach 239 Using LIME for NLP 264
The preparations 239 Trying SHAP for NLP 272
Loading the libraries 240 Comparing SHAP with LIME 275
Understanding and preparing the data 241 Mission accomplished 276
Leveraging SHAP's Summary 277
KernelExplainer for local Dataset sources 277
interpretations with SHAP
Further reading 277
7
Anchor and Counterfactual Explanations
Technical requirements 280 Exploring counterfactual
The mission 280 explanations 303
Unfair bias in recidivisim risk Counterfactual explanations
assessments 281 guided by prototypes 304
Counterfactual instances and much
The approach 282 more with the What-If Tool (WIT) 308
The preparations 282
Comparing with CEM 319
Loading the libraries 283
Mission accomplished 324
Understanding and preparing the data 284
Summary 325
Understanding anchor
Dataset sources 325
explanations 295
Further reading 326
Preparations for anchor and
counterfactual explanations with alibi 296
Local interpretations for anchor
explanations 299
Table of Contents v
8
Visualizing Convolutional Neural Networks
Technical requirements 328 Saliency maps 361
The mission 328 Grad-CAM 364
Integrated gradients 367
The approach 330
Tying it all together 370
Preparations 331
Understanding classifications
Loading the libraries 331
with perturbation-based
Understanding and preparing the data 333
attribution methods 374
Assessing the CNN classifier with
traditional interpretation methods 339 Occlusion sensitivity 375
LIME's ImageExplainer 377
Visualizing the learning process
CEM 380
with activation-based methods 349
Tying it all together 387
Intermediate activations 351
Bonus method: SHAP's DeepExplainer 391
Activation maximization 356
Mission accomplished 393
Evaluating misclassifications
Summary 394
with gradient-based
attribution methods 360 Dataset and image sources 394
Further reading 394
9
Interpretation Methods for Multivariate Forecasting
and Sensitivity Analysis
Technical requirements 399 Computing global and local
The mission 399 attributions with SHAP's
KernelExplainer 427
The approach 401
Identifying influential features
The preparation 401
with factor prioritization 434
Loading the libraries 402
Quantifying uncertainty and
Understanding and preparing the data 403
cost sensitivity with factor
Assessing time series models fixing 442
with traditional interpretation Mission accomplished 451
methods 414
Summary 453
Generating LSTM attributions
Dataset and image sources 453
with integrated gradients 421
References 453
