Hyperparameter Tuning with Python: Boost your machine learning model’s performance via hyperparameter tuning

306 Pages·2022·11.079 MB·English

by  Louis Owen

Download Packt Books Free
https://t.me/prog_books_Packt
Hyperparameter
Tuning with Python
Boost your machine learning model’s performance
via hyperparameter tuning
Louis Owen
BIRMINGHAM—MUMBAI
Download Packt Books Free
https://t.me/prog_books_Packt
Hyperparameter Tuning with Python
Copyright © 2022 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted
in any form or by any means, without the prior written permission of the publisher, except in the case
of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information
presented. However, the information contained in this book is sold without warranty, either express
or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable
for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and
products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot
guarantee the accuracy of this information.
Group Product Manager: Gebin George
Publishing Product Manager: Dinesh Chaudhary
Senior Editor: David Sugarman
Technical Editor: Devanshi Ayare
Copy Editor: Safis Editing
Project Coordinator: Farheen Fathima
Proofreader: Safis Editing
Indexer: Pratik Shirodkhar
Production Designer: Ponraj Dhandapani
Marketing Coordinator: Shifa Ansari and Abeer Riyaz Dawe
First published: July 2022
Production reference: 1280722
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham
B3 2PB, UK.
ISBN 978-1-80323-587-5
www.packt.com
To Mom and Dad, thanks for everything!
– Louis
Download Packt Books Free
https://t.me/prog_books_Packt
Co n t r i b u t o r s
About the author
Louis Owen is a data scientist/AI engineer from Indonesia who is always hungry for new knowledge.
Throughout his career journey, he has worked in various fields of industry, including NGOs, e-commerce,
conversational AI, OTA, Smart City, and FinTech. Outside of work, he loves to spend his time helping
data science enthusiasts to become data scientists, either through his articles or through mentoring
sessions. He also loves to spend his spare time doing his hobbies: watching movies and conducting side
projects. Finally, Louis loves to meet new friends! So, please feel free to reach out to him on LinkedIn
if you have any topics to be discussed.
About the reviewer
Jamshaid Sohail is passionate about data science, machine learning, computer vision, and natural
language processing and has more than 2 years of experience in the industry. He has worked at a Silicon
Valley-based start-up named FunnelBeam, the founders of which are from Stanford University, as a
data scientist. Currently, he is working as a data scientist at Systems Limited. He has completed over
66 online courses from different platforms. He authored the book Data Wrangling with Python 3.X for
Packt Publishing and has reviewed multiple books and courses. He is also developing a comprehensive
course on data science at Educative and is in the process of writing books for multiple publishers.

Download Packt Books Free
https://t.me/prog_books_Packt
Table of Contents
Preface
Section 1: The Methods
1
Evaluating Machine Learning Models
Technical requirements 4 Discovering Leave-One-Out
Understanding the concept cross-validation 12
of overfitting 4 Discovering LPO cross-validation 13
Creating training, validation, Discovering time-series
and test sets 5 cross-validation 14
Exploring random and stratified splits 6 Summary 16
Discovering repeated k-fold Further reading 16
cross-validation 11
2
Introducing Hyperparameter Tuning
What is hyperparameter tuning? 17 Understanding hyperparameter
Demystifying hyperparameters space and distributions 19
versus parameters 18 Summary 20
viii Table of Contents
3
Exploring Exhaustive Search
Understanding manual search 22 Understanding random search 25
Understanding grid search 23 Summary 28
4
Exploring Bayesian Optimization
Introducing BO 30 Understanding TPE 51
Understanding BO GP 40 Understanding Metis 55
Understanding SMAC 42 Summary 58
5
Exploring Heuristic Search
Understanding simulated annealing 60 Understanding Population-Based
Understanding genetic algorithms 65 Training 82
Understanding particle swarm Summary 86
optimization 74
6
Exploring Multi-Fidelity Optimization
Introducing MFO 88 Understanding hyper band 100
Understanding coarse-to-fine search 89 Understanding BOHB 103
Understanding successive halving 95 Summary 107
Table of Contents ix
Section 2: The Implementation
7
Hyperparameter Tuning via Scikit
Technical requirements 112 Implementing Bayesian
Introducing Scikit 112 Optimization Gaussian Process 130
Implementing Grid Search 114 Implementing Bayesian
Optimization Random Forest 133
Implementing Random Search 121
Implementing Bayesian
Implementing Coarse-to-Fine Search 123
Optimization Gradient Boosted Trees 134
Implementing Successive Halving 124
Summary 136
Implementing Hyper Band 128
8
Hyperparameter Tuning via Hyperopt
Technical requirements 138 Implementing Tree-structured
Introducing Hyperopt 138 Parzen Estimators 146
Implementing Random Search 141 Implementing Adaptive TPE 146
Implementing simulated annealing 148
Summary 150
9
Hyperparameter Tuning via Optuna
Technical requirements 152 Implementing Simulated Annealing 162
Introducing Optuna 152 Implementing Successive Halving 164
Implementing TPE 159 Implementing Hyperband 166
Implementing Random Search 160 Summary 167
Implementing Grid Search 161