Surrogate Model-Based Engineering Design and Optimization

246 Pages·2020·10.403 MB·English

by  Ping Jiang, Qi Zhou, Xinyu Shao

Springer Tracts in Mechanical Engineering
Ping Jiang
Qi Zhou
Xinyu Shao
Surrogate
Model-Based
Engineering
Design and
Optimization
Springer Tracts in Mechanical Engineering
Series Editors
Seung-Bok Choi, College of Engineering, Inha University, Incheon, Korea
(Republic of)
Haibin Duan, Beijing University of Aeronautics and Astronautics, Beijing, China
Yili Fu, Harbin Institute of Technology, Harbin, China
Carlos Guardiola, CMT-Motores Termicos, Polytechnic University of Valencia,
Valencia, Spain
Jian-Qiao Sun, University of California, Merced, CA, USA
Young W. Kwon, Naval Postgraduate School, Monterey, CA, USA
Springer Tracts in Mechanical Engineering (STME) publishes the latest develop-
ments in Mechanical Engineering - quickly, informally and with high quality. The
intentistocoverallthemainbranchesofmechanicalengineering,boththeoretical
and applied, including:
(cid:129) Engineering Design
(cid:129) Machinery and Machine Elements
(cid:129) Mechanical structures and Stress Analysis
(cid:129) Automotive Engineering
(cid:129) Engine Technology
(cid:129) Aerospace Technology and Astronautics
(cid:129) Nanotechnology and Microengineering
(cid:129) Control, Robotics, Mechatronics
(cid:129) MEMS
(cid:129) Theoretical and Applied Mechanics
(cid:129) Dynamical Systems, Control
(cid:129) Fluids mechanics
(cid:129) Engineering Thermodynamics, Heat and Mass Transfer
(cid:129) Manufacturing
(cid:129) Precision engineering, Instrumentation, Measurement
(cid:129) Materials Engineering
(cid:129) Tribology and surface technology
Within the scopes of the series are monographs, professional books or graduate
textbooks,editedvolumesaswellasoutstandingPh.D.thesesandbookspurposely
devoted to support education in mechanical engineering at graduate and
post-graduate levels.
Indexed by SCOPUS and Springerlink. The books of the series are submitted
for indexing to Web of Science.
Tosubmitaproposalorrequestfurtherinformation,pleasecontact:Dr.LeontinaDi
Cecco [e-mail] or Li Shen [e-mail].
PleasecheckourLectureNotesinMechanicalEngineeringathttp://www.springer.
com/series/11236ifyouareinterestedinconferenceproceedings.Tosubmitapro-
posal,[e-mail]@springer.com.
More information about this series at http://www.springer.com/series/11693
Ping Jiang Qi Zhou Xinyu Shao
(cid:129) (cid:129)
Surrogate Model-Based
Engineering Design
and Optimization
123
PingJiang QiZhou
TheState KeyLaboratory of Digital Schoolof Aerospace Engineering
Manufacturing Equipment andTechnology HuazhongUniversity of Science
Schoolof MechanicalScience and andTechnology
Engineering Wuhan, Hubei, China
HuazhongUniversity of Science
andTechnology
Wuhan, China
XinyuShao
TheState KeyLaboratory of Digital
Manufacturing Equipment andTechnology
Schoolof MechanicalScience and
Engineering
HuazhongUniversity of Science
andTechnology
Wuhan, China
ISSN 2195-9862 ISSN 2195-9870 (electronic)
SpringerTracts inMechanical Engineering
ISBN978-981-15-0730-4 ISBN978-981-15-0731-1 (eBook)
https://doi.org/10.1007/978-981-15-0731-1
©SpringerNatureSingaporePteLtd.2020
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
orinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodologynowknownorhereafterdeveloped.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom
therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, expressed or implied, with respect to the material contained
hereinorforanyerrorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregard
tojurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSingaporePteLtd.
The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721,
Singapore
Preface
Numerical simulations have been wildly used in engineering design and optimiza-
tiontostudyorimitatetherealphysicalsystems.Withtheincreasingcomputational
power and the development of simulation software, simulation models can reflect
moredetailsoftherealsystemandthediscrepancybetweenthephysicalsystemand
thesimulationmodelisshrinking.However,conductingahigh-fidelitysimulationis
still time-consuming due tothe increasing complexityof the simulation model, for
example, the utilization of more solving equations and finer mesh, etc. If directly
applying these time-consuming simulation models in optimization problems may
also result in unaffordable design cost. Under this background, surrogate models
have gained increasing attention in recent years. They are constructed based on
available input parameter values and the corresponding output performance or
quantity of interests (QOIs) to provide predictions. Especially the multi-fidelity
surrogate models, which integrate the information from the simulation model of
differentfidelity,canachieveahighmodelingaccuracywithinthesamesimulation
cost. Applying surrogate models to engineering design and optimization can sig-
nificantly reduce therequired number of simulations andshorten thedesign cycle.
For this book, our motivation is to provide a systematic introduction for the
designers about how to use surrogate models in engineering design and optimiza-
tion. It mainly consists of two parts: how to construct a surrogate model with the
desiredaccuracyandhowtoapplyittodifferentoptimizationproblems.Therefore,
this book will cover some of the most popular methods in design space sampling,
the ensemble of the surrogate model, multi-fidelity surrogate model construction,
surrogate model selection, and validation, surrogate-based robust design opti-
mization, and surrogate model-based evolutionary optimization. Some real-life
engineering design problems, such as three-dimensional aircraft design, are also
provided to illustrate the ability of Surrogate models in support of complex engi-
neeringdesign.Also,lotsofillustrativeexamplesareadoptedthroughoutthebook
inthehopethattheapproachesareexplainedmoreclearlyinthisway.Webelieve
that the methods for the ensemble of the surrogate model and multi-fidelity model
will be particular interest to the readers, as the presented methods demonstrate a
good balance between surrogate modeling accuracy and building cost. This book
v
vi Preface
introducesdifferentapproachesinadidacticway,anditcanbeusedasareference
book for undergraduate students, graduate students, or engineers. We assume the
intended readers have some mathematical backgrounds, for example, matrix
operations, basic optimization algorithms and so on.
We thank Prof. Qi Zhou for his laborious work, Prof. Xinyu Shao for his
valuable comments, and the following graduate students: Jiexiang Hu, Leshi Shu,
TingliXie,HuaWei,XiongfengRuan,JiCheng,YutongPeng,LinjunZhong,and
Yuda Wu. Ping Jiang, the first author, acknowledges financial support from the
NationalNaturalScienceFoundationofChina(NSFC)underGrantNo.51775203.
Qi Zhou, the second author, is grateful for the financial support from the National
Natural Science Foundation of China (NSFC) under Grant No. 51805179. Xinyu
Shao,thethirdauthor,isgratefulforthefinancialsupportfromtheNationalNatural
Science Foundation of China (NSFC) under Grant No. 51721092.
Wuhan, China Ping Jiang
August 2019
Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Reasons for Using Surrogate Models. . . . . . . . . . . . . . . . . . . . . . 1
1.2 Symbols and Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 Classic Types of Surrogate Models. . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1 Polynomial Response Surface Models . . . . . . . . . . . . . . . . . . . . . 7
2.2 Radial Basis Function Models. . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.3 Support Vector Regression Models . . . . . . . . . . . . . . . . . . . . . . . 11
2.4 Gaussian Process Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.5 Backpropagation Neural Network Models . . . . . . . . . . . . . . . . . . 20
2.6 Performance Comparison of Different Surrogate Models. . . . . . . . 24
2.6.1 Influence of Sample Size . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.6.2 Influence of Noise Level . . . . . . . . . . . . . . . . . . . . . . . . . 29
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3 Ensembles of Surrogate Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.1 Weighted Average Surrogate Models. . . . . . . . . . . . . . . . . . . . . . 36
3.1.1 Weight Factor Selection Based on the Generalized
Mean Square Cross-Validation Error (GMSE) . . . . . . . . . . 36
3.1.2 Optimal Weighted Ensemble of Surrogates (OWS) . . . . . . 39
3.1.3 Optimal Average Weight Method Based on Recursive
Arithmetic Averaging. . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.2 Pointwise Weighted Ensembles of Surrogate Models (EMs) . . . . . 42
3.2.1 Weight Coefficient Selection Based on Prediction
Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.2.2 Adaptive Hybrid Function (AHF)-Based Pointwise
Weighting Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
vii
viii Contents
3.2.3 Pointwise Weighted EM Using v Nearest Points
Cross-Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.2.4 Optimal Weighted Pointwise Ensemble (OWPE)
Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4 Multi-fidelity Surrogate Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.1 Scaling-Function-Based Approaches . . . . . . . . . . . . . . . . . . . . . . 56
4.1.1 Multiplicative Scaling Approach. . . . . . . . . . . . . . . . . . . . 56
4.1.2 Additive Scaling Approach. . . . . . . . . . . . . . . . . . . . . . . . 58
4.1.3 Hybrid Scaling Approach. . . . . . . . . . . . . . . . . . . . . . . . . 59
4.1.4 Examples and Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4.2 Space Mapping (SM) Approaches . . . . . . . . . . . . . . . . . . . . . . . . 62
4.2.1 The Concept of Output-Output Space Mapping (OOSM)
Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4.2.2 The Radial Basis Function (RBF)-Based OOSM
Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.2.3 The Gaussian Process-Based OOSM Approach . . . . . . . . . 71
4.2.4 The Support Vector Regression (SVR)-Based OOSM
Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
4.3 Co-Kriging Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
4.3.1 Traditional Co-Kriging Approach . . . . . . . . . . . . . . . . . . . 78
4.3.2 Extended Co-Kriging Approaches. . . . . . . . . . . . . . . . . . . 80
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5 Verification Methods for Surrogate Models . . . . . . . . . . . . . . . . . . . 89
5.1 Metrics Relying on Testing Methods . . . . . . . . . . . . . . . . . . . . . . 91
5.1.1 Jackknife Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.1.2 Bootstrap Error (BE) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
5.1.3 Cross-Validation Error . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
5.1.4 Prediction Variance (PV) . . . . . . . . . . . . . . . . . . . . . . . . . 95
5.1.5 Predictive Estimation of Model Fidelity (PEMF) Error. . . . 97
5.2 Metrics Relying on Sampling Methods . . . . . . . . . . . . . . . . . . . . 97
5.2.1 Coefficient of Determination. . . . . . . . . . . . . . . . . . . . . . . 98
5.2.2 Mean Square Error (MSE) . . . . . . . . . . . . . . . . . . . . . . . . 99
5.2.3 Root Mean Square Error (RMSE). . . . . . . . . . . . . . . . . . . 99
5.2.4 Maximum Absolute Error (MaxAE) . . . . . . . . . . . . . . . . . 100
5.2.5 Relative Maximum Absolute Error (RMAE) . . . . . . . . . . . 100
5.2.6 Mean Absolute Error (MeanAE). . . . . . . . . . . . . . . . . . . . 100
5.2.7 Relative Average Absolute Error (RAAE). . . . . . . . . . . . . 100
5.3 Selection of Error Metrics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
Contents ix
5.3.1 Review of Error Metrics for Surrogate Models . . . . . . . . . 101
5.3.2 Performance Comparison of Commonly Used
Error Metrics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
6 Sampling Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
6.1 One-Shot Sampling Methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
6.1.1 Uniform Design (UD) . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
6.1.2 Monte Carlo Sampling (MCS) . . . . . . . . . . . . . . . . . . . . . 116
6.1.3 Latin Hypercube Design (LHD) . . . . . . . . . . . . . . . . . . . . 117
6.1.4 Orthogonal Array Sampling . . . . . . . . . . . . . . . . . . . . . . . 117
6.2 Adaptive Sequential Sampling. . . . . . . . . . . . . . . . . . . . . . . . . . . 118
6.2.1 For Single Surrogate Models . . . . . . . . . . . . . . . . . . . . . . 118
6.2.2 For Ensembles of Surrogate Models . . . . . . . . . . . . . . . . . 121
6.2.3 For Multi-fidelity Models. . . . . . . . . . . . . . . . . . . . . . . . . 124
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
7 Surrogate-Model-Based Design and Optimization. . . . . . . . . . . . . . . 135
7.1 Surrogate-Model-Based Deterministic Design Optimization. . . . . . 135
7.1.1 Expected Improvement Criteria. . . . . . . . . . . . . . . . . . . . . 139
7.1.2 Probability of Improvement Criteria . . . . . . . . . . . . . . . . . 155
7.1.3 Lower Confidence Bound Criteria. . . . . . . . . . . . . . . . . . . 163
7.2 Surrogate-Model-Based Robust Design Optimization . . . . . . . . . . 172
7.2.1 Surrogate-Model-Based Probabilistic Robust
Optimization (RO) Approaches. . . . . . . . . . . . . . . . . . . . . 173
7.2.2 Surrogate-Model-Based Deterministic RO Approaches. . . . 190
7.3 Surrogate-Model-Based Evolutionary Optimization. . . . . . . . . . . . 207
7.3.1 A Kriging-Model-Assisted Multi-objective Genetic
Algorithm (K-MOGA). . . . . . . . . . . . . . . . . . . . . . . . . . . 208
7.3.2 A Multi-objective Variable-Fidelity Optimization
Method for GAs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
7.3.3 An Online Multi-fidelity Surrogate-Model-Assisted
Multi-objective Genetic Algorithm (OLVFM-MOGA) . . . . 212
7.3.4 Examples and Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
Reference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240