Numerical Analysis and Optimization: An Introduction to Mathematical Modelling and Numerical Simulation (Numerical Mathematics and Scientific Computation)

472 Pages·2007·6.77 MB·English

by  Gregoire Allaire

NUMERICAL MATHEMATICS AND SCIENTIFIC COMPUTATION
Series Editors
G.H. GOLUB, A. GREENBAUM
A.M. STUART, E. SU¨LI
NUMERICAL MATHEMATICS AND SCIENTIFIC COMPUTATION
Books in the series
Monographs marked with an asterix (*) appeared in the series ‘Monographs in Numerical
Analysis’ which has been folded into, and is continued by, the current series.
* P. Dierckx: Curve and surface fittings with splines
* J.H. Wilkinson: The algebraic eigenvalue problem
* I. Duff, A. Erisman, and J. Reid: Direct methods for sparse matrices
* M.J. Baines: Moving finite elements
* J.D. Pryce: Numerical solution of Sturm–Liouville problems
K. Burrage: Parallel and sequential methods for ordinary differential equations
Y. Censor and S.A. Zenios: Parallel optimization: theory, algorithms and applications
M. Ainsworth, J. Levesley, W. Light, and M. Marletta: Wavelets, multilevel methods and
elliptic PDEs
W.Freeden,T.Gervens,andM.Schreiner: Constructiveapproximationonthesphere: theory
and applications to geomathematics
C.H. Schwab: p- and hp- finite element methods: theory and applications to solid and fluid
mechanics
J.W. Jerome: Modelling and computation for applications in mathematics, science, and
engineering
Alfio Quarteroni and Alberto Valli: Domain decomposition methods for partial differential
equations
G.E. Karniadakis and S.J. Sherwin: Spectral/hp element methods for CFD
I. Babuˇska and T. Strouboulis: The finite element method and its reliability
B. Mohammadi and O. Pironneau: Applied shape optimization for fluids
S. Succi: The Lattice Boltzmann Equation for fluid dynamics and beyond
P. Monk: Finite element methods for Maxwell’s equations
A. Bellen and M. Zennaro: Numerical methods for delay differential equations
J. Modersitzki: Numerical methods for image registration
M. Feistauer, J. Felcman, and I. Straˇskraba: Mathematical and computational methods for
compressible flow
W. Gautschi: Orthogonal polynomials: computation and approximation
M.K. Ng: Iterative methods for Toeplitz systems
Michael Metcalf, John Reid, and Malcolm Cohen: Fortran 95/2003 explained
George Em Karniadakis and Spencer Sherwin: Spectral/hp element methods for CFD,
second edition
DarioA.Bini,GuyLatouche,andBeatriceMeini: NumericalmethodsforstructuredMarkov
chains
Howard Elman, David Silvester, and Andy Wathen: Finite elements and fast iterative
solvers: with applications in incompressible fluid dynamics
Moody Chu and Gene Golub: Inverse eigenvalue problems: theory and applications
Jean-Fr´ed´eric Gerbeau, Claude Le Bris, and Tony Leli`evre: Mathematical methods for the
magnetohydrodynamics of liquid metals
Gr´egoire Allaire: Numerical analysis and optimization
umerical nalysis and ptimization
N A O
An introduction to mathematical modelling
and numerical simulation
Gr´egoire Allaire
´
Ecole Polytechnique
Translated by Dr Alan Craig
University of Durham
1
3
GreatClarendonStreet,OxfordOX26DP
OxfordUniversityPressisadepartmentoftheUniversityofOxford.
ItfurtherstheUniversity’sobjectiveofexcellenceinresearch,scholarship,
andeducationbypublishingworldwidein
Oxford NewYork
Auckland CapeTown DaresSalaam HongKong Karachi
KualaLumpur Madrid Melbourne MexicoCity Nairobi
NewDelhi Shanghai Taipei Toronto
Withofficesin
Argentina Austria Brazil Chile CzechRepublic France Greece
Guatemala Hungary Italy Japan Poland Portugal Singapore
SouthKorea Switzerland Thailand Turkey Ukraine Vietnam
OxfordisaregisteredtrademarkofOxfordUniversityPress
intheUKandincertainothercountries
PublishedintheUnitedStates
byOxfordUniversityPressInc.,NewYork
(cid:1)c Gr´egoireAllaire2007
Themoralrightsoftheauthorhavebeenasserted
DatabaserightOxfordUniversityPress(maker)
Firstpublished2007
Allrightsreserved. Nopartofthispublicationmaybereproduced,
storedinaretrievalsystem,ortransmitted,inanyformorbyanymeans,
withoutthepriorpermissioninwritingofOxfordUniversityPress,
orasexpresslypermittedbylaw,orundertermsagreedwiththeappropriate
reprographicsrightsorganization. Enquiriesconcerningreproduction
outsidethescopeoftheaboveshouldbesenttotheRightsDepartment,
OxfordUniversityPress,attheaddressabove
Youmustnotcirculatethisbookinanyotherbindingorcover
andyoumustimposethesameconditiononanyacquirer
BritishLibraryCataloguinginPublicationData
Dataavailable
LibraryofCongressCataloginginPublicationData
Dataavailable
TypesetbyNewgenImagingSystems(P)Ltd.,Chennai,India
PrintedinGreatBritain
onacid-freepaperby
BiddlesLtd.,King’sLynn,Norfolk
ISBN978–0–19–920521–9(Hbk.)
ISBN978–0–19–920522–6(Pbk.)
1 3 5 7 9 10 8 6 4 2
Contents
1 Introduction ix
1 Introduction to mathematical modelling and numerical simulation 1
1.1 General introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 An example of modelling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Some classical models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.3.1 The heat flow equation . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.3.2 The wave equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.3.3 The Laplacian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.3.4 Schr¨odinger’s equation . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.3.5 The Lam´e system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.3.6 The Stokes system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.3.7 The plate equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.4 Numerical calculation by finite differences . . . . . . . . . . . . . . . . . . . . 14
1.4.1 Principles of the method . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.4.2 Numerical results for the heat flow equation . . . . . . . . . . . . . . . 17
1.4.3 Numerical results for the advection equation . . . . . . . . . . . . . . 21
1.5 Remarks on mathematical models. . . . . . . . . . . . . . . . . . . . . . . . . 25
1.5.1 The idea of a well-posed problem . . . . . . . . . . . . . . . . . . . . . 25
1.5.2 Classification of PDEs . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2 Finite difference method 31
2.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.2 Finite differences for the heat equation . . . . . . . . . . . . . . . . . . . . . . 32
2.2.1 Various examples of schemes . . . . . . . . . . . . . . . . . . . . . . . 32
2.2.2 Consistency and accuracy . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.2.3 Stability and Fourier analysis . . . . . . . . . . . . . . . . . . . . . . . 36
2.2.4 Convergence of the schemes . . . . . . . . . . . . . . . . . . . . . . . . 42
2.2.5 Multilevel schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.2.6 The multidimensional case. . . . . . . . . . . . . . . . . . . . . . . . . 46
2.3 Other models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
2.3.1 Advection equation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
2.3.2 Wave equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
v
vi CONTENTS
3 Variational formulation of elliptic problems 65
3.1 Generalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.1.2 Classical formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.1.3 The case of a space of one dimension . . . . . . . . . . . . . . . . . . . 67
3.2 Variational approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.2.1 Green’s formulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.2.2 Variational formulation . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.3 Lax–Milgram theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
3.3.1 Abstract framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
3.3.2 Application to the Laplacian . . . . . . . . . . . . . . . . . . . . . . . 76
4 Sobolev spaces 79
4.1 Introduction and warning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.2 Square integrable functions and weak differentiation . . . . . . . . . . . . . . 80
4.2.1 Some results from integration . . . . . . . . . . . . . . . . . . . . . . . 80
4.2.2 Weak differentiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.3 Definition and principal properties . . . . . . . . . . . . . . . . . . . . . . . . 84
4.3.1 The space H1(Ω) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
4.3.2 The space H1(Ω) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
0
4.3.3 Traces and Green’s formulas . . . . . . . . . . . . . . . . . . . . . . . 89
4.3.4 A compactness result. . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
4.3.5 The spaces Hm(Ω) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
4.4 Some useful extra results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
4.4.1 Proof of the density theorem 4.3.5 . . . . . . . . . . . . . . . . . . . . 98
4.4.2 The space H(div) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
4.4.3 The spaces Wm,p(Ω) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
4.4.4 Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
4.5 Link with distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
5 Mathematical study of elliptic problems 109
5.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
5.2 Study of the Laplacian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
5.2.1 Dirichlet boundary conditions . . . . . . . . . . . . . . . . . . . . . . . 109
5.2.2 Neumann boundary conditions . . . . . . . . . . . . . . . . . . . . . . 116
5.2.3 Variable coefficients . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
5.2.4 Qualitative properties . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
5.3 Solution of other models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
5.3.1 System of linear elasticity . . . . . . . . . . . . . . . . . . . . . . . . . 136
5.3.2 Stokes equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
6 Finite element method 149
6.1 Variational approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
6.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
6.1.2 General internal approximation . . . . . . . . . . . . . . . . . . . . . . 150
6.1.3 Galerkin method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
6.1.4 Finite element method (general principles). . . . . . . . . . . . . . . . 153
6.2 Finite elements in N =1 dimension . . . . . . . . . . . . . . . . . . . . . . . 154
CONTENTS vii
6.2.1 P1 finite elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
6.2.2 Convergence and error estimation. . . . . . . . . . . . . . . . . . . . . 159
6.2.3 P2 finite elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
6.2.4 Qualitative properties . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
6.2.5 Hermite finite elements . . . . . . . . . . . . . . . . . . . . . . . . . . 168
6.3 Finite elements in N ≥2 dimensions . . . . . . . . . . . . . . . . . . . . . . . 171
6.3.1 Triangular finite elements . . . . . . . . . . . . . . . . . . . . . . . . . 171
6.3.2 Convergence and error estimation. . . . . . . . . . . . . . . . . . . . . 184
6.3.3 Rectangular finite elements . . . . . . . . . . . . . . . . . . . . . . . . 191
6.3.4 Finite elements for the Stokes problem . . . . . . . . . . . . . . . . . . 195
6.3.5 Visualization of the numerical results. . . . . . . . . . . . . . . . . . . 201
7 Eigenvalue problems 205
7.1 Motivation and examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
7.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
7.1.2 Solution of nonstationary problems . . . . . . . . . . . . . . . . . . . . 206
7.2 Spectral theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
7.2.1 Generalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
7.2.2 Spectral decomposition of a compact operator. . . . . . . . . . . . . . 210
7.3 Eigenvalues of an elliptic problem . . . . . . . . . . . . . . . . . . . . . . . . . 213
7.3.1 Variational problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
7.3.2 Eigenvalues of the Laplacian . . . . . . . . . . . . . . . . . . . . . . . 218
7.3.3 Other models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
7.4 Numerical methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
7.4.1 Discretization by finite elements . . . . . . . . . . . . . . . . . . . . . 224
7.4.2 Convergence and error estimates . . . . . . . . . . . . . . . . . . . . . 227
8 Evolution problems 231
8.1 Motivation and examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
8.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
8.1.2 Modelling and examples of parabolic equations . . . . . . . . . . . . . 232
8.1.3 Modelling and examples of hyperbolic equations . . . . . . . . . . . . 233
8.2 Existence and uniqueness in the parabolic case . . . . . . . . . . . . . . . . . 234
8.2.1 Variational formulation . . . . . . . . . . . . . . . . . . . . . . . . . . 234
8.2.2 A general result. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
8.2.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
8.3 Existence and uniqueness in the hyperbolic case. . . . . . . . . . . . . . . . . 246
8.3.1 Variational formulation . . . . . . . . . . . . . . . . . . . . . . . . . . 246
8.3.2 A general result. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
8.3.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
8.4 Qualitative properties in the parabolic case . . . . . . . . . . . . . . . . . . . 253
8.4.1 Asymptotic behaviour . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
8.4.2 The maximum principle . . . . . . . . . . . . . . . . . . . . . . . . . . 255
8.4.3 Propagation at infinite velocity . . . . . . . . . . . . . . . . . . . . . . 256
8.4.4 Regularity and regularizing effect . . . . . . . . . . . . . . . . . . . . . 257
8.4.5 Heat equation in the entire space . . . . . . . . . . . . . . . . . . . . . 259
8.5 Qualitative properties in the hyperbolic case. . . . . . . . . . . . . . . . . . . 261
8.5.1 Reversibility in time . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
viii CONTENTS
8.5.2 Asymptotic behaviour and equipartition of energy . . . . . . . . . . . 262
8.5.3 Finite velocity of propagation . . . . . . . . . . . . . . . . . . . . . . . 263
8.6 Numerical methods in the parabolic case . . . . . . . . . . . . . . . . . . . . . 264
8.6.1 Semidiscretization in space . . . . . . . . . . . . . . . . . . . . . . . . 264
8.6.2 Total discretization in space-time . . . . . . . . . . . . . . . . . . . . . 266
8.7 Numerical methods in the hyperbolic case . . . . . . . . . . . . . . . . . . . . 269
8.7.1 Semidiscretization in space . . . . . . . . . . . . . . . . . . . . . . . . 270
8.7.2 Total discretization in space-time . . . . . . . . . . . . . . . . . . . . . 271
9 Introduction to optimization 277
9.1 Motivation and examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
9.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
9.1.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278
9.1.3 Definitions and notation . . . . . . . . . . . . . . . . . . . . . . . . . . 284
9.1.4 Optimization in finite dimensions . . . . . . . . . . . . . . . . . . . . . 285
9.2 Existence of a minimum in infinite dimensions. . . . . . . . . . . . . . . . . . 287
9.2.1 Examples of nonexistence . . . . . . . . . . . . . . . . . . . . . . . . . 287
9.2.2 Convex analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
9.2.3 Existence results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
10 Optimality conditions and algorithms 297
10.1 Generalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
10.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
10.1.2 Differentiability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298
10.2 Optimality conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303
10.2.1 Euler inequalities and convex constraints . . . . . . . . . . . . . . . . 303
10.2.2 Lagrange multipliers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
10.3 Saddle point, Kuhn–Tucker theorem, duality . . . . . . . . . . . . . . . . . . 317
10.3.1 Saddle point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
10.3.2 The Kuhn–Tucker theorem . . . . . . . . . . . . . . . . . . . . . . . . 318
10.3.3 Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320
10.4 Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
10.4.1 Dual or complementary energy . . . . . . . . . . . . . . . . . . . . . . 323
10.4.2 Optimal command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326
10.4.3 Optimization of distributed systems . . . . . . . . . . . . . . . . . . . 330
10.5 Numerical algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332
10.5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332
10.5.2 Gradient algorithms (case without constraints) . . . . . . . . . . . . . 333
10.5.3 Gradient algorithms (case with constraints) . . . . . . . . . . . . . . . 336
10.5.4 Newton’s method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342
11 Methods of operational research
(Written in collaboration with St´ephane Gaubert) 347
11.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347
11.2 Linear programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348
11.2.1 Definitions and properties . . . . . . . . . . . . . . . . . . . . . . . . . 348
11.2.2 The simplex algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
11.2.3 Interior point algorithms. . . . . . . . . . . . . . . . . . . . . . . . . . 357
CONTENTS ix
11.2.4 Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358
11.3 Integer polyhedra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
11.3.1 Extreme points of compact convex sets. . . . . . . . . . . . . . . . . . 362
11.3.2 Totally unimodular matrices . . . . . . . . . . . . . . . . . . . . . . . 364
11.3.3 Flow problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368
11.4 Dynamic programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371
11.4.1 Bellman’s optimality principle. . . . . . . . . . . . . . . . . . . . . . . 372
11.4.2 Finite horizon problem. . . . . . . . . . . . . . . . . . . . . . . . . . . 372
11.4.3 Minimum cost path, or optimal stopping, problem . . . . . . . . . . . 375
11.5 Greedy algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380
11.5.1 General points about greedy methods . . . . . . . . . . . . . . . . . . 380
11.5.2 Kruskal’s algorithm for the minimum
spanning tree problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 381
11.6 Separation and relaxation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383
11.6.1 Separation and evaluation (branch and bound) . . . . . . . . . . . . . 383
11.6.2 Relaxation of combinatorial problems . . . . . . . . . . . . . . . . . . 388
12 Appendix Review of hilbert spaces 399
13 Appendix Matrix Numerical Analysis 405
13.1 Solution of linear systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405
13.1.1 Review of matrix norms . . . . . . . . . . . . . . . . . . . . . . . . . . 406
13.1.2 Conditioning and stability . . . . . . . . . . . . . . . . . . . . . . . . . 409
13.1.3 Direct methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411
13.1.4 Iterative methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424
13.1.5 The conjugate gradient method . . . . . . . . . . . . . . . . . . . . . . 428
13.2 Calculation of eigenvalues and eigenvectors . . . . . . . . . . . . . . . . . . . 435
13.2.1 The power method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436
13.2.2 The Givens–Householder method . . . . . . . . . . . . . . . . . . . . . 438
13.2.3 The Lanczos method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442
Index 451
Index notations 455