Foundations of Applied Mathematics, Volume 1: Mathematical Analysis

708 Pages·2017·198.183 MB·English

by  Jeffrey Humpherys, Tyler J. Jarvis, Emily J. Evans

Foundations of Applied Mathematics
Volume 1
Mathematical Analysis
JEFFREY HUMPHERYS
TYLER J. JARVIS
EMILY J. EVANS
BRIGHAM YOUNG UNIVERSITY
•
"
SOCIETY FOR INDUSTRIAL
AND APPLIED MATHEMATICS
PHILADELPHIA
Copyright© 2017 by the Society for Industrial and Applied Mathematics
10987654321
All rights reserved. Printed in the United States of America. No part of this book may be reproduced, stored, or
transmitted in any manner without the written permission of the publisher. For information, write to the Society
for Industrial and Applied Mathematics, 3600 Market Street, 6th Floor, Philadelphia, PA 19104-2688 USA
No warranties, express or implied, are made by the publisher, authors, and their employers that the programs
contained in this volume are free of error. They should not be relied on as the sole basis to solve a problem
whose incorrect solution could result in injury to person or property. If the programs are employed in such
a manner, it is at the user's own risk and the publisher, authors, and their employers disclaim all liability for
such misuse.
Python is a registered trademark of Python Software Foundation.
PUBLISHER David Marshall
EXECUTIVE EDITOR Elizabeth Greenspan
DEVELOPMENTAL EDITOR Gina Rinelli Harris
MANAGING EDITOR Kelly Thomas
PRODUCTION EDITOR Louis R. Primus
COPY EDITOR Louis R. Primus
PRODUCTION MANAGER Donna Witzleben
PRODUCTION COORDINATOR Cally Shrader
COMPOSITOR Lumina Datamatics
GRAPHIC DESIGNER Lois Sellers
COVER DESIGNER Sarah Kay Miller
Library of Congress Cataloging-in-Publication Data
Names: Humpherys, Jeffrey, author. I Jarvis, Tyler ,Jamison, author. I Evans,
Emily J , author.
Title: Foundations of applied mathematics I Jeffrey Humpherys, Tyler J
Jarvis, Emily J. Evans, Brigham Young University, Provo, Utah.
Description Philadelphia: Society for Industrial and Applied Mathematics,
[2017]-I Series: Other titles in applied mathematics ; 152 I Includes
bibliographical references and index.
Identifiers LCCN 2017012783 I ISBN 978161197 4898 (v. 1)
Subjects: LCSH: Calculus. I Mathematical analysis. I Matrices.
Classification LCC QA303.2 .H86 20171 DDC 515--dc23 LC record available at https//lccn.loc
gov /2017012783
•
5.la.J1L
is a registered trademark.
Contents
List of Notation ix
Preface xiii
I Linear Analysis I 1
1 Abstract Vector Spaces 3
1.1 Vector Algebra . . . . . . . . . . . . 3
1.2 Spans and Linear Independence .. 10
1.3 Products, Sums, and Complements 14
1.4 Dimension, Replacement, and Extension 17
1.5 Quotient Spaces 21
Exercises .... . ... ........ .. . 27
2 Linear Transformations and Matrices 31
2.1 Basics of Linear Transformations I . 32
2.2 Basics of Linear Transformations II 36
2.3 Rank, Nullity, and the First Isomorphism Theorem 40
2.4 Matrix Representations .. . ......... . 46
2.5 Composition, Change of Basis, and Similarity 51
2.6 Important Example: Bernstein Polynomials 54
2. 7 Linear Systems 58
2.8 Determinants I . 65
2.9 Determinants II 70
Exercises ....... . 78
3 Inner Product Spaces 87
3.1 Introduction to Inner Products. 88
3.2 Orthonormal Sets and Orthogonal Projections 94
3.3 Gram-Schmidt Orthonormalization .. 99
3.4 QR with Householder Transformations 105
3.5 Normed Linear Spaces . . . 110
3.6 Important Norm Inequalities ..... . 117
3.7 Adjoints .......... . ..... . 120
3.8 Fundamental Subspaces of a Linear Transformation 123
3.9 Least Squares 127
Exercises ....... . .... . 131
v
vi Contents
4 Spectral Theory 139
4.1 Eigenvalues and Eigenvectors . 140
4.2 Invariant Subspaces 147
4.3 Diagonalization . . . . . . . . 150
4.4 Schur's Lemma . . . . . . . . 155
4.5 The Singular Value Decomposition 159
4.6 Consequences of the SYD . 165
Exercises . .. . .. . 171
II Nonlinear Analysis I 177
5 Metric Space Topology 179
5.1 Metric Spaces and Continuous Functions 180
5.2 Continuous Functions and Limits . ... 185
5.3 Closed Sets, Sequences, and Convergence 190
5.4 Completeness and Uniform Continuity . 195
5.5 Compactness . .. .... . . . ..... . 203
5.6 Uniform Convergence and Banach Spaces . 210
5.7 The Continuous Linear Extension Theorem . 213
5.8 Topologically Equivalent Metrics . 219
5.9 Topological Properties .. 222
5.10 Banach-Valued Integration 227
Exercises . . ...... .. . .. . 233
6 Differentiation 241
6.1 The Directional Derivative 241
6.2 The Frechet Derivative in ]Rn .. 246
6.3 The General Frechet Derivative 252
6.4 Properties of Derivatives ... . 256
6.5 Mean Value Theorem and Fundamental Theorem of Calculus 260
6.6 Taylor's Theorem 265
Exercises . ...... . ... . . . .. . ... . 272
7 Contraction Mappings and Applications 277
7.1 Contraction Mapping Principle . .. .. 278
7.2 Uniform Contraction Mapping Principle 281
7.3 Newton's Method ............ . 285
7.4 The Implicit and Inverse Function Theorems 293
7.5 Conditioning 301
Exercises . .. . ... . .. ..... . .. . ... . . 310
III Nonlinear Analysis II 317
8 Integration I 319
8.1 Multivariable Integration . . . . . . ... . 320
8.2 Overview of Daniell-Lebesgue Integration 326
8.3 Measure Zero and Measurability . . . . . . 331
Contents vii
8.4 Monotone Convergence and Integration on
Unbounded Domains .... . . . . ... . 335
8.5 Fatou's Lemma and the Dominated Convergence Theorem 340
8.6 Fubini's Theorem and Leibniz's Integral Rule 344
8.7 Change of Variables 349
Exercises . .. .. 356
9 * Integration II 361
9.1 Every Normed Space Has a Unique Completion 361
9.2 More about Measure Zero . . 364
9.3 Lebesgue-Integrable Functions .. . ... . 367
9.4 Proof of Fubini's Theorem .... . ... . 372
9.5 Proof of the Change of Variables Theorem 374
Exercises ..... . ..... . ........ .. . 378
10 Calculus on Manifolds 381
10.l Curves and Arclength . 381
10.2 Line Integrals .... . 386
10.3 Parametrized Manifolds . 389
10.4 * Integration on Manifolds 393
10.5 Green's Theorem 396
Exercises .. . ... . 403
11 Complex Analysis 407
11. l Holomorphic Functions 407
11.2 Properties and Examples 411
11.3 Contour Integrals . . . . 416
11.4 Cauchy's Integral Formula 424
11.5 Consequences of Cauchy's Integral Formula . 429
11.6 Power Series and Laurent Series . . .... . 433
11. 7 The Residue Theorem . . . . . . . . . . . . . 438
11.8 *The Argument Principle and Its Consequences . 445
Exercises .. . .. . .. .. . . .............. . 451
IV Linear Analysis II 457
12 Spectral Calculus 459
12.l Projections .... . .. . 460
12.2 Generalized Eigenvectors 465
12.3 The Resolvent . . . . . . 470
12.4 Spectral Resolution ... 475
12.5 Spectral Decomposition I 480
12.6 Spectral Decomposition II 483
12.7 Spectral Mapping Theorem . 489
12.8 The Perron-Frobenius Theorem 494
12.9 The Drazin Inverse . ... 500
12.10 * Jordan Canonical Form . 506
Exercises ............. . 511
viii Contents
13 Iterative Methods 519
13.l Methods for Linear Systems ..... ... . 520
13.2 Minimal Polynomials and Krylov Subspaces 526
13.3 The Arnoldi Iteration and GMRES Methods 530
*
13.4 Computing Eigenvalues I . 538
*
13.5 Computing Eigenvalues II 543
Exercises ..... . ...... . 548
14 Spectra and Pseudospectra 553
14.l The Pseudospectrum ....... . . . 554
14.2 Asymptotic and Transient Behavior .. 561
*
14.3 Proof of the Kreiss Matrix Theorem . 566
Exercises ... . .. ... . 570
15 Rings and Polynomials 573
15.l Definition and Examples ....... .. . 574
15.2 Euclidean Domains ........ . .. . . 583
15.3 The Fundamental Theorem of Arithmetic . 588
15.4 Homomorphisms . . . . . . . .. ..... . 592
15.5 Quotients and the First Isomorphism Theorem . 598
15.6 The Chinese Remainder Theorem .. .. ... . 601
15.7 Polynomial Interpolation and Spectral Decomposition . 610
Exercises .................... . .... .. .. . 618
V Appendices 625
A Foundations of Abstract Ma.thematics 627
A. l Sets and Relations . 627
A.2 Functions .......... ... .. . 635
A.3 Orderings . . . . . . . . . . . . . . . . 643
A.4 Zorn's Lemma, the Axiom of Choice, and Well Ordering 647
A.5 Cardinality . . . . . . . . . . . . . . . . . . . . . . . . . . 648
B The Complex Numbers and Other Fields 653
B.1 Complex Numbers. 653
B.2 Fields .... . ........ ... . .. . . 659
C Topics in Matrix Analysis 663
C.l Matrix Algebra 663
C.2 Block Matrices . 665
C.3 Cross Products 667
D The Greek Alphabet 669
Bibliography 671
Index 679
List of Notation
t indicates harder exercises xvi
* indicates advanced material that can be skipped xvi
isomorphism 39, 595
EB direct sum 16
EB addition in a quotient 24, 575, 599, 641
[] multiplication in a quotient 24, 575, 599, 641
>0 (for matrices) positive definite 159
~o (for matrices) positive semidefinite 159
>- componentwise inequality 494, 513
(-, . ) inner product 89
(-) the ideal generated by · 581
J_ orthogonal complement 123
x Cartesian product 4, 14, 596, 630
I divides 585
I· I absolute value or modulus of a number; componentwise mod
ulus of a matrix 4, 513, 654
II · II a norm 111
JJ · JJF the Frobenius norm 113, 115
II· Jiu the L1-norm 134, 327
II · 11£ the L2-norm 134
2
JI · JIL= the L00-norm, also known as the sup norm 113, 134
II· llv,w the induced norm on @(V, W) 114
II· lip the p-norm, with p E [l, oo], either for vectors or operators
111, 112, 115, 116
[[·]] an equivalence class 22
[-,·,·] a permutation 66
]. the all ones vector ]. = [1 1] T 499
lie the indicator function of E 228, 323
2s the power set of S 628
[a,b] a closed interval in ]Rn 321
a.e. almost everywhere 329, 332
argminx f(x) the value of x that minimizes f 533
ix
x List of Notation
B(xo, r) the open ball with center at xo and radius r 182
Bf!> the jth Bernstein polynomial of degree n 55
J
8E the boundary of E 192
@(V;W) the space of bounded linear transformations from V to W
114
@k(V;W) the space of bounded linear transformations from V to
@k-1(V, W) 266
@(V) the space of bounded linear operators on V 114
@(X;IF) the space of bounded linear functionals on X 214
c
the complex numbers 4, 407, 627
C(X;Y) the set of continuous functions from X to Y 5, 185
Co([a,b];IF) the space of continuous IF-valued functions that vanish at the
endpoints a and b 9
Cb(X; JR) the space of continuous functions on X with bounded L00
-
norm 311
cn(X;Y) the space of Y-valued functions whose nth derivative is con
tinuous on X 9, 253, 266
C00(X; Y) the space of smooth Y-valued functions on X 266
Csr the transition matrix from T to S 48
D(xo, r) the closed ball with center at x and radius r 191
0
Df(x) the Frechet derivative of f at x 246
Dd the ith partial derivative of f 245
Dk f(x) the kth Frechet derivative of f at x 266
Dvf(x) the directional derivative of f at x in the direction v 244
DU(x) kth directional derivative of f at x in the direction v 268
Dk f(x)v(k) same as D~f (x) 268
D>. the eigennilpotent associated to the eigenvalue ,\ 483
diag(.A1, ... , An) the diagonal matrix with (i, i) entry equal to Ai 152
d(x,y) a metric 180
8ij the Kronecker delta 95
Eo the set of interior points of E 182
E the closure of E 192
6°>, the generalized eigenspace corresponding to eigenvalue ,\ 468
ei the ith standard basis vector 13, 51
ep the evaluation map at p 33, 594
IF a field, always either C or JR 4
IFn n-dimensional Euclidean space 5
IF[A] the ring of matrices that are polynomials in A 576
List of Notation xi
lF[x] the space of polynomials with coefficients in lF 6
JF[x, y] the space of polynomials in x and y with coefficients in lF 576
lF[x;n] the space of polynomials with coefficients in lF of degree at
most n 9
the preimage {x I f(x) EU} off 186, 187
the greatest common divisor 586
the graph of f 635
the Householder transformation of x 106
I("Y,zo) the winding number of 'Y with respect to z 441
0
ind(B) the index of the matrix B 466
SS(z) the imaginary part of z 450, 654
K(A) the Kreiss matrix constant of A 562
JtA,(A, b) the kth Krylov subspace of A generated by b 506, 527
11;(A) the matrix condition number of A 307
the relative condition number 303
/\;
Pi, the absolute condition number 302
/\;spect (A) the spectral condition number of A 560
L1(A; X) the space of integrable functions on A 329, 338, 363
LP([a, b]; lF) the space of p-integrable functions 6
Li the ith Lagrange interpolant 603
L00(S; X) the set of bounded functions from S to X, with the sup norm
216
£(V;W) the set of linear transformations from V to W 37
len( a) the arclength of the curve a 383
£(a, b) the line segment from a to b 260
£P
the space of infinite sequences (xj)~1 such that 2::~1 x~
converges 6
>.(R) the measure of an interval R C lFn 323
Mn(lF) the space of square n x n matrices with entries in lF 5
Mmxn(lF) the space of m x n matrices with entries in lF 5
N the natural numbers {O, 1, 2, ... } 577, 628
Ni the ith Newton interpolant 611
N the unit normal 392
JY (L) the kernel of L 35, 594
the eigenprojection associated to all the nonzero eigenvalues
500
P;.. the eigenprojection associated to the eigenvalue >. 475
PA(z) the characteristic polynomial of A 143
proh(v) the orthogonal projection of v onto the subspace X 96
xii List of Notation
the orthogonal projection of v onto span( { x}) 93
a subdivision 323
xn x
the ith projection map from to 33
the rational numbers 628
the points of ]Rn with dyadic rational coordinates 37 4
JR the real numbers 4, 627
R[x] the ring of formal power series in x with coefficients in R 576
R(A,z) the resolvent of A, sometimes denoted R( z) 47 0
Res(!, zo) the residue off at zo 440
r(A) the spectral radius of A 474
re:(A) the c:-pseudospectral radius of A 561
~([a, b];X) the space of regulated integrable functions 324
~(L) the range of L 35
~(z) the real part of z 450, 654
p(L) the resolvent set of L 141
S([a, b]; X) the set of all step functions mapping [a, b] into X 228, 323
sign(z) the complex sign z/JzJ of z 656
Skewn(IF) the space of skew-symmetric n x n matrices 5, 29
Symn(IF) the space of symmetric n x n matrices 5, 29
~>.(L) the >.-eigenspace of L 141
a(L) the spectrum of L 141
ae:(A) the pseudospectrum of A 554
the unit tangent vector 382
the tangent space to M at p 390
the set of compact n-cubes with corners in Qk 374
v(a) a valuation function 583
the integers{ ... , -2, -1, 0, 1, 2, ... } 628
the positive integers {1, 2, ... } 628
the set of equivalence classes in Z modulo n 575, 633, 660