Table Of Content52
Texts in Applied Mathematics
Editors
J.E. Marsden
L. Sirovich
S.S. Antman
Advisors
G. Iooss
P. Holmes
D. Barkley
M. Dellnitz
P. Newton
Texts in Applied Mathematics
1. Sirovich:Introduction to Applied Mathematics.
2. Wiggins:Introduction to Applied Nonlinear Dynamical Systems and Chaos.
3. Hale/Koçak:Dynamics and Bifurcations.
4. Chorin/Marsden:A Mathematical Introduction to Fluid Mechanics, 3rd ed.
5. Hubbard/Weist:Differential Equations: A Dynamical Systems Approach:
Ordinary Differential Equations.
6. Sontag:Mathematical Control Theory: Deterministic Finite Dimensional
Systems, 2nd ed.
7. Perko:Differential Equations and Dynamical Systems, 3rd ed.
8. Seaborn:Hypergeometric Functions and Their Applications.
9. Pipkin:A Course on Integral Equations.
10. Hoppensteadt/Peskin:Modeling and Simulation in Medicine and
the Life Sciences, 2nd ed.
11. Braun:Differential Equations and Their Applications, 4th ed.
12. Stoer/Bulirsch:Introduction to Numerical Analysis, 3rd ed.
13. Renardy/Rogers:An Introduction to Partial Differential Equations.
14. Banks:Growth and Diffusion Phenomena: Mathematical Frameworks
and Applications.
15. Brenner/Scott:The Mathematical Theory of Finite Element Methods, 2nd ed.
16. Van de Velde:Concurrent Scientific Computing.
17. Marsden/Ratiu:Introduction to Mechanics and Symmetry, 2nd ed.
18. Hubbard/West:Differential Equations: A Dynamical Systems Approach:
Higher-Dimensional Systems.
19. Kaplan/Glass:Understanding Nonlinear Dynamics.
20. Holmes:Introduction to Perturbation Methods.
21. Curtain/Zwart:An Introduction to Infinite-Dimensional Linear Systems Theory.
22. Thomas:Numerical Partial Differential Equations: Finite Difference Methods.
23. Taylor:Partial Differential Equations: Basic Theory.
24. Merkin:Introduction to the Theory of Stability of Motion.
25. Naber:Topology, Geometry, and Gauge Fields: Foundations.
26. Polderman/Willems:Introduction to Mathematical Systems Theory:
A Behavioral Approach.
27. Reddy:Introductory Functional Analysis with Applications to Boundary-Value
Problems and Finite Elements.
28. Gustafson/Wilcox:Analytical and Computational Methods of Advanced
Engineering Mathematics.
29. Tveito/Winther:Introduction to Partial Differential Equations:
A Computational Approach.
30. Gasquet/Witomski:Fourier Analysis and Applications: Filtering,
Numerical Computation, Wavelets.
(continued after index)
Mark H. Holmes
Introduction to
Numerical Methods in
Differential Equations
Mark H. Holmes
Academic Science of the Material Science and Engineering
Rensselaer Polytechnic Institute
Troy, NY 12180
[email protected]
Series Editors
J.E. Marsden L. Sirovich
Control and Dynamical Systems, 107–81 Division of Applied Mathematics
California Institute of Technology Brown University
Pasadena, CA 91125 Providence, RI 02912
USA USA
[email protected] [email protected]
S.S. Antman
Department of Mathematics
and
Institute for Physical Science
and Technology
University of Maryland
College Park, MD 20742-4015
USA
[email protected]
Mathematics Subject Classification (2000): 65L05, 65L06, 65L07, 65L12, 65M12, 65M70,
65N12, 65N22, 65N35, 65N40, 68U05, 74S20
Library of Congress Control Number: 2006927786
ISBN-10: 0-387-30891-1 Printed on acid-free paper.
ISBN-13: 978-0387-30891-3
© 2007 Springer Science+Business Media, LLC
All rights reserved. This work may not be translated or copied in whole or in part without the
written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring
Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or
scholarly analysis. Use in connection with any form of information storage and retrieval,
electronic adaptation, computer software, or by similar or dissimilar methodology now known
or hereafter developed is forbidden.
The use in this publication of trade names, trademarks, service marks, and similar terms, even
if they are not identified as such, is not to be taken as an expression of opinion as to whether
or not they are subject to proprietary rights.
9 8 7 6 5 4 3 2 1
springer.com
To my parents
Preface
The title gives a reasonable first-order approximation to what this book is
about.Toexplainwhy,let’sstartwiththeexpression“differentialequations.”
Theseareessentialinscienceandengineering,becausethelawsofnaturetyp-
icallyresultinequationsrelatingspatialandtemporalchangesinoneormore
variables.Todevelopanunderstandingofwhatisinvolvedinfindingsolutions,
thebookbeginswithproblemsinvolvingderivativesforonlyoneindependent
variable, and these give rise to ordinary differential equations. Specifically,
the first chapter considers initial value problems (time derivatives), and the
second concentrates on boundary value problems (space derivatives). In the
succeeding four chapters problems involving both time and space derivatives,
partial differential equations, are investigated.
This brings us to the next expression in the title: “numerical methods.”
This is a book about how to transform differential equations into problems
that can be solved using a computer. The fact is that computers are only
able to solve discrete problems and generally do this using finite-precision
arithmetic. What this means is that in deriving and then using a numerical
algorithmthecorrectnessofthediscreteapproximationmustbeconsidered,as
must the consequences of round-off error in using floating-point arithmetic to
calculatetheanswer.Oneoftheinterestingaspectsofthesubjectisthatwhat
appears to be an obviously correct numerical method can result in complete
failure. Consequently, although the book concentrates on the derivation and
use of numerical methods, the theoretical underpinnings are also presented
and used in the development.
This brings us to the remaining principal word in the title: “introduc-
tion.” This has several meanings for this book, and one is that the material
is directed to those who are first learning the subject. Typically this includes
upper-division undergraduates and beginning graduate students. The objec-
tive is to learn the fundamental ideas of what is involved in deriving a nu-
merical method, including the role of truncation error, and the importance
of stability. It is also essential that you actually use the methods to solve
problems. In other words, you run code and see for yourself just how success-
viii Preface
ful, or unsuccessful, the method is for solving the problem. In conjunction
with this it is essential that those who do computations develop the ability
to effectively communicate the results to others. The only way to learn this
is to do it. Consequently, homework assignments that involve an appreciable
amount of computing are important to learning the material in this book.
To help with this, a library of sample code for the topics covered is available
at www.holmes.rpi.edu. Speaking of which, many of the problems considered
in the book result in solutions that are time-dependent. To help visualize the
dynamicalnatureofthesolution,moviesareprovidedforsomeoftheexample
problems. These are identified in the book with an (M) in the caption of the
associated figure.
Another meaning for “introduction” as concerns this textbook is that the
subject of each chapter can easily produce one or more volumes in its own
right. The intent here is to provide an introduction to the subject, and that
means certain topics are either not discussed or they are presented in an
abbreviatedform.Alltold,thematerialincludedshouldfillasemestercourse.
For those who might want a more in-depth presentation on a specific topic,
references are provided throughout the text.
The prerequisites for this text include an introductory undergraduate
course in differential equations and a basic course in numerical computing.
The latter would include using LU to solve matrix equations, polynomial
interpolation, and numerical differentiation and integration. Some degree of
computing capability is also required to use the methods that are derived.
Although no specific language or program is required to read this book, the
codes provided at www.holmes.rpi.edu use mostly MATLAB, and the movies
provided require QuickTime.
I would like to express my gratitude to the many students who took my
course in numerical methods for differential equations at Rensselaer. They
helped me immeasurably in understanding the subject and provided much-
neededencouragementtowritethisbook.Itisalsoapleasuretoacknowledge
the suggestions of Yuri Lvov, who read an early version of the manuscript.
Troy, New York Mark H. Holmes
January, 2006
Contents
Preface ........................................................ vii
1 Initial Value Problems..................................... 1
1.1 Introduction ............................................ 1
1.1.1 Examples of IVPs ................................. 2
1.2 Methods Obtained from Numerical Differentiation ........... 5
1.2.1 The Five Steps.................................... 5
1.2.2 Additional Difference Methods ...................... 15
1.3 Methods Obtained from Numerical Quadrature.............. 18
1.4 Runge–Kutta Methods ................................... 22
1.5 Extensions and Ghost Points.............................. 24
1.6 Conservative Methods.................................... 26
1.6.1 Velocity Verlet .................................... 27
1.6.2 Symplectic Methods ............................... 29
1.7 Next Steps ............................................. 31
Exercises ................................................... 33
2 Two-Point Boundary Value Problems...................... 45
2.1 Introduction ............................................ 45
2.1.1 Birds on a Wire................................... 45
2.1.2 Chemical Kinetics ................................. 45
2.2 Derivative Approximation Methods ........................ 46
2.2.1 Matrix Problem................................... 49
2.2.2 Tridiagonal Matrices............................... 50
2.2.3 Matrix Problem Revisited .......................... 52
2.2.4 Error Analysis .................................... 55
2.2.5 Extensions ....................................... 58
2.3 Residual Methods ....................................... 62
2.3.1 Basis Functions ................................... 63
2.3.2 Residual ......................................... 66
2.4 Shooting Methods ....................................... 69
x Contents
2.5 Next Steps ............................................. 72
Exercises ................................................... 74
3 Diffusion Problems ........................................ 83
3.1 Introduction ............................................ 83
3.1.1 Heat Equation .................................... 83
3.2 Derivative Approximation Methods ........................ 88
3.2.1 Implicit Method...................................100
3.2.2 Theta Method ....................................102
3.3 Methods Obtained from Numerical Quadrature..............105
3.3.1 Crank–Nicolson Method............................106
3.3.2 L-Stability .......................................109
3.4 Methods of Lines ........................................112
3.5 Collocation .............................................113
3.6 Next Steps .............................................118
Exercises ...................................................119
4 Advection Equation .......................................127
4.1 Introduction ............................................127
4.1.1 Method of Characteristics ..........................127
4.1.2 Solution Properties ................................130
4.1.3 Boundary Conditions ..............................131
4.2 First-Order Methods.....................................132
4.2.1 Upwind Scheme ...................................132
4.2.2 Downwind Scheme.................................132
4.2.3 Numerical Domain of Dependence ...................134
4.2.4 Stability .........................................138
4.3 Improvements...........................................139
4.3.1 Lax–Wendroff Method .............................140
4.3.2 Monotone Methods ................................144
4.3.3 Upwind Revisited .................................145
4.4 Implicit Methods ........................................146
Exercises ...................................................148
5 Numerical Wave Propagation..............................155
5.1 Introduction ............................................155
5.1.1 Solution Methods .................................155
5.1.2 Plane Wave Solutions ..............................160
5.2 Explicit Method.........................................164
5.2.1 Diagnostics .......................................167
5.2.2 Numerical Experiments ............................169
5.3 Numerical Plane Waves ..................................171
5.3.1 Numerical Group Velocity ..........................174
5.4 Next Steps .............................................176
Exercises ...................................................176
Contents xi
6 Elliptic Problems ..........................................181
6.1 Introduction ............................................181
6.1.1 Solutions .........................................183
6.1.2 Properties of the Solution ..........................186
6.2 Finite Difference Approximation...........................187
6.2.1 Building the Matrix ...............................190
6.2.2 Positive Definite Matrices ..........................192
6.3 Descent Methods ........................................196
6.3.1 Steepest Descent Method...........................198
6.3.2 Conjugate Gradient Method ........................199
6.4 Numerical Solution of Laplace’s Equation...................204
6.5 Preconditioned Conjugate Gradient Method ................207
6.6 Next Steps .............................................212
Exercises ...................................................214
A Appendix..................................................223
A.1 Order Symbols ..........................................223
A.2 Taylor’s Theorem........................................224
A.3 Round-Off Error ........................................225
A.3.1 Function Evaluation ...............................225
A.3.2 Numerical Differentiation...........................226
A.4 Floating-Point Numbers..................................227
References.....................................................231
Index..........................................................235
