Introduction to numerical programming: a practical guide for scientists and engineers using Python and C/C++

663 Pages·2015·8.35 MB·English

by  Titus A. Beu

Computer Science/Scientific Programming
B
e
SERIES IN COMPUTATIONAL PHYSICS u
Steven A. Gottlieb and Rubin H. Landau, Series Editors
“Your journey will be a real pleasure since the book focuses on thorough explanations, hands-on I
N
code examples, and graphical representations.”
T
—Professor Dr. Alexander K. Hartmann, Institute for Physics, University of Oldenburg R
O
“… an invaluable resource for aspects that are often not sufficiently emphasised, despite their
D
importance for reliable calculations. I strongly recommend it for everyone’s bookshelf.”
U
—Professor Joan Adler, Technion, Israel Institute of Technology
C
“… a comprehensive introduction to classical numerical methods… via clear and simple codes T
I
in Python and C/++. … I will recommend it to my students.” O
—Professor Mike Wheatland, The University of Sydney
N

Makes Numerical Programming More Accessible to a Wider Audience T
O
Bearing in mind the evolution of modern programming, most specifically emergent program-

N
ming languages that reflect modern practice, Numerical Programming: A Practical Guide for
U
Scientists and Engineers Using Python and C/C++ utilizes the author’s many years of practi-
M
cal research and teaching experience to offer a systematic approach to relevant programming
concepts. Adopting a practical, broad appeal, this user-friendly book offers guidance to anyone E
R
interested in using numerical programming to solve science and engineering problems. Empha-
I
sizing methods generally used in physics and engineering—from elementary methods to com- C
plex algorithms—it gradually incorporates algorithmic elements with increasing complexity. A
L
Develop a Combination of Theoretical Knowledge, Efficient Analysis Skills,
P
and Code Design Know-How R
O
The book encourages algorithmic thinking, which is essential to numerical analysis. Establish-
G
ing the fundamental numerical methods, application numerical behavior and graphical output
needed to foster algorithmic reasoning, coding dexterity, and a scientific programming style, it R
A
enables readers to successfully navigate relevant algorithms, understand coding design, and de-
M
velop efficient programming skills. The book incorporates real code and includes examples and
problem sets to assist in hands-on learning. M
I
This text introduces platform-independent numerical programming using Python and C/C++ and N
appeals to advanced undergraduate and graduate students in natural sciences and engineering, G
researchers involved in scientific computing, and engineers carrying out applicative calculations.
K16451
K16451_cover.indd 1 7/16/14 11:03 AM
INTRODUCTION TO
NUMERICAL
PROGRAMMING
SERIES IN COMPUTATIONAL PHYSICS
Steven A. Gottlieb and Rubin H. Landau
Series Editors
Parallel Science and Engineering Applications: The Charm++ Approach
Sanjay Kale and Abhinav Bhatele, Eds.
Introduction to Numerical Programming: A Practical Guide for Scientists
and Engineers Using Python and C/C++
Titus Adrian Beu
Forthcoming
Visualization in Computational Physics and Materials Science
Joan Adler
Introduction to Classical Dynamics: A Computational View
Kelly Roos and Joseph Driscoll
SERIES IN COMPUTATIONAL PHYSICS
Steven A. Gottlieb and Rubin H. Landau, Series Editors
INTRODUCTION TO
NUMERICAL
PROGRAMMING
A Practical Guide for Scientists and Engineers
Using Python and C/C++
Titus Adrien Beu
Babeș-Bolyai University
Faculty of Physics
Cluj-Napoca, Romania
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2015 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Version Date: 20140716
International Standard Book Number-13: 978-1-4665-6968-3 (eBook - PDF)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been
made to publish reliable data and information, but the author and publisher cannot assume responsibility for the valid-
ity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright
holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this
form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may
rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or uti-
lized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopy-
ing, microfilming, and recording, or in any information storage or retrieval system, without written permission from the
publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://
www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923,
978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For
organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for
identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
ToMihaelaandVictor

Contents
SeriesPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix
1 ApproximateNumbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 SourcesofErrorsinNumericalCalculations . . . . . . . . . . . . . . . . 1
1.2 AbsoluteandRelativeErrors . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 RepresentationofNumbers . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 SignificantDigits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.5 ErrorsofElementaryOperations . . . . . . . . . . . . . . . . . . . . . . . 7
ReferencesandSuggestedFurtherReading . . . . . . . . . . . . . . . . . . . . 10
2 BasicProgrammingTechniques . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.1 ProgrammingConcepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2 FunctionsandParameters. . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3 PassingArgumentstoPythonFunctions . . . . . . . . . . . . . . . . . . 15
2.4 PassingArgumentstoC/C++Functions . . . . . . . . . . . . . . . . . . 17
2.5 ArraysinPython . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.6 DynamicArrayAllocationinC/C++ . . . . . . . . . . . . . . . . . . . . 19
2.7 BasicMatrixOperations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
ReferencesandSuggestedFurtherReading . . . . . . . . . . . . . . . . . . . . 30
3 ElementsofScientificGraphics . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.1 TheTkinterPackage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.2 TheCanvasWidget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.3 SimpleTkinterApplications . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.4 PlottingFunctionsofOneVariable . . . . . . . . . . . . . . . . . . . . . 39
3.5 GraphicsLibrarygraphlib.py . . . . . . . . . . . . . . . . . . . . . . 44
3.6 CreatingPlotsinC++UsingtheLibrarygraphlib.py . . . . . . . . 57
ReferencesandSuggestedFurtherReading . . . . . . . . . . . . . . . . . . . . 61
4 SortingandIndexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.2 BubbleSort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.3 InsertionSort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
vii
viii Contents
4.4 Quicksort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.5 IndexingandRanking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.6 ImplementationsinC/C++ . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.7 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
ReferencesandSuggestedFurtherReading . . . . . . . . . . . . . . . . . . . . 84
5 EvaluationofFunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.1 EvaluationofPolynomialsbyHorner’sScheme . . . . . . . . . . . . . . 85
5.2 EvaluationofAnalyticFunctions. . . . . . . . . . . . . . . . . . . . . . . 88
5.3 ContinuedFractions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.4 OrthogonalPolynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
5.5 SphericalHarmonics—AssociatedLegendreFunctions . . . . . . . . . . 98
5.6 SphericalBesselFunctions . . . . . . . . . . . . . . . . . . . . . . . . . . 102
5.7 ImplementationsinC/C++ . . . . . . . . . . . . . . . . . . . . . . . . . . 105
5.8 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
ReferencesandSuggestedFurtherReading . . . . . . . . . . . . . . . . . . . . 125
6 AlgebraicandTranscendentalEquations . . . . . . . . . . . . . . . . . . . . . . 127
6.1 RootSeparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
6.2 BisectionMethod. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
6.3 MethodofFalsePosition . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
6.4 MethodofSuccessiveApproximations . . . . . . . . . . . . . . . . . . . 134
6.5 Newton’sMethod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
6.6 SecantMethod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
6.7 Birge–VietaMethod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
6.8 Newton’sMethodforSystemsofNonlinearEquations . . . . . . . . . . 147
6.9 ImplementationsinC/C++ . . . . . . . . . . . . . . . . . . . . . . . . . . 151
6.10 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
ReferencesandSuggestedFurtherReading . . . . . . . . . . . . . . . . . . . . 168
7 SystemsofLinearEquations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
7.2 GaussianEliminationwithBackwardSubstitution . . . . . . . . . . . . 169
7.3 Gauss–JordanElimination . . . . . . . . . . . . . . . . . . . . . . . . . . 179
7.4 LUFactorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
7.5 InversionofTriangularMatrices . . . . . . . . . . . . . . . . . . . . . . . 195
7.6 CholeskyFactorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
7.7 TridiagonalSystemsofLinearEquations . . . . . . . . . . . . . . . . . . 203
7.8 BlockTridiagonalSystemsofLinearEquations . . . . . . . . . . . . . . 207
7.9 ComplexMatrixEquations . . . . . . . . . . . . . . . . . . . . . . . . . . 208
7.10 JacobiandGauss–SeidelIterativeMethods . . . . . . . . . . . . . . . . . 209
7.11 ImplementationsinC/C++ . . . . . . . . . . . . . . . . . . . . . . . . . . 213
7.12 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
ReferencesandSuggestedFurtherReading . . . . . . . . . . . . . . . . . . . . 231
8 EigenvalueProblems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
8.2 DiagonalizationofMatricesbySimilarityTransformations . . . . . . . 234
8.3 JacobiMethod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
8.4 GeneralizedEigenvalueProblemsforSymmetricMatrices . . . . . . . . 243
8.5 ImplementationsinC/C++ . . . . . . . . . . . . . . . . . . . . . . . . . . 246
Contents ix
8.6 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
ReferencesandSuggestedFurtherReading . . . . . . . . . . . . . . . . . . . . 264
9 ModelingofTabulatedFunctions . . . . . . . . . . . . . . . . . . . . . . . . . . 265
9.1 InterpolationandRegression . . . . . . . . . . . . . . . . . . . . . . . . . 265
9.2 LagrangeInterpolationPolynomial . . . . . . . . . . . . . . . . . . . . . 268
9.3 Neville’sInterpolationMethod . . . . . . . . . . . . . . . . . . . . . . . . 273
9.4 CubicSplineInterpolation . . . . . . . . . . . . . . . . . . . . . . . . . . 276
9.5 LinearRegression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
9.6 MultilinearRegressionModels . . . . . . . . . . . . . . . . . . . . . . . . 287
9.7 NonlinearRegression:TheLevenberg–MarquardtMethod . . . . . . . 293
9.8 ImplementationsinC/C++ . . . . . . . . . . . . . . . . . . . . . . . . . . 301
9.9 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
ReferencesandSuggestedFurtherReading . . . . . . . . . . . . . . . . . . . . 331
10 IntegrationofFunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
10.2 TrapezoidalRule;AHeuristicApproach . . . . . . . . . . . . . . . . . . 333
10.3 TheNewton–CotesQuadratureFormulas . . . . . . . . . . . . . . . . . 335
10.4 TrapezoidalRule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337
10.5 Simpson’sRule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339
10.6 AdaptiveQuadratureMethods . . . . . . . . . . . . . . . . . . . . . . . . 341
10.7 Romberg’sMethod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344
10.8 ImproperIntegrals:OpenFormulas . . . . . . . . . . . . . . . . . . . . . 348
10.9 MidpointRule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352
10.10 GaussianQuadratures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354
10.11 MultidimensionalIntegration . . . . . . . . . . . . . . . . . . . . . . . . 361
10.12 AdaptiveMultidimensionalIntegration . . . . . . . . . . . . . . . . . . . 369
10.13 ImplementationsinC/C++ . . . . . . . . . . . . . . . . . . . . . . . . . . 372
10.14 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
ReferencesandSuggestedFurtherReading . . . . . . . . . . . . . . . . . . . . 393
11 MonteCarloMethod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
11.2 IntegrationofFunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396
11.3 ImportanceSampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399
11.4 MultidimensionalIntegrals . . . . . . . . . . . . . . . . . . . . . . . . . . 402
11.5 GenerationofRandomNumbers . . . . . . . . . . . . . . . . . . . . . . 408
11.6 ImplementationsinC/C++ . . . . . . . . . . . . . . . . . . . . . . . . . . 415
11.7 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
ReferencesandSuggestedFurtherReading . . . . . . . . . . . . . . . . . . . . 426
12 OrdinaryDifferentialEquations . . . . . . . . . . . . . . . . . . . . . . . . . . . 427
12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427
12.2 TaylorSeriesMethod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429
12.3 Euler’sMethod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431
12.4 Runge–KuttaMethods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434
12.5 AdaptiveStepSizeControl . . . . . . . . . . . . . . . . . . . . . . . . . . 440
12.6 MethodsforSecond-OrderODEs . . . . . . . . . . . . . . . . . . . . . . 447
12.7 Numerov’sMethod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454
12.8 ShootingMethodsforTwo-PointProblems . . . . . . . . . . . . . . . . 457