Table Of ContentWRITING SCIENTIFIC SOFTWARE:A GUIDE
TO GOOD STYLE
Thecoreofscientificcomputingisdesigning,writing,testing,debuggingandmod-
ifying numerical software for application to a vast range of areas: from graphics,
weatherforecasting,andchemistrytoengineering,biology,andfinance.Scientists,
engineers,andcomputerscientistsneedtowritegood,clearcode,forspeed,clarity,
flexibility,andeaseofre-use.
Oliveira and Stewart provide here a guide to writing numerical software,
pointing out good practices to follow, and pitfalls to avoid. By following their
advice, the reader will learn how to write efficient software, and how to test it
for bugs, accuracy, and performance. Techniques are explained with a variety of
programming languages, and illustrated with two extensive design examples, one
inFortran90andoneinC++,alongwithotherexamplesinC,C++,Fortran90
andJavascatteredthroughoutthebook.
Commonissuesinnumericalcomputingaredealtwith:forexample,whetherto
allocateorpass“scratch”memoryfortemporaryuse,howtopassparameterstoa
functionthatisitselfpassedtoaroutine,howtoallocatemultidimensionalarrays
in C/C++/Java, and how to create suitable interfaces for routines and libraries.
Advanced topics, such as recursive data structures, template programming and
typebindersfornumericalcomputing,blockingandunrollingloopsforefficiency,
how to design software for deep memory hierarchies, and amortized doubling for
efficientmemoryuse,arealsoincluded.
Thismanualofscientificcomputingstylewillprovetobeanessentialaddition
tothebookshelfandlabofeveryonewhowritesnumericalsoftware.
WRITING SCIENTIFIC SOFTWARE:
A GUIDE FOR GOOD STYLE
SUELY OLIVEIRA AND DAVID E. STEWART
UniversityofIowa
cambridge university press
Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo
Cambridge University Press
TheEdinburghBuilding,Cambridgecb22ru,UK
Published in the United States of America by Cambridge University Press, New York
www.cambridge.org
Informationonthis title: www.cambridge.org/9780521858960
Thispublicationisincopyright.Subjecttostatutoryexceptionandtotheprovisionof
relevantcollectivelicensingagreements,noreproductionofanypartmaytakeplace
withoutthewrittenpermissionofCambridgeUniversityPress.
Firstpublishedinprintformat 2006
isbn-13 978-0-521-85896-0hardback
isbn-10 0-521-85896-8 hardback
isbn-13 978-0-521-67595-6 paperback
isbn-10 0-521-67595-2 paperback
CambridgeUniversityPresshasnoresponsibilityforthepersistenceoraccuracyofurls
forexternalorthird-partyinternetwebsitesreferredtointhispublication,anddoesnot
guaranteethatanycontentonsuchwebsitesis,orwillremain,accurateorappropriate.
Contents
Preface page ix
PartI NumericalSoftware 1
1 Whynumericalsoftware? 3
1.1 Efficientkernels 4
1.2 Rapidchange 5
1.3 Large-scaleproblems 6
2 Scientificcomputationandnumericalanalysis 8
2.1 Thetroublewithrealnumbers 8
2.2 Fixed-pointarithmetic 18
2.3 Algorithmstabilityvs.problemstability 19
2.4 Numericalaccuracyandreliability 23
3 Priorities 30
3.1 Correctness 30
3.2 Numericalstability 32
3.3 Accuratediscretization 32
3.4 Flexibility 33
3.5 Efficiency:timeandmemory 35
4 Famousdisasters 36
4.1 Patriotmissiles 36
4.2 Ariane5 37
4.3 SleipnerAoilrigcollapse 38
5 Exercises 39
PartII DevelopingSoftware 43
6 Basicsofcomputerorganization 45
6.1 Underthehood:whataCPUdoes 45
6.2 Callingroutines:stacksandregisters 47
6.3 Allocatingvariables 51
6.4 Compilers,linkers,andloaders 53
v
vi Contents
7 Softwaredesign 57
7.1 Softwareengineering 57
7.2 Softwarelife-cycle 57
7.3 Programminginthelarge 59
7.4 Programminginthesmall 61
7.5 Programminginthemiddle 67
7.6 Interfacedesign 70
7.7 Top-downandbottom-updevelopment 75
7.8 Don’thard-wireitunnecessarily! 77
7.9 Comments 78
7.10 Documentation 80
7.11 Cross-languagedevelopment 82
7.12 Modularityandallthat 87
8 Datastructures 90
8.1 Packageyourdata! 90
8.2 Avoidglobalvariables! 91
8.3 Multidimensionalarrays 92
8.4 Functionalrepresentationvs.datastructures 96
8.5 Functionsandthe“environmentproblem” 97
8.6 Somecommentsonobject-orientedscientificsoftware 106
9 Designfortestinganddebugging 118
9.1 Incrementaltesting 118
9.2 Localizingbugs 120
9.3 Themighty“print”statement 120
9.4 Getthecomputertohelp 122
9.5 Usingdebuggers 129
9.6 Debuggingfunctionalrepresentations 130
9.7 Errorandexceptionhandling 132
9.8 Compareandcontrast 135
9.9 Trackingbugs 136
9.10 Stresstestingandperformancetesting 137
9.11 Randomtestdata 141
10 Exercises 143
PartIII EfficiencyinTime,EfficiencyinMemory 147
11 Bealgorithmaware 149
11.1 Numericalalgorithms 149
11.2 Discretealgorithms 151
11.3 Numericalalgorithmdesigntechniques 153
12 Computerarchitectureandefficiency 156
12.1 Cachesandmemoryhierarchies 156
Contents vii
12.2 AtourofthePentium4TM architecture 158
12.3 Virtualmemoryandpaging 164
12.4 Thrashing 164
12.5 Designingformemoryhierarchies 165
12.6 Dynamicdatastructuresandmemoryhierarchies 168
12.7 Pipeliningandloopunrolling 168
12.8 BasicLinearAlgebraSoftware(BLAS) 170
12.9 LAPACK 178
12.10 Cache-obliviousalgorithmsanddatastructures 184
12.11 Indexingvs.pointersfordynamicdatastructures 185
13 Globalvs.localoptimization 187
13.1 Pickingalgorithmsvs.keyholeoptimization 187
13.2 Whatoptimizingcompilersdo 188
13.3 Helpingthecompileralong 191
13.4 Practicalitiesandasymptoticcomplexity 192
14 Grabbingmemorywhenyouneedit 195
14.1 Dynamicmemoryallocation 195
14.2 Givingitback 197
14.3 Garbagecollection 198
14.4 Lifewithgarbagecollection 199
14.5 Conservativegarbagecollection 202
14.6 Doingityourself 203
14.7 Memorytips 205
15 Memorybugsandleaks 208
15.1 Beware:unallocatedmemory! 208
15.2 Beware:overwritingmemory! 208
15.3 Beware:danglingpointers! 210
15.4 Beware:memoryleaks! 214
15.5 Debuggingtools 215
PartIV Tools 217
16 Sourcesofscientificsoftware 219
16.1 Netlib 220
16.2 BLAS 220
16.3 LAPACK 221
16.4 GAMS 221
16.5 Othersources 221
17 Unixtools 223
17.1 Automatedbuilds:make 223
17.2 Revisioncontrol:RCS,CVS,SubversionandBitkeeper 226
viii Contents
17.3 Profiling:profandgprof 228
17.4 Textmanipulation:grep,sed,awk,etc. 230
17.5 Othertools 232
17.6 WhataboutMicrosoftWindows? 233
PartV DesignExamples 237
18 Cubicsplinefunctionlibrary 239
18.1 Creationanddestruction 242
18.2 Output 244
18.3 Evaluation 244
18.4 Splineconstruction 247
18.5 Periodicsplines 257
18.6 Performancetesting 260
19 Multigridalgorithms 262
19.1 Discretizingpartialdifferentialequations 262
19.2 Outlineofmultigridmethods 264
19.3 Implementationofframework 265
19.4 Commonchoicesfortheframework 272
19.5 Afirsttest 273
19.6 Theoperatorinterfaceanditsuses 276
19.7 Dealingwithsparsematrices 279
19.8 Asecondtest 282
AppendixA Reviewofvectorsandmatrices 287
A.1 Identitiesandinverses 288
A.2 Normsanderrors 289
A.3 Errorsinsolvinglinearsystems 291
AppendixB Trademarks 292
References 293
Index 299
Preface
Mathematical algorithms, though usually invisible, are all around us. The micro-
computerinyourcarcontrollingthefuelignitionusesacontrolalgorithmembody-
ing mathematical theories of dynamical systems; a Web search engine might use
large-scalematrixcomputations;a“smartmap”usingaGlobalPositioningSystem
totellwhereyouareandthebestwaytogethomeembodiesnumerousnumerical
and non-numerical algorithms; the design of modern aircraft involves simulating
the aerodynamic and structural characteristics on powerful computers including
supercomputers.
Behind these applications is software that does numerical computations. Often
it is called scientific software, or engineering software; this software uses finite-
precision floating-point (and occasionally fixed-point) numbers to represent con-
tinuousquantities.
If you are involved in writing software that does numerical computations, this
book is for you. In it we try to provide tools for writing effective and efficient
numericalsoftware.Ifyouareanumericalanalyst,thisbookmayopenyoureyes
to software issues and techniques that are new to you. If you are a programmer,
thisbookwillexplainpitfallstoavoidwithfloating-pointarithmeticandhowtoget
good performance without losing modern design techniques (or programming in
Fortran66).Peopleinotherareaswithcomputingprojectsthatinvolvesignificant
numerical computation can find a bounty of useful information and techniques in
thisbook.
But this is not a book of numerical recipes, or even a textbook for numerical
analysis(numericalanalysisbeingthestudyofmathematicalalgorithmsandtheir
behavior with finite precision floating-point arithmetic or other sources of com-
putational errors). Nor is it a handbook on software development. It is about the
development of a particular kind of software: numerical software. Several things
makethiskindofsoftwarealittledifferentfromotherkindsofsoftware:
ix
