Data Analysis: Statistical and Computational Methods for Scientists and Engineers

532 Pages·2014·8.76 MB·English

by  Siegmund Brandt

Siegmund Brandt
Data Analysis
Statistical and Computational Methods
for Scientists and Engineers
Fourth Edition
Data Analysis

Siegmund Brandt
Data Analysis
Statistical and Computational Methods
for Scientists and Engineers
Fourth Edition
Translated by Glen Cowan
123
SiegmundBrandt
DepartmentofPhysics
UniversityofSiegen
Siegen,Germany
Additionalmaterialtothisbookcanbedownloadedfromhttp://extras.springer.com
ISBN978-3-319-03761-5 ISBN978-3-319-03762-2(eBook)
DOI10.1007/978-3-319-03762-2
SpringerChamHeidelbergNewYorkDordrechtLondon
LibraryofCongressControlNumber:2013957143
©SpringerInternationalPublishingSwitzerland2014
Thisworkissubjecttocopyright. AllrightsarereservedbythePublisher, whetherthewholeorpartofthema-
terialisconcerned,specifically therightsoftranslation, reprinting, reuseofillustrations, recitation, broadcasting,
reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformationstorageandretrieval,elec-
tronicadaptation,computersoftware,orbysimilarordissimilarmethodologynowknownorhereafterdeveloped.
Exemptedfromthislegalreservationarebriefexcerptsinconnectionwithreviewsorscholarlyanalysisormaterial
suppliedspecificallyforthepurposeofbeingenteredandexecutedonacomputersystem,forexclusiveusebythe
purchaser ofthework. Duplication ofthis publication orparts thereof ispermitted onlyunder theprovisions of
theCopyrightLawofthePublisher’s location, initscurrentversion,andpermissionforusemustalways beob-
tainedfromSpringer.PermissionsforusemaybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.
ViolationsareliabletoprosecutionundertherespectiveCopyrightLaw.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublicationdoesnot
imply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevantprotectivelawsand
regulationsandthereforefreeforgeneraluse.
Whiletheadviceandinformationinthisbookarebelievedtobetrueandaccurateatthedateofpublication,neither
theauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityforanyerrorsoromissionsthatmay
bemade.Thepublishermakesnowarranty,expressorimplied,withrespecttothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Preface to the Fourth English Edition
For the present edition, the book has undergone two major changes: Its
appearance was tightened significantly and the programs are now written in
themodernprogramminglanguageJava.
Tightening was possible without giving up essential contents by expedi-
ent use of the Internet. Since practically all users can connect to the net, it is
no longer necessary to reproduce program listings in the printed text. In this
way,thephysicalsizeofthebookwas reduced considerably.
TheJavalanguageoffersanumberofadvantagesovertheolderprogram-
ming languages used in earlier editions. It is object-oriented and hence also
more readable. It includes access to libraries of user-friendly auxiliary rou-
tines, allowing for instance the easy creation of windows for input, output,
orgraphics. Formost popularcomputers,Javais either preinstalledorcan be
downloadedfromtheInternetfreeofcharge.(SeeSect.1.3fordetails.)Since
by now Java is often taught at school, many students are already somewhat
familiarwiththelanguage.
Our Java programs for data analysis and for the production of graphics,
including many example programs and solutions to programming problems,
can bedownloadedfrom thepageextras.springer.com.
I am grateful to Dr. Tilo Stroh for numerous stimulatingdiscussions and
technicalhelp. Thegraphicsprogramsare based onpreviouscommonwork.
Siegen,Germany SiegmundBrandt
v

Contents
Preface to the FourthEnglishEdition v
ListofExamples xv
Frequently UsedSymbols andNotation xix
1 Introduction 1
1.1 Typical ProblemsofDataAnalysis . . . . . . . . . . . . . . . 1
1.2 On theStructureofthisBook . . . . . . . . . . . . . . . . . . 2
1.3 AbouttheComputerPrograms . . . . . . . . . . . . . . . . . . 5
2 Probabilities 7
2.1 Experiments,Events,SampleSpace . . . . . . . . . . . . . . . 7
2.2 TheConcept ofProbability . . . . . . . . . . . . . . . . . . . . 8
2.3 Rules ofProbabilityCalculus:ConditionalProbability . . . . 10
2.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.4.1 ProbabilityfornDotsin theThrowingofTwo
Dice . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.4.2 Lottery6 Outof49 . . . . . . . . . . . . . . . . . . . 12
2.4.3 Three-DoorGame . . . . . . . . . . . . . . . . . . . . 13
3 Random Variables:Distributions 15
3.1 Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.2 Distributionsofa SingleRandom Variable . . . . . . . . . . . 15
3.3 FunctionsofaSingleRandomVariable,ExpectationValue,
Variance, Moments . . . . . . . . . . . . . . . . . . . . . . . . 17
3.4 DistributionFunctionand ProbabilityDensityofTwo
Variables: ConditionalProbability . . . . . . . . . . . . . . . . 25
3.5 ExpectationValues, Variance, Covariance,and Correlation. . 27
vii
viii Contents
3.6 MorethanTwo Variables:Vectorand MatrixNotation . . . . 30
3.7 TransformationofVariables . . . . . . . . . . . . . . . . . . . 33
3.8 Linearand OrthogonalTransformations:Error
Propagation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4 Computer Generated Random Numbers: The Monte
Carlo Method 41
4.1 Random Numbers . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.2 Representation ofNumbersinaComputer . . . . . . . . . . . 42
4.3 LinearCongruentialGenerators . . . . . . . . . . . . . . . . . 44
4.4 MultiplicativeLinearCongruentialGenerators. . . . . . . . . 45
4.5 Qualityofan MLCG:Spectral Test . . . . . . . . . . . . . . . 47
4.6 Implementationand Portabilityofan MLCG. . . . . . . . . . 50
4.7 CombinationofSeveral MLCGs. . . . . . . . . . . . . . . . . 52
4.8 Generation ofArbitrarilyDistributedRandom Numbers . . . 55
4.8.1 Generation byTransformationoftheUniform
Distribution . . . . . . . . . . . . . . . . . . . . . . . . 55
4.8.2 GenerationwiththevonNeumannAcceptance–Re-
jectionTechnique . . . . . . . . . . . . . . . . . . . . 58
4.9 Generation ofNormallyDistributedRandomNumbers . . . . 62
4.10 Generation ofRandomNumbersAccording
to aMultivariateNormal Distribution . . . . . . . . . . . . . . 63
4.11 TheMonteCarlo MethodforIntegration . . . . . . . . . . . . 64
4.12 TheMonteCarlo MethodforSimulation . . . . . . . . . . . . 66
4.13 JavaClasses andExamplePrograms . . . . . . . . . . . . . . 67
5 Some ImportantDistributions and Theorems 69
5.1 TheBinomialand MultinomialDistributions . . . . . . . . . 69
5.2 Frequency: TheLawofLarge Numbers . . . . . . . . . . . . 72
5.3 TheHypergeometricDistribution . . . . . . . . . . . . . . . . 74
5.4 ThePoissonDistribution . . . . . . . . . . . . . . . . . . . . . 78
5.5 TheCharacteristicFunction ofa Distribution . . . . . . . . . 81
5.6 TheStandard NormalDistribution. . . . . . . . . . . . . . . . 84
5.7 TheNormalorGaussianDistribution . . . . . . . . . . . . . . 86
5.8 QuantitativeProperties oftheNormalDistribution . . . . . . 88
5.9 TheCentral LimitTheorem . . . . . . . . . . . . . . . . . . . 90
5.10 TheMultivariateNormalDistribution . . . . . . . . . . . . . . 94
5.11 ConvolutionsofDistributions . . . . . . . . . . . . . . . . . . 100
5.11.1 FoldingIntegrals . . . . . . . . . . . . . . . . . . . . . 100
5.11.2 ConvolutionswiththeNormalDistribution . . . . . . 103
5.12 ExamplePrograms . . . . . . . . . . . . . . . . . . . . . . . . 106
Contents ix
6 Samples 109
6.1 Random Samples.Distribution
ofaSample. Estimators. . . . . . . . . . . . . . . . . . . . . . 109
6.2 Samples fromContinuousPopulations:Mean
and Varianceofa Sample . . . . . . . . . . . . . . . . . . . . . 111
6.3 Graphical Representation ofSamples:Histograms
and Scatter Plots . . . . . . . . . . . . . . . . . . . . . . . . . . 115
6.4 Samples fromPartitionedPopulations . . . . . . . . . . . . . 122
6.5 Samples WithoutReplacement from FiniteDiscrete
Populations.MeanSquare Deviation.Degrees of
Freedom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
6.6 Samples fromGaussian Distributions:χ2-Distribution . . . . 130
6.7 χ2 and EmpiricalVariance . . . . . . . . . . . . . . . . . . . . 135
6.8 Samplingby Counting:Small Samples . . . . . . . . . . . . . 136
6.9 Small SampleswithBackground . . . . . . . . . . . . . . . . 142
6.10 DeterminingaRatio ofSmallNumbersofEvents . . . . . . . 144
6.11 Ratio ofSmallNumbers ofEventswithBackground . . . . . 147
6.12 JavaClasses andExamplePrograms . . . . . . . . . . . . . . 149
7 The Method ofMaximum Likelihood 153
7.1 LikelihoodRatio: LikelihoodFunction . . . . . . . . . . . . . 153
7.2 TheMethodofMaximumLikelihood . . . . . . . . . . . . . . 155
7.3 InformationInequality.MinimumVariance
Estimators.Sufficient Estimators . . . . . . . . . . . . . . . . 157
7.4 AsymptoticProperties oftheLikelihoodFunction
and Maximum-LikelihoodEstimators. . . . . . . . . . . . . . 164
7.5 SimultaneousEstimationofSeveral Parameters:
Confidence Intervals . . . . . . . . . . . . . . . . . . . . . . . 167
7.6 ExamplePrograms . . . . . . . . . . . . . . . . . . . . . . . . 173
8 Testing StatisticalHypotheses 175
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
8.2 F-Test onEqualityofVariances . . . . . . . . . . . . . . . . . 177
8.3 Student’sTest:ComparisonofMeans . . . . . . . . . . . . . . 180
8.4 Concepts oftheGeneral TheoryofTests . . . . . . . . . . . . 185
8.5 TheNeyman–PearsonLemmaandApplications . . . . . . . . 191
8.6 TheLikelihood-RatioMethod . . . . . . . . . . . . . . . . . . 194
8.7 Theχ2-Test forGoodness-of-Fit . . . . . . . . . . . . . . . . 199
8.7.1 χ2-Test withMaximalNumberofDegrees
ofFreedom . . . . . . . . . . . . . . . . . . . . . . . . 199
8.7.2 χ2-Test withReduced NumberofDegrees
ofFreedom . . . . . . . . . . . . . . . . . . . . . . . . 200
8.7.3 χ2-Test and EmpiricalFrequency Distribution . . . . 200