Bayesian Signal Processing: Classical, Modern and Particle Filtering Methods (Adaptive and Learning Systems for Signal Processing, Communications and Control Series)

466 Pages·2009·9.16 MB·English

by  James V. Candy

ffirs.tex 1/4/2009 16:25 Pagei
BAYESIAN SIGNAL
PROCESSING
ffirs.tex 1/4/2009 16:25 Pageiii
BAYESIAN SIGNAL
PROCESSING
Classical, Modern, and
Particle Filtering Methods
James V. Candy
LawrenceLivermoreNationalLaboratory
UniversityofCaliforniaSantaBarbara
A JOHN WILEY & SONS, INC., PUBLICATION
ffirs.tex 1/4/2009 16:25 Pageiv
Copyright©2009byJohnWiley&Sons,Inc.Allrightsreserved
PublishedbyJohnWiley&Sons,Inc.,Hoboken,NewJersey
PublishedsimultaneouslyinCanada
Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,ortransmittedin
anyformorbyanymeans,electronic,mechanical,photocopying,recording,scanning,orotherwise,
exceptaspermittedunderSection107or108ofthe1976UnitedStatesCopyrightAct,withouteither
thepriorwrittenpermissionofthePublisher,orauthorizationthroughpaymentoftheappropriate
per-copyfeetotheCopyrightClearanceCenter,Inc.,222RosewoodDrive,Danvers,MA01923,
[phone],fax[phone],oronthewebatwww.copyright.com.RequeststothePublisher
forpermissionshouldbeaddressedtothePermissionsDepartment,JohnWiley&Sons,Inc.,
IllRiverStreet,Hoboken,NJ07030,[phone],fax[phone],oronlineat
http://www.wiley.com/go/permission.
LimitofLiability/DisclaimerofWarranty:Whilethepublisherandauthorhaveusedtheirbestefforts
inpreparingthisbook,theymakenorepresentationsorwarrantieswithrespecttotheaccuracyor
completenessofthecontentsofthisbookandspecificallydisclaimanyimpliedwarrantiesof
merchantabilityorfitnessforaparticularpurpose.Nowarrantymaybecreatedorextendedby
salesrepresentativesorwrittensalesmaterials.Theadviceandstrategiescontainedhereinmaynot
besuitableforyoursituation.Youshouldconsultwithaprofessionalwhereappropriate.Neither
thepublishernorauthorshallbeliableforanylossofprofitoranyothercommercialdamages,
includingbutnotlimitedtospecial,incidental,consequential,orotherdamages.
Forgeneralinformationonourotherproductsandservicesorfortechnicalsupport,pleasecontact
ourCustomerCareDepartmentwithintheUnitedStatesat(800)762-2974,outsidetheUnitedStates
at(317)572-3993orfax(317)572-4002.
Wileyalsopublishesitsbooksinavarietyofelectronicformats.Somecontentthatappearsinprint
maynotbeavailableinelectronicformats.FormoreinformationaboutWileyproducts,visitour
websiteatwww.wiley.com.
LibraryofCongressCataloging-in-PublicationData:
Bayesiansignalprocessing:classical,modern,andparticlefilteringmethods/JamesV.Candy.,
p.cm.
Includesbibliographicalreferences.
ISBN978-0-470-18094-5(cloth)
1. Signalprocessing—Mathematics. 2. Bayesianstatisticaldecisiontheory. I. Title.
TK5102.9.C3187 2008
621.382(cid:1)2–dc22
2008032184
PrintedintheUnitedStatesofAmerica
10 9 8 7 6 5 4 3 2 1
ffirs.tex 1/4/2009 16:25 Pagev
Peace,realpeace,canonlybefoundthroughthe
Lord,ourSavior,JesusChrist!
ftoc.tex 12/2/2009 16:28 Pagevii
CONTENTS
Preface xiii
ReferencestothePreface xix
Acknowledgments xxiii
1 Introduction 1
1.1 Introduction 1
1.2 BayesianSignalProcessing 1
1.3 Simulation-BasedApproachtoBayesianProcessing 4
1.4 BayesianModel-BasedSignalProcessing 8
1.5 NotationandTerminology 12
References 14
Problems 15
2 BayesianEstimation 19
2.1 Introduction 19
2.2 BatchBayesianEstimation 19
2.3 BatchMaximumLikelihoodEstimation 22
2.3.1 Expectation-MaximizationApproach
toMaximumLikelihood 25
2.3.2 EMforExponentialFamilyofDistributions 30
2.4 BatchMinimumVarianceEstimation 33
2.5 SequentialBayesianEstimation 36
2.5.1 JointPosteriorEstimation 39
2.5.2 FilteringPosteriorEstimation 41
2.6 Summary 43
References 44
Problems 45
vii
ftoc.tex 12/2/2009 16:28 Pageviii
viii CONTENTS
3 Simulation-BasedBayesianMethods 51
3.1 Introduction 51
3.2 ProbabilityDensityFunctionEstimation 53
3.3 SamplingTheory 56
3.3.1 UniformSamplingMethod 58
3.3.2 RejectionSamplingMethod 62
3.4 MonteCarloApproach 64
3.4.1 MarkovChains 70
3.4.2 Metropolis-HastingsSampling 71
3.4.3 RandomWalkMetropolis-HastingsSampling 73
3.4.4 GibbsSampling 75
3.4.5 SliceSampling 78
3.5 ImportanceSampling 81
3.6 SequentialImportanceSampling 84
3.7 Summary 87
References 87
Problems 90
4 State–SpaceModelsforBayesianProcessing 95
4.1 Introduction 95
4.2 Continuous-TimeState–SpaceModels 96
4.3 Sampled-DataState–SpaceModels 100
4.4 Discrete-TimeState–SpaceModels 104
4.4.1 DiscreteSystemsTheory 107
4.5 Gauss-MarkovState–SpaceModels 112
4.5.1 Continuous-Time/Sampled-DataGauss-MarkovModels 112
4.5.2 Discrete-TimeGauss-MarkovModels 114
4.6 InnovationsModel 120
4.7 State–SpaceModelStructures 121
4.7.1 TimeSeriesModels 121
4.7.2 State–SpaceandTimeSeriesEquivalenceModels 129
4.8 Nonlinear(Approximate)Gauss-MarkovState–SpaceModels 135
4.9 Summary 139
References 140
Problems 141
5 ClassicalBayesianState–SpaceProcessors 147
5.1 Introduction 147
5.2 BayesianApproachtotheState–Space 147
5.3 LinearBayesianProcessor(LinearKalmanFilter) 150
5.4 LinearizedBayesianProcessor(LinearizedKalmanFilter) 160
5.5 ExtendedBayesianProcessor(ExtendedKalmanFilter) 167
ftoc.tex 12/2/2009 16:28 Pageix
CONTENTS ix
5.6 Iterated-ExtendedBayesianProcessor(Iterated-Extended
KalmanFilter) 174
5.7 PracticalAspectsofClassicalBayesianProcessors 182
5.8 CaseStudy:RLCCircuitProblem 186
5.9 Summary 191
References 191
Problems 193
6 ModernBayesianState–SpaceProcessors 197
6.1 Introduction 197
6.2 Sigma-Point(Unscented)Transformations 198
6.2.1 StatisticalLinearization 198
6.2.2 Sigma-PointApproach 200
6.2.3 SPTforGaussianPriorDistributions 205
6.3 Sigma-PointBayesianProcessor(UnscentedKalmanFilter) 209
6.3.1 ExtensionsoftheSigma-PointProcessor 218
6.4 QuadratureBayesianProcessors 218
6.5 GaussianSum(Mixture)BayesianProcessors 220
6.6 CaseStudy:2D-TrackingProblem 224
6.7 Summary 230
References 231
Problems 233
7 Particle-BasedBayesianState–SpaceProcessors 237
7.1 Introduction 237
7.2 BayesianState–SpaceParticleFilters 237
7.3 ImportanceProposalDistributions 242
7.3.1 MinimumVarianceImportanceDistribution 242
7.3.2 TransitionPriorImportanceDistribution 245
7.4 Resampling 246
7.4.1 MultinomialResampling 249
7.4.2 SystematicResampling 251
7.4.3 ResidualResampling 251
7.5 State–SpaceParticleFilteringTechniques 252
7.5.1 BootstrapParticleFilter 253
7.5.2 AuxiliaryParticleFilter 261
7.5.3 RegularizedParticleFilter 264
7.5.4 MCMCParticleFilter 266
7.5.5 LinearizedParticleFilter 270
7.6 PracticalAspectsofParticleFilterDesign 272
7.6.1 PosteriorProbabilityValidation 273
7.6.2 ModelValidationTesting 277
7.7 CaseStudy:PopulationGrowthProblem 285
7.8 Summary 289
ftoc.tex 12/2/2009 16:28 Pagex
x CONTENTS
References 290
Problems 293
8 JointBayesianState/ParametricProcessors 299
8.1 Introduction 299
8.2 BayesianApproachtoJointState/ParameterEstimation 300
8.3 Classical/ModernJointBayesianState/ParametricProcessors 302
8.3.1 ClassicalJointBayesianProcessor 303
8.3.2 ModernJointBayesianProcessor 311
8.4 Particle-BasedJointBayesianState/ParametricProcessors 313
8.5 CaseStudy:RandomTargetTrackingUsingaSynthetic
ApertureTowedArray 318
8.6 Summary 327
References 328
Problems 330
9 DiscreteHiddenMarkovModelBayesianProcessors 335
9.1 Introduction 335
9.2 HiddenMarkovModels 335
9.2.1 Discrete-TimeMarkovChains 336
9.2.2 HiddenMarkovChains 337
9.3 PropertiesoftheHiddenMarkovModel 339
9.4 HMM ObservationProbability:EvaluationProblem 341
9.5 StateEstimationinHMM:TheViterbiTechnique 345
9.5.1 IndividualHiddenStateEstimation 345
9.5.2 EntireHiddenStateSequenceEstimation 347
9.6 ParameterEstimationinHMM:
TheEM/Baum-WelchTechnique 350
9.6.1 ParameterEstimationwithStateSequenceKnown 352
9.6.2 ParameterEstimationwithStateSequenceUnknown 354
9.7 CaseStudy:Time-ReversalDecoding 357
9.8 Summary 362
References 363
Problems 365
10 BayesianProcessorsforPhysics-BasedApplications 369
10.1 OptimalPositionEstimationfortheAutomaticAlignment 369
10.1.1 Background 369
10.1.2 StochasticModelingofPositionMeasurements 372
10.1.3 BayesianPositionEstimationandDetection 374
10.1.4 Application:BeamLineData 375
10.1.5 Results:BeamLine(KDPDeviation)Data 377
10.1.6 Results:AnomalyDetection 379
ftoc.tex 12/2/2009 16:28 Pagexi
CONTENTS xi
10.2 BroadbandOceanAcousticProcessing 382
10.2.1 Background 382
10.2.2 BroadbandState–SpaceOceanAcoustic
Propagators 384
10.2.3 BroadbandBayesianProcessing 389
10.2.4 BroadbandBSPDesign 393
10.2.5 Results 395
10.3 BayesianProcessingforBiothreats 397
10.3.1 Background 397
10.3.2 ParameterEstimation 400
10.3.3 BayesianProcessorDesign 401
10.3.4 Results 403
10.4 BayesianProcessingfortheDetectionofRadioactiveSources 404
10.4.1 Background 404
10.4.2 Physics-BasedModels 404
10.4.3 Gamma-RayDetectorMeasurements 407
10.4.4 BayesianPhysics-BasedProcessor 410
10.4.5 Physics-BasedBayesianDeconvolutionProcessor 412
10.4.6 Results 415
References 417
AppendixA Probability&StatisticsOverview 423
A.1 ProbabilityTheory 423
A.2 GaussianRandomVectors 429
A.3 UncorrelatedTransformation:
GaussianRandomVectors 430
References 430
Index 431