Digital Signal Processing with Kernel Methods

655 Pages·2018·26.51 MB·English

by  José Luis Rojo-Álvarez

DigitalSignalProcessingwithKernelMethods
Digital Signal Processing with Kernel Methods
JoséLuisRojo-Álvarez
DepartmentofSignalTheoryandCommunications
UniversityReyJuanCarlos
Fuenlabrada(Madrid)
and
CenterforComputationalSimulation
UniversidadPolitécnicadeMadrid,Spain
ManelMartínez-Ramón
DepartmentofElectricalandComputerEngineering
TheUniversityofNewMexico
Albuquerque,NewMexico
USA
JordiMuñoz-Marí
DepartmentofElectronicsEngineering
UniversitatdeValència
Paterna(València),Spain
GustauCamps-Valls
DepartmentofElectronicsEngineering
UniversitatdeValència
Paterna(València),Spain
Thiseditionfirstpublished
©JohnWiley&SonsLtd
Allrightsreserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,or
transmitted,inanyformorbyanymeans,electronic,mechanical,photocopying,recordingorotherwise,
exceptaspermittedbylaw.Adviceonhowtoobtainpermissiontoreusematerialfromthistitleisavailable
athttp://www.wiley.com/go/permissions.
TherightofJoséLuisRojo-Álvarez,ManelMartínez-Ramón,JordiMuñoz-Marí,GustauCamps-Vallstobe
identifiedastheauthorsoftheeditorialmaterialinthisworkhasbeenassertedinaccordancewithlaw.
RegisteredOffices
JohnWiley&Sons,Inc.,RiverStreet,Hoboken,NJ,USA
JohnWiley&SonsLtd,TheAtrium,SouthernGate,Chichester,WestSussex,POSQ,UK
EditorialOffice
TheAtrium,SouthernGate,Chichester,WestSussex,POSQ,UK
Fordetailsofourglobaleditorialoffices,customerservices,andmoreinformationaboutWileyproducts
visitusatwww.wiley.com.
Wileyalsopublishesitsbooksinavarietyofelectronicformatsandbyprint-on-demand.Somecontentthat
appearsinstandardprintversionsofthisbookmaynotbeavailableinotherformats.
LimitofLiability/DisclaimerofWarranty
MATLABⓇandSimulinkisatrademarkofTheMathWorks,Inc.andisusedwithpermission.The
MathWorksdoesnotwarranttheaccuracyofthetextorexercisesinthisbook.Thiswork’suseordiscussion
ofMATLABⓇsoftwareorrelatedproductsdoesnotconstituteendorsementorsponsorshipbyThe
MathWorksofaparticularpedagogicalapproachorparticularuseoftheMATLABⓇsoftware.
Whilethepublisherandauthorshaveusedtheirbesteffortsinpreparingthiswork,theymakeno
representationsorwarrantieswithrespecttotheaccuracyorcompletenessofthecontentsofthisworkand
specificallydisclaimallwarranties,includingwithoutlimitationanyimpliedwarrantiesofmerchantability
orfitnessforaparticularpurpose.Nowarrantymaybecreatedorextendedbysalesrepresentatives,written
salesmaterialsorpromotionalstatementsforthiswork.Thefactthatanorganization,website,orproductis
referredtointhisworkasacitationand/orpotentialsourceoffurtherinformationdoesnotmeanthatthe
publisherandauthorsendorsetheinformationorservicestheorganization,website,orproductmay
provideorrecommendationsitmaymake.Thisworkissoldwiththeunderstandingthatthepublisherisnot
engagedinrenderingprofessionalservices.Theadviceandstrategiescontainedhereinmaynotbesuitable
foryoursituation.Youshouldconsultwithaspecialistwhereappropriate.Further,readersshouldbeaware
thatwebsiteslistedinthisworkmayhavechangedordisappearedbetweenwhenthisworkwaswrittenand
whenitisread.Neitherthepublishernorauthorsshallbeliableforanylossofprofitoranyother
commercialdamages,includingbutnotlimitedtospecial,incidental,consequential,orotherdamages.
LibraryofCongressCataloging-in-Publicationdata
Names:Rojo-Álvarez,JoséLuis,–author.|Martínez-Ramón,Manel,–author.|
Muñoz-Marí,Jordi,author.|Camps-Valls,Gustau,–author.
Title:Digitalsignalprocessingwithkernelmethods/byDr.JoséLuisRojo-Álvarez,
Dr.ManelMartínez-Ramón,Dr.JordiMuñoz-Marí,Dr.GustauCamps-Valls.
Description:Firstedition.|Hoboken,NJ:JohnWiley&Sons,.|Includesbibliographical
referencesandindex.|
Identifiers:LCCN(print)|LCCN(ebook)|ISBN(pdf)|
ISBN(epub)|ISBN(cloth)
Subjects:LCSH:Signalprocessing–Digitaltechniques.
Classification:LCCTK.(ebook)|LCCTK..R(print)|DDC./–dc
LCrecordavailableathttps://lccn.loc.gov/
CoverdesignbyWiley
Coverimage:©AF-studio/Gettyimages
Setin/ptWarnockbySPiGlobal,Pondicherry,India
         
v
Contents
AbouttheAuthors xiii
Preface xvii
Acknowledgements xxi
ListofAbbreviations xxiii
PartI FundamentalsandBasicElements 
1 FromSignalProcessingtoMachineLearning 
. ANewScienceisBorn:SignalProcessing 
.. SignalProcessingBeforeBeingCoined 
.. :BirthoftheInformationAge 
.. s:AudioEngineeringCatalyzesSignalProcessing 
. FromAnalogtoDigitalSignalProcessing 
.. s:DigitalSignalProcessingBegins 
.. s:DigitalSignalProcessingBecomesPopular 
.. s:SiliconMeetsDigitalSignalProcessing 
. DigitalSignalProcessingMeetsMachineLearning 
.. s:NewApplicationAreas 
.. s:NeuralNetworks,FuzzyLogic,andGeneticOptimization 
. RecentMachineLearninginDigitalSignalProcessing 
.. TraditionalSignalAssumptionsAreNoLongerValid 
.. EncodingPriorKnowledge 
.. LearningandKnowledgefromData 
.. FromMachineLearningtoDigitalSignalProcessing 
.. FromDigitalSignalProcessingtoMachineLearning 
2 IntroductiontoDigitalSignalProcessing 
. OutlineoftheSignalProcessingField 
.. FundamentalsonSignalsandSystems 
.. DigitalFiltering 
.. SpectralAnalysis 
.. Deconvolution 
.. Interpolation 
vi Contents
.. SystemIdentification 
.. BlindSourceSeparation 
. FromTime–FrequencytoCompressedSensing 
.. Time–FrequencyDistributions 
.. WaveletTransforms 
.. Sparsity,CompressedSensing,andDictionaryLearning 
. MultidimensionalSignalsandSystems 
.. MultidimensionalSignals 
.. MultidimensionalSystems 
. SpectralAnalysisonManifolds 
.. TheoreticalFundamentals 
.. LaplacianMatrices 
. TutorialsandApplicationExamples 
.. RealandComplexSignalProcessingandRepresentations 
.. Convolution,FourierTransform,andSpectrum 
.. Continuous-TimeSignalsandSystems 
.. FilteringCardiacSignals 
.. NonparametricSpectrumEstimation 
.. ParametricSpectrumEstimation 
.. SourceSeparation 
.. Time–FrequencyRepresentationsandWavelets 
.. ExamplesforSpectralAnalysisonManifolds 
. QuestionsandProblems 
3 SignalProcessingModels 
. Introduction 
. VectorSpaces,Basis,andSignalModels 
.. BasicOperationsforVectors 
.. VectorSpaces 
.. HilbertSpaces 
.. SignalModels 
.. ComplexSignalModels 
.. StandardNoiseModelsinDSP 
.. TheRoleoftheCostFunction 
.. TheRoleoftheRegularizer 
. DigitalSignalProcessingModels 
.. SinusoidalSignalModels 
.. SystemIdentificationSignalModels 
.. SincInterpolationModels 
.. SparseDeconvolution 
.. ArrayProcessing 
. TutorialsandApplicationExamples 
.. ExamplesofNoiseModels 
.. AutoregressiveExogenousSystemIdentificationModels 
.. NonlinearSystemIdentificationUsingVolterraModels 
.. SinusoidalSignalModels 
Contents vii
.. Sinc-basedInterpolation 
.. SparseDeconvolution 
.. ArrayProcessing 
. QuestionsandProblems 
.A MATLABsimpleInterpToolboxStructure 
4 KernelFunctionsandReproducingKernelHilbertSpaces 
. Introduction 
. KernelFunctionsandMappings 
.. MeasuringSimilaritywithKernels 
.. Positive-DefiniteKernels 
.. ReproducingKernelinHilbertSpaceandReproducingProperty 
.. Mercer’sTheorem 
. KernelProperties 
.. Tikhonov’sRegularization 
.. RepresenterTheoremandRegularizationProperties 
.. BasicOperationswithKernels 
. ConstructingKernelFunctions 
.. StandardKernels 
.. PropertiesofKernels 
.. EngineeringSignalProcessingKernels 
. ComplexReproducingKernelinHilbertSpaces 
. SupportVectorMachineElementsforRegressionandEstimation 
.. SupportVectorRegressionSignalModelandCostFunction 
.. MinimizingFunctional 
. TutorialsandApplicationExamples 
.. KernelCalculationsandKernelMatrices 
.. BasicOperationswithKernels 
.. ConstructingKernels 
.. ComplexKernels 
.. ApplicationExampleforSupportVectorRegressionElements 
. ConcludingRemarks 
. QuestionsandProblems 
PartII FunctionApproximationandAdaptiveFiltering 
5 ASupportVectorMachineSignalEstimationFramework 
. Introduction 
. AFrameworkforSupportVectorMachineSignalEstimation 
. PrimalSignalModelsforSupportVectorMachineSignalProcessing 
.. NonparametricSpectrumandSystemIdentification 
.. OrthogonalFrequencyDivisionMultiplexingDigitalCommunications 
.. ConvolutionalSignalModels 
.. ArrayProcessing 
. TutorialsandApplicationExamples 
viii Contents
.. NonparametricSpectralAnalysiswithPrimalSignalModels 
.. SystemIdentificationwithPrimalSignalModelγ-filter 
.. ParametricSpectralDensityEstimationwithPrimalSignalModels 
.. TemporalReferenceArrayProcessingwithPrimalSignalModels 
.. SincInterpolationwithPrimalSignalModels 
.. OrthogonalFrequencyDivisionMultiplexingwithPrimalSignalModels 
. QuestionsandProblems 
6 ReproducingKernelHilbertSpaceModelsforSignalProcessing 
. Introduction 
. ReproducingKernelHilbertSpaceSignalModels 
.. KernelAutoregressiveExogenousIdentification 
.. KernelFiniteImpulseResponseandtheγ-filter 
.. KernelArrayProcessingwithSpatialReference 
.. KernelSemiparametricRegression 
. TutorialsandApplicationExamples 
.. NonlinearSystemIdentificationwithSupportVector
Machine–AutoregressiveandMovingAverage 
.. NonlinearSystemIdentificationwiththeγ-filter 
.. ElectricNetworkModelingwithSemiparametricRegression 
.. PromotionalData 
.. SpatialandTemporalAntennaArrayKernelProcessing 
. QuestionsandProblems 
7 DualSignalModelsforSignalProcessing 
. Introduction 
. DualSignalModelElements 
. DualSignalModelInstantiations 
.. DualSignalModelforNonuniformSignalInterpolation 
.. DualSignalModelforSparseSignalDeconvolution 
.. SpectrallyAdaptedMercerKernels 
. TutorialsandApplicationExamples 
.. NonuniformInterpolationwiththeDualSignalModel 
.. SparseDeconvolutionwiththeDualSignalModel 
.. DopplerUltrasoundProcessingforFaultDetection 
.. SpectrallyAdaptedMercerKernels 
.. InterpolationofHeartRateVariabilitySignals 
.. DenoisinginCardiacMotion-ModeDopplerUltrasoundImages 
.. IndoorLocationfromMobileDevicesMeasurements 
.. ElectroanatomicalMapsinCardiacNavigationSystems 
. QuestionsandProblems 
8 AdvancesinKernelRegressionandFunctionApproximation 
. Introduction 
. Kernel-BasedRegressionMethods 
.. AdvancesinSupportVectorRegression 
.. Multi-outputSupportVectorRegression 
Contents ix
.. KernelRidgeRegression 
.. KernelSignal-to-NoiseRegression 
.. Semi-supervisedSupportVectorRegression 
.. ModelSelectioninKernelRegressionMethods 
. BayesianNonparametricKernelRegressionModels 
.. GaussianProcessRegression 
.. RelevanceVectorMachines 
. TutorialsandApplicationExamples 
.. ComparingSupportVectorRegression,RelevanceVectorMachines,and
GaussianProcessRegression 
.. Profile-DependentSupportVectorRegression 
.. Multi-outputSupportVectorRegression 
.. KernelSignal-to-NoiseRatioRegression 
.. Semi-supervisedSupportVectorRegression 
.. BayesianNonparametricModel 
.. GaussianProcessRegression 
.. RelevanceVectorMachines 
. ConcludingRemarks 
. QuestionsandProblems 
9 AdaptiveKernelLearningforSignalProcessing 
. Introduction 
. LinearAdaptiveFiltering 
.. LeastMeanSquaresAlgorithm 
.. RecursiveLeast-SquaresAlgorithm 
. KernelAdaptiveFiltering 
. KernelLeastMeanSquares 
.. DerivationofKernelLeastMeanSquares 
.. ImplementationChallengesandDualFormulation 
.. ExampleonPredictionoftheMackey–GlassTimeSeries 
.. PracticalKernelLeastMeanSquaresAlgorithms 
. KernelRecursiveLeastSquares 
.. KernelRidgeRegression 
.. DerivationofKernelRecursiveLeastSquares 
.. PredictionoftheMackey–GlassTimeSerieswithKernelRecursiveLeast
Squares 
.. BeyondtheStationaryModel 
.. ExampleonNonlinearChannelIdentificationandReconvergence 
. ExplicitRecursivityforAdaptiveKernelModels 
.. RecursivityinHilbertSpaces 
.. RecursiveFiltersinReproducingKernelHilbertSpaces 
. OnlineSparsificationwithKernels 
.. SparsitybyConstruction 
.. SparsitybyPruning 
. ProbabilisticApproachestoKernelAdaptiveFiltering 
.. GaussianProcessesandKernelRidgeRegression 
.. OnlineRecursiveSolutionforGaussianProcessesRegression 
x Contents
.. KernelRecursiveLeastSquaresTracker 
.. ProbabilisticKernelLeastMeanSquares 
. FurtherReading 
.. SelectionofKernelParameters 
.. Multi-KernelAdaptiveFiltering 
.. RecursiveFilteringinKernelHilbertSpaces 
. TutorialsandApplicationExamples 
.. KernelAdaptiveFilteringToolbox 
.. PredictionofaRespiratoryMotionTimeSeries 
.. OnlineRegressionontheKINKDataset 
.. TheMackey–GlassTimeSeries 
.. ExplicitRecursivityonReproducingKernelinHilbertSpace
andElectroencephalogramPrediction 
.. AdaptiveAntennaArrayProcessing 
. QuestionsandProblems 
PartIII Classification,Detection,andFeatureExtraction 
10 SupportVectorMachineandKernelClassificationAlgorithms 
. Introduction 
. SupportVectorMachineandKernelClassifiers 
.. SupportVectorMachines 
.. MulticlassandMultilabelSupportVectorMachines 
.. Least-SquaresSupportVectorMachine 
.. KernelFisher’sDiscriminantAnalysis 
. AdvancesinKernel-BasedClassification 
.. LargeMarginFiltering 
.. Semi-supervisedLearning 
.. MultipleKernelLearning 
.. Structured-OutputLearning 
.. ActiveLearning 
. Large-ScaleSupportVectorMachines 
.. Large-ScaleSupportVectorMachineImplementations 
.. RandomFourierFeatures 
.. ParallelSupportVectorMachine 
.. Outlook 
. TutorialsandApplicationExamples 
.. ExamplesofSupportVectorMachineClassification 
.. ExampleofLeast-SquaresSupportVectorMachine 
.. Kernel-FilteringSupportVectorMachineforBrain–ComputerInterface
SignalClassification 
.. ExampleofLaplacianSupportVectorMachine 
.. ExampleofGraph-BasedLabelPropagation 
.. ExamplesofMultipleKernelLearning 
. ConcludingRemarks 
. QuestionsandProblems 