Fundamentals of Adaptive Signal Processing

725 Pages·2015·7.89 MB·English

by  Aurelio Uncini (auth.)

Signals and Communication Technology
Aurelio Uncini
Fundamentals
of Adaptive
Signal
Processing
Signals and Communication Technology
For furthervolumes:
http://www.springer.com/series/4748
ThiSisaFMBlankPage
Aurelio Uncini
Fundamentals of Adaptive
Signal Processing
AurelioUncini
DIET
SapienzaUniversityofRome
Rome
Italy
ISSN1860-4862 ISSN1860-4870(electronic)
ISBN978-3-319-02806-4 ISBN978-3-319-02807-1(eBook)
DOI10.1007/978-3-319-02807-1
SpringerChamHeidelbergNewYorkDordrechtLondon
LibraryofCongressControlNumber:2014958676
©SpringerInternationalPublishingSwitzerland2015
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionor
informationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodologynowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerpts
inconnectionwithreviewsorscholarlyanalysisormaterialsuppliedspecificallyforthepurposeofbeing
enteredandexecutedonacomputersystem,forexclusiveusebythepurchaserofthework.Duplication
ofthispublicationorpartsthereofispermittedonlyundertheprovisionsoftheCopyrightLawofthe
Publisher’s location, in its current version, and permission for use must always be obtained from
Springer.PermissionsforusemaybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.
ViolationsareliabletoprosecutionundertherespectiveCopyrightLaw.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexempt
fromtherelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
While the advice and information in this book are believed to be true and accurate at the date of
publication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityfor
anyerrorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,with
respecttothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Preface
Sincetheancienttimesofhumanhistory,therehavebeenmanyattemptstodefine
intelligence. Aristotle argued that all persons express similar intellectual faculties
and that the differences were due to the teaching and example. In more recent
times,intelligencehasbeendefinedasasetofinnatecognitivefunctions,adaptive,
imaginative, etc., arising from a human or animal biological brain. Among these,
the capacity of adaptation is the main prerogative present in all definitions of
intelligent behavior. From the biological point of view, adaptation is a property
thatalllivingorganismspossessandthatcanbeinterpretedbothasapropensityto
the species improvement and as a conservative process tending to the species
preservationovertimeofthelife.Fromthepsychologicalpointofview,adaptation
issynonymouswithlearning.Inthissense,learningisabehavioralfunction,more
orlessconscious,ofasubjectthatadaptsherorhisattitudeasaresultofexperience:
learningistoadapt.
In intelligent systems, whether biologically inspired, or entirely
artificial,adaptationandmethodstocarryitoutrepresentanessentialprerogative.
In this framework, adaptive filters are defined as information processing systems,
analog or digital, capable of autonomously adjusting their parameters in response
to external stimuli. In other words, the system learns independently and adapts
its parameters to achieve a certain processing goal such as extracting the useful
informationfromanacquiredsignalandtheremovalofdisturbancesduetonoiseor
other sources interfering or, more generally, the adaptive filter provides the elim-
ination of the redundant information. In fact, in support of this hypothesis, the
BritishneuroscientistHoraceB.Barlowin1953discoveredthatthefrogbrainhas
neuronswhichfireinresponsetospecificvisualstimuliandconcludedthatoneof
themainaimsofvisualprocessingisthereductionofredundancy.Hisworkshave
been milestones in the study of the properties of the biological nervous system.
Indeed,hisresearchesdemonstratethatthemainfunctionofamachineperception
istoeliminateredundantinformationcomingfromreceptors.
The usability of adaptive signal processing methods to the solution of real
problems is extensive and represents a paradigm for many strategic applications.
Adaptivesignalprocessingmethodsareusedineconomicandfinancialsciences,in
v
vi Preface
engineering and social sciences, in medicine, biology, and neuroscience, and
inmanyotherareasofhighstrategicinterest.Adaptivesignalprocessingisalsoa
veryactivefieldofstudyandresearchthat,forathoroughunderstanding,requires
advancedinterdisciplinaryknowledge.
Objectives of the Text
The aim of this book is to provide advanced theoretical and practical tools for
the study and determination of circuit structures and robust algorithms for the
adaptive signals processing in different application scenarios. Example can be
found in multimodal and multimedia communications, the biological and bio-
medical areas, economic model, environmental sciences, acoustics, telecommuni-
cations, remote sensing, monitoring, and, in general, modeling and prediction of
complexphysicalphenomena.
Inparticular,inadditiontopresentingthefundamentaltheoreticalbaseconcepts,
the most important adaptivealgorithms are introduced, while also providing tools
to evaluate the algorithms’ performance. The reader, in addition to acquiring the
basic theories, will be able to design and implement the algorithms and evaluate
theirperformanceforspecificapplications.
The idea of the text is based on years of teaching activities of the author,
during the course Algorithms for Adaptive Signal Processing held at the Faculty
of Information Engineering of “Sapienza” University of Rome. In preparing
the book, particular attention was paid in the first chapters and in mathematics
appendices,whichmakethetextsuitabletotheirreaderswithoutspecialprerequi-
sites, other than those common to all first 3-year courses of the Information
EngineeringFacultyandotherscientificfaculties.
Adaptivefiltersarenonstationarynonlinearandtime-varyingdynamicsystems,
and,attimes,toavoidasimplisticapproach,theargumentsmayhavesomeconceits
that might be difficult to understand. For this reason, many of the subjects are
introduced by considering different points of view and with multiple levels of
analysis.
In the literature on this topic numerous and authoritative texts are available.
The reasons that led to the writing of this work are linked to a philosophically
differentvisionofintelligent signalprocessing.Infact,adaptivefilteringmethods
canbeintroducedstartingfromdifferenttheories.Inthisworkwewantedtoavoid
an “ideological” approach related to some specific discipline, but we wanted to
put an emphasis on interdisciplinarity presenting the most important topics with
differentparadigms.
For example, a central importance argument, as the least mean squares (LMS)
algorithm, is exposed in three distinct and, to some extent, original ways.
Inthefirst,followingamoresystemisticcriteria,theLMSispresentedbyconsid-
ering an energy approach through the Lyapunov attractor. In the second, with a
“classic” statistical approach, is introduced the stochastic approximation of the
Preface vii
gradient-descent optimization method. In the third mode, following a different
approach, is presented considering the simple axiomatic properties of minimal
perturbation. Moreover, it should be noted that this philosophy is not only a
pedagogicalexercise,butitisoffundamentalimportanceinmoreadvancedtopics
andtheoreticaldemonstrationswhere,followingaphilosophyratherthantheother,
ithappensveryoftentotreadwindingroadsanddeadend.
Organization and Structure of the Book
Thesequenceofargumentsispresentedinclassicalmode.Thefirstpartintroduces
thebasicconceptsofoptimallinearfiltering.Followingthefirstandsecondorder,
onlineandbatchprocessingtechniquesarepresented.Aparticularefforthasbeen
made in trying to present arguments with a common formalism, while trying to
remainfaithfultotheconsideredoriginalreferences.Theentirenotationisdefined
indiscretetimeandthealgorithmsarepresentedinordertofacilitatethereaderto
the writing of computer programs, for the practical realization of the applications
describedinthetext.
The book consists of nine chapters, each one containing the references
where the reader can independently deepen topics of particular interest, and three
mathematicsappendices.
Chapter1coverspreliminarytopicsofthediscrete-timesignalsandcircuitsand
somebasicmethodsofdigitalsignalprocessing.
Chapter 2 introduces the basic definitions of adaptive filtering theory and
the main filters topologies are discussed. In addition, the concept of cost function
to be minimized and the main philosophies concerning adaptation methods are
introduced. Finally, the main application fields of adaptive signal processing
techniquesarepresentedanddiscussed.
In Chap. 3, the Wiener optimal filtering theory is presented. In particular,
the problems of the mean square error minimization and of its optimal value
determination are addressed. The formulation of the normal equations and the
optimal Wiener filter in discrete time is introduced. Moreover, the type 1, 2, and
3multichannelnotationsanditsmulti-input-outputoptimalfiltergeneralizationare
presented.Arealsodiscussed corollaries, and presentedsomeapplications related
totherandomsequencespredictionandestimation.
In Chap. 4, adaptation methods, in the case that the input signals are not
statistically characterized, are addressed. The principle of least squares (LS),
bringing back the estimation problem into an optimization algorithm, is intro-
duced.ThenormalequationsintheYule–Walkerformulationareintroducedand
the similarities and differences with Wiener optimal filtering theory are also
discussed. Moreover, the minimum variance optimal estimators, the normal
equations weighing techniques, the regularization LS approach, and the linearly
constrained and the nonlinear LS techniques are introduced. The algebraic
methods to matrix decomposition forsolving the LS systems in the cases and of
viii Preface
over/under-determined equations system are also introduced and discussed. The
technique of singular value decomposition in the solution of the LS systems is
discussed. The method of Lyapunov attractor for the iterative LS solution is
presented, and the least mean squares and Kaczmarz algorithms, seen as an
iterative LS solution, are introduced. Finally, the methodology of total least
squares (TLS) and the matching pursuit algorithms for underdetermined sparse
LSsystemsarepresentedanddiscussed.
Chapter 5 introduces the first-order adaptation algorithms for online adaptive
filtering. The methods are presented with a classical statistical approach and
the LMS algorithm with the stochastic gradient paradigm. In addition, methods
for performance evaluation of adaptation algorithms, with particular reference to
the convergence speed and tracking analysis, are presented and discussed. Some
general axiomatic properties of the adaptive filters are introduced. Moreover, the
methodologyofstochasticdifferenceequations,asageneralmethodforevaluating
the performance of online adaptation algorithms, is introduced. Finally, variants
of the LMS algorithm, some multichannel algorithms applications, and delayed
learning algorithms, such as the class filtered-x LMS in its various forms, the
method filtered error LMS, and the method of the adjoint network, are presented
anddiscussed.
In Chap. 6, the most important second-order algorithms for the solution of
LSequationswithrecursivemethodsareintroduced.Inthefirstpartofthechapter,
the Newton’s method and its version with time-average correlation estimation,
definingthe classofadaptive algorithms such as sequential regression, are briefly
exposed. Subsequently, in the context of the second-order algorithms, a variant
of the NLMS algorithm, called affine projection algorithm (APA), is presented.
Thereafter the family of algorithms called recursive least squares (RLS) is
presented, and their convergence characteristics are studied. In the following,
some RLS variants and generalizations as the Kalman filter are presented.
Moreover,somecriteriatostudytheperformanceofadaptivealgorithmsoperating
in nonstationary environments are introduced. Finally, a more general adaptation
lawbasedonnaturalgradientapproach,consideringsparsityconstraints,isbriefly
introduced.
InChap.7,structuresandalgorithmsfortheimplementationofadaptivefiltersin
batch and online mode, operating in transformed domain (typically the frequency
domain),areintroduced.Inthefirstpartofthechapter,theblockLMSalgorithmis
introduced.Successively,twoparagraphsaboutthefrequencydomainconstrained
algorithmsknownasfrequencydomainadaptivefilters(FDAF),theunconstrained
FDAF, and the partitioned FDAF are introduced. In the third paragraph, the
transformeddomainadaptivealgorithms,referredtoastransform-domainadaptive
filters (TDAF), are presented. The chapter also introduces the multirate methods
andthesubbandadaptivefilters(SAFs).
In Chap. 8, the forward and backward linear prediction and the issue of the
order recursive algorithms are considered. Both of these topics are related to
implementative structures with particular robustness and efficiency properties. In
connection with this last aspect, the subject of the filter circuit structure and the
Preface ix
adaptation algorithm is introduced, in relation to the problems of noise control,
scalingandefficientcomputation,andeffectsduetocoefficientsquantization.
Chapter 9 introduces the problem space-time domain adaptive filtering, in
whichthesignalsareacquiredbyhomogeneoussensorarrays,arrangedindifferent
spatial positions. This issue, known in the literature as array processing (AP), is
of fundamental interest in many application fields. In particular, the basic
conceptsofdiscretespace-timefilteringareintroduced.Thefirstpartofthechapter
introduces the basics of the anechoic and echoic wave propagation model, the
sensors directivity functions, the signal model, and steering vectors of some
typical array geometries. The characteristics of noise field in various application
contextsandthearray qualityindices arealsodiscussed.Inthesecondpartofthe
chapter, methods for conventional beamforming are introduced, and the radiation
characteristicsarediscussed,themaindesigncriteriainrelationtotheoptimization
of quality indices. Moreover, the broadband beamformer with spectral decompo-
sition and the methods of direct synthesis of the spatial response are introduced
and discussed. In the third part of the chapter, the statistically optimal static
beamformingisintroduced.TheLSmethodologyisextendedinordertominimize
theinterferencerelatedtothenoisefield.Inaddition,thesuper-directivemethods,
the related regularized solution techniques, and the post-filtering method are
discussed.Theminimumvariancebroadbandmethod(theFrostalgorithm)isalso
presented.Inthefourthpart,theadaptivemodeforthedeterminationoftheonline
beamformingoperatingnonstationarysignalconditionispresented.Inthefinalpart
ofthechapter,theissueofthetime-delayestimation(TDE)anddirectionofarrival
(DOA) estimation in the case of free-field narrow-band signals and in the case of
broadbandsignalsinreverberantenvironmentispresented.
In addition, in order to have a possible self-contained text there are three
appendices,withacommonformalismtoallthearguments,thatrecalltothereader
somebasicnecessaryprerequisitesforaproperunderstandingofthetopicscovered
inthisbook.
In Appendix A, some basic concepts and quick reference of linear algebra
arerecalled.
In Appendix B, the basic concepts of the nonlinear programming are briefly
introduced. In particular, some fundamental concepts of the unconstrained and
theconstrainedoptimizationmethodsarepresented.
Finally, in Appendix C some basic concepts on random variables, stochastic
processes,andestimationtheoryarerecalled.
Foreditorialchoicefurtherstudyandinsights,exercises,projectproposals,the
studyofrealapplication,andalibrarycontainingMATLAB(®registeredtrademark
ofTheMathWorks,Inc.)codesforthecalculationofmainalgorithmsdiscussedin
thethistextareinsertedintoasecondvolumewhichiscurrentlybeingwritten.
Additionalmaterialstothetextcanbefoundat:http://www.uncini.com/FASP
Rome,Italy AurelioUncini