Table Of ContentSecond Generation Wavelets
and Applications
Maarten Jansen and Patrick Oonincx
Second Generation
Wavelets and
Applications
With 58 Figures
MaartenJansen,MScEngineering,PhDComputerScience
DepartmentofMathematicsandComputerScience
TUEindhoven,POBox513
5600MBEindhoven,TheNetherlands
and
DepartmentofComputerScience
K.U.Leuven,Celestijnenlaan200A
3001Leuven,Belgium
PatrickOonincx,MScMathematics,PhDMathematics
RoyalNetherlandsNavalCollege
POBox10000
1780CADenHelder,TheNetherlands
BritishLibraryCataloguinginPublicationData
Jansen,Maarten
Secondgenerationwaveletsandapplications
1. Signalprocessing—Mathematics 2. Imageprocessing—
Mathematics 3. Wavelets(Mathematics)
I. Title II. Oonincx,Patrick
515.2′433
ISBN1852339160
LibraryofCongressCataloging-in-PublicationData
Jansen,Maarten.
Secondgenerationwaveletsandapplications/MaartenJansenandPatrickOonincx.
p.cm.
Includesbibliographicalreferencesandindex.
ISBN1-85233-916-0(alk.paper)
1. Signalprocessing—Mathematics. 2. Imageprocessing—Mathematics. 3. Wavelets
(Mathematics) I. Oonincx,Patrick. II. Title.
TK5102.9.J3623 2005
515′.2433—dc22 2004058967
Apartfromanyfairdealingforthepurposesofresearchorprivatestudy,orcriticismorreview,
aspermittedundertheCopyright,DesignsandPatentsAct1988,thispublicationmayonlybe
reproduced,storedortransmitted,inanyformorbyanymeans,withthepriorpermissionin
writing of the publishers, or in the case of reprographic reproduction in accordance with the
termsoflicencesissuedbytheCopyrightLicensingAgency.Enquiriesconcerningreproduction
outsidethosetermsshouldbesenttothepublishers.
ISBN1-85233-916-0
SpringerScience+BusinessMedia
springeronline.com
©Springer-VerlagLondonLimited2005
The use of registered names, trademarks, etc. in this publication does not imply, even in the
absenceofaspecificstatement,thatsuchnamesareexemptfromtherelevantlawsandregula-
tionsandthereforefreeforgeneraluse.
Thepublishermakesnorepresentation,expressorimplied,withregardtotheaccuracyofthe
informationcontainedinthisbookandcannotacceptanylegalresponsibilityorliabilityforany
errorsoromissionsthatmaybemade.
Typesetting:Electronictextfilespreparedbyauthors
PrintedintheUnitedStatesofAmerica
69/3830-543210 Printedonacid-freepaper SPIN10974018
Preface
Theongoingsuccessofwavelettheoryandsimilarmultiscaledatadecompositions
is largely due to its flexibility in continuously expandingits horizon towards new
applications. The first success of wavelets came in the 1980swith the continuous
wavelettransformasamethodinthetime-scaleanalysisofsignals.Whilethisre-
searchwenton(andstillgoeson),itwassoonrealizedthattheconceptoftime-scale
analysisiscloselyrelatedtothesortofsignalprocessingthathappenedinso-called
Laplacianpyramids.Theformaldefinitionofmultiresolution,thecorrespondingfast
wavelettransformalgorithm(Mallat)andthediscoveryofcompactlysupportedor-
thogonal wavelets (Daubechies) were major breakthroughsin the developmentof
thediscretewavelettransformasatoolinsignalandimageprocessing.Inspiteof
this immediate success, the classical discrete wavelet transform is somehow lim-
ited.Atypicalexampleofsuchalimitationistheassumptionthattheinputhasto
be a regularly observed (sampled) signal, where the number of observations is a
power of two. Obviously,these limitations can be dealt with by some pre- and/or
post-processingofthe data:thinkaboutinterpolationforirregularityandsymmet-
ric extension if the number of data is not a power of two. Nevertheless, it seems
moreinterestingifwecanincorporatethegridstructureandtheintervalboundaries
intotheactualconstructionofthemultiresolutionanalysis.Thisisexactlywhatwas
aimedby‘second-generationwavelets’whentheywerefirstproposed(Sweldens),
about10yearsago.
Theconstructionofsecond-generationwaveletsisbasedontheliftingscheme.
In the first instance, this scheme is an alternative implementation of the fast dis-
cretewavelettransform.Itisabitfasterthantheclassicalrepeatedfilterbankimple-
mentationanditallowsin-placecomputations,i.e.,noadditionalworkingmemory
is needed, as all steps in the algorithm may overwrite input data without making
thetransformirreversible.Itwassoonrealizedthatthisliftingschemehasanother,
importantadvantage:unliketheclassicalfilterbankimplementation,theconceptis
readily extensible to differentlystructured data. This includesnot only the above-
mentioned irregularly observed data and data on an interval, but nearly any type
ofstructure.Onecouldnowthink,forinstance,aboutamultiresolutionanalysisof
DNA-molecule structures, large networks, surfaces in computer graphics applica-
tionsandso on.Evenforcommonimages,it paysoffusingtheliftingscheme,as
wasillustratedbythenewcompressionstandardinJPEG-2000,whichmakesuseof
lifting.Liftingalsoallowsonetoaddamultiscaleelementtopreviouslydeveloped
vi Preface
methods.Instatistics,forinstance,liftingcanbeusedincombinationwithexisting
smoothingmethods,suchassplinesmoothing.
Another extension,equally easy to constructwith the lifting scheme, are non-
linear and data-adaptive multiscale analyses. A well known example of a nonlin-
ear multiresolution decomposition is the integer wavelet transform that maps an
array of integersto another array of integers. Data-adaptivemultiresolutiontrans-
formsareextensivelydiscussedfurtherinthisbook.Intwodimensions,a specific
data-adaptive,anisotropicapproach,discussedinthelastchapter,showsinteresting
behaviourintheneighbourhoodofedgesinimages.Whiletheconceptofthisedge-
adaptive multiscale decomposition is quite easy to understand, the resulting con-
vergenceratecomparestothatofmorecomplicatedapproaches,suchascurvelets,
contourlets,edgeletsandsoon.
Wavelettheoryandapplicationsaresituatedontheedgesbetweenmanyfields:
signalandimageprocessing,numericalanalysisandscientificcomputing,statistics,
approximationtheoryandothermathematicalfields.Eachofthesedomainshasits
ownterminologyandrequiresa specificwritingstyle.Whilepeopleinsignalpro-
cessingareusedtoillustrationsanddiagrams,mathematicianspreferclearformulas
tothinkandreasonabout.Samplesinsignalprocessingcorrespondtoobservations
instatistics,whileasampleinstatisticalliteratureismostlytheentiresetofobser-
vations.Signalsinsignalprocessingarefunctionsinmathematics.Wehavetriedto
offer both figures and formulas, fully aware that some readers might spend some
time on puzzling out the meaning of each symbol in an expression or a diagram.
Yet,webelievethatdoingsoisakeytoopeningthedoorsofmanypapersthatoth-
erwiseremainunread.Also,wetriedtoprovidesufficientvariation,inthesensethat
amathematicalapproachisillustratedwithfiguresandplotsinasubsequentsection
andviceversa.
Asforthemathematicaldepthofourapproach,ourpointofviewistoprovidea
guidefortheexplorationoftheliterature,ratherthanafullyelaboratedmathematical
treatmentofallaspectsofsecond-generationwavelets.We thereforeomitsomeof
thefull,rigourousproofsiftheywouldtaketoomuchspaceandattention.Wehope
tocomposeacleargeneraloverviewonthetopicofsecond-generationwavelets.
Writing and even thinking this book would be a much harder job without the
presenceofourfamilies.
I, Maarten Jansen, would like to express my gratitute to my wife, Gerda
Claeskens, of course for her support and patience, but also for her keen curiosity
andinterestinmywork.Herremarksaftercarefulproofreadingmychaptersmade
asubstantialimprovementinreadability.Assheiswritingabookherselfnow,Iam
surethat,ifsheshowsthesameenthusiasmthereaswell,thatbookisgoingtobe
anotherpieceofherworkshe(andI!)canbeproudof.
I, Patrick Oonincx, would like to thank my loving fiancee Birgit Bobeck for
runningourfamilyandhavingpatiencewhenIwasspendingweekendsandholidays
strugglingwith figuresandformulas.Specialthanksshouldgoto her,becausefor
thesecondtimeinourlivessheiscarryingababywhileI’mwritingabookand,as
inthepreviouscase,bothwillseedaylightaroundthesamedate.Lastbutnotleast
Preface vii
I havetomakemyapologiestomydaughterSachaforbeingalessplayfuldadin
thelastyear.Afterthislastsentencemanyformulaswillbereplacedbyplaytime.
Contents
1. The Classical Wavelet Transform for Continuous-time and Discrete-
timeSignals ................................................ 1
1.1 TheContinuousWaveletTransform ........................... 1
1.1.1 TheHaarwavelet .................................... 3
1.1.2 TheMexicanHatWavelet............................. 5
1.2 TheDiscreteWaveletTransform.............................. 7
1.2.1 DiscretizingtheContinuousWaveletTransform........... 7
1.2.2 MultiresolutionAnalysis.............................. 8
1.2.3 TheTwo-scaleEquation .............................. 9
1.2.4 Duality............................................. 12
1.2.5 VanishingMoments .................................. 13
1.3 MultiresolutionAnalysisandFilterbanks....................... 14
1.3.1 TheDecompositionAlgorithm......................... 14
1.3.2 TheReconstructionAlgorithm ......................... 16
1.3.3 ImplementationoftheDiscreteWaveletTransformforDiscrete-
timeSignals......................................... 17
2. Second-generationWavelets................................... 21
2.1 WaveletsonIrregularPointSets .............................. 21
2.2 TheLiftingScheme......................................... 24
2.2.1 TheHaarDecompositionbyLifting..................... 24
2.2.2 TheLiftingScheme:Split,PredictandUpdate............ 26
2.2.3 BasisFunctions...................................... 29
2.2.4 Subdivision ......................................... 33
2.2.5 LiftingExistingWaveletTransforms .................... 35
2.2.6 LiftingandPolyphase ................................ 38
2.3 TheConstructionofSecond-generationWavelets................ 40
2.3.1 MultiscaleGrids..................................... 41
2.3.2 TheUnbalancedHaarTransform ....................... 41
2.3.3 PredictionMethods .................................. 45
2.3.4 UpdatesforVanishingMoments........................ 52
2.4 LiftinginTwo(andMore)Dimensions ........................ 53
2.4.1 DefinitionsandConstructionofTriangulations ........... 54
2.4.2 DelaunayTriangulation............................... 54
x Contents
2.4.3 MultiscaleTriangulations ............................. 55
2.4.4 MultiscaleDelaunayTriangulations..................... 57
3. NonlinearandAdaptiveLifting................................ 61
3.1 NonlinearFilters ........................................... 61
3.1.1 TheMax-liftingScheme .............................. 61
3.1.2 TheMedian-liftingScheme............................ 64
3.2 AdaptiveLifting ........................................... 66
3.3 ReconstructionofAdaptiveLifting............................ 70
3.3.1 AutomaticPerfectReconstruction ...................... 70
4. NumericalCondition ........................................ 77
4.1 StabilityinWaveletSmoothingonIrregularPointSets ........... 78
4.2 ConditionfromFinitetoInfiniteDimensions ................... 81
4.2.1 ConditionNumbers .................................. 81
4.2.2 StableBases ........................................ 82
4.3 NumericalConditionofWaveletTransforms.................... 86
4.3.1 GeneralStabilityCriteria.............................. 86
4.3.2 Smoothness,Convergence(ofApproximation)andStability 89
4.4 NumericalConditionofLiftedWaveletTransforms .............. 91
4.4.1 LiftingofExistingStableSchemes ..................... 91
4.4.2 NumericalConditionandPrimalVanishingMoments...... 92
4.4.3 StabilizingUpdates .................................. 94
4.4.4 Prediction .......................................... 95
4.4.5 SplittingStrategies................................... 99
5. ApplicationsofNonlinearLiftinginImaging .................... 103
5.1 ImageRetrievalUsingAdaptiveLifting........................103
5.1.1 QuincunxLifting ....................................103
5.1.2 AdaptiveLifting .....................................106
5.1.3 RedundantLifting ...................................108
5.1.4 FeatureVectors......................................109
5.1.5 IllustrationofImageRetrieval .........................111
5.2 AdaptiveSplittingusingNormalOffsets .......................113
5.2.1 2D > (1D)2 ......................................113
5.2.2 SomeTheoreticalBackground .........................118
5.2.3 NormalOffsets ......................................120
References ..................................................... 129
Index.......................................................... 135
1. The Classical Wavelet Transform for
Continuous-time and Discrete-time Signals
Since its first appearance about 20 years ago the wavelet transform and its repre-
senations by means of filterbanks has been discussed in many books and papers,
amongst which are some standard works [16, 22, 51, 58, 59, 73]. Although most
readersofthisbookwillbefamiliarwithatleastoneofthesepublications,wewill
start with a wrap up of the wavelet transformas it has been used for the past two
decades.
1.1 The Continuous WaveletTransform
The continuous wavelet transform (CWT) transform was introduced in 1984 by
Morletandco-workerstoanalysegeophysicalsignalswithsomekindofmodified
windowedFouriertransform(WFT),whichreads
(cid:1)
1
F [s](t,ω)= √ s(y)h(y−t)e−iωydy, (1.1)
h
2π
IR
fors ∈ L2(IR)atime-continuoussignalandh ∈ L2(IR)a windowfunction.Here
L2(IR)denotesthespaceofsquareintegrablefunctionsonIR.Themodificationof
theWFTwasestablishedbycombiningwindowfunctionandFouriermodee−iωy
intoonewindowfunctionψthatcanbescaled.ThismodificationbecametheCWT
givenby
(cid:1) (cid:2) (cid:3)
1 t−b
W [s](a,b)= √ s(t)ψ dt, (1.2)
ψ a a
IR
for some wavelet function ψ ∈ L2(IR), scaling parameter a > 0 and translation
parameterb∈IR.Although,thisseemedtobeanewtransform,inthemathematical
societythistransformwasalreadyknowasCalde´ron’sreproducingformula[5].
TheCWTcanbewritteninadifferentwaybyintroducingthetranslationoper-
atorTb anddilationoperatorDaonL2(IR)by
(cid:2) (cid:3)
1 t
T [s](t)=s(t−b) and D [s](t)= √ s , (1.3)
b a a a
forsomea∈IR+,b∈IR.Usingthesetransformswecanalsowrite(1.2)as
2 1. TheClassicalWaveletTransformforContinuous-timeandDiscrete-timeSignals
W [s](a,b)=(cid:3)s,T D ψ(cid:4), (1.4)
ψ b a
andbyusingParseval’sidentityfortheFouriertransformF alsoas
(cid:1)
√
W [s](a,b)=(cid:3)Fs,FT D ψ(cid:4)= a sˆ(ω)ψ(cid:4)(aω)eibωdω, (1.5)
ψ b a
IR
withsˆtheFouriertransformofthesignals.
¿From(1.4)twopropertiesoftheCWTcanbeobtainedinaratherstraightfor-
wardway,namelythattheCWT isbothboundedandcontinuousinitsparameters
(a,b).ToshowthecontinuityoftheCWTwefirstassumeψisafunctionofcompact
support.Inthiscaseψisuniformlycontinuous,meaningthat
(cid:5)TbDaψ−Tb(cid:1)Da(cid:1)ψ(cid:5)2 → 0, (a,b)→(a(cid:1),b(cid:1)), (1.6)
with (cid:5)·(cid:5)2 the L2-norm. Since the space of all compactly supported functions in
L2(IR) is dense in L2(IR), this result is inherited by all ψ ∈ L2(IR), yielding the
continuityofW [s]initsparameters(a,b)following(1.4).Furthermore,alsofrom
ψ
thisdefinitionweobtainbySchwarz’sinequality
|Wψ[f](a,b)|≤(cid:5)f(cid:5)2·(cid:5)ψ(cid:5)2 ∀a∈IR+∀b∈IR,
i.e.,theCWTisboundedforall(a,b).
Untilnowwe havenotdiscussedwhichfunctionscanbe usedasananalyzing
wavelet functionψ. One can thinkof functionsthatare compactlysupported,like
arealwindowthatslidesalongthesignalss.Mathematicallyspeaking,adesirable
propertyofawaveletisthataninversionformulaexists,i.e.,scanberecoveredfrom
W [s].ForderivinganinversionformulawestartwithakindofParseval’sidentity
ψ
fortheCWT.Weintegrate|W [s](a,b)|2overIH=IR+×IRusingtheHaarmeasure
ψ
dbda/a2.Furthermore,wewritetheCWTasaconvolution(usingthestandardu∗v
todenotetheconvolutionoftwofunctionsuandv)W [s](a,b)=s∗D ψˇ(b),with
ψ a
ψˇ(t) = ψ(−t), which follows straightforward, from the definition of the CWT.
ApplyingParseval’sidentityonFouriertransforms,following[52],wearriveat
(cid:1) (cid:1)
(cid:5) (cid:6)
da da
|W [s](a,b)|2db = |s∗D ψˇ(b)|2db
ψ a2 a a2
IH IH (cid:1)
= 2π |sˆ(ω)|2|ψ(cid:4)(aω)|2dω da
a
IH ⎛ ⎞
(cid:1) (cid:1)
= |sˆ(ω)|2⎝2π |ψ(cid:4)(aω)|2 da⎠ dω.
a
IR IR+
Bydefining
(cid:1)
|ψ(cid:4)(aω)|2
C (ω)=2π da (1.7)
ψ a
IR+
