Digital signal compression : principles and practice

439 Pages·2011·7.162 MB·English

by  William A Pearlman, Amir Said

DigitalSignalCompression
Withclearandeasy-to-understandexplanations,thisbookcoversthefundamentalcon-
cepts and coding methods of signal compression, while still retaining technical depth
andrigor.Itcontainsawealthofillustrations,step-by-stepdescriptionsofalgorithms,
examples, and practice problems, which make it an ideal textbook for senior under-
graduate and graduate students, as well as a useful self-study tool for researchers and
professionals.
Principles of lossless compression are covered, as are various entropy coding tech-
niques,includingHuffmancoding,arithmeticcoding,run-lengthcoding,andLempel–
Zivcoding.Scalarandvectorquantization,andtheiruseinvariouslossycompression
systems,arethoroughlyexplained,andafullchapterisdevotedtomathematicaltrans-
formations,includingtheKarhunen–Loevetransform,discretecosinetransform(DCT),
and wavelet transforms. Rate control is covered in detail, with several supporting
algorithms to show how to achieve it. State-of-the-art transform and subband/wavelet
image and video coding systems are explained, including JPEG2000, SPIHT, SBHP,
EZBC, and H.264/AVC. Also,unique tothis book is achapter on setpartition coding
thatshedsnewlightonSPIHT,SPECK,EZW,andrelatedmethods.
WilliamA.Pearlman is a Professor Emeritus in the Electrical, Computer, and Systems
Engineering Department at the Rensselear Polytechnic Institute (RPI), where he has
beenafacultymembersince1979.Hehasmorethan35yearsofexperienceinteaching
and researching in the fields of information theory, data compression, digital signal
processing,anddigitalcommunicationstheory,andhehaswrittenabout250published
worksinthesefields.HeisaFellowoftheIEEEandtheSPIE,andheistheco-inventor
oftwocelebratedimagecompressionalgorithms:SPIHTandSPECK.
AmirSaidiscurrentlyaMasterResearcheratHewlett-PackardLaboratories,wherehe
has worked since 1998. His research interests include multimedia communications,
coding and information theory, image and video compression, signal processing, and
optimization, and he has more than 100 publications and 20 patents in these fields.
He is co-inventor with Dr. Pearlman of the SPIHT image compression algorithm and
co-recipient, also with Dr. Pearlman, of two Best Paper Awards, one from the IEEE
CircuitsandSystemsSocietyandtheotherfromtheIEEESignalProcessingSociety.

Digital Signal Compression
Principles and Practice
WILLIAM A. PEARLMAN
RensselearPolytechnicInstitute,NewYork
AMIR SAID
Hewlett-PackardLaboratories,PaloAlto,California
CAMBRIDGE UNIVERSITY PRESS
Cambridge,NewYork,Melbourne,Madrid,CapeTown,
Singapore,SãoPaulo,Delhi,Tokyo,MexicoCity
CambridgeUniversityPress
TheEdinburghBuilding,CambridgeCB28RU,UK
PublishedintheUnitedStatesofAmericabyCambridgeUniversityPress,NewYork
www.cambridge.org
Informationonthistitle:www.cambridge.org/9780521899826
(cid:2)c CambridgeUniversityPress2011
Thispublicationisincopyright.Subjecttostatutoryexception
andtotheprovisionsofrelevantcollectivelicensingagreements,
noreproductionofanypartmaytakeplacewithoutthewritten
permissionofCambridgeUniversityPress.
Firstpublished2011
PrintedintheUnitedKingdomattheUniversityPress,Cambridge
AcatalogrecordforthispublicationisavailablefromtheBritishLibrary
LibraryofCongressCataloginginPublicationdata
Pearlman,WilliamA.(WilliamAbraham)
Digitalsignalcompression:principlesandpractice/WilliamA.Pearlman,AmirSaid.
p. cm.
Includesbibliographicalreferencesandindex.
ISBN978-0-521-89982-6(hardback)
1. Datacompression(Telecommunication) 2. Signalprocessing–Digitaltechniques.
I. Said,Amir. II. Title.
TK5102.92.P43 2011
(cid:3)
005.746–dc23
2011012972
ISBN978-0-521-89982-6Hardback
Additionalresourcesforthispublicationatwww.cambridge.org/9780521899826
CambridgeUniversityPresshasnoresponsibilityforthepersistenceor
accuracyofURLsforexternalorthird-partyinternetwebsitesreferredto
inthispublication,anddoesnotguaranteethatanycontentonsuch
websitesis,orwillremain,accurateorappropriate.
ToEleanor
ToCelloandRicardo

Contents
Preface pagexv
Acknowledgments xix
1 Motivation 1
1.1 Theimportanceofcompression 1
1.2 Datatypes 2
1.2.1 Symbolicinformation 2
1.2.2 Numericalinformation 3
1.3 Basiccompressionprocess 4
1.4 Compressionapplications 5
1.5 Designofcompressionmethods 6
1.6 Multi-disciplinaryaspect 8
Note 8
References 9
2 Bookoverview 10
2.1 Entropyandlosslesscoding 10
2.2 Quantization 11
2.3 Sourcetransformations 12
2.3.1 Prediction 12
2.3.2 Transforms 13
2.4 Setpartitioncoding 16
2.5 Codingsystems 17
2.5.1 Performancecriteria 17
2.5.2 Transformcodingsystems 18
2.5.3 Subbandcodingsystems 19
2.6 Distributedsourcecoding 20
Notes 21
References 22
3 Principlesoflosslesscompression 23
3.1 Introduction 23
3.2 Losslesssourcecodingandentropy 23
viii Contents
3.3 Variablelengthcodes 28
3.3.1 Uniquedecodabilityandprefix-freecodes 28
3.3.2 Constructionofprefix-freecodes 28
3.3.3 Kraftinequality 30
3.4 Optimalityofprefix-freecodes 32
3.4.1 Sourceswithmemory 36
3.5 Concludingremarks 37
Problems 37
References 40
4 Entropycodingtechniques 41
4.1 Introduction 41
4.2 Huffmancodes 41
4.3 Shannon–Fano–Eliascodes 47
4.3.1 SFEcodeexamples 48
4.3.2 DecodingtheSFEcode 49
4.4 Arithmeticcode 50
4.4.1 Preliminaries 50
4.4.2 Arithmeticencoding 51
4.4.3 Arithmeticdecoding 53
4.5 Run-lengthcodes 55
4.6 Alphabetpartitioning:modifiedHuffmancodes 57
4.6.1 ModifiedHuffmancodes 57
4.6.2 Alphabetpartitioning 58
4.7 Golombcode 60
4.8 Dictionarycoding 63
4.8.1 TheLZ78code 64
4.8.2 TheLZWalgorithm 65
4.8.3 TheLZ77codingmethod 67
4.9 Summaryremarks 72
Problems 72
Notes 75
References 76
5 Lossycompressionofscalarsources 77
5.1 Introduction 77
5.2 Quantization 77
5.2.1 Scalarquantization 77
5.2.2 Uniformquantization 81
5.3 Non-uniformquantization 87
5.3.1 Highrateapproximations 89
5.4 Companding 91
5.4.1 Distortionathighrates 93
Contents ix
5.5 Entropycodingofquantizeroutputs 95
5.5.1 Entropycodedquantizercharacteristics 98
5.5.2 Null-zonequantization 99
5.6 Boundsonoptimalperformance 101
5.6.1 Rate-distortiontheory 102
5.6.2 TheGish–Piercebound 104
5.7 Concludingremarks 107
5.8 Appendix:quantizationtables 107
Problems 109
Note 113
References 114
6 Codingofsourceswithmemory 116
6.1 Introduction 116
6.2 Predictivecoding 116
6.2.1 Optimallinearprediction 117
6.2.2 DPCMsystemdescription 120
6.2.3 DPCMcodingerrorandgain 121
6.3 Vectorcoding 122
6.3.1 Optimalperformancebounds 122
6.3.2 Vector(block)quantization(VQ) 129
6.3.3 Entropyconstrainedvectorquantization 135
6.4 Tree-structuredvectorquantization 141
6.4.1 VariablelengthTSVQcoding 144
6.4.2 PrunedTSVQ 145
6.5 Treeandtrelliscodes 146
6.5.1 Trelliscodes 148
6.5.2 Encodinganddecodingoftrelliscodes 150
6.5.3 Codevectoralphabets 152
6.6 Trelliscodedquantization(TCQ) 152
6.6.1 Entropy-codedTCQ 154
6.6.2 Improvinglow-rateperformanceinTCQ 155
6.7 Searchalgorithms 155
6.7.1 M-algorithm 155
6.7.2 TheViterbialgorithm 158
6.8 Concludingremarks 160
Problems 160
Notes 163
References 164
7 Mathematicaltransformations 166
7.1 Introduction 166
7.1.1 Transformcodinggain 169
x Contents
7.2 TheoptimalKarhunen–Loevetransform 171
7.2.1 Optimaltransformcodinggain 172
7.3 Suboptimaltransforms 172
7.3.1 ThediscreteFouriertransform 172
7.3.2 Thediscretecosinetransform 173
7.3.3 TheHadamard–Walshtransform 174
7.4 Lappedorthogonaltransform 175
7.4.1 Exampleofcalculationoftransformcodinggain 178
7.5 Transformsviafilterbanks 179
7.6 Two-dimensionaltransformsforimages 181
7.7 Subbandtransforms 184
7.7.1 Introduction 184
7.7.2 Codinggainofsubbandtransformation 187
7.7.3 Realizableperfectreconstructionfilters 192
7.7.4 Orthogonalwavelettransform 194
7.7.5 Biorthogonalwavelettransform 199
7.7.6 Usefulbiorthogonalfilters 204
7.7.7 Theliftingscheme 205
7.7.8 Transformswithintegeroutput 208
7.8 Concludingremarks 211
Problems 212
Notes 214
References 216
8 Ratecontrolintransformcodingsystems 218
8.1 Rateallocation 218
8.1.1 Optimalrateallocationforknownquantizercharacteristics 220
8.1.2 Realizingtheoptimalrateallocation 223
8.1.3 Fixedlevelquantization 225
8.1.4 Optimalbitallocationforarbitrarysetofquantizers 226
8.1.5 Buildinguptooptimalratesforarbitraryquantizers 228
8.1.6 Transformcodinggain 230
8.2 Subbandrateallocation 233
8.2.1 Practicalissues 237
8.2.2 Subbandcodinggain 239
8.3 Algorithmsforrateallocationtosubbands 241
8.4 Conclusions 242
Problems 242
Notes 243
References 244
9 Transformcodingsystems 245
9.1 Introduction 245