Quantization Noise: Roundoff Error in Digital Computation, Signal Processing, Control, and Communications

781 Pages·2008·9.41 MB·English

by  Bernard Widrow

This page intentionally left blank
Quantization Noise
Ifyouareworkingindigitalsignalprocessing,control,ornumericalanalysis,youwill
findthisauthoritativetreatmentofquantizationnoise(roundofferror)tobean
invaluableresource.
Doyouknowwherethetheoryofquantizationnoisecomesfrom,andunderwhat
circumstancesitistrue?Expertauthors,includingthefounderofthefieldand
formulatorofthetheoryofquantizationnoise,BernardWidrow,answertheseand
otherimportantpracticalquestions.Theydescribeandanalyzeuniformquantization,
floating-pointquantization,andtheirapplicationsindetail.
Keyfeaturesinclude:
– heuristicexplanationsalongwithrigorousproofs;
– workedexamples,sothattheoryisunderstoodthroughexamples;
– focusonpracticalcases,engineeringapproach;
– analysisoffloating-pointroundoff;
– dithertechniquesandimplementationissuesanalyzed;
(cid:1)
– programpackageforMatlab availableontheweb,forsimulationandanalysis
offixed-pointandfloating-pointroundoff;
– homeworkproblemsandsolutionsmanual;and
– activelymaintainedwebsitewithadditionaltextonspecialtopicsonquantization
noise.
Theadditionalresourcesareavailableonlinethrough
www.cambridge.org/9780521886710
Bernard Widrow,aninternationallyrecognizedauthorityinthefieldof
quantization,isaProfessorofElectricalEngineeringatStanfordUniversity,
California.Hepioneeredthefieldandoneofhispapersonthetopicisthestandard
reference.HeisaFellowoftheIEEEandtheAAAS,amemberoftheUSNational
AcademyofEngineering,andthewinnerofnumerousprestigiousawards.
István Kollár isaProfessorofElectricalEngineeringattheBudapestUniversity
ofTechnologyandEconomics.AFellowoftheIEEE,hehasbeenresearchingthe
theoryandpracticeofquantizationandroundoffforthelast30years.Heistheauthor
ofabout135scientificpublicationsandhasbeeninvolvedinseveralindustrial
developmentprojects.

Quantization Noise
Roundoff Error in Digital Computation, Signal Processing,
Control, and Communications
Bernard Widrow and
István Kollár
CAMBRIDGEUNIVERSITY PRESS
Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo
Cambridge University Press
The Edinburgh Building, Cambridge CB2 8RU, UK
Published in the United States of America by Cambridge University Press, New York
www.cambridge.org
Information on this title: www.cambridge.org/9780521886710
© Cambridge University Press 2008
This publication is in copyright. Subject to statutory exception and to the provision of
relevant collective licensing agreements, no reproduction of any part may take place
without the written permission of Cambridge University Press.
First published in print format 2008
ISBN-13 978-0-511-40990-5 eBook (NetLibrary)
ISBN-13 978-0-521-88671-0 hardback
Cambridge University Press has no responsibility for the persistence or accuracy of urls
for external or third-party internet websites referred to in this publication, and does not
guarantee that any content on such websites is, or will remain, accurate or appropriate.
We dedicate this work to our fathers and our
teachers. Theyinfluencedourlivesandourthink-
inginaverypositiveway.
I would like to dedicate this book to Professors
David Middleton and John G. Linvill, and to the
memory of Professor William K. Linvill and my
father,MosesWidrow.
BernardWidrow
I would like to dedicate this book to Professors
Andra´s Pre´kopa and Pe´ter Osva´th, and to the
memory of Professor La´szlo´ Schnell and my fa-
ther,LajosKolla´r.
Istva´nKolla´r

Contents
Preface XIX
Acknowledgments XXI
GlossaryofSymbols XXIII
AcronymsandAbbreviations XXVII
Part I Background
1 Introduction 3
1.1 DefinitionoftheQuantizer 3
1.2 SamplingandQuantization(Analog-to-DigitalConversion) 9
1.3 Exercises 10
2 SamplingTheory 13
2.1 Linvill’sFrequencyDomainDescriptionofSampling 14
2.2 The Sampling Theorem; Recovery of the Time Function from its
Samples 18
2.3 Anti-AliasFiltering 22
2.4 AStatisticalDescriptionofQuantization,BasedonSampling
Theory 25
2.5 Exercises 28
3 ProbabilityDensityFunctions,CharacteristicFunctions,Moments 31
3.1 ProbabilityDensityFunction 31
3.2 CharacteristicFunctionandMoments 33
3.3 JointProbabilityDensityFunctions 35
3.4 JointCharacteristicFunctions,Moments,andCorrelation
Functions 40
3.5 First-Order Statistical Description of the Effects of Memoryless
OperationsonSignals 43
VII
VIII Contents
3.6 Addition of Random Variables and Other Functions of Random
Variables 46
3.7 TheBinomialProbabilityDensityFunction 47
3.8 TheCentralLimitTheorem 49
3.9 Exercises 53
Part II Uniform Quantization
4 StatisticalAnalysisoftheQuantizerOutput 61
4.1 PDFandCFoftheQuantizerOutput 61
4.2 Comparison of Quantization with the Addition of Independent
UniformlyDistributedNoise,thePQNModel 66
4.3 QuantizingTheoremsIandII 69
4.4 RecoveryofthePDFoftheInputVariable x fromthePDFofthe
(cid:1)
OutputVariable x 70
4.5 Recovery of Moments of the Input Variable x from Moments of
(cid:1)
theOutputVariable x whenQTIIisSatisfied;Sheppard’s
CorrectionsandthePQNModel 80
4.6 GeneralExpressionsoftheMomentsoftheQuantizerOutput,and
oftheErrorsofSheppard’sCorrections: DeviationsfromthePQN
Model 84
4.7 Sheppard’sCorrectionswithaGaussianInput 84
4.8 Summary 85
4.9 Exercises 87
5 StatisticalAnalysisoftheQuantizationNoise 93
5.1 AnalysisoftheQuantizationNoiseandthePQNModel 93
5.2 SatisfactionofQuantizingTheoremsIandII 99
5.3 QuantizingTheoremIII/A 99
5.4 General Expressions of the First- and Higher-Order Moments of
theQuantizationNoise: DeviationsfromthePQNModel 102
5.5 QuantizationNoisewithGaussianInputs 106
5.6 Summary 107
5.7 Exercises 108
6 CrosscorrelationsbetweenQuantizationNoise,QuantizerInput,
andQuantizerOutput 113
6.1 CrosscorrelationswhenQuantizingTheoremIIisSatisfied 113
6.1.1 CrosscorrelationbetweenQuantizationNoiseandthe
QuantizerInput 113
6.1.2 CrosscorrelationbetweenQuantizationNoiseandthe
QuantizerOutput 115