Handbook of Fourier Analysis & Its Applications

812 Pages·2006·12.19 MB·English

by  Marks II R.J.

viii
The Joy of Fourier:
Analysis, Sampling Theory, Systems, Multidimensions, Stochastic
Processes, Random Variables, Signal Recovery, POCS, Time
Scales, & Applications
by Robert J. Marks II
Baylor University
(cid:176)c2006 by R.J. Marks II
ix
Acknowledgement
“There is one God, the Father, ever-living, omnipresent, omniscient, almighty, the maker
of heaven and earth, and one Mediator between God and man, the man Jesus Christ...”
Isaac Newton [328].
‘There is a God-shaped vacuum in the heart of every man, which only God can fill through
His Son Jesus Christ.” Blaise Pascal [909, 1154].
“Christ [is] the power of God and the wisdom of God. The Christian religion is a
revelation and that revelation is the Word of God.” Michael Faraday [854].
“For the invisible things of Him from the creation of the world are clearly seen, being
understood by the things that are made, even His eternal power and Godhead; so that
they are without excuse.” Romans 1:20.
Dedicated to the Christ Jesus who makes things clear.
Preface
The emphasis of this text is the broad field of Fourier analysis and some of its immediate
outgrowths including convolution, systems theory, Shannon sampling theory and the mod-
elling of random variables and stochastic processes. The contents of this book evolved from
a set of lecture notes for courses in advanced engineering mathematics (Chapters 2, 3, 4, 12
and 13), multidimensional signal processing (Chapters 2, 9 and 11), signal reconstruction
(Chapters 2, 10 and 11) and Shannon sampling and interpolation theory (Chapters 2, 5, 7,
6 and 7). Chapter 4 is useful as a supplement a graduate course on random variables and
stochastic processes. The book is written at the level of the advanced senior or first year
graduate student who has an introductory foundation in Fourier analysis and stochastic
processes. Rudimentary familiarity with image processing is helpful but not necessary.
The book is written as a text, but can also serve as a handbook or source of material for
self study. In order to present material from an alternate point of view and to facilitate self
study,solutionstoselectedproblemsareincludedattheendofeachchapter.Ingeneral,the
problems are not designed for linear extension of concepts presented in the text. They are,
rather, often used to present new material or to establish concepts from a new viewpoint.
An extensive bibliography is also included.
Theauthoristakengreatpainstokeepthebookerrorfree,butiscertainthatthegoalhas
not been achieved. All errors herein are the responsibility of the author. If you find errors
in the text, please e-mail the specifics to [email protected]. Corrections will be
posted on TheJoyOfFourier.com. Additional problems and solutions will also be posted.
Contents
1 Introduction 2
1.1 Ubiquitous Fourier Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Jean Baptiste Joseph Fourier . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3.1 Surveying the Contents . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 Fundamentals of Fourier Analysis 10
2.1 Signal Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2 The Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2.1 The Continuous Time Fourier Transform . . . . . . . . . . . . . . . . 14
2.2.2 The Fourier Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2.2.1 Parseval’s theorem for the Fourier Series . . . . . . . . . . . 17
2.2.2.2 Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.2.3 Relationship Between Fourier and Laplace Transforms . . . . . . . . . 18
2.2.4 Some Continuous Time Fourier Transform Theorems . . . . . . . . . . 19
2.2.4.1 The Derivative Theorem. . . . . . . . . . . . . . . . . . . . . 19
2.2.4.2 The Convolution Theorem. . . . . . . . . . . . . . . . . . . . 19
2.2.4.3 The Inversion Theorem. . . . . . . . . . . . . . . . . . . . . . 20
2.2.4.4 The Duality Theorem. . . . . . . . . . . . . . . . . . . . . . . 20
2.2.4.5 The Scaling Theorem. . . . . . . . . . . . . . . . . . . . . . . 21
2.2.5 Some Continuous Time Fourier Transform Pairs . . . . . . . . . . . . 21
2.2.5.1 The Dirac Delta Function . . . . . . . . . . . . . . . . . . . . 21
2.2.5.2 Trigonometric Functions . . . . . . . . . . . . . . . . . . . . 25
2.2.5.3 The Sinc and Related Functions . . . . . . . . . . . . . . . . 28
2.2.5.4 The Rectangle Function . . . . . . . . . . . . . . . . . . . . . 29
2.2.5.5 The Signum Function . . . . . . . . . . . . . . . . . . . . . . 30
2.2.5.6 The Gaussian . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.2.5.7 The Comb Function . . . . . . . . . . . . . . . . . . . . . . . 33
2.2.5.8 The Triangle Function . . . . . . . . . . . . . . . . . . . . . . 34
2.2.5.9 The Array Function . . . . . . . . . . . . . . . . . . . . . . . 34
2.2.5.10 The Gamma Function . . . . . . . . . . . . . . . . . . . . . . 38
xii
2.2.5.11 The Beta Function . . . . . . . . . . . . . . . . . . . . . . . . 39
2.2.5.12 Bessel Functions . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.2.5.13 The Sine Integral . . . . . . . . . . . . . . . . . . . . . . . . 42
2.2.5.14 Orthogonal Polynomials . . . . . . . . . . . . . . . . . . . . . 42
2.2.6 Other Properties of the Continuous Time Fourier transform . . . . . . 47
2.2.6.1 The Signal Integral Property . . . . . . . . . . . . . . . . . . 47
2.2.6.2 Conjugate Symmetry of a Real Signal’s Spectrum. . . . . . . 48
2.2.6.3 Hilbert Transform Relationships Within a Causal Signals’
Spectrum. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.2.6.4 The Power Theorem. . . . . . . . . . . . . . . . . . . . . . . 50
2.2.6.5 The Poisson Sum Formula. . . . . . . . . . . . . . . . . . . . 50
2.3 Orthogonal Basis Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
2.3.1 Parseval’s theorem for an Orthonormal Basis . . . . . . . . . . . . . . 51
2.3.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
2.3.2.1 The Cardinal Series . . . . . . . . . . . . . . . . . . . . . . . 52
2.3.2.2 The Fourier Series . . . . . . . . . . . . . . . . . . . . . . . . 52
2.3.2.3 Prolate Spheroidal Wave Functions . . . . . . . . . . . . . . 53
2.3.2.4 Complex Walsh Functions. . . . . . . . . . . . . . . . . . . . 53
2.3.2.5 Orthogonal Polynomials . . . . . . . . . . . . . . . . . . . . . 53
2.3.2.6 FourierTransformsofOrthonormalFunctionsareOrthonormal 53
2.4 The Discrete Time Fourier Transform . . . . . . . . . . . . . . . . . . . . . . 53
2.4.1 Relation to the Continuous Time Fourier Transform . . . . . . . . . . 54
2.5 The Discrete Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . . 56
2.5.1 Relation to the Continuous Time Fourier Transform . . . . . . . . . . 57
2.5.2 DFT Leakage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
2.6 Related Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
2.6.1 The Cosine Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
2.6.2 The Sine Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
2.6.3 The Hartley Transform . . . . . . . . . . . . . . . . . . . . . . . . . . 61
2.6.4 The z Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
2.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
2.8 Solutions for Chapter 2 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . 77
3 Fourier Analysis in Systems Theory 110
3.1 System Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
3.1.1 System Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
3.1.1.1 Homogeneous Systems. . . . . . . . . . . . . . . . . . . . . . 110
3.1.1.2 Additive Systems . . . . . . . . . . . . . . . . . . . . . . . . 110
3.1.1.3 Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . 110
3.1.1.4 Time-Invariant Systems . . . . . . . . . . . . . . . . . . . . . 111
3.1.1.5 LTI Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
3.1.1.6 Causal Systems . . . . . . . . . . . . . . . . . . . . . . . . . 112
3.1.1.7 Memoryless Systems . . . . . . . . . . . . . . . . . . . . . . . 113
xiii
3.1.1.8 Stable Systems . . . . . . . . . . . . . . . . . . . . . . . . . . 114
3.1.1.9 Invertible Systems . . . . . . . . . . . . . . . . . . . . . . . . 114
3.1.2 Example Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
3.1.2.1 The Magnifier . . . . . . . . . . . . . . . . . . . . . . . . . . 114
3.1.2.2 The Fourier Transformer . . . . . . . . . . . . . . . . . . . . 115
3.1.2.3 Convolution . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
3.2 System Characterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
3.3 Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
3.4 Causal Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
3.5 Linear Time Invariant (LTI) Systems . . . . . . . . . . . . . . . . . . . . . . . 121
3.5.1 Convolution Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
3.5.2 Convolution Mechanics . . . . . . . . . . . . . . . . . . . . . . . . . . 123
3.5.3 Step and Ramp Response Characterization . . . . . . . . . . . . . . . 125
3.5.4 Characterizing an LTI System . . . . . . . . . . . . . . . . . . . . . . 126
3.5.5 Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
3.6 Goertzel’s Algorithm for Computing the DFT . . . . . . . . . . . . . . . . . . 128
3.7 Fractional Fourier Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
3.7.1 Periodicity of the Fourier Transform Operator. . . . . . . . . . . . . . 129
3.7.2 Fractional Fourier Transform Criteria . . . . . . . . . . . . . . . . . . 130
3.7.3 The Weighted Fractional Fourier Transform . . . . . . . . . . . . . . . 130
3.8 Approximating a Linear System with LTI Systems . . . . . . . . . . . . . . . 132
3.8.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
3.8.1.1 The Magnifier. . . . . . . . . . . . . . . . . . . . . . . . . . . 133
3.8.1.2 The Fourier Transformer . . . . . . . . . . . . . . . . . . . . 134
3.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
3.10 Solutions for Chapter 3 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . 141
4 Fourier Transforms in Random Variables and Stochastic Processes 149
4.1 Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
4.1.1 ProbabilityDensityFunctions,Expectation,andCharacteristicFunc-
tions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
4.1.1.1 Discrete Random Variables . . . . . . . . . . . . . . . . . . . 150
4.1.1.2 Moments from the Characteristic Functions . . . . . . . . . . 150
4.1.1.3 Chebyshev’s inequality . . . . . . . . . . . . . . . . . . . . . 153
4.1.2 Example Probability Functions, Characteristic Functions, Means and
Variances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
4.1.2.1 The Uniform Random Variable . . . . . . . . . . . . . . . . . 154
4.1.2.2 The Triangle Random Variable . . . . . . . . . . . . . . . . . 158
4.1.2.3 The Gaussian Random Variable . . . . . . . . . . . . . . . . 158
4.1.2.4 The Exponential Random Variable . . . . . . . . . . . . . . . 159
4.1.2.5 The Laplace Random Variable . . . . . . . . . . . . . . . . . 159
4.1.2.6 The Hyperbolic Secant Random Variable . . . . . . . . . . . 160
4.1.2.7 The Gamma Random Variable . . . . . . . . . . . . . . . . . 161
xiv
4.1.2.8 The Beta Random Variable . . . . . . . . . . . . . . . . . . . 163
4.1.2.9 The Cauchy Random Variable . . . . . . . . . . . . . . . . . 164
4.1.2.10 The Deterministic Random Variable . . . . . . . . . . . . . . 166
4.1.2.11 The Bernoulli Random Variable . . . . . . . . . . . . . . . . 167
4.1.2.12 The Discrete Uniform Random Variable . . . . . . . . . . . . 168
4.1.2.13 The Binomial Random Variable . . . . . . . . . . . . . . . . 168
4.1.2.14 The Poisson Random Variable . . . . . . . . . . . . . . . . . 170
4.1.3 Distributions of Sums of Random Variables . . . . . . . . . . . . . . . 172
4.1.3.1 The Sum of Independent Random Variables . . . . . . . . . 172
4.1.3.2 Distributions Closed Under Independent Summation . . . . . 173
4.1.3.3 The Sum of i.i.d. Random Variables . . . . . . . . . . . . . . 175
4.1.3.4 The Average of i.i.d. Random Variables . . . . . . . . . . . 175
4.1.4 A Law of Large Numbers . . . . . . . . . . . . . . . . . . . . . . . . . 176
4.1.4.1 Stochastic Resonance . . . . . . . . . . . . . . . . . . . . . . 176
4.1.5 The Central Limit Theorem . . . . . . . . . . . . . . . . . . . . . . . . 177
4.1.5.1 The Central Limit Theorem Applied to Randomly Gener-
ated Probability Density Functions . . . . . . . . . . . . . . 179
4.1.5.2 Random Variables That Are Asymptotically Gaussian . . . . 179
4.2 Uncertainty in Wave Equations . . . . . . . . . . . . . . . . . . . . . . . . . . 180
4.3 Stochastic Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
4.3.1 First and Second Order Statistics . . . . . . . . . . . . . . . . . . . . . 183
4.3.1.1 Stationary Processes . . . . . . . . . . . . . . . . . . . . . . . 184
4.3.2 Power Spectral Density . . . . . . . . . . . . . . . . . . . . . . . . . . 185
4.3.3 Some Stationary Noise Models . . . . . . . . . . . . . . . . . . . . . . 186
4.3.3.1 Stationary Continuous White Noise . . . . . . . . . . . . . . 186
4.3.3.2 Stationary Discrete White Noise . . . . . . . . . . . . . . . . 186
4.3.3.3 Laplace Autocorrelation . . . . . . . . . . . . . . . . . . . . . 187
4.3.3.4 Ringing Laplace Autocorrelation . . . . . . . . . . . . . . . . 187
4.3.4 Linear Systems with Stationary Stochastic Inputs . . . . . . . . . . . 187
4.3.5 Properties of the Power Spectral Density . . . . . . . . . . . . . . . . 188
4.3.5.1 Second Moment. . . . . . . . . . . . . . . . . . . . . . . . . . 189
4.3.5.2 Realness. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
4.3.5.3 Nonnegativity. . . . . . . . . . . . . . . . . . . . . . . . . . . 189
4.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
4.5 Solutions for Chapter 4 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . 205
5 The Sampling Theorem 214
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
5.1.1 The Cardinal Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
5.1.2 History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
5.2 Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
5.3 Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
5.3.1 Using Comb Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
xv
5.3.2 Fourier Series Proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
5.3.3 Papoulis’ Proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
5.4 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
5.4.1 Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
5.4.1.1 For Finite Energy Signals . . . . . . . . . . . . . . . . . . . . 221
5.4.1.2 For Bandlimited Functions with Finite Area Spectra . . . . . 222
5.4.2 Trapezoidal Integration . . . . . . . . . . . . . . . . . . . . . . . . . . 223
5.4.2.1 Of Bandlimited Functions . . . . . . . . . . . . . . . . . . . . 223
5.4.2.2 Of Linear Integral Transforms . . . . . . . . . . . . . . . . . 224
5.4.2.3 Derivation of the Low Passed Kernel. . . . . . . . . . . . . . 224
5.4.2.4 Parseval’s Theorem for the Cardinal Series . . . . . . . . . . 229
5.4.3 The Time-Bandwidth Product . . . . . . . . . . . . . . . . . . . . . . 229
5.5 Application to Spectra Containing Distributions . . . . . . . . . . . . . . . . 230
5.6 Application to Bandlimited Stochastic Processes . . . . . . . . . . . . . . . . 231
5.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
5.8 Solutions for Chapter 5 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . 236
6 Generalizations of the Sampling Theorem 242
6.1 Generalized Interpolation Functions . . . . . . . . . . . . . . . . . . . . . . . 242
6.1.1 Oversampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
6.1.2 Restoration of lost samples . . . . . . . . . . . . . . . . . . . . . . . . 243
6.1.2.1 Sample Dependency . . . . . . . . . . . . . . . . . . . . . . . 243
6.1.2.2 Restoring a Single Lost Sample . . . . . . . . . . . . . . . . . 243
6.1.2.3 Restoring M Lost Samples . . . . . . . . . . . . . . . . . . . 245
6.1.2.4 Direct Interpolation From M Lost Samples . . . . . . . . . . 246
6.1.2.5 Relaxed Interpolation Formulae . . . . . . . . . . . . . . . . 247
6.1.3 Criteria for Generalized Interpolation Functions. . . . . . . . . . . . . 248
6.1.3.1 Interpolation Functions . . . . . . . . . . . . . . . . . . . . . 249
6.1.4 Reconstruction from a Filtered Signal’s Samples . . . . . . . . . . . . 250
6.1.4.1 Restoration of Samples from an Integrating Detector . . . . 251
6.2 Papoulis’ Generalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
6.2.1 Derivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
6.2.2 Interpolation Function Computation . . . . . . . . . . . . . . . . . . . 257
6.2.3 Example Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
6.2.3.1 Recurrent Nonuniform Sampling . . . . . . . . . . . . . . . . 258
6.2.3.2 Interlaced Signal–Derivative Sampling . . . . . . . . . . . . . 259
6.2.3.3 Higher Order Derivative Sampling . . . . . . . . . . . . . . . 260
6.2.3.4 Effects of Oversampling . . . . . . . . . . . . . . . . . . . . . 262
6.3 Derivative Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
6.3.1 Properties of the Derivative Kernel . . . . . . . . . . . . . . . . . . . . 263
6.4 A Relation Between the Taylor and Cardinal Series . . . . . . . . . . . . . . . 267
6.5 Sampling Trigonometric Polynomials . . . . . . . . . . . . . . . . . . . . . . . 267
6.6 Sampling Theory for Bandpass Functions . . . . . . . . . . . . . . . . . . . . 269
xvi
6.6.1 Heterodyned Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
6.6.2 Direct Bandpass Sampling . . . . . . . . . . . . . . . . . . . . . . . . . 271
6.7 A Summary of Sampling Theorems for Directly Sampled Signals . . . . . . . 273
6.8 Lagrangian Interpolation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
6.9 Kramer’s Generalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
6.10 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
6.11 Solutions for Chapter 6 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . 281
7 Noise and Error Effects 292
7.1 Effects of Additive Data Noise . . . . . . . . . . . . . . . . . . . . . . . . . . 292
7.1.1 On Cardinal Series Interpolation . . . . . . . . . . . . . . . . . . . . . 292
7.1.1.1 Interpolation Noise Level . . . . . . . . . . . . . . . . . . . . 293
7.1.1.2 Effects of Oversampling and Filtering . . . . . . . . . . . . . 293
7.1.2 Interpolation Noise Variance for Directly Sampled Signals . . . . . . . 297
7.1.2.1 Interpolation with Lost Samples . . . . . . . . . . . . . . . . 298
7.1.2.2 Bandpass Functions . . . . . . . . . . . . . . . . . . . . . . . 305
7.1.3 On Papoulis’ Generalization . . . . . . . . . . . . . . . . . . . . . . . . 306
7.1.3.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
7.1.3.2 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
7.1.4 On Derivative Interpolation . . . . . . . . . . . . . . . . . . . . . . . . 312
7.1.5 A Lower Bound on the NINV . . . . . . . . . . . . . . . . . . . . . . . 314
7.1.5.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
7.2 Jitter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320
7.3 Filtered Cardinal Series Interpolation. . . . . . . . . . . . . . . . . . . . . . . 320
7.3.1 Unbiased Interpolation from Jittered Samples . . . . . . . . . . . . . . 321
7.3.2 In Stochastic Bandlimited Signal Interpolation . . . . . . . . . . . . . 323
7.3.2.1 NINV of Unbiased Restoration . . . . . . . . . . . . . . . . . 324
7.3.2.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
7.4 Truncation Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326
7.4.1 An Error Bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326
7.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330
7.6 Solutions for Chapter 7 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . 332
8 Multidimensional Signal Analysis 335
8.1 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
8.2 Visualizing Higher Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . 337
8.2.1 N Dimensional Tic Tac Toe . . . . . . . . . . . . . . . . . . . . . . . . 337
8.2.2 Solving Simultaneous Linear Equations and Matrix Manipulation in
Higher Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338
8.3 Continuous Time Multidimensional Fourier Analysis . . . . . . . . . . . . . . 339
8.3.1 The Shift Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341
8.3.2 Multidimensional Convolution . . . . . . . . . . . . . . . . . . . . . . 341
8.3.2.1 The Mechanics of Two Dimensional Convolution. . . . . . . 342