Applied Fourier Analysis. From Signal Processing to Medical Imaging

307 Pages·2017·8.318 MB·english

by  Tim Olson

Tim Olson
Applied Fourier Analysis
From Signal Processing to Medical Imaging
Tim Olson
Department ofMathematics
University of Florida
Gainesville, FL
USA
ISBN978-1-4939-7391-0 ISBN978-1-4939-7393-4 (eBook)
https://doi.org/10.1007/978-1-4939-7393-4
LibraryofCongressControlNumber:2017949170
MathematicsSubjectClassification(2010): 42A38,42B10,65T50,92C55,94A08,94A12,94A20
©SpringerScience+BusinessMedia,LLC2017
Preface
Purpose
Thisbook istheoutgrowthofmany years oftrying toquicklyteach engineers and
mathematicsmajorsFourierAnalysisatUniversityofFlorida.Themajorconstraints
were the following:
1. The students will not necessarily have a prior course in modern analysis;
therefore, Lebesgue measure is suppressed. Linear Algebra and calculus is
all that is assumed.
2. The students generally do not have two or three semesters to master the
subject. They have one semester to get an overview of the subject. While
thosearetheconstraintsinGainesville,Ibelievethatthisbookwillsupporta
two-semestercoursetaughtslowerwithcompletedetail.Intenyears,Ihave
never covered everything in this book.
3. Most of the students will not be interested in an analysis class which does
notcoversomeapplications.Understandingsomeapplicationsneedstobea
goal of the class.
4. The course should have mathematical rigor, but this rigor should not be
emphasized at the cost of applications.
5. The students should be exposed to the concepts of isometry, sampling,
interpolation, and aliasing.
6. WhenIwrotethefirsttestforthisclass,manyofthequestionshadbeenon
previous Numerical Analysis Ph.D. qualifying exam. Thus, I had to find a
way to let undergraduate students learn, without expecting heavy proof
ability as would be expected at a much later date.
7. Some exercises which involve the calculation of Fourier Series or integrals
arenecessaryandhelpful.Calculationofcomplicatedintegralsshouldnotbe
the only emphasis of the course.
8. Projects seem to be the best way of having them “get their feet wet”. Most
notably numerical projects are using MATLAB, although other packages
could suffice. I let them program in any language they wish, but refuse to
answer questions about anything other than MATLAB. I have included
some of the projects I used for one semester. Please feel free to alter them.
Whilethereweremanymathematicsstudentsinthesecourses,manyofthemare
EE,CS,ME,Physics,etc.Thus,thepurposeistogiveamathematicalintroduction
to Fourier Analysis, while emphasizing the applications. The goal is not to give a
finalcourseonFourierAnalysiswithcompletemathematical rigor.Thegoalisnot
tomakethecourseafinalcourseonanyoftheappliedsubjecttopics.Thegoalisto
settheplateforfurtherinvestigationbythestudentsintothetopicswhichtheyfind
most interesting or which their individual study plans demand.
Suggestions to Professors
Covering the Material
One obvious observation is that there is a lot of material in the book for one
semester. I may have come close to covering all of it in one semester, but that is
never my goal. I would love to be at a place where the students do not need to
graduate,andwecouldtaketwosemesterstocoverthematerial.Inathirteen-week
semester, I generally spend 7–8 weeks on Chapters 2-4. I try to emphasize to the
studentsmybeliefthatiftheyknowthismaterial,theycanreadtheresteasily.Ido
emphasize Chapter 5, since sampling is at the forefront of most of modern signal
processing, communications, etc.
AfterChapters.2-4,thingsbecomefareasierforthestudents,andgenerally,itis
funtimetoplaywithwhattheylearned.Iusuallycoverthelaterchaptersinaweek
or so. I pick and choose them according to what the students are interested in.
Normally, Imove to Image Processing after Chapter 5, because nearly all students
want to be able to play with images. The Medical Imaging chapter is of personal
interest to me, and I find it a lot offun. I believe that students do also. I generally
endupwithsomecleanuptime,i.e.,justcoveringthehighlightsofchapterswhich
we did not cover, sometimes one lecture per chapter.
Insummary,Idonotfeelitisnecessarytocovermorethanoneortwochapters
after the base knowledge in Chapters 2-5. If I have time, I keep going.
Projects
You will notice that the chapter projects are merely collections of exercises. There
is no magic in how I have arranged them. They seem to work, but they are
continually evolving. You may want to change small things because code from
prior semesters will always hang around and is hard for students to resist. For
instance, merely changing expansions on ½(cid:2)…;…(cid:3) to ½(cid:2)2;2(cid:3) will insure that the
students have to put some of their own thought into the projects.
Itiscompletelyviabletobuildcustomprojectsfromindividualexercises.Itend
to stay with ones that have traditionally worked. Whether other projects are
designed is certainly up to the individual instructor.
Testing
Part of the reason I wrote this book was that testing on this broad material was
difficult,andprobablyabovethelevelofanundergraduateclass.Ontheotherhand,
I do give one test per semester, generally over the material in Chapter 4. The
purposeofthattestistomakesurethatthestudentsknowsomeofFourierAnalysis
off the top of their head. Required material includes the basics of dilation, trans-
lation, differentiation, convolution, etc. I refer to these as the “easy” problems.
I challenge them to know some of the “hard” problems, such as the uncertainty
principle, the proof of the Inverse Fourier Transform, and the Shannon Sampling
Theorem. I do not require all of them.
Thereisnomagichere,justwhatIfoundnecessarytofeelthatIhadtransferred
the knowledge. Please experiment with a mix of testing vs. projects. If you find
something that works, please let me know.
Acknowledgements and History
Undergraduate and Graduate Years
This book is somewhat of an oral history of my training in mathematics. I was an
undergraduate student at University of Montana in Missoula. It was only through
theeffortsofsomededicatedProfessors,mostnotablyGeorgeMcRaeandWilliam
Derrick,thatIsurvivedmytimeasanundergraduatewithsomeacademicintegrity
inplace.TheyconvincedmethatIcouldsucceedatMathematicsifonlyIdecided
to study.
I attended graduate school at Auburn University and began to look at
Mathematics as a career and something that I loved, rather than a distraction from
my other pursuits. The entire faculty at Auburn was very engaged in educating
the graduate students, and I have to commend all of them. My thesis advisor
Dr. Richard Zalik treated me as one of his family and guided me through my
experience there. My experience at Auburn included a wide number of the
Professors, and I would like to thank all of them. I was also blessed with a great
number of excellent colleagues at Auburn, and I have to mention Saady, Bobby,
and Jeffy, who received their Ph.Ds with me.
WhileatAuburn,Istudied tworather different disciplines. IwrotemyMaster’s
Thesis on Tomography [20] and learned a great deal about dealing with real data
and applied numerical analysis. I then wrote a fairly pure mathematical thesis on
wavelet-type basis for Fourier Analysis [21]. These two different approaches, one
very applied and one fairly pure, are hopefully illustrated in this book.
The Dartmouth Years
IwasprivilegedtoleaveAuburntoworkwithDennisHealyetalatDartmouthon
apostdoctoralfellowship.Dartmouthwasagreatplacetobeatthetime.Dennisand
Iworkedonanumberofapplications,medicalimaging,radarprocessing,statistical
pattern recognition, as well as others.
I also had the blessing of working with a great group of others, including Dan
Rockmore and John Weaver. We eventually named our group the “Wavelet
Warriors.” While we rarely did anything directly with wavelets, we believed that
Fourier Analysis, appropriately used through wavelets or otherwise, could solve
problems.Wesolvedsome,andweleftmanyunsolved,butitwasagreattimeand
place to work. Most of the Wavelet Warriors are shown in Figure 1, including the
waveletdogs,CharlieOlsonandDiggerRockmore.WhileatDartmouth,wewerea
family that generally played and worked together. I will always remember it as a
very special time.
University of Florida
IspentayearatJohnsHopkins,studyingunderCareyPriebe.Weenjoyedthetime
andheenlightenedmetothewondersofspatialstatistics.IhavebeenatUniversity
of Florida since 1997. My colleagues have all been great. I must acknowledge the
greatluck IhadtofirststudyatDartmouth withtheWaveletWarriors,andthento
endupatFlorida.JohnKlauderhasdonehisbesttokeepmyknowledgeofFourier
Analysisaliveandkeepmeoutoftrouble.Botharefull-timejobs.SergeiShabanov
has tolerated my ignorance of physics and answered any questions I had.
Figure1: WaveletWarriorsaredepictedinthisphotographfromJuneof1996.Inthetoproware
PeterKostelec,DougWarner,SumitChawla,GeoffDavis,andDennisHealy.Atthebottomare
DanRockmore,DiggerRockmore,CharlieOlson,andTimOlson.TheWaveletWarriorViking
regaliaiscirca1995andbroughtmuchstrengthtothegroup.
Contents
1 Introduction: From Linear Algebra to Linear Analysis . . . . . . . . . 1
1.1 Three variations of Fourier Analysis . . . . . . . . . . . . . . . . . . . . 2
1.1.1 Fourier Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.2 The Discrete Fourier Transform . . . . . . . . . . . . . . . . . 3
1.1.3 The Continuous Fourier Transform . . . . . . . . . . . . . . . 4
1.2 Motivations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2.1 Exploration and Understanding . . . . . . . . . . . . . . . . . 5
1.2.2 Manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2.3 Compression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3 Linear Algebra, Linear Analysis, and Fourier Analysis . . . . . . 8
1.3.1 The Dot Product, Inner Product, and Orthogonality . . . 9
1.3.2 Eigenvectors and Eigenvalues in Linear Algebra . . . . . 12
1.3.3 Orthogonal Diagonalization . . . . . . . . . . . . . . . . . . . . 14
1.3.4 Diagonalization in Linear Analysis: Eigenfunctions . . . 14
1.3.5 Fourier Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.3.6 Notational Differences . . . . . . . . . . . . . . . . . . . . . . . . 16
2 Basic Fourier Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.1 Fourier Series on L2½a;b(cid:3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.1.1 Calculating a Fourier Series . . . . . . . . . . . . . . . . . . . . 22
2.1.2 Periodicity and Equality . . . . . . . . . . . . . . . . . . . . . . 27
2.1.3 Problems and Exercises . . . . . . . . . . . . . . . . . . . . . . . 29
2.2 Orthogonality and Hilbert Spaces . . . . . . . . . . . . . . . . . . . . . . 30
2.2.1 Orthogonal Expansions . . . . . . . . . . . . . . . . . . . . . . . 33
2.2.2 Problems and Exercises: . . . . . . . . . . . . . . . . . . . . . . 35
2.3 The Pythagorean Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.3.1 The Isometry between L2½a;b(cid:3) and l2 . . . . . . . . . . . . . 37
2.3.2 Complex Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.3.3 Estimating Truncation Errors . . . . . . . . . . . . . . . . . . . 42
2.3.4 Problems and Exercises: . . . . . . . . . . . . . . . . . . . . . . 45
2.4 Differentiation and Convergence Rates . . . . . . . . . . . . . . . . . . 46
2.4.1 A Quandary between Calculus
and Fourier Analysis . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.4.2 Derivatives and Rates of Decay . . . . . . . . . . . . . . . . . 48
2.4.3 Fourier Derivatives and Induced Discontinuities . . . . . 51
2.4.4 Problems and Exercises: . . . . . . . . . . . . . . . . . . . . . . 52
2.5 Sine and Cosine Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
2.5.1 Problems and Exercises: . . . . . . . . . . . . . . . . . . . . . . 58
2.6 Perhaps Cosine Series Only . . . . . . . . . . . . . . . . . . . . . . . . . . 59
2.6.1 Induced Discontinuities vs. True Discontinuities . . . . . 61
2.6.2 Problems and Exercises: . . . . . . . . . . . . . . . . . . . . . . 62
2.7 Gibbs Ringing Phenomenon . . . . . . . . . . . . . . . . . . . . . . . . . . 63
2.7.1 Problems and Exercises . . . . . . . . . . . . . . . . . . . . . . . 64
2.8 Convolution and Correlation . . . . . . . . . . . . . . . . . . . . . . . . . 65
2.8.1 A couple of classic examples . . . . . . . . . . . . . . . . . . . 67
2.8.2 Problems and Exercises: . . . . . . . . . . . . . . . . . . . . . . 68
2.9 Chapter Project: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
2.10 Summary of Expansions: . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3 The Discrete Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.1 The Fourier Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
3.1.1 Orthogonality of the Fourier Matrix . . . . . . . . . . . . . . 77
3.2 The Complex N’th roots of unity and their structure . . . . . . . . 79
3.2.1 Problems and Exercises: . . . . . . . . . . . . . . . . . . . . . . 79
3.2.2 The roots of unity and frequency . . . . . . . . . . . . . . . . 81
3.2.3 Making the Fourier Matrix Understandable . . . . . . . . . 82
3.2.4 Problems and Exercises: . . . . . . . . . . . . . . . . . . . . . . 83
3.2.5 Group structure of the roots of unity . . . . . . . . . . . . . 84
3.2.6 Subgroups and Coset Representations . . . . . . . . . . . . . 86
3.2.7 Problems and Exercises: . . . . . . . . . . . . . . . . . . . . . . 87
3.3 The Fast Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . . 88
3.3.1 Speed Enabling Algorithm . . . . . . . . . . . . . . . . . . . . . 89
3.3.2 The simplest examples: F , F , and F . . . . . . . . . . . . 89
2 4 8
3.3.3 The FFT in the Classic Case N ¼2q . . . . . . . . . . . . . 93
3.3.4 The FFT when N ¼N N . . . . . . . . . . . . . . . . . . . . . 98
1 2
3.3.5 Problems and Exercises: . . . . . . . . . . . . . . . . . . . . . . 100
3.4 Discrete Convolution and Correlation . . . . . . . . . . . . . . . . . . . 101
3.4.1 Circulant Toeplitz Matrices . . . . . . . . . . . . . . . . . . . . 103
3.5 Applications of Convolution and Correlation . . . . . . . . . . . . . . 105
3.5.1 Derivatives via Convolution . . . . . . . . . . . . . . . . . . . . 105
3.5.2 Averaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
3.5.3 Narrowband and Nonlinear filtering . . . . . . . . . . . . . . 110
3.5.4 Matched Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
3.5.5 Problems and Exercises: . . . . . . . . . . . . . . . . . . . . . . 114
3.6 Computing Sine and Cosine Expansions Using the FFT:
Compression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
3.6.1 Cosine expansions . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
3.6.2 Sine expansions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
3.6.3 Piecewise Cosine and Sine expansions . . . . . . . . . . . . 117
3.6.4 Problems and Exercises: . . . . . . . . . . . . . . . . . . . . . . 118
3.7 Chapter Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
4 The Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
4.1 From Fourier Series to the Fourier Transform . . . . . . . . . . . . . 121
4.2 The Fourier Isometry from L2ðRÞ to L2ðRÞ . . . . . . . . . . . . . . . 125
4.2.1 Problems and Exercises: . . . . . . . . . . . . . . . . . . . . . . 126
4.3 The Basic Theorems of Fourier Analysis . . . . . . . . . . . . . . . . . 126
4.3.1 Differentiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
4.3.2 Translation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
4.3.3 Convolution and Correlation . . . . . . . . . . . . . . . . . . . 129
4.3.4 Dilation and Uncertainty . . . . . . . . . . . . . . . . . . . . . . 133
4.3.5 The Fourier Transform of a Gaussian . . . . . . . . . . . . . 137
4.3.6 Problems and Exercises: . . . . . . . . . . . . . . . . . . . . . . 138
4.4 The Inverse Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
4.4.1 Proving the Fourier Inversion Formula . . . . . . . . . . . . 139
4.4.2 Problems and Exercises: . . . . . . . . . . . . . . . . . . . . . . 141
4.5 Partition of Unity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
4.6 Revisiting Gibbs’ ringing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
4.6.1 Problems and Exercises: . . . . . . . . . . . . . . . . . . . . . . 145
4.7 Suggested Test Review: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
5 Sampling and Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
5.1 The Shannon Sampling Theorem . . . . . . . . . . . . . . . . . . . . . . 150
5.1.1 Recalling Polynomial Interpolation . . . . . . . . . . . . . . . 154
5.1.2 Shannon’s Theorem: Advantages and
Disadvantages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
5.2 Advanced Interpolation Theory . . . . . . . . . . . . . . . . . . . . . . . . 156
5.2.1 Basic Ideas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
5.2.2 Oversampling and Adaptive Windows . . . . . . . . . . . . 159
5.2.3 Interpolating Window Functions . . . . . . . . . . . . . . . . 161
5.2.4 Approximate Window functions: . . . . . . . . . . . . . . . . 164
5.2.5 Problems and Exercises: . . . . . . . . . . . . . . . . . . . . . . 167
5.3 Aliasing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
5.3.1 Anti-Aliasing Filters . . . . . . . . . . . . . . . . . . . . . . . . . 170
5.3.2 Problems and Exercises: . . . . . . . . . . . . . . . . . . . . . . 171
5.4 Numerical Computation of the Shannon Series . . . . . . . . . . . . 171
5.4.1 Example 1: Exact Interpolation . . . . . . . . . . . . . . . . . 172
5.4.2 Example 2: Approximate Interpolation . . . . . . . . . . . . 173
5.4.3 Problems and Exercises: . . . . . . . . . . . . . . . . . . . . . . 175
5.5 Chapter Project: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
6 Digital Communications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
6.1 Introduction and Basic Terms . . . . . . . . . . . . . . . . . . . . . . . . . 177
6.1.1 Definitions from Communications . . . . . . . . . . . . . . . 180
6.2 Modulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
6.2.1 Analogue vs. Digital . . . . . . . . . . . . . . . . . . . . . . . . . 181
6.2.2 AM vs FM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
6.2.3 RZ vs. NRZ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
6.3 The Eye Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
6.3.1 Interference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
6.4 Fourier Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
6.4.1 Uncertainty, Dilation, and ISI . . . . . . . . . . . . . . . . . . 190
6.4.2 Uncertainty, Dilation, and CCI . . . . . . . . . . . . . . . . . . 191
6.5 Statistics and Matched Filtering . . . . . . . . . . . . . . . . . . . . . . . 192
6.5.1 Gaussian White Noise . . . . . . . . . . . . . . . . . . . . . . . . 195
6.5.2 Averaging Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
6.5.3 The Matched Filter . . . . . . . . . . . . . . . . . . . . . . . . . . 198
6.5.4 Transmitter and Receiver Details . . . . . . . . . . . . . . . . 201
6.5.5 Chapter Project: . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
7 Radar Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
7.1 Range and Velocity Measurements . . . . . . . . . . . . . . . . . . . . . 205
7.1.1 Range Measurements . . . . . . . . . . . . . . . . . . . . . . . . . 205
7.1.2 Simple Range Resolution . . . . . . . . . . . . . . . . . . . . . . 206
7.1.3 Maximum Unambiguous Range . . . . . . . . . . . . . . . . . 207
7.1.4 Velocity Estimation and Resolution . . . . . . . . . . . . . . 208
7.2 Radar Range Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
7.2.1 Problems and Exercises: . . . . . . . . . . . . . . . . . . . . . . 210
7.3 Signal to Noise Management in Radar . . . . . . . . . . . . . . . . . . 211
7.3.1 Matched Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
7.3.2 Signal Averaging . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
7.3.3 Heterodyne Receivers . . . . . . . . . . . . . . . . . . . . . . . . 214
7.3.4 IF Band-Pass Filters . . . . . . . . . . . . . . . . . . . . . . . . . 216
7.3.5 Problems and Exercises: . . . . . . . . . . . . . . . . . . . . . . 217
7.4 Revisiting Range Resolution: Uncertainty and Bandwidth . . . . 218
7.5 Doppler Measurement and Resolution . . . . . . . . . . . . . . . . . . . 221
7.5.1 Doppler from a moving Target . . . . . . . . . . . . . . . . . 221
7.5.2 Doppler from a moving Source . . . . . . . . . . . . . . . . . 223
7.5.3 Doppler Resolution . . . . . . . . . . . . . . . . . . . . . . . . . . 224