Table Of ContentD I G I TA L L I G H T F I E L D P H O T O G R A P H Y
a dissertation
submitted to the department of computer science
and the committee on graduate studies
of stanford university
in partial fulfillment of the requirements
for the degree of
doctor of philosophy
RenNg
July
© Copyright by Ren Ng
All Rights Reserved
ii
I certify that I have read this dissertation and that, in my opinion, it is fully
adequate in scope and quality as a dissertation for the degree of Doctor of
Philosophy.
PatrickHanrahan PrincipalAdviser
I certify that I have read this dissertation and that, in my opinion, it is fully
adequate in scope and quality as a dissertation for the degree of Doctor of
Philosophy.
MarcLevoy
I certify that I have read this dissertation and that, in my opinion, it is fully
adequate in scope and quality as a dissertation for the degree of Doctor of
Philosophy.
MarkHorowitz
ApprovedfortheUniversityCommitteeonGraduateStudies.
iii
iv
Acknowledgments
I feel tremendously lucky to have had the opportunity to work with Pat Hanrahan, Marc
LevoyandMarkHorowitzontheideasinthisdissertation,andIwouldliketothankthem
fortheirsupport. Patinstilledinmealoveforsimulatingtheflowoflight,agreedtotakeme
onasagraduatestudent,andencouragedmetoimmersemyselfinsomethingIhadapassion
for. Icouldnothaveaskedforafinermentor. MarcLevoyistheonewhooriginallydrewme
tocomputergraphics,hasworkedsidebysidewithmeattheopticalbench,andisvigorously
carryingtheseideastonewfrontiersinlightfieldmicroscopy. MarkHorowitzinspiredme
toassemblemycamerabysharinghislovefordismantlingoldthingsandbuildingnewones.
Ihavenevermetaprofessormoregenerouswithhistimeandexperience.
IamgratefultoBrianWandellandDwightNishimuraforservingonmyoralscommit-
tee. DwighthasbeenanunfailingsourceofencouragementduringmytimeatStanford.
Iwouldliketoacknowledgethefineworkoftheotherindividualswhohavecontributed
tothiscameraresearch.MathieuBrédifworkedcloselywithmeindevelopingthesimulation
system, and he implemented the original lens correction software. Gene Duval generously
donated his time and expertise to help design and assemble the prototype, working even
through illness to help me meet publication deadlines. Andrew Adams and Meng Yu con-
tributed software to refocus light fields more intelligently. Kayvon Fatahalian contributed
themosttoexplaininghowthesystemworks,andmanyoftheraydiagramsinthesepages
areduetohisartistry.
Assemblingtheprototyperequiredcustomsupportfromseveralvendors.Specialthanks
to Keith Wetzel at Kodak Image Sensor Solutions for outstanding support with the photo-
sensor chips, Thanks also to John Cox at Megavision, Seth Pappas and Allison Roberts at
AdaptiveOpticsAssociates,andMarkHolzbachatZebraImaging.
v
In addition, I would like to thank Heather Gentner and Ada Glucksman at the Stan-
ford Graphics Lab for providing mission-critical administrative support, and John Gerth
forkeepingthecomputinginfrastructurerunningsmoothly.
ThanksalsotoPeterCatrysse, BrianCurless, JoyceFarrell, KeithFife, AbbasElGamal,
Joe Goodman, Bert Hesselink, Brad Osgood, and Doug Osheroff for helpful discussions
relatedtothiswork.
AMicrosoftResearchFellowshiphassupportedmyresearchoverthelasttwoyears.This
fellowshipgavemethefreedomtothinkmorebroadlyaboutmygraduatework,allowingme
torefocusmygraphicsresearchondigitalphotography.AStanfordBirdseedGrantprovided
the resources to assemble the prototype camera. I would also like to express my gratitude
toStanfordUniversityandScotchCollegeforalltheopportunitiesthattheyhavegivenme
overtheyears.
IwouldliketothankallmywonderfulfriendsandcolleaguesattheStanfordGraphicsLab.
I can think of no finer individual than Kayvon Fatahalian, who has been an exceptional
friend to me both in and out of the lab. Manu Kumar has been one of my strongest sup-
porters,andIamverygratefulforhisencouragementandpatientadvice. JeffKlingnerisa
source of inspiration with his infectious enthusiasm and amazing outlook on life. I would
especiallyliketothankmycollaborators: EricChan,MikeHouston,GregHumphreys,Bill
Mark, Kekoa Proudfoot, Ravi Ramamoorthi, Pradeep Sen and Rui Wang. Special thanks
alsotoJohnOwens,MattPharrandBennettWilburnforbeingsogenerouswiththeirtime
andexpertise.
Iwouldalsoliketothankmyfriendsoutsidethelab,theclimbingposse,whohavehelped
makemygraduateyearssoenjoyable,includingMarshallBurke,StephCheng,AlexCooper,
PollyFordyce,NamiHayashi,LisaHwang,JoeJohnson,ScottMatula,ErikaMonahan,Mark
Pauly, Jeff Reichbach, Matt Reidenbach, Dave Weaver and Mike Whitfield. Special thanks
are due to Nami for tolerating the hair dryer, spotlights, and the click of my shutter in the
nameofscience.
Finally,Iwouldliketothankmyfamily,YiFoong,BengLymnandCheeKeongNg,for
theirloveandsupport.Myparentshavemadecountlesssacrificesforme,andhaveprovided
mewithsteadyguidanceandencouragement. Thisdissertationisdedicatedtothem.
vi
(cid:1224)(cid:1)(cid:39)(cid:80)(cid:83)(cid:1)(cid:46)(cid:66)(cid:78)(cid:66)(cid:1)(cid:66)(cid:79)(cid:69)(cid:1)(cid:49)(cid:66)(cid:81)(cid:66)(cid:1)(cid:1224)
vii
viii
Contents
Acknowledgments v
1 Introduction 1
. TheFocusProbleminPhotography . . . . . . . . . . . . . . . . . . . . . . .
. TrendsinDigitalPhotography . . . . . . . . . . . . . . . . . . . . . . . . . .
. DigitalLightFieldPhotography . . . . . . . . . . . . . . . . . . . . . . . . .
. DissertationOverview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2 LightFieldsandPhotographs 11
. PreviousWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. TheLightFieldFlowingintotheCamera . . . . . . . . . . . . . . . . . . . .
. PhotographFormation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. ImagingEquations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3 RecordingaPhotograph’sLightField 23
. APlenopticCameraRecordstheLightField . . . . . . . . . . . . . . . . . .
. ComputingPhotographsfromtheLightField. . . . . . . . . . . . . . . . . .
. ThreeViewsoftheRecordedLightField . . . . . . . . . . . . . . . . . . . . .
. ResolutionLimitsofthePlenopticCamera . . . . . . . . . . . . . . . . . . .
. GeneralizingthePlenopticCamera. . . . . . . . . . . . . . . . . . . . . . . .
. PrototypeLightFieldCamera . . . . . . . . . . . . . . . . . . . . . . . . . . .
. RelatedWorkandFurtherReading . . . . . . . . . . . . . . . . . . . . . . . .
ix
x contents
4 DigitalRefocusing 49
. PreviousWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. ImageSynthesisAlgorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. TheoreticalRefocusingPerformance . . . . . . . . . . . . . . . . . . . . . . .
. TheoreticalNoisePerformance . . . . . . . . . . . . . . . . . . . . . . . . . .
. ExperimentalPerformance . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. TechnicalSummary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. PhotographicApplications . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5 SignalProcessingFramework 79
. PreviousWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. PhotographicImagingintheFourierDomain . . . . . . . . . . . . . . . . .
.. GeneralizationoftheFourierSliceTheorem . . . . . . . . . . . . . .
.. FourierSlicePhotographTheorem . . . . . . . . . . . . . . . . . . .
.. PhotographicEffectofFilteringtheLightField . . . . . . . . . . . .
. Band-LimitedAnalysisofRefocusingPerformance . . . . . . . . . . . . . .
. FourierSliceDigitalRefocusing . . . . . . . . . . . . . . . . . . . . . . . . .
. LightFieldTomography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6 SelectableRefocusingPower 113
. SamplingPatternoftheGeneralizedLightFieldCamera . . . . . . . . . . .
. OptimalFocusingofthePhotographicLens. . . . . . . . . . . . . . . . . . .
. ExperimentswithPrototypeCamera . . . . . . . . . . . . . . . . . . . . . . .
. ExperimentswithRay-TraceSimulator . . . . . . . . . . . . . . . . . . . . .
7 DigitalCorrectionofLensAberrations 131
. PreviousWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. TerminologyandNotation . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. VisualizingAberrationsinRecordedLightFields . . . . . . . . . . . . . . . .
. ReviewofOpticalCorrectionTechniques . . . . . . . . . . . . . . . . . . . .
. DigitalCorrectionAlgorithms . . . . . . . . . . . . . . . . . . . . . . . . . .
Description:DIGITAL LIGHT FIELD PHOTOGRAPHY adissertation submittedtothedepartmentofcomputerscience andthecommitteeongraduatestudies ofstanforduniversity
