Table Of ContentAdvances in Computer Vision and Pattern
Recognition
Forfurthervolumes:
www.springer.com/series/4205
Antonio Robles-Kelly (cid:2) Cong Phuoc Huynh
Imaging
Spectroscopy for
Scene Analysis
AntonioRobles-Kelly CongPhuocHuynh
NationalICTAustralia NationalICTAustralia
Canberra,ACT Canberra,ACT
Australia Australia
SeriesEditors
Prof.SameerSingh Dr.SingBingKang
ResearchSchoolofInformatics MicrosoftResearch
LoughboroughUniversity MicrosoftCorporation
Loughborough Redmond,WA
UK USA
ISSN2191-6586 ISSN2191-6594(electronic)
AdvancesinComputerVisionandPatternRecognition
ISBN978-1-4471-4651-3 ISBN978-1-4471-4652-0(eBook)
DOI10.1007/978-1-4471-4652-0
SpringerLondonHeidelbergNewYorkDordrecht
LibraryofCongressControlNumber:2012951401
©Springer-VerlagLondon2013
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’slocation,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.
PermissionsforusemaybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violations
areliabletoprosecutionundertherespectiveCopyrightLaw.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Whiletheadviceandinformationinthisbookarebelievedtobetrueandaccurateatthedateofpub-
lication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityforany
errorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,withrespect
tothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
To mywife,forherloveand helpover the
years. To mychildren,for thejoythey
brought intomylife.Andtomyparents and
sisters, fortheircontinuousand
unconditionalsupport.
AntonioRobles-Kelly
To myparents and brother, fortheirendless
loveand continuingsupport.
CongPhuocHuynh
Foreword
Current Computer Vision systems, developed to solve many and varied problems
insceneunderstandingandobjectrecognition,typicallyusethethreecolourchan-
nelsdesignedtomimicthethree-colourvisionsystemofhumans.However,human
vision is complex and involves significant perceptual learning, knowledge acqui-
sition and, in general, post retinal neural processing to arrive at the type of inter-
pretations of objects, scenes and events that predominate our perception. Indeed,
oneofthemoststrikingaspectsofhumanvisualperceptionistheabilitytoappar-
ently“directly”perceivetheworldaroundasmaterials,objectsandphysicalevents
“out there” and not, purely, as images. The challenge for Computer Vision is to
create elegant and efficient ways to achieve these goals and, as this book shows,
suchachievementsarepossiblewiththeuseofnewcamerasystemsthatgobeyond
the fixed three colour channels. Indeed the evolution of Imaging Spectroscopy (in
remotesensingcalledHyperspectralImaging),capableofimagingspectralinforma-
tionbeyondthevisibleregionandatspectralresolutionsfar greaterthanthefixed
threecolourchannels,offersgreatpotentialandnewwaysofextractingthephysical
propertiesofmaterials,objects,scenesandilluminants,muchmorepowerfullythan
thestandardthreecoloursystemscanpossiblydo.
This book is a landmark and timely contribution in this direction as it offers,
forthefirsttime,detaileddescriptionsandanalysisofthesenewcamerasandhow
they can be used to extract physical properties of what is being sensed in elegant
and efficient ways relevant to material identification, object recognition and scene
understanding.
ImagingSpectroscopyalsoenablesthecreationofvisionsystems“ondemand”
thatbestsuitspecificenvironmentsandapplicationswellbeyondthesystemsavail-
ableincurrentfixedthree-coloursystemcameras.Itisalsotheidealcamerasystem
foradvancingtheareaofComputationalPhotographyandmovingComputerVision
awayfromtheimage,perse,towhatisbeingimaged.
InthismonographAntonioandConghavebroughttogethertheirworkinImag-
ingSpectroscopyoverthepastdecadetoresultinabookthatwillbecomeastandard
forthearea.Welldone.
VictoriaResearchLaboratory TerryCaelli
NationalICTAustralia(NICTA)
vii
Preface
The vast majority of ground-based image capture (e.g. for general photography,
surveillance and industrial machine vision) is currently performed using RGB fil-
ters on imaging sensors. This is particularly true for digital photography which,
sinceitsinceptioninthe1980s,hasalmostexclusivelyemployedtrichromaticcam-
erasbasedonCMOSorCCDsensorsusingBayerarrays,three-CCDcamerasora
set of stacked photodiodes with a layer for each colour channel. Despite their dif-
ferences,allofthesesensortypesinvolvecapturinglightatthewavelengthsofthe
threeadditiveprimarycolours(red,greenandblue).Thesebandsareusedbecause
they are a close approximation to the bands to which the human eye is sensitive.
Therefore,theyarewellsuitedtocapturingphotosandvideosforimmediatehuman
perception.
Nowadays,theimagedataisnotonlyusedforimmediatehumanperceptionas
originally intended, but also for a wide range of processing, contributing to cam-
erautility,qualityandperformance.Overthepastfiveyears,manydigitalcameras
have featured integrated circuits and firmware for sophisticated image processing
(suchasCanon’sDIGICandNikon’sExpeedchips).Thesechipsinitiallyperformed
functionssuchasautomaticfocus,exposureandwhitebalance.Morerecently,such
chips have performed a wider range of higher-level scene analysis features such
asfacedetection,smiledetectionandobjecttracking.Suchsceneanalysisfeatures
provide a high value to professional and consumer camera users and are typically
significant selling points in particular models. Many industrial cameras now also
includehigh-levelsceneanalysisfunctionssuchaspeoplecountingforsurveillance
andobjectrecognitionforindustrialmachinevision.
In practise, every scene comprises a rich tapestry of light sources, material re-
flectance, lighting and other photometric effects due to object curvature and shad-
ows.Despitebeingreasonablyeffectiveforsceneanalysis,trichromatic(i.e.RGB)
technologydoeshavelimitsinitssceneanalysiscapabilities.Forexample,acamera
with an RGB sensor cannot determine the constituent material of an object in the
scene.Similarly,cameraswithRGBsensorscannot,ingeneral,deliverphotometric
invariantscharacteristictoamaterialandindependentofthelightingcondition,for
robusttracking,identificationandrecognitiontasks.
ix
x Preface
Inthisbook,weexploretheopportunities,applicationareasandchallengescon-
cerningtheuseofimagingspectroscopyasameansforsceneunderstanding.This
isimportant,sincesceneanalysisinthescopeofimagingspectroscopyinvolvesthe
ability to robustly encode material properties, object composition and concentra-
tions of primordial components. The combination of spatial and spectral informa-
tionpromisesavastnumberofapplicationpossibilities.Forinstance,spectroscopic
scene analysis can enable advanced capabilities for surveillance by permitting ob-
jectsto betracked basedontheir composition.In computationalphotography,im-
age colours may be enhanced taking into account each specific material type in
the scene. For food security, health and precision agriculture the analysis of spec-
troscopicimagescanbethebasisforthedevelopmentofnon-intrusivediagnostic,
monitoringandsurveyingtools.
The ability to combine spatial and compositional information of the scene re-
quiressolvingseveraldifficultproblems.Withtheseproblemssolved,spectroscopic
sceneanalysisoffersthepossibilityofperformingshapeanalysisfromasingleview
for non-diffuse surfaces (Huynh and Robles-Kelly 2009), recovering photometric
invariantsandmaterial-specificsignatures(FuandRobles-Kelly2011a),recovering
the illuminant power spectrum (Huynh and Robles-Kelly 2010a) and visualising
digitalmedia(Kimetal.2010).
With the availability of imaging spectroscopy in ground-based cameras, it will
nolongerbenecessarytolimitthecameradatacapturedtothreeRGBcolourchan-
nels. Hyperspectral cameras offer an alternative number and range of bands that
provide the best trade-off between functionality, performance and cost for a par-
ticular market segment or application need. Rather than having the same spectra
capturedasdisplayed,itwillbepracticaltodecouplethem,capturingarichspectral
representation, performing processing on this representation and then rendering it
intrichromaticformwhenneeded.
Theuseoftheinformation-richrepresentationofthescenethatspectralimaging
providesis,byitself,anewapproachtosceneanalysiswhichmakesuseofthespec-
tralsignaturesandtheircontextsoastoprovideabetterunderstandingofmaterials
andobjectsthroughouttheimage.Theuseofpolarisationandreflectiontorecover
objectprofilesakintothoseinphaseimagingcandelivernovelmethodscapableof
recovering an optimal representation of the scene which captures shape, material,
objectprofilesandphotometricparameterssuchasindexofrefraction.
Furthermore,thehighdimensionalityinherentinspectroscopydataimpliesthat
these algorithms may not be exclusive to imaging spectroscopy, but could also be
appliedtotheprocessingofotherhigh-dimensionaldata.Thus,thesemethodsmay
beextendibletomanyothersensingtechnologiesbeyondjustspectralimagery.
References
Fu,Z.,&Robles-Kelly,A.(2011).Discriminantabsorptionfeaturelearningformaterialclassifi-
cation.IEEETransactionsonGeoscienceandRemoteSensing,49(5),1536–1556.
Preface xi
Huynh,C.P.,&Robles-Kelly,A.(2010).Asolutionofthedichromaticmodelformultispectral
photometricinvariance.InternationalJournalofComputerVision,90(1),1–27.
Huynh,C.P.,&Robles-Kelly,A.(2009).Simultaneousphotometricinvarianceandshaperecovery.
InIEEEinternationalconferenceoncomputervision.
Kim,S.J.,Zhuo,S.,Deng,F.,Fu,C.W.,&Brown,M.(2010).Interactivevisualizationofhy-
perspectralimagesofhistoricaldocuments.IEEETransactionsonVisualizationandGraphics,
16(6),1441–1448.
Acknowledgements
We wish to thank Professor Terry Caelli, Professor Edwin Hancock and Mr. Bill
Simpson-Young for their support, impartial assessment and constructive criticism
onthematerialinthisbook.ProfessorTerryCaelliwasinstrumentalinourefforts
towardswritingthisdocument.Withouthiscontinuingsupport,someoftheresearch
presented here would have never been accomplished. The insights that Professor
Edwin Hancock provided into reflectance modelling and polarisation have greatly
influencedthisbook.Mr.BillSimpson-Youngprovidedavaluableinsightintothe
commercialuseandapplicationsofimagingspectroscopy.
Wewouldliketoexpressoursincereappreciationtoourcollaborators.Inpartic-
ular,wethankDr.ZhouyuFu,Dr.RanWei,Mr.LinGuandMs.SejutiRahmanfor
theirpatienceanddedicationtotheteamworkthatdeliveredsomeoftheresultspre-
sentedhere.Wealsoextendourappreciationtoallourcolleaguesfromthecomputer
visiongroupatNICTA.
Finally,wewouldliketoacknowledgeNICTA,1andinparticularitsCEO,Hugh
Durrant-Whyte, for fostering this technology. We also thank the ANU and the
UNSW@ADFAforallowingustosupervisetheirgraduatestudents.
1NICTAisfundedbytheAustralianGovernmentasrepresentedbytheDepartmentofBroadband,
CommunicationsandtheDigitalEconomyandtheAustralianResearchCouncilthroughtheICT
CentreofExcellenceprogram.
xiii
