Visual Texture: Accurate Material Appearance Measurement, Representation and Modeling

304 Pages·2013·7.15 MB·English

by  Michal Haindl

Advances in Computer Vision and Pattern
Recognition
Forfurthervolumes:
www.springer.com/series/4205
Michal Haindl (cid:2) Jiˇrí Filip
Visual Texture
Accurate Material Appearance
Measurement, Representation
and Modeling
MichalHaindl JiˇríFilip
Inst.ofInformationTheory&Automation Inst.ofInformationTheory&Automation
Acad.ofSciencesoftheCzechRepublic Acad.ofSciencesoftheCzechRepublic
Prague,CzechRepublic Prague,CzechRepublic
SeriesEditors
Prof.SameerSingh Dr.SingBingKang
ResearchSchoolofInformatics MicrosoftResearch
LoughboroughUniversity MicrosoftCorporation
Loughborough Redmond,WA
UK USA
ISSN2191-6586 ISSN2191-6594(electronic)
AdvancesinComputerVisionandPatternRecognition
ISBN978-1-4471-4901-9 ISBN978-1-4471-4902-6(eBook)
DOI10.1007/978-1-4471-4902-6
SpringerLondonHeidelbergNewYorkDordrecht
LibraryofCongressControlNumber:2013930058
©Springer-VerlagLondon2013
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’slocation,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.
PermissionsforusemaybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violations
areliabletoprosecutionundertherespectiveCopyrightLaw.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Whiletheadviceandinformationinthisbookarebelievedtobetrueandaccurateatthedateofpub-
lication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityforany
errorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,withrespect
tothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
To ourfamilies,fortheircontinuoussupport
Preface
Themainpurposeofthisbookistoprovideacomprehensivestate-of-the-artsurvey
of the newly emerging area of physically correct visual texture modeling. Multi-
dimensional visual texture is the appropriate paradigm for physically correct rep-
resentation of material visual properties. The book presents recent advance in the
texture modeling methodologyused in computer vision, pattern recognition, com-
putergraphics,andvirtualandaugmentedrealityapplications.
Whiletextureanalysisisawell-establishedresearchfield,itisstillpredominantly
restrictedtothesimplestandmostapproximatetexturerepresentation—eithergray-
scaleorcolortextures.Severalbooksdevotedtosuchsimplestatictextureanalysis
havebeenpublished,butthereisnobookdedicatedtoeithertheareaofmoregeneral
texturemodelingorrecentstate-of-the-arttexturalrepresentations.
Severalfeaturessetourbookapartfromthefewothervisualtexturebookspub-
lished.
• Theonlybookwithcomprehensivetreatmentoftexturesynthesis.
• The only book covering all known aspects of the most advanced visual surface
representationwhichcanberecentlyapplied—theBidirectionalTextureFunction
(BTF).
• Therighttiming.Thisbookarrivesatatimeofadvancedcomputingandgraph-
icshardwarewhichcanprocessandstoreenormousamountsofdataneededfor
physicallycorrectmaterialmodelingandrecognition;likewise,recentGPUpro-
grammingprogressallowsuserstoutilizerelativelyintuitiveandeconomicalpro-
gramming.This allows for fast implementation,therebyenablingreal industrial
applicationsofthepresentedmethods.
• A complete reference. This self-contained book covers the entire pipeline from
materialappearancerepresentation,measurement,analysis,andcompression,to
modeling,editing,visualization,andperceptualevaluation.
Recent progress in computing and acquisition technology of advanced visual
data,togetherwithadvancesintheoriesofmathematicalmodeling,provideuswith
timelyopportunitytoachievenewbreakthroughsbeyondthecurrentstateofcom-
putervisionart.Finally,itispossibletomeasurenotonlytheordinarystaticcolor
vii
viii Preface
textures, but also the far more complicated and accurate high-dimensional visual
texturerepresentations.
Naturalvisualtexturesprovideampleinformationaboutlocallightingfieldstruc-
ture as well as the surface relief, accounting for such effects as self-occlusions,
self-shadowing,inter-reflectionorsubsurfacescattering.Moreover,theappearance
ofrealmaterialsdramaticallychangeswith,forexample,illuminationandviewing
variations.Theprevailingcomputervisionmethodologyusesonlyasmallfraction
ofthisreadilyavailableandpotentiallyrichinformationsource,butwebelievethat
thisemergingresearchareawillsoonhavesignificantimpactsonfurtherprogressin
artificialvisualcognitionandrelatedapplications.Ouraimisthustoofferthefirst
bookwiththisfocus,inordertofosterthisdevelopment.
The book builds on the authors’ work in this field over two decades and was
inspiredbypositivefeedbacktoseveralofourtutorials:BidirectionalTextureFunc-
tionModellingatCVPR2010,SanFrancisco,AccurateMaterialAppearanceMod-
ellingatSCIA2011,Ystad,AdvancedTexturalRepresentationofMaterialsAppear-
anceatSIGGRAPH2011,HongKong,andAdvancedNatureExteriorsModelling
atICPR2012,Tsukuba.
The book starts from the basic principles and builds on the fundamentals and
basicvisualtexturetaxonomyintroducedasafoundationforusingthelatesttech-
niquesintexturemodeling.Thereaderisexpectedtopossessgraduatelevelknowl-
edge in statistics and probability theory as well as competence in basic computer
graphicsprinciples.However,itisalsosuitablefornewcomerstothefieldofcom-
puter graphics and computer vision, as well as for practitioners who wish to be
brought up to date on the state-of-the-art methodology of texture modeling. This
surveybookwillprovideausefulreferenceandtextbookforresearchers,lecturers,
industrypractitioners,andstudentsinterestedinthisnewandprogressiveresearch
area.
We tried to keep the book as concise as possible to maintain its scope at an
acceptable level. Rather than explaining mathematical and implementation details
of all methods, we refer to the original publications. Our ambition was to provide
the reader with general knowledge about state-of-the-art visual texture modeling.
Attempting to rigorously explain, for example, the Markovian or mixture models
usedinthisbookwouldrequireatleasttwiceasmanypages.
Prague,CzechRepublic MichalHaindl
JiˇríFilip
Acknowledgements
ManycolleaguesfromthedepartmentofPatternRecognitionoftheInstituteofIn-
formationTheoryandAutomationoftheAcademyofSciencesoftheCzechRepub-
licinPraguehavecontributedinvaluablepartstothisworkinourjointpublications
or doing some experimental work. We would like to thank Dr. Jiˇrí Grim, Martin
Hatka,MichalHavlícˇek,Dr.VojteˇchHavlícˇek,RadekHolub,Dr.StanislavMikeš,
VáclavRemeš,Dr.PetrSomol,Dr.PavelVácha,RadomírVávra,andDr.PavelŽid.
We are also grateful for many stimulating discussions to colleagues—Prof.
MichaelJ. Chantler,Prof.PatrickR.Greenfrom theHeriot-WattUniversity,Prof.
ReinhardKleinfromtheBonnUniversity,Dr.GiuseppeScarpafromtheUniversity
FedericoII,andProf.MineichiKudofromtheHokkaidoUniversity.
WewouldlikealsotothankBonnUniversityandYaleUniversityforproviding
ussomeBTFmeasurementsusedinthisbook,DaimlerChryslerAGfor3Dmodelof
carinterior,variousparticipantsfortakingpartinthepsycho-physicalexperiments,
andforthepartialsupportoftheCzechScienceFoundationprojects102/08/0593,
103/11/0335.
Finally,wewouldliketothankourfamiliesfortheirsupportandpatienceduring
manyyearsofresearchleadingtothisbook.
ix
Contents
1 Motivation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 VisualTextureDefinition . . . . . . . . . . . . . . . . . . . . . . 1
1.2 ContentsOverview. . . . . . . . . . . . . . . . . . . . . . . . . . 6
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2 Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1 GeneralReflectanceFunction . . . . . . . . . . . . . . . . . . . . 9
2.2 TexturedModelRepresentationTaxonomy . . . . . . . . . . . . . 12
2.2.1 BidirectionalSurfaceScatteringReflectanceDistribution
Function . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2.2 Bidirectional Reflectance and Transmittance Texture
Function . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2.3 BidirectionalTextureFunction . . . . . . . . . . . . . . . 14
2.2.4 SpatiallyVaryingBRDF . . . . . . . . . . . . . . . . . . . 14
2.2.5 SurfaceLightField . . . . . . . . . . . . . . . . . . . . . 15
2.2.6 SurfaceReflectanceField . . . . . . . . . . . . . . . . . . 16
2.2.7 MultispectralTexture . . . . . . . . . . . . . . . . . . . . 16
2.3 RepresentationTaxonomyofHomogeneousModels . . . . . . . . 17
2.3.1 BidirectionalScatteringDistributionFunction . . . . . . . 18
2.3.2 BidirectionalReflectanceDistributionFunction . . . . . . 18
2.3.3 BidirectionalTransmittanceDistributionFunction . . . . . 19
2.3.4 Isotropic Bidirectional Reflectance Distribution
Function . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.4 AttributesofTaxonomicalClasses . . . . . . . . . . . . . . . . . 21
2.4.1 TaxonomicalClassAdvantages . . . . . . . . . . . . . . . 21
2.4.2 TaxonomicalClassDrawbacks . . . . . . . . . . . . . . . 22
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3 TextureAcquisition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.1 HighDynamicRangeTextureAcquisition . . . . . . . . . . . . . 23
3.2 StaticTexturesAcquisition . . . . . . . . . . . . . . . . . . . . . 24
xi
xii Contents
3.3 DynamicTexturesAcquisition. . . . . . . . . . . . . . . . . . . . 25
3.4 BRDFAcquisition . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.4.1 Gonioreflectometers-BasedBRDFSetups . . . . . . . . . 28
3.4.2 Mirror-BasedBRDFSetups . . . . . . . . . . . . . . . . . 30
3.4.3 Image-BasedBRDFAcquisition . . . . . . . . . . . . . . 32
3.4.4 PortableBRDFAcquisitionSystems . . . . . . . . . . . . 34
3.5 SpatiallyVaryingBRDFAcquisition . . . . . . . . . . . . . . . . 35
3.6 BTFAcquisition . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.6.1 Gonioreflectometers-BasedBTFSetups. . . . . . . . . . . 38
3.6.2 Mirror-BasedBTFSetups . . . . . . . . . . . . . . . . . . 42
3.6.3 OtherBTFSetups . . . . . . . . . . . . . . . . . . . . . . 42
3.6.4 SparseSamplingofBTF. . . . . . . . . . . . . . . . . . . 43
3.6.5 BTFSetupsOverview . . . . . . . . . . . . . . . . . . . . 44
3.6.6 ABTFSetupDesign. . . . . . . . . . . . . . . . . . . . . 44
3.7 MeasurementofTime-VaryingSurfaces . . . . . . . . . . . . . . 47
3.8 BSSRDFMeasurement . . . . . . . . . . . . . . . . . . . . . . . 48
3.8.1 Diffuse-Specular Separation of Reflectance
Measurements . . . . . . . . . . . . . . . . . . . . . . . . 48
3.8.2 HomogeneousSubsurfaceScatteringMeasurement . . . . 50
3.8.3 SpatiallyVaryingSubsurfaceScatteringMeasurement . . . 51
3.9 SurfaceLightandReflectanceFieldsMeasurements . . . . . . . . 53
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4 StaticMultispectralTextures . . . . . . . . . . . . . . . . . . . . . . 63
4.1 TextureModelingApproaches . . . . . . . . . . . . . . . . . . . . 63
4.2 Model-BasedRepresentations . . . . . . . . . . . . . . . . . . . . 64
4.2.1 SpectralFactorization . . . . . . . . . . . . . . . . . . . . 65
4.2.2 SpatialFactorization . . . . . . . . . . . . . . . . . . . . . 66
4.2.3 FractalModels . . . . . . . . . . . . . . . . . . . . . . . . 67
4.2.4 RandomMosaics . . . . . . . . . . . . . . . . . . . . . . 68
4.2.5 MarkovianModels . . . . . . . . . . . . . . . . . . . . . . 70
4.2.6 MixtureModels . . . . . . . . . . . . . . . . . . . . . . . 80
4.2.7 ProbabilisticDiscrete-Mixture2DModel . . . . . . . . . . 83
4.2.8 BernoulliDistributionMixtureModel. . . . . . . . . . . . 84
4.2.9 Gaussian-Mixture2DModel . . . . . . . . . . . . . . . . 85
4.2.10 MixtureModel-BasedTextureSynthesis . . . . . . . . . . 86
4.2.11 ProbabilisticMixtureModelsProperties . . . . . . . . . . 87
4.3 TextureSampling . . . . . . . . . . . . . . . . . . . . . . . . . . 88
4.3.1 TextureSamplingMethods . . . . . . . . . . . . . . . . . 88
4.3.2 Roller . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
4.3.3 SamplingMethodsSummary . . . . . . . . . . . . . . . . 89
4.4 HybridModeling. . . . . . . . . . . . . . . . . . . . . . . . . . . 90
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
5 DynamicTextures . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97