Methods for the automatic alignment of colour histograms

191 Pages·2009·9.15 MB·English

1
Methods For The Automatic Alignment Of
Colour Histograms
Christopher Rohan Senanayake
Adissertationsubmittedinpartialfulfillment
oftherequirementsforthedegreeof
EngineeringDoctorate
of
UniversityCollegeLondon.
DepartmentofComputerScience
UniversityCollegeLondon
2009
2
I,ChristopherRohanSenanayake,confirmthattheworkpresentedinthisthesisismyown.Where
informationhasbeenderivedfromothersources,Iconfirmthatthishasbeenindicatedinthethesis.
Abstract 3
Abstract
Colourprovidesimportantinformationinmanyimageprocessingtaskssuchasobjectidentificationand
tracking. Differentimagesofthesameobjectfrequentlyyielddifferentcolourvaluesduetoundesired
variationsinlightingandthecamera.Inpractice,controllingthesourceofthesefluctuationsisdifficult,
uneconomicalorevenimpossibleinaparticularimagingenvironment.Thisthesisisconcernedwiththe
questionof howto bestalign the correspondingclustersof colourhistogramsto reduceor removethe
effectoftheseundesiredvariations.
Weintroducefeaturebasedhistogramalignment(FBHA)algorithmsthatenableflexiblealignment
transformationsto be applied. The FBHA approachhas three steps, 1) feature detection in the colour
histograms, 2)featureassociation and3) featurealignment. We investigatethe choicesforthese three
stepsontwocolourdatabases:1)astructuredandlabeleddatabaseofRGBimageryacquiredundercon-
trolledcamera,lightingandobjectvariationand2)grey-levelvideostreamsfromanindustrialinspection
application.ThedesignandacquisitionoftheRGBimageandgrey-levelvideodatabasesareakeycon-
tribution of the thesis. The databases are used to quantitatively compare the FBHA approach against
existingmethodologiesandshowittobeeffective. FBHA isintendedtoprovidea genericmethodfor
aligningcolourhistograms, it onlyuses informationfrom the histogramsandthereforeignoresspatial
informationintheimage. Spatialinformationandothercontextsensitivecuesaredeliberatelyavoided
tomaintainthegenericnatureofthealgorithm;byignoringsomeofthisimportantinformationwegain
usefulinsightsintotheperformancelimitsofa colouralignmentalgorithmthatworksfromthecolour
histogramalone,thishelpsunderstandthelimitsofagenericapproachtocolouralignment.
Acknowledgements 4
Acknowledgements
Thisthesisisdedicatedtomywife,MariaTeresaAguilera-Peon
IembarkedontheEngDprogramasamaturestudent,itfeltlikearisktoleavepaidemployment
in a respectablejob atthe BBC Research andDevelopmentfacility inKingswoodSurrey. Friendsask
meifIamgladthatIundertooktheprogram,ItellthemthatIcan’timaginegivingupthelessonsthatI
havelearntandthenewperspectivesIhold. TheEngDperiodhasquitesimplybeenthebiggestperiod
ofpersonalgrowthandchangeinmylife,duringthistimeImarriedanamazingwomanandwehavea
beautifuldaughtertogether,Daniela. We arecurrentlyexpectingoursecondchildandIhaveaccepted
an exciting position in a greattechnologycompanyin the United States. This thesis owes a debtto a
numberofpeoplewhohaveeitherdirectlyor indirectlyhelped. I brieflysummarizetheircontribution
here.
My supervisor Dr. Daniel Alexander is a mentor and friend. His constant corrective guidance
hasbeenakeyfactorinovercomingmanyoftheunforseenobstaclesthatoccurredduringtheresearch
and writing up periods of this thesis. His willingness to meet regularly and freely have made signifi-
cantchallengessurmountable,theexplorationofmanyblindalleyswashaltedearlyandnewdirections
weredefinedduringthesemeetings. Iwouldrecommendanyprospectivedoctoralstudentwhogetsthe
opportunitytoworkwithDannytograbtheopportunitywithbothhands.
This thesis was based around a problem at Buhler Sortex as defined by Dr. Gabriel Hamid. Dr.
Hamidhasgivenhistimefreelyandhasmadehimselfavailabletoexplainanddefineproblemsrelevant
to the industrially related portions of this thesis. I am grateful for the joint grant received from the
ESPRCandBuhlerSortex. MysecondsupervisorDr. LewisGriffinandDr. SimonPrincebothmade
invaluablecontributionsduringthefirstandsecondyearreviews,theirsuggestionsforapproachingthe
research process were insightful and timely. Members of the UCL computer vision laboratorieshave
provided a useful sounding board and source of ideas and references. Dr. Martin Lillholm deserves
specialmention,hejoinedthelabandIwasluckyenoughtositnexttohim. Hisdeepknowledgeand
sharp insighthave helped immensely; Martin is one of those rare guyswho seems to have the correct
answer to any questionyouask him. Other memberssuch as Tony Shepherd,Alistair Mooreand Bill
Crumallrespondedenthusiasticallytomyinterrogationsoftheirknowledgeofvarioustechniques.Other
peopleatUCLhaveprovedinvaluablyhelpfulattimes,RomyBeattieinparticularisastar. Sheresolved
numerousEngDadministrativeissuesthatweregettingpushedbetweenpeople’sdesksuntilshegoton
thecase.
Acknowledgements 5
Nobodyhasbeenmoreconnectedtothehighsandlowsofthedoctoralprocessthanmywife. She
hasworkedhard,ofteninhighpressuresituationstosupportourfamilythroughthistime. Withouther
hard work and support, it would have not been financially possible to finish this thesis. Maria-Teresa
has constantly encouragedme and never considered that we give up, even when precarioussituations
arose. Her love and sacrifice have been a driving force to finish this thesis. Daniela has taught me
somethingeverystepoftheway,dealingwithdoctors,hospitalvisitsandweeksoffathomeduetobugs
andsicknesswhilemanagingEngDdeadlineshastaughtmepatienceandcalm. Seeingherjoyfulsmile
haskeptmefocussedonthebigpicture.
My parents have been fantastic, they have supported and encouraged me every step of the way.
Becominga parentmyself hasgivenme an evengreaterappreciationofthe continuouseffort,support
andlovethattheyhavegivenme throughoutmyentire life. I will neverforgetthe encouragingwords
ofmymother,whohasalwaystoldmefromayoungagethat“youcanachieveanythingyouputyour
mind to”. My father is significantly responsible for getting me interested in science, technology and
computers. Hisloveoftinkeringwiththingsandbrowsingathousanddifferentsubjectsrubbedoffon
me. I remember our first computer, a Sinclair Spectrum, 48K with rubber keys and a cassette tape to
loadprograms-justgreat.
Contents 6
Contents
1 Introduction 20
1.1 ColourconsistencyinComputerVision . . . . . . . . . . . . . . . . . . . . . . . . . . 20
1.2 Colourconsistencyinthefoodprocessingindustry . . . . . . . . . . . . . . . . . . . . 22
1.3 Theautomatichistogramalignmentproblem . . . . . . . . . . . . . . . . . . . . . . . . 23
1.4 GoalsandContributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
1.5 ThesisPlan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2 LiteratureReview 26
2.1 Colourfundamentals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.1.1 Thethreeelementsofcolour . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.1.2 Asimplecolourmodel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.1.3 Colourambiguities:MetamerismandColourinconsistency . . . . . . . . . . . 32
2.1.4 Physicalbasisforinconsistency . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.1.5 ColourSpaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.2 ColourHistograms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.4 Ataxonomyofcolourinconsistencycorrectionmethods . . . . . . . . . . . . . . . . . 40
2.5 TransformationMethods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
2.5.1 Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
2.5.2 MethodsForComputingTheTransformations . . . . . . . . . . . . . . . . . . 50
2.6 Motivation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3 Featurebasedhistogramalignment 55
3.1 Featurebasedhistogramalignmentalgorithm . . . . . . . . . . . . . . . . . . . . . . . 55
3.1.1 FeatureDetection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.1.2 FeatureMatching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.1.3 FeatureAlignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.1.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.2 QualitativeEvaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.2.1 1DFBHA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.2.2 2DFBHA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Contents 7
3.2.3 3DFBHA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.2.4 Shapepreservinghistogramtransformations. . . . . . . . . . . . . . . . . . . . 63
3.3 SummaryConclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4 AnimagedatabasefortestingRGBcolouralignment 73
4.1 Databasedesign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
4.1.1 Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
4.1.2 Captureconditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
4.1.3 Objectlabeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.1.4 Imagevariationsets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.2 ExistingColourDatasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.3 Histogramalignmentmetrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4.3.1 BintoBinMeasures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4.3.2 Cross-BinMeasures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
4.3.3 Manuallydefinedmetrics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
4.3.4 Empiricalcomparisonofmetrics . . . . . . . . . . . . . . . . . . . . . . . . . . 85
4.3.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
4.4 QuantitativeevaluationofRGBcolouralignment . . . . . . . . . . . . . . . . . . . . . 92
4.4.1 Experiment1:FeatureBasedAlignmentHypothesis . . . . . . . . . . . . . . . 93
4.4.2 Experiment2:ClosestEuclideanFeatureMatchHypothesis . . . . . . . . . . . 105
4.4.3 Experiment3:FBHAcomparison . . . . . . . . . . . . . . . . . . . . . . . . . 105
4.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
5 ApplicationoffeaturebasedhistogramalignmenttoBuhlerSortexmachines 125
5.1 TheBuhlerSortexZ-series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
5.1.1 Histogramalignmentproblem . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
5.1.2 CurrentApproachandCommercialConfidence . . . . . . . . . . . . . . . . . . 127
5.1.3 Motivationforimprovedcalibrationroutines . . . . . . . . . . . . . . . . . . . 128
5.2 Productcolourinconsistencyreduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
5.2.1 Non-segmentationalignmentmethods . . . . . . . . . . . . . . . . . . . . . . 130
5.2.2 Segmentationbasedalignmentmethods . . . . . . . . . . . . . . . . . . . . . . 140
5.3 ExperimentalEvaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
5.3.1 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
5.3.2 Qualitativeevaluationoffeaturedetectionandassociation . . . . . . . . . . . . 153
5.3.3 Quantitativeevaluationofcolourinconsistencycorrections . . . . . . . . . . . . 154
5.4 SummaryConclusionsandDiscussion . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
6 ConclusionsandFurtherWork 164
6.1 Commercialrelevanceandcontributions . . . . . . . . . . . . . . . . . . . . . . . . . . 164
6.1.1 Directapplicabilityoffindings . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
Contents 8
6.1.2 Futureapplications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
6.2 FBHA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
6.3 Existingmethods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
6.4 Furtherwork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
7 SummaryofAchievements 170
8 Glossary 172
9 Appendix 173
9.1 ThePseudoinverse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
9.2 OrderingResults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
ListofFigures 9
List of Figures
1.1 Atomatowithablackrottenpatchontheleftandahealthierlookingtomatoontheright. 20
1.2 Two scenes captured by an Aibo robotthat have been segmented using the method of
Rofer [1]. 1.2(a) shows a scene and 1.2(b) shows a segmented version of the image.
Each unique colour in 1.2(b) indicates a unique class label. 1.2(c) and 1.2(d) show a
differentsceneanditssegmentedversion.Inthisimagethescaleoftheballandrobotis
largerwhencomparedto1.2(a). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
1.3 Input,AcceptandRejectSamplesforWhiteLongGrainParboiledRiceusingaBuhler
SortexZ-seriesmachine. Picture ccopyrightBuhlerSortexLtd,2008. Reprintedwith
(cid:13)
permission. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.1 Theelectromagneticspectrumdescribesradiationatdifferentwavelengths.Visiblelight
occupiesasmallpartofthespectrumbetween380nmand780nm. PicturebyL.Keiner
(Reprintedwithpermission) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.2 Spectral power distributions from GE Lighting [2]. 2.2(a) shows the spectral power
distributionofoutdoordaylightand2.2(b)showsthespectralpowerdistributionunder
Spx35florescentlighting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.3 Schematicviewofthehumaneye(Source:CreativeCommons[3]). . . . . . . . . . . . 30
2.4 Colour matching functionsfor the RGB and the CIE X,Y,Z primaries shown in 2.4(a)
and2.4(b)respectively. RedisusedtoindicateRandX,greenindicatesGandY,blue
indicates B and Z primaries. The curves indicate the relative amounts of the primary
coloursneededtomatchatesttargetcolourwiththeindicatedwavelength. . . . . . . . 31
2.5 Thepinholecamera(Source:CreativeCommons[4]). . . . . . . . . . . . . . . . . . . . 32
2.6 Two different arrangements for sampling different spectral ranges. 2.6(a) shows the
single chipBayerpatternarraywhereared, greenorbluefilter elementis placedover
eachpixel.2.6(b)showsamulti-chiparrangementthatsplitslightwithaprismanduses
threeseparateimagingchipsforthered,greenandbluebands.Imagesreproducedfrom
Dalsa[5]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.7 Spectral reflectance curves comparing two different sets of objects. 2.7(a) shows the
reflectivityofBarleyseeds(red)andBark(green). 2.7(b)showsthereflectivityofRed-
wood(red)andabrownpaperbag(green).AlldataisfromGlassner[6]. . . . . . . . . 39
2.8 Introducedtaxonomyofcolourinconsistencycorrectionmethods. . . . . . . . . . . . . 42
ListofFigures 10
2.9 AMacbethchart,commonlyusedforcolourcalibrationtasks. . . . . . . . . . . . . . . 48
3.1 Exampleofthelocalmaximainaonedimensionalhistogram.3.1(a)showsahistogram
obtained from the green channel of an image of skittles in the image database. All
localmaximaareshownasreddots,maximainlowlevelnoiseishighlightedas1)and
multiplelocalmaximaonaclusterarehighlightedas2).3.1(b)showsazoomedviewof
thelocalmaximaintheclusterlabelled2). . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.2 3.2(a)showsarepresentationofthescalespaceofthehistograminFigure3.1(a);hori-
zontalslicesindicatehistogramsblurredatincreasingscalemovingfrombottomtotop,
denserregionsofthescalespacearerenderedclosertowhiteandlocalmaximaateach
scale aredrawnascircles. The maximaformpathsacrossscales, with pathsfromless
significantpeaksendingearlierinthe scale space. 3.2(b)showsthe persistentmaxima
usingT=20. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.3 Thesourceimagein3.3(a)andtargetimagein3.3(b)exhibitcolourinconsistency.3.3(c)
showsthecoloursofthesourceimagetransformedusing1DFBHAwithalineartrans-
formineachchannel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.4 Source and target histograms are shown as overlayed line plots in the top portion of
eachsub-figure.Theredplotsshowthetargethistogramintheredchannel3.4(a),green
channel3.4(b)and bluechannel3.4(c). The correspondingblue plotsshow the source
histograms. Detected features are marked on the source and target histograms with a
star. The bottom portions of each sub-plot show an exploded view of the source and
targethistogramsandthematchedfeatures. . . . . . . . . . . . . . . . . . . . . . . . . 68
3.5 Exploded view of the histograms of the corrected data plotted with the solid line and
thetargethistogramplottedwiththedottedline. Thealignedfeaturesareshownusinga
linetoconnectthealignedfeatureandtargetfeature.Subfigures3.5(a),3.5(b)and3.5(c)
showthered,greenandbluechannelsrespectively. . . . . . . . . . . . . . . . . . . . . 69
3.6 Imageofplasticskittles. Figure3.6(a)showsthesourceimageandfigure3.6(b)shows
thetargetimagewherearedlightmodifiestheappearanceoftheskittles. Figure3.6(c)
showsthemodifiedsourceimagewhere2DFBHA isusedintheRG channelsand1D
FBHAisusedinthebluechannel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.7 Sub-figure3.7(a)showsthesquarerootofRGhistogramforthesourceimageinFigure
3.6(a) and sub-figure 3.7(b) shows the square root of the RG histogram for the target
image in Figure 3.6(b). Taking the square root of the histogramsallows the shapes of
localfeaturesatdifferentscalestobeobservedmoreeasilyonasingleplot. . . . . . . . 71