Table Of ContentSpringer Tracts in Advanced Robotics
Volume 75
Editors:BrunoSiciliano·OussamaKhatib·FransGroen
Javier Civera, Andrew J. Davison,
and José María Martínez Montiel
Structure from Motion
Using the Extended Kalman
Filter
ABC
ProfessorBrunoSiciliano,DipartimentodiInformaticaeSistemistica,UniversitàdiNapoliFedericoII,
ViaClaudio21,80125Napoli,Italy,E-mail:[email protected]
ProfessorOussamaKhatib,ArtificialIntelligenceLaboratory,DepartmentofComputerScience,
StanfordUniversity,Stanford,CA94305-9010,USA,E-mail:[email protected]
ProfessorFransGroen,DepartmentofComputerScience,UniversiteitvanAmsterdam,Kruislaan403,
1098SJAmsterdam,TheNetherlands,E-mail:[email protected]
Authors
Dr.JavierCivera Dr.AndrewJ.Davison
UniversidaddeZaragoza ImperialCollegeLondon
InstitutoUniversitariodeInvestigaciónen DepartmentofComputing
IngenieríadeAragón(I3A) 180Queen’sGate
EdificioAdaByron SW72AZLondon,UK
MaríadeLuna,1,50018Zaragoza,Spain E-mail:[email protected]
E-mail:[email protected]
Dr.JoséMaríaMartínezMontiel
UniversidaddeZaragoza
InstitutoUniversitariodeInvestigaciónen
IngenieríadeAragón(I3A)
EdificioAdaByron
MaríadeLuna,1,50018Zaragoza,Spain
E-mail:[email protected]
ISBN978-3-642-24833-7 e-ISBN978-3-642-24834-4
DOI 10.1007/978-3-642-24834-4
SpringerTractsinAdvancedRobotics ISSN1610-7438
LibraryofCongressControlNumber:2011940008
(cid:2)c 2012Springer-VerlagBerlinHeidelberg
Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthematerialis
concerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,broadcasting,
reproductiononmicrofilmorinanyotherway,andstorageindatabanks.Duplicationofthispublication
orpartsthereofispermittedonlyundertheprovisionsoftheGermanCopyrightLawofSeptember 9,
1965,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.Violations
areliableforprosecutionundertheGermanCopyrightLaw.
Theuseofgeneral descriptive names,registered names,trademarks, etc. inthis publication does not
imply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevantprotective
lawsandregulationsandthereforefreeforgeneraluse.
Typeset&CoverDesign:ScientificPublishingServicesPvt.Ltd.,Chennai,India.
Printedonacid-freepaper
543210
springer.com
EditorialAdvisoryBoard
OliverBrock,TUBerlin,Germany
HermanBruyninckx,KULeuven,Belgium
RajaChatila,LAAS,France
HenrikChristensen,GeorgiaTech,USA
PeterCorke,QueenslandUniv.Technology,Australia
PaoloDario,ScuolaS.AnnaPisa,Italy
RüdigerDillmann,Univ.Karlsruhe,Germany
KenGoldberg,UCBerkeley,USA
JohnHollerbach,Univ.Utah,USA
MakotoKaneko,OsakaUniv.,Japan
LydiaKavraki,RiceUniv.,USA
VijayKumar,Univ.Pennsylvania,USA
SukhanLee,SungkyunkwanUniv.,Korea
FrankPark,SeoulNationalUniv.,Korea
TimSalcudean,Univ.BritishColumbia,Canada
RolandSiegwart,ETHZurich,Switzerland
GauravSukhatme,Univ.SouthernCalifornia,USA
SebastianThrun,StanfordUniv.,USA
YangshengXu,ChineseUniv.HongKong,PRC
Shin’ichiYuta,TsukubaUniv.,Japan
N
SdeTrAtRhe(aSupsrpinicgeesrTofraEcUtsRinOANd(vEaunrcoepdeRanobRootbicost)ichsasRbeeseeanrpcrhoNmeottwedorukn)- RERuersoOepaeracBnhROO************TICS
NetworkU
E
To Mar´ıa,Mar´ıaA´ngeles,PedroJose´ and
Sergio.
To Lourdes, Rafael,Blanca andAdela.
To Jose´ Mar´ıaand Olga.To Virginia,Elena
andMikel
Foreword
Robotics is undergoing a major transformation in scope and dimension. From a
largely dominant industrial focus, robotics is rapidly expanding into human envi-
ronmentsandvigorouslyengagedinitsnewchallenges.Interactingwith,assisting,
serving, and exploring with humans, the emerging robots will increasingly touch
peopleandtheirlives.
Beyondits impacton physicalrobots,the bodyof knowledgeroboticshaspro-
duced is revealing a much wider range of applications reaching across diverse
research areas and scientific disciplines, such as: biomechanics, haptics, neuro-
sciences,virtualsimulation,animation,surgery,andsensornetworksamongothers.
Inreturn,thechallengesofthenewemergingareasareprovinganabundantsource
ofstimulationandinsightsforthefieldofrobotics.Itisindeedattheintersectionof
disciplinesthatthemoststrikingadvanceshappen.
TheSpringerTractsin AdvancedRobotics(STAR)isdevotedtobringingtothe
research community the latest advances in the robotics field on the basis of their
significance and quality. Through a wide and timely dissemination of critical re-
searchdevelopmentsin robotics,ourobjectivewith this seriesis to promotemore
exchanges and collaborations among the researchers in the community and con-
tributetofurtheradvancementsinthisrapidlygrowingfield.
ThemonographwrittenbyJavierCivera,AndrewJ.DavisonandJ.M.M.Montiel
isdevotedtothewidefieldofcomputervisionforrobotics,whichhasbeenreceiv-
ing a great deal of attention by the research community throughoutthe last three
decades.Thecontentsarefocusedonthedevelopmentofmodelsandmethodsfor
sequentialStructurefromMotionormonocularSLAM,wheretheprojectivenature
of the camera is fitted undera Bayesian filtering framework.Pointfeature model,
efficient and robust correspondences search, model selection for degenerate con-
figurations and internal self-calibration are essential to the theoretical framework
underlyingthesequentialestimationalgorithmsdescribedinthebook.
X Foreword
Sparsepoint-based3Dreconstructionandmotionestimationfromanimagese-
quenceareachievedassumingnoknowledgeoverthescenenortheinternalcamera
calibrationparameters,makingtheresultingsystemtoeffectivelyworkinreal-time
atvideorateintypicalroom-sizedscenarios.
STAR is proud to welcome yet another volume in the series dedicated to the
popularareaofSLAM!
Naples,Italy BrunoSiciliano
August2011 STAREditor
Preface
The book you have in your hands is the result of the joint research of the au-
thors from 2006 to 2009, that led to Javier Civera’s PhD thesis. The contribu-
tions on it were first published in several international conferences and journals;
specifically and in chronological order [Montieletal.2006, Civeraetal.2007a,
Civeraetal.2007b, Civeraetal.2008a, Civeraetal.2008b, Civeraetal.2009b,
Civeraetal.2009a, Civeraetal.2009c, Civeraetal.2010]. This book provides a
unifiedviewoftheresearchspreadonthosepapers,presentingacompleteandro-
bustsystemaimedtosequential3Dvisioninreal-time.
Alongtheyears,manypeoplehavecontributedinoneoranothermannertothe
resultspresentedhere.Firstofall,weareverygratefultoourcoauthorsinthetopics
relatedwiththisbookO´scarG.Grasa,DianaR.BuenoandJuanA.Magallo´n.Gabe
Sibley andJoan Sola` reviewedthe first versionofthe PhD thesismanuscript.The
juryofJavierCivera’sPhDdefence;WolframBurgard,J.D.Tardo´s,PatricJensfelt,
MiguelA´ngelSalichsandV´ıctorMun˜oz,contributedtothisfinalversionwithvery
insightfulcommentsandannotations.TheexperteyesofAntonioAgudo,O´scarG.
Grasa,LinaPazyPedroPinie´salsoreviewedthefinalversionofthisbookandcon-
tibutedtopolishsomediscussionsandtofindthelatestandmoredifficultbugs.The
authors are also extremely grateful to their respective research groups: the Robot
VisionGroupatImperialCollegeinLondonandtheRobotics,PerceptionandReal-
TimeGroupintheI3A(Arago´nInstituteonEngineeringResearch)intheUniversity
ofZaragozainSpain.Hundredsofrelevantconversations,thestimulatingresearch
atmosphereandthededicationatbothgroupsarefertilebreedinggroundsthathave
played an essential role in our research. An exhaustivelist of names would be an
unfeasibletask,butstillwewouldliketoexpressourgratitudetosomeofourcol-
leaguesforveryinterestingdiscussions:Jose´ Neira,PedroPinie´s,LinaPaz,Rube´n
Mart´ınez-Cant´ın,LuisMontesano,AlejandroMosteo, Ana Cristina Murillo,In˜aki
Ran˜o, Antonio Agudo, Margarita Chli, Hauke Strasdat, Ian Reid, Brian Williams
andCyrillStachniss.Lastbutnotleast,weareindebtwithourrespectivefamilies,
fortheirconstantsupport.
The research results in this book were financially supported by the following
sources:TheSpanishGovermentresearchgrantsDPI2003-07986,DPI2006-13578,
XII Preface
DPI2009-07130andPR2007-0427;regionalgrantDGA(CONAI+D)-CAIIT12-06;
theEuropeanUnionprojectsRAWSEEDSFP6-045144andRoboEarthFP7-248942;
andA.J.Davison’sERCStartingGrant210346.
ZaragozaandLondon JavierCivera
July2011 AndrewJ.Davison
J.M.M.Montiel
Contents
1 Introduction................................................... 1
1.1 StructurefromMotionandMonocularSLAM................... 3
1.1.1 StructurefromMotion................................ 3
1.1.2 MonocularSLAM ................................... 3
1.1.3 StructureoftheProblem .............................. 4
1.2 Background ............................................... 8
1.3 OutlineoftheBook......................................... 9
2 PointsatInfinity:MosaicsUsingtheExtendedKalmanFilter ....... 13
2.1 Introduction ............................................... 13
2.1.1 PointsatInfinity..................................... 13
2.1.2 Real-TimeEKF-BasedMosaicing ...................... 14
2.2 RelatedWork.............................................. 15
2.2.1 SLAM ............................................. 15
2.2.2 Off-LineSFMvs.EKFSLAM ......................... 16
2.2.3 ImageMosaicing .................................... 17
2.3 GeometricalModeling ...................................... 19
2.3.1 FeatureModel....................................... 19
2.3.2 CameraMotionModel................................ 19
2.3.3 MeasurementModel ................................. 20
2.4 SimultaneousLocalizationandMapping ....................... 21
2.4.1 Matching........................................... 22
2.4.2 StateInitialization ................................... 23
2.4.3 FeatureInitializationandDeletion ...................... 23
2.5 MeshingandMosaicing ..................................... 24
2.5.1 UpdatingtheMesh................................... 24
2.5.2 TileTextureMapping................................. 25
2.6 ExperimentalResults ....................................... 26
◦
2.6.1 360 PanandCyclotorsion ............................ 26
2.6.2 Real-TimeMosaicBuilding ........................... 29
2.6.3 ProcessingTime ..................................... 30
2.7 Discussion ................................................ 31
