Table Of ContentSpringer Tracts in Advanced Robotics 122
Emilio Garcia-Fidalgo
Alberto Ortiz
Methods for
Appearance-based
Loop Closure Detection
Applications to Topological Mapping
and Image Mosaicking
Springer Tracts in Advanced Robotics 122
Editors
Prof. Bruno Siciliano Prof. Oussama Khatib
Dipartimento di Ingegneria Elettrica Artificial Intelligence Laboratory
e Tecnologie dell’Informazione Department of Computer Science
Università degli Studi di Napoli Stanford University
Federico II Stanford, CA 94305-9010
Via Claudio 21, 80125 Napoli USA
Italy E-mail: [email protected]
E-mail: [email protected]
Editorial Advisory Board
Nancy Amato, Texas A & M, USA
Oliver Brock, TU Berlin, Germany
Herman Bruyninckx, KU Leuven, Belgium
Wolfram Burgard, Univ. Freiburg, Germany
Raja Chatila, ISIR - UPMC & CNRS, France
Francois Chaumette, INRIA Rennes - Bretagne Atlantique, France
Wan Kyun Chung, POSTECH, Korea
Peter Corke, Queensland Univ. Technology, Australia
Paolo Dario, Scuola S. Anna Pisa, Italy
Alessandro De Luca, Sapienza Univ. Rome, Italy
Rüdiger Dillmann, Univ. Karlsruhe, Germany
Ken Goldberg, UC Berkeley, USA
John Hollerbach, Univ. Utah, USA
Lydia Kavraki, Rice Univ., USA
Vijay Kumar, Univ. Pennsylvania, USA
Bradley Nelson, ETH Zürich, Switzerland
Frank Park, Seoul National Univ., Korea
Tim Salcudean, Univ. British Columbia, Canada
Roland Siegwart, ETH Zurich, Switzerland
Gaurav Sukhatme, Univ. Southern California, USA
More information about this series at http://www.springer.com/series/5208
Emilio Garcia-Fidalgo Alberto Ortiz
(cid:129)
Methods for
Appearance-based
Loop Closure Detection
Applications to Topological Mapping
and Image Mosaicking
123
Emilio Garcia-Fidalgo AlbertoOrtiz
Department ofMathematics Department ofMathematics
andComputer Science andComputer Science
University of the BalearicIslands University of the BalearicIslands
Palma deMallorca Palma deMallorca
Spain Spain
ISSN 1610-7438 ISSN 1610-742X (electronic)
SpringerTracts inAdvanced Robotics
ISBN978-3-319-75992-0 ISBN978-3-319-75993-7 (eBook)
https://doi.org/10.1007/978-3-319-75993-7
LibraryofCongressControlNumber:2018932198
©SpringerInternationalPublishingAG2018
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
orinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodologynowknownorhereafterdeveloped.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom
therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authorsortheeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinor
for any errors or omissions that may have been made. The publisher remains neutral with regard to
jurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations.
Printedonacid-freepaper
ThisSpringerimprintispublishedbySpringerNature
TheregisteredcompanyisSpringerInternationalPublishingAG
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
To my father Valentín. I hope you read these
lines, wherever you are.
Foreword
Robotics is undergoing a major transformation in scope and dimension. From a
largely dominant industrial focus, robotics is rapidly expanding into human envi-
ronmentsandvigorouslyengagedinitsnewchallenges.Interactingwith,assisting,
serving, and exploring with humans, the emerging robots will increasingly touch
people and their lives.
Beyond its impact on physical robots, the body of knowledge robotics has
produced is revealing a much wider range of applications reaching across diverse
research areas and scientific disciplines, such as biomechanics, haptics, neuro-
sciences,virtualsimulation,animation,surgery,andsensornetworksamongothers.
Inreturn,thechallengesofthenewemergingareasareprovinganabundantsource
ofstimulationandinsightsforthefieldofrobotics.Itisindeedattheintersectionof
disciplines that the most striking advances happen.
TheSpringer TractsinAdvancedRobotics(STAR)isdevotedtobringingtothe
research community the latest advances in the robotics field on the basis of their
significance and quality. Through a wide and timely dissemination of critical
researchdevelopmentsinrobotics,ourobjectivewiththisseriesistopromotemore
exchanges and collaborations among the researchers in the community and con-
tribute to further advancements in this rapidly growing field.
ThemonographbyE.Garcia-FidalgoandA.Ortizisbasedonthefirstauthor’s
Ph.D. thesis work. It deals with the mapping and localization problem for autono-
mous mobile robot navigation in cluttered environments, adopting methods for
appearance-basedloopclosuredetection.Noveltopologicalmappingalgorithmsare
proposedandexperimentallytestedonstate-of-the-artdatasets.Afastmulti-threaded
image mosaicking algorithm is also proposed as a particular case of a topological
map,leadingtosuccessfulresultsformotionestimation.
STAR is proud to welcome yet another volume in the series dedicated to the
popular area of SLAM!
Naples, Italy Bruno Siciliano
December 2017 STAR Editor
vii
Preface
Mapping and localization are two essential processes in autonomous mobile
roboticssincetheyarethebasisofotherhigherlevelandmorecomplextasks,such
as obstacle avoidance or path planning. Mapping is the process through which a
robot builds its own representation of the environment when a map of the envi-
ronment is not available. In this context, there exist mainly two types of maps:
metric and topological. While metric maps represent the world as accurate as
possible with regard to a global coordinate system, topological maps represent the
environment in an abstract manner by means of a graph, which implies several
benefits in front of the classic metric approaches.
Regardless of the type of map to be built, any sensor used to perceive the
environment presents an unavoidable noise that should be taken into account in
order to not compromise the accuracy of the resulting map and the localization
process.For this reason,an additional input isrequiredto correct theinefficiencies
producedinthemap bythisnoise.Due tothisreason,mappingalgorithms usually
relyonloopclosuredetectiontechniques,whichentailsthecorrectidentificationof
previously seen places to reduce the uncertainty of the resulting maps.
Inthelastdecades,therehasbeenasignificantincreaseinthenumberofvisual
solutions to solve the loop closure detection problem, specially motivated by the
low cost of cameras, the richness of the sensor data provided, and the existence of
morepowerfulcomputers.Theproblemofidentifyingifthecurrentreceivedimage
hasbeentakenfromanalreadyvisitedplaceisusuallyknownasappearance-based
loopclosuredetection.Ingeneral,giventhatthisproblemdealswithsetsofimages,
any technique proposed to solve it should be devised to efficiently manage a huge
volume of visual information, which normally grows along with the map.
This book deals with the problem of generating topological maps of the envi-
ronment using efficient appearance-based loop closure detection techniques. Since
the quality of a visual loop closure detection algorithm is related to the image
descriptionmethodanditsabilitytoindexpreviouslyseenimages,severalmethods
for loop closure detection adopting different approaches are developed and asses-
sed. Then, these methods are used as basic components in three novel topological
mappingalgorithms.Theresultsobtainedindicatethatthesolutionsproposedattain
ix
x Preface
a better performance than several state-of-the-art approaches. To conclude, given
that loop closure detection is also a key component in other research areas, a
multi-threaded image mosaicking algorithm is proposed, which can be seen as a
particular case of a topological map. This approach makes use of one of the loop
closure detection techniques previously introduced in the book in order to find
overlapping pairs between images and finally obtain seamless mosaics of different
environments in a reasonable amount of time.
The work reported in this book was supported by the European Social Fund
through Ph.D. scholarship FPI11-43123621R (Conselleria d'Educació, Cultura i
Universitats, GoverndelesIlles Balears),byFP7projectsMINOAS(GA 233715)
and INCASS (GA 605200), and by H2020 project ROBINS (GA 779776).
Palma de Mallorca, Spain Emilio Garcia-Fidalgo
December 2017 Alberto Ortiz
Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.1 Basic Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.2 Mobile Robot Navigation. . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.3 Mapping, Localization and SLAM . . . . . . . . . . . . . . . . . . 4
1.1.4 Loop Closure Detection . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Book Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1 Topological Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 Appearance-Based Loop Closure Detection . . . . . . . . . . . . . . . . . 11
2.2.1 Image Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2.2 Image Indexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3 Literature Review. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.2 Methods Based on Global Descriptors . . . . . . . . . . . . . . . . . . . . . 29
3.2.1 Histograms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.2.2 The Gist Descriptor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.2.3 Vertical Regions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.2.4 Discrete Fourier Transform. . . . . . . . . . . . . . . . . . . . . . . . 33
3.2.5 Biologically-Inspired Approaches . . . . . . . . . . . . . . . . . . . 34
3.2.6 Other Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.3 Methods Based on Local Features . . . . . . . . . . . . . . . . . . . . . . . . 36
3.4 Methods Based on Bag-of-Words Schemes . . . . . . . . . . . . . . . . . 41
3.4.1 Offline Visual Vocabulary Approaches . . . . . . . . . . . . . . . 41
3.4.2 Online Visual Vocabulary Approaches . . . . . . . . . . . . . . . 45
xi
