Relational Affordances and their Applications

218 Pages·2015·14.9 MB·English

by  Bogdan Moldovan

ARENBERG DOCTORAL SCHOOL
FACULTY OF ENGINEERING SCIENCE
Relational Affordances
and their Applications
Bogdan Moldovan
Supervisor: Dissertation presented in partial
Prof. dr. Luc De Raedt fulfillment of the requirements for the
degree of PhD in Engineering
March 2015

RELATIONAL AFFORDANCES
AND THEIR APPLICATIONS
Bogdan MOLDOVAN
Supervisor:
Prof. dr. Luc De Raedt
Members of the
Examination committee:
Prof. dr. ir. Paul Van Houtte, chair
Prof. dr. Luc De Raedt, supervisor
Prof. dr. ir. Herman Bruyninckx
Prof. dr. ir. Tinne Tuytelaars
Prof. dr. Claude Sammut
(University of New South Wales, Australia)
Prof. dr. José Santos-Victor Dissertation presented in
(Instituto Superior Técnico, Portugal) partial fulfillment of the
Prof. dr. Martijn van Otterlo requirements for the
(Nijmegen, The Netherlands) degree of PhD in Engineering
March 2015
©2015KULeuven–FACULTYOFENGINEERINGSCIENCE
Uitgegevenineigenbeheer,BogdanMoldovan,Celestijnenlaan200A,B-3001Heverlee,Belgium(Belgium)
Allerechtenvoorbehouden. Nietsuitdezeuitgavemagwordenvermenigvuldigden/ofopenbaargemaaktworden
door middel van druk, fotokopie, microfilm, elektronisch of op welke andere wijze ook zonder voorafgaande
schriftelijketoestemmingvandeuitgever.
Allrightsreserved. Nopartofthepublicationmaybereproducedinanyformbyprint,photoprint,microfilm,
electronicoranyothermeanswithoutwrittenpermissionfromthepublisher.
Abstract
The concept of affordances is used in robotics to model action opportunities of
a robot on objects in the environment, and as such they play a role in building
basic cognitive capabilities of the robot. Affordances generally model isolated
objects in the environment and capture the interdependencies between object
properties, executed actions on those objects, and the effects of those respective
actions. However, many real-world scenarios involve configurations of multiple
objects that interact with each other when manipulated.
This thesis proposes the use of recent advances in statistical relational learning
to build relational affordance models, where the (spatial) relations between
the different objects, such as relative distances between pairs of objects, are
taken into account. Two-object interaction models are learned from the robot
interacting with the objects in the world in a behavioural babbling stage.
These models can then be employed in situations with arbitrary numbers of
objects. This model thus generalizes over objects and can deal effectively with
uncertainty.
Toshowtherelevanceofrelationalaffordancemodels, wefurtherinvestigatethe
use of relational affordances in three different applications. These applications
are illustrated by the use of the iCub and PR2 robots in simulation, and the
use of the iCub robot in a real setting.
Firstly,weuseSRLmethodstocreatearelationalaffordancemodelfortwo-arm
robotactions,forsettingswherethesecanbeapproximatedbyacombinationof
the two single-arm actions composing them. The arms may act simultaneously
or sequentially, and the robot is given background knowledge about possible
actions in its environment. SRL is used here to generalise and build a higher-
level model for a set of two-arm actions settings in a household environment,
by using symmetries of the arms to model effects of actions executed with the
arm not involved in the behavioural babbling stage, and background knowledge
about two-arm actions.
i
ii ABSTRACT
Secondly, we employ relational affordance models in a planning setting with
high-level goals which are specified by (spatial) relations between the objects.
The relational affordance model can be used to define a state transition model
in a table-top setting, and we provide a planning algorithm to be used to infer
thenextbestpossibleactionfortherobottoexecutetowardsreachingthegiven
goal. The robot is then given a series of planning tasks of increasing difficulty
in a table-top environment, which it needs to solve.
Finally,totackleanoccludedobjectsearchtask,weusetheconceptofrelational
affordances to search for any object affording a given action in a kitchen
environment with many shelves. Multiple object types can afford the action,
and each type allows for many different objects with size and shape modelled
by probability distributions, thus relaxing some of the previous assumptions in
the field. Moreover, we allow for stacked objects, a more realistic modelling of
objects in shelves, introducing more complex object spatial relations.
Beknopte samenvatting
In de robotica worden affordances gebruikt voor het modelleren van mogelijke
acties die een robot kan uitvoeren met een bepaald object in de omgeving. Ze
spelen daarom een belangrijke rol in de opbouw van de primaire cognitieve
vermogens van de robot. Over het algemeen modelleren affordances geïsoleerde
objecten in de omgeving en beschrijven ze de relatie tussen de eigenschappen
vaneenobject,actiesdieophetobjectuitgevoerdkunnenworden,endeeffecten
van deze acties. In realiteit treden echter vaak scenario’s op waarbij meerdere
objecten met elkaar interageren.
Deze thesis stelt het gebruik voor van recente ontwikkelingen in statistisch
relationeel leren (SRL) voor het opbouwen van relationele affordance modellen,
waarbij rekening wordt gehouden met de (ruimtelijke) relaties tussen de
verschillende objecten, zoals de relatieve afstanden tussen objectparen.
Interactiemodellen voor twee objecten worden geleerd uit interactie van de
robot met de objecten in de wereld tijdens een gedragsbrabbelfase. Deze
modellen kunnen dan worden toegepast in situaties met een willekeurig aantal
objecten. Dit model generaliseert dus over objecten en kan effectief omgaan
met onzekerheden.
Omderelevantievanrelationeleaffordancemodellenaantetonen,onderzochten
we verder het gebruik van relationele affordances in drie verschillende
toepassingen. Deze toepassingen worden geïllustreerd door het gebruik van de
iCub en PR2 robots in simulatie, en het gebruik van de iCub robot in een echte
setting.
Ten eerste, gebruiken we SRL methoden om een relationeel affordance model
voor twee-armige robotacties te maken, in situaties waar deze kunnen worden
benaderd door een combinatie van twee enkelarmige acties. De armen kunnen
gelijktijdig gebruikt worden of in sequentie, en de robot heeft toegang tot
achtergrondkennis over mogelijke acties in zijn omgeving. SRL wordt hier
gebruikt om te generaliseren en om een hoger niveau model op te bouwen
iii
iv BEKNOPTESAMENVATTING
voor twee-armige acties in een huiselijke omgeving, waarbij gebruik gemaakt
wordt van symmetrieën tussen de armen om effecten te modelleren van acties
uitgevoerd met de arm die niet betrokken is in de gedragsbrabbelfase, en
achtergrondkennis over twee-armige acties.
Ten tweede, maken we gebruik van relationele affordance modellen in een
planningsomgeving met hoog niveau doelen die worden gespecificeerd door
(ruimtelijke) relaties tussen de objecten. Het relationele affordance model kan
gebruikt worden om een toestandsovergangsmodel in een tafelblad omgeving
te definiëren. We bieden een planningsalgoritme voor het bepalen van de best
mogelijkevolgendeactieuittevoerendoorderobotmethetoogophetbereiken
vanhetgegevendoel. Derobotwordtdaneenreeksvanplanningstakengegeven
van toenemende moeilijkheidsgraad in een tafelblad omgeving die opgelost
moeten worden.
Tenslotte, gebruiken we relationele affordances om te zoeken naar een object
dat een gegeven actie aanbiedt in een keuken omgeving met veel planken.
Meerdere types objecten kunnen de actie bieden, en elke type object zorgt
voor veel verschillende voorwerpen met afmetingen en vorm gemodelleerd door
kansverdelingen. Hierbijrelaxerenwesommigevandeeerdereveronderstellingen
in het vakgebied. Bovendien laten we voor gestapelde voorwerpen een meer
realistischemodelleringvanobjecteninschappentoe,waarbijwemeercomplexe
object ruimtelijke relaties introduceren.
Acknowledgments
Pursuing this Ph.D. has been a challenging, exciting and rewarding experience.
Many people were involved along the way, and through their ideas, support and
advice helped me achieve this goal. I would like to thank all of them here for
their contribution.
First of all, I would like to thank my supervisor Luc De Raedt. He encouraged
me to look for ideas, ask the right questions, and inspired me with pointers and
suggestions. We explored together research ideas in the field of robotics, and I
am thankful for his support and encouragement throughout all the stages of
my Ph.D. at KU Leuven.
Secondly,Iwanttokindlythankallthemembersofmyjury,HermanBruyninckx,
TinneTuytelaars, ClaudeSammut, JoséSantosVictor, andMartijnvanOtterlo
for carefully reading this text, and for the constructive feedback which helped
me improve it. Special thanks go to Martijn van Otterlo who as a postdoc at
KU Leuven also guided my first research steps in this field. I would also like to
thank Prof. Paul Van Houtte for chairing my defence.
Over the years, I collaborated with many people in order to publish all the
papers on which this thesis is based, and so I would want to thank here all of
my co-authors throughout my Ph.D. studies: Davide, Ingo, Jesse, José, Laura,
Luc, Martijn, McElory, and Plinio. Your input was invaluable, and had an
impactonthefinalshapeofallourpapers. Theywouldnothavebeenthesame
without you.
Ofcourse,pursuingaPh.D.wouldnothavebeenthesamewithoutastimulating,
and enjoyable work environment, and for this I am thankful to all the past and
present members of the DTAI research group. I would like to thank Jesse Davis
for his input in the early stages of my Ph.D. studies. Next, I would want to
thank all the people with whom I had the pleasure of sharing an office during
my time at KU Leuven: Aaron, Davide, Laura, Martijn, Mathias, McElory, and
Parisa, for our many interesting discussions and the exchange of research ideas,
v
vi ACKNOWLEDGMENTS
and for the tea-breaks during the day. I am especially thankful to Davide for
showing me how to use his Distributional Clauses, which I used in several of
my experiments, and for his Italian cookies. During my time at KU Leuven, I
also enjoyed the daily lunches with all the current and former members of our
“Alma-group”. Our regular lunches were a bright spot every day throughout
all these years. And during these years I also enjoyed the various activities
around Leuven and Belgium with several past and present members of DTAI:
Andrew, Antoine, Anton, Benjamin, Clement, Daniele, Davide, Gitte, Irma,
Jessa, Kurt, McElory, Sebastijan, Sergey, Siegfried, Vladimir, and others. By
the same token, I am grateful for all the friends from Europe and all over the
world that I met throughout these past four years, and who had an impact on
my life.
I gratefully acknowledge the financial support received for the work performed
during this Ph.D. thesis from the IWT (agentschap voor Innovatie door
Wetenschap en Technologie), and by the First-MM project (Flexible Skill
Acquisition and Intuitive Robot Tasking for Mobile Manipulation in the Real
World)fundedbytheEuropeanCommunity’s7thFrameworkProgramme,grant
agreement First-MM-248258.
I would also want to thank all the people at IST Lisbon that I interacted with
during our collaboration in the First-MM project. Your input, suggestions and
contributions were invaluable, and many of the papers and research presented
in this thesis would not have been the same without you. I especially want to
thank Plinio, who helped a lot to get the iCub robot to do what we wanted it
to do in our experiments.
Finally, I would like to thank my parents for supporting and helping me
throughout all the study years, for always encouraging me to do what I was
passionate about, and follow my dreams.
Bogdan Moldovan
Leuven, March 2015