Learning with Attributes for Object Recognition

158 Pages·2015·11.56 MB·English

Learning with Attributes for Object
Recognition: Parametric and
Non-parametric Views
by
Viktoriia Sharmanska
THESIS
Presented to the Faculty of the Graduate School of the
Institute of Science and Technology Austria, Klosterneuburg, Austria
in Partial Fulfillment of the Requirements for the Degree of
DOCTOR OF PHILOSOPHY
Supervisor:
Prof. Christoph Lampert, IST Austria, Klosterneuburg, Austria
Committee Member:
Prof. Chris Wojtan, IST Austria, Klosterneuburg, Austria
Committee Member:
Prof. Horst Bischof, TU Graz, Graz, Austria
Program Chair:
Prof. Herbert Edelsbrunner, IST Austria, Klosterneuburg, Austria
April, 2015
c by Viktoriia Sharmanska, April, 2015
￿
All Rights Reserved
Declaration
I hereby declare that this dissertation is my own work, and it does not contain
other people’s work without this being stated; and this thesis does not contain
my previous work without this being stated, and that the bibliography contains
all the literature that I used in writing the dissertation, and that all references
refer to this bibliography. I declare that this is a true copy of my thesis, including
any final revisions, as approved by my thesis committee, and that this thesis has
not been submitted for a higher degree to any other University or Institution.
Viktoriia Sharmanska
Brighton, UK
April, 2015

To Novi Quadrianto,
the most wonderful person I have ever met.

Acknowledgments
Completing the PhD would have been unthinkable without the help and support
of many wonderful people, who I was delighted to meet during my studies. I
would like to thank my supervisor, Christoph Lampert, for guidance throughout
my studies and for patience in transforming me into a scientist, and my thesis
committee, Chris Wojtan and Horst Bischof, for their help and advice. I would
like to thank Elisabeth Hacker who perfectly assisted all my administrative needs
and was always nice and friendly to me, and the campus team for making the
IST Austria campus my second home. I was honored to collaborate with brilliant
researchers and to learn from their experience. Undoubtedly, I learned most of all
from Novi Quadrianto: brainstorming our projects and getting exciting results
was the most enjoyable part of my work – thank you! I am also grateful to David
Knowles, Zoubin Ghahramani, Daniel Hernández-Lobato, Kristian Kersting and
AnastasiaPentinaforthefantasticprojectsweworkedontogether,andtoKristen
GraumanandAdrianaKovashkafortheexceptionalexperienceworkingwithuser
studies. I would like to thank my colleagues at IST Austria and my office mates
who shared their happy moods, scientific breakthroughs and thought-provoking
conversations with me: Chao, Filip, Rustem, Asya, Sameh, Alex, Vlad, Mayu,
Neel, Csaba, Thomas, Vladimir, Cristina, Alex Z., Avro, Amelie and Emilie,
Andreas H. and Andreas E., Chris, Lena, Michael, Ali and Ipek, Vera, Igor,
Katia. Special thanks to Morten for the countless games of table soccer we
played together and the tournaments we teamed up for: we will definitely win
nexttime:) AverywarmhugtoAsyaforalwaysbeingsoinspiringandsupportive
to me, and for helping me to increase the proportion of female computer scientists
in our group. Arigatou Ryoichi and Morten for the Japanese tea time we shared
together, for inspiring meetings and enjoyable gatherings at Heuriger. I would
like to express words of gratitude to Olga, little Maya and Chris for welcoming me
and making me feel at home when living abroad. Thank you, Filip, Magdalena
and Veronika, for coming to support me during my defense. Grazie Tatiana and
vii
viii
Michele for sharing the happiest days of our lives in Vienna and in Rome. I
would like to thank Claudia for her support and understanding during the most
demandingpartofmyPhDway, andforourdinnertimesthatwesharedtogether.
Danke Marta for our Salzburg trip and Barcelona exploration together. I would
like to thank my Texas friends at UT Austin who made my stay in the USA a
wonderful experience: Adriana, Dinesh, Aron, Chao-Yeh, Suyog, Bo and my host
Kristen Grauman. Thank you, my dear friend Aleksandra, for introducing me
to the gym and for taking care of me during my stay in Austin! I would like to
acknowledge the kind labmates with whom I shared my Cambridge experience:
Sae,David,Tomo,Miguel,Sebastien,ChristianandmyhostZoubinGhahramani.
IamverygratefultoJoeforhelpingmetoproofreadmythesis. Sonnchen, дякую
за нашi домашнi зустрiчi у Вiденських музеях! Зiрочко, дякую за те, що ти
освiтлювала i зiгрiвала мiй довгий шлях своєю посмiшкою. Iрчик, дякую за
те,щотибулазавждипоруч,вiдколиясебепам’ятаю.ДякуюдорогiйНаталiї
Олександрiвнi за мiцну i щиру дружбу. Дякую Мамi, Татовi, Iгорчику, Баб
Валiчцi,ДiдовiiмоїмдорогимЛьвiв’янамзапостiйнупiдтримку,вiруiтеплу
любов! Terima kasih Pa, Kak, Pipi, Ko Aseng atas dukungan dan doa Anda. My
last and most important words of love and gratitude are reserved for Novi: it is
thanks to you I was able to find myself and to come to the finish line of the PhD.
Thank You!
Kindly,
Viktoriia
Abstract
The human ability to recognize objects in complex scenes has driven research in
the computer vision field over couple of decades. This thesis focuses on the ob-
ject recognition task in images. That is, given the image, we want the computer
systemtobeabletopredicttheclassoftheobjectthatappearsintheimage. Are-
cent successful attempt to bridge semantic understanding of the image perceived
by humans and by computers uses attribute-based models. Attributes are se-
mantic properties of the objects shared across different categories, which humans
and computers can decide on. To explore the attribute-based models we take a
statistical machine learning approach, and address two key learning challenges
in view of object recognition task: learning augmented attributes as mid-level
discriminative feature representation, and learning with attributes as privileged
information. Our main contributions are parametric and non-parametric mod-
els and algorithms to solve these frameworks. In the parametric approach, we
explore an autoencoder model combined with the large margin nearest neighbor
principle for mid-level feature learning, and linear support vector machines for
learningwithprivilegedinformation. Inthenon-parametricapproach, wepropose
a supervised Indian Buffet Process for automatic augmentation of semantic at-
tributes, and explore the Gaussian Processes classification framework for learning
with privileged information. A thorough experimental analysis shows the effec-
tiveness of the proposed models in both parametric and non-parametric views.
Keywords Attributes, Object Classification, Mid-level Feature Representa-
tion, Learning using Privileged Information, Autoencoder, Large Margin Nearest
Neighbor, Indian Buffet Process, SVM, Gaussian Processes.
ix