Table Of ContentLecture Notes in Artificial Intelligence 6925
Subseries of Lecture Notes in Computer Science
LNAISeriesEditors
RandyGoebel
UniversityofAlberta,Edmonton,Canada
YuzuruTanaka
HokkaidoUniversity,Sapporo,Japan
WolfgangWahlster
DFKIandSaarlandUniversity,Saarbrücken,Germany
LNAIFoundingSeriesEditor
JoergSiekmann
DFKIandSaarlandUniversity,Saarbrücken,Germany
Jyrki Kivinen Csaba Szepesvári
Esko Ukkonen Thomas Zeugmann (Eds.)
Algorithmic
Learning Theory
22nd International Conference, ALT 2011
Espoo, Finland, October 5-7, 2011
Proceedings
1 3
SeriesEditors
RandyGoebel,UniversityofAlberta,Edmonton,Canada
JörgSiekmann,UniversityofSaarland,Saarbrücken,Germany
WolfgangWahlster,DFKIandUniversityofSaarland,Saarbrücken,Germany
VolumeEditors
JyrkiKivinen
UniversityofHelsinki,DepartmentofComputerScience
P.O.Box68(GustafHällströminkatu2b),00014Helsinki,Finland
E-mail:[email protected].fi
CsabaSzepesvári
UniversityofAlberta,DepartmentofComputingScience
Edmonton,AB,T6G2E8,Canada
E-mail:[email protected]
EskoUkkonen
UniversityofHelsinki,DepartmentofComputerScience
P.O.Box68(GustafHällströminkatu2b),00014Helsinki,Finland
E-mail:[email protected].fi
ThomasZeugmann
HokkaidoUniversity,DivisionofComputerScience
N-14,W-9,Sapporo060-0814,Japan
E-mail:[email protected]
ISSN0302-9743 e-ISSN1611-3349
ISBN978-3-642-24411-7 e-ISBN978-3-642-24412-4
DOI10.1007/978-3-642-24412-4
SpringerHeidelbergDordrechtLondonNewYork
LibraryofCongressControlNumber:2011936965
CRSubjectClassification(1998):I.2,F.4.1,F.1,F.2,I.2.3,I.2.6
LNCSSublibrary:SL7–ArtificialIntelligence
©Springer-VerlagBerlinHeidelberg2011
Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthematerialis
concerned,specificallytherightsoftranslation,reprinting,re-useofillustrations,recitation,broadcasting,
reproductiononmicrofilmsorinanyotherway,andstorageindatabanks.Duplicationofthispublication
orpartsthereofispermittedonlyundertheprovisionsoftheGermanCopyrightLawofSeptember9,1965,
initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.Violationsareliable
toprosecutionundertheGermanCopyrightLaw.
Theuseofgeneraldescriptivenames,registerednames,trademarks,etc.inthispublicationdoesnotimply,
evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevantprotectivelaws
andregulationsandthereforefreeforgeneraluse.
Typesetting:Camera-readybyauthor,dataconversionbyScientificPublishingServices,Chennai,India
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Preface
This volume contains the papers presented at the 22nd International Confer-
ence on Algorithmic Learning Theory (ALT 2011), which was held in Espoo,
Finland, October 5–7,2011.The conference was co-locatedwith the 14th Inter-
national Conference on Discovery Science (DS 2011). The technical program of
ALT2011contained28papersselectedfrom61submissions,and5invitedtalks.
The invited talks were presented in joint sessions at both conferences.
ALT 2011 was dedicated to the theoretical foundations of machine learning
andtookplaceonthecampusofAaltoUniversity,Espoo,Finland.ALTprovides
aforumforhigh-qualitytalkswithastrongtheoreticalbackgroundandscientific
interchange in areas such as inductive inference, universal prediction, teaching
models, grammatical inference, formal languages, query learning, complexity of
learning,on-linelearningandrelativelossbounds,semi-supervisedandunsuper-
visedlearning,clustering,activelearning,statisticallearning,regression,bandit
problems, support vector machines, Vapnik-Chervonenkis dimension, probably
approximately correct learning, Bayesian and causal networks, boosting and
bagging, information-based methods, minimum description length, Kolmogorov
complexity,kernels,graphlearning,decisiontreemethods,Markovdecisionpro-
cesses, reinforcement learning, intelligent agents, and real-world applications of
algorithmic learning theory.
DS 2011wasthe 14thInternationalConference onDiscoveryScience andfo-
cused on the development and analysis of methods for intelligent data analysis,
knowledge discovery and machine learning, as well as their application to sci-
entific knowledge discovery.Traditionally, it was co-located and held in parallel
with Algorithmic Learning Theory.
The present volume contains the texts of the 28 papers presented at ALT
2011, divided into groups of papers on inductive inference, regression, bandit
problems,onlinelearning,kernelsandmargin-basedmethods,intelligentagents,
and on other learning models. The volume also contains the texts or abstracts
of the invited talks:
– PeterAuer(Montanuniversita¨tLeoben,Austria),“ModelsforAutonomously
MotivatedExplorationinReinforcementLearning”(invitedspeakerforALT
2011)
– YoshuaBengio(Universit´edeMontr´eal,Canada),“OntheExpressivePower
of Deep Architectures” (joint invited speaker for ALT 2011 and DS 2011)
– Jorma Rissanen (Helsinki Institute for Information Technology, Finland),
“Optimal Estimation” (invited speaker for ALT 2011)
– Eyke Hu¨llermeier jointly with Johannes Fu¨rnkranz (Philipps-Universita¨t
Marburg, Germany, and Technische Universita¨t Darmstadt, Germany, re-
spectively),“LearningfromLabelPreferences”(invitedspeakerforDS2011)
– Ming Li (University of Waterloo, Canada), “Information Distance and Its
Extensions” (invited speaker for DS 2011).
VI Preface
Papers presented at DS 2011 are contained in the DS 2011 proceedings.
Since 1999, ALT has been awarding the E. M. Gold Award for the most
outstanding student contribution. This year, the award was given to Malte
Darnsta¨dt for his paper “Supervised Learning and Co-training,” co-authored
with Hans Ulrich Simon and Bala´zs Sz˝or´enyi.
ALT2011wasthe22ndintheALTconferenceseries,establishedinJapanin
1990.A second root is the conference series Analogicaland Inductive Inference,
previouslyheldin1986,1989,1992,whichmergedwiththeconferenceseriesALT
afteracollocationintheyear1994.Fromthenon,ALTbecameaninternational
conferenceserieswhichkeptitsstronglinkstoJapanbutwasalsoregularlyheld
in other destinations including Australia, Germany, Hungary, Italy, Portugal,
Singapore, Spain, the USA, and Finland.
The ALT series was supervised by its Steering Committee: Naoki Abe (IBM
ThomasJ.WatsonResearchCenter,Yorktown,USA),ShaiBen-David(Univer-
sityofWaterloo,Canada),JyrkiKivinen(UniversityofHelsinki,Finland),Philip
M. Long (Google, Mountain View, USA), Akira Maruoka (Ishinomaki Senshu
University, Japan), Takeshi Shinohara (Kyushu Institute of Technology, Iizuka,
Japan), Frank Stephan (National University of Singapore, Republic of Singa-
pore), EinoshinSuzuki (KyushuUniversity, Fukuoka,Japan), Csaba Szepesva´ri
(University of Alberta, Canada), Eiji Takimoto (Kyushu University, Fukuoka,
Japan), Gyo˝rgy Tura´n (University of Illinois at Chicago, USA and University
ofSzeged,Hungary),OsamuWatanabe (TokyoInstitute of Technology,Japan),
ThomasZeugmann(Chair,HokkaidoUniversity,Japan),andSandraZilles(Pub-
licity Chair, University of Regina, Saskatchewan, Canada).
We would like to thank the many people and institutions who contributed
to the success of the conference. In particular, we want to thank our authors
for contributing to the conference and for coming to Espoo in October 2011.
Without their efforts and their willingness to choose ALT 2011 as a forum to
report on their research,this conference would not have been possible.
We would like to thank the Aalto University, School of Science, Department
of Information and Computer Science, the University of Helsinki, Department
of Computer Science, the Helsinki Institute for Information Technology, and
Algodan–FinnishCentreofExcellenceforAlgorithmicDataAnalysisResearch
for generously sponsoring the conference.
We are furthermore grateful to Aalto University for hosting the event. The
support of Aalto University, the University of Helsinki, the Helsinki Institute
for Information Technologyand Algodanwas a greathelp, organizationallyand
financially, for the ALT 2011 and DS 2011 conferences.
We also thank the journal Artificial Intelligence for its generous financial
support of ALT 2011 and DS 2011.
We are also grateful that we could use the excellent conference management
system EasyChair for putting together the program for ALT 2011; EasyChair
was developed mainly by Andrei Voronkov and is hosted at the University of
Manchester. The system is cost-free.
Preface VII
The conference series ALT was this year, as in many previous years, co-
located with the series Discovery Science. We are grateful for this continuous
collaboration.Inparticular,wewouldliketothankthe ConferenceChairHeikki
MannilaandtheProgramCommitteeChairsTapioElomaaandJaakkoHollm´en
of Discovery Science 2011.
We would like to thank Olli Simula for organizing the conference and the
tremendous amount of work he put into making ALT 2011 a success. We want
to extend our thanks to the other members of the local Organizing Committee,
who were there to organize the reception, to sit at the information desk and to
carry out the other duties connected to organizing and hosting a conference.
We are grateful to the members of the ALT 2011 Program Committee and
the subreferees for their hard work in selecting a good program for ALT 2011.
Reviewing papers and checking the correctness of results is demanding in time
andskillsandweverymuchappreciatethis contributiontothe conference.Last
but not least we thank Springer for their support in preparing and publishing
this volume of the Lecture Notes in Artificial Intelligence series.
July 2011 Jyrki Kivinen
Csaba Szepesv´ari
Esko Ukkonen
Thomas Zeugmann
Organization
Conference Chair
Esko Ukkonen University of Helsinki, Finland
Program Committee
Dana Angluin Yale University, USA
Jean-Yves Audibert Universit´e Paris Est, France
Shai Ben-David University of Waterloo, Canada
Avrim Blum Carnegie Mellon University, USA
Nader Bshouty Technion, Israel
John Case University of Delaware, USA
Ricard Gavalda` Universitat Polit`ecnica de Catalunya, Spain
Andra´s Gyo˝rgy Hungarian Academy of Sciences, Hungary
Sanjay Jain National University of Singapore
Jyrki Kivinen (Chair) University of Helsinki, Finland
Ga´bor Lugosi Pompeu Fabra University, Spain
R´emi Munos InstitutNationaldeRechercheenInformatique
et en Automatique Lille, France
Ronald Ortner University of Leoben, Austria
John Shawe-Taylor University College London, UK
Hans Ulrich Simon Ruhr-Universita¨t Bochum, Germany
Frank Stephan National University of Singapore
Csaba Szepesv´ari (Chair) University of Alberta, Edmonton, Canada
Eiji Takimoto Kyushu University, Japan
Vladimir Vovk Royal Holloway, University of London, UK
Akihiro Yamamoto Kyoto University, Japan
Sandra Zilles University of Regina, Canada
Local Arrangements Chair
Olli Simula Aalto University, Espoo, Finland
X Organization
Subreferees
Jacob Abernethy Robert Kleinberg
Margareta Ackerman Timo Ko¨tzing
Andras Antos Mikhail Langovoy
Peter Auer Tor Lattimore
Maria-Florina Balcan Franc¸ois Laviolette
Andras A. Benczur Alessandro Lazaric
Alina Beygelzimer Shane Legg
Antoine Bordes Guy Lever
S´ebastien Bubeck Julien Mairal
Christos Dimitrakakis Andre Martins
Frank Drewes David McAllester
Aurelien Garivier Gergely Neu
Claudio Gentile Alexandru Niculescu-Mizil
Oded Goldreich Daniel Reidenbach
Peter Gru¨nwald Afshin Rostamizadeh
Joe Halpern Cynthia Rudin
Zaid Harchaoui Daniil Ryabko
Kohei Hatano Yevgeny Seldin
Daniel Hsu Ambuj Tewari
Rodolphe Jenatton Peter Torma
Hachem Kadri Jia Yuan Yu
Yuri Kalnishkan Thomas Zeugmann
Sponsoring Institutions
Aalto University, School of Science, Department of Information and Computer
Science
University of Helsinki, Department of Computer Science
Helsinki Institute for Information Technology
Algodan – Finish Centre of Excellence for Algorithmic Data Analysis Research
The journal Artificial Intelligence
Table of Contents
Editors’ Introduction ............................................. 1
Jyrki Kivinen, Csaba Szepesv´ari, Esko Ukkonen, and
Thomas Zeugmann
Invited Papers
Models for Autonomously Motivated Exploration in Reinforcement
Learning (Extended Abstract) ..................................... 14
Peter Auer, Shiau Hong Lim, and Chris Watkins
On the Expressive Power of Deep Architectures...................... 18
Yoshua Bengio and Olivier Delalleau
Optimal Estimation .............................................. 37
Jorma Rissanen
Learning from Label Preferences ................................... 38
Eyke Hu¨llermeier and Johannes Fu¨rnkranz
Information Distance and Its Extensions ............................ 39
Ming Li
Inductive Inference
Iterative Learning from Positive Data and Counters .................. 40
Timo Ko¨tzing
Robust Learning of Automatic Classes of Languages.................. 55
Sanjay Jain, Eric Martin, and Frank Stephan
Learning and Classifying.......................................... 70
Sanjay Jain, Eric Martin, and Frank Stephan
Learning Relational Patterns ...................................... 84
Michael Geilke and Sandra Zilles
Regression
Adaptive and Optimal Online Linear Regression on (cid:2)1-Balls ........... 99
S´ebastien Gerchinovitz and Jia Yuan Yu
XII Table of Contents
Re-adapting the Regularization of Weights for Non-stationary
Regression ...................................................... 114
Nina Vaits and Koby Crammer
Competing against the Best Nearest Neighbor Filter in Regression ..... 129
Arnak S. Dalalyan and Joseph Salmon
Bandit Problems
Lipschitz Bandits without the Lipschitz Constant .................... 144
S´ebastien Bubeck, Gilles Stoltz, and Jia Yuan Yu
Deviations of Stochastic Bandit Regret ............................. 159
Antoine Salomon and Jean-Yves Audibert
On Upper-Confidence Bound Policies for Switching Bandit Problems ... 174
Aur´elien Garivier and Eric Moulines
Upper-Confidence-Bound Algorithms for Active Learning in
Multi-armed Bandits ............................................. 189
Alexandra Carpentier, Alessandro Lazaric,
Mohammad Ghavamzadeh, R´emi Munos, and Peter Auer
Online Learning
The Perceptronwith Dynamic Margin.............................. 204
Constantinos Panagiotakopoulos and Petroula Tsampouka
Combining Initial Segments of Lists ................................ 219
Manfred K. Warmuth, Wouter M. Koolen, and David P. Helmbold
Regret Minimization Algorithms for Pricing Lookback Options ........ 234
Eyal Gofer and Yishay Mansour
Making Online Decisions with Bounded Memory..................... 249
Chi-Jen Lu and Wei-Fu Lu
Universal Prediction of Selected Bits ............................... 262
Tor Lattimore, Marcus Hutter, and Vaibhav Gavane
Semantic Communication for Simple Goals Is Equivalent to On-line
Learning........................................................ 277
Brendan Juba and Santosh Vempala
Kernel and Margin Based Methods
Accelerated Training of Max-Margin Markov Networks with Kernels.... 292
Xinhua Zhang, Ankan Saha, and S.V.N. Vishwanathan
