Table Of ContentLecture Notes in Artificial Intelligence 1634
Subseries ofLectureNotesin ComputerScience
Editedby J.G. Carbonelland J. Siekmann
Lecture Notes in Computer Science
Editedby G.Goos,J. Hartmanisand J.van Leeuwen
3
Berlin
Heidelberg
NewYork
Barcelona
HongKong
London
Milan
Paris
Singapore
Tokyo
Sasˇo Dzˇeroski Peter Flach (Eds.)
Inductive
Logic Programming
9th International Workshop, ILP-99
Bled, Slovenia, June 24-27, 1999
Proceedings
1 3
SeriesEditors
JaimeG.Carbonell, CarnegieMellonUniversity,Pittsburgh,PA,USA
Jo¨rgSiekmann, UniversityofSaarland,Saarbru¨cken,Germany
VolumeEditors
SasˇoDzˇeroski
Joz˘efStefanInstitute,DepartmentofIntelligentSystems
Jamova39,SI-1000Ljubljana,Slovenia
E-mail:[email protected]
PeterFlach
UniversityofBristol,DepartmentofComputerScience
MerchantVenturersBuilding,WoodlandRoad,BristolBS81UB,UK
E-mail:[email protected]
Cataloging-in-Publicationdataappliedfor
DieDeutscheBibliothek-CIP-Einheitsaufnahme
Inductivelogicprogramming:9thinternationalworkshop;
proceedings/ILP-99,Bled,Slovenia,June24-27,1999.Sasˇo
Dzˇeroski;PeterFlach(ed.).-Berlin;Heidelberg;NewYork;
Barcelona;Budapest;HongKong;London;Milan;Paris;
Singapore;Tokyo:Springer,1999
(Lecturenotesincomputerscience;Vol.1634:Lecturenotesin
artificialintelligence)
ISBN3-540-66109-3
CRSubjectClassification(1998):I.2,D.1.6
ISBN3-540-66109-3Springer-VerlagBerlinHeidelbergNewYork
Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthematerialis
concerned,specificallytherightsoftranslation,reprinting,re-useofillustrations,recitation,broadcasting,
reproductiononmicrofilmsorinanyotherway,andstorageindatabanks.Duplicationofthispublication
orpartsthereofispermittedonlyundertheprovisionsoftheGermanCopyrightLawofSeptember9,1965,
initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer-Verlag.Violationsare
liableforprosecutionundertheGermanCopyrightLaw.
(cid:1)c Springer-VerlagBerlinHeidelberg1999
PrintedinGermany
Typesetting:Camera-readybyauthor
SPIN10703406 06/3142–543210 Printedonacid-freepaper
Foreword
This volume contains 3 invited and 24 submitted papers presented at the Ninth
InternationalWorkshoponInductiveLogicProgramming,ILP-99.The24accep-
tedpaperswereselectedbytheprogramcommitteefromthe40paperssubmitted
to ILP-99. Each paper was reviewed by three referees, applying high reviewing
standards.
ILP-99 was held in Bled, Slovenia, 24{27 June 1999. It was collocated with
theSixteenthInternationalConferenceonMachineLearning,ICML-99,held27{
30 June 1999. On 27 June, ILP-99 and ICML-99 were given a joint invited talk
by J. Ross Quinlan and a joint poster session where all the papers accepted at
ILP-99 and ICML-99 were presented. The proceedings of ICML-99 (edited by
Ivan Bratko and Sa(cid:20)so D(cid:20)zeroski) are published by Morgan Kaufmann.
We wish to thank all the authors who submitted their papers to ILP-99, the
program committee members and other reviewers for their help in selecting a
high-qualityprogram,andtheinvitedspeakers:DaphneKoller,HeikkiMannila,
andJ.RossQuinlan.ThanksareduetoTanjaUrban(cid:20)ci(cid:20)candherteamandMajda
Zidanskiandherteamfortheorganizationalsupportprovided.Wewishtothank
Alfred Hofmann and Anna Kramer of Springer-Verlag for their cooperation in
publishing these proceedings. Finally, we gratefully acknowledge the (cid:12)nancial
support provided by the sponsors of ILP-99.
April 1999 Sa(cid:20)so D(cid:20)zeroski
Peter Flach
ILP-99 Program Committee
Francesco Bergadano (University of Torino)
Henrik Bostro¨m (University of Stockholm)
Ivan Bratko (University of Ljubljana)
William Cohen (AT&T Research Labs)
James Cussens (University of York)
Luc De Raedt (University of Leuven)
Sa(cid:20)so D(cid:20)zeroski (Jo(cid:20)zef Stefan Institute, co-chair)
Peter Flach (University of Bristol, co-chair)
Alan Frisch (University of York)
Koichi Furukawa (Keio University)
Roni Khardon (University of Edinburgh)
Nada Lavra(cid:20)c (Jo(cid:20)zef Stefan Institute)
John Lloyd (Australian National University)
Stan Matwin (University of Ottawa)
Raymond Mooney (University of Texas)
Stephen Muggleton (University of York)
Shan-Hwei Nienhuys-Cheng (University of Rotterdam)
David Page (University of Louisville)
Bernhard Pfahringer (Austrian Research Institute for AI)
Celine Rouveirol (University of Paris)
Claude Sammut (University of New South Wales)
Michele Sebag (Ecole Polytechnique)
Ashwin Srinivasan (University of Oxford)
Prasad Tadepalli (Oregon State University)
Stefan Wrobel (GMD Research Center for Information Technology)
Organizational Support
• The Albatross Congress Tourist Agency, Bled
• Center for Knowledge Transfer in Information Technologies, Jo(cid:20)zef Stefan
Institute, Ljubljana
Sponsors of ILP-99
• ILPnet2, Network of Excellence in Inductive Logic Programming
• COMPULOG Net, European Network of Excellence in Computational Logic
• Jo(cid:20)zef Stefan Institute, Ljubljana
• LPA Software, Inc.
• University of Bristol
Table of Contents
I Invited Papers
Probabilistic Relational Models
D. Koller .................................................................. 3
Inductive Databases (Abstract)
H. Mannila ............................................................... 14
Some Elements of Machine Learning (Extended Abstract)
J.R. Quinlan .............................................................. 15
II Contributed Papers
Re(cid:12)nement Operators Can Be (Weakly) Perfect
L. Badea, M. Stanciu ..................................................... 21
CombiningDivide-and-ConquerandSeparate-and-ConquerforE(cid:14)cientand
E(cid:11)ective Rule Induction
H. Bostro¨m, L. Asker ..................................................... 33
Re(cid:12)ning Complete Hypotheses in ILP
I. Bratko .................................................................. 44
Acquiring Graphic Design Knowledge
with Nonmonotonic Inductive Learning
K. Chiba, H. Ohwada, F. Mizoguchi ....................................... 56
Morphosyntactic Tagging of Slovene Using Progol
J. Cussens, S. D(cid:20)zeroski, T. Erjavec ....................................... 68
Experiments in Predicting Biodegradability
S. D(cid:20)zeroski, H. Blockeel, B. Kompare, S. Kramer,
B. Pfahringer, W. Van Laer .............................................. 80
1BC: A First-Order Bayesian Classi(cid:12)er
P. Flach, N. Lachiche ..................................................... 92
Sorted Downward Re(cid:12)nement: Building Background Knowledge
into a Re(cid:12)nement Operator for Inductive Logic Programming
A.M. Frisch ............................................................. 104
A Strong Complete Schema for Inductive Functional Logic Programming
J. Herna(cid:19)ndez-Orallo, M.J. Ram(cid:19)(cid:16)rez-Quintana ............................ 116
Application of Di(cid:11)erent Learning Methods
to Hungarian Part-of-Speech Tagging
T. Horva(cid:19)th, Z. Alexin, T. Gyimo(cid:19)thy, S. Wrobel .......................... 128
VIII Table of Contents
Combining LAPIS and WordNet for the Learning of LR Parsers with
Optimal Semantic Constraints
D. Kazakov .............................................................. 140
Learning Word Segmentation Rules for Tag Prediction
D. Kazakov, S. Manandhar, T. Erjavec .................................. 152
Approximate ILP Rules by Backpropagation Neural Network:
A Result on Thai Character Recognition
B. Kijsirikul, S. Sinthupinyo ............................................. 162
Rule Evaluation Measures: A Unifying View
N. Lavra(cid:20)c, P. Flach, B. Zupan ........................................... 174
Improving Part-of-Speech Disambiguation Rules by Adding
Linguistic Knowledge
N. Lindberg, M. Eineborg ................................................ 186
On Su(cid:14)cient Conditions for Learnability of Logic Programs from
Positive Data
E. Martin, A. Sharma ................................................... 198
A Bounded Search Space of Clausal Theories
H. Midelfart ............................................................. 210
Discovering New Knowledge from Graph Data
Using Inductive Logic Programming
T. Miyahara, T. Shoudai, T. Uchida, T. Kuboyama,
K. Takahashi, H. Ueda ................................................... 222
Analogical Prediction
S. Muggleton, M. Bain ................................................... 234
Generalizing Re(cid:12)nement Operators to Learn
Prenex Conjunctive Normal Forms
S.-H. Nienhuys-Cheng, W. Van Laer, J. Ramon, L. De Raedt ............ 245
Theory Recovery
R. Parson, K. Khan, S. Muggleton ....................................... 257
Instance Based Function Learning
J. Ramon, L. De Raedt .................................................. 268
Some Properties of Inverse Resolution in Normal Logic Programs
C. Sakama ............................................................... 279
An Assessment of ILP-Assisted Models for Toxicology and
the PTE-3 Experiment
A. Srinivasan, R.D. King, D.W. Bristol .................................. 291
Author Index .......................................................... 303
Probabilistic Relational Models
Daphne Koller
Computer Science Department, Stanford University, Stanford, CA 94305-9010
http://robotics.stanford.edu/~koller
[email protected]
Abstract. Probabilistic models provide a sound and coherent founda-
tionfordealingwiththenoiseanduncertaintyencounteredinmostreal-
world domains. Bayesian networks are a language for representing com-
plex probabilistic models in a compact and natural way. A Bayesian
networkcanbeusedtoreasonaboutanyattributeinthedomain,given
anysetofobservations.Itcanthusbeusedforavarietytasks,including
prediction, explanation, and decision making. The probabilistic seman-
tics also gives a strong foundation for the task of learning models from
data. Techniques currently exist for learning both the structure and the
parameters, for dealing with missing data and hidden variables, and for
discovering causal structure.
One of the main limitations of Bayesian networks is that they represent
theworldintermsofa(cid:12)xedsetof\attributes".Likepropositionallogic,
theyareincapableofreasoningexplicitlyaboutentities,andthuscannot
representmodelsoverdomainswherethesetofentitiesandtherelations
betweenthemarenot(cid:12)xedinadvance.Asaconsequence,Bayesiannet-
works are limited in their ability to model large and complex domains.
Probabilisticrelationalmodels arealanguagefordescribingprobabilistic
models based on the signi(cid:12)cantly more expressive basis of relational lo-
gic. They allow the domain to be represented in terms of entities, their
properties, and the relations between them. These models represent the
uncertaintyoverthepropertiesofanentity,representingitsprobabilistic
dependence both on other properties of that entity and on properties of
related entities. They can even represent uncertainty over the relational
structureitself.SomeofthetechniquesforBayesiannetworklearningcan
be generalized to this setting, but the learning problem is far from sol-
ved. Probabilistic relational models provide a new framework, and new
challenges, for the endeavor of learning relational models for real-world
domains.
1 Relational Logic
Relationallogichastraditionallyformedthebasisformostlarge-scaleknowledge
representation systems. The advantages of relational logic in this context are
obvious:thenotionsof\individuals",theirproperties,andtherelationsbetween
them provide an elegant and expressive framework for reasoning about many
diverse domains. The use of quanti(cid:12)cation allows us to compactly represent
general rules, that can be applied in many di(cid:11)erent situations. For example,
S.D(cid:20)zeroskiandP.Flach(Eds.):ILP-99,LNAI1634,pp.3{13,1999.
(cid:13)c Springer-VerlagBerlinHeidelberg1999
4 D. Koller
whenreasoningaboutgenetictransmissionofcertainproperties(e.g.,genetically
transmitted diseases), we can write down general rules that hold for all people
and many properties.
One of the most signi(cid:12)cant gaps in the expressive power of the logical fra-
mework, and one of the most signi(cid:12)cant barriers to its use in many real-world
applications,isitsinabilitytorepresentandreasonwithuncertainandnoisyin-
formation.Uncertaintyisunavoidableintherealworld:ourinformationisoften
inaccurate and always incomplete, and only a few of the \rules" that we use for
reasoning are true in all (or even most) of the possible cases.
2 Probabilistic Models
This limitation, which is crucial in many domains (e.g., medical diagnosis), has
ledoverthelastdecadetotheresurgenceofprobabilisticreasoninginAI.Proba-
bility theory models uncertainty by assigning a probability to each of the states
of the world that the agent considers possible. Most commonly, these states are
the set of possible assignments of values to a set of attributes or random varia-
bles. For example, in a medical expert system, the random variables could be
diseases, symptoms, and predisposing factors. A probabilistic model speci(cid:12)es a
jointdistributionoverallpossibleassignmentsofvaluestothesevariables.Thus,
it speci(cid:12)es implicitly the probability of any event.
As a consequence, unlike standard predictive models, a probability distribu-
tion is a model of the domain as a whole, and can be used to deal with a much
richer range of problems. It is not limited to conclusions about a prespeci(cid:12)ed
set of attributes, but rather can be used to answer queries about any variable
or subset of variables. Nor does it require that the values of all other variables
be given; it applies in the presence of any evidence. For example, a probabilistic
model can be used to predict the probability that a patient with a history of
smoking will get cancer. As new evidence about symptoms and test results is
obtained, Bayesian conditioning can be used to update this probability, so that
theprobabilityofcancerwillgoupifweobserveheavycoughing.Thesamemo-
del is used to do the predictive and the evidential reasoning. Most impressively,
a probabilistic model can perform explaining away, a reasoning pattern that is
very common in human reasoning, but very di(cid:14)cult to obtain in other formal
frameworks. Explaining away uses evidence supporting one cause to decrease
the probability in another, not because the two are incompatible, but simply
because the one cause explains away the evidence, removing the support for the
other cause. For example, the probability of cancer will go down if we observe
highfever,whichsuggestsbronchitisasanalternativeexplanationforthecough.
The same probabilistic model supports all of these reasoning patterns, allowing
it to be used in many di(cid:11)erent tasks.
The traditional objection to probabilistic models has been their computatio-
nalcost.Acompletejointprobabilitydistributionoverasetofrandomvariables
mustspecifyaprobabilityforeachoftheexponentiallymanydi(cid:11)erentinstantia-
tionsoftheset.Thus,anaiverepresentationisinfeasibleforallbutthesimplest
