Table Of ContentCASE-BASED APPROXIMATE
REASONING
by
EYKE HÜ LLERMEIER
University of Magdeburg, Germany
AC.I.P. Catalogue record for this book is available from the Library of Congress.
ISBN-10 1-4020-5694-X (HB)
ISBN-13 978-1-4020-5694-9 (HB)
ISBN-10 1-4020-5695-8 (e-book)
ISBN-13 978-1-4020-5695-6 (e-book)
Published by Springer,
P.O. Box 17, 3300 AADordrecht, The Netherlands.
www.springer.com
Printed on acid-free paper
All Rights Reserved
© 2007 Springer
No part of this work may be reproduced, stored in a retrieval system, or transmitted
in any form or by any means, electronic, mechanical, photocopying, microfilming, recording
or otherwise, without written permission from the Publisher, with the exception
of any material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work.
To Christiane, Jana, and Jae¨l
Table of Contents
Dedication ................................................... v
Foreword..................................................... xiii
Preface....................................................... xv
Notation ..................................................... 1
1. Introduction.................................................. 5
1.1 Similarity and case-based reasoning ........................... 5
1.2 Objective of this book ....................................... 6
1.2.1 Making case-based inference more reliable................. 9
1.2.2 The important role of models............................ 10
1.2.3 Formal models of case-based inference .................... 11
1.3 Overview .................................................. 13
2. Similarity and Case-Based Inference .......................... 17
2.1 Model-based and instance-based approaches .................... 17
2.1.1 Model-based approaches ................................ 18
2.1.2 Instance-based approaches .............................. 19
2.1.3 Knowledge representation............................... 19
2.1.4 Performance in generalization ........................... 20
2.1.5 Computational complexity .............................. 21
2.2 Similarity-based methods .................................... 22
2.2.1 Nearest neighbor (NN) estimation........................ 22
2.2.2 Instance-based learning................................. 29
2.2.3 Case-based reasoning................................... 30
vii
viii TableofContents
2.3 The concept of similarity..................................... 33
2.3.1 Similarity in case-based reasoning........................ 33
2.3.2 Similarity and fuzzy sets................................ 37
2.3.3 Aggregation of local similarity measures .................. 38
2.4 Case-based inference ........................................ 41
2.4.1 Deterministic inference problems......................... 45
2.4.2 Non-deterministic inference problems..................... 48
2.4.3 Formal models of case-based inference .................... 52
2.5 Summary and remarks....................................... 53
3. Constraint-Based Modeling of Case-Based Inference .......... 59
3.1 Basic concepts.............................................. 59
3.1.1 Similarity profiles and hypotheses ........................ 59
3.1.2 Generalized similarity profiles ........................... 62
3.2 Constraint-based inference ................................... 65
3.2.1 A constraint-based inference scheme...................... 65
3.2.2 Non-deterministic problems ............................. 69
3.3 Case-based approximation.................................... 72
3.3.1 Properties of case-based approximation ................... 73
3.3.2 Local similarity profiles................................. 79
3.4 Learning similarity hypotheses................................ 81
3.4.1 The learning task ...................................... 81
3.4.2 A learning algorithm ................................... 85
3.4.3 Properties of case-based learning......................... 86
3.4.4 Experimental results ................................... 94
3.5 Application to statistical inference ............................ 97
3.5.1 Case-based parameter estimation ........................ 97
3.5.2 Case-based prior elicitation.............................. 98
3.6 Summary and remarks....................................... 99
4. Probabilistic Modeling of Case-Based Inference............... 103
4.1 Basic probabilistic concepts .................................. 106
4.1.1 Probabilistic similarity profiles and hypotheses............. 107
TableofContents ix
4.1.2 Generalized probabilistic profiles......................... 110
4.2 Case-basedinference,probabilisticreasoning,andstatisticalinference112
4.3 Learning probabilistic similarity hypotheses .................... 117
4.3.1 Simple hypotheses and credible case-based inference ........ 117
4.3.2 Extended case-based learning............................ 118
4.4 Experiments with regression and label ranking .................. 121
4.4.1 Regression: artificial data ............................... 122
4.4.2 Regression: real-world data.............................. 124
4.4.3 Label ranking ......................................... 126
4.5 Case-based inference as evidential reasoning .................... 129
4.5.1 Transformation of probabilistic evidence .................. 131
4.5.2 Inference from individual cases .......................... 133
4.5.3 Combining evidence from several cases.................... 134
4.6 Assessment of cases ......................................... 140
4.6.1 Similarity-weighted approximation ....................... 140
4.6.2 More general criteria ................................... 142
4.6.3 Assessment of individual cases........................... 144
4.7 Complex similarity hypotheses................................ 146
4.7.1 Inference schemes of higher order ........................ 147
4.7.2 Partially admissible profiles ............................. 150
4.8 Approximate probabilistic inference ........................... 152
4.8.1 Generalized uncertainty measures and profiles ............. 152
4.8.2 An approximate inference scheme ........................ 155
4.9 Summary and remarks....................................... 159
5. Fuzzy Set-Based Modeling of Case-Based Inference I ......... 165
5.1 Background on possibility theory.............................. 165
5.1.1 Possibility distributions as generalized constraints .......... 166
5.1.2 Possibility as evidential support ......................... 168
5.2 Fuzzy rule-based modeling of the CBI hypothesis................ 169
5.2.1 Possibility rules........................................ 170
5.2.2 Modeling the CBI hypothesis............................ 171
5.3 Generalized possibilistic prediction ............................ 176
x TableofContents
5.3.1 Control of compensation and accumulation of support ...... 177
5.3.2 Possibilistic support and weighted NN estimation .......... 178
5.3.3 Upper and lower possibility bounds ...................... 179
5.3.4 Fuzzy logical evaluation ................................ 180
5.3.5 Comparison of extrapolation principles ................... 181
5.3.6 From predictions to decisions............................ 186
5.3.7 An illustrative example ................................. 188
5.3.8 Complexity issues...................................... 191
5.4 Extensions of the basic model ................................ 191
5.4.1 Dealing with incomplete information ..................... 192
5.4.2 Discounting noisy and atypical instances .................. 194
5.4.3 From instances to rules ................................. 198
5.4.4 Modified possibility rules ............................... 200
5.4.5 Combination of several rules............................. 202
5.4.6 Locally restricted extrapolation.......................... 204
5.4.7 Incorporation of background knowledge................... 205
5.5 Experimental studies ........................................ 206
5.5.1 Preliminaries.......................................... 206
5.5.2 Classification accuracy.................................. 207
5.5.3 Statistical assumptions and robustness.................... 209
5.5.4 Variation of the aggregation operator..................... 210
5.5.5 Representation of uncertainty ........................... 211
5.6 Calibration of CBI models ................................... 212
5.7 Relations to other fields...................................... 219
5.7.1 Fuzzy and possibilistic data analysis...................... 219
5.7.2 Fuzzy set-based approximate reasoning ................... 222
5.8 Summary and remarks....................................... 224
6. Fuzzy Set-Based Modeling of Case-Based Inference II......... 229
6.1 Gradual inference rules ...................................... 230
6.1.1 The basic model....................................... 230
6.1.2 Modification of gradual rules ............................ 232
6.2 Certainty rules ............................................. 235
TableofContents xi
6.3 Cases as information sources ................................. 239
6.3.1 A probabilistic model .................................. 239
6.3.2 Combination of information sources ...................... 241
6.4 Exceptionality and assessment of cases......................... 243
6.5 Local rules................................................. 247
6.6 Summary and remarks....................................... 249
7. Case-Based Decision Making ................................. 253
7.1 Case-based decision theory ................................... 255
7.2 Nearest Neighbor decisions ................................... 258
7.2.1 Nearest Neighbor classification and decision making ........ 258
7.2.2 Nearest Neighbor decision rules.......................... 260
7.2.3 An axiomatic characterization ........................... 261
7.3 Fuzzy modeling of case-based decisions......................... 265
7.3.1 Basic measures for act evaluation ........................ 266
7.3.2 Modification of the basic measures ....................... 267
7.3.3 Interpretation of the decision criteria ..................... 268
7.4 Fuzzy quantification in act evaluation.......................... 270
7.5 A CBI framework of CBDM.................................. 274
7.5.1 Generalized decision-theoretic setups ..................... 274
7.5.2 Decision making using belief functions .................... 277
7.5.3 Possibilistic decision making............................. 279
7.6 CBDM models: A discussion of selected issues .................. 282
7.6.1 The relation between similarity, preference, and belief....... 282
7.6.2 The effect of observed cases ............................. 285
7.6.3 Dynamic aspects of decision making...................... 286
7.7 Experience-based decision making............................. 287
7.7.1 Compiled decision models............................... 290
7.7.2 Satisficing decision trees ................................ 292
7.7.3 Experimental evaluation ................................ 296
7.8 Summary and remarks....................................... 301
xii TableofContents
8. Conclusions and Outlook ..................................... 307
A. Possibilistic Dominance in Qualitative Decisions .............. 309
B. Implication-Based Fuzzy Rules as Randomized Gradual Rules 313
B.1 Implication-based fuzzy rules ................................. 314
B.1.1 Gradual rules ......................................... 315
B.1.2 Other implication-based rules............................ 317
B.2 Randomized gradual rules.................................... 320
B.3 A probabilistic representation of implication-based fuzzy rules..... 322
C. Similarity-Based Reasoning as Logical Inference .............. 333
D. Simulation Results of Section 3.4.4 ........................... 335
E. Experimental Results of Section 5.5.4......................... 339
F. Simulation Results of Section 7.4 ............................. 341
G. Computation of an Extended Splitting Measures.............. 343
H. Experimental Results of Section 7.7.2......................... 345
References ................................................... 346
Index......................................................... 369
Foreword
Inthelastthreedecades,importanttrendsofresearchinartificialintelligencehave
beendevotedtothedesignandthestudyofinferencesystemsthatexhibithuman-
likereasoningcapabilities,withspecialemphasisontolerancetowardincomplete
informationanduncertainty.Thisprogram,whichcanbealsorelatedtocognitive
psychology concerns, has led to the development of computational models, in
particularfordefaultreasoningthataccommodatesexceptionsandinconsistency,
for approximate reasoning with interpolative abilities, as well as for case-based
reasoning.
Case-basedreasoning(CBR)reliesontheconceptofsimilarity,andmorepartic-
ularlyontheideathatsituationsrecognizedassimilarinimportantaspectsmay
be also similar in other respects. CBR thus appears as a yet simple instance of
analogical reasoning, but as powerful as a general problem solving method. This
explains the success encountered by case-based reasoning, and beyond that, the
interest for similarity-based reasoning that has been increasing in the last ten
years.TheveryideaofCBRisthustosolvenewproblemsonthebasisofexpe-
rience that is represented by already solved problems of the same type, referred
to as cases. Thus, a new problem is solved by adapting the solution of a similar
case, hoping that the adaptation can be done with much less effort than solving
the problem from scratch.
ACBRsystemrequiresefficienttechniquesforseveralimportantsubtasks,such
as organizing and maintaining the case base, retrieving cases (which are maxi-
mally similar to the problem) from the case base, and adapting stored cases to
theproblemathand.ThebasicinferencemechanismunderlyingCBR,wherethe
concept of similarity plays a major role, is built upon the principle of instance-
based learning and nearest neighbor classification.
CBR has always been motivated by real-world problems, and research in this
area has largely focused on building efficient computer systems. Less work has
been done, however, on the theoretical foundations of case-based and similarity-
based inference. Note that this strongly contrasts with the situation in default
reasoning,andtoalessextentinapproximatereasoning.Thepresentbookseeks
toremedythisflaw.ItsobjectiveistocontributetoasoundfoundationofCBR
and related fields, such as instance-based learning and analogical reasoning, by
providing formal models of similarity-based inference.
To accomplish this objective, Eyke Hu¨llermeier embeds case-based inference
into different frameworks of knowledge representation and reasoning, namely
xiii
