Do the Right Thing: Studies in Limited Rationality

216 Pages·1991·14.401 MB·English

by  Stuart Russell, Eric H. Wefald

Do the Right Thing

Copyrighted Material
Do the Right Thing
Studies in Limited Rationality
Stuart Russell and Eric Wefald
The MIT Press
Cambridge, Massachusetts
London, England
Copyrighted Material
©1991 Massachusetts Institute of Technology
All rights reserved. No part of this hook may be reproduced in any form
by any electronic or mechanical means (including photocopying,
recording, or information storage and retrieval) without permission in
writing from the publisher.
This book was typeset by the author using TEX3.0 and converted to
PostScript using Y & Y's DVIPSONE™. Camera-ready copy was
produced by Chiron, Inc.
Library of Congress Cataloging-in-Publication Data
Russell, Stuart J. (Stuart Jonathan)
Do the right thing: studies in limited rationality / Stuart Russell and
Eric Wefald.
p. cm. — (Artificial intelligence)
Includes bibliographical references and index.
ISBN 0-262-18144-4
1. Artificial intelligence. 2. Reasoning. 3. Decision-making. I. Wefald,
Eric, d. 1989. II. Title. III. Title: Limited rationality. IV. Series: Artificial
intelligence (Cambridge, Mass.)
Q335.R87 1991
006.3—dc20 91-17290
CIP
Copyrighted Material
To the memory of Eric Wefald
Copyrighted Material
Contents
List of Figures xi
List of Tables xiii
Series Foreword xv
Preface xvii
Acknowledgments xix
1 Limited Rationality 1
1.1 Introduction to Artificial Intelligence 1
1.2 Agents, architectures and programs 4
1.3 Problems with traditional approaches to agent design 8
1.4 Bounded optimality 11
1.5 Historical background 14
1.5.1 Simon's "Bounded Rationality" 14
1.5.2 Good's "Type II Rationality" 15
1.5.3 Information value 16
1.5.4 Limited rationality in philosophy 16
1.6 Achieving Bounded Optimality 17
1.6.1 Architectural constraints 17
1.6.2 Do the right thinking 23
1.7 Summary 27
2 Metareasoning Architectures 31
2.1 Introduction 31
2.2 Metareasoning 32
2.2.1 What is metareasoning? 32
2.2.2 Why and when to metareason? 34
2.2.3 How to metareason 36
2.3 Execution architectures 39
2.3.1 Why compilation? 40
2.3.2 An uncompiled architecture 42
2.3.3 Categories of knowledge 42
2.3.4 Types of execution architecture 46
2.3.5 The space of compilation methods 46
Copyrighted Material
Contents
viii
2.3.6 Approximate compilation 50
2.4 Metalevel execution architectures 51
2.4.1 TEIRESIAS 52
2.4.2 MRS 52
2.4.3 SOAR 53
2.4.4 Thoughts on a new architecture 55
2.5 The basis of metalevel knowledge 57
3 Rational Metareasoning 59
3.1 Introduction 59
3.2 Models of Deliberation 61
3.2.1 Notation 62
3.3 External analysis of metalevel decisions 63
3.3.1 The value of computation 64
3.3.2 Ideal and approximate control 66
3.3.3 Time and its Cost 67
3.4 Systems using utility estimates 68
3.4.1 Estimates and partial information 69
3.4.2 Analysis for complete computations 71
3.4.3 Simplifying assumptions 72
3.4.4 Partial computations 74
3.4.5 Qualitative behaviour 75
3.5 The concrete level 76
3.6 Summary 79
4 Application to Game-Playing 81
4.1 Introduction 81
4.2 The standard model of game-playing 83
4.3 Subtree independence 84
4.4 Controlling search by node expansion 87
4.4.1 How node values are propagated 87
4.4.2 Obtaining irrelevance criteria 89
4.4.3 Final expressions for the value of computation 90
Copyrighted Material
Contents ix
4.4.4 Estimating the probability distributions 91
4.5 Implementation of MGSS* 92
4.5.1 Examples of search trees 95
4.5.2 Performance of MGSS* 96
4.6 Controlling individual successor generation 98
4.6.1 The value of a partially expanded node 99
4.6.2 How node values are propagated 101
4.6.3 The new value of the node being expanded 104
4.6.4 Obtaining irrelevance criteria 104
4.6.5 Final formulae for the value of computation 106
4.7 Implementation of MGSS2 107
4.7.1 Examples of search trees 108
4.7.2 Performance 109
4.8 Chess and backgammon 110
4.9 Related work 113
4.10 Summary 116
5 Application to Problem-Solving Search 119
5.1 Search problems 120
5.2 The RTA* family of algorithms 121
5.2.1 Minimin search with alpha-pruning 121
5.2.2 Cycle avoidance 125
5.2.3 Choosing an update procedure 128
5.2.4 The SRTA* Algorithm 131
5.2.5 Trading off solution quality and search cost 134
5.3 The DTA* Algorithm 137
5.3.1 Generalized Heuristic Search Routines 137
5.3.2 Selecting a node to expand 139
5.3.3 The cost of a minimal useful computation 142
5.3.4 Possible computation steps 143
5.3.5 The expected value of search 144
5.3.6 Details of the algorithm 149
5.4 Performance of DTA* 150
Copyrighted Material
Contents
X
5.4.1 Experiments on the 15-puzzle 151
5.4.2 Experiments in robotic path-planning 151
5.4.3 An application in computer vision 153
5.5 Summary and further work 155
6 Learning the Valu e of Computation 159
6.1 Introduction 159
6.2 Incremental learning of the effects of computations 160
6.2.1 Prior distributions 162
6.2.2 Updating the distributions 162
6.2.3 Experimental results 163
6.3 Incremental learning of the value of computation 164
6.3.1 Metalevel feedback 165
6.3.2 Utility learning and credit assignment 167
6.4 Summary 169
7 Towards Limited Rational Agents 171
7.1 Problem-solving and planning systems 172
7.1.1 Bayesian search 172
7.1.2 Bayesian planning 174
7.1.3 Metareasoning about planning 176
7.1.4 Situated decision-making 177
7.2 Composing real-time systems 178
7.2.1 Interruptible vs contract algorithms 179
7.2.2 Generating composite anytime algorithms 180
7.2.3 Summary 182
7.3 Provably optimal systems 183
7.4 Conclusion 185
Bibliography 187
Index 195
Copyrighted Material
List of Figures
1.1 A simple mapping from percept sequences to actions 5
1.2 Illustration of ideal, optimizing and optimal action se
lection 14
2.1 Search trees produced by minimax and alpha-beta 35
2.2 Stages in the uncompiled execution architecture 44
2.3 Decision stages and shortcuts 45
3.1 The metalevel decision situation 63
3.2 Three basic situations 76
4.1 A minimax game tree 83
4.2 Revision of action preference under subtree indepen
dence 86
4.3 A partial game tree generated by node expansions 88
4.4 The MGSS* algorithm 93
4.5 Error frequency plot for a bucket with 174 data points 94
4.6 The algorithm, playing white, decides that move e7 is
obviously best 96
4.7 MGSS* decides to investigate the line cl-b4-b2-b6... in
depth 97
4.8 MGSS* and alpha-beta trees for the position above. 98
4-9 Pio,q f°r 0 == standard normal curve No,i 101
4.10 tpf _ , the density function after two successors 102
q 17
4.11 bf (m) plotted with pf 102
0iQ 0ifl
4.12 A partial game tree containing partially expanded nodes 103
4.13 MGSS2 and MGSS* trees for the same position. 109
4.14 A schematic backgammon search tree 111
5.1 The 8-puzzle 120
5.2 The minimin back-up procedure 123
5.3 Procedure for calculating backed-up value of a move. 124
5.4 Modification to keep previously visited nodes in a hash
table. 125
Copyrighted Material
List of Figures
5.5 Top-level loop with hashing 125
5.6 Final minimin backing-up procedure 129
5.7 Top-level loop for SRTA*. 131
5.8 Move selection for SRTA*. 132
5.9 RTA*, SRTA*: Nodes generated per move vs. search
depth 136
5.10 RTA*, SRTA*: Total solution cost vs. search depth 137
5.11 RTA*, SRTA*: Total solution cost vs. cost ratio, opti
mal depth 138
5.12 Generalized heuristic search procedure 139
5.13 Hardest-first search 142
5.14 The DTA* procedure for selecting moves. 150
5.15 Path-planning problems (50 and 100 obstacles). 152
5.16 Performance comparison for path-planning problems. 153
5.17 Original image of pallets 154
5.18 Image after edge detection 154
6.1 Learning curve for LDTA* 164
6.2 The metalevel decision situation 165
7.1 Belief network for a partially-explored state space 173
7.2 Initial network for Bayesian planner 175
7.3 Composition of anytime algorithms. 179
Copyrighted Material