Table Of ContentTASK-DIRECTED SENSOR FUSION AND PLANNING
THE KLUWER INTERNATIONAL SERIES IN
ENGINEERING AND COMPUTER SCIENCE
ROBOTICS: VISION, MANIPULATION AND SENSORS
Consulting Editor
Takeo Kanade
Carnegie Mellon University
Other books in the series:
Robotic Grasping and Fine Manipulation. M. Cutkosky.
ISBN 0-89838-200-9.
Shadows and Silhouettes in Computer Vision. S. Shafer.
ISBN 0-89838-167-3.
Perceptual Organization and Visual Recognition. D. Lowe.
ISBN 0-89838-172-X.
Robot Dynamics Algorithms. R. Featherstone.
ISBN 0-89838-230-0.
Three Dimensional Machine Vision. T. Kanade (editor).
ISBN 0-89838-188-6.
Kinematic Modeling, Identification and Control oj Robot Manipulators.
H.W. Stone. ISBN 0-89838-237-8.
Object Recognition Using Vision and Touch. P .K. Allen.
ISBN 0-89838-245-9.
Integration, Coordination and Control oj Multi-Sensor Robot Systems.
H.F. Durrant-Whyte. ISBN 0-89838-247-5.
Motion Understanding: Robot and Human Vision.
W.N. Martin and 1.K. Aggrawal (editors). ISBN 0-89838-258-0.
Bayesian Modeling oj Uncertainty in Low-Level Vision. R. Szeliski.
ISBN 0-7923-9039-3.
Vision and Navigation: The CMU NA VLAB. C. Thorpe (editor).
ISBN 0-7923-9068-7.
TASK-DIRECTED SENSOR
FUSION AND PLANNING
A Computational Approach
by
Gregory D. Hager
University of Pennsylvania
....
"
KLUWER ACADEMIC PUBLISHERS
Boston/DordrechtiLondon
DiSI,ibu!(ln for Norlh Amerin:
Kluwer Acad~mic Publishers
10) Philip Drive
Assinippi Park
Norwell, Massachusem 02061 USA
llistributon for all olher countnn:
Kluwcr Academic Publishen Group
Dis1Tibution Cenlre
I'ost Office Sox 122
3300 AH Dordrechl. THE NETHERLANDS
Ubrary of Coal''" CllaIOling.in.Publkalloll Dill
Hager, Gregory, D., 1961-
Task-4irected sensor fusion and planning: a computational
approach / Gregory D. Hager.
p. cm. _ (The Kluwer international series in engin«ring and
computer science: 99)
ISBN·)): 97&·1-46) 2·8828·2 e-[SBN~)): 978-1-4613-1.545_2
001: 10.10011918-1-4613-1545-2
1. Robots-Design and construction. 2. Artificial intelligence.
[. Title. II. Series: Kluwcr inlernalional serin in engin=ing
and computer science: SECS 99.
TJ21 LH23 1990
629.8 '92-dc20 90-4550
CW
Cop>right © 1990 by Kluwer Academic Publishers
Softcover reprint of the hardcover 1st edition 1990
All rights reserved. No pari of this publication may be reproduced. stored in a retrieval system or
transmilled in any form or by any means, mechanical. photocopying, recordins. Or otherwise,
without the prior writtcn permission of the publisher. KIU"'cr Academic Publishers. 101 Philip
Dri'·e. Assinippi Park. Norwell, Massachusetts 02061.
Contents
Foreword xiii
Preface xv
Acknowledgements xvii
1 Introduction 1
1.1 A Model for Information Gathering. . . . . . . 3
1.2 A Strategy for Realizing Information Gathering 7
1.3 Organizations for Information Gathering . 8
1.4 An Overview of this Book 11
1.5 Literature . . . 13
2 Modeling Sensors 15
2.1 Modeling Sensing Geometry ....... . 17
2.2 Modeling Sensor Observation Uncertainty 30
2.3 Additional Modeling Considerations 40
2.4 An Example System 42
2.5 Discussion 48
2.6 Literature ..... . 51
3 Task Modeling and Decision Making 53
3.1 Task Modeling . 54
3.2 Decision Theory 62
3.3 Discussion 68
3.4 Literature .... 70
4 Mean-Square Estimation 73
4.1 Derivation of Mean Square Estimation Techniques 75
4.2 Robustness to System Variation .......... . 81
4.3 Robust Rules for Nonlinear Systems ....... . 89
4.4 Additional Comments On Moment-Based Representations 101
4.5 Discussion 104
4.6 Literature ................. . 105
5 Grid-Based Probability Density Methods 107
5.1 Grid-Based Probability Density Updating 109
5.2 Estimation and Payoff Computation 118
5.3 Robustness ..... . 120
5.4 Error Analysis ... . 125
5.5 Simulation Evaluation 129
5.6 Extensions ...... . 133
VI
5.7 Discussion 134
5.8 Literature 136
6 Choosing Viewpoints and Features 137
6.1 Describing the Sensor Action Space. 138
6.2 Implementing Sensor Planning ... 143
6.3 Simulation Analysis of Sensor Planning 146
6.4 Discussion and Extensions 151
6.5 Literature........ ........ . 152
7 Towards a Task-Level Programming Environment 155
7.1 Sensor Fusion . . . . . 157
7.2 Task Specification ........ . 166
7.3 Observation Planning ...... . 169
7.4 Summary and Future Development 173
7.5 Literature ............. . 174
8 An Experimental System 175
8.1 Implementation Description 175
8.2 Experimental Results. 189
8.3 Discussion ... 197
9 Future Extensions 199
9.1 System Organization ............. . 200
9.2 Information Gathering with Multiple Sensors 202
9.3 The Model Selection Problem . . . . .. 205
9.4 Sensor Fusion and Artificial Intelligence 206
9.5 Summary ................ . 208
A Review of Probability 211
A.1 Basic Probability .. 211
A.2 Conditional Probability 214
A.3 Expectations ..... . 215
A.4 Transforming Probability 217
A.5 Convergence. . . . . . . . 218
B Review of Methods for Estimation 221
B.1 Stochastic Approximation .. 222
B.2 Least Squares Methods ...... . 223
B.3 Maximum Likelihood Method .. . 224
B.4 Maximum A Posteriori Probability 224
B.5 Decision Theory 225
B.6 Game Theory . . . . . . . . . . . . 227
Vll
B.7 f-maximin Game Theory 229
C System Hardware 231
References 233
Glossary of Mathematical Notation 245
Glossary of Symbols 241
Index 250
List of Figures
1.1 The logical organization of an information-gathering system. 9
1.2 A photograph of a postal sorting test system. 9
2.1 The geometry of sonar observing a wall. 18
2.2 A picture of a camera-in-hand system. . 22
2.3 A picture of tactile sensors. . ..... . 23
2.4 Fitting a rectangular box to a non-rectangular object. 28
2.5 Three different types of superellipsoid model variation. 29
2.6 An example of camera lens distortion. . . . . . . . . . 31
2.7 The geometry of error in estimation due to a calibration error. 32
2.8 Noise in a sub-pixel interpolation algorithm .......... . 34
2.9 Densities resulting from Gaussian distributed disparity errors .. 36
2.10 Four histograms of residuals. .................. . 38
2.11 The structure of a stereo-based position determination system. 43
2.12 Slider stereo hardware .................. . 43
2.13 The geometry of sonar and cameras observing a wall .. 47
3.1 Several (scalar) loss functions .............. . 59
4.1 The effects of system specification error on an MMSE estimate. 77
4.2 The error in point position estimation over time. . ... 79
4.3 Simulation of position tracking with standard methods. 80
4.4 The geometry of a scalar minimax problem. . . . . . . . 84
4.5 An illustration of filter divergence. . . . . ....... . 86
4.6 The error of a linear rule applied to a nonlinear system. 91
4.7 Piecewise approximation of a nonlinear function. . ... 93
4.8 Performance of the minimax estimator on a nonlinear system. 97
4.9 The randomization criteria of a, minimax estimator.. . . 98
4.10 The randomization criteria of a minimax estimator ..... . gg
4.11 Simulation of position tracking with minimax methods. 100
4.12 A comparison of moment-based uncertainty representations. 103
5.1 Approximation by a piecewise-constant density function .. 108
5.2 Approximation by a piecewise-constant density function .. 109
5,3 An example of a scalar update. . . . . . . . . . . . . . .. 110
5.4 The transformation of a piecewise-constant density function .. 111
5.5 An illustration of two different grid projection methods. . . . 113
5.6 Computing position from two monocular camera observations .. 117
5.7 The geometry of a solution space. . . . . . . . . . 123
5.8 An illustration of the geometry of model fitting .. 124
5.9 Updating errors as a function of observations ... 127
x
5.10 Updating errors as a function of unknown parameters. ..... 127
5.11 A comparison of the grid-based methods to the MMSE method. 130
5.12 A simulation of estimation of the position of a block ... 131
5.13 Simulated estimation of the size and position of a block. 132
5.14 Estimation bias for grids of 3, 5 and 7 elements ... 133
5.15 Estimation bias after 3 observations and after 30. 133
6.1 Stereo geometry for planning. ........... 141
6.2 A plot of payoff for camera/object relationship l. 147
6.3 A plot of payoff for camera/object relationship 2. 148
6.4 A plot of payoff for camera/object relationship 3. 149
6.5 Comparison of estimated and true average marginal gain. 150
6.6 The qualitative behavior of a stopping rule. 150
7.1 The logical structure of the software system .. 156
8.1 The logical configuration of the experimental system. . 176
8.2 The statistical edge operator mask. . . . . . . . . . . 176
8.3 A typical test scene for the static cameras. . . . . . . 190
8.4 A superellipsoidal model with shape parameter .95 .. 195
8.5 A superellipsoidal model with shape parameter .78 .. 196
8.6 A superellipsoidal model with shape parameter .15 .. 196
9.1 Extending an information-gathering system with a supervisor .. 201
List of Tables
6.1 Stopping rule performance for a unit priority. 149
6.2 Stopping rule performance for a priority of five. 151
8.1 The results of calibrating the camera focal length. 190
8.2 The results of estimating the size and position of the object. . 191
8.3 Estimating the size, position and rotation of a rectangular object. 192
8.4 Estimation results for a more complex positioning problem 192
8.5 Estimator performance on a series of rotations. . . . . . . 193
8.6 Estimator performance on three dimensional positioning. . 194
8.7 Estimator performance on position, width, and rotation. . 194
