Perception as Bayesian Inference

528 Pages·1996·59.989 MB·English

by  Edited by David C. Knill, Whitman Richards

PERCEPTION AS BAYESIAN INFERENCE

PERCEPTION AS BAYESIAN
INFERENCE
Edited by
DAVID C. KNILL
University of Pennsylvania
WHITMAN RICHARDS
Massachusetts Institute of Technology
CAMBRIDGE
UNIVERSITY PRESS
CAMBRIDGE UNIVERSITY PRESS
Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, Sao Paulo
Cambridge University Press
The Edinburgh Building, Cambridge CB2 8RU, UK
Published in the United States of America by Cambridge University Press, New York
www.cambridge.org
Information on this title: www.cambridge.org/9780521461092
© Cambridge University Press 1996
This publication is in copyright. Subject to statutory exception
and to the provisions of relevant collective licensing agreements,
no reproduction of any part may take place without the written
permission of Cambridge University Press.
First published 1996
This digitally printed version 2008
A catalogue record for this publication is available from the British Library
ISBN 978-0-521-46109-2 hardback
ISBN 978-0-521-06499-6 paperback
Contents
List of contributors page vii
Preface ix
0 Introduction D.C. Knill, D. Kersten &A. Yuille 1
Part one: Bayesian frameworks 23
1 Pattern theory: A unifying perspective D. Mumford 25
2 Modal structure and reliable inference A. Jepson, W. Richards
& D.C. Knill 63
3 Priors, preferences and categorical percepts W. Richards, A. Jepson
& J. Feldman 93
4 Bayesian decision theory and psychophysics A.L. Yuille &
H.H. Bulthojf 123
5 Observer theory, Bayes theory, and psychophysics B.M. Bennett,
D.D. Hoffman, C. Prakash & S.N. Richman 163
Commentaries 213
Part two: Implications and applications 237
6 Implications of a Bayesian formulation of visual information for
processing for psychophysics D.C. Knill, D. Kersten & P. Mamassian 239
7 Shape from texture: Ideal observers and human psychophysics
A. Blake, H.H. Bulthoff & D. Sheinberg 287
8 A computational theory for binocular stereopsis P.N. Belhumeur 323
9 The generic viewpoint assumption in a Bayesian framework
W.T. Freeman 365
10 Experiencing and perceiving visual surfaces K. Nakayama &
S. Shimojo 391
11 The perception of shading and reflectance E.H. Adelson &
A.P. Pentland 409
12 Banishing the homunculus H. Barlow 425
Commentaries 451
Author index 507
Subject index 513

Contributors
E.H. Adelson
Department of Brain & Cognitive Sciences, Massachusetts Institute of Technology,
Cambridge MA 02139
H. Barlow
Physiological Laboratory, Downing St., Cambridge, England CB3 OEH
RN. Belhumeur
Department of Electrical Engineering, Yale University, New Haven, CT 06520
B.M. Bennett
Department of Mathematics, University of California at Irvine, Irvine, CA 92717
A. Blake
Department of Engineering Science, University of Oxford, Oxford, OXl 3PJ England
H.H. Bulthoff
Max-Planck Institutfur biologische Kybernetik, 72076 Tubingen, Germany
J. Feldman
Center for Cognitive Science, Rutgers University, Piscataway, NJ 08855
W.T. Freeman
Mitsubishi Electric Research Laboratories, 201 Broadway, Cambridge, MA 02139
D.D. Hoffman
Department of Cognitive Science, University of California at Irvine, Irvine, CA 92717
A.Jepson
Department of Computer Science, University of Toronto, Toronto, M5S 1A4 Canada
D. Kersten
Department of Psychology, University of Minnesota, Minneapolis, MN 55455
D.C. Knill
Department of Psychology, University of Pennsylvania, Philadelphia, PA 19104
P. Mamas sian
Department of Psychology, University of Minnesota, Minneapolis, MN 55455
D.D. Mumford
Department of Mathematics, Harvard University, Cambridge, MA 02138
vil
viii Contributors
K. Nakayama
Vision Sciences Laboratory, Department of Psychology, Harvard University, Cambridge,
MA 02138
A.R Pentland
Media Arts & Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139
C. Prakash
Department of Mathematics, California State University, San Bernardino, CA 92407
W. Richards
Media Arts & Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139
S. Richman
Department of Program in Mathematical Behavioral Science, University of California at
Irvine, Irvine, CA 92717
R. Rensink
Vision Sciences Laboratory, Department of Psychology, Harvard University, Cambridge,
MA 02138
D. Sheinberg
Department of Cognitive & Linguistic Sciences, Brown University, Providence, RI02912
S. Shimojo
Department of Psychology, University of Tokyo, Komaba, Meguro-ku, Tokyo 153 Japan
A.L. Yuille
Division of Applied Sciences, Harvard University, Cambridge, MA 02138
Preface
By the late eighties, the computational approach to perception advocated by Marr
(1982) was well established. In vision, most properties of the 2 1/2 D sketch such as
surface orientation and 3D shape admitted solutions, especially for machine vision
systems operating in constrained environments. Similarly, tactile and force sensing
was rapidly becoming a practicality for robotics and prostheses. Yet in spite of this
progress, it was increasingly apparent that machine perceptual systems were still
enormously impoverished versions of their biological counterparts. Machine sys-
tems simply lacked the inductive intelligence and knowledge that allowed biological
systems to operate successfully over a variety of unspecified contexts and environ-
ments. The role of "top-down" knowledge was clearly underestimated and was
much more important than precise edge, region,"textural", or shape information. It
was also becoming obvious that even when adequate "bottom-up" information was
available, we did not understand how this information should be combined from
the different perceptual modules, each operating under their often quite different
and competing constraints (Jain, 1989). Furthermore, what principles justified the
choice of these "constraints" in the first place? Problems such as these all seemed
to be subsumed under a lack of understanding of how prior knowledge should be
brought to bear upon the interpretation of sensory data. Of course, this conclusion
came as no surprise to many cognitive and experimental psychologists (e.g. Gregory,
1980; Hochberg, 1988; Rock, 1983), or to neurophysiologists who were exploring
the role of massive reciprocal descending pathways (Maunsell & Newsome, 1987;
Van Essen et ah, 1992). But the contributions of these groups were principally
facts and observations; there were no really comprehensive models. Missing was
an overarching framework within which a variety of computational and experi-
mental results might fit together and be assimilated. Here, we offer several such
frameworks, woven together by Bayesian threads.
Not surprisingly, at roughly the same time, several laboratories saw the need
for a well articulated, formal framework for perception that would show how prior
IX
x Preface
knowledge could drive the interpretation of sensory observations. Seeds had already
been planted in adjacent fields (e.g., Pearl, 1988; Skilling, 1991), as well as in our
own (e.g., Bennett et al, 1989; Clark & Yuille, 1990). The time seemed ripe to bring
together a small group to compare several of these new frameworks, and to place the
burden upon the authors to show how their proposals might suggest new directions
for experimental research. I asked David Knill, who had recently written with Dan
Kersten one of the keynote papers, if he would assume the principal role of orga-
nizing a meeting, assisted by a committee of myself, Heinrich Biilthoff, and Alan
Yuille. Our intent was to choose participants from computer science, mathematics,
cognitive science and psychophy sics, but to favor those who were already engaged in
collaborative studies of human vision from a computational perspective. After an en-
thusiastic group of participants agreed to meet, David Knill subsequently contacted
Dr. John Tangney of AFOSR, who subsequently provided support. The meeting
was then held in January 1993 at Chatham Bars Inn, Chatham, Massachusetts. This
collection represents a partial distillation of the results of the Chatham meeting.
As mentioned, our main goal was to evaluate whether any of the new formal
frameworks for perception could have any practical impact upon the kinds of ques-
tions asked by the experimentalists. In the process of this evaluation, we expected
that the common ground underlying the different, more theoretical proposals would
be revealed. Indeed, the title of the book, "Perception as Bayesian Inference" reflects
the unifying theme. However the reader should not be misled into concluding that all
contributors accept the hypotheses that biological perceptual systems indeed make
strict Bayesian inferences. Rather, the more representative view is that the Bayesian
formulation captures the essence common to most of the frameworks, and allows
the distinctions to be articulated clearly. Consequently, we begin this collection with
a tutorial by Knill et al. to aid the newcomer to Bayesian inference. But others may
wish to start at the end, reading first the final chapter (12) by Horace Barlow on "Ban-
ishing the Homunculus", which provides an entirely different motivation for contin-
uing through this volume. The remainder of the book then reflects the main goals:
the first part presents several different frameworks for understanding the perceptual
process, whereas the second part is committed more to implications and applica-
tions. Finally, we have added commentaries to the contributions that enlarge upon
the discussions that took place at the Chatham meeting. These commentaries, then,
are probably the most critical indicator of the extent to which we managed to meet
our main goal of fleshing out the theoretical frameworks and integrating them with
practical psychophysics and computation. Our hope is that we have kindled interest
in developing new, more powerful approaches to understanding the perceptual act.
Whitman Richards