Table Of ContentEditors:
Werner Rheinboldt
University of Pittsburgh
Pittsburgh, Pennsylvania
Daniel Siewiorek
Carnegie-Mellon University
Pittsburgh, Pennsylvania
Editorial Advisory Board:
Kazuhiro Fuchi, Director
Institute for New Generation Computer Technology (ICOT)
Tokya Japan
Makoto Nagao
Kyoto University
Kyota Japan
PERSPECTIVES IN COMPUTING, Vol. 20
(Formerly "Notes and Reports in Computer Science and
Applied Mathematics")
Machine Vision
Algorithms, Architectures, and Systems
Edited by
Herbert Freeman
The Center for Computer Aids for Industrial Productivity
Rutgers University
Piscataway New Jersey
ACADEMIC PRESS, INC
Harcourt Brace Jovanovich, Publishers
Boston San Diego New York
Berkeley London Sydney
Tokyo Toronto
Copyright © 1988 by Academic Press, Inc.
All rights reserved.
No part of this publication may be reproduced or
transmitted in any form or by any means, electronic
or mechanical, including photocopy, recording, or
any information storage and retrieval system, without
permission in writing from the publisher.
ACADEMIC PRESS, INC
1250 Sixth Avenue, San Diego, CA 92101
United Kingdom Edition published by
ACADEMIC PRESS INC (LONDON) LTD
24-28 Oval Road, London NW1 7DX
Library of Congress Cataloging-in-Publication Data
Machine vision: algorithms, architectures, and systems/edited by
Herbert Freeman.
p. cm.—(Perspectives in computing : vol. 20)
Proceedings of a workshop "Machine vision: where are we going?",
organized in April 1987 at the Hyatt Regency Hotel in New Brunswick,
NJ, by the Rutgers University CAIP Center.
Includes bibliographies.
ISBN 0-12-266720-4
1. Computer vision—Congresses. I. Freeman, Herbert.
II. Rutgers University. Center for Computer Aids for Industrial
Productivity III. Series.
TA1632.M335 1988
006.3'7-dc19 88-6320
CIP
88 89 90 91 9 8 7 6 5 4 3 2 1
Printed in the United States of America
Preface
In April 1987 the Rutgers University CAIP Center organized and
hosted a workshop in machine vision at the Hyatt Regency Hotel in
New Brunswick, NJ. The theme was "Machine Vision: Where Are
We and Where Are We Going?" The objective of the workshop was
to assess the current state of the art of machine vision and set a
course for future research.
The workshop drew some 75 attendees, and was purposefully lim
ited to achieve an environment that would be conducive to free in
teraction and exchange of ideas. Participants came from all over the
United States as well as from some foreign countries, and collectively
represented a major fraction of the leading researchers in machine vi
sion in the world. Representatives from industry and academe were
present in nearly equal numbers. The two-day program consisted
of 12 formal presentations of 40 minutes each, two one-and-one-half
hour panel sessions, and three evening open-discussion forums. This
book, though assembled subsequently, represents, in effect, the pro
ceedings of that workshop.
Machine vision is concerned with achieving electronically what
the human eye does so masterfully: visual perception. It involves
image sensing, image processing, pattern recognition and image un
derstanding. In the industrial sector the promising applications are
in the areas of materials handling, autonomous vehicle guidance
(with collision avoidance), inspection, continuous-process control,
and surveillance, with inspection looming as the application with
the likely greatest payoff.
Research on machine vision has been in progress for over 25 years,
yet practical accomplishments have been modest. To date most in
stalled machine vision systems are relatively simple and involve only
simple computer processing. In most cases, only silhouette images
are utilized and frequently the illumination is specially structured
to reduce the burden on the pattern recognition computations. The
vii
viii Preface
installations often are in hazardous or extreme clean-room environ
ments where the high cost of such systems can be more readily jus
tified. Generally, the difficulties in bringing machine vision systems
to the market place have been both technical and economical - cur
rent performance is still very much limited and their costs compare
unfavorably with the same task performed by humans.
It has been stated that "...the best vision system is a solution that
avoids the need for it." This may have some validity today; however,
it is not an inherent truism. Vision is by far the most powerful and
versatile sensing mechanism in the animal kingdom. This is true for
direct human vision as well as in those cases where human vision
is enhanced through the use of microscopes or telescopes. There is
every reason to believe that once we achieve machine vision with
a capability even modestly approaching that of human vision and
at competitive cost, the installation of machine vision systems will
virtually explode - not unlike the explosion of computer graphics
systems once these reached acceptable levels of performance and cost
in the late 1970's.
The prevalent opinion in machine vision today is that any sig
nificant increase in processing complexity for industrial applications
must await the arrival of new special-purpose, parallel-type computer
architectures. There are projects underway today in virtually every
machine vision laboratory in the world to develop "machine vision
engines", that is, special- purpose computers that will have the req
uisite capabilities at affordable costs. Rapid progress is being made,
and new exciting developments are surfacing almost daily.
The workshop, and thus indirectly also this book, were made
possible through the generous support of the New Jersey Commission
on Science and Technology. This support is gratefully acknowledged.
The editor would also like to express appreciation to Ted Wu for his
painstaking efforts in formatting the manuscript with lAT^X.
Herbert Freeman
Contributors
Numbers in parentheses indicate the pages on which authors' contributions
begin.
J. K. Aggarwal (193), Computer and Vision Research Center, The
University of Texas at Austin, Austin, TX 78712
Paul J. Besl (221), Computer Science Department, General Motors
Research Laboratories, Warren, MI 48090-9055
Peter J. Burt (1), David Sarnoff Research Center, Subsidiary of SRI
International, Princeton, Ν J 08543-5300
T. J. Fountain (31,103), Department of Physics and Astronomy,
University College London, London WClE 6BT, England
H. Freeman (299), CAIP Center, Rutgers University, CN-1390, Pis-
cataway, Ν J 08855-1390
Ramesh C. Jain (221), Electrical Engineering and Computer Science
Dept., University of Michigan, Ann Arbor, MI 48109-2122
Mark A. Lavin (161), Automation Research Department, Manufac
turing Research, IBM Research Division, Yorktown Heights,
NY 10598.
James J. Little (57), Artificial Intelligence Laboratory, Massachu
setts Institute of Technology, Cambridge, MA 02139
Joseph L. Mundy (109), Artificial Intelligence Branch, GE Cor
porate Research and Development Center, Schenectady, NY
12305
ix
χ Contributors
Azriel Rosenfeld (97), Center for Automation Research, University
of Maryland, College Park, MD 20742
Jakub Segen (257), ATkT Bell Laboratories, Holmdel, Ν J 07733
J. Sklansky (187), Department of Electrical Engineering and Com
puter Science, University of California at Irvine, Irvine, CA
92717
Steven L. Tanimoto (273), Computer Science Department, Univer
sity of Washington Seattle, WA 98195
Y. F. Wang (193), Computer and Vision Research Center, The
University of Texas at Austin, Austin, TX 78712
Wayne Wiitanen (149), Computer Science Department, General
Motors Research Laboratories, Warren, MI 48090-9055
'Smart Sensing' in
Machine Vision
Peter J. Burt
David Sarnoff Research Center
Abstract
In human vision, sensing is a distinctly active process: the eyes con
tinually move to explore the visual world, and to locate and gather
specific visual information. Similar 'smart sensing' strategies may
be essential for practical computer vision systems. Through intel
ligent selection of data at early stages of analysis such systems can
reduce the data and computations required to perform many tasks
by three or more orders of magnitude. Modest vision systems can
then perform challenging tasks in real time.
Smart sensing places particular demands on the architecture of
computer vision systems. These should support multiresolution,
pyramid processing and pipeline data flow. We have designed and
built a system of this type, the Pipeline Pyramid Machine.
1 Introduction
Images contain vast amounts of information, enough to challenge
the processing capabilities of even the most powerful computers. Yet
only a small fraction of this information may be relevant to a given
task. It is expedient, often essential, that a vision system avoid or
discard excessive detail at the very earliest stages of analysis, at the
sensor itself.
Still there is a tendency among system designers, at least in the
research community, to develop vision systems that process images
in full detail through the early stages of analysis, and only begin to
distinguish between useful and extraneous information as objects are
MACHINE VISION 1 Copyright © 1988 by Academic Press, Inc.
All rights of reproduction in any form reserved.
ISBN 0-12-266720-4
2 Peter J. Burt
identified, at a relatively late analysis stage. How, it may be asked,
can a vision system determine that particular image information is
unimportant until it has completed at least rudimentary analysis
of that information. Besides, the initial stages of analysis involve
only iconic processing. While such operations are data intensive
and time consuming in present computers, they are amenable to
implementation in special purpose hardware. Once such hardware
is available, it is argued, there will be little need for concern about
efficiency.
A closer examination of the nature of visual information and the
requirements of early vision suggest, however, that it is possible to
identify certain generic strategies for selecting salient information at
the sensor. These 'smart sensing' strategies lead to very significant
improvements in the speed of a vision system, or reductions in its
cost, that are critical in practical applications. These strategies are
the subject of the present paper.
It is also possible to determine essential architectural features
of a computer system for smart sensing. For example, such systems
must be capable of processing image data at multiple resolutions and
within restricted windows, or 'regions of interest'. Considerations of
such requirements lead to a proposed pipeline, pyramid architecture
for smart sensing.
2 What is Smart Sensing
Smart sensing may be defined as the selective, task oriented, gath
ering of information from the visual world. The viewer, be it man
or machine, takes an active role in the sensing processes, probing
and exploring the environment in order to extract information for
the task at hand. This active approach to vision is already quite
familiar to us from our own visual experience, for the human eye is
the preeminent example of a smart sensor.
Imagine, for example, that you are driving a car down a country
road. Your visual task as driver is to locate and follow the road,
observe oncoming traffic, and detect and avoid any unusual objects
in the road. At the same time you look for signs or other landmarks
'Smart Sensing1 in Machine Vision 3
that indicate progress towards your destination. However you do
not examine the world in uniform detail. Rather your eyes fixate in
turn on certain critical patterns, an oncoming car, the center line,
an object in the road, a sign, devoting perhaps a quarter of a second
or a second to each fixation.
The eye is capable of gathering detailed information about a scene
only in the small region centered on the fovea. Its resolving power
falls rapidly outside this region, towards peripheral vision. Thus
while you fixate the road, you hardly notice individual leaves, or
even trees by the side of the road, or objects in the fields beyond.
What is remarkable is the skill with which your visual systems locates
and extracts just that visual information required to drive the car,
while it ignores the vast flow of information not relevant to this task.
What are the essential elements of smart sensing? In the case of
human vision three elements are apparent. First there is the concen
tration of resolving power within the fovea. This serves at once as a
probe by which the visual system can explore the visual world and
as a device by which the system can limit the amount of detailed
information it must process at any moment in time. The second is
the broad field of view provided by peripheral vision. This serves
an alerting function; while sensing the world at much reduced res
olution it detects unexpected objects or events. It also provides a
guidance function, directing the fovea for close object examination
or tracking. Finally there is high level cognitive control of eye move
ments. As a viewer performs a task such as driving, he understands
the requirements of this task and moves his eyes to meet its needs.
These three features of human vision, the fovea, the periphery,
and high level control, are essential for smart sensing in computer vi
sion as well. These and several other basic elements of smart sensing
will be examined more closely below.
3 Why Smart Sensing
It can be argued that selective information gathering, smart sensing,
is both necessary and sufficient for most vision tasks. It is the nature
of the physical world that objects are localized in space and events
