Table Of ContentLecture Notes
in Control and Information Sciences
237
Editor: M. Thoma
regnirpS
London
Berlin
Heidelberg
New York
Barcelona
Budapest
Hong Kong
Milan
Paris
Santa Clara
Singapore
Tokyo
divaD .J ,namgeirK yrogerG .D regaH dna
.A nehpetS esroM )sdE(
ehT ecneulfnoC of
noisiV dna lortnoC
~ regnirpS
Series Advisory Board
A. Bensoussan • M.J. Grimble • P. Kokotovic • H. Kwakernaak
J.L. Massey • Y.Z. Tsypkin
Editors
David J. Kriegman PhD
Gregory D. Hager PhD
A. Stephen Morse PhD
Departments of Electrical Engineering and Computer Science, Yale University
PO Box 208267 Yale Station, New Haven CT 06520-8267, USA
ISBN 1-85233-025-2 Springer-Verlag Berlin Heidelberg New York
British Library Cataloguing in Publication Data
The confluence of vision and control. - (Lecture notes in
control and information sciences ; )732
1.Robot vision 2.Robots - Control systems
I.Kriegman, David J. II.Hager, Gregory D., 1961- III.Morse,
A. Stephen, 1939-
629.8'92
ISBN 1852330252
Library of Congress Cataloging-in-Publication Data
A catalog record for this book is available from the Library of Congress
Apart from any fair dealing for the purposes of research or private study, or criticism or review, as
permitted under the Copyright, Designs and Patents Act 1988, this publication may only be
reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing
of the publishers, or in the case of reprographic reproduction in accordance with the terms oflicences
issued by the Copyright Licensing Agency. Enquiries concerning reproduction outside those terms
should be sent to the publishers.
© Springer-Verlag London Limited 8991
Printed in Great Britain
The use of registered names, trademarks, etc. in this publication does not imply, even in the absence
of a specific statement, that such names are exempt from the relevant laws and regulations and
therefore free for general use.
The publisher makes no representation, express or implied, with regard to the accuracy of the
information contained in this book and cannot accept any legal responsibility or liability for any errors
or omissions that may be made.
Typesetting: Camera ready by editors
Printed and bound at the Atheneum Press Ltd., Gateshead, Tyne & Wear
6913830-543210 Printed on acid-free paper
Preface
The past decade has seen an increasing interest in the areas of real-time vi-
sion, visual tracking, active focus of attention in vision, vehicle navigation,
and vision-based control of motion. Although these topics encompass a di-
verse set of problems spanning the fields of vision, control, robotics, and
artificial intelligence, they all share a common focus: the application or pro-
cessing of visual information in a way which entails the design and analysis
of algorithms incorporating concepts studied in the field of control.
This collection emerged from the Block Island Workshop on Vision and
Control, held from June 23-27, 1997 on Block Island, Rhode Island. The
workshop, organized by J. Malik and .S Sastry from the University of Cal-
ifornia Berkeley, and G. Hager, D. Kriegman, and A.S. Morse from Yale
University, included participants from the U.S., Canada, Australia, France,
Germany, Israel and Italy, and in the fields of computer vision, control the-
ory, and robotics. It provided a forum for presenting new theoretical results,
empirical investigations, and applications as well as an opportunity to discuss
future directions for research. The contributions contained in this collection
touch on many of the same topics, from foundational issues such as estima-
tion, feedback, stability, delay, and task encoding, to the use of vision for
control in the context of visual servoing and non-holonomic systems, the use
of control within vision processes, and the application of vision and control in
vehicle navigation, grasping, and micro-electro-mechanical systems (MEMS).
We have also included a summary of the discussions which took place at the
workshop.
The Block Island Workshop on Vision and Control was generously sup-
ported by the National Science Foundation, the Army Research Office, and
Yale University Faculty of Engineering.
Table of Contents
Preface ....................................................... V
Research Issues in Vision and Control
Gregory D. Hager, David J. Kriegman, and A. Stephen Morse ........ 1
Visual Homing: Surfing on the Epipoles
Ronen Basri, Ehud Rivlin, and Ilan Shimshoni ...................... 11
Role of Active Vision in Optimizing Visual Feedback for
Robot Control
Rajeev Shaxma ................................................. 24
An Alternative Approach for Image-Plane Control of Robots
Michael Seelinger, Steven B. Skaar, and Matthew Robinson .......... 41
Potential Problems of Stability and Convergence in
Image-Based and Position-Based Visual Servoing
Franqois Chaumette ............................................. 66
What Can Be Done with an Uncalibrated Stereo System?
Jo~o Hespanha, Zachary Dodds, Gregory D. Hager, and A.S. Morse ... 79
Visual Tracking of Points as Estimation on the Unit Sphere
Alessandro Chiuso and Giorgio Picci .............................. 90
Extending Visual Servoing Techniques to Nonholonomic
Mobile Robots
Dimitris P. Tsakiris, Patrick Rives, and Claude Samson .............. 106
A Lagrangian Formulation of Nonholonomic Path Following
Ruggero Frezza, Giorgio Picci, and Stefano Soatto .................. 118
Vision Guided Navigation for a Nonholonomic Mobile Robot
Yi Ma, Jana KovseckA, and Shankar Sastry ........................ 134
VIII Table of Contents
Design, Delay and Performance in Gaze Control:
Engineering and Biological Approaches
Peter Corke .................................................... 146
The Separation of Photometry and Geometry Via Active
Vision
Ruzena Bajcsy and Max Mintz ................................... 159
Vision-Based System Identification and State Estimation
William A. Wolovich and Mustafa Unel ............................ 171
Visual Tracking, Active Vision, and Gradient Flows
Allen Tannenbaum and Anthony Yezzi, Jr .......................... 183
Visual Control of Grasping
Billibon H. Yoshimi and Peter K. Allen ............................ 195
Dynamic Vision Merging Control Engineering and AI
Methods
Ernst D. Dickmanns ............................................. 210
Real-Time Pose Estimation and Control for Convoying
Applications
R. L. Carceroni, C. Harman, C. K. Eveland, and C. M. Brown ........ 230
Visual Routines for Vehicle Control
Garbis Salgian and Dana H. Ballard ............................... 244
Microassembly of Micro-electro-mechanical Systems
(MEMS) using Visual Servoing
John Feddema and Ronald W. Simon ............................. 257
The Block Island Workshop: Summary Report
Gregory D. Hager, David J. Kriegman, and A. Stephen Morse with con-
tributions from P. Allen, D. Forsyth, S. Hutchinson, J. Little, N. Harris
McClamroch, A. Sanderson, and .S Skaar .......................... 273
List of Contributors
Peter K. Allen Alessandro Chiuso
Department of Computer Science Dipartimento di Elettronica e Infor-
Columbia University matica
New York, NY 10027 USA Universit~ di Padova
[email protected] Padua, Italy
[email protected]
Ruzena Bajcsy
GRASP Laboratory Christopher M. Brown
University of Pennsylvania Department of Computer Science
Philadelphia, PA 19104 USA University of Rochester
[email protected] Rochester, NY 14627 USA
[email protected]
Dana H. Ballard
Computer Science Department Peter Corke
University of Rochester CSIRO Manufacturing Science and
Rochester, NY 14627 USA Technology
[email protected] PO Box 388
Kenmore, 4069 Australia
Ronen Basri [email protected]
Department of Applied Math
The Weizmann Inst. of Science Ernst D. Dickmanns
Rehovot 76100 Israel Universit~t der Bundeswehr Mfinchen
[email protected] D-85577 Neubiberg, Germany
[email protected]
Rodrigo L. Carceroni
Department of Computer Science Zachary Dodds
University of Rochester Center for Computational Vision &
Rochester, NY 14627 USA Control
[email protected] Department of Computer Science
Yale University
Francois Chaumette New Haven, CT 06520-8285 USA
IRISA / INRIA Rennes [email protected]
Campus de Beaulieu
53 240 Rennes cedex, France
Francois.ChaumetteQirisa.fr
X List of Contributors
Chris K. Eveland David J. Kriegman
Department of Computer Science Center for Computational Vision &
University of Rochester Control
Rochester, NY 14627 USA Dept. of Electrical Engineering
[email protected] Yale University
New Haven, CT 06520-8285 USA
Dr. John Feddema [email protected]
Sandia National Laboratories
PO Box 5800, SM 3001 Yi Ma
Albuquerque, NM 87185 USA Electronics Research Laboratory
[email protected] University of California at Berkeley
Berkeley, CA 94720 USA
Ruggero Frezza [email protected]
Universit~ di Padova
via Gradenigo 6a Max Mintz
00153 Padova - Italy GRASP Laboratory
frezzaQveronica.dei.unipd.it University of Pennsylvania
Philadelphia, PA 19104 USA
Gregory D. Hager [email protected]
Center for Computational Vision &
Control A. Stephen Morse
Department of Computer Science Center for Computational Vision &
Yale University Control
New Haven, CT 06520-8285 USA Dept. of Electrical Engineering
hagerQcs.yale.edu Yale University
New Haven, CT 06520-8267 USA
C. Harman [email protected]
Department of Computer Science
University of Rochester Giorgio Picci
Rochester, NY 14627 USA Dipartimento di Elettronica e Infor-
matica
Jo~o Hespanha Universit~ di Padova
Center for Computational Vision Padua, Italy
Control [email protected]
Dept. of Electrical Engineering
Yale University Patrick Rives
New Haven, CT 06520-8267 USA INRIA Sophia-Antipolis
[email protected] ,4002 Route des Lucioles, B.P. 39
,20960 Sophia Antipolis Cedex
Jana Ko~eck~ France
Electronics Research Laboratory [email protected]
University of California at Berkeley
Berkeley, CA 94720 USA
j [email protected]
List of Contributors IX
Ehud Rivlin 713 Pond Laboratory,
Department of Computer Science University Park, PA 6016-20861
The Technion [email protected]
Haifa 32000 Israel
[email protected] Dr. Ronald W. Simon
Sandia National Laboratories
Matthew Robinson PO Box 5800, SM 1003
Department of Aerospace and Me- Albuquerque, NM 87185 USA
chanical Engineering
Fitzpatrick Hall of Engineering Steven B. Skaar
University of Notre Dame Department of Aerospace and Me-
Notre Dame, Indiana 7365-65564 chanical Engineering
[email protected] Fitzpatrick Hall of Engineering
University of Notre Dame
Garbis Salgian Notre Dame, Indiana 7365-65564
Computer Science Department [email protected]
University of Rochester
Rochester, NY 14627 USA Stefano Soatto
[email protected] Dept. of Electrical Engineering
Washington University
Claude Samson St. Louis - MO 63130 USA
INRIA Sophia-Antipolis [email protected]
,4002 Route des Lucioles, B.P. 39
,20960 Sophia Antipolis Cedex Ilan Shimshoni
France Department of Computer Science
[email protected] The Technion
Haifa 32000 Israel
Shankar Sastry [email protected]
Electronics Research Laboratory
University of California at Berkeley Allen Tannenbaum
Berkeley, CA 94720 USA Department of Electrical and Com-
[email protected] puter Engineering
University of Minnesota
Michael Seelinger Minneapolis, MN 55455
Department of Aerospace and Me- [email protected]
chanical Engineering
Fitzpatrick Hall of Engineering Dimitris P. Tsakiris
University of Notre Dame INRIA Sophia-Antipolis
Notre Dame, Indiana 7365-65564 2004, Route des Lucioles, B.P. 39
Michael.J.Seelinger. 1 @nd.edu ,20960 Sophia Antipolis Cedex
France
Rajeev Sharma [email protected]
Department of Computer Science &
Engineering,
The Pennsylvania State University
