Table Of ContentSPRINGER SERIES
IN PERCEPTION ENGINEERING
Series Editor: Ramesh Jain
Springer Series
In Perception Engineering
P.J. Besl: Sutfaces in Range Image Understanding
J.L.C. Sanz (ed.): Advances in Machine Vision
R.C. Vogt: Automatic Generation of Morphological Set Recognition
Algorithms
E.P. Krotkov: Active Computer Vision by Cooperative Focus and Stereo
T.-J. Fan: Describing and Recognizing 3-D Objects Using Sutface
Properties
Ting-Jun Fan
Describing and Recognizing
3-D Objects
U sing Surface Properties
With 68 Illustrations
Springer-Verlag
New Yark Berlin Heidelberg
London Paris Tokyo Hong Kong
Ting-Jun Fan
IBM T.J. Watson Research Center
Yorktown Heights, NY 10598
USA
Series Editor
Ramesh Jain
Electrical Engineering
and Computer Science Department
The University of Michigan
Ann Arbor, MI 48109
USA
Library of Congress Cataloging-in-Publication Data
Fan, Ting-Jun.
Describing and recognizing 3-D objects using surface properties I
Ting-Jun Fan.
p. cm.-(Springer series in perception engineering)
Includes bibliographical references.
I. Image processing. 2. Surfaces (Technology) I. Title.
II. Series.
TA1632.F36 1990
006.4'2-dc20 89-21848
© 1990 Springer-Verlag New York Inc.
Softcover reprint of the hardcover I st edition 1990
All rights reserved. This work may not be translated or copied in whole or in part without the written
permission of the publisher (Springer-Verlag New York, Inc., 175 Fifth Avenue, New York, NY
10010, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in
connection with any form of information storage and retrieval, electronic adaptation, computer
software, or by similar or dissimilar methodology now known or hereafter developed is forbidden.
The use of general descriptive names, trade names, trademarks, etc., in this publication, even if the
former are not especially identified, is not to be taken as a sign that such names, as understood by
the Trade Marks and Merchandise Act, may accordingly be used freely by anyone.
Camera-ready text prepared by the author using LaTeX.
9 8 7 6 5 4 3 2 I
ISBN-13:978-1-4612-8787-2 e-ISBN-13:978-1-4612-4466-0
DOl: 10.1007/978-1-4612-4466-0
Preface
Surface properties play a very important role in many perception tasks.
Object recognition, navigation, and inspection use surface properties ex
tensively. Characterizing surfaces at different scales in given data is often
the first and possibly the most important step. Most early research in ma
chine perception relied on only very coarse characterization of surfaces. In
the last few years, surface characterization has been receiving due attention.
Dr. T. J. Fan is one of the very few researchers who designed and im
plemented a complete system for object recognition. He studied issues re
lated to characterization of surfaces in the context of object recognition,
and then uses the features thus developed for recognizing objects. He uses
a multi-view representation of 3-D objects for recognition, and he devel
ops techniques for the segmentation of range images to obtain features for
recognition. His matching approach also allows him to recognize objects
from their partial views in the presence of other occluding objects. The
efficacy of his approach is demonstrated in many examples.
Since the emphasis in this work is on surface characterization, range
images offer an excellent domain to study issues without trivializing the
problem and without any unnecessary complexity. Range images have be
come increasing in popular due to the above and to the fact that techniques
for acquiring and processing range images are now making them applicable
to many industrial problems. Fan's book is the third book in the Springer
Series in Perception Engineering on this topic. Besl's book addresses sur
face characterization and segmentation, the book by Jain and Jain gives
an overview of the field in collected articles from several experts, and now,
T. J. Fan describes a complete system for object recognition.
Ramesh Jain
Ann Arbor, Michigan
Contents
1 Introduction 1
1.1 The Input . . . . . . . . . . . . . . . 2
1.2 Issues in Shape Description . . . . . 2
1.2.1 Criteria for shape description 2
1.2.2 Choosing segmented surface descriptions . 4
1.3 Issues of Recognition . . . . . . . . . . . . . 5
1.3.1 Description of models .. . . . . . . 5
1.3.2 Matching primitives and algorithms 6
1.4 Questions for the Research .... 7
1.5 The Contribution of the Research . 8
1.6 Organization of the Book .. 8
2 Survey of Previous Work 11
2.1 Survey of Shape Descriptions 11
2.1.1 Volume descriptions 12
2.1.2 Curve/line descriptions 13
2.1.3 Surface descriptions .. 14
2.1.4 Summary ....... . 17
2.2 Survey of Recognition Systems 17
2.2.1 3DPO ....... . 18
2.2.2 Nevatia and Binford . . . 18
2.2.3 ACRONyM ....... . 19
2.2.4 Extended Gaussian Image (EGI) 21
2.2.5 Oshima and Shirai . . . . . 21
2.2.6 Grimson and Lozanl;>-Perez 22
2.2.7 Faugeras and Hebert . 23
2.2.8 Bhanu.. 24
2.2.9 Ikeuchi 25
2.2.10 Summary 26
3 Surface Segm.entation and Description 27
3.1 Curvature Properties and Surface
Discontinuities . . . . . . . . . . . . . . 27
viii Contents
3.2 Detecting Surface Features ............. 30
3.2.1 Method 1: using directional curvatures and
scale-space tracking ............. 32
3.2.2 Method 2: using principal curvatures at a single scale 39
3.2.3 Method 3: using anisotropic filtering 40
3~ Spa~Grooping........... ~
3.4 Spatial Linking . . . . . . . . . . . 42
3.5 Segmentation into Surface Patches 43
3.6 Surface Fitting . . . . . . . 45
3.7 Object Inference .......... 47
3.7.1 Labeling boundaries . . . . 47
3.7.2 Occlusion and connectivity 50
3.7.3 Inferring and describing objects. 50
3.8 Representing Objects by Attributed Graphs 52
3.8.1 Node attributes. 53
3.8.2 Link attributes . . . . . . . . . . . . 53
4 Object Recognition 55
4.1 Representation of Models ..... 55
4.2 Overview of the Matching Process 55
4.3 Module 1: Screener. . . . . . . . . 58
4.4 Module 2: Graph Matcher . . . . . 59
4.4.1 Compatibility between nodes of the model view and
scene graph . . . . . . . . . . . . . . . . . . . . . .. 61
4.4.2 Compatibility between two pairs of matching nodes 62
4.4.3 Computing the geometric transform . . . . . . . 64
4.4.4 Modifications based on the geometric transform. 66
4.4.5 Measuring the goodness of a match. 67
4.5 Module 3: Analyzer. . . 67
4.5.1 Splitting objects 68
4.5.2 Merging objects 69
4.6 Summary . . . . . . 71
5 Experimental Results 73
5.1 The Models . . . . . . . . . . . . . . . . . . . 73
5.2 A Detailed Case Study . . . . . . . . . . . . . 82
5.2.1 Search nodes expanded in recognition 82
5.3 Results for Other Scenes . . . . . 94
5.4 Parallel Versus Sequential Search 108
5.5 Unknown Objects. 115
5.6 Occlusion . . . . . . . . . . . . . 115
Contents ix
6 Discussion and Conclusion 121
6.1 Discussion ............. . 121
6.1.1 Problems of segmentation . 121
6.1.2 Problems of approximation 123
6.2 Contribution ......... . 123
6.3 Future Research ....... . 125
6.3.1 From surface to volume 125
6.3.2 Applications ..... . 126
A Directional Curvatures 121
B Surface Curvature 131
C Approximation by Quadric Surfaces 133
Bibliography 135
List of Tables
3.1 Classification of patches. . . . . . . . . . . . . . . . . . . 46
5.1 Summary of the recognition results for the chosen scene. 93
5.2 Summary of recognition results for scene 1. 105
5.3 Summary of recognition results for scene 2. 105
5.4 Summary of recognition results for scene 3. 106
5.5 Summary of recognition results for scene 4. 106
5.6 Summary of recognition results for scene 5. 107
5.7 Summary of recognition results for scene 6. 107
5.8 Summary of recognition results for scene 7. 109
5.9 Results of parallel searches of scene 1. 110
5.10 Results of parallel searches of scene 2. 110
5.11 Results of parallel searches of scene 3. 112
5.12 Results of parallel searches of scene 4. 112
5.13 Results of parallel searches of scene 5. 113
5.14 Results of parallel searches of scene 6. 113
5.15 Results of parallel searches of scene 7. 114
5.16 Results of parallel searches of scene 8. 114
5.17 Results ofrecognition with unknown objects. 116
List of Figures
1.1 Levels of descriptions. 3
3.1 Local surface shape. . 29
3.2 Block diagram of Method 1. 31
3.3 Block dia.gram of Method 2. 32
3.4 Block diagram of Method 3. 33
3.5 Block diagram for the segmentation process. . 34
3.6 3-D plot of the "cup" image and its shading representation. 35
3.7 Derivative masks of 4 directions: (a) 0°, (b) 45°, (c) 90°, (d)
135°. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 36
3.8 Extrema and Zero-crossings in the vertical direction for
(1'
increasing from 0 to 1.5. . . . . . . . . . . . . . . . . . . .. 38
3.9 Results of the scale-space tracking in vertical direction for
the "cup" image using Method 1. . . . . . . . . . . . . . .. 39
3.10 Features detected for "cup" image using Method 2. . . .. 40
3.11 Features detected for "cup" image using Method 3 ... " 42
3.12 Results of the space grouping procedure for "cup" image. 44
3.13 Closed boundaries of the "cup" image. 45
3.14 Reconstruction of the "cup" image. . . . . . 48
3.15 Segmentation of a real range image. .... 49
3.16 Objects and graphs of the real range image. 52
4.1 Block diagram for the Screener ..... 56
4.2 Block diagram for the Graph matcher. 57
4.3 Block diagram for the Analyzer. ... 57
4.4 Block diagram for the graph matcher. 60
4.5 Geometric similarity by mirror effect .. 64
4.6 Computing transforms .. 66
4.7 Splitting objects. 68
4.8 Merging objects. .. . . 70
5.1 The four views of the model car. 75
5.2 The four views ofthe model chair. 76
5.3 The four views of the model telephone .. 77
