Computer Analysis of Scenes of 3-Dimensional Curved Objects

134 Pages·1976·4.712 MB·English

by  Ramakant Nevatia (auth.)

Editor I Herausgeber:
Prof. Salomon Klaczko-Ryndziun. Frankfurt a.M.
Co-Editors I Mitherausgaber:
Prof. Ranan Banerji, Temple University, Phi/Bdelphia
Prof. Jerome A. Feldman, University of Rochester, Rocheste
Prof. Mohamed Abdelrahman Mansour, ETH, Zurich
Prof. Ernst Billeter, Universitat Fribourg, Fribourg
Prof. Christof Burckhardt, EPF, Lausanne
Prof. lvar Ugi, Technische Universitlit Miinchen
Prof. King-Sun Fu, Purdue University, West Lafayette
Interdisciplinary Systems Research lnterdiszipliniire Systemforschung
Analysis -Modelling-Simulation Analyse-Formalisierung-Simulation
The system science has been developed from several Die Systemwissenschaft hat sich BUS der Varbindung
scientific fields: control and communication theory, mahren~r Wissenschaftszweige entwickelt: der Regs
model theory and computer science. Nowadays it ful lungs-und SteiiBI'ungstheorie, der Kommunikationswis
fills the l"flqUirements which Norbert Wiener fotmulatsd senschaft, tier Modelltheorie und der lnformatik. Sie
originally for cybernetics; and were not fHsible at his erfiillt heute des Programm, des Norbert Wiener mit
time, because of insufficient development of computer seiner Definition von Kybernatik urspriing/ich vorgelegt
science in the pest. hilt und dessen Durchfiihrung zu seiner Zeit dun:h die
ResHrch and precticalapp/icstion of system science noch ungeniigend entwickelte Computerwissenschaft
involve works of specialists of system science BS well starlc eingeschrlinkt WilT.
as of those from vBrious fields of Bpp/icBtion. Up to Die Forschung und die praktische Anwendung der Sy
now, the efficiency of this co-operBtion hils been proved stemwissenschaft bezieht heute sowohl die FBChleute
in many theoreticBIBnd prBCtical works. tier Systemwissenschaft als auch die Spezialisten dar
The series 'lnterdisciplinBry Systems Rese11rch' is in Anwendungsgebiete sin. In vie/en Bereichen hat sich
tended to be 11 source of inform11tion for university diese Zu1111mmenarbeit mittlerweile bewiihrt.
students Bnd scientists involvsd in theoreticBI 11nd lip Die Reihe clnterdiszip/iniire Systemforschung• setzt
plied systems resesrch. The rHder shBII be informed sich zum Ziel, dem Studenten, dem Theon~tiker und
Bbout the most Bdv11nced state of the Brt in reseBrch, dem Praktiker iiber den neuesten Stand aus Lehre und
Bpplicetion, lecturing Bnd metatheoreticBI criticism in Forschung, aus der Anwendung und der metBtheore
this ~~res. It is 11/so intended to enlarge this •res by in tischen Kritik dieser Wissenschaft zu berichten.
cluding diverse mathematics/ modeling procedures Dieser R11hmen soli noch insofern erweitert werden, 11/s
developed in m11ny decBdes for the description Bnd op die Reihe in ihn1n Publikationen die mBthemBtischen
timization of systems. Modellierungsverfahren mit einbezieht, die in verschie
In contr11st to the former trBdition, which restricted the dansten Wissenschaften in vie/en Jahrzehnten zur
theoreticBI control Bnd computer science to mathemBti Beschn~ibung und Optimierung von Systemen erarbeite~
cians, physicists Bnd engineers, the present series em wurden.
.phBsizes the interdisciplinBrity which system science Entgegen del' friiheren Tradition, in der die theoretische
hils rHched until now, 11nd which tends to expend. City Regelungs-und Computerwissenschllft auf den Kn~is
and regional p/Bnners, psychologists, physiologists, der Mathematiker, Physiker und lngenieun~ beschriinkt
economists, ecologists, food scientists, sociologists. war, liegt die Betonung dieser Reihe auf der lntenliszi
politicBI scientists, lawyers. pedBgogues, philologists, p/inaritlit, die die Systemwissenschaft mittlerweile er
m11nagers, diplomats, military scientists 11nd other spe reicht hat und welter Bnstn~bt. Stadt-und Regionalpla
cialists Bl'tl increasingly confronted or even chllrgsd ner, Psycholofl.en. Physiologen, Betriebswirte, Volks
with problems of system science. wirtschBfter, Okologen, ErniihrungswissenschBfter,
The /SR series will contain research reports-including Soziologen, Politologen, Juristen, Plidagogen, MBna
PhD-theses -lectun1 notes, n~eders for lectun~s 11nd ger, Dip/omaten, Militiirwissenschllfter und Bndal'tl Fach
proceedings of scientific symposia. The use of less ex /eute sehen sich zunehmend mit Aufgaben der System
pensive printing methods is provided to BSsure thBt the forschung konfrontiert oder sogBr beauftragt.
Buthors' results m11y be offered for discussion in the Die ISR-Reihe win/ Forschungsberichte-einschliess
shortest time to a brolld, interested community. In or lich Dissertationan...., Vorlesungsskripten, Readers zu
der to assun~ the reproducibility of the published n~sults Vorlesungen und Tagungsberichte enthalten. Die Ver
the coding lists of the used progrBms should be in wendung wenig aufwendiger Herstellungsverfahl'tln sol/
cluded in reports about computer simulation. dazu dienen, die Ergebnisse der Autoren in kiirzester
The international character of this series is intended to Frist einer mog/ichst breiten, interessierten Offentlich
be accomplished by including reports in German, Eng keit zur Diskussion zu stellen. Um auch die Reprodu
lish and French, both from universities 11nd n~search zierbarkeit der Ergebnisse zu gewlihrleisten, werden in
centers in the whole world. To assure this goB/, the edi Berichten iiber Arbeiten mit dem Computer wenn im
tors' board will be composed of n~presentatives of the mer moglich such die Befehlslisten im Anhang mitge
different countries and arHS of interest. druckt.
Der internationals Charekter der Reihe sol/ durch die
AufnBhme von Arbeiten in Deutsch, Englisch und FrBnzO
sisch aus Hochschulen und Forschungszentren Bus aller
Weh verwirklicht werden. Dafiir sol/ sine entspre
chende Zusammensetzung des Herausgebergremiums
sorgen.
ISR22
Interdisciplinary Systems Research
lnterdisziplinare Systemforschung
Ramakant Nevatia
Computer Analysis
of Scenes of
3-Dimensional Curved Objects
1976 Springer Basel AG
CIP-Kurztitelaufnahme der Deutschen Bibliothek
Nevatia, Ramakant
Computer analysis of scenes of 3-d curved objects.
l.Aufl.
(lnterdisciplinary systems research; 22)
ISBN 978-3-7643-0845-2 ISBN 978-3-0348-5206-7 (eBook)
DOI 10.1007/978-3-0348-5206-7
Ali rights reserved. No part of this publicat ion may be reproduced. stored in a
retrieval system. or transmitted. in any form or by any means. electronic.
mechanical. photocopying. recording or otherwise. without the prior permission
of the copyright owner.
© Springer Basel AG 1976
Originally published by Birkhăuser Verlag Basel in 1976
ISBN 978-3-7643-0845-2
ABSTRACT
Previous work in computer scene analysis has concentrated on
scenes of polyhedral objects. This work is concerned with the descrip
tion and recognition of curved objects. Toy animals and hand tools are
typical of the objects used. Three dimensional range data, acquired by a
laser triangulation system, is used as the input.
A complex object is represented by decomposition into sub-parts.
The individual parts are described by generalized cones, which are de
fined by a space curve known as the axis, and arbitrarily shaped normal
cross-sections along this axis. The parts are connected at joints, which
are described by the relationships of these parts. Structured, symbolic
graph-like descriptions result for a complete object.
Recognition is performed by matching of the symbolic descriptions
of an object with descriptions of models stored in the memory. A limited
amount of indexing capability exists to retrieve a sub-class of similar ob
jects from the memory, using selected, important features of the object
description. This permits recognition without matching with each object
in the memory.
Results of working programs on many actual scenes, consisting of
single as well as multiple objects are presented. The techniques developed
here have the potential of being useful in real industrial automation appli
cations.
ii
ACKNOWLEDGEMENTS
I am grateful to my advisor, Dr, Thomas 0. Binford, for his
constant guidance and many helpful insights. He provided numerous
suggestions and many ideas used in this work developed in discussions
with him.
I would like to thank Prof. Jerome Feldman for his reading of
this work and taking interest in its progress; and Prof, Bernard Widrow
for his attempts at making it more readable, Prof. John McCarthy pro
vided the opportunity to work at the Stanford Artificial Intelligence Labo
ratory.
An experimental work of this nature would not have been possible
without drawing on many other people's support, In particular, I wish to
acknowledge the following: Dr. Gerald J, Agin, for his data acquisition
programs and some picture :files; Alan Borning for his preliminary proces
sing ,programs; Wing Hui for help with the programming of the "Projection"
routines; and Joe Zingheim and Karl Pingle for their support of the "Hand/
Eye" hardware and software,
The research reported here and the preparation of this manuscript
were supported in part by the Advanced Research Projects Agency of the
U.S. Department of Defense under Contracts DAHC-15-73-C-0435 and
F0606-72-C-0008 respectively.
lll
TABLE OF CONTENTS
SECTION PACE
1.0 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 AN OVERVIEW . . . . . . . . . . . . . . . . . . . . . . . . 3
2.0 REPRESENT A IION . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1 GENERALIZED CONE PRIMITIVES. . . . . . . . . . . . . . . . . . 8
2.2 OTHER PRIMITIVES . . . . . . . . . . . . . . . . . . . . . . . . . II
3.0 DATA ACQ..UISITION AND BOUNDARY ORGANIZATION . . . . . . . . . 12
3.1 LASER TRIANGULATION RANGING . . . . . . . . . . . . . . . . 12
3.2 BOUNDARY ORGANIZATION . . . . . . . . . . . . . . . . . . . . 16
3.3 BODY SEPARATION . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.0 BODY SEGMENTATION AND PRIMITIVE DESCRIPTIONS. . . . . . . . . 22
4.1 CONE DEFINITIONS . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.2 LOCAL CONE SAMPLES: METHOD OF PROJECTIONS. . . . . . . 24
4.3 EXTENSION OF LOCAL CONES. . . . . . . . . . . . . . . . . . . . 25
4:4 OTHER CONE DESCRIPTION METHODS . . . . . . . . . . . . . . 34
4.5 CHOICE OF SEGMENTATIONS . . . . . . . . . . . . . . . . . . . . 37
4.6 OTHER DESCRIPTIONS OF PRIMITIVES . . . . . . . . . . . . . . 43
5.0 SYMBOLIC DESCRIPTIONS . . . . . . . . . . . . . . . . . . . . . . . . . 44
5.1 PIECE DESCRIPTIONS. . . . . . . . . . . . . . . . . . . . . . . . . 46
5.2 LINKING OF PIECES AND JOINT DESCRIPTIONS . . . . . . . . . 47
5.3 OBJECT DESCRIPTION . . . . . . . . . . . . . . . . . . . . . . . . 51
5.4 LINKING OF SHADOWED PARTS . . . . . . . . . . . . . . . . . . 52
5.5 JOINT/PARI SEMANTICS. . . . . . . . . . . . . . . . . . . . . . . . 53
6.0 RECOGNITION . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
6.1 AN EXAMPLE . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
6.2 MATCHING WITH A MODEL. . . . . . . . . . . . . . . . . . . . . . 61
6.3 MATCHING OF OBJECT PIECES. . . . . . . . . . . . . . . . . . . . 64
6.4 OCCLUSION AND SHADOWS. . . . . . . . . . . . . . . . . . . . . . 67
6.5 SELECTION OF A MATCH . . . . . . . . . . . . . . . . . . . . . . 69
6.6 VERIFICATION . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
6.7 INDEXING INTO VISUAL MEMORY. . . . . . . . . . . . . . . . . . 71
6.8 MODEL ACQ..UISITION AND LEARNING . . . . . . . . . . . . . . 74
7.0 RESULTS AND CONCLUSIONS . . . . . . . . . . . . . . . . . . . . . . . . 76
7.1 RESULTS . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
7.2 EXECUTION TIMES AND MEMORY REQ..UIREMENTS . . . . . . . 90
7.3 FURTHER RESEARCH. . . . . . . . . . . . . . . . . . . . . . . . . 92
7.4 CONCLUSIONS . . . . . . . . . . . . . . . . . . . . . . . . . 93
APPENDIX I BOUNDARY ORGANIZATION . . . . . . . . . . . . . . . 96
APPENDIX 2 DESCRIPTION OF ALGORITHMS . . . . . . . . . . . . . 101
APPENDIX 3 DATA STRUCTURES . . . . . . . . . . . . . . . . . . . . 116
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . 118
iv
LIST OF ILLUSTRATIONS
FIGURE PAGE
1.1 Block Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1 Segmentation of a Human Form into Sub-parts . . . . . . . . . . . . . . . . . . g
3.1 Schematic View of Triangulation Ranging. . . . . . . . . . . . . . . . . . . . . . 13
3.2 Schematic of the Ranging Apparatus . . . . . . . . . . . . . . . . . . . . . . . . 14
3.3 Laser Scans for a Doll . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.4 Boundary Constructed from the Scans in Fig. 3.3 . . . . . . . . . . . . . . . . . . 17
3.5 A TV Picture of a Doll and a Snake . . . . . . . . . . . . . . . . . . . . . . . . 19
3.6 Laser Scans for the Scene of Fig. 3.5 . . . . . . . . . . . . . . . . . . . . . . . . 20
3.7 Boundary Derived from the Scans of Fig. 3.6 . . . . . . . . . . . . . . . . . . . . 21
4.1 Two choices for Axis Points . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.2 Cross-sections from Projection in a Chosen Direction. . . . . . . . . . . . . . . . 26
4.3 Local Cones Generated from Fig. 4.2 . . . . . . . . . . . . . . . . . . . . . . . . 27
4.4 Local Cones Generated by Projection in Eight Directions. . . . . . . . . . . . . . 28
4.5 Extension of a Cone . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.6 Axes of Extended Cones for the Doll of Fig. i.2 . . . . . . . . . . . . . . . . . . 31
4.7 Cones for a Hammer 32
4.8 Laser Scans for a Hammer 32
4.9 Cone Descriptions for a Glove . 35
4.10 Cone Descriptions for a Horse . 36
4.11 Different Cones for a Rectangle . . . . . . . . . . . . . . . . 38
4.12 Selected Cones from fig. 4.6 for a Doll. . . . . . . . . . . . . . . . . . . . . . . . 39
4.13 A II Extended Cones for Scene of Fig. 3.5 • • • • • . . . • • . • . . . • • . . • • . 41
4.14 Selected Cones from Fig. 4.13 . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
5.1 Connection Graph of a Doll . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
5.2 Joints between Pieces of the Doll of fig. 4.12 . . . . . . . . . . . . . . . . . . . . 48
5.3 Different Joint Types . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
6.1 Another View of the Glove of Fig. 4.9. . . . . . . . . . . . . . . . . . . . . . . . 55
6.2 Stick Figures of Two Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
6.3 A View of a Doll . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
6.4 Connection Graph of the Doll in Fig. 6.3 . . . . . . . . . . . . . . . 60
6.5 Matching Results for the Doll of fig. 6.3 . . . . . . . . . . . . . . . . 62
7.1 A TV Picture of a Horse and a Ring . . . . . . . . . . . . . . . . . . 77
7.2 A TV Picture of a Horse and a Doll . . . . . . . . . . . . . . . . . . . 77
7.3 Segmentation of Scene Corresponding to Fig. 7.1 . . . . . . . . . . . . . . . . . . 78
7.4 Recognition Results for Horse in Fig. 7.3 . . . . . . . . . . . . . . . . . . . . . . 79
7.5 Segmentation of Scene Corresponding to Fig. 7.2 . . . . . . . . . . . . . . . . . . 80
7.6 A View of a Horse . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
7.7 Recognition Results for Horse of Fig. 7.6 . . . . . . . . . . . . . . . . . . . . . . 82
7.8 A Snake . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
7.9 Another View of a Doll . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
7.10 Recognition Results for the Doll in fig. 7.9. . . . . . . . . . . . . . . . . . . . . . 84
7.11 A Glove . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
7.12 Recognition Results for the Glove in Fig. 7.11. . . . . . . . . . . . . . . . . . . . 85
LIST OF ILLUSTRATIONS v
7.13 Another View of a Doll 86
7.14 Another Hammer 86
7.15 Another View of a Horse 87
A 1.1 Boundary Linking for Different Cases . . . . . . . . . . . . . . . . . . . . . . 97
A 1.2 Examples of Segment Overlaps and Non-overlaps . . . . . . . . . . . . . . . . 99
A2.1 Cross-sections for a Selected Projection Direction ................. . 102
A2.2 Axis Refinement .......................... . 104