Geometric Estimation with Local Affine Frames and Free-form Surfaces

224 Pages·2008·35.06 MB·English

Geometric Estimation with
Local Affine Frames and Free-form
Surfaces
Dissertation
zur Erlangung des akademischen Grades
Doktor der Ingenieurwissenschaften
(Dr.-Ing.)
der Technischen Fakult(cid:228)t
der Christian-Albrechts-Universit(cid:228)t zu Kiel
Kevin K(cid:246)ser
Kiel
2008
1. Gutachter
2. Gutachter
3. Gutachter
Datum der m(cid:252)ndlichen Pr(cid:252)fung
Contents
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
2 Basic Concepts 5
2.1 Camera Model and Geometry . . . . . . . . . . . . . . . . . . 5
2.1.1 Projective Geometry and Notation . . . . . . . . . . . 5
2.1.2 Geometric Objects . . . . . . . . . . . . . . . . . . . . 6
2.1.3 Rotation . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1.4 Camera Model . . . . . . . . . . . . . . . . . . . . . . 9
2.1.5 Pose . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2 Photometric Image Creation . . . . . . . . . . . . . . . . . . . 12
2.2.1 Plenoptic Function . . . . . . . . . . . . . . . . . . . . 12
2.2.2 Piecewise Continuous Surfaces . . . . . . . . . . . . . . 13
2.2.3 Camera Hardware, CCD . . . . . . . . . . . . . . . . . 13
2.2.4 Lens E(cid:27)ects . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2.5 Brightness Models . . . . . . . . . . . . . . . . . . . . 15
2.3 Relations between Local Regions in a Scene . . . . . . . . . . 15
2.3.1 Displacement, Euclidean, Similarity, A(cid:30)ne Transform . 16
2.3.2 General Homography . . . . . . . . . . . . . . . . . . . 18
2.3.3 Perspectivity . . . . . . . . . . . . . . . . . . . . . . . 20
2.3.4 Homography induced by a Scene Plane . . . . . . . . . 21
2.3.5 In(cid:28)nite Homography and Conjugate Rotation . . . . . 22
3 Primitives in the Literature 23
3.1 Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.2 Lines and Planes . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.3 Conics, Quadrics and Convex Hull Regions . . . . . . . . . . . 26
3.4 Intensities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.5 Local Curves and Lines . . . . . . . . . . . . . . . . . . . . . . 27
3.6 Local Regions . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.7 Summary and Relation to this Thesis . . . . . . . . . . . . . . 31
I
II CONTENTS
4 Di(cid:27)erential Constraints 33
4.1 Robust Local Image Features . . . . . . . . . . . . . . . . . . 35
4.1.1 Regions of Interest . . . . . . . . . . . . . . . . . . . . 35
4.1.2 Local A(cid:30)ne Frame . . . . . . . . . . . . . . . . . . . . 35
4.1.3 Descriptors . . . . . . . . . . . . . . . . . . . . . . . . 39
4.1.4 Matching Strategies . . . . . . . . . . . . . . . . . . . . 40
4.2 The LAF Correspondence Constraint . . . . . . . . . . . . . . 40
4.2.1 Concatenation of Local A(cid:30)ne Frames . . . . . . . . . . 40
4.2.2 Warp Constraints . . . . . . . . . . . . . . . . . . . . . 42
4.2.3 Physically Motivated Interpretation . . . . . . . . . . . 43
4.2.4 Triangle Decomposition . . . . . . . . . . . . . . . . . 44
4.2.5 Re(cid:28)nement and Upgrade from Simpler Features . . . . 47
4.3 Alignment in Scale Space . . . . . . . . . . . . . . . . . . . . . 47
4.3.1 Related Work on Gradient-based Alignment . . . . . . 50
4.3.2 Parametric Image Alignment with Uncertainty . . . . . 52
4.3.3 Evaluation of the Optimization . . . . . . . . . . . . . 59
4.3.4 Summary on Gradient-based Optimization . . . . . . . 62
4.3.5 Practical Optimization for LAF correspondences . . . . 63
4.4 LAF Uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.4.1 Obtaining and Representing Uncertainty . . . . . . . . 64
4.4.2 Empiric Covariance . . . . . . . . . . . . . . . . . . . . 66
4.4.3 Incidence and Outlier Detection . . . . . . . . . . . . . 67
4.4.4 Maximum-Likelihood Estimation . . . . . . . . . . . . 69
4.4.5 Evaluation: Measuring and Finite Area Approximation 70
4.5 Relation to other Primitives . . . . . . . . . . . . . . . . . . . 72
4.5.1 Triple of Points . . . . . . . . . . . . . . . . . . . . . . 74
4.5.2 Conic Correspondence . . . . . . . . . . . . . . . . . . 74
4.5.3 Curves . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
5 Applications: Geometric Estimation 79
5.1 General Homography . . . . . . . . . . . . . . . . . . . . . . . 81
5.1.1 Previous Work on Homography Estimation . . . . . . . 81
5.1.2 Obtaining a General Homography
from Two Feature Correspondences . . . . . . . . . . . 81
5.1.3 Generalizing to n Correspondences . . . . . . . . . . . 86
5.1.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.2 Conjugate Rotation . . . . . . . . . . . . . . . . . . . . . . . . 90
5.2.1 Previous Work . . . . . . . . . . . . . . . . . . . . . . 91
5.2.2 A Minimal Parameterization . . . . . . . . . . . . . . . 93
5.2.3 Estimation . . . . . . . . . . . . . . . . . . . . . . . . . 98
CONTENTS III
5.2.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.2.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 104
5.3 Triangulation and Normal Estimation . . . . . . . . . . . . . . 105
5.3.1 Previous Work . . . . . . . . . . . . . . . . . . . . . . 105
5.3.2 Patchlet Estimation . . . . . . . . . . . . . . . . . . . . 107
5.3.3 Maximum Likelihood Estimation . . . . . . . . . . . . 111
5.3.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 111
5.3.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 113
5.4 Pose Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . 113
5.4.1 Perspectivity . . . . . . . . . . . . . . . . . . . . . . . 115
5.4.2 Previous Work . . . . . . . . . . . . . . . . . . . . . . 115
5.4.3 Pose Estimation from a LAF correspondence . . . . . 116
5.4.4 Optimization, Tracking and Maximum Likelihood Es-
timation . . . . . . . . . . . . . . . . . . . . . . . . . . 119
5.4.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 119
5.4.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 126
5.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
6 Free-form Surface Models 129
6.1 O(cid:31)ine Modeling . . . . . . . . . . . . . . . . . . . . . . . . . 131
6.1.1 Structure from Motion from Spherical Images . . . . . 133
6.1.2 Bundle Adjustment . . . . . . . . . . . . . . . . . . . . 139
6.1.3 Dense 3D Reconstruction . . . . . . . . . . . . . . . . . 140
6.2 Initialization using a Descriptor Database . . . . . . . . . . . 141
6.2.1 Previous Work on View Registration . . . . . . . . . . 141
6.2.2 Scene Database . . . . . . . . . . . . . . . . . . . . . . 143
6.3 Tracking Free-form Surface Models . . . . . . . . . . . . . . . 148
6.3.1 Spherical Camera . . . . . . . . . . . . . . . . . . . . . 149
6.3.2 Camera Tracking . . . . . . . . . . . . . . . . . . . . . 151
6.3.3 System Evaluation . . . . . . . . . . . . . . . . . . . . 156
6.3.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 164
7 Conclusion 169
7.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
7.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
A Analysis 173
A.1 Taylor Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
A.2 Homographies in P1: Rational Functions in R1 . . . . . . . . 175
IV CONTENTS
B Probability Theory 179
B.1 Basic Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . 179
B.1.1 Cumulative Distribution and Density . . . . . . . . . . 179
B.1.2 Moments . . . . . . . . . . . . . . . . . . . . . . . . . . 179
B.1.3 Mahalanobis Distance . . . . . . . . . . . . . . . . . . 181
B.2 Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
B.2.1 Normal Distribution . . . . . . . . . . . . . . . . . . . 181
B.2.2 χ2 distribution . . . . . . . . . . . . . . . . . . . . . . 183
B.3 Statistical Testing . . . . . . . . . . . . . . . . . . . . . . . . . 183
B.3.1 Incidence Test . . . . . . . . . . . . . . . . . . . . . . . 183
B.4 Uncertainty Propagation . . . . . . . . . . . . . . . . . . . . . 184
B.4.1 Linear Error Propagation . . . . . . . . . . . . . . . . 184
B.4.2 Monte Carlo Methods . . . . . . . . . . . . . . . . . . 185
B.4.3 Unscented Transform . . . . . . . . . . . . . . . . . . . 185
C Robust Estimation 187
C.1 Robust Estimation . . . . . . . . . . . . . . . . . . . . . . . . 187
C.1.1 Observations and Uncertainty . . . . . . . . . . . . . . 187
C.1.2 Least Squares and Covariance Estimation . . . . . . . 187
C.1.3 Covariance Estimation . . . . . . . . . . . . . . . . . . 188
C.1.4 Newton-like methods . . . . . . . . . . . . . . . . . . . 188
C.1.5 Gross Errors and Breakdown Point . . . . . . . . . . . 190
C.1.6 Robust Error Functions . . . . . . . . . . . . . . . . . 190
C.1.7 RANSAC-like methods . . . . . . . . . . . . . . . . . . 191
D Source Code 193
D.1 Conjugate Rotation Parameterization . . . . . . . . . . . . . . 193
List of Abbreviations V
List of Abbreviations
AC Absolute Conic
CAD Computer Aided Design
CCD Charged Coupled Device
DIAC Dual Image of the Absolute Conic
DLT Direct Linear Transformation
DOF Degrees of Freedom
DoG Di(cid:27)erence of Gaussians
EBR Edge-based Region
EKF Extended Kalman Filter
FoV Field of View
GPU Graphics Processing Unit
IAC Image of the Absolute Conic
IBCA Image Brightness Constancy Assumption
IBR Intensity-based Region
LAF Local A(cid:30)ne Frame
LDA Linear Discriminant Analysis
MAD Mean Absolute Di(cid:27)erence
MDA Multiple Discriminant Analysis
MLE Maximum Likelihood Estimation
MSER Maximally Stable Extremal Region
NCC Normalized Cross Correlation
PCA Principal Component Analysis
pdf probability density function
PLI Pre-image of the line at in(cid:28)nity
PMD Photonic Mixer Device
RANSAC Random Sampling Consensus
VI List of Abbreviations
SAD Sum of Absolute Di(cid:27)erences
SfM Structure from Motion
SIFT Scale Invariant Feature Transform
SLAM Simultaneous Localization and Mapping
SSD Sum of Squared Di(cid:27)erences
SVD Singular Value Decomposition
ToF Time of Flight
VRML Virtual Reality Modeling Language
Symbols and Notation
To improve readability and for clarity of the equations, the following fonts,
styles, and symbols have been used:
R the set of the real numbers
C the set of the complex numbers
Pn the projective space representing vectors of Rn and the ideal points
s a scalar number (real-valued if not denoted otherwise)
x Euclidean vector from Rn
x element from Pn
R Matrix acting on elements from Rn
K Matrix acting on elements from Pn
I n×n identity matrix
n×n
0 n×n zero matrix
n×n
0 zero vector of dimension n
n
π plane at in(cid:28)nity
∞
l line at in(cid:28)nity
∞
[x] 3×3 cross product matrix of x
×
∀ for all
∃ there exist(s)
I[x] image I’s grey value at x
H[x] function H acting on x
∂f(cid:12)(cid:12) derivative of f with respect to x, evaluated at b
∂x
b
sup[f] supremum of f[x] for x ∈ [a;b]
[a;b]
det[M] determinant of square matrix M
trace[M] trace of square matrix M
(cid:39) equality up to scale, collinearity
euc[x] homogeneous to euclidean mapping
hom[x] euclidean to homogeneous mapping
vec[M] vectorization of matrix, stacking rows on top of one another
dim[x] dimension of x
exp[x] exponential function of x
Throughout the document, boldface italic serif letters x will always de-
VII
VIII List of Abbreviations
note Euclidean vectors while boldface upright serif letters x denote homo-
geneous vectors. Matrices appear as capital letters, where those acting on
Euclidean vectors are denoted by italic font without serifs (as A), while ma-
trices acting on homogeneous vectors are denoted by upright serif letters (as
A). FunctionsareindicatedbytypewriterfontTwiththeargumentinsquare
brackets [arg], while matrices are indicated by round brackets.