Similarity and affine invariant point detectors and descriptors

66 Pages·2011·7.04 MB·English

Introduction Two-viewgeometry Invariantinterestpoints InvariantDescriptors Matching Viewpointsimulation Conclusion
Similarity and affine invariant point
detectors and descriptors
Fr´ed´eric Sur
Loria, E´cole des Mines de Nancy & Inria, France
[email protected]
with ASIFT material from
Jean-Michel Morel
Cmla, E´cole Normale Sup´erieure de Cachan, France
Guoshen Yu
Cmap, E´cole Polytechnique, France
CVPR11tutorial-ToolsandMethodsforImageRegistration-June24,2011
1/42
Introduction Two-viewgeometry Invariantinterestpoints InvariantDescriptors Matching Viewpointsimulation Conclusion
Interest Points in Computer Vision
Feature extraction in images (especially Interest Point detection)
is the very first step of many Computer Vision applications, e.g.:
photography stitching,
object recognition,
stereovision,
pose estimation,
structure from motion,
robot localization
...
Second step: define correspondences between Interest Points
(i.e. pairs of IP which are images of the same physical 3D point).
2/42
Introduction Two-viewgeometry Invariantinterestpoints InvariantDescriptors Matching Viewpointsimulation Conclusion
Why Interest Points?
Features = points instead of edges/lines.
Attneave1954
“Informationisconcentratedalongcontoursandisfurtherconcentratedatthose
pointsonacontouratwhichitsdirectionchangesmostrapidly.”
3/42
Introduction Two-viewgeometry Invariantinterestpoints InvariantDescriptors Matching Viewpointsimulation Conclusion
Why Interest Points?
Features = points instead of edges/lines.
Robustness to the aperture problem.
+ Local method (vs global method):
robust to occlusions/clutter and to local deformations.
3/42
Introduction Two-viewgeometry Invariantinterestpoints InvariantDescriptors Matching Viewpointsimulation Conclusion
Interest Points and invariant descriptors
Feature = interest point + descriptor of a local patch.
source:http://www.cs.ubc.ca/∼lowe/keypoints/+Vedaldi’sVLFEAT.
−→ local descriptor to ease IP matching.
Pioneering work: Schmid-Mohr 1996.
4/42
Introduction Two-viewgeometry Invariantinterestpoints InvariantDescriptors Matching Viewpointsimulation Conclusion
Interest Points and invariant descriptors
Feature = interest point + descriptor of a local patch.
source:http://www.cs.ubc.ca/∼lowe/keypoints/+Vedaldi’sVLFEAT.
−→ local descriptor to ease IP matching.
Pioneering work: Schmid-Mohr 1996.
4/42
Introduction Two-viewgeometry Invariantinterestpoints InvariantDescriptors Matching Viewpointsimulation Conclusion
Desirable properties
IP and descriptor contents should be “invariant” to:
illumination change
and to some geometric transformations.
−→ Which ones?
i.e. Which geometric transformations are likely to map the query
image to the test image?
5/42
Introduction Two-viewgeometry Invariantinterestpoints InvariantDescriptors Matching Viewpointsimulation Conclusion
Outline
1 The two-view geometry
−→ requirements on geometric invariance of IP/descriptors.
2 Invariant Interest Points
−→ How to define an Interest Point?
3 Invariant Descriptors
−→ How to describe the patch around each IP?
4 Matching IP: limitations
5 A new way to attain affine invariance: viewpoint simulation
−→ Morel and Yu’s ASIFT.
6/42
Introduction Two-viewgeometry Invariantinterestpoints InvariantDescriptors Matching Viewpointsimulation Conclusion
Viewpoint invariance?
Invariance to central projection in the pinhole camera.
−→ unreachable when the structure of the underlying object is
unknown (problems e.g. with self-occlusions.)
7/42
Introduction Two-viewgeometry Invariantinterestpoints InvariantDescriptors Matching Viewpointsimulation Conclusion
The pinhole camera model
Ω
U
V Y X
M(x,y,z) C: optical center
P: image plane
C(cid:48) C Z
m(u,v) f: focal length
(d(C,P))
P
    
u fα fγ u x
0
v (cid:39)  0 fβ v0y
1 0 0 1 z
8/42