Method and apparatus for three dimensional surface contouring and ranging using a digital video ...

37 Pages·2013·2.93 MB·English

US006788210B1
(12) United States Patent (10) Patent N0.2 US 6,788,210 B1
Huang et al. (45) Date of Patent: Sep. 7, 2004
(54) METHOD AND APPARATUS FOR THREE 4,641,972 A 2/1987 Halioua et a1. ........... .. 356/604
DIMENSIONAL SURFACE CONTOURING 4,657,394 A 4/1987 Halioua .......... .. 356/604
AND RANGING USING A DIGITAL VIDEO , , 2 lg2a1 lungme e1 Wa1 - . - - - .-. - - .- .
PROJECTION SYSTEM 5,069,548 A 12/1991 Boehnlein ................. .. 356/605
(75) Inventors: Peisen Huang, S. Setauket, NY (US); 5’O75’562 A 12/1991 Grelvenkamp’ Jr‘
. . . et a1. ................... .. 250/55905
Qmgylng Hm Ch?on Park NY (Us); 5,307,151 A 4/1994 HOfei a1. ................. .. 356/604
Fu-Pell ChlaHg, Setauket, NY (Us) 5,307,152 A 4/1994 Boehnlein et a1. 356/605
5,337,289 A * 8/1994 Fasching et a1. .... .. 340/612
(73) Assigneer The Research Foundation of State 5,561,526 A * 10/1996 Huber et a1. ........ .. 356/604
University of New York, Albany, NY 5,633,755 A 5/1997 Manabe et a1. 359/443
(US) 5,636,025 A 6/1997 Bieman et a1. ........... .. 356/619
( * ) Notice: Subject to any disclaimer, the term of this (Llst Con?rmed on next page‘)
patent is extended or adjusted under 35 FOREIGN PATENT DOCUMENTS
U.S.C. 154(b) by 171 days.
EP 0076866 4/1983
(21) Appl. No.: 10/088,133 EP 0769674 A 4/1997
(22) PCT Filed: Sep. 15, 2000 OTHER PUBLICATIONS
(86) PCT No; PCT/US00/25220 Angel, J. R. P. and WiZinoWich, P., “A Method for Phase
Shifting Interferometry the Presence of Vibration,” Euro
§ 371 (C)(1)> pean Southern Observatory Conf. Proc., 30, 561; (1988).
(2), (4) Date: Aug. 5, 2002 _ _
(List continued on neXt page.)
(87) PCT Pub' NO" Wool/20539 Primary Examiner—Van T. Trieu
PCT Pub. Date: Mar. 22, 2001 (74) Attorney, Agent, or Firm—Hoffmann & Baron, LLP
Related US. Application Data (57) ABSTRACT
(60) Provisional application NO' 60/154,063’ ?led on Sep' 16’ Athree dimensional surface contouring and ranging system
1999' (10) based on a digital fringe projection (18) and phase
(51) Int. Cl.7 .............................................. .. G08B 21/00 shifting technique is disclosed. In the system, three phase
(52) 340/612; 382/154; 382/286 shifted fringe patterns and an absolute phase mark pattern
(58) Field of Search ............................... .. 340/435, 436, are used to determine the absolute Phase map of the Object
340/612, 9951, 99514, 580; 382/141, 144, (12). The phase map is then converted to the absolute X, y,
145, 154, 286; 250/204, 205, 55923; 356/359, and Z coordinates of the object surface by a transformation
376, 512 algorithm. A calibration procedure is used to determine
accurate values of system parameters required by the trans
(56) References Cited formation algorithm. The parameters are initially indirectly
measured through experiments to determine their approxi
U~S~ PATENT DOCUMENTS mate values. A calibration plate is then measured by the
3,694,088 A 9/1972 Gallagher et aL ________ __ 356/495 system at various positions. An iteration algorithm is then
3,749,493 A 7/1973 Macovski ................. .. 356/2 used to estimate the System Parameters
4,634,278 A 1/1987 Ross et a1. . . . . . . . . . .. 356/376
4,639,139 A 1/1987 Wyant et a1. ............. .. 356/497 19 Claims, 21 Drawing Sheets
US 6,788,210 B1
Page 2
U.S. PATENT DOCUMENTS Giovanna Sansoni et al.; “A Novel, Adaptive System for
3—D Optical Pro?lometry Using a Liquid Crystal Light
5,680,215 A * 10/1997 Huber etal. .............. .. 356/600 Projector”; IEEE Transactions on Instrumentation and Mea
5,825,495 A 10/1998 Huber . . . . . . . . . . . . .. 356/600 surement 43(4): 558—566; Aug. 4, 1994; US.
5,835,218 A 11/1998 Harding . . . . . . . . . . . . . .. 356/521 Greivenkamp, J. E., “Generalized Data Reduction for Het
6,522,777 B1 * 2/2003 Paulsen et al. . . . . . . . . .. 382/154
erodyne Interferometry,” Opt. Eng, 23, 350 (1984).
6,603,103 B1 * 8/2003 Ulrich et al. ............. .. 250/205
Grievenkamp, J .E. and Bruning, J .H., “Phase Shifting Inter
OTHER PUBLICATIONS ferometry,” in OPtical Ship Testing, 2d Ed., Daniel Malac
ara, John Wiley & Sons, pp. 501—598 (1992).
Arai, Yasuhiko, Yekozeki, Shunsuke, and Yamada, Tomo
haru, “3—D automatic precision measurement system by Gove, R.J., “DMD Display Systems: The Impact of an
All—Digital Display,” SID 94 Digest, 673—676 (1994).
liquid crystal plate on moiré—topography,” Proc. SPIE, vol.
1554B, 266—274 (1991). Halioua, M., Krishnamurthy, R.S., Liu, H. —C., and Chiang,
Asundi, A., Digital Moire—Techniques and Applications, F.P., “Automated 360° Pro?lometry of 3—D Diffuse
Objects,” Appl. Opt., 24(12), 2193—2196 (1985).
Proc. SPIE, vol. 3407, 468—476 (1998).
Halioua, M., Krishnamurthy, R. S., Liu, H., and Chiang, F.
Asundi, A., “Novel grating methods for optical inspection,”
P., “Projection moire With moving gratings for automated
Proc. SPIE, vol. 1554B, 708—715 (1991).
3—D topography,” Appl. Opt. 22, 850—855 (1983).
Asundi, A., “Projection moiré using PSALM,” Proc. SPIE,
Halioua, M. and Liu, H. —C., “Optical Three—Dimensional
vol. 1554B, 257—265 (1991).
Sensing by Phase Measuring Pro?lometry,” Opt. Lasers
Asundi, A., “Fringe analysis in moiré interferometry,” Proc.
Eng, 11(3), 185—215 (1989).
SPIE, vol. 1554B, 472—480 (1991).
Harding, K.G., Coletta, Michael P., Van Dommelen Carl H.,
Bieman, Leonard H., “Survey of design considerations for
“Color encoded moiré contouring,” SPIE Proc. vol. 1005
3—D imaging systems,” Proc. SPIE, vol. 1005, 138—144
Optics, Illumination, and Image Sensing for Machine Vision
(1988).
III, 169—178 (1988).
Bieman, Leonard H., Harding, Kevin G., and Boehnlein,
Hariharan, P., Oreb, B. F., and Eiju, T., “Digital Phase—Shift
Albert, “Absolute measurement using ?eld shifted moire,”
ing Interferometry: A Simple Error—Compensating Phase
Proc. SPIE, 1614, 259—264 (1991).
Calculation Algorithm,” Appl. Opt., 26, 2504 (1987).
Blais, F. and Rioux, M., “BIRIS: A Simple 3—D Sensor,”
Hobrough, G. and Hobrough, T., “Stereopsis for robots by
Proc. SPIE, vol. 728, 235 (1986).
interative stereo image matching,” Proc. SPIE, vol. 449,
Boehnlein, A.J. and Harding, K.G., “Adaptation of a Parallel 94—102 (1983).
Architecture Computer to Phase Shifted Moire Interferom
Hornbeck, Larry, J., “Current Status of the Digital Micro
etry,” Proc. SPIE, vol. 728, 183—193 (1986).
mirror Device (DMD) for Projection Television Applica
Boehnlein, Albert, and Harding, Kevin G., “Field shift tions,” IEEE IEDM 93—381—384.
moiré, a neW technique for absolute range measurement,”
Hornbeck, Larry, J ., “Deformable—Mirror Spacial Light
Proc. SPIE, vol. 1163, 2—9 (1989).
Modulators,” Proc. SPIE, vol. 1150 (1989).
Bruning, J. H., Herriott, D.R., Gallagher, J.E., Rosenfeld,
Hornbeck, Larry, J ., “Digital Light Processing and MEMS:
D.P., White, A. D., and Brangaccio, D. J., “Digital Wave
Timely Convergence for a Bright Future,” available at
front Measuring Interferometer for Testing Optical Surfaces http://WWW.ti.com (1996).
and Lenses,” Appl. Opt., 13, 2693 (1974).
Huang, P.S., Hu, Q., Jin, F., and Chiang F.P., “Color—en
Carre, P., “Installation et utilisation du comparateur photo coded fringe projection and phase shifting for 3D surface
electrique et interferentiel du Bureau International des Poids contouring,” Proc. SPIE vol. 3407, 477—482 (1998).
et Mesures,” Metrologia 2, 13 (1996).
KujaWinska, M., “Use of Phase—Stepping Automatic Fringe
Chiang, F.P., “Moire Methods of Strain Analysis,” in Manual Analysis in Moire Interferometry,” Appl. Opt. 26(22),
on Experimental Stress Analysis, A.S. Kobayashi, Ed., Soc. 4712—4714 (1987).
for Exp. Stress Anal., Brook?eld Center, Ct, 1983, pp.
Moore, D.T. and Truax, B.E., “Phase—Locked Moire Fringe
51—69.
Analysis for Automated Contouring of Diffuse Surfaces,”
Chiang, F. P., “Moire Methods for Contouring, Displace Appl. Opt., 18(1), 91—96 (1979).
ment, De?ection, Slope, and Curvature,” Proc. SPIE, vol. Sampsell, J .B., “An OvervieW of the Performance Envelope
153, 113—119 (1978). of Digital—Micromirror—Device—Based Projection Display
Chiang, F.P., “Techniques of optical signal ?ltering parallel Systems,” SID Digest, 669—672 (1994).
to the processing of moiré—fringe patterns,” Exp. Mech.,
Sansoni, G., Docchio F., Minoni U., and Bussolati C.,
9(11), 523—526 (1969).
“Development and characterization of a liquid crystal pro
Creath, K. “Phase—Measurement Interferometry Tech jection unit for adaptive structured illumination,” Proc.
niques,” in Progress in Optics. vol. XXVI, E. Wolf, Ed., SPIE, 1614, 78—86 (1991).
Elsevier Science Publishers, Amsterdam, 1988, pp. Sansoni, G., Rodella, R., “Fast digitization of heritage
349—393).
objects by means of a 3D vision system based on the
Dai, Y.Z. and Chiang, F.P., “Moire interferometry applied to projection of structured light,” University of Brescia, pp.
topographic contour measurement,” Proc. SPIE, 954, 1—11.
(1988).
SchWider, J. BuroW, R., Elssner, K. E., Grzanna, J ., Spolac
Geng, Z. Jason, “RainboW three—dimensional camera: neW zyk, R., and Merkel K., “Digital Wavefront Measuring
concept of high—speed three—dimensional vision systems,” Interferometry: Some Systematic Error Sources,” Appl.
Optical Engineering, 35 (2), 376—383 (1996). Opt., 22, 3421 (1983).
US 6,788,210 B1
Page 3
ShaW, M.M., Harvey, D.M., Hobson, CA, and Lalor, M.J., WiZinoWich, P. L., “System for Phase Shifting Interferom
“Non—contact ranging using dynamic fringe projection,”
etry in the Presence of Vibration,” Proc. SPIE, 1164, 25
Proc. SPIE, vol. 1163, 22—29 (1989).
(1989).
Srinivasan, V. H., Liu, H. —C., and Halioua, M., “Automated
Phase—Measuring Pro?lometry of 3—D Diffuse Objects,” WiZinoWich, P. L., “Phase Shifting Interferometry in the
Appl. Opt., 23(18), 3015—3018 (1984).
Presence of Vibration: ANeW Algorithm and System,” Appl.
Srinivasan, V. H., Liu, H. —C., and Halioua, M., “Automated
Opt., 29, 3271 (1990).
Phase—Measuring Pro?lometry: a phase mapping
approach.” Appl. Opt., 24(2), 185—188 (1985).
Wyant, J .C., “Use of an ac heterodyne lateral shear inter
Stahl, Philip H., “White—light phase—measuring interferom
ferometer With real—time Wavefront correction systems,”
etry,” Proc. SPIE, vol. 1332, 720—728 (1990).
Stumpf, K.D., “Real—Time Interferometer,” Opt. Eng., 18, Appl. Opt., 14, 2622 (1975).
648 (1979).
YoshiZaWa, Toru and TomisaWa, Teiyu, “Moire topography
Svetkoff, D.J., Leonard, P.F., Sampson, RE, and Jain, R.,
With the aid of phase shift method,” Proc. SPIE, 1554B,
“Techniques for Real—Time, 3D, Feature Extraction Using
Range Information,” Proc. SPIE, vol. 521, 302 (1984). 441—450 (1991).
Takasaki, H., “Moire Topography,” Applied Optics 9 (6),
Younse, J .M., “Mirror on a chip,” IEEE Spectrum, 30(11),27
1457—1472 (1970).
(1993).
Toyooka, S. and IWaasa, Y., “Automatic Pro?lometry of 3—D
Diffuse Objects by Spatial Phase Detection,” Appl. Opt.,
25(10), 1630—1633 (1986). * cited by examiner
U.S. Patent Sep. 7, 2004 Sheet 1 0f 21 US 6,788,210 B1
wNas.N;\ _ \ mobqmwzmw
mozmwzwo
TUE
U.S. Patent Sep. 7, 2004 Sheet 2 0f 21 US 6,788,210 B1
F SET FRINGE PHASEA, PITCHp , f23 25
COLOR, AND AREA mxn /
‘V CALCULATE INTENSITY
COLOR-ENCODED
FRINGE PATTERN
SINUSOIDAL INTENSITY BINARY
PROFILE
?
27 29 31
TRIANGULAR
/ V / ‘ /
CALCULATE INTENSITY CALCULATE INTENSITY CALCULATE INTENSITY
33 GRAYSCALE 37
j v / v z
Ira) =10), {gm = lb(x) =0 1,.(0 =1gm=lb00= [(X) 1rc0=1bm = o. lg(x) =10)
I > _ _I
II
CIRCULAR
41
/ v
DRAW LINEAR FRINGE PATTERN DRAW CIRCULAR FRINGE PATrERN
LINE(x,0) - (m). Rcslsur ,lg ,Ib) C1RCLE(m /2,n/2),X, RGBUF ,lgJb)
LAST LINE
OR CIRCLE
?
YES
END
U.S. Patent Sep. 7, 2004 Sheet 3 0f 21 US 6,788,210 B1
/
O O O
\OOQ
O O ' \
O O
O O
[TUIIIHIHTHHIIIII
I,
FIG-3
24/
U.S. Patent Sep. 7, 2004 Sheet 4 0f 21 US 6,788,210 B1
FIG—4
30
-10° +10°
26
U.S. Patent Sep. 7, 2004 Sheet 5 0f 21 US 6,788,210 B1
@105
U.S. Patent Sep. 7, 2004 Sheet 6 0f 21 US 6,788,210 B1
0-0.
U.S. Patent
Sep. 7, 2004 Sheet 7 0f 21 US 6,788,210 B1