The F-Table: A Data Structure for Rendering Photo-Accurate Images of Faces from Experimentally ...

89 Pages·2015·6 MB·English

by  Hui Song

Florida State University Libraries
Electronic Theses, Treatises and Dissertations The Graduate School
2004
The F-Table: A Data Structure for
Rendering Photo-Accurate Images of Faces
from Experimentally Acquired Reflectance
Hui Song
Follow this and additional works at the FSU Digital Library. For more information, please contact [email protected]
THE FLORIDA STATE UNIVERSITY
COLLEGE OF ARTS AND SCIENCE
THE F-TABLE:
A DATA STRUCTURE FOR RENDERING PHOTO-ACCURATE IMAGES
OF FACES FROM EXPERIMENTALLY ACQUIRED REFLECTANCE
By
HUI SONG
A Dissertation submitted to the
Department of Computer Science
in partial fulfillment of the
requirements for the degree of
Master of Science
Degree Awarded:
Summer Semester, 2004
The members of the Committee approve the dissertation of Hui Song defended on July
3rd, 2004.

David Banks
Professor Directing Thesis

Anuj Srivastava
Committee Member

Xiuwen Liu
Committee Member

Approved:

Sudhir Aggarwal, Chair Department of Computer Science

Donald Foss, Dean, College of Arts and Science

The Office of Graduate Studies has verified and approved the above named committee
members.
ii
To Mama, Baba ...
iii
ACKNOWLEDGEMENTS
I am grateful to Dr. Myles Hollander of the Department of Statistics at Florida State
University for providing me with space to create the prototype for a Reflectance Laboratory.
It was in this facility that I conducted my initial experiments to measure the reflectance
of a surface. I thank Dr. Joe Travis, director of the School of Computational Science and
Information Technology for allowing me to use the space in the large Seminar Room in
the Dirac Science Library, and for upgrading the projection system for the powerWall to
increase the lumens by a factor of 10; without this technology I could not have conducted
this research.
I have had the distinct privilege of starting my thesis project the first year of graduate
school. Plus I have two great advisers. Dr. David Banks from the department of Computer
Science introduced me to the exciting world of computer graphics from drawing triangles on
the computer to using Open Inventor to building complicated applications. He also gave me
lots of wonderful ideas of how to use resources on research projects. Dr. Anuj Srivastava
from ithe department of Statistics taught me how to use statistical methods to analyze data
and fillin missing data using the existing data. Thank you both so much for the knowledge
and skills you gave me.
IamindebtedtoKevinBeasonforhisimplementationofHenrikJensen’s photonmapping
algorithm. Mr. Beason developed this software tool (called “pane”) in support of his
own masters thesis on photo-realistic rendering of isosurfaces for scientific visualization;
he graciously shared it with all of us in the Visualization Laboratory so that I could use
it to create photo-realistic renderings in our own research projects. He also modified pane
to incorporate my F-table, which allowed me to make side-by-side comparisons of different
reflectancefunctions. ThiscapabilityprovedtobecrucialforgatheringthedatainChapter4.
This research was supported by the Laboratory of Computational Visualization and the
CSIT Visualization Laboratory, and by NSF Grant FRG DMS-0101429 and NSF Grant
0083898.
iv
TABLE OF CONTENTS
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii
Abstract. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
1. INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Problem Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 The BRDF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Past Literature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.4 Our Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2. SAMPLING THE REFLECTANCE OF A SPHERE . . . . . . . . . . . . . . . . . 14
2.1 Measuring the Mean Reflectance of a Sphere . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.2 Determining the Local Spherical Coordinates at a Point on a Sphere . . . . . . . 18
2.3 Constructing the F-table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.4 Applying the F-table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3. METHODOLOGY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.1 Scanning the Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.1.1 Improving the Mesh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.1.2 Recovering Texture Coordinates in the Mesh . . . . . . . . . . . . . . . . . . . . . 27
3.2 Improving the Texture Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.2.1 Image Alignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.3 Light Calculation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.4 Automatic Acquisition of Reflectance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.4.1 Reducing Stray Light. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.4.2 Automating the powerDot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.4.3 Automating the Snapshots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.4.4 Reducing the Ambient Light . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.5 Filling the F-table for a Face . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.5.1 Sources of Error in Geometry and Color . . . . . . . . . . . . . . . . . . . . . . . . 42
3.5.2 Ray-casting the Mesh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.6 Summary of Tools Used. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4. RESULTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.1 The F-table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.2 BRDF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.3 Rendered Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
v
5. CONCLUSIONS AND FUTURE WORK. . . . . . . . . . . . . . . . . . . . . . . . . . . 60
APPENDIX A: Construct Texture Coordinates given information from
Minolta Vivid scanner software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
APPENDIX B: My program list . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
REFERENCES. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
BIOGRAPHICAL SKETCH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
vi
LIST OF TABLES
3.1 The output matrix M from scanalyze recorded in a .xf file. The upper-left
3×3 submatrix M (shown in bold face) of M defines the rotation in R3. The
r
top three rows of the right-most column form a vector that determines the
translation in R3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.2 Text formatofthetransposeMT ofthescanalyze matrixM. AnOpenInventor
object of type SbMatrix stores this array in order to transform the vertices of
the face. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.3 The cubic polynomial coefficients I used for each side of the powerWall. The
resulting brightness is Lˆ−1 (x) = a +a x+a x2 +a x3. . . . . . . . . . . . . . . . 36
actual 0 1 2 3
3.4 Pseudocode of the ray-casting process that fills the F-table. . . . . . . . . . . . . . . . 45
3.5 Tools used for acquiring the F-table and their cost . . . . . . . . . . . . . . . . . . . . . . 47
4.1 Results in table. Every cell shows a pair of images. The left one is the rendered
image using different BRDFs and the right one is the difference image (between
original-photographand the rendered image). The error metrics are also shown
in each cell. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
vii
LIST OF FIGURES
1.1 Photographs of the same person under different lighting. Images are from the
Yale Face Database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Minolta Vivid 3D scanner. Picture is from the manufacturer’s website:
http://www.minoltausa.com/vivid/products/vi700-en.asp. . . . . . . . . . . . . . . . . 3
1.3 Top: (left) Photograph of a face; (right) 3D mesh captured by the laser
range imaging system. Bottom: (left) conventional rendering of the face using
OpenGL, with a headlight shining forward onto the polygonal mesh and with
reflectance modeled as pure diffuse with equal components of red, green, and
blue; (right) OpenGL rendering of the same mesh with texture mapping applied. 4
1.4 Local spherical coordinates at a point on a surface. The surface normal points
up and to the right. The incident light shines down and left into the surface.
The exitant direction v points up and left in the image. The yellow arrows
out
define the local rectangular coordinate system of the tangent plane: one basis
vector is the normalized projection of v ; the other basis vector is orthogonal
in
to it. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.5 Left: goniometer for measuring a BRDF. The light source and the camera can
each pivot independently in order to sample four degrees of freedom for the
BRDF. Right: Ward’s BRDF measuring setup using a fisheye lens pointed at a
hemispherical mirror that collects light from a surface sample (positioned near
the lens). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.6 Left: theGonioreflectometerpresentedbySing-ChongFooinhismaster’sthesis
[1]. Right: a ball rendered using the measured blue spray paints BRDF. . . . . . 8
1.7 Marschner et al’s setup to measure reflectance of the forehead skin. . . . . . . . . 9
1.8 Different versions of Debevec’s light stage. Left: light stage 1 with a single
light source that whirls around spherical path enclosing the subject. Middle:
long-exposure photograph of the data-acquisition process. Right: light stage 2
withmultiple lightsourcesapproximatelydistributed aroundasphereenclosing
the subject. Images are from http://www.debevec.org. . . . . . . . . . . . . . . . . . . . 10
1.9 Han and Perlin’s kaleidoscope. Left: Schematic of optical components for
measuring surface reflectance. Middle: Experimental setup. The kaleidoscope
is the stack of triangular mirrors in the foreground, positioned atop the test
surface. Right: A multi-view image of a copper penny using the kaleidoscope.
Each triangular image represents a different view of the penny. . . . . . . . . . . . . 11
1.10 The powerWall in the CSIT seminar room. . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
viii
1.11 Our setup. The background is the powerWall we use as light source. The
object (mannequin head in this picture) is placed 1 meter in front of the light
source. A camera (Canon EOS 10D ) is placed in front of the object. A laptop
on the right side is used to drive the camera. Black cloth is used in the back to
block light from other directions. The two pictures show the setup before and
after we turn off the other light source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.12 Top row: A photograph of the scene. Middle row: OpenGL rendering of the
scene using local illumination. Soft shadows and inter-reflections present in
the photograph are absent in the rendering, producing a large pixel-by-pixel
difference. Bottom row: The same scene rendered using my collected BRDF
table and global illumination. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.1 Pink spray paint, three tones of CoverGirl make-up base, and powder applied
to a test sheet of white cardboard. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.2 Panorama of the setup in the makeshift Reflectance Laboratory. . . . . . . . . . . . 17
2.3 Photographs of the make-up ball from different viewing directions. . . . . . . . . . 18
2.4 Local coordinate system at a point on the make-up ball. Left: light directions
V (white arrow), V (blue arrow), and normal n (green arrow). Right: Basis
in out
vectors t and t (yellow) define the local spherical coordinate system of the
1 2
tangent space. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.5 Five slices (subtables) of the F-table with specific values of parameter φ .
out
Along the horizontal axis of each subtable I have θ ∈ [0,2π]; along the
out
vertical axis I have φ ∈ [0,π] (increasing downward). Top row: values in the
in
F-table corresponding to pixels in the photographs. Bottom row: Gaussian
smoothing of the data fills in missing regions in the table. . . . . . . . . . . . . . . . . 20
2.6 The F-map, with 1003 values. (total size 32MB). Left: Volume rendering of
F−tablewiththergbvalueasthecolormapandthegrayscalevalue(computed
using Equation 2.6) as the transparency. Right: One isosurface of F−table
(grayscale value = 0.2009) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.7 A sphere rendered with different render model. Left: Diffuse only reflectance.
Middle: Specular only (Phong model with specular exponent equals to 5).
Right: Rendered using acquired BRDF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.8 Left: photo of the make up ball. Right: a sphere rendered with the acquired
BRDF aligned with the photo. The mean square difference between these two
images is 0.0907 (in a scale ∈ [0,1]) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.1 The scanning rig: the Minolta Vivid 700 3D scanner is mounted on the
Manfrotto tripod. A desktop computer is also setup to drive the scanner. . . . . 25
3.2 The merged mesh generated by Minolta Vivid tool. There are obvious seams
where partial meshes are joined. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.3 Left: two meshes are registered with the registration information from scana-
lyze. Right: the optimized mesh output from vrip. . . . . . . . . . . . . . . . . . . . . . 28
ix