Handbook Of Face Recognition

405 Pages·2005·17.12 MB·English

by  Stan Z. Li

Handbook of Face Recognition
Stan Z. Li Anil K. Jain
Editors
Handbook of
Face Recognition
With 210 Illustrations
StanZ.Li AnilK.Jain
CenterforBiometricsResearchandTesting& DepartmentofComputerScience
NationalLabofPatternRecognition &Engineering
InstituteofAutomation MichiganStateUniversity
ChineseAcademyofSciences EastLansing,MI48824-1226
Beijing100080 USA
China [email protected]
[email protected]
LibraryofCongressCataloging-in-PublicationData
Handbookoffacerecognition/editors,StanZ.Li&AnilK.Jain.
p.cm.
Includesbibliographicalreferencesandindex.
ISBN0-387-40595-X(alk.paper)
1. Humanfacerecognition(Computerscience I. Li,S.Z.,1958– II. Jain,
AnilK.,1948–
TA1650.H36 2004
006.4′2—dc22 2004052453
ISBN0-387-40595-X Printedonacid-freepaper.
©2005SpringerScience+BusinessMedia,Inc.
All rights reserved. This work may not be translated or copied in whole or in part
without the written permission of the publisher (Springer Science+Business Media,
Inc., 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in con-
nectionwithreviewsorscholarlyanalysis.Useinconnectionwithanyformofinfor-
mation storage and retrieval, electronic adaptation, computer software, or by similar
ordissimilarmethodologynowknownorhereafterdevelopedisforbidden.
The use in this publication of trade names, trademarks, service marks, and similar
terms, even if they are not identified as such, is not to be taken as an expression of
opinionastowhetherornottheyaresubjecttoproprietaryrights.
PrintedintheUnitedStatesofAmerica. (MP)
9 8 7 6 5 4 3 2 1 SPIN10946602
springeronline.com
Preface
Facerecognitionhasalargenumberofapplications,includingsecurity,personverification,In-
ternetcommunication,andcomputerentertainment.Althoughresearchinautomaticfacerecog-
nitionhasbeenconductedsincethe1960s,thisproblemisstilllargelyunsolved.Recentyears
have seen significant progress in this area owing to advances in face modeling and analysis
techniques. Systems have been developed for face detection and tracking, but reliable face
recognitionstilloffersagreatchallengetocomputervisionandpatternrecognitionresearchers.
There are several reasons for recent increased interest in face recognition, including ris-
ing public concern for security, the need for identity verification in the digital world, and the
needforfaceanalysisandmodelingtechniquesinmultimediadatamanagementandcomputer
entertainment. Recent advances in automated face analysis, pattern recognition, and machine
learninghavemadeitpossibletodevelopautomaticfacerecognitionsystemstoaddressthese
applications.
Thisbookwaswrittenbasedontwoprimarymotivations.Thefirstwastheneedforhighly
reliable,accuratefacerecognitionalgorithmsandsystems.Thesecondwastherecentresearch
in image and object representation and matching that is of interest to face recognition re-
searchers.
Thebookisintendedforpractitionersandstudentswhoplantoworkinfacerecognitionor
whowanttobecomefamiliarwiththestate-of-the-artinfacerecognition.Italsoprovidesref-
erencesforscientistsandengineersworkinginimageprocessing,computervision,biometrics
andsecurity,Internetcommunications,computergraphics,animation,andthecomputergame
industry. The material fits the following categories: advanced tutorial, state-of-the-art survey,
andguidetocurrenttechnology.
The book consists of 16 chapters, covering all the subareas and major components nec-
essary for designing operational face recognition systems. Each chapter focuses on a specific
topicorsystemcomponent,introducesbackgroundinformation,reviewsup-to-datetechniques,
presentsresults,andpointsoutchallengesandfuturedirections.
Chapter1introducesfacerecognitionprocessing,includingmajorcomponentssuchasface
detection,tracking,alignment,andfeatureextraction,anditpointsoutthetechnicalchallenges
ofbuildingafacerecognitionsystem.Weemphasizetheimportanceofsubspaceanalysisand
learning,notonlyprovidinganunderstandingofthechallengesthereinbutalsothemostsuc-
VI Preface
cessfulsolutionsavailablesofar.Infact,mosttechnicalchaptersrepresentsubspacelearning-
basedtechniquesforvariousstepsinfacerecognition.
Chapter 2 reviews face detection techniques and describes effective statistical learning
methods. In particular, AdaBoost-based learning methods are described because they often
achieve practical and robust solutions. Techniques for dealing with nonfrontal face detection
arediscussed.Resultsarepresentedtocompareboostingalgorithmsandotherfactorsthataf-
fectfacedetectionperformance.
Chapters 3 and 4 discuss face modeling methods for face alignment. These chapters de-
scribe methods for localizing facial components (e.g., eyes, nose, mouth) and facial outlines
and for aligning facial shape and texture with the input image. Input face images may be ex-
tractedfromstaticimagesorvideosequences,andparameterscanbeextractedfromtheseinput
imagestodescribetheshapeandtextureofaface.Theseresultsarebasedlargelyonadvances
intheuseofactiveshapemodelsandactiveappearancemodels.
Chapters5and6covertopicsrelatedtoilluminationandcolor.Chapter5describesrecent
advances in illumination modeling for faces. The illumination invariant facial feature repre-
sentationisdescribed;thisrepresentationimprovestherecognitionperformanceundervarying
illuminationandinspiresfurtherexplorationsofreliablefacerecognitionsolutions.Chapter6
deals with facial skin color modeling, which is helpful when color is used for face detection
andtracking.
Chapter7providesatutorialonsubspacemodelingandlearning-baseddimensionreduction
methods,whicharefundamentaltomanycurrentfacerecognitiontechniques.Whereasthecol-
lectionofallimagesconstituteshighdimensionalspace,imagesoffacesresideinasubspaceof
thatspace.Facialimagesofanindividualareinasubspaceofthatsubspace.Itisofparamount
importance to discover such subspaces so as to extract effective features and construct robust
classifiers.
Chapter 8 addresses problems of face tracking and recognition from a video sequence of
images. The purpose is to make use of temporal constraints present in the sequence to make
trackingandrecognitionmorereliable.
Chapters 9 and 10 present methods for pose and illumination normalization and extract
effective facial features under such changes. Chapter 9 describes a model for extracting illu-
mination invariants, which were previously presented in Chapter 5. Chapter 9 also presents a
subregionmethodfordealingwithvariationinpose.Chapter10describesarecentinnovation,
calledMorphableModels,forgenerativemodelingandlearningoffaceimagesunderchanges
inilluminationandposeinananalysis-by-synthesisframework.Thisapproachresultsinalgo-
rithmsthat,inasense,generalizethealignmentalgorithmsdescribedinChapters3and4tothe
situationwherethefacesaresubjecttolargechangesinilluminationandpose.Inthiswork,the
three-dimensionaldataoffacesareusedduringthelearningphasetotrainthemodelinaddition
tothenormalintensityortextureimages.
Chapters 11 and 12 provide methods for facial expression analysis and synthesis. The
analysispart,Chapter11,automaticallyanalyzesandrecognizesfacialmotionsandfacialfea-
turechangesfromvisualinformation.Thesynthesispart,Chapter12,describestechniqueson
three-dimensionalfacemodelingandanimation,facelightingfromasingleimage,andfacial
expressionsynthesis.Thesetechniquescanpotentiallybeusedforfacerecognitionwithvary-
ing poses, illuminations, and facial expressions. They can also be used for human computer
interfaces.
Preface VII
Chapter13reviews27publiclyavailabledatabasesforfacerecognition,facedetection,and
facial expression analysis. These databases provide a common ground for development and
evaluation of algorithms for faces under variations in identity, face pose, illumination, facial
expression,age,occlusion,andfacialhair.
Chapter14introducesconceptsandmethodsforfaceverificationandidentificationperfor-
manceevaluation.ThechapterfocusesonmeasuresandprotocolsusedinFERETandFRVT
(facerecognitionvendortests).Analysisofthesetestsidentifiesadvancesofferedbystate-of-
the-arttechnologiesforfacerecognition,aswellasthelimitationsofthesetechnologies.
Chapter 15 offers psychological and neural perspectives suggesting how face recognition
might go on in the human brain. Combined findings suggest an image-based representation
thatencodesfacesrelativetoaglobalaverageandevaluatesdeviationsfromtheaverageasan
indicationoftheuniquepropertiesofindividualfaces.
Chapter 16 describes various face recognition applications, including face identification,
security,multimediamanagement,andhuman-computerinteraction.Thechapteralsoreviews
manyfacerecognitionsystemsanddiscussesrelatedissuesinapplicationsandbusiness.
Acknowledgments
Anumberofpeoplehelpedinmakingthisbookareality.VincentHsu,DirkColbry,Xiaoguang
Lu,KarthikNandakumar,andAnoopNamboodiriofMichiganStateUniversity,andShiguang
Shan,ZhenanSun,ChenghuaXuandJiangweiLioftheChineseAcademyofScienceshelped
proofread several of the chapters. We also thank Wayne Wheeler and Ann Kostant, editors at
Springer,fortheirsuggestionsandforkeepingusonschedulefortheproductionofthebook.
ThishandbookprojectwasdonepartlywhenStanLiwaswithMicrosoftResearchAsia.
December2004 StanZ.Li
Beijing,China
AnilK.Jain
EastLansing,Michigan

Contents
Chapter1.Introduction
StanZ.Li,AnilK.Jain ..................................................... 1
Chapter2.FaceDetection
StanZ.Li................................................................ 13
Chapter3.ModelingFacialShapeandAppearance
TimCootes,ChrisTaylor,HaizhuangKang,VladimirPetrovic´ ...................... 39
Chapter4.ParametricFaceModelingandTracking
Jo¨rgenAhlberg,FadiDornaika .............................................. 65
Chapter5.IlluminationModelingforFaceRecognition
RonenBasri,DavidJacobs.................................................. 89
Chapter6.FacialSkinColorModeling
J.BirgittaMartinkauppi,MattiPietika¨inen ..................................... 113
ColorPlatesforChapters6and15
........................................................................ 137
Chapter7.FaceRecognitioninSubspaces
GregoryShakhnarovich,BabackMoghaddam ................................... 141
Chapter8.FaceTrackingandRecognitionfromVideo
RamaChellappa,ShaohuaKevinZhou ........................................ 169
Chapter9.FaceRecognitionAcrossPoseandIllumination
RalphGross,SimonBaker,IainMatthews,TakeoKanade ......................... 193
Chapter10.MorphableModelsofFaces
SamiRomdhani,VolkerBlanz,CurzioBasso,ThomasVetter ...................... 217
X Contents
Chapter11.FacialExpressionAnalysis
Ying-LiTian,TakeoKanade,JeffreyF.Cohn .................................... 247
Chapter12.FaceSynthesis
ZichengLiu,BainingGuo .................................................. 277
Chapter13.FaceDatabases
RalphGross.............................................................. 301
Chapter14.EvaluationMethodsinFaceRecognition
P.JonathonPhillips,PatrickGrother,RossMicheals ............................. 329
Chapter15.PsychologicalandNeuralPerspectivesonHumanFaceRecognition
AliceJ.O’Toole........................................................... 349
Chapter16.FaceRecognitionApplications
ThomasHuang,ZiyouXiong,ZhenqiuZhang ................................... 371
Index................................................................... 391