Linear Disentangled Representation Learning for Facial Actions

1.7 MB·

by  Xiang Xiang

LINEARDISENTANGLEDREPRESENTATIONLEARNINGFORFACIALACTIONS
XiangXiang1 andTracD.Tran2
1Dept. ofComputerScience 2Dept. ofElectrical&ComputerEngineering
JohnsHopkinsUniversity,3400N.CharlesStreet,Baltimore,MD21218,USA
7
1
0
2

n
a
J

1 Fig.1. Theseparabilityoftheneutralfacey andexpression
n
1 component y . We find y is better for identity recognition
e n

] thanyandyeisbetterforexpressionrecognitionthany.
V
Fig. 2. Action unit number and FACS name shown using
C
ABSTRACT imagesfromMPI-VDBwith27distinctAUs.Thepeakframe
s. isshown. AU12LandAU12Raredistinct;similarforAU14.
Limitedannotateddataavailablefortherecognitionoffacial
c
[ expression and action units embarrasses the training of deep

networks, which can learn disentangled invariant features. units (AU, see Fig. 2). The widely-used 6 basic emotions
1
v However, a linear model with just several parameters nor- defined by Paul Ekman include surprise, sadness, disgust,
2 mallyisnotdemandingintermsoftrainingdata.Inthispaper, anger,fear andhappiness. HealsodefinestheFacialAction
0 weproposeanelegantlinearmodeltountangleconfounding Coding System (FACS), using which we can code almost
1 factors in challenging realistic multichannel signals such as any expression. Recently, feature learning [1, 2, 3] using
3
2D face videos. The simple yet powerful model does not auto-encoder or adversarial training has shown to be able to
0
. rely on huge training data and is natural for recognizing disentangle facial expression from identity and pose. Unlike
1 facial actions without explicitly disentangling the identity. face recognition, limited labelled training data are available
0
Base on well-understood intuitive linear models such as forfacialexpressionsandAUsinparticular.
7
1 Sparse Representation based Classification (SRC), previous As shown in Fig.1, an expressive face can be separated
: attemptsrequireaprepossessingofexplicitdecouplingwhich into a principal component of the neutral face encoding
v
i is practically inexact. Instead, we exploit the low-rank identitycuesandanactioncomponentencodingmotioncues
X property across frames to subtract the underlying neutral such as the highlighted brow, cheek, lip, lid and nose which
r faces which are modeled jointly with sparse representation relatetoAUsinFACS.Asrecognitionisalwaysbrokendown
a
on the action components with group sparsity enforced. intomeasuringsimilarity[4],asimilaridentitycanconfusea
On the extended Cohn-Kanade dataset (CK+), our one-shot similaraction. Todecouplethem,[5]firstrulesouttheneural
automaticmethodonrawfacevideosperformsascompetitive faceexplicitlyandthendiscriminatebetweendifferentaction
asSRCappliedonmanuallypreparedactioncomponentsand components[6]insteadofrawfaces[7].Thefirststepisbased
performsevenbetterthanSRCintermsoftruepositiverate. on the observation that the underlying neutral face stays the
We apply the model to the even more challenging task of same. Ifwestackvectorsofneutralfacesoverthetimeofan
facialactionunitrecognition,verifiedontheMPIFaceVideo action as a matrix, it should be low-rank, ideally with rank
Database (MPI-VDB) achieving a decent performance. All 1. While theoretically the low-rank Principal Component
theprogramsanddatahavebeenmadepubliclyavailable. Pursuit [8] can be exact under certain conditions, it is of
approximate nature in practice. Their second step is based
1. INTRODUCTION
on the idea of describing an action component as a sparse
Inthispaper,theproblemisrecognizingfacialactionsgiven representation over an over-complete dictionary formed by
a face video and action categories in the granularity of actioncomponentsofallcategories.
either the holistic expression (emotion, see Fig.1) or action Our intuition is to remain both facets in a joint manner.
Foronething,weimplicitlygetridoftheneutralface.Foran- tionerror,whichcanbeseenasamax-marginclassifier.
other,weuseequally-spacedsampledframes,sinceallframes Particularly for facial actions, we treat videos as multi-
collaboratively yet redundantly represent the expression and channelsignals[10,19],differentfromimage-basedmethods
neutral face. Then, the sparse coefficient vectors form a [5, 6]. [5] explicitly separates the neutral face and action
joint-sparsecoefficientmatrix. Thatdrivesustoinduceboth component, and then exploits the class-wise sparsity sepa-
the joint Sparse representation [9, 10] and the implicit Low- rately for the recognition of identity from neutral faces and
Rankapproximation[8]inonemodel(SLR)[11],whichalso expression from action components. Differently, with the
inducesconsistentclassificationsacrossframes. focus of facial actions we exploit the low-rank property for
Furthermore,ideallynon-zerocoefficientsalldroptothe disentanglingidentityaswellasstructuredsparsitybyinter-
ground-truthcategory. Therefore,theclass-levelsparsityis1 channel observation. Furthermore, there is tradeoff between
and the coefficient matrix exhibits group sparsity. However, simplicityandperformance. Asvideosaresequentialsignals,
coefficientvectorsshareclass-wiseyetnotnecessarilyatom- the above appearance-based methods including ours cannot
wisesparsity[12]. Thus,wepreferenforcingboththegroup modelthedynamicsgivenbyatemporalmodel[20]orspatio-
sparsity and atom-wise sparsity. We name this extended temporal models [21, 22, 23]. Other linear models include
model Collaborative-Hierarchical Sparse and Low-Rank (C- ordinalregression[24,25,26]andboosting[27].
HiSLR)model[11]followingthenamingofC-HiLasso[12].
3. LINEARREPRESENTATIONMODEL
InSec. 2, wereviewtheclassicideaoflearningasparse
representationforclassificationanditsrelatedapplicationon In this section, we explain how to model X using Y and
expression recognition. In the remainder, we first elaborate training data D, which contains K ∈ Z+ types of actions.
ourmodelinSec. 3,thendiscusssolvingthemodelviajoint WewouldliketoclassifyatestvideoasoneoftheK classes.
optimizationinSec. 4,andfinallyquantitativelyevaluateour 3.1. SLR:jointSparserepresentationandLow-Rankness
modelinSec. 5,withaconclusionfollowedinSec. 6. Firstofall,weneedanexplicitrepresentationYofanexpres-
sive face. The matrix Y ∈ Rd×τ can be an arrangement of
2. RELATEDWORKS d-dimensionalfeaturevectorsy ∈ Rd (i = 1,2,...,τ)ofthe
(cid:2) (cid:3)
Among non-linear models, one line of work is kernel-based τ frames: Y = Y |...|Y . Weemphasizeourmodel’s
1 τ d×τ
methods [13] while another is deep learning [14, 15, 16, 1]. powerbysimplyusingtherawpixelintensities.
Similarideaswithdisentanglingfactorshavebeenpresented Now,weseekanimplicitlatentrepresentationX∈Rn×τ
in[3,2,1].Byintroducingextracues,onelineofworksis3D ofaninputtestface’semotionY ∈ Rd×τ asasparselinear
e
models[17]whileanotherismulti-modalmodels[18].Butin combinationofpreparedfixedtrainingemotionsD∈Rd×n:
thelinearworld,observingarandomsignalyforrecognition, Y =DX.
e
we just hope to send the classifier a discriminative compact Since an expressive face y = y +y is a superposition of
e n
representation x over a dictionary D such that Dx=y. anemotiony ∈Rdandaneutralfacey ∈Rd,wehave
e n
Normallyxiscomputedbypursuingthebestreconstruction. Y =Y +L,
e
For example, when D is under-complete (skinny), a closed- whereL ∈ Rd×τ isideallyτ-timesrepetitionofthecolumn
formapproximatesolutioncanbeobtainedbyLeast-Squares: vector of a neutral face y ∈ Rd. Presumably L =
n
x∗ =argmin (cid:107)y−Dx(cid:107)2 ≈(DTD)−1DTy. (cid:2)y |...|y (cid:3) . AsshowninFig. 3,Xsubjectsto
x 2 n n d×τ
When D is over-complete (fat), add a Tikhonov regularizer: Y =DX+L,
x∗ =argminx(cid:107)y−Dx(cid:107)22+λr(cid:107)√x(cid:107)22 =argminx(cid:107)y(cid:101)−D(cid:101)x(cid:107)22 where the dictionary matrix Dd×n is an arrangement of all
where y(cid:101) =[y,0]T and D(cid:101) =[D, λrI]T is under-complete. sub-matrices D[j], j = 1,...,(cid:98)nτ(cid:99). Only for training, we
Notably,x∗ =(DTD+λ I)−1DTyisgenerallydense. have(cid:98)n(cid:99)trainingemotionswithneutralfacessubtracted.The
r τ
Alternatively, we can seek a sparse usage of D. Sparse above constraint of X characterizes an affine transformation
Representation based Classification [9] (SRC) expresses a fromthelatentrepresentationXtotheobservationY.
test sample y as a weighted linear combination y=Dx of IfwewriteXandYinthehomogeneousform,wehave
(cid:20) (cid:21) (cid:20) (cid:0) (cid:1) (cid:21) (cid:20) (cid:21)
trainingsamplessimplystackedcolumnwiseinthedictionary Yd×τ = Dd×n yn d×1 × Xn×τ .
D. Presumably, non-zero weight coefficients drop to the 11×τ 01×n 1 11×τ
ground-truthclass, whichinducesasparsecoefficientvector Intheidealcasewithrank(L) = 1,iftheneutralfacey is
n
or the so-called sparse representation. In practice, non-zero pre-obtained [6, 5], it is trival to solve for X. Normally, y
n
coefficients also drop to other classes due to noises and isunknownandLisnotwithrank1duetonoises. AsXis
correlationsamongclasses. Onceaddinganerrorterme,we supposedtobesparseandrank(L)isexpectedtobeassmall
canformandictionaryD(cid:101) whichisalwaysover-complete: aspossible(maybeeven1),intuitivelyourobjectiveisto
[x∗,e∗]T =argmin sparsity(x) min sparsity(X)+λ ·rank(L),
x(cid:101) (cid:20) (cid:21)(cid:101) X,L L
x where rank(L) can be seen as the sparsity of the vector
s.t. y=Dx+e=[D|I]× e =D(cid:101)x(cid:101). formed by the singular values of L. Here λ is a non-
L
SRCevaluateswhichclassleadstotheminimumreconstruc- negativeweightingparameterweneedtotune.Whenλ =0,
L
astheSchatten(cid:96) matrixnormandcanbeseenasagroup’s
2
magnitude. λ isanon-negativeweightingparameterforthe
G
groupregularizer,whichisgeneralizedfroman(cid:96) regularizer
1
(cid:8) (cid:9)
(considerG = {1},{2},...,{n} forsingletongroups)[12].
Whenλ = 0,C-HiSLRdegeneratesintoSLR.Whenλ =
G L
0,wegetbacktocollaborativeSparseGroupLasso.
3.3. Classification
FollowingSRC,foreachclassc∈{1,2,...,K},letD de-
[Gc]
Fig. 3. Pictorial illustration of the linear constraint of the notethesub-matrixofDwhichconsistsofallthecolumnsof
proposedmodelshownfordisgust. Dispreparedandsimply DthatcorrespondtoemotionclasscandsimilarlyforX[Gc].
fixed. Dependingontheobjectvethemodelhastwoversions. WeclassifyYbyassigningittotheclasswithminimalresid-
ualasc∗ =argmin r (Y):=||Y−D X −L|| .
c c [Gc] [Gc] F
4. OPTIMIZATION
BothSLRandC-HiSLRmodelscanbeseenassolving
minf(X)+λL(cid:107)L(cid:107)∗ s.t. Y =DX+L (3)
X,L
TofollowastandarditerativeADMMprocedure,wewrite
downtheaugmentedLagrangianfunctionfor(3)as
Fig.4.PictorialillustrationofthegroupsparsityinC-HiSLR. L(X,L,Λ)=f(X)+λ ||L||
L ∗
β (4)
+(cid:104)Λ,Y−DX−L(cid:105)+ ||Y−DX−L||2,
2 F
the optimization problem reduces to that in SRC. With both
terms relaxed to be convex norms, we alternatively solve where Λ is the matrix of multipliers, (cid:104)·,·(cid:105) is inner product,
min (cid:107)X(cid:107) +λ (cid:107)L(cid:107) , and β is a positive weighting parameter for the penalty
X,L 1 L ∗
where(cid:107)·(cid:107) istheentry-wise(cid:96) matrixnorm,whereas(cid:107)·(cid:107) is (augmentation). Asingleupdateatthek-thiterationincludes
1 1 ∗
cthaenSbcehasteteenn(cid:96)a1smaaptprilxyinnogrm(cid:96)1(nnuocrlmeartonotrhme,vtreaccteornoorfms)inwghuilcahr Lk+1=argLminλL||L||∗+ β2||Y−DXk−L+ β1Λk||2F (5)
values. Now,theproposedjointSLRmodelisexpressedas β 1
min(cid:107)X(cid:107)1+λL(cid:107)L(cid:107)∗ s.t. Y =DX+L (1) Xk+1=argXminf(X)+ 2||Y−DX−Lk+1+ βΛk||2F (6)
X,L Λk+1=Λk+β(Y−DXk+1−Lk+1). (7)
We solve (1) for matrices X and L by the Alternating Thesub-stepofsolving(5)hasaclosed-formsolution:
DirectionMethodofMultipliers(ADMM)(seeSec. 4). 1
L =D (Y−DX + Λ ), (8)
k+1 λL k β k
3.2. C-HiSLR:aCollaborative-HierarchicalSLRmodel β
whereDistheshrinkagethresholdingoperator.InSLRwhere
If there is no low-rank term L, (1) becomes a problem of
f(X) = (cid:107)X(cid:107) , (6) is a Lasso problem, which we solve by
1
multi-channelLasso(LeastAbsoluteShrinkageandSelection
using the Illinois fast solver. When f(X) follows (2) of C-
Operator). For a single-channel signal, Group Lasso has
HiSLR, computing X needs an approximation based on
k+1
explored the group structure for Lasso yet does not enforce
the Taylor expansion at X [28, 12]. We refer the reader to
k
sparsitywithinagroup, whileSparseGroupLassoyieldsan
[12]fortheconvergenceanalysisandrecoveryguarantee.
atom-wise sparsity as well as a group sparsity. Then, [12]
5. EXPERIMENTALRESULTS
extends Sparse Group Lasso to multichannel, resulting in a
Collaborative-Hierarchical Lasso (C-HiLasso) model. For Weevaluateourmodelonexpressions(CK+)andactionunits
our problem, we do need L, which induces a Collaborative- (MPI-VDB). Images are cropped using the Viola-Jones face
HierarchicalSparseandLow-Rank(C-HiSLR)model: detector. Percategoryaccuraciesareaveragedover20runs.
(cid:88) 5.1. Holisticfacialexpressionrecognition
min(cid:107)X(cid:107) +λ (cid:107)L(cid:107) +λ (cid:107)X (cid:107)
X,L 1 L ∗ GG∈G [G] F (2) ExperimentsareconductedontheCK+dataset[29]consist-
ing of 321 videos with labels1. For SLR and C-HiSLR, we
s.t. Y =DX+L
assumenopriorknowledgeoftheneutralface. Atestingunit
whereX[G] isthesub-matrixformedbyalltherowsindexed containsthelast(τ −1)framestogetherwiththefirstframe,
tst
bytheelementsingroupG ⊆ {1,...,n}. AsshowninFig. 4, whichisnotexplicitlyknownaprioriasaneutralface. But
given a group G of indices, the sub-dictionary of columns
indexed by G is denoted as D . G = {G ,...,G } is a 1Contempt is discarded in [5, 6] due to its confusion with anger and
[G] 1 K disgustbutwechoosetokeepitisforthecompletenessoftheexperimenton
non-overlapping partition of {1,...,n}. Here (cid:107)·(cid:107) denotes
F CK+.Seehttps://github.com/eglxiang/icassp15_emotion
theFrobeniusnorm,whichistheentry-wise(cid:96)2 normaswell forcroppedfacedataandprogramsofC-HiSLR,SLR,SRCandEigenface.
An Co Di Fe Ha Sa Su
An 0.77 0.01 0.09 0.02 0 0.07 0.04
Co 0.08 0.84 0 0 0.03 0.04 0
Di 0.05 0 0.93 0.01 0.01 0.01 0
Fe 0.09 0.01 0.03 0.53 0.12 0.07 0.15
Ha 0.01 0.02 0.01 0.02 0.93 0 0.03
Sa 0.19 0.02 0.02 0.05 0 0.65 0.07
Su 0 0.02 0 0.02 0 0.02 0.95
Table 1. Confusion matrix for C-HiSLR on CK+ without
explicitlyknowingneutralfaces. Theoptimizerrunsfor600
iterationsandtherecognitionrateis0.80withastdof0.05.
Fig.5. Effectofgroupsparsity. τ =8. (a)isthetestinput
trn
Y.(b)(c)arerecoveredLandDX,givenbyC-HiSLRwhich An Co Di Fe Ha Sa Su
correctlyclassifies(a)ascontempt.(d)(e)arerecoveryresults An 0.51 0 0.10 0.02 0 0.31 0.06
givenbySLRwhichmis-classifies(a)assadness. (i),(ii),(iii) Co 0.03 0.63 0.03 0 0.04 0.26 0.01
denote results of frame #1, #4, #8 respectively, whereas (iv) Di 0.04 0 0.74 0.02 0.01 0.15 0.04
displays the recovered X (left for C-HiSLR and right for Fe 0.08 0 0.01 0.51 0.03 0.19 0.18
SLR).XgivenbyC-HiSLRisgroup-sparseasweexpected. Ha 0 0.01 0 0.03 0.85 0.08 0.03
Sa 0.09 0 0.04 0.04 0 0.70 0.13
Su 0 0.01 0 0.02 0.01 0.02 0.94
forformingthedictionary,wesubtractthefirstframefromthe Table 2. Confusion matrix for SLR on CK+. We let the
lastτtrnframespervideo.Theparametersaresetasτtrn =8, optimizer run for 100 iters and Lasso run for 100 iters. The
τtst = 8, λL = 10 and λG = 4.5. We randomly choose totalrecognitionrateis0.70withastdof0.14.
10 videos for training and 5 for testing per class. Fig. 5
visualizestherecoveryresultsgivenbyC-HiSLR.Table1and
2 present their confusion matrix, respectively. Columns are (i.e., forming dictionary) and the other 5 for testing (namely
predictions and rows are ground truths. Table 4 summarizes τtrn =5andτtst =5).Inthisway,thedatasetisdividedinto
the true positive rate (i.e., sensitivity). We have anticipated atrainingsetandadisjointtestingsetbothwith5sub-videos
that SLR (0.70) performs worse than SRC (0.80) since SRC per category. When λL = 15, SLR’s performance is shown
is equipped with neutral faces. However, C-HiSLR’s result in Fig. 6 with an average recognition rate of 0.80. When
(0.80) is comparable with SRC’s. C-HiSLR performs even λG = 4.5, C-HiSLR’s performance is shown in Fig. with a
better in terms of sensitivity, which verifies that the group average recognition rate of 0.84. They both perform poorly
sparsityindeedbooststheperformance. on AU10R (right upper lip raiser), which confuse with 12R
Asacomparsion,wereplicatetheimage-basedSRCused (rightlipcorner),13(cheekpuffer),14R(rightdimpler)and
in [5, 6, 7] and assume the neutral face is provided. We 15(lipcornerdepressor)becausetheyareallaboutlips.
representanactioncomponentbysubtractingtheneutralface 6. CONCLUSION
which is the first frame from the last frame per video. We In this paper, we propose a identity-decoupled linear model
choosehalfofCK+fortrainingandtheotherhalffortesting to learn a facial action representation, unlike [6] requiring
per class. When sparsity level is set to 35%, SRC achieves neutralfacesasinputsand[5]generatinglabelsoftheidentity
a recognition rate of 0.80 shown by Table 3. Accuracies for andfacialactionasmutualby-productsyetwithextraefforts.
fear&sadarelowastheyconfuseeachother.
5.2. Facialactionunitrecognition
To be pose-independant, the following experiments are con-
ducted on a profile view of MPI-VDB 2 containing 27 long
videoallwithover100frames(1videopercategory,seeFig.
2).Fromeachvideowesample10disjointsub-videoseachof
whichcontains10equally-spacedsampledframes. Different
fromSec.5.1,allframesaredirectlyusedwithoutsubtracting
the first frame as the sub-videos do not start with neutral
states. However, there implicitly exist underlying neural Fig.6. ConfusionmatrixforSLR(left)andCHi-SLR(right)
statesandpresumablytheproposedmodelisstillvalid. Then on27AUsfromMPI-VDB.SLRachievesarecognitionrate
we randomly sample 5 sub-videos from the 10 for training of0.80whentheoptimizerrunsfor30itersandLassorunfor
3iters.CHi-SLRachievesarecognitionrateof0.84whenthe
2Seehttp://vdb.kyb.tuebingen.mpg.defortherawdataand
optimizerrunsfor1000iters. Inbothcasethestdis0.04.
https://github.com/eglxiang/FacialAUforcroppedfacedata.
An Co Di Fe Ha Sa Su [9] John Wright, Allen Y. Yang, Arvind Ganesh, Shankar S Sastry, and
An 0.71 0.01 0.07 0.02 0.01 0.03 0.16 YiMa, “Robustfacerecognitionviasparserepresentation,” IEEET-
PAMI,vol.31,no.2,pp.210–227,2009.2,5
Co 0.07 0.60 0.02 0 0.16 0.03 0.12
Di 0.04 0 0.93 0.02 0.01 0 0 [10] Yonina C. Eldar and Holger Rauhut, “Average case analysis of
multichannelsparserecoveryusingconvexrelaxation,” IEEETrans.
Fe 0.16 0 0.09 0.25 0.25 0 0.26
Inf.Theory,vol.56,no.1,pp.505–519,2010.2
Ha 0.01 0 0 0.01 0.96 0 0.02
[11] Xiang Xiang, Minh Dao, Gregory D Hager, and Trac D Tran,
Sa 0.22 0 0.13 0.01 0.04 0.24 0.35
“Hierarchical sparse and collaborative low-rank representation for
Su 0 0.01 0 0 0.01 0 0.98 emotionrecognition,”inICASSP.IEEE,2015,pp.3811–3815.2
Table3. ConfusionmatrixforSRC[9]onCK+withneutral [12] Pablo Sprechmann, Ignacio Ramrez, Guillermo Sapiro, and Yonina
facesexplicitlyprovided. Recognitionrateis0.80(std:0.05). Eldar, “C-HiLasso: A collaborative hierarchical sparse modeling
framework,” IEEETrans.Sig.Proc., vol.59, no.9, pp.4183–4198,
2011.2,3
Model An Co Di Fe Ha Sa Su
[13] C. Fabian Benitez-Quiroz, Ramprakash Srinivasan, and Aleix M.
SRC 0.71 0.60 0.93 0.25 0.96 0.24 0.98
Martinez, “Emotionet: An accurate, real-time algorithm for the
SLR 0.51 0.63 0.74 0.51 0.85 0.70 0.94 automaticannotationofamillionfacialexpressionsinthewild,” in
C-HiSLR 0.77 0.84 0.93 0.53 0.93 0.65 0.95 CVPR,June2016.2
Table 4. Comparison of sensitivity. The bold and italics [14] Xiangyun Zhao, Xiaodan Liang, Luoqi Liu, Teng Li, Yugang Han,
NunoVasconcelos,andShuichengYan, “Peak-piloteddeepnetwork
denotethehighestandlowestrespectively. Differencewithin
forfacialexpressionrecognition,”inECCV,2016,pp.425–442.2
0.05istreatedascomparable. C-HiSLRperformsthebest.
[15] Heechul Jung, Sihaeng Lee, Junho Yim, Sunjeong Park, and Junmo
Kim, “Jointfine-tuningindeepneuralnetworksforfacialexpression
recognition,”inICCV,December2015.2
Our contribution is two-fold. First, we do not recover
[16] PingLiu,ShizhongHan,ZiboMeng,andYanTong,“Facialexpression
the action component explicitly. Instead, the video-based
recognitionviaaboosteddeepbeliefnetwork,”inCVPR,2014.2
sparse representation is jointly modelled with the low-rank
[17] HuiChen,JiangdongLi,FengjunZhang,YangLi,andHonganWang,
property across frames so that the neutral face underneath “3dmodel-basedcontinuousemotionrecognition,”inCVPR,2015.2
is automatically subtracted. Second, we preserve the label [18] ZhengZhang,JeffM.Girard,YueWu,XingZhang,PengLiu,Umur
consistencybyenforcingatom-wiseaswellasgroupsparsity. Ciftci,ShaunCanavan,MichaelReale,AndyHorowitz,HuiyuanYang,
JeffreyF.Cohn, QiangJi, andLijunYin, “Multimodalspontaneous
For the CK+ dataset, C-HiSLR’s performance on raw faces
emotioncorpusforhumanbehavioranalysis,”inCVPR,June2016.2
is comparable with SRC given neutral faces, which verifies
[19] KailiZhao,Wen-ShengChu,FernandoDelaTorre,JeffreyF.Cohn,
thatactioncomponentsareautomaticallyseparablefromraw
andHonggangZhang, “Jointpatchandmulti-labellearningforfacial
faces as well as sparsely representable by training data. We actionunitdetection,”inCVPR,June2015.2
alsoapplythemodelonrecognizingactionsunitswithlimited [20] Arnaud Dapogny, Kevin Bailly, and Severine Dubuisson, “Pairwise
trainingdata,whichmayembarrassdeeplearningtechniques. conditionalrandomforestsforfacialexpressionrecognition,”inICCV,
2015.2
[21] Mengyi Liu, Shiguang Shan, Ruiping Wang, and Xilin Chen,
7. REFERENCES
“Learning expressionlets on spatio-temporal manifold for dynamic
[1] Salah Rifai, Yoshua Bengio, Aaron Courville, Pascal Vincent, and facialexpressionrecognition,”inCVPR,2014,pp.1749–1756.2
MehdiMirza, “Disentanglingfactorsofvariationforfacialexpression [22] Ziheng Wang, Shangfei Wang, and Qiang Ji, “Capturing complex
recognition,”inECCV.Springer,2012,pp.808–822.1,2 spatio-temporal relations among facial muscles for facial expression
recognition,”inCVPR,2013,pp.3422–3429.2
[2] ScottReed,KihyukSohn,YutingZhang,andHonglakLee, “Learning
todisentanglefactorsofvariationwithmanifoldinteraction,”inICML, [23] Yimo Guo, Guoying Zhao, and Matti Pietika¨inen, “Dynamic facial
2014,pp.1431–1439.1,2 expressionrecognitionusinglongitudinalfacialexpressionatlases,” in
ECCV,pp.631–644.Springer,2012.2
[3] Ping Liu, Joey Tianyi Zhou, Ivor Wai-Hung Tsang, Zibo Meng,
ShizhongHan,andYanTong,“Featuredisentanglingmachine-anovel [24] RuiZhao,QuanGan,ShangfeiWang,andQiangJi,“Facialexpression
approach of feature selection and disentangling in facial expression intensityestimationusingordinalinformation,”inCVPR,June2016.2
analysis,”inECCV.Springer,2014,pp.151–166.1,2 [25] OgnjenRudovic,VladimirPavlovic,andMajaPantic, “Multi-output
[4] Xiang Xiang and Trac D. Tran, “Pose-selective max pooling for laplaciandynamicordinalregressionforfacialexpressionrecognition
measuringsimilarity,”inICPRworkshops,2016.1 andintensityestimation,”inCVPR.IEEE,2012,pp.2634–2641.2
[5] SimaTaheri,VishaM.Patel,andRamaChellappa,“Component-based [26] Minyoung Kim and Vladimir Pavlovic, “Structured output ordinal
recognitionoffacesandfacialexpressions,” IEEETrans.onAffective regressionfordynamicfacialemotionintensityprediction,” inECCV.
Computing,vol.4,no.4,pp.360–371,2013.1,2,3,4 Springer,2010,pp.649–662.2
[27] PengYang,QingshanLiu,andDimitrisNMetaxas, “Exploringfacial
[6] StefanosZafeiriouandMariaPetrou,“Sparserepresentationsforfacial
expressionswithcompositionalfeatures,”inCVPR.IEEE,2010.2
expressionsrecognitionvial1optimization,”inIEEECVPRWorkshop,
2010.1,2,3,4 [28] Minh Dao, Nam H Nguyen, Nasser M Nasrabadi, and Trac D
Tran, “Collaborative multi-sensor classification via sparsity-based
[7] RaymondPtucha,GrigoriosTsagkatakis,andAndreasSavakis,“Man-
representation,”IEEETrans.onSig.Proc.,vol.64,no.9.3
ifold based sparse representation for robust expression recognition
withoutneutralsubtraction,”inIEEEICCVWorkshops,2011.1,4 [29] PatrickLucey,JeffreyF.Cohn,TakeoKanade,JasonSaragih,andZara
Ambadar, “TheExtendedCohn-KanadeDataset(CK+): Acomplete
[8] EmmanuelJ.Candes,XiaodongLi,YiMa,andJohnWright, “Robust
dataset for action unit and emotion-specified expression,” in IEEE
principalcomponentanalysis?,”JournaloftheACM,vol.58,no.3,pp.
CVPR,2010.3
1–37,2011.1,2