HMOG: New Behavioral Biometric Features for Continuous Authentication of Smartphone Users

2.9 MB·English

by  Zdenka Sitova

1
HMOG: New Behavioral Biometric Features for
Continuous Authentication of Smartphone Users*
Zdenˇka Sitova´† Jaroslav Sˇedeˇnka† Qing Yang‡# Ge Peng‡ Gang Zhou‡ Paolo Gasti† Kiran S. Balagani†(cid:91)
†New York Institute of Technology – {sitovaz, sedenka}@mail.muni.cz, {pgasti, kbalagan}@nyit.edu
‡College of William and Mary – {qyang, gpeng, gzhou}@cs.wm.edu
6 Abstract—We introduce Hand Movement, Orientation, and I. INTRODUCTION
1 Grasp (HMOG), a set of behavioral features to continuously
0 authenticate smartphone users. HMOG features unobtrusively Currently, popular smartphone authentication mechanisms
2 capturesubtlemicro-movementandorientationdynamicsresult- suchasPINs,graphicalpasswords,andfingerprintscansoffer
ing from how a user grasps, holds, and taps on the smartphone. limited security. They are susceptible to guessing [16] (or
n Weevaluatedauthenticationandbiometrickeygeneration(BKG)
a performance of HMOG features on data collected from 100 spoofing [59] in the case of fingerprint scans), and to side
J subjects typing on a virtual keyboard. Data was collected under channel attacks such as smudge [4], reflection [66], and video

5 two conditions: sitting and walking. We achieved authentication capture [57] attacks. Additionally, a fundamental limitation
2 EERs as low as 7.16% (walking) and 10.05% (sitting) when we of PINs, passwords, and fingerprint scans is that they are
combined HMOG, tap, and keystroke features. We performed
well-suitedforone-timeauthentication,andthereforearecom-
] experiments to investigate why HMOG features perform well
R monly used to authenticate users at login. This renders them
during walking. Our results suggest that this is due to the
C abilityofHMOGfeaturestocapturedistinctivebodymovements ineffective when the smartphone is accessed by an adversary
. caused by walking, in addition to the hand-movement dynamics afterlogin.Continuousoractiveauthenticationaddressesthese
s from taps. With BKG, we achieved EERs of 15.1% using challenges by frequently and unobtrusively authenticating the
c
HMOGcombinedwithtaps.Incomparison,BKGusingtap,key
[ user via behavioral biometric signals, such as touchscreen in-
hold, and swipe features had EERs between 25.7% and 34.2%.
teractions[26],handmovementsandgait[17],[8],voice[40],
3 We also analyzed the energy consumption of HMOG feature
v extraction and computation. Our analysis shows that HMOG and phone location [55].
9 featuresextractedat16Hzsensorsamplingrateincurredaminor In this paper, we present Hand Movement, Orientation, and
9 overhead of 7.9% without sacrificing authentication accuracy. Grasp (HMOG), a new set of behavioral biometric features
1 Two points distinguish our work from current literature: 1) we
for continuous authentication of smartphone users. HMOG
1 present the results of a comprehensive evaluation of three types
uses accelerometer, gyroscope, and magnetometer readings
0 of features (HMOG, keystroke, and tap) and their combinations
. under the same experimental conditions; and 2) we analyze to unobtrusively capture subtle hand micro-movements and
1
the features from three perspectives (authentication, BKG, and orientation patterns generated when a user taps on the screen.
0
energy consumption on smartphones).
5 HMOGfeaturesarefoundedupontwocorebuildingblocks
1 Index Terms—Behavioral biometrics, continuous authentica- of human prehension [46]: stability grasp, which provides
: tion, biometric key generation, energy evaluation, HMOG. stability to the object being held; and precision grasp, which
v
i involves precision-demanding tasks such as tapping a target.
X
Weviewtheactofholdingaphoneasastabilitygraspandthe
r *Thispaperwaspublishedas[58] actoftouchingtargetsonthetouchscreenasaprecisiongrasp.
a
#WorkdoneinpartwhilevisitingtheNewYorkInstituteofTechnology. We hypothesize that the way in which a user “distributes” or
(cid:91)Correspondingauthor.Mailingaddress:RoomHSH226C,NorthernBlvd., “shares” stability and precision grasps while interacting with
NY11568,USA.Phone:[phone].Fax:[phone]. thesmartphoneresultsindistinctivemovementandorientation
ZdenˇkaSitova´ andJaroslavSˇedeˇnkawerestudentsatMasarykUniversity, behavior. The rationale for our hypothesis comes from the
CzechRepublic.ThisworkwasdonewhilevisitingNYIT. following two bodies of research.
This work was supported in part by DARPA Active Authentication grant First, there is evidence (see [32], [5], [65]) that users have
FA8750-13-2-0266 and a 2013 NYIT ISRC grants. The views, findings,
postural preferences for interacting with hand-held devices
recommendations,andconclusionscontainedhereinarethoseoftheauthors
andshouldnotbeinterpretedasnecessarilyrepresentingtheofficialpolicies suchassmartphones.Dependingupontheposturalpreference,
or endorsements, either expressed or implied, of the sponsoring agencies or it is possible that the user can have her own way of achieving
theU.S.Government.
stabilityandprecision—forexample,theusercanachieveboth
ZhouwassupportedinpartbyNSFCAREERgrantCNS-1253506. stabilityandprecisionwithonehandiftheposturalpreference
Computing and storage resources were provided in part under program involves holding and tapping the phone with the same hand;
“ProjectsofLargeInfrastructureforResearch,Development,andInnovations” or distribute stability and precision between both hands, if the
(LM2010005).
posture involves using both hands for holding and tapping; or
Copyright (c) 2013 IEEE. Personal use of this material is permitted.
achieve stability with one hand and precision with the other.
However, permission to use this material for any other purposes must be
[email protected] Second, studies in ergonomics, biokinetics, and human-
2
computer interaction have reported that handgrip strength keys must not be stored on the smartphone’s memory—but
strongly correlates with an individual’s physiological and so- rather generated from biometric signals and/or passwords.
matic traits like hand length, handedness, age, gender, height, To our knowledge, we are the first to evaluate BKG on
body mass, and musculature (see, e.g., [24], [12], [35]). If smartphones. We instantiated BKG using normalized general-
themicro-movementscausedbytappingreflectanindividual’s ized Reed-Solomon codes in Lee metric. (See section VI for
handgrip strength, then the distinctiveness of HMOG may formulationandevaluation.)WecomparedBKGonHMOGto
have roots, at least in part, in an individual’s distinctive BKG on tap, key hold, and swipe features under two metrics:
physiological and somatic traits. equal error rate (EER) and guessing distance.
Motivated by the above, we designed 96 HMOG features Our results on BKG can be summarized as follows: we
and evaluated their continuous user authentication and bio- achieved lower EERs with HMOG features compared to key
metric key generation performance during typing. Because hold, tap, and swipe features in both walking and sitting
walkinghasbeenshowntoaffecttypingperformance[42],we conditions. For walking, EER of HMOG-based BKG was
evaluated HMOG under both walking and sitting conditions. 17%, vs. 29% with key hold and 28% with tap features. By
combiningHMOGandtapfeatures,weachieved15.1%EER.
Forsitting,weobtainedanEERof23%withHMOGfeatures,
A. Contributions and Novelty of This Work
26% with tap features, and 20.1% by combining both. In
New HMOG Features for Continuous Authentication. We contrast, we obtained 34% EER with swipes. HMOG features
propose two types of HMOG features: resistance features, also provided higher guessing distance (i.e., 2.9 for walking,
which measure the micro-movements of the phone in re- and 2.8 for sitting) than all other features extracted from our
sponse to the forces exerted by a tap gesture; and stability dataset (1.9 for taps and for key holds in walking and 1.6 for
features, which measure how quickly the perturbations in taps in sitting conditions).
movement and orientation, caused by tap forces, dissipate. Energy Consumption Analysis of HMOG Features. Be-
Our extensive evaluation of HMOG features on a dataset of cause smartphones are energy constrained, it is crucial that
100 users1 who typed on the smartphone led to the following a continuous user authentication method consumes as little
findings: (1) HMOG features extracted from accelerometer energy as possible, while maintaining the desired level of
and gyroscope signals outperformed HMOG features from authentication performance. To evaluate the feasibility of
magnetometer; (2) Augmenting HMOG features with tap HMOGfeaturesforcontinuousauthenticationonsmartphones,
characteristics (e.g., tap duration and contact size) lowered we measured the energy consumption of accelerometer and
equal error rates (EERs): from 14.34% to 11.41% for sitting, gyroscope, sampled at 100Hz, 50Hz, 16Hz and 5Hz. We then
and from 14.73% to 8.53% for walking. This shows that measured the energy required for HMOG feature computation
combining tap information with HMOG features considerably from sensor signals, and reported the tradeoff between energy
improvesauthenticationperformance;and(3)HMOGfeatures consumption and EER.
complement tap and keystroke dynamics features, especially Our analysis shows that a balance between authentication
for low authentication latencies at which tap and keystroke performanceandenergyoverheadcanbeachievedbysampling
dynamics features fare poorly. For example, for 20-second HMOG features at 16Hz. The energy overhead with 16Hz
authentication latency, adding HMOG to tap and keystroke is 7.9%, compared to 20.5% with 100Hz sampling rate, but
dynamics features reduced the equal error rate from 17.93% comes with minor increase (ranging from 0.4% to 1.8%) in
to11.74%forwalkingandfrom19.11%to15.25%forsitting. EERs.However,byfurtherreducingthesamplingrateto5Hz,
Empirical Investigation Into Why HMOG Authentication we observed a significant increase in EER (11.0% to 14.1%).
Performs Well During Walking. HMOG features achieved
B. Organization
lower authentication errors (13.62% EER) for walking com-
pared to sitting (19.67% EER). We investigated why HMOG WepresentthedescriptionofHMOGfeaturesinSectionII,
had a superior performance during walking by comparing the anddetailsonourdatasetinSectionIII.InsectionsIVandV,
performance of HMOG features during taps and between taps wedescribetheauthenticationexperimentsandpresentresults.
(i.e., the segments of the sensor signal that lie between taps). WeintroduceandevaluateBKGonHMOGinSectionVI.We
Ourresultssuggestthatthehigherauthenticationperformance analyzetheenergyconsumptionofHMOGfeaturesinSection
during walking can be attributed to the ability of HMOG VII.InSectionVIII,wereviewrelatedresearch.Weconclude
features to capture distinctive movements caused by walking in Section IX.
in addition to micro-movements caused by taps.
II. DESCRIPTIONOFHMOGFEATURES
BKG with HMOG Features. BKG is closely related to
We define two types of HMOG features: grasp resistance
authentication,buthasadifferentobjective:toprovidecrypto-
and grasp stability. These features are computed from data
graphicaccesscontroltosensitivedataonthesmartphone.We
collected using three sensors: accelerometer, gyroscope, and
believe that designing a secure BKG scheme on smartphones
magnetometer. Because HMOG features aim to capture the
isveryimportant,becausetheadversaryisusuallyassumedto
subtle micro-movements and orientation patterns of a user
havephysicalaccesstothedevice,andthereforecryptographic
while tapping on the screen, we extract HMOG features from
signals collected during or close to tap events. Computation
1We made the dataset available at http://www.cs.wm.edu/∼qyang/hmog.
html.Wealsodescribedthedataanditsreleasein[68]. of grasp stability and resistance features is discussed next.
3
A. Grasp Resistance Features
Grasp resistance features measure the resistance of a hand 2)s/m end of tap = |Zj - avg100msBefore|
grasp to the forces (or pressures) exerted by touch/gesture ( six maxTap
a
einvemntosv.eWmeenqtu(aunstiinfyg rreeasidsitnagnscefraosmthaecccehlaenrogme,eoterr)p,eortruiernbtaattiioonn, -z eht n 100 ms 100 ms avg100msAfter
(from gyroscope) and magnetic field (from magnetometer), i re avg100msBefore
te
caused by a tap event. m
o
rele avgTap
cc start of tap
TABLEI a m
NOTATION. orf sg
Sensor AccelerometerorGyroscopeorMagnetometer. nid tafter
X,Y,Z Timeseriesofsensorreadingsinx,y,andzaxesrespectively ae tap
Z1,...,Zn Individualsensorreadingsinzaxiscollectedattime R non-tap
M tT1im,.e..s,ertinesreosfpmecatgivneiltyudeof(cid:113)sensorreading,whereeach 100 tbefore_c5Te0in mtmerse2 0(tms0tairltlist=eecn tdomna5dx0_s inm)_tsatpa3fte0r0_center tmin400 t2e0n0dm +s
elementMiiscomputedas (Xi2+Yi2+Zi2)
tstart Starttimeofatapevent Fig. 1. Illustration of key variables for computing grasp resistance features
tend Endtimeofatapevent 3-5andgraspstabilityfeatures1-3.
tmaxintap Timebetweentstartandtendatwhichthereadingfroma
sensorreachesitshighestvalue
tbefotrmeicnenter TCiemnteerwohfenthseta1b0i0litmysiswaicnhdioewvedbeaffoterertahetaptapeventhasended Algorithm 1: Computation of tmin on Z readings
taftercenter Centerofthe100mswindowafteratap input :avg100msBefore,t1,...,tn (timestampsbetweentend and
avg100msBefore Averageofsensorreadingsina100mswindowbeforestart tend+200ms,andZ1,...,Zn (Z readingsatt1,...,tn)
avg100msAfter timeandafterendtime,respectively output:tmin (timestampbetweentend andtend+200msatwhich
avgTap Averageofreadingsduringtapevents thesensorreadingareclosesttothosemeasuredbeforethetap.)
tmin Timewhenstabilityisachievedafterthetapeventhasended 1: fori=1...ndo
2: avgDiffs[i]= (cid:80)nj=i(|Zj−avg100msBefore|)
Weextractedfivegraspresistancefeaturesfromaccelerome- 3: endfor n−i+1
ter,gyroscope,andmagnetometer,overfourdimensions(mag- 4: min=argmini(avgDiffs[i])//min is the index at
which avgDiffs has its minimum value
nitude,x,y,andzaxes),leadingto5×3×4=60features.For
5: returntmin
simplicity of exposition, we describe grasp resistance features
only on the z axis. We also extracted the same features from
X,Y,andM.OurnotationissummarizedinTableI.Figure1
Algorithm 1. This feature is calculated as t −t .
illustrates variables used in features 3 through 5. min end
1) Mean of Z during taps.
2) Normalized time duration for mean sensor value to
2) Standard deviation of Z during taps.
change from before tap to after tap event, calculated as:
3) Difference in Z readings before and after a tap event.
Let avg100msBefore be the average of Z read- ∆ = tafter center−tbefore center
duration avg100msAfter−avg100msBefore
ings in a 100 ms window before tap start time, and
avg100msAfter be the average of Z readings in a where t is the center of the 100ms window
after center
100 ms window after tap end time. We calculated this after a tap event, and t is the center of the
before center
featureasthedifferencebetweenavg100msAfterand 100ms window before the tap event.
avg100msBefore. 3) Normalized time duration for mean sensor values to
4) Net change in Z readings caused by a tap. Let avgTap change from maxTap to avg100msAfter in response
be the average of Z readings during a tap event. We cal- to a tap event, calculated as:
culate this feature as avgTap - avg100msBefore.
t −t
∆ = after center maxintap
5) Maximum change in Z readings caused by a tap. max to avg avg100msAfter−maxTap
Let maxTap be the maximum Z reading during a
where maxTap is the maximum sensor value during a
tap event. This feature is calculated as maxTap -
tap,andt isthetimewhenthisvalueoccurred.
avg100msBefore. max in tap
We extracted the above three grasp stability features for three
sensors and four types of sensor readings (X, Y, Z and M),
B. Grasp Stability Features for a total of 3×3×4=36 features.
Stability features quantify how quickly the perturbations
Complexity of computing HMOG features is linear (O(n))
caused by a finger-force from a tap event disappear after the
in the sampling frequency, except for Grasp Stability Feature
tap event is complete. We compute grasp stability features as
1, which is quadratic (O(n2)).
follows: (Figure 1 illustrates variables used in the features.)
1) Time duration to achieve movement and orientation sta-
III. DATASET
bilityafteratapevent.Lett denotetheendtimeofthe
end
tap event, and t the time when stability is achieved To evaluate HMOG features, we used sensor data collected
min
after the tap event has ended, computed as shown in from 100 smartphone users (53 male, 47 female) during eight
4
freetexttypingsessions[68].2 Usersansweredthreequestions A. Design of Authentication Experiments
per session, typing at least 250 characters for each answer.
1-Class Verifiers. We performed verification experiments
In four sessions, the users typed while sitting. In another
using three verifiers [34]: scaled Manhattan (SM), scaled
four sessions, users typed while walking in a controlled
Euclidian(SE),and1-classSVM.(Henceforth,weuse“SVM”
environment.
to refer to “1-class SVM”.) We chose these verifiers because
Foreachuser,wecollectedanaverageof1193tapsperses-
previous work on behavioral authentication has shown that
sion (standard deviation: 303) and 1019 key presses (standard
they perform well. For instance, SM and SVM were top per-
deviation: 258). The average duration of a session was 11.6
formersinastudyonkeystrokeauthenticationofdesktopusers
minutes, with a standard deviation of 4.6 minutes. Data was
byMaxionetal.[34].SVMperformedwellinexperimentson
collectedusingthesamesmartphonemodel(SamsungGalaxy
touch-based authentication of smartphone users by Serwadda
S4). We used a total of ten Samsung Galaxy S4. Data was
et al. [53]. SE is a popular verifier in biometrics (see for
collected over multiple days, and the same user might have
example [7], [28]).
received a different device during each visit.
ParametertuningwasnotrequiredforSMandSE.However,
We recorded accelerometer, gyroscope and magnetometer for SVM [11], we used RBF kernel and performed a grid
sensor readings (sampling rate 100 Hz) as well as raw touch searchtofindtheparameters(forγ,wesearchedthrough2−13,
data collected from the touchscreen, touch gestures (e.g., tap, 2−11,2−9,...,213;andforν,wesearchedthrough0.01,0.03,
scale, scroll, and fling), key press, and key release latencies 0.05, 0.1, 0.15 and 0.5). We used cross-validation to choose
on the virtual keyboard. Due to security concerns, Android the parameter values (see Section IV-C).
OS forbids third-party applications to access touch and key We did not include 2-class verifiers in our evaluation. To
press data generated on the virtual keyboard. Therefore, we train a 2-class verifier, in addition to data from smartphone
designed a virtual keyboard for data collection that mimicked owner, biometric data from other users (non-owners) is re-
the look, feel, and functionality of default Android keyboard, quired. Because sharing of biometric information between
includingtheautocorrectandautocompleteoptions,whichthe smartphoneusersleadstoprivacyconcerns,webelievethat1-
users were free to use. classverifiersaremoresuitableforsmartphoneauthentication.
During data collection users were allowed to choose the (A similar argument was made in [54].)
orientation of the smartphone (i.e., landscape or portrait).
Training and Testing. For experiments in sitting and walk-
Because less than 20 users typed in landscape orientation, we
ing conditions, we used the first two sessions for training
performed all authentication experiments with data collected
and the remaining two for testing. We extracted HMOG
in portrait mode.
features during each tap. Thus, each training/testing vector
corresponded to one tap. With keystroke dynamics features,
each training/testing vector corresponded to one key press on
IV. EVALUATIONOFHMOGFEATURES
the virtual keyboard.
For SM and SE, the template consisted of the feature-wise
average of all training vectors. We used user-wise standard
deviations for each feature for scaling. We used all training
vectors to construct the template (hypersphere) with SVM.
Users with less than 80 training vectors were discarded from
authentication. As a consequence, ten users failed to enroll
(and were not included in our experiments).
We created authentication vectors by averaging test vectors
sampledduringt-secondsscan.Wereportresultsforauthenti-
cation scans of t = 20, 40, 60, 80, 100, 120 and 140 seconds.
We chose these scan lengths to cover both low and higher
authentication latencies. Our preliminary experiments showed
that for scans longer than 140 seconds, there is minimal
improvement in authentication performance.
Fig.2. Flow-diagramdepictingourexperimentworkflow. QuantifyingAuthenticationPerformance.Wegeneratedtwo
types of scores, genuine (authentication vector was matched
Our experiment workflow involves: (1) computing features against template of the same user) and zero-effort impostor
from data collected during typing; (2) performing feature (authentication vector of one user was matched against the
selection; (3) performing feature transformation (PCA); (4) template of another). We used population equal error rate
performingoutlierremoval;and(5)performingauthentication (EER) to measure the authentication performance.
using Scaled Manhattan, Scaled Euclidean, SVM verifiers,
B. Comparing HMOG to Other Feature Sets
and score-level fusion. Figure 2 summarizes the experiment
workflow. We compared the authentication performance of HMOG
features with touchscreen tap and keystroke dynamic features
2Ourdatasetisavailableathttp://www.cs.wm.edu/∼qyang/hmog.html (key hold and digraph latencies).
5
TABLEII For each set of parameter values, 10-CV yielded ten EERs,
PARAMETERSEVALUATEDUSINGCROSS-VALIDATION which we averaged to get an estimate of the EER corre-
sponding to that set of parameter values. We then selected
Method Parameter
Fisherscore PercentageofthesumofallFisherscores parameter values which had the lowest (average) EER. For
mRMR ThresholdonthemRMRscore 10-CV experiments involving 20- to 140-second scan lengths,
PCA Percentageoftotalvariance
the sets of parameter values that led to the lowest EERs were
SVM γ,ν
not always identical. In this case, we took a majority vote to
select the most common parameter values.
Touchscreen Features from Tap Events. We extracted 11 Feature Selection. During training, we evaluated two feature
commonly used touchscreen-based features for tap events selection methods: Fisher score ranking [20], and minimum-
(see Table VIII). Some papers (e.g., [53] and [26]) defined Redundancy Maximum-Relevance (mRMR) [48]. Our pre-
these features for swipes, while we extracted them from taps liminary experiments showed that Fisher score performed
due to very low availability of swipes during typing, and to betterforHMOGfeatures,whilemRMRperformedwellwith
provide a more meaningful comparison with HMOG features, tap features. With key hold and digraph features, the best
whicharecollectedduringtaps.Thefeaturesweextractedare: performing feature set contained all the features.
• Duration of the tap Fisher score ranking was computed independently for each
• Contact size features: mean, median, standard deviation, HMOG feature as the ratio of between-user to within-user
1st, 2nd and 3rd quartile, first contact size of a tap, variance.(HigherFisherscoresuggestshigherdiscriminability
minimum and maximum of the contact size during the of the corresponding feature.) Using 10-CV, we tested feature
tap (9 features) subsets whose sum of Fisher scores accounted for 80% to
• Velocity (in pixels per second) between two consecutive 100% of the sum of Fisher scores of all features.
press events belonging to two consecutive taps. We selected HMOG features for each verifier separately.
The following parameters for Fisher score ranking provided
Key Hold Features. Key hold latency is the down-up time
the best authentication results: 82% (17 features) for SM
between press and release of a key. We used 89 key hold
during sitting; 81% (13 features) for SM and SVM during
features, each corresponding to a key on the virtual keyboard.
walking; and 80% (16 features) for SVM during sitting. For
Digraph Features. Digraph latency is the down-down time SE, we achieved lowest EER by including resistance features
between two consecutive key presses. We used digraph fea- only, compared to the feature subset obtained from feature
tures for combinations of the 35 most common keys in our selection. Figure 3 reports the ranking of the features during
dataset.3 Thus we have 352 =1225 digraph features. sitting (3(a)) and walking (3(b)).
Score-level Fusion. To determine whether HMOG features For tap features, with SM verifier we achieved the best
complement existing feature sets, we combined tap, key hold, results with 3 features chosen by mRMR (threshold 0) and
digraph and HMOG features using weighted sum score-level for SE and SVM with 2 features (threshold 0.1). The best
fusion. We chose this method because it is simple to imple- threefeaturesaccordingtomRMRare(inthisorder):duration
ment, and has been shown to perform well in biometrics [30]. of the tap; mean of contact size; and velocity between two
We used the technique of Locklear et al. [39] to ensure that consecutive down events.
weights sum to one and proportion of weights is preserved OutlierRemoval.ForHMOGandtaptemplates,weevaluated
when scores from some feature sets were missing (e.g. due to the interquartile outlier removal (i.e., different subsets of the
lack of accelerometer data). We used grid-search to find the values from the first and fourth quartile are removed). Ex-
weights which led to the best authentication performance. periments with SM verifier showed that outlier removal does
not improve authentication accuracy, so we did not consider
it further in our experiments.
C. Feature Selection, Preprocessing, and Transformation
For key hold and digraph latencies, using only outlier
To improve authentication performance, we performed fea- removalandnotperformingfeatureselectionortransformation
ture selection, feature transformation with Principal Compo- led to the best results. Outlier removal was done using two
nent Analysis (PCA), and outlier removal. parameters: (1) latencies longer than l ms were discarded and
Parameter selection. We used 10-fold cross-validation (10- (2) if a feature occurs less than m times in a user’s template,
CV) on training data to choose feature selection method the feature was discarded). The values evaluated for l were
(mRMR or Fisher score ranking), as well as to set the 100, 200, 300, 400, 500 and 1000 for key hold and 200, 350,
parameters for feature selection, PCA, and SVM. The param- 500, 650 and 800 for digraph. For m, we experimented with
eters are presented in Table II. We evaluated all parameters 2, 5, 10, 15, 20, 40 and 60. The best l value was 200 for key
independently for each combination of feature set, verifier, hold,andbetween350–500fordigraph;thebestmvaluewas
authentication scan-length and body-motion condition. between 2–60 for key hold and between 2–5 for digraph.
Feature Transformation.WeusedPCAtotransformoriginal
3All 26 alphabetic keys, 5 keyboard switches (shift, switch between features into principal components, that were subsequently
numerical and alphabetical keyboard, delete, done, return) and 4 special
used in authentication experiments. Our motivation for using
characters(space,dot,commaandapostrophe).Theavailabilityofotherkeys
inourtrainingdatawasextremelylow(<1onaverageperuser). PCA are: (1) to remove correlation between features to meet
6
40%
SVM − sit
2.0 accelerometer resistance ate30% SE − sit
R
gyroscope resistance or SM − sit
e1.5 accelerometer stability Err20%
or al SVM − walk
sc gyroscope stability qu10%
her 1.0 E SE − walk
Fis 0% SM − walk
20 40 60 80 100 120 140
0.5 Time (s)
0.0 Fig.4. ComparisonofHMOGfeaturesinsittingandwalkingconditionsfor
threeverifiers.ThereportedEERsarewithPCAforSE,andforSVM-sitting;
Features and without PCA for SM and SVM-walking. X-axis shows authentication
timeinseconds.
(a) FisherscoresofHMOGfeaturesduringsitting.
Digraph − sit
40%
Digraph − walk
2.0 accelerometer resistance ate30% Key hold − sit
R
gyroscope resistance Error 20% Key hold − walk
e1.5 accelerometer stability al Tap − sit
scor gyroscope stability Equ10% Tap − walk
her 1.0 HMOG − sit
s 0%
Fi 20 40 60 80 100 120 140 HMOG − walk
0.5 Time (s)
Fig. 5. Comparison of EERs of HMOG with keystroke dynamics (i.e.,
0.0
key hold and digraph) and tap features with SM verifier. X-axis shows
authenticationtimeinseconds.
Features
(b) FisherscoresofHMOGfeaturesduringwalking. Finally, we present our findings on why HMOG features
achieve lower EERs during walking.
Fig.3. HMOGfeaturesextractedfromaccelerometerandgyroscope,sorted
by Fisher score computed from training data. Higher scores correspond to
featureswithhigherdiscriminativepower.Magnetometerfeatures(notshown) A. Performance of HMOG Features
rankedbelowaccelerometerandgyroscopefeatures.
HMOG features extracted from both accelerometer and gy-
roscopeoutperformedthoseextractedfromindividualsensors.
the assumptions in SE and SM, and (2) to reduce dimen- HMOG features from magnetometer performed consistently
sionality by using only those principal components, which worse than accelerometer and gyroscope features with all
explain most of the variance. We performed PCA under two verifiers, in both sitting and walking conditions. Combining
settings: (1) on all features (except magnetometer features, magnetometer features with features from accelerometer and
which performed poorly), and (2) on a subset of features gyroscope did not improve performance.
selectedusingFisherscoreandmRMR.Weperformed10-CV Resistance features outperformed stability features in both
experimentswithcomponentsexplaining90%,95%,98%,and walking and sitting conditions (and also had higher Fisher
100%oftotalvariance,tosetthethresholdfordimensionality score,seeFigure3).Thissuggeststhattheabilityofresistance
reduction. PCA improved EER for HMOG features with SE features to discriminate between users is higher than that
when performed on resistance features, and for SVM during of stability features. In fact, feature selection on HMOG
sittingwhenperformedonfeaturesselectedusingFisherscore. with 10-CV resulted in selecting resistance features only.
PCA performed on all tap features improved results with SM In some cases, using PCA after feature selection further
and SE. lowered EERs. Table III summarizes the sensors and feature
selection/transformation that led to the lowest EERs.
In Figure 4, we show the EERs of all verifiers under sitting
V. AUTHENTICATIONRESULTS
and walking conditions, when the authentication scans varied
In this section, we report authentication performance of
between 20 and 140 seconds. Among the three verifiers, SM
HMOGfeatures.WecomparetheperformanceofHMOGwith
overallhadlowerEERsforbothsittingandwalkingconditions
keystroke and tap features and report results with fusion.4
and therefore we present the results only with SM hereafter.
Comparison of HMOG with Keystroke Dynamics and
4SeeSupplementforthenumberofgenuineandimpostorscoresusedfor
calculatingEERsinthispaper. Tap Features. Tap features and HMOG features in walking
7
TABLEIII
SUMMARYOFLOWESTEERSACHIEVEDUSINGONLYHMOGFEATURES.
Verifier BestPerformingFeatures Sensors Sitting Walking
ScaledManhattan WithFisherScoreRanking Accelerometer+Gyroscope 19.67% 13.62%
ScaledEuclidean WithPCA,noFeatureSelection Accelerometer+Gyroscope 25% 15.31%
1-ClassSVM WithFisherScoreRanking;withPCAforsitting,withoutPCAforwalking Accelerometer+Gyroscope 27.45% 15.71%
TABLEIV
SUMMARYOFLOWESTEERSACHIEVEDWITHSCORE-LEVELFUSIONOFHMOG,TAP,ANDKEYSTROKEDYNAMICS(KD)FEATURES.
Score-LevelFusionwithSMVerifier Sitting Walking
HMOG,Tap,andKeystrokeDynamics 10.05% 7.16%
HMOGandTap 11.41% 8.53%
TapandKD 11.02% 10.79%
30% 40%
Tap HMOG
e
at e30% between taps
R20% HMOG at (sit)
Error TDaigpr a+p Khey hold + Error R20% d(suitr)ing taps
Equal 10% HMOG + Tap Equal 10% b(dweuatrwilnkeg)e tna ptasps
HMOG + Tap + (walk)
Key hold + Digraph
0%
0%
20 40 60 80 100 120 140
Time (s) 20 40 60 80 100 120 140
Time (s)
(a) Sitting
Fig.9. PerformanceofHMOGfeaturesextractedduringandbetweentaps.
X-axisshowsauthenticationtimeinseconds.
30%
e Tap results for sitting and walking conditions are presented in
at
R20% HMOG figures 6(a) and 6(b), respectively. The lowest EERs achieved
or withfusionaresummarizedinTableIV,vandthecorrespond-
al Err10% TDaigpr a+p Khey hold + ingDETcurvesforfusionon60-and120-secondscanlengths
qu HMOG + Tap are shown in Figure 7.
E
HMOG + Tap + Our results show that: (1) for both walking and sitting
Key hold + Digraph
0% conditions, score-level fusion of all signals led to the lowest
20 40 60 80 100 120 140 EER;and(2)fusingHMOGwithtapfeaturesledtoadecrease
Time (s)
in EERs and either outperformed (in the case of walking and
shorter scans in sitting) or was comparable (in the case of
(b) Walking
longerscansinsitting)tofusionoftapandkeystrokedynamics
Fig.6. Score-levelfusionofcombinationsoffeaturetypeswithSMverifier. (see figures 6(a)) and 6(b)). Both (1) and (2) indicate that
X axesshowauthenticationtimeinseconds.
HMOG provides additional distinctiveness to that of tap and
keystroke dynamics, especially in walking condition.
condition performed better than keystroke dynamics features;
HMOG in sitting outperforms keystroke dynamics for shorter B. Why HMOG Features Perform Better During Walking
scans and is comparable for longer scans (see Figure 5).
We investigated why HMOG features performed better
HMOG features outperformed tap features in walking con- duringwalking.Specifically,weinvestigatedwhetherthehigh
dition, while tap outperformed HMOG in sitting. The per- authentication accuracies of HMOG features during walking
formance of tap and keystroke dynamics features did not were due to hand movements caused by taps, or due to
changesignificantlybetweensittingandwalking.However,the movements caused by walking, or a combination of both.
performance of HMOG improved considerably (up to 6.11%)
Experimentsetup.Weextracted64HMOGfeaturesfromtwo
during walking.
segmentsofanaccelerometer/gyroscopesignal:(1)duringtap,
Fusion of HMOG, Tap, and Keystroke Features. We used as discussed in previous sections; and (2) between taps, in
SM verifier and performed score-level fusion with the follow- which HMOG features were extracted when the user was not
ingfeaturecombinations:{HMOG,tap,keystrokedynamics}; tapping the screen (see Figure 8). In (2), the signal between
{tap,keystrokedynamics};and{tap,HMOG}.Detailedfusion taps was segmented into non-overlapping blocks of 91 ms;
8
1 1
Tap + Key hold + Digraph Tap + Key hold + Digraph
0.9 HMOG + Tap + Key hold + Digraph 0.9 HMOG + Tap + Key hold + Digraph
0.8 0.8
e0.7 e0.7
at at
ct R0.6 ct R0.6
eje0.5 eje0.5
R R
e 0.4 e 0.4
s s
Fal0.3 Fal0.3
0.2 0.2
0.1 0.1
0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
False Accept Rate False Accept Rate
(a) Sitting(60-secondscans) (b) Walking(60-secondscans)
1 1
Tap + Key hold + Digraph Tap + Key hold + Digraph
0.9 HMOG + Tap + Key hold + Digraph 0.9 HMOG + Tap + Key hold + Digraph
0.8 0.8
e0.7 e0.7
at at
ct R0.6 ct R0.6
eje0.5 eje0.5
R R
e 0.4 e 0.4
s s
Fal0.3 Fal0.3
0.2 0.2
0.1 0.1
0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
False Accept Rate False Accept Rate
(c) Sitting(120-secondscans) (d) Walking(120-secondscans)
Fig.7. DETcurvesforfusionofallfeaturetypesincludingandexcludingHMOG.Thescanlengthsare60-and120-seconds.
during taps
er
et
m
o tap
er
accel2m/s) non-tap
gs from z-axis (
eadinn the between taps
R i
0 Time (seconds)
Fig.8. HMOGfeaturesextractedduringandbetweentaps.Thefigureshowsasampleofreadingsfromthez-axisofaccelerometerinsittingcondition.
oneHMOGfeaturevectorwasextractedfromeachblock.We HMOGduringtapswas1122forsitting,and1186forwalking.
selected 91ms as the block size because it was the median Forbetweentaps,itwas7692forsittingand7462forwalking.
duration of a tap in our training data. This ensured that the The average number of testing vectors per user for HMOG
number of sensor readings used to extract a HMOG feature features during taps was 897 for sitting and 972 for walking.
vector between and during tap remained same. Forbetweentaps,itwas5885forsittingand5768forwalking.
HMOG features extracted during taps use sensor readings Verification experiments were performed using SM.
from 100 ms before and 200 ms after a tap event (see Section Performance of HMOG Features Extracted During vs.
II).WeextractedHMOGfeaturesbetweentapsstarting300ms Between Taps. We compared HMOG features extracted dur-
after a tap until 300 ms before the next tap, to avoid any ing taps with the same features extracted between taps for
overlap between during and between HMOG features. sitting and walking conditions. For sitting, HMOG features
The average number of the training vectors per user for extracted during taps performed consistently better than those
9
extracted between taps (see EERs in Figure 9). This indicates biometric. Each feature F is assigned a discretization range
i
that HMOG features were able to capture distinctive hand [0,d range ], where d range ∈ {(p−1)/2,...,p−1} is
Fi Fi
micro-movementpatternswhentheuserstappedonthephone. negatively correlated to the standard deviation σ of F (i.e.,
i i
Similarly, for walking, HMOG features extracted during taps if σ <σ , then d range >d range ).
i j Fi Fj
performed better than those extracted between taps (see EERs Let x be an instance of F , and min and max be
i i Fi Fi
inFigure9).ThisagainindicatesthatHMOGfeaturescapture respectively the typical minimum and maximum value of F .
i
user’sdistinctivehandmicro-movementpatternswhentheuser Discretization and scaling are performed as:
is tapping, regardless of the motion condition.
ITobinmueadttppwpicsaeea.rcetfHtenosoMrtmftahOpWeaGdtEaHlEftkheRMiaenstOugsfrGaooemrsnfeseeHixatMwtttriunahrOcgeetsnGeadcneFadbxpeettawrtuatwaruceletrkeeedindnsisgtEtdaixipnuntscrritFndaivgiucgetrueismndriegtotBiv9wnee)gat.mwlkTe(eisnhneetingess DSFi(xi)=0(cid:106)ddrarnagnegeFi·(cid:16)maxxiF−im−imnFiniFi(cid:17)(cid:107) xxmiin<>Fimm≤ianxxFii≤maxFi
Fi i Fi
induced by walking, even in the absence of tap activity.
Committing a Key. To commit a cryptographic key using
Supported by the above results, the high authentication
n biometric features, the user selects a random codeword
accuraciesachievedbyHMOGfeaturesduringwalkingcanbe c of length n from C ⊂ (Z )n. The key is computed as
p
jointlyattributedto:(a)thedistinctivenessinhandmovements
k = PRF (z|0), where PRF is a pseudorandom function
c
causedbytapactivityand(b)thedistinctivenessinmovements
family, z is a system-wide public constant and “|” denotes
caused by walking.
string concatenation. (BKG can be augmented with a second
authenticationfactorbysettingztoauser-providedpassword.)
VI. BIOMETRICKEYGENERATION c is then committed using the user’s biometrics as discussed
FROMHMOGFEATURES next. Let x=(x ,...,x ) be a scaled and discretized feature
1 n
In this section, we evaluate the performance of HMOG vector.Theusercomputesδ =(x−c)=(x1−c1,...,xn−cn)
features for biometric key generation (BKG). For this pur- and publishes commitment γ =(PRFc(z|1),δ).
pose, we introduce our BKG construction, which extends and Theusercomputesk fromγ andherbiometricsignals(and
generalizes the fuzzy commitment scheme of Juels et al. [31]. possibly a password z) as follows. She extracts biometric fea-
While the technique in [31] operates on features represented tures from the signal, and encodes them as y =(y ,...,y ).
1 n
using a single bit, our BKG construction represents features Then, she computes c(cid:48) = decode(y −δ). If PRF (z|1) =
c(cid:48)
as symbols of an alphabet of arbitrary prime size p. Our PRF (z|1), then k =PRF (z|0) with overwhelming proba-
c c(cid:48)
BKG relies on Reed-Solomon error correcting codes [52] in bility.
Lee metric [37]. Notation used in this section is presented in Asymptotic complexity of BKG key retrieval is dominated
Table V. by one instance of Euclidean algorithm and one matrix-vector
Preliminaries. BKG uses biometric information to prevent multiplication, both in O(n2) finite field operations in a field
unauthorizedaccesstocryptographickeys;thesekeyscanthen of size p ≥ n. Security of our construction is analyzed in
be used, e.g., to encrypt/decrypt sensitive information. The Appendix A.
process of protecting a key is referred to as committing, and
Using Lee-metric Decoding for BKG. Distance between
theoutcomeofthisprocessisacommitment.Givenacommit-
feature vectors is defined using the Lee distance [37]—a
ment, the cryptographic key is reconstructed by decommiting
discrete approximation of SM:
(or opening) it, using information from a biometric signal.
Informally, a BKG construction is secure if a key committed Definition 1 (Lee weight). Let p be an odd prime. The Lee
using a biometric signal s can be opened only using a signal weight of element x∈Z is defined as w (x)=min|x(cid:48)|, for
p L
s(cid:48) ≈s, and both s and s(cid:48) are from the same user. x(cid:48) ≡xmodp. The Lee weight of vector x=(x ,...,x )∈
1 n
BKG techniques use error-correcting codes to address nat- (Z )n is defined as the sum of Lee weights of its elements,
p
ural variations among different biometric samples from the i.e., w (x)=(cid:80)n w (x ).
L i=1 L i
same users. An error-correcting code is defined as a set C of
Definition 2 (Lee distance). The Lee distance of vectors
codewords. Typically, there are two functions associated with
x,y ∈Z istheLeeweightoftheirdifference,i.e.,d (x,y)=
a code: encode(·) and decode(·). The former maps a message p L
w (x−y).
to a codeword; the latter—a possibly perturbed codeword to L
the original codeword. The decode(·) function is designed to In Z , the Lee weight coincides with Hamming weight.
2
maximize the probability of correct decoding.
WeusednormalizedgeneralizedReed-Solomoncodesfrom
[52],presentednext,toimplementtheencode(·)anddecode(·)
A. Our Construction functions.
ScalingandDiscretization.BKGtechniquesworkondiscrete Definition 3. Let l ≤ n and n ≤ p. A linear [n,l]-code over
values,insteadofrealvalues.Therefore,theuserfirstperforms Z isal-dimensionalvectorsubspaceof(Z )n.Anormalized
p p
scalinganddiscretizationofthefeaturevectorrepresentingher Reed-Solomon [n,l]-code over Z is a linear [n,l]-code over
p
10
TABLEV
LISTOFSYMBOLSUSEDINTHISSECTION.
C Error-correctingcode(vectorspaceoverZp). δ=(x−c) Discretizebiometricsamplexmaskedbyc.
n Numberofbiometricfeatures,aswellaslengthofC. γ Fuzzycommitment
p AlphabetsizeofC.(pisaprimenumberlargern.) d rangeFi UpperboundoftherangeofDSFi
l DimensionofC (sizeofthebasisofC). minFi,maxFi MinimumandmaximumvaluesofFi
wL(·),dL(·,·) LeeweightandLeedistancefunctions. σi StandarddeviationofFi
z System-widepublicconstantoruserPIN/password. xi InstanceofFi
c=(c1,...,cn) CodewordfromC. Fi Biometricfeaturei
x=(x1,...,xn) Discretizedbiometricsample.
Z with parity-check matrix: of a BKG scheme, because it does not take into account all
p
  theinformationreadilyavailabletotheadversary.Inparticular,
1 1 ... 1
withBKGtheadversaryhasaccessto:(1)thecommitmentγ;
 1 2 ... n 
H = 1 22 ... n2  and (2) an approximation of the distribution of the user’s
 .  biometric signals obtained from population data.
 . 
 . 
Accesstoγ allowstheadversarytotestwhetheraparticular
1 2n−l−1 ... nn−l−1
feature vector decommits the key. The adversary can perform
and generator matrix: this test offline, i.e., with no restrictions on the number of at-
 v v ... v  temptsperformed(withinthelimitsoftheavailableresources).
1 2 n
 v1 2v2 ... nvn  Therefore, the hardness of “guessing” a user’s feature vector
G= v1 22v2 ... n2vn  given γ is an upper bound on the security of a BKG scheme.
 . 
 ..  The adversary can use (2) to guess the user’s feature vector
v 2l−1v ... nl−1v more efficiently, under the assumption that biometric signals
1 2 n
from different users are not completely independent. To this
The rows of G form a basis of the nullspace of HT.
end,Ballardetal.[6]proposedthenotionofguessingdistance.
To obtain a random codeword from C, we select a l-tuple
It is defined as the logarithm of the number of guesses
m=(m ,...,m )∈(Z )l uniformly at random, and encode
1 l p necessary to open a commitment using feature vectors from
m as c=mG. The Lee distance of C is 2(n−l), so for any
multiple impostors.
errore=(e ,...,e )withw (e)<n−l,decode(c+e)=c.
1 n L We instantiated guessing distance in our setting as follows.
First, we built a commitment γ =(PRF (z|1),δ ) from the
B. Evaluation of HMOG Features on BKG feature vector of each user i. iThen, weciused thei biometric
To evaluate HMOG features for BKG, we determined sample from user j to open all γ such that i(cid:54)=j, and ranked
i
their authentication accuracy and security against population users according to how many commitments they were able to
attacks. We then compared our results with HMOG features open.Finally,foreachuseriweselectusersj (cid:54)=ifollowingto
to that of tap, key hold, and swipe features under the same thisranking,anddeterminedhowmanyattemptswererequired
metrics. to open γ . Guessing distance was computed as the binary
i
logarithm of this value. There might be users i for which no
Biometric Accuracy. Figures 10(a) and 10(b) summarize the
feature vector from other users could open γ . We refer to the
resultsofourexperiments,performedonour100-userdataset. i
commitments of these users as non-guessed.
We evaluated BKG using the features that performed best for
authentication. We used 17 and 13 HMOG features in sitting TableVIsummarizestheresultsofourexperimentsforone
and walking conditions, respectively (see Figure 3). minute scans. The lowest EER was achieved by combining
We ran experiments on four different feature subsets: HMOG features with the tap features selected by mRMR.
(1) HMOG-only features; (2) 11 tap-only features; (3) 12 Overall, HMOG outperformed tap features for biometric key
(sitting) and 8 (walking) key hold-only features; and (4) generation. Nevertheless, our results show that most commit-
HMOG and 3 best-performing tap features. For both walking ments can be guessed using population data.
and sitting experiments, feature subset (4) provided the best Comparison of HMOG and Swipe Features. Because there
results, i.e., 15% and 20% EER respectively, for both one- is no previous work on BKG using touch-, accelerometer-, or
minute and two-minute scan lengths. For sitting experiment, gyroscope-basedfeatures,wecomparedBKGonHMOGwith
key generation was not possible with (3), as the within-user BKG on touch features extracted from swipes (swipe features
variability of the biometrics signals was too high. hereafter).Forthispurpose,wecomputedswipefeaturesfrom
Sedenka et al.[64] showed that Linear Discriminant Analy- the datasets of Serwadda et al. [53]. As in [53], we used the
sis (LDA) [25] improves BKG accuracy on desktop keystroke wholefirstsessionforcomputingcommitmentsandtenswipes
dynamics. However, HMOG, tap, and key hold features on a from the second session to perform impostor/genuine open
virtual keyboard did not benefit from LDA. Therefore we do attempts. Results are reported in Table VI.
not report BKG results with LDA for these features. We also performed experiments on the touch dataset of
Security Against Population Attacks. EER computed via Franketal.[26].However,alargemajorityoftheuserscould
zero-effortattacksprovideslimitedinformationonthesecurity notreliablydecommittheirownkeys.Thiswasduetothelarge