Pedestrian dead reckoning employing simultaneous activity recognition cues

21 Pages·2012·1.99 MB·English

Home Search Collections Journals About Contact us My IOPscience
Pedestrian dead reckoning employing simultaneous activity recognition cues
This article has been downloaded from IOPscience. Please scroll down to see the full text article.
2012 Meas. Sci. Technol. 23 025103
(http://iopscience.iop.org/0957-0233/23/2/025103)
View the table of contents for this issue, or go to the journal homepage for more
Download details:
IP Address: 139.179.12.122
The article was downloaded on 09/03/2012 at 10:22
Please note that terms and conditions apply.
IOPPUBLISHING MEASUREMENTSCIENCEANDTECHNOLOGY
Meas.Sci.Technol.23(2012)025103(20pp) doi:10.1088/0957-0233/23/2/025103
Pedestrian dead reckoning employing
simultaneous activity recognition cues
KeremAltun1 andBillurBarshan
DepartmentofElectricalandElectronicsEngineering,BilkentUniversity,Bilkent,TR-06800Ankara,
Turkey
E-mail:[email protected],[email protected]
Received14September2011,infinalform18November2011
Published11January2012
Onlineatstacks.iop.org/MST/23/025103
Abstract
Weconsiderthehumanlocalizationproblemusingbody-worninertial/magneticsensorunits.
Inertialsensorsarecharacterizedbyadrifterrorcausedbytheintegrationoftheirrateoutput
toobtainpositioninformation.Becauseofthisdrift,thepositionandorientationdataobtained
frominertialsensorsarereliableoveronlyshortperiodsoftime.Therefore,positionupdates
fromexternallyreferencedsensorsareessential.However,ifthemapoftheenvironmentis
known,theactivitycontextoftheusercanprovideinformationabouthisposition.In
particular,theswitchesintheactivitycontextcorrespondtodiscretelocationsonthemap.By
performinglocalizationsimultaneouslywithactivityrecognition,wedetecttheactivitycontext
switchesandusethecorrespondingpositioninformationaspositionupdatesinalocalization
filter.Thelocalizationfilteralsoinvolvesasmootherthatcombinesthetwoestimatesobtained
byrunningthezero-velocityupdatealgorithmbothforwardandbackwardintime.We
performedexperimentswitheightsubjectsinindoorandoutdoorenvironmentsinvolving
walking,turningandstandingactivities.Usingaspatialerrorcriterion,weshowthatthe
positionerrorscanbedecreasedbyabout85%ontheaverage.Wealsopresenttheresultsof
two3Dexperimentsperformedinrealisticindoorenvironmentsanddemonstratethatitis
possibletoachieveover90%errorreductioninpositionbyperforminglocalization
simultaneouslywithactivityrecognition.
Keywords:inertialsensing,wearablecomputing,pedestriandeadreckoning,human
localization,humanactivityrecognition
(Somefiguresmayappearincolouronlyintheonlinejournal)
1.Introduction combined with other types of position sensing to improve
positionaccuracy.
Dead reckoning is the process of estimating the current Historically, dead reckoning has been used in ship
position of a moving entity using the position estimate navigationforcenturies.Reference[1]explainsitsuseinship
(orfix)calculatedatprevioustimeinstantsandthevelocity(or navigation in detail. It has been used in air navigation since
speed)estimateatthecurrenttimeinstant.Itcanalsobeused the beginning of 1900s; a thorough survey appears in [2, 3].
topredictthefuturepositionbyprojectingthecurrentknown A survey on the positioning and navigation methods for
positionandspeedtoafutureinstant[1].Sincethepastposition vehiclesappearsin[4].Deadreckoningisemployedinmobile
estimatesareprojectedthroughtimetoobtainnewestimatesin robotics through the use of odometry [5] and/or inertial
deadreckoning,positionerrorsaccumulateovertime.Because navigationsystems(INSs).
ofthiscumulativeerrorpropagation,dead-reckoningestimates INSs [6] can be used for both indoor and outdoor
areunreliableifcalculatedoverlongperiodsoftime.Hence, positioning and navigation. Fundamentally, gyroscopes
dead reckoning is seldom used alone in practice and is often provide angular rate information and accelerometers provide
velocity rate information. Although the rate information is
reliable over long periods of time, it must be integrated to
1 Present address: Department of Computer Science, University of British
Columbia,Vancouver,BC,Canada. provide position, orientation and velocity estimates. Thus,
0957-0233/12/025103+20$33.00 1 ©2012IOPPublishingLtd PrintedintheUK&theUSA
Meas.Sci.Technol.23(2012)025103 KAltunandBBarshan
Figure1.Strap-downINSintegration.
even very small errors in the rate information provided by one step will not be carried over to the next step. As an
inertial sensors cause unbounded growth in the error of the alternative,insteadofdirectlyresettingthevelocitiestozero,
integratedmeasurements.Asaconsequence,anINSbyitself this information can be used as a measurement in a Kalman
is characterized by position errors that grow with time and filter[11,12].In[10],theZUPTmethodisusedtoestimatethe
distance, usually referred to as the ‘drift error.’ One way distancetravelledandahigh-gradegyroscopeisemployedto
of overcoming this problem is to periodically reset inertial estimate the orientation. Alternative methods for orientation
sensors with external absolute sensing mechanisms and to estimation also exist in the literature. In [13], a Kalman
eliminate this accumulated error. Thus, in most cases, data filter is used to estimate the orientation. Accelerometers
fromanINSmustbeintegratedwithabsolutelocation-sensing and magnetometers can also be used interchangeably with
mechanismstoprovideusefulinformationaboutposition. gyroscopes depending on whether the body is in motion or
An inertial measurement unit (IMU) consists of not [14]. Another approach is to use the orientation output
orthogonallymountedaccelerometersandgyroscopesinthree of a commercially available sensor module that integrates
spatial directions. If the IMU is directly mounted on the accelerometer, gyroscope and magnetometer measurements
[15]. An extensive survey on orientation estimation
moving object, the system is called a strap-down INS [6].
methods using body-worn sensors appears in [16]. Heuristic
The IMU provides three acceleration and three angular
methodsthatexploittheusualwalkingpatternsofpeoplecan
velocity (or angular rate) outputs in the object coordinate
alsobeappliedfordriftreduction[17]andelimination[18]in
frame. A basic block diagram of a strap-down INS is given
gyroscopes.
in figure 1. To estimate the orientation (or attitude) of the
InordertoapplytheZUPTmethod,correctdetectionof
moving object, the gyroscope outputs should be integrated.
gaiteventssuchasthesteppinginstantsandcorrectestimation
Then, using the estimated orientation, accelerometer outputs
of gait parameters such as stride length are crucial for many
should be transformed to the Earth coordinate frame. The
PDR systems. This detection can be performed using only
accelerationvaluesintheEarthcoordinateframeareintegrated
inertialsensorsasin[19].Zero-velocitydetectionalgorithms
twicetogettheposition.Becauseoftheintegrationoperations
using inertial sensors are compared in [20, 21]. In [22], an
involved in the position calculation, any error in the sensor
external pressure sensor is used to detect the steps. It is also
outputs accumulates in the position output, causing a rapid
possible to perform activity recognition with inertial sensors
drift in both the gyroscope and accelerometer outputs. Thus,
to detect the stepping instants and estimate the stride length
the reliability of position estimates decreases with time. For
[23,24].
example,aconstantbiasinthegyroscopewillcauseanerror
IntegratingexternalreferencesensorswithPDRsystems
in the position that grows proportional to the cube of time,
isalsocommonintheliterature.In[13,25],ashoe-mounted
and a constant bias in the accelerometer will cause an error
inertial/magnetic system is used together with a quaternion-
that grows proportional to the square of time [7]. For this
based extended Kalman filter (EKF) to estimate the 3D path
reason, inertial sensors are usually used in conjunction with
travelled by a walking person. Magnetic sensors are used
othersensingsystemsthatprovideabsoluteexternalreference
in the initialization of the EKF. Reference [26] combines
information.
dead reckoning with GPS in outdoor environments. For
One application of INSs is in pedestrian dead reckoning
indoor environments, WiFi fingerprinting method is used
(PDR). PDR systems are generally used in GPS-denied
for localization. Reference [27] uses the GPS data for error
environments such as inside buildings, tunnels, underground correction.ThepedestriantrajectoryisestimatedusingaPDR
ordenseforestsandaroundtallbuildingsinurbanareaswhere systemandawirelesssensornetworkin[8].
GPSdataarenotaccurateoralwaysavailable.References[8] Another alternative for integrating external references is
and[9]providebriefsurveysonPDRsystems.Suchsystems map matching. If a map of the environment is available, this
are usually developed for security personnel and emergency information can be used to provide drift error correction. In
responders [10]. Unlike land vehicles and robots, a method [28],thisideaisappliedinanoutdoorenvironment,combined
called‘zerovelocityupdate’(ZUPT)enablesthestand-alone withaheuristicdrifteliminationproceduredescribedin[18].
usageofINSsonpedestrians,withoutanyexternalreference In indoor environments, activity-based map matching can be
sensor.TheZUPTmethodexploitsthefactthatduringwalking, used[24].Thisideaexploitsthefactthattheactivitycontext
the velocity of the foot is zero at some time interval during of the pedestrian gives information about his location. For
the stance phase (see section 3.1). If this time interval example, if the pedestrian is ascending stairs, most locations
is correctly detected, the drift in the velocities calculated on an indoor map can be ruled out, improving the position
instrap-downintegrationcanberesettozeroandthedriftin estimate.Here,wefollowasimilarapproach.
2
Meas.Sci.Technol.23(2012)025103 KAltunandBBarshan
In this study, we perform pedestrian localization using
five inertial and magnetic sensor units worn on the body
[29]. Localization is performed simultaneously with activity
recognition, where activity recognition cues are used in the
positionupdatestocorrectthedrifterrorsofinertialsensors.
Apart from being inherent in inertial sensors, drift errors
and offsets in body-worn systems can also arise from initial
misplacement, occasional slips from the initial position and
orientation during operation, or loose mounting on the body.
Even though the initial errors are expected to be small, they
Figure2.MTx3-DOForientationtracker
are accumulated and result in larger errors over long periods
(reprintedfromhttp://www.xsens.com/en/general/mtx).
of time. To the best of our knowledge, these issues have not
been addressed before in the literature. We demonstrate that
Xsens Technologies [32]. Each MTx unit has a tri-axial
usingagivenmapoftheenvironmentandactivityrecognition
accelerometer, a tri-axial gyroscope and a tri-axial
cues, these errors can be reduced considerably and accurate
magnetometersothatthesensorunitsacquire3Dacceleration,
localizationcanbeachievedwithouthavingtouseanyexternal
rate of turn and the strength of the Earth’s magnetic field.
referencesensor.Inpractice,theproposedmethodcanbeused
Accelerometers of two of the MTx trackers can sense in the
inapplicationswhereamapisavailableandGPSdataarenot
range±50ms−2(standardrange)andtheotherthreecansense
reliableornotavailableatall(e.g.,undergroundmines,indoor
intherangeof±180ms−2(customizedrange).Allgyroscopes
areas and urban outdoor areas with tall buildings). We note
intheMTxunitscansenseintherangeof±1200◦s−1angular
that although here we use activity recognition information
velocities; magnetometers can sense magnetic fields in the
to improve localization performance, the converse is also
rangeof±75μT.Additionally,eachsensorunithasabuilt-in
possible,i.e.localizationinformationandamapcanimprove
Kalman filter that outputs the orientation of the sensor with
theactivityrecognitionperformance.However,inourprevious
respect to a global coordinate frame (see section 3.1). Three
studies,sincewehaveobservedthatactivityrecognitionwith
orientationoutputmodescanbeusedfortheoutput:direction
high accuracy can already be achieved using proper signal
cosinematrix,quaternion andEulerangles.Inthisstudy,we
processingandpatternrecognitiontechniques[30],wefocus
usethequaternionoutputmode.
ononlyonesideoftheloopinthispaper.Inaveryrecentstudy,
The sensors are placed on five different positions on the
activityrecognitionandbodyposeestimationarecombinedin
subject’s body as shown in figure 3. Two of the customized
averysimilarway[31].
sensorunitsareplacedonthefeet,theremainingcustomized
We have performed experiments in both 2D and 3D
unitisplacedonthesubject’schestandthestandardunitsare
environments. In the 2D experiments, walking, standing
placed on the sides of the knees (the right side of the right
and turning activities are considered. In 3D localization
experiments,ascending/descendingstairsactivityisaddedto kneeandtheleftsideoftheleftknee).Thecustomizedunits
these activities. We assume that a map of the environment areusedonthefeettoavoidsaturationinthesensoroutputs,
is available and that the switches between these activities becausefeetaccelerationsareexpectedtobelargerthanknee
usually correspond to multiple locations on the map. For accelerations(upto±90ms−2inourexperiments).Thesensor
example, in an indoor environment, switching from walking units on the feet and chest are used to estimate the distance
to turning activity might correspond to the end of a corridor travelledandtheheading,respectively.Thesensorunitsonthe
or to the front of a room, whereas switching from walking legsarenotusedinthelocalizationprocess;theyareusedfor
to standing activity might correspond to a location in front activityrecognitioninthe3Dexperiments.
of a lift. Therefore, activity switches usually correspond to
several discrete locations in the environment. If one can 3.Methodology
detecttheactivityswitchescorrectly,itispossibletousethe
correspondingpositioninformationinordertocorrectthedrift In the following, we refer to several different coordinate
intheposition. frames,whicharetheglobalcoordinateframe,localnavigation
Therestofthispaperisorganizedasfollows:insection2, coordinateframesandthesensorcoordinateframes(figure4).
wedescribethesensorsusedinthisstudy.Section3explains There is a single global coordinate frame. In the default
thetheoreticalbackgroundoftheappliedmethods.Sections4 configurationofthiscoordinateframe,thezaxispointsupward
and 5 present the results of 2D and 3D experiments, along the vertical (opposite to the direction of the gravity
respectively.Weprovideadiscussionoftheresults,limitations vectorg),thexaxispointstowardsthemagneticnorthandthey
of the proposed method and related issues in section 6 and axispointstothewest,completingtheright-handedcoordinate
conclude with section 7, providing some future research frame(figure4(a)).Thelocalnavigationcoordinateframesare
directions. translatedversionsoftheglobalframetothepositionofeach
sensorunit,andtherefore,inthedefaultcase,alsohavetheir
2.Inertial/magneticsensingequipment zaxespointingupwardsalongthevertical,xaxespointingin
themagneticnorthdirectionandyaxespointingtothewest.
In this study, we use five MTx three-degree-of-freedom In other words, there is a single global coordinate frame but
(3-DOF) orientation trackers (figure 2), manufactured by five local navigation coordinate frames, one for each sensor
3
Meas.Sci.Technol.23(2012)025103 KAltunandBBarshan
one of which is for localization and the other is for activity
recognition.
3.1.Localization
Theprocessingforlocalizationisdoneintwomainsteps.In
thefirststep,thetrajectoriesarefoundusingtheZUPTmethod,
mentionedinsection1.Inthesecondstep,aKalmanfilter-like
state estimation procedure is employed to utilize the activity
recognitioncuesandimprovetheresults.
Weperformtheregularstrap-downintegrationprocedure,
using the orientation data output from the MTx sensor and
ZUPTs. A block diagram that summarizes this procedure is
depicted in figure 5. As shown in the diagram, calculations
for the distance travelled and the heading are performed
separately. To estimate the heading, it is possible to use the
orientationoutputoftheMTxuniteitheronthechestoronthe
feet.Weusethechestsensoroutputbecauseduringwalking,
thechestisarelativelystablereferencetomeasuretheperson’s
heading as opposed to the feet. That is, the signals recorded
on the chest are less oscillatory than the signals acquired
fromotherlocations.Thequaternionoutputmodeisusedfor
orientationtoavoidtheoccurrenceofanysingularitiespossible
in the Euler angle mode, even though this is unlikely for the
chest.Atthebeginningoftheexperiments,aresetoperationis
performedonthecoordinateframessuchthattheyawangleis
initiallysettozeroandismeasuredwithrespecttothevertical
Figure3.Thelocationsofthesensorunitsonthebody.(Theoutline
axisduringthemotion(seesection4.1).Then,theorientation
ofthehumanbodyistakenfrom
data are converted to Euler angles (see, for example, [33]).
http://www.anatomyacts.co.uk/learning/primary/Montage.htm.)
In the Euler angle domain, the yaw angle (ψ) represents the
instantaneous heading. Here, it is assumed that the left and
unit.Theaxesofthelocalnavigationcoordinateframesalways
right turns performed during motion are about the vertical
remainparalleltotheaxesoftheglobalcoordinateframebut
axis.
their origins are shifted to the locations of the sensor units.
To estimate the distance travelled, the sensor signals on
The sensor coordinate frames also have their origins at the
eitherfootcanbeused.First,usingtheorientationoutputofthe
positionsofthesensorunitsbuttheirthreeaxeshavearbitrary sensorunit,theaccelerationsaretransformedfromthesensor
orientationinitially,asshowninfigure4(a). coordinateframetothelocalnavigationcoordinateframe.The
Asstatedabove,theMTxunitsproviderawacceleration, transformationcansimplybeperformedas
angular velocity and magnetic field data, in addition to the
a =q a q∗ =q a q , (1)
orientation data that are calculated by the built-in Kalman L LS S LS LS S SL
filter.Inthissection,thestepsusedforprocessingthesedata where a is the acceleration vector in the local navigation
L
areexplained.Theprocessingisdoneintwoseparatetracks, frame, a is the acceleration vector in the sensor coordinate
S
N
N
xG x xG xL,xS
L
y
G
y ,y
x L S
yG zG yL zL S zG zL,zS
y
global frame zS S global frame sensor unit
sensor unit
before alignment reset after alignment reset
(a) (b)
Figure4.Topviewsoftheglobal(G),localnavigation(L)andsensor(S)coordinateframes(a)beforeand(b)immediatelyafterthe
alignmentresetoperation.
4
Meas.Sci.Technol.23(2012)025103 KAltunandBBarshan
Figure5.Blockdiagramforthefirstprocessingstep.
Figure6.Thehumangaitcycle
(figurefromhttp://www.sms.mavt.ethz.ch/research/projects/prostheses/GaitCycle).
frameandq isthequaternionrepresentingtheorientationof the angular velocity (rate), the following binary signal is
LS
thesensorcoordinateframewithrespecttothelocalnavigation constructed:
(cid:2)
frame. To estimate the position from the acceleration signal, 1, |ω(k)|(cid:2)(cid:4)
I (k)= T (2)
theaccelerationdatamustbeintegratedtwice.Becauseofthis step 0, |ω(k)|>(cid:4) ,
T
integration procedure, the errors in the sensor readings are (cid:3)
accumulated,causingunboundeddriftintheposition. wherekisthetimestep,|ω(k)|= ωx(k)2+ωy(k)2+ωz(k)2
We use the ZUPT method [10] to reduce the drift in and(cid:4)T isapre-setthresholdvalue.Thissignalisconstructed
position. When a person is walking, the motion of the leg separately for the left foot and the right foot sensors. When
is quasiperiodic. The collection of these motions within one thissignalis1,thefootisassumedtobeinthestancephase;
periodiscalledthegaitcycle.Thehumangaitcycleisroughly otherwiseitisassumedtobeintheswingphase.Toeliminate
dividedintotwophasescalledthestancephaseandtheswing possibleinstantaneous0-1-0or1-0-1switchesinthissignal,a
phase.Thestancephaseisdefinedasthetimeintervalduring medianfilterisused.Then,thevelocitiesandaccelerationsare
which the foot is in contact with the ground, and the swing settozerowhenthissignalis1,andtheintegrationsintheblock
phaseisthetimeintervalduringwhichthefootdoesnottouch diagraminfigure5areperformed. Notethattheintegrations
theground.Stancephasetakesapproximately60%ofthegait on the plane and in the z direction are performed separately,
cycle, as shown in figure 6. During a sub-interval (cid:3)T of the resulting in the signals d(k) and dz(k), which correspond to
stancephase,thefootvelocityandaccelerationareexpectedto thedistancetravelledonthex−yplaneandthepositiononthe
bezero.Thus,thetruevaluesofthevelocityandacceleration zaxis,respectively.
areknown.Ifonecansuccessfullydetectthissub-interval,the Because of the slight movement of the chest during
sensor signals can be reset to zero and the drift error in one walking,theheadingsignalcontainsripples,asshownbythe
stepwillnotbecarriedovertothenextstep. blue-dashed line in figure 7(a). This signal can be smoothed
The problem is now converted to successfully detecting using the gait phase data obtained using the aforementioned
the(cid:3)T intervalwherethefootvelocityisexactlyzero.There method. The Istep signal of the right foot is superimposed on
areanumberofdetectorsusedintheliteratureforthispurpose: thisplotinthegreen-solidlineinthesamefigure.Thesedata
accelerationmovingvariancedetector,accelerationmagnitude are obtained in an experiment where the subject stands for
detectorandangularratemagnitudedetector[20].Inarecent 5s,thenstartswalkingalongastraightline,thenturns90◦ to
study,analternativedetectorwasproposedthatgivesslightly therightataboutt = 25sandcontinueswalking.Ascanbe
better results than the angular rate magnitude detector [20]. observed in the figure, when a right step is taken (i.e. when
However, in most of the studies, the angular rate magnitude I = 0fortherightfoot),thechestangleswingsslightlyto
step
detector outperforms the others. We use the angular rate the left, and vice versa. To remove the ripples, the mean of
magnitude detector in this study because of its performance theheadingdatabetweenrisingedgesoftheI signalcanbe
step
and simplicity of implementation. Using the magnitude of calculatedandreplacedasacorrectedheadingsignal.Thisis
5
Meas.Sci.Technol.23(2012)025103 KAltunandBBarshan
1.5 1.5
1 1
0.5 0.5
0 0
d) d)
a a
ψ (r−0.5 ψ (r−0.5
−1 −1
−1.5 −1.5
−2 −2
0 10 20 30 40 0 10 20 30 40
t (s) t (s)
(a) (b)
Figure7.(a)Originalheadingsignal(blue-dashedline)andswing-stancephaseindicatorvariable(green-solidline)superimposed;
(b)originalheadingsignal(blue-dashedline)andcorrectedheadingsignal(red-solidline).
showninfigure7(b).Inthisfigure,theoriginalheadingdata long periods of walking. This could be caused by attaching
areshownbytheblue-dashedlineandthecorrectedheadingis the sensors to loose rather than tight clothing. Magnetic
shownbythered-solidline.Obviously,thiscorrectionshould disturbance caused by the ferromagnetic materials in the
bemadeseparatelyforeitherfootdependingonwhichfoot’s environmentisanothersourceoferrorforthemagnetometers
data are used in evaluating d(k), using the I indicator for that directly affects the heading. Accelerometer data can be
step
that foot. In this case, the correction is made using the right used to estimate the inclination angle, but the only external
footdata.Thecorrectedheadingdataaredenotedasψ(k)in referenceavailablefordeterminingtheheadingisthemagnetic
therestofthistext. field data. Furthermore, the thresholds that we use are fixed
Afterdeterminingd(k),d (k)andψ(k),thepathcanbe constants, i.e. they are not selected specifically for the
z
reconstructedusingthesimplestatemodelgivenbelow: person wearing the sensors. Considering the age, height and
weightvariationsamongpeople,sucherrorsareunavoidable.
x(k)=x(k−1)+(cid:3)d(k−1)cos[ψ(k−1)] Therefore,weusecuesobtainedfromactivityrecognitionand
y(k)=y(k−1)+(cid:3)d(k−1)sin[ψ(k−1)] (3) performpositionupdateswhensuchcuesareavailable,inorder
toimprovetheresults.
z(k)=z(k−1)+(cid:3)d (k−1),
z
3.2.Activityrecognition
withtheinitialconditionsx(0),y(0)andz(0).Here,(cid:3)d(k−1)
=d(k)−d(k−1)representsthedistancetravelledontheplane Inourearlierwork[30],wedemonstratedthatitispossibleto
and(cid:3)d (k−1)=d (k)−d (k−1)representsthedisplacement distinguishbetweenvariousactivitiesusingbody-worninertial
z z z
inthezdirection,duringthekthtimestep. and magnetic sensors and provided an extensive comparison
By defining a state vector ξ(k) = between various classifiers. Simple Bayes classifiers with
[x(k), y(k), z(k)]T and an input vector u(k) = Gaussianprobabilitydensityfunctionsaresufficienttoobtain
[(cid:3)d(k)cosψ(k), (cid:3)d(k)sinψ(k), (cid:3)d (k)]T, the equa- over95%correctclassificationratesiftrainingdatafromthat
z
tionbecomes specificpersonareavailable.However,ifsuchtrainingdataare
notavailabletotheclassifiers,morecomplexclassifierssuch
ξ(k)=ξ(k−1)+u(k−1) (4)
as the k-nearest neighbour method (k-NN) or support vector
with the initial condition ξ(0) = [x(0), y(0), z(0)]T. In the machines (SVM) can be utilized that have expected correct
2D experiments, we do not consider the z direction. That is, classification rates of about 85%. The reader is referred to
d (k) is not calculated and the state z(k) is deleted from the [30,34–36]forsurveysoftheliteratureonactivityrecognition
z
statevectorintheseexperiments. usingbody-wornsensors.
The performance of the above model depends on the Inourexperimentsin2D,weconsiderareducedactivity
performances of the distance and the heading estimation set, comprised of walking, standing and turning activities.
methods. In our experiments, we observed that both have Sincethesethreeactivitiesarequitedifferentfromeachother,
errors,whichcausesthereconstructedpathtodriftovertime. usingcomplexclassifiersisnotnecessary.Weusearule-based
This drift is naturally amplified as the length of the walking classifierforthesethreeactivities,inwhichthefollowingrules
path increases. The most dominant cause of error is the areappliedinthegivenorder:
dislocation of the mounted sensors during the experiments, (i) ifthefilteredheadingvalueisaboveacertainthreshold,
especiallytheheadingsensor.Forexample,aslightdislocation theactivityisclassifiedasturning;
of the chest sensor causes a slight measurement error in (ii) ifbothfeetarestationary,thentheactivityisclassifiedas
the heading that causes the path to drift drastically over standing;
6
Meas.Sci.Technol.23(2012)025103 KAltunandBBarshan
(iii) if the above conditions do not hold, then the activity is withtheinitialconditionξ(0)modelledasaGaussianrandom
classifiedaswalking. vector with mean μξ(0) and covariance matrix Pξ(0). Note
For the first rule, the heading signal is passed through a that here ξ(k) is a random process and is different from the
first-orderdifferencefilteroflength1sandthresholded.The deterministicstatevectorinequation(4).However,weusethe
second rule is realized by performing an AND operation on samenotationforsimplicity.Theinputu(k)isthesameasin
the I indicator variables (equation (2)) for the left and the equation (4). In equation (5), Rθ represents a rotation on the
step
rightfeet. planebyanarbitrary⎛angleθ: ⎞
Forour3Dexperiments,weintroducethe‘stairs’activity cosθ −sinθ 0
totheactivitysetthatrepresentstheactivitystateofthesubject Rθ =⎝sinθ cosθ 0⎠, (6)
whileascendingordescendingstairs.Distinguishingbetween 0 0 1
walking and stairs activities is not straightforward, and a and w(k) is the process noise modelled as a white Gaussian
simple rule-based method like the one applied above cannot
noise with a diagonal covariance matrix Q. In equation (5),
beusedinthiscase.Therefore,weusethek-NNclassifier.The the noise vector is rotated by ψ(k) at each time step k. This
data acquired in our previous work [30] are employed as the
way,thenoiseintroducedtothesystemismodelledsuchthat
trainingdatafortheclassifier.Fromthatarticle,wecombine
itisuncorrelated(andindependent,sinceitisGaussian)inthe
the data of the activities walking in a parking lot (A9) and
current heading direction and in the perpendicular direction
walking on a treadmill (A10) to get the ‘walking’ class, and
to the heading. If there were no rotation, the noise would be
data of ascending stairs (A5) and descending stairs (A6) to
uncorrelated in the global x and y directions, as long as the
getthe‘stairs’class.Weusethestanding(A2)activityforthe
covariancematrixQisdiagonal.Webelievethatintroducing
‘standing’ class directly. To recognize these three activities,
this rotation matrix is a more realistic assumption for our
weusethesensorsontherightandtheleftlegs,sincetheyare
model than assuming the noise in the x and y directions as
mountedatthesamepositionasinthatarticle.Therefore,the
beinguncorrelated.
dataareexpectedtobesimilar.Wecalculatetherunningmean
Suppose that an AB activity switch is detected and a
and running variance values from the test data as features,
positionupdateisperformedataprevioustimek = k .Until
using a sliding window of length 5 s. This length is chosen 1
thenextpositionupdate,equation(5)canbeusedtomodelthe
since the same length is also used in the training data for
position. The prediction equations using this forward model
feature extraction. We do not use magnetometer data, since
aregivenas
theaccuracyofmagnetometersisknowntodegradeinindoor
environments[16].Thek-NNclassifierisusedtodistinguish ξˆ (k|k ) =ξˆ (k−1|k )+u(k−1)
f 1 f 1
between walking, standing and stairs activities, whereas the (cid:4)f(k|k1)=R(cid:3)ψ(k)(cid:4)f(k−1|k1)RT(cid:3)ψ(k)+Rψ(k)QRTψ(k) (7)
turningactivityisrecognizedusingthesameruleasintherule-
fork>k ,wherethesubscript f standsfortheforwardmodel
based method described above. Then, the switches between 1
and(cid:3)ψ(k)=ψ(k)−ψ(k−1).Theinitialconditionsforthese
activities and corresponding time values are determined and
predictionequationsdependontheactivityswitchatk = k .
usedforpositionupdates,asexplainedinthefollowingsection. 1
Theyaregivenasξˆf(k1|k1) = μAB,n and(cid:4)f(k1|k1) = PAB,n,
wherenistheindexofthecorrespondingactivityswitchpoint
3.3.Simultaneouslocalizationandactivityrecognition
onthemap.Ifnopositionupdateisperformeduptotimek,then
In this section, we combine the localization results with k = 0andtheinitialconditionsfortheforwardfilterarethe
1
position updates simultaneously obtained from activity initialconditionsofthestatemodel.Thatis,ξˆf(0|0)=μξ(0)
recognition cues. We assume that a map of the environment and(cid:4)f(0|0)=Pξ(0).
is available and some of the switches between recognized WhenanactivityswitchfromactivityCtoactivityD(i.e.
activities correspond to multiple locations on the map, in a CD switch) is detected at k = k , we run the same system
2
general. That is, knowledge of an activity switch provides backwards in time, all the way back to the previous activity
informationaboutthepossiblepositionsonthemap. switchABandpositionupdateatk =k .Thebackwardfilter
1
Supposethat,foragivenmap,aswitchfromactivityAto equationsare
activityBcanoccuratNABdifferentpoints.TheplaceholdersA ξˆ (k−1|k )=ξˆ (k|k )−u(k−1)
andBcanstandforanyactivityinouractivityset,i.e.walking b 2 b 2
(W), standing (S), turning (T) or stairs (R). For example, a (cid:4)b(k−1|k2)=R(cid:3)ψ(k−1)(cid:4)b(k|k2)RT(cid:3)ψ(k−1)
walking-to-standing activity switch is denoted as WS and a +Rψ(k−1)QRTψ(k−1) (8)
walking-to-stairs activity switch is denoted as WR. In the for k < k (cid:2) k , where the subscript b stands for the
1 2
following, the nth AB activity switch point is modelled as backward model and (cid:3)ψ(k−1) = ψ(k−1)−ψ(k). The
a Gaussian random vector with mean μ and covariance
AB,n initialconditionsforthesepredictionequationsagaindepend
PAB,n, where n = 1,...,NAB. The mean corresponds to the on the current activity switch at k = k , and are given by
2
cthoeorcdoivnaartieasncoefmthoedeexlsptehceteudncloecrtaatiinotnyoonftthheelogcivaetinonm.ap, and ξˆb(k2|k2) = μCD,n∗ and(cid:4)b(k2|k2) = PCD,n∗.Thesubscriptn∗
indicatesthepredefinedCDswitchlocationonthemapthatis
In the previous section, we use the state equation (4) to
theclosesttotheforwardstateestimatejustbeforetheposition
predicttheposition.Tomodeltheuncertaintyintheposition,
update.Moreprecisely,
considerthestateequation
ξ(k)=ξ(k−1)+u(k−1)+Rψ(k)w(k), (5) n∗ =argmnin||ξˆf(k2|k1)−μCD,n||. (9)
7
Meas.Sci.Technol.23(2012)025103 KAltunandBBarshan
15 the backward filter should be run all the way back to the
previous activity switch. Since there is no previous activity
switch, the backward filter is run to the beginning, k = 0.
10 This path is shown by the magenta-dashed line. Then, these
estimatesarecombinedtogettheimprovedestimate,whichis
shownbytheblue-solidlineinthefigure.Thereconstruction
5 almost coincides with the ground truth after the update, as
confirmedbythefigure.
)
m
0 4.2DExperiments
(
y
4.1.Experimentalsetup
−5 Atotalof11experimentsareperformedin2D,intwodifferent
environments. The first set of experiments is performed
outdoorsonastraightlineof66mlength.Thelineisdivided
−10
intofoursegmentsofequallength,andtheendpointsofeach
segmentaremarkedwitha+ora×sign.Thepathisillustrated
infigure9.
−15
0 5 10 15 20 25 30 Acoordinateframeisassignedinthisenvironmentsuch
x (m) that the line coincides with the x axis. The origin of the
coordinate frame is at the leftmost point of the line. The ×
Figure8.Optimalcombination(blue-solidline)oftheforward marksindicatepossiblelocationstoperformthe‘walking-to-
(green-dash-dottedline)andbackward(magenta-dashedline)
standing’ (WS) activity switch, and the + marks indicate the
estimates.Thethinred-solidlineshowsthetruepath.
locations to perform the ‘walking-to-turning’ (WT) activity
switch. Note that point (66,0) is marked with both symbols
Atthispoint,foreachk=k +1,...,k −1,wehavetwo
1 2 meaningthatitispossibletoperformbothWSandWTactivity
estimatesavailablefortheposition.Thelinearcombinationof
switchesatthislocation.
thesetwoestimateswiththeminimumcovarianceis(seethe
In this outdoor environment, four experiments are
appendix)
performed:
ξˆ(k|k1,k2)=(cid:4)(k|k1,k2) (1) startfrompoint(0,0),stopat(16.5,0),stopat(49.5,0),
×[(cid:4)f(k|k1)−1ξˆf(k|k1)+(cid:4)b(k|k2)−1ξˆb(k|k2)], (10) stopat(66,0);
where (cid:4)(k|k ,k ) = [(cid:4) (k|k )−1 + (cid:4) (k|k )−1]−1 is the (2) start from point (0,0), stop at (16.5,0), turn back at
1 2 f 1 b 2 (33,0),stopat(16.5,0),stopat(0,0);
covarianceofthecombinedestimate.
(3) startfrompoint(0,0),stopat(16.5,0),stopat(49.5,0),
In practice, we run the forward filter in a causal manner
turn back at (66,0), stop at (49.5,0), stop at (16.5,0),
untilanactivityswitchisdetected.Whenanactivityswitchis
detectedatk =k ,thebackwardfilterisrunallthewayback stopat(0,0);
to the previous p2osition update at k = k , and the position (4) start from point (0,0), stop at (49.5,0), turn back at
estimatesfork=k +1,...,k −1areca1lculated.Ifthereis (66,0),stopat(16.5,0),stopat(0,0).
1 2
nopreviouspositionupdate,thenk1 =0.Aftertheupdateand Notethatitisnotrequiredtostopatevery×mark,orturn
the smoothing operation, the new k1 value is assigned as k2. backatevery+mark,butthesemarksindicatesomenonzero
Thisisillustratedinfigure8thatincludesaportionofoneof likelihoodthattheseeventswilloccuratthatlocation.
our2Dexperiments.Intheexperiment,thesubjectstartsfrom ThesportshallofBilkentUniversityisusedasthesecond
point (0,0) and walks in the +x direction, which is shown environment.Thesubjectsarerequiredtowalkonlinesdrawn
bythethinred-solidlineandrepresentsthegroundtruth.The on the floor. The map of this indoor environment is shown
green-dash-dotted line shows the reconstructed path until an in figure 10. Similar to the first setup, the × marks indicate
activityswitchisdetected,whichoccursatpoint(16.5,0).The possiblelocationstoperformthestandingactivity.Eachcorner
reconstructed path is drifting from the actual path, as shown inthefigureindicatesapossiblelocationtoperformtheturning
inthefigure. activity. Thus, the WS and WT activity switch points on the
Theaverageheadingerrorisabout18◦.Suchlargeheading map are assigned manually; all corners are defined as WT
errorsarenotfrequentlyobservedinourexperiments;however, switchpointsandWSswitchpointsareassignedarbitrarily.
this experiment is chosen to demonstrate the performance of Thesevenexperimentsperformedinthisenvironmentare
combiningactivityrecognitioncues.Aftertheactivityswitch, asfollows:
Figure9.Thepathfollowedinthefirstfourexperiments(alldimensionsinm).
8
Meas.Sci.Technol.23(2012)025103 KAltunandBBarshan
Table2.Profilesoftheeightsubjects.
Subjectno Gender Age Height(cm) Weight(kg)
S1 f 32 158 45
S2 f 34 161 51
S3 m 25 180 79
S4 f 22 166 47
S5 f 24 178 60
S6 m 33 175 95
S7 m 22 187 75
S8 m 25 182 75
reset, the global and local navigation frames are in their
default configuration. However, at the reset instant, the x–y
orientationoftheseframesmaychangearbitrarily,whiletheir
zaxesremainperpendiculartothehorizontalplane,opposite
to the direction of the gravity vector. Immediately after
Figure10.Thepathfollowedinthesecondsetofexperiments(all the alignment reset, the local navigation and the sensor
dimensionsinm). coordinate frames are coincident. All orientation outputs
duringtheexperimentsareobtainedwithrespecttothelocal
Table1.Totalpathlengthsoftheexperiments.
navigation coordinate frames, illustrated in figure 4(b) for
Experimentno Pathlength(m) a single sensor unit. After the alignment reset, the sensor
coordinate frames may rotate and translate with the motion
1 66
oftheperson,whereastheglobalframeremainsfixedandthe
2 66
localnavigationframesmaytranslatebutnotrotate.
3 132
4 132
5 222 4.2.Experimentalresults
6 222
7 90 In this section, we present and compare the results of the
8 90
reconstructionwithandwithoutusinganyactivityrecognition
9 33.9
cues.Wecalculatetheerrorbetweenthereconstructedpathand
10 96.2
thetruepathbydiscretizingthetruepathwithequallyspaced
11 96.2
points on the path, and consider either path as a finite set of
points.Weuseasymmetricerrorcriterionbetweentwopoint
(5) walkforthreelapsonarectangleofsize24m×13m; sets P and Q, proposed in [37]. The well-known Euclidean
(6) walk for three laps on a rectangle of size 24 m × 13 m, distanced(p,q ):R3×R3 →R(cid:2)0oftheithpointintheset
i j
stoppingatthemidpointofthelongerside; Pwiththepositionvectorp = (p ,p ,p )T tothe jthpoint
i xi yi zi
(7) walkforthreelapsonarectangleofsize9m×6m; q = (q ,q ,q )T insetQisgivenby
j xj yj zj
(8) walk for three laps on a rectangle of size 9 m × 6 m, (cid:8)
stoppingatthemidpointofthelongerside; d(p,q )= (p −q )2+(p −q )2+(p −q )2,
i j xi xj yi yj zi zj
(9) walkforthreelapsonacircleofdiameter3.6m,stopping (11)
eachtimeattheendpointsofthediameter;
(10) walkforonelaponarectilinearpolygon; where i ∈ {1,...,N1} and j ∈ {1,...,N2}. In [37], we
(11) walkforonelaponarectilinearpolygon,stoppingatthree consider and compare three different metrics to measure
differentpoints. the similarity between two sets of points, each with certain
advantagesanddisadvantages.Inthiswork,weusethemost
Thetotalpathlengthsoftheseexperimentsaretabulated
favourable of them to measure the closeness or similarity
intable1.These11experimentsareperformedbyfourmale
betweenthesetsPandQ:
andfourfemalesubjects,whoseages,heightsandweightsare
1
presentedintable2. E =
(P−Q)
Before starting the experiments, an ‘alignment reset’ is ⎛ 2 ⎞
performedoneachsensorunittoresetthecoordinateframes ×⎝1 (cid:9)N1 min{d(p,q )}+ 1 (cid:9)N2 min{d(p,q )}⎠.
stoucthhethuatnittheopienriatitaolro(rtiheanttaitsi,onthetrainnsiftioarlmoartiieonntactoiornresopuotnpdust N1 i=1 qj∈Q i j N2 j=1 pi∈P i j
is I3×3 in the direction cosine matrix mode, q = 1 in the (12)
quaternion output mode or zero Euler angles in the Euler Accordingtothiscriterion,wetakeintoaccountallpointsin
angleoutputmode),andthezaxesareintheverticaldirection. the two sets and find the distance of every point in the set
The top views of the global, local navigation and the sensor PtothenearestpointinthesetQandaveragethem,andvice
coordinateframesbeforeandimmediatelyafterthealignment versa. The two terms in equation (12) are also averaged, so
reset are shown in figure 4. Note that before the alignment thatthecriterionissymmetricwithrespecttoPandQ.
9