Visual Multiple-Object Tracking for Unknown Clutter Rate

0.33 MB·

by  Du Yong Kim

Visual Multi-Object Tracking for Unknown Clutter Rate
DuYongKim
DepartmentofElectricalandComputerEngineering,CurtinUniversity,Bentley,6102
7
1
0
2 Abstract

r
a Inmostmulti-objecttrackingalgorithms,tuningofmodelparametersisofcriticalim-
M
portanceforreliableperformance. Inparticular,weareinterestedindesigningarobust

1
tracking algorithm that is able to handle unknown false measurement rate. The pro-
3

posedalgorithmisbasedoncouplingoftworandomfinitesetfiltersthatsharetracking
]
V
parameters. Performanceevaluationwithvisualsurveillanceandcellmicroscopyim-
C
agesdemonstratestheeffectivenessofthetrackingalgorithmforreal-worldscenarios.
.
s
c Keywords: Multi-objecttracking,Visualtracking,randomfinitesets
[

2
v 1. Introduction
3
7
2 Multi-objecttrackingisoneoffundamentalproblemsinmanyapplications. There
2
are abundant research works, however, it is still far from practical use. The over-
0
.
1 whelmingmajorityofmulti-targettrackingalgorithmsarebuiltontheassumptionthat
0
multi-object system model parameters are known a priori, which is generally not the
7
1
caseinpractice[1]. Whiletrackingperformanceisgenerallytoleranttomismatchesin
:
v
i thedynamicandmeasurementnoise,thesamecannotbesaidaboutmisseddetections
X
and false detections. In particular, mismatches in the specification of false detection
r
a
model parameters such as clutter intensity can lead to a significant bias or even erro-
neousestimates[2].
Unfortunately, except for a few application areas, exact knowledge of model pa-
rametersisnotavailable. Thisisespeciallytrueinvisualtracking,inwhichthemissed
detection and false detection processes vary with the detection methods. The clutter
intensitymodelcanbeobtainedbytrialanderror.Amajorproblemisthetime-varying
natureoffalsedetectionprocesses. Consequently,thereisnoguaranteethatthemodel
PreprintsubmittedtoPatternRecognitionLetters April3,2017
parameterschosenfromtrainingdatawillbesufficientforthemulti-objectfilteratsub-
sequentframes.
Stochasticmulti-objecttrackingalgorithmsbasedonKalmanfilteringorSequential
MonteCarlo(SMC)methodhavebeenwidelyusedincludingvisualmulti-objecttrack-
ing research [3], [4] where detection profile and false measurement rate are assumed
tobeknown. Ontheotherhand,deterministicapproachsuchasnetworkflow[5],con-
tinuousenergyoptimisation[6],hasbecomeapopularmethodforvisualmulti-object
tracking problem. This approach is known to be free from measurement model pa-
rameters,however,itrequiresotheruser-specifiedparametersandisusefulonlywhen
reliableobjectdetectionisavailablewithnegligiblefalsemeasurements.
Unknownobservationmodelparameters(i.e.,clutterrate,detectionprofile)inon-
linemulti-objectfilteringwasrecentlyformulatedinajointestimationframeworkus-
ingrandomfiniteset(RFS)approach[1]. Recently,Mahler[2]showedthatcleveruse
of the CPHD filter can accommodate unknown clutter rate and detection profile. [7]
extendedittomulti-BernoullifilterwithSMCimplementation. Whilethesolutionfor
filteringwithunknownclutterrateexists,thesefiltersdonotprovidetracksthatiden-
tify different objects. To the best of our knowledge this paper is the first attempt for
handlingunknownfalsemeasurementinformationinonlinetrackingwithtrajectories.
The main contribution of this paper is to design a multi-object tracker that produces
trajectoriesofindividualobjectsandestimatesunknownclutterrateonthefly.
2. ProblemFormulation
LetX = Rnx denotethespaceofthetargetkinematicstate,and{0,1}denotethe
discretespaceoflabelsforcluttermodelandactualtargets. Then,theaugmentedstate
spaceisgivenbyX˘ =X×{0,1},where×denotesaCartesianproduct.Consequently,
thestatevariablecontainsthe kinematicstate, andtarget/clutterindicator. Wefollow
theconventionfrom[7]thatthelabelu = 0willbeusedasasubscripttodenotethe
cluttergeneratorsandthelabelu=1foractualtargets.
Suppose that there are T target and clutter object, and we have O observations
k k
(i.e., detections). In the RFS framework, the collections of targets (including clutter
2
objects)andmeasurementscanbedescribedasfinitesubsetsofthestateandobserva-
tion spaces, respectively as X˘ ={x˘ }Tk ⊂X˘, Z ={z }Ok ⊂Z, where x˘
k k,i i=1 k k,j j=1 k,i
representeitherthekinematicstateofactualtargetorcluttertarget;z isameasure-
k,j
ment,andZisthespaceofmeasurement,respectively.
Considering the dynamic of the state, the RFS model of the multi-target state at
timekconsistsofsurvivingtargetsandnewtargetsenteringinthescene. Thisnewset
isrepresentedastheunion
X˘ = (cid:91) S (x˘ ) (cid:91)Γ ,
k k|k−1 k−1 k (1)
x˘k−1∈X˘k−1
whereΓ isasetofspontaneousbirthobjects(actualorcluttertargets)andS (·)
k k|k−1
isthesetofsurvivedobjectstatesattimekwithsurvivalprobabilityp (x˘)<1.
S
Thesetofobservationsgiventhemulti-targetstateisexpressedas
Z =Z (cid:83)Z , (2)
k T,0,k T,1,k
whereZ andZ are,respectively,setsofclutterandtarget-originatedobserva-
T,0,k T,1,k
tionswithunknowndetectionprobabilityp (x˘)<1.
D
WiththeRFSmulti-targetdynamicandmeasurementmodel,themulti-objectfilter-
ingproblemamountstopropagatingmulti-targetposteriordensityrecursivelyforward
intimeviatheBayesrecursion. Notethatintheclassicalsolutiontothisfilteringprob-
lem such as PHD, CPHD [1], and multi-Bernoulli filters [4], instead of clutter target
measurementset,thePoissonclutterintensityisgivenandthedetectionprofilep (x)
D
isalsoknownapriori[1].
3. Multi-objecttrackerwithunknownclutterrate
The aim of this paper is to propose a new online multi-object tracker that is able
to accommodate unknown clutter rate. For this purpose, the Robust Multi-Bernoulli
(RMB) filter [7] is employed to adapt unknown clutter rate. Then, the RMB filter
is coupled with the Generalized Multi-Bernoulli (GLMB) tracker [8] by exchanging
parameterstoachieverobustperformanceaswillbegiveninSection4.
3
3.1. RobustMulti-BernoulliFilter
Themulti-Bernoullifilterparametrizesthemulti-objectposteriordensitybyusing
asetofpairs,i.e.,Bernoulliparameters,{(r(i),p(i))}M wherer(i) andp(i) represent
i=1
theexistenceprobabilityandthedensityofthestateamongM Bernoullicomponents.
The multi-Bernoulli filter recursion for extended state space called RMB filter [7] is
summarizedtomakethepaperself-contained.
If the posterior multi-object density of the multi-Bernoulli form at time k −1 is
given as {(r(i) ,p(i) )}Mk−1. Then, the predicted intensity is approximated by a
k−1 u,k−1 i=1
setoffollowingmulti-Bernoullicomponents{(r(i) ,p(i) )}Mk|k−1.Asetofpre-
k|k−1 u,k|k−1 i=1
dicted Bernoulli components is a union of birth components and surviving compo-
nents. The birth Bernoulli components, {(r(i),p(i) )}MΓ,k, are chosen a priori by
Γ,k Γ,u,k i=1
consideringtheentranceregionofthevisualscene,e.g.,imageborder. Thesurviving
componentsarecalculatedby
r(i) =r(i) (cid:80) (cid:104)p(i) ,p (cid:105), p(i) (x)= (cid:68)fu,k|k−1(x˘|·,)p(ui,)k−1,pS,u,k(cid:69),
P,k|k−1 k−1u=0,1 u,k−1 S,u,k u,k|k−1 (cid:68)p(ui,)k−1,pS,u,k(cid:69)
(3)
wherep isthesurvivalprobabilitytotimekandf (x˘|·)isthestatetransition
S,u,k u,k|k−1
densityspecifiedbyeitherforactualtargetorforcluttersasdenotedbyu.
If at time k, the predicted multi-target density is multi-Bernoulli form, then the
updatedmulti-Bernoullidensityapproximationiscomposedofthelegacycomponents
withthesubscriptLandthemeasurementupdatedcomponentswiththesubscriptU as
{(r(i) ,p(i) )}Mk|k−1∪{(r (z),p (·;z))} . Thelegacyandmeasurements
L,k L,u,k i=1 U,k U,u,k z∈Zk
updatedcomponentsarecalculatedbyaseriesofequations(4)asfollows.
r(i) =(cid:80) r(i) , r(i) = rk(i|)k−1(cid:68)p(ui,)k|k−1,1−pD,u,k(cid:69) ,
L,k u=0,1 (cid:0)L,u,k (cid:1)L,u,k 1−rk(i|)k−1(cid:80)u(cid:48)=0,1(cid:68)p(ui(cid:48)),k|k−1,pD,u(cid:48),k(cid:69)
p(i) (x)= 1−pD,u,k p(ui,)k|k−1(x) , r (z)=(cid:80) r (z),
L,u,k (cid:80)u(cid:48)=0,1(cid:68)p(ui(cid:48)),k|k−(cid:0)1,1−pD,u(cid:1)(cid:48),k(cid:69) U,k u=0,1 U,u,k
(cid:80)Mk|k−1 rk(i|)k−1 1−rk(i|)k−1 (cid:104)p(ui,)k|k−1,gu,k(z,|·)pD,u,k(cid:105)
r (z)= i=1 (cid:0)1−rk(i|)k−1(cid:80)u(cid:48)=0,1(cid:28)p(ui(cid:48))=0,1,pD,u(cid:48),k(cid:29)(cid:1)2 , (4)
U,u,k (cid:80)Mk|k−1 rk(i|)k−1(cid:80)u(cid:48)=0,1(cid:104)p(ui,)k|k−1,gu(cid:48),k(z|·)pD,u(cid:48),k(cid:105)
i=1 1−rk(i|)k−1(cid:80)u(cid:48)=0,1(cid:28)p(ui(cid:48)),k|k−1,pD,u(cid:48),k(cid:29)
p (x;z)= (cid:80)iM=k1|k−1 1−rk(ri|k()ki|)−k−11p(ui,)k|k−1(x)gu,k(z|x)·pD,u,k
U,u,k (cid:80)u(cid:48)=0,1(cid:80)Mi=k1|k−1 1−rk(ri|k()ki|)−k−11(cid:68)p(ui(cid:48)),k|k−1,gu(cid:48),k(z|·)pD,u(cid:48),k(cid:69)
4
where p is the state dependent detection probability, g (z|x) is the measure-
D,u,k u,k
ment likelihood function that will be defined in the following section. Note that the
implementationdetailscanbefoundin[7].
3.2. GeneralizedlabeledMulti-BernoulliFilter
Thegeneralizedlabeledmulti-Bernoulli(GLMB)filterprovidesasolutionofmulti-
objectBayesfilterwithuniquelabels[8].Inmulti-objecttrackingwithlabels,formally,
the state of an object at time k is defined as x = (x ,(cid:96) ) ∈ X×L , where L de-
k k k k k
notesthelabelspaceforobjectsattimek (includingthosebornpriortok). Notethat
L isgivenbyB ∪L ,whereB denotesthelabelspaceforobjectsbornattimek
k k k−1 k
(andisdisjointfromL )andwedonotconsidercluttertargetsindesigningGLMB,
k−1
thus,thelabeluisnotintroduced.
Suppose that there are N actual targets at time k, with label x ,...,x , in
k k,1 k,Nk
the context of multi-object tracking, X = {x ,...,x } ∈ F(X×L ), where
k k,1 k,Nk k
F(X×L )denotesthespaceoffinitesubsetsofX×L .Wedenotecardinality(num-
k k
berofelements)ofXby|X|andthesetoflabelsofX,{(cid:96):(x,(cid:96))∈X},byL . Note
X
thatsincethelabelisunique,notwoobjectshavethesamelabel,i.e. δ (|L |)=1.
|X| X
Hence∆(X)(cid:44)δ (|L |iscalledthedistinctlabelindicator.
|X| X
In the GLMB the posterior density takes the form of a generalized labeled multi-
Bernoulli
π (X)=∆(X)(cid:88)ω(c) (L )(cid:104)p(ξ) (cid:105)X. (5)
k−1 k−1 X k−1
c∈C
Given the posterior multi-object density of the form (5), the predicted multi-object
densitytotimekisgivenby
π (X)=∆(X)(cid:88)ω(c) (L )(cid:104)p(c) (cid:105)X (6)
k|k−1 k|k−1 X k|k−1
c∈C
where
ω(c) (L)=w (L∩B )ω(c)(L∩L ),
k|k−1 B,k k S,k k−1
p(kc|)k−1(x,(cid:96))=1Lk−1((cid:96))p(Sc,)k(x,(cid:96))+(1−1Lk−1((cid:96)))pB,k(x,(cid:96)),
p(c)(x,(cid:96))= (cid:68)pS,k−1(·,(cid:96))fk|k−1(x|·,(cid:96)),p(kc−)1(·,(cid:96))(cid:69), η(c)((cid:96))=(cid:68)p (·,(cid:96)),p(c) (·,(cid:96))(cid:69),
S,k η(c)((cid:96)) S,k S,k−1 k−1
ω(c)(L)=[η(c)]L (cid:88) [qS(,kc)]I−Lω(c) , q(c)((cid:96))=(cid:68)1−p (·,(cid:96)),p(c) (·,(cid:96))(cid:69),
S,k S,k S k−1 S S,k−1 k−1
I⊆Lk−1
5
wherecistheindexfortrackhypothesis,Lisaninstanceoflabelset,I istracklabels
fromprevioustimestep.
Moreover,theupdatedmulti-objectdensityisgivenby
πk|k(X|Zk)=∆(X)(cid:88) (cid:88) ωZ(ck,θ)(LX)(cid:104)p(kc|,kθ)(·|Zk)(cid:105)X (7)
c∈Cθ∈Θk
whereΘ isthespaceofmappingsθ :L →{0,1,...,|Z |},suchthatθ(i)=θ(i(cid:48))>
k k k
0impliesi=i(cid:48),and
ω(c,θ)(L)∝δ (L)ω(c) (L)[η(c,θ)]L,
Zk θ−1({0:|Zk|}) k|k−1 Zk
p(c,θ)(x,(cid:96)|Z )= p(kc|)k−1(x,(cid:96))ψZk(x,(cid:96);θ), η(c,θ)((cid:96))=(cid:68)p(c) (·,(cid:96)),ψ (·,(cid:96);θ)(cid:69),
k|k k η(c,θ)((cid:96)) Zk k|k−1 Zk
Zk
ψ (x,(cid:96);θ)=δ (θ((cid:96)))(1−p (x,(cid:96)))+(1−δ (θ((cid:96))))pD,k(x,(cid:96))gk(zθ((cid:96))|x,(cid:96))
Zk 0 D,k 0 κk(zθ((cid:96)))
whereκ ∼λ U(Z)denotestheclutterdensity.λ istheclutterratewhichisassumed
k c c
toknownasaprior.
3.3. BoostedGeneralizedlabeledMulti-BernoulliFilter
In this paper, we propose a new RFS filter called Boosted GLMB filter that is a
principled combination of RMB filter [7] and GLMB filter [8]. The GLMB filter is
used as a main tracker that produces track parameters. The one-step RMB filter is
employedtoadaptunknownclutterratebasedonprovidedtrackparameters(position
meanandcovariance)andtrackexistenceprobabilityoftheGLMBfilterfromthepre-
vious time step. Then, the estimated clutter rate from the one-step RMB filter is fed
intotheGLMBfilterforcluttermodeltoboostaccuratetrackingresults. Figure1rep-
resentstherelationbetweentheGLMBfilterandone-stepRMBfilter.
Aswillbespecifiedinthefollowingsection,weconsideralinearGaussianmodel,
thus,theBoostedGLMBfilterisimplementedbyusingKalmanfilters. Trackparam-
etersinthisimplementationisasetofGaussianparameters,i.e.,meanandcovariance
matrixthatrepresentkinematicstatesoftheobject.
Once the tracker is initialized, state mean, covariance matrix and track existence
probabilityareproducedbytheGLMBfilter. Inaddition,thetrackexistenceprobabil-
ityr((cid:96)) isobtainedbyenumeratingtrackhypothesisweightsfromtheGLMBdensity
6
as
(cid:88)(cid:88)
r((cid:96)) = 1 ((cid:96))ω(I,ξ), (8)
I
ξ∈ΞI⊆L
where I and ξ are track labels and data association map from the previous time step
andthecalculatedtrackexistenceprobabilityr((cid:96)) isusedastheinitialvaluesoftrack
existenceprobabilityrinRMBpredictionEq. (3).
Then,theclutterrateissimplyobtainedbytheEAPestimateofcluttertargetnum-
beras
λˆ =(cid:88)Mk r(i)p (9)
c 0,k D,0,k
i=1
wherer(i) istheexistenceprobabilityofcluttertargetintroducedintheprevioussec-
0,k
tion,p istheprobabilityofdetectionforcluttertargets.
D,0,k
4. Experimentalresults
In this section, two kinds of real-world experimental results are given. First, the
proposed tracker is evaluated for visual multi-object tracking datasets [9], [10], [11]
with unknown false measurement rate. Second, cell tracking in microscopy image
dataisillustrated. Trackingperformanceisenumeratedwithrespecttothewell-known
performancemetrics,in[5],[12].
4.1. Objectmotionandmeasurementmodelsandbasicparameters
Dynamicmotionmodel: Individualobjectkinematicsaredescribedbya4Dstate
vectorx =[p ,p˙ ,p ,p˙ ]Tofpositionandvelocity,whichfollowsaconstant
k x,k x,k y,k y,k
velocity model with T equal to the sampling period of the video data, and process
s
noiseismodeledasGaussianv ∼ N(v ;0,Q),whereN(z;m,P)denotesanormal
k k
distributionwithmeanmandcovarianceP;Q=σ2I ,I isthe2×2identitymatrix,
v 2 2
andσ = 3pixels/frameisthenoisestandarddeviation(setbyconsideringthemaxi-
v
mumspeedoftheobjectwithregardtotheframerate). Thesevaluesaredetermined
bythemaximumallowableobjectmotionwithrespecttotheimageframerate. Hence,
thetransitiondensity
f (x |x )=N(x ;Fx ,Q),
1,k|k−1 k k−1 k k−1
7
where
 
1 T
s
F =I2⊗ .
0 1
Forcluttertargets,thetransitiondensityf ,isgivenasarandomwalkmodelto
0,k|k−1
describearbitrarymotion[7].
Observationmodel:Assumingthattheobjectstatex =[p p˙ ,p p˙ ]T (x-
k x,k x,k y,k y,k
,y-positionsandvelocities)isobservedwithadditiveGaussiannoise,themeasurement
likelihoodfunctionisgivenbyg (z |x ) = N(z ;Hx ,Σ),z istheresponsefrom
k k k k k k
designateddetector;H =[1 0 1 0],i,e.,x-,y-positionisobservedbythedetector,Σ
isthecovariancematrixoftheobservationnoise.
4.2. Peopletrackinginvision
For the evaluation in real-world data, we are interested in tracking of multiple
pedestrians.Inthisexperiment,weapplytheACFdetector[13]forwell-knowndatasets
incomputervision,i.e.,”S2.L1”fromPETS’09dataset[11],”TUD-Stadtmitte”from
TUDdataset[9],and”Bahnhof”and”Sunnyday”fromETHdataset[10]. Thenumber
of targets varies in time due to births and deaths, and the measurement set includes
target-originated detections and clutter. In our experiments, we use the low thresh-
old for nonmaximum suppression so as to have less number of miss-detections but
increasedfalsealarmswithtime-varyingrate. Itismorerealisticsettingespeciallyin
ETHdatasetthatisrecordedwithfrequentcameraviewchanges. TheBoostedGLMB
iscomparedwiththeoriginalGLMB(withfixedclutterrate)[8], andstate-of-the-art
online Bayesian multi-object tracker called RMOT [14]. Quantitative experiment re-
sults are summarized in Table 1 using well-known performance indexes given in [5].
InTable1,BoostedGLMBshowssuperiorperformancecomparedtotheGLMBinall
indexesandcompatiblewiththerecentonlinetracker,RMOT.TheproposedBoosted
GLMBoutperformsothertrackerswithrespecttoFPFwheretrackerisabletoeffec-
tivelyrejectclutterswithestimatedclutterrate. Onthehand, inferiorperformancein
Fragmentation and ID switches are observed compared to RMOT due to the simple
motionmodel.
8
4.3. Cellmigrationanalysisinmicroscopyimage
Theproposedalgorithmisalsotestedwithlive-cellmicroscopyimagedataforcell
migrationanalysis. TheproposedGLMBtrackingmethodistestedonarealstemcell
migration sequence. The image sequence is recorded for 3 days, i.e., 4320 min and
eachimageistakeninevery16min.
Performance comparison is conducted with the state-of-the-art Multiple Hypoth-
esis Tracker (MHT) [15]. The same motion and measurement models are used as in
thefirstexperimentsandspotdetectionin[15]isappliedforthefaircomparison. As
showninFigure2,theproposedGLMBprovidesreliabletrackingresultscomparedto
theMHT.TheMHTistunedtoobtainthebesttrackingresults.TheGLMBtrackerpro-
ducessignificantlylessfalsetracksandalleviatefragmentedtracksbecausethetracker
efficientlymanagestime-varyingclutterinformationandkeepconfidenttracks. Quan-
titatively,ascanbeseeninTable2,timeaveragedOSPAdistances[12]forbothtrackers
verifythattheBoostedGLMBshowsreliableperformanceevenwhentheclutterrate
isunknown.
5. Conclusion
Inthispaper,weproposeanewmulti-objecttrackingalgorithmforunknownclutter
ratebasedonRFSfiltersthatisfreefromclutterparametertuning. Theone-stepRMB
filterforclutterrateestimationandGLMBfiltersfortrajectoryestimationarecoupled
byexchangingparametersforrobustonlinevisualmulti-objecttracking. Experimental
resultsviapeopleandcelltrackingdatasetswithstate-of-the-artonlinetrackersillus-
trate that the proposed multi-object tracker shows reliable performance. Interesting
futureresearchdirectionwouldbetheextensionofthetrackingalgorithmtoadaptive
survivalprobabilityandhandlingofmissed-detectionsforfurtherimprovement.
Acknowledgement
This work was supported by the Australian Research Council through a research
grantDP160104662.
9
References
References
[1] R.Mahler,StatisticalMultisource-MultitargetInformationFusion,ArtechHouse,
Inc.,Norwood,MA,USA,2007.
[2] R.Mahler,B.-T.Vo,B.-N.Vo,Cphdfilteringwithunknownclutterrateandde-
tectionprofile,IEEETrans.SignalProcessing59(8)(2011)3497–3513.
[3] M. D. Breitenstein, F. Reichlin, B. Leibe, E. Koller-Meier, L. V. Gool, On-
linemultipersontracking-by-detectionfromasingle,uncalibratedcamera,IEEE
Trans.PatternAnal.Mach.Intell.33(9)(2011)1820–1833.
[4] R. Hoseinnezhad, B.-N. Vo, B.-T. Vo, D. Suter, Visual tracking of numerous
targets via multi-bernoulli filtering of image data, Pattern Recognition 45 (10)
(2012)3625–3635.
[5] C. H. Kuo, R. Nevatia, How does person identity recognition help multi-person
tracking?, in: Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2011, pp. 1217–
1224.
[6] A.Milan,S.Roth,K.Schinlder,Continuousenergyminimizationformultitarget
tracking,IEEETrans.PatternAnal.Mach.Intell.36(1)(2014)58–72.
[7] B.-T.,B.-N.Vo,R.Hoseinnezhad,Robustmulti-bernoullifiltering,IEEEJour.of
Select.TopicsinSignalProcess.7(3)(2013)399–409.
[8] B.-T.Vo,B.-N.Vo,Labeledrandomfinitesetsandmulti-objectconjugatepriors,
IEEETrans.SignalProcessing61(13)(2013)3460–3475.
[9] M.Andriluka,S.Roth,B.Schiele,Monocular3dposeestimationandtrackingby
detection,in: Proc.IEEEConf.Comput.Vis.PatternRecog.,2010,pp.623–630.
[10] A. Ess, B. Leibe, K. Schindler, L. V. Gool, A mobile vision system for robust
multi-persontracking,in: Proc.IEEEConf.Comput.Vis.PatternRecog.,2008,
pp.1–8.
10