C. Aggarwal. A Survey of Uncertain Data Clustering Algorithms, In Data Clustering

26 Pages·2013·0.34 MB·English

Chapter18
ASurveyofUncertainDataClusteringAlgorithms
CharuC.Aggarwal
IBMT.J.WatsonResearchCenter
YorktownHeights,NY10598
[email protected]
18.1 Introduction ....................................................... 455
18.2 MixtureModelClusteringofUncertainData ........................ 457
18.3 DensityBasedClusteringAlgorithms ............................... 458
18.3.1 FDBSCANAlgorithm ..................................... 458
18.3.2 FOPTICSAlgorithm ...................................... 459
18.4 PartitionalClusteringAlgorithms ................................... 460
18.4.1 TheUK-meansAlgorithm ................................. 460
18.4.2 TheCK-MeansAlgorithm ................................. 461
18.4.3 ClusteringUncertainDatawithVoronoiDiagrams .......... 462
18.4.4 ApproximationAlgorithmsforClusteringUncertainData ... 462
18.4.5 SpeedingupDistanceComputations ....................... 463
18.5 ClusteringUncertainDataStreams ................................. 464
18.5.1 TheUMicroAlgorithm .................................... 464
18.5.2 TheLuMicroAlgorithm ................................... 469
18.5.3 EnhancementstoStreamClustering ........................ 469
18.6 ClusteringUncertainDatainHighDimensionality .................. 470
18.6.1 SubspaceClusteringofUncertainData ..................... 471
18.6.2 UPStream:ProjectedClusteringofUncertainDataStreams . 472
18.7 ClusteringwiththePossibleWorldsModel ......................... 475
18.8 ClusteringUncertainGraphs ....................................... 475
18.9 ConclusionsandSummary ......................................... 476
Bibliography ......................................................... 477
18.1 Introduction
Many data sets which are collected often have uncertainty built into them. In many cases, the
underlying uncertainty can be easily measured and collected. When this is the case, it is possible
to use the uncertainty in order to improve the results of data mining algorithms. This is because
theuncertaintyprovidesaprobabilisticmeasureoftherelativeimportanceofdifferentattributesin
dataminingalgorithms.Theuseofsuchinformationcanenhancetheeffectivenessofdatamining
algorithms, because the uncertainty provides a guidance in the use of different attributes during
the mining process. Some examples of real applications in which uncertainty may be used are as
follows:
(cid:129) Impreciseinstrumentsandhardwarearesometimesusedinordertocollectthedata.Insuch
455
456 DataClustering: AlgorithmsandApplications
cases,thelevelofuncertaintycanbemeasuredbypriorexperimentation.Aclassicexample
ofsuchhardwarearesensors,inwhichthemeasurementsareoftenimprecise.
(cid:129) The data may be imputed by statistical methods, such as forecasting. In such cases, the un-
certaintymaybeinferredfromthemethodologyusedinordertoperformtheimputation.
(cid:129) Manyprivacy-preservingdataminingtechniquesuseprobabilisticperturbations[11]inorder
toreducethefidelityoftheunderlyingdata.Insuchcases,theuncertaintymaybeavailable
asanendresultoftheprivacy-preservationprocess.Recentwork[8]hasexplicitlyconnected
the problem of privacy-preservation with that of uncertain data mining, and has proposed a
methodwhichgeneratesdata,whichisfriendlytotheuseofuncertaindataminingmethods.
Theproblemofuncertaindatahasbeenstudiedinthetraditionaldatabaseliterature[14,44],though
theissuehasseenarevivalinrecentyears[4,8,15,19,21,22,42,49,51,52].Thedrivingforce
behind this revival has been the evolution of new hardware technologies such as sensors which
cannot collect the data in a completely accurate way. In many cases, it has become increasingly
possible to collect the uncertainty along with the underlying data values. Many data mining and
managementtechniquesneedtobecarefullyre-designedinordertoworkeffectivelywithuncertain
data. This is because the uncertainty in the data can change the results in a subtle way, so that
deterministicalgorithmsmayoftencreatemisleadingresults[4].Whiletherawvaluesofthedata
canalwaysbeusedinconjunctionwithdataminingalgorithms,theuncertaintyprovidesadditional
insightswhicharenototherwiseavailable.Asurveyofrecenttechniquesforuncertaindatamining
maybefoundin[2].
Theproblemofclusteringisawellknownandimportantoneinthedataminingandmanagement
communities.Theproblemhasbeenwidelyexploredinthecontextofdeterministicdata.Detailsof
avarietyofclusteringalgorithmsmaybefoundin[37,34].Theclusteringproblemhasbeenwidely
studiedinthetraditionaldatabase literature[27,38,56]because ofitsapplications toavarietyof
customersegmentationanddataminingproblems.
Uncertainty modeling isvery relevant inthe context ofa number of differentclustering appli-
cations.Anexampleisillustratedin[41]inwhichuncertaintywasincorporatedintotheclustering
processinthecontextofasalesmerchandisingapplication.Sincetheproblemofdataclusteringis
closelyrelatedtothatofclassification,themethodsforuncertaindataclusteringcanalsobeusedto
enablealgorithmsforothercloselyrelateddataminingproblemssuchasoutlierdetection[6]and
classification[4].Thisisbecauseclusteringservesasageneral-purposesummarizationtool,which
canbeusedinthecontextofawidevarietyofproblems.
The presence of uncertainty significantly affects the behavior of the underlying clusters. This
isbecausethepresenceofuncertaintyalongaparticularattributemayaffecttheexpecteddistance
betweenthedatapointandthatparticularattribute.Inmostrealapplications,thereisconsiderable
skew in the uncertainty behavior across different attributes. The incorporation of uncertainty into
theclusteringbehaviorcansignificantlyaffectthequalityoftheunderlyingresults.
The problem of uncertain data clustering is often confused with that of fuzzy clustering [50].
Inthecaseofuncertaindataclustering,theuncertaintybelongstotherepresentationofthesource
objects which are being clusters, and the actual clustering model may be either probabilistic or
deterministic.Inthecaseoffuzzyclustering[50],thesourceobjectsaretypicallydeterministic,and
the membership of objects to clusters is probabilistic. In other words, each object has a degree of
belongingnesstothedifferentclusters,whichis“fuzzy”orprobabilisticinnature.
Inthischapter,wewillprovideasurveyofclusteringalgorithmsforuncertaindata.Themain
classesofclusteringalgorithmsforuncertaindataareasfollows:
(cid:129) Mixture-Modeling Algorithms:Mixturemodeling techniques useprobabilisticmodelsfor
clusteringuncertaindata.Aclassicexampleofsuchanapproachisgivenin[33],whichuses
anEM-approach[23]fortheclusteringprocess.
ASurveyofUncertainDataClusteringAlgorithms 457
(cid:129) Density-based Methods: A density-based method for uncertain data was proposed in [42].
ThisisreferredtoastheFDBSCANalgorithm.ThisapproachmodifiestheDBSCANalgo-
rithmtothecaseofuncertaindata.AnalternativemethodmodifiestheOPTICSalgorithmto
thecaseofuncertaindata[43].ThisisreferredtoastheFOPTICSalgorithm.
(cid:129) Partitional Methods: The K-means algorithm has been modified for the case of uncertain
data[16,48,45,20,28].Typically,themainchallengeinthesemethodsisthattheuncertain
distance computations for the k-means algorithms are too slow. Therefore, the focus is on
improvingefficiencybyusingpruningmethods[48],speedingupdistancecomputations[45],
orbyusingfastapproximationalgorithms,whichprovideworst-casebounds[20,28].
(cid:129) Streaming Algorithms: The problem of clustering uncertain data has been extended to the
case of data streams [5]. For this purpose, we extend the micro-clustering approach [10] to
thecaseofdatastreams.
(cid:129) HighDimensionalAlgorithms:Highdimensionaldataposesaspecialchallengeintheun-
certaindata,becausethedataisdistributedinaverysparsewaytobeginwith.Theaddition
ofuncertaintyandnoisefurtheraddstothesparsity.Therefore,effectivemethodsneedtobe
designedforapproximatelydeterminingclustersinsuchapplications.
Inthischapter,wewillprovideadetaileddiscussionofeachoftheabovealgorithmsforuncer-
taindata.Thischapterisorganizedasfollows.Inthenextsection,wewilldiscussmixturemodel
clusteringofuncertaindata.Insection18.3,wewilldescribedensity-basedclusteringalgorithmsfor
uncertaindata.TheseincludeextensionsofpopulardeterministicalgorithmssuchastheDBSCAN
and OPTICS algorithms. In section 18.4, we will discuss partitional algorithms for clustering un-
certain data. Most of these methods are extensions of the k-means and k-median algorithms. This
includes methods such as the UK-means, CK-means, and a number of approximation algorithms
for clustering uncertain data. Section 18.5 discusses streaming algorithms for clustering uncertain
data.Section18.6discusseshighdimensionalalgorithmsforclusteringuncertaindata.Theuncer-
taindataclusteringproblemhasalsobeenexploredinthecontextofthepossibleworldsmodelin
section18.7.Section18.9containstheconclusionsandsummary.
18.2 MixtureModelClusteringofUncertainData
Mixture model clustering [23] is a popular method for clustering of deterministic data, and it
models the clusters in the underlying data in terms of a number of probabilistic parameters. For
example, the data can be modeled as a mixture of gaussian clusters, and then the parameters of
this mixture can be learned from the underlying data. The core idea [23] is to determine model
parameters,whichensureamaximumlikelihoodfitoftheobservedinstantiationsofthedatawith
the proposed model. A popular method in order to determine these model parameters is the EM
algorithm, which uses an expectation maximization approach to iteratively update the parameters
withtheobserveddatainstances.
Theworkin[33]generalizesthisapproachtothecaseofuncertaindata,whereeachdatavalue
maybedrawnfromaninterval.Themaindifferencebetweentheuncertainversionofthealgorithm
andthedeterministicversionisthateachinstantiationisnowanuncertainvalueoftherecord,rather
thanadeterministicvalue.Correspondingly,theEMalgorithmisalsochangedinordertoevaluate
the expressions in the E-step and M-step as an expectation over the uncertain range of the data
value. We note that the approach can be used fairly easily for any uncertain distribution, which is
representedintheformofaprobabilityhistogramofvalues.
458 DataClustering: AlgorithmsandApplications
80
70
60
DENSITY ESTIMATE876543000000 1DENSITY ESTIMATE.5112345000000
20 2 0.5
10 1 0
0 −0.5
0 −1
1.5 1 0.5 0 −0.5 −1 −1 1.5 1 0.5 0 −0.5 −1
FIGURE 18.1: Density Based Profile with FIGURE 18.2: Density Based Profile with
LowerDensityThreshold HigherDensityThreshold
Another algorithm known as the MMVar algorithm has been proposed in [29], in which the
centroid of a clusterC is defined as an uncertain objectCMM, that represents the mixture model
ofC. The cluster compactness criterion used by the MMVar algorithm is the minimization of the
varianceoftheclustercentroid.
18.3 DensityBasedClusteringAlgorithms
Density-based methods are very popular in the deterministic clustering literature, because of
theirabilitytodetermineclustersofarbitraryshapesintheunderlyingdata.Thecore-ideainthese
methodsistocreateadensityprofileofthedatasetwiththeuseofkerneldensityestimationmeth-
ods.Thisdensityprofileisthenusedinordertocharacterizetheunderlyingclusters.Inthissection,
wewilldiscusstwovariationsofsuchdensity-basedmethods,whicharetheFDBSCANandFOP-
TICSmethods.
18.3.1 FDBSCANAlgorithm
The presence of uncertainty changes the nature of the underlying clusters, since it affects the
distance function computations between different data points. A technique has been proposed in
[42]inordertofinddensitybasedclustersfromuncertaindata.Thekeyideainthisapproachisto
computeuncertaindistanceseffectivelybetweenobjectswhichareprobabilisticallyspecified.The
fuzzy distance is defined in terms of the distance distribution function. This distance distribution
functionencodestheprobabilitythatthedistancesbetweentwouncertainobjectsliewithinacertain
user-definedrange.Letd(X,Y)betherandomvariablerepresentingthedistancebetweenX andY.
Thedistancedistributionfunctionisformallydefinedasfollows:
Definition18.3.1 Let X and Y be two uncertain records, and let p(X,Y) represent the distance
densityfunctionbetweentheseobjects.Then,theprobabilitythatthedistancelieswithintherange
(a,b)isgivenbythefollowingrelationship:
(cid:2)
b
P(a≤d(X,Y)≤b)= p(X,Y)(z)dz (18.1)
a
ASurveyofUncertainDataClusteringAlgorithms 459
Basedonthistechniqueandthedistancedensityfunction,themethodin[42]definesareachabil-
ityprobabilitybetweentwodatapoints.Thisdefinestheprobabilitythatonedatapointisdirectly
reachablefromanotherwiththeuseofapath,suchthateachpointonithasdensitygreaterthana
particularthreshold.Wenotethatthisisadirectprobabilisticextensionofthedeterministicreach-
ability concept which is defined in the DBSCAN algorithm [24]. In the deterministic version of
thealgorithm[24],datapointsaregroupedintoclusterswhentheyarereachablefromoneanother
by a path which is such that every point on this path has a minimum threshold data density. To
this effect, the algorithm uses the condition that the ε-neighborhood of a data point should con-
tain at least MinPts data points. The algorithm starts off at a given data point and checks if the
ε neighborhood contains MinPts data points. If this is the case, the algorithm repeats the process
foreach point inthiscluster and keeps adding points untilno more points can be added. One can
plot the density profile of a data set by plotting the number of data points in the ε-neighborhood
of various regions, and plotting a smoothed version of the curve. This is similar to the concept of
probabilisticdensityestimation.Intuitively,thisapproachcorrespondstothecontinuouscontoursof
intersectionbetweenthedensitythresholdsofFigures18.1and18.2withthecorrespondingdensity
profiles.ThedensitythresholddependsuponthevalueofMinPts.Notethatthedatapointsinany
contiguousregionwillhavedensitygreaterthanthethreshold.Notethattheuseofahigherdensity
threshold (Figure 18.2) results in 3 clusters, whereas the use of a lower density threshold results
in2clusters.ThefuzzyversionoftheDBSCANalgorithm(referredtoasFDBSCAN)worksina
similarwayastheDBSCANalgorithm,exceptthatthedensityatagivenpointisuncertainbecause
oftheunderlinguncertaintyofthedatapoints.Thiscorrespondstothefactthatthenumberofdata
pointswithintheε-neighborhoodofagivendatapointcanbeestimatedonlyprobabilistically,and
isessentiallyanuncertainvariable.Correspondingly,thereachabilityfromonepointtoanotheris
nolongerdeterministic,sinceotherdatapointsmayliewithintheε-neighborhoodofagivenpoint
with a certain probability, which may be less than 1. Therefore, the additional constraint that the
computedreachabilityprobabilitymustbegreaterthan0.5isadded.Thus,thisisageneralization
ofthedeterministicversionofthealgorithminwhichthereachabilityprobabilityisalwayssetto1.
18.3.2 FOPTICSAlgorithm
Anotherrelatedtechniquediscussedin[43]isthatofhierarchicaldensitybasedclustering.An
effective (deterministic) density based hierarchical clustering algorithm is OPTICS [12]. We note
thatthecoreideainOPTICSisquitesimilartoDBSCANandisbasedontheconceptofreachability
distance between data points. While the method in DBSCAN defines a global density parameter
whichisusedasathresholdinordertodefinereachability,theworkin[43]pointsoutthatdifferent
regions in the data may have different data density, as a result of which it may not be possible to
definetheclusterseffectivelywithasingledensityparameter.Rather,manydifferentvaluesofthe
densityparameterdefinedifferent(hierarchical)insightsabouttheunderlyingclusters.Thegoalis
todefineanimplicitoutputintermsoforderingdatapoints,sothatwhentheDBSCANisapplied
with this ordering, once can obtain the hierarchical clustering at any level for different values of
the density parameter. The key is to ensure that the clusters at different levels of the hierarchy
areconsistentwithoneanother.Oneobservationisthatclustersdefinedoveralowervalueofεare
completelycontainedinclustersdefinedoverahighervalueofε,ifthevalueofMinPtsisnotvaried.
Therefore,thedatapointsareorderedbasedonthevalueofεrequiredinordertoobtainMinPtsin
theε-neighborhood.Ifthedatapointswithsmallervaluesofεareprocessedfirst,thenitisassured
thathigherdensityregionsarealwaysprocessedbeforelowerdensityregions.Thisensuresthatif
theDBSCANalgorithmisusedfordifferentvaluesofεwiththisordering,thenaconsistentresultis
obtained.Thus,theoutputoftheOPTICSalgorithmisnottheclustermembership,butitistheorder
inwhichthedatapointsareprocessed.WenotethatthatsincetheOPTICSalgorithmsharessomany
characteristicswiththeDBSCANalgorithm,itisfairlyeasytoextendtheOPTICSalgorithmtothe
uncertaincaseusingthesameapproachasthatwasusedforextendingtheDBSCANalgorithm.This
460 DataClustering: AlgorithmsandApplications
isreferredtoastheFOPTICSalgorithm.Notethatoneofthecore-conceptsneededtoordertodata
pointsistodeterminethevalueofεwhichisneededinordertoobtainMinPtsinthecorresponding
neighborhood. In the uncertain case, this value is defined probabilistically, and the corresponding
expectedvaluesareusedtoorderthedatapoints.Adifferenthierarachicalclusteringalgorithmwith
theuseofaninformation-theoreticapproachwasproposedin[31].
18.4 PartitionalClusteringAlgorithms
Partitionalclusteringmethodsarealgorithmswhichextendthek-meansandk-medoidprinciples
tothecaseofuncertaindata.Inthissection,wewilldiscussthesemethods.Theadvantageofusing
partitionalclusteringmethodsistheirrelativesimplicityandquickexecution.
18.4.1 TheUK-meansAlgorithm
Acommonapproachtoclusteringisthek-meansalgorithm.Inthek-meansalgorithm,wecon-
structclustersaroundapre-definednumberofclustercenters.Avarietyofdistancefunctionsmay
beusedinordertomapthepointstothedifferentclusters.Ak-meansapproachtoclusteringuncer-
taindatawasstudiedinthecontextofmovingobjectdata[16,48].Inthecaseofmovingobjects,
theactuallocationsoftheobjectsmaychangeovertimeasthedataisreportedintermittently.Thus,
thepositionofavehiclecouldbeanarbitraryorcircleregionwhichusesthereportedlocationasits
centerandhasasizewhichisdependentuponthespeedanddirectionofthevehicle.Aprobability
densityfunctioncouldbeusedtomodeltheprobabilityofpresenceofthevehicleatagivenlocation
ataparticulartime.
Onepossibilityistosimplyreplaceeachuncertaindatapointbyarepresentativepointsuchas
itscentroid,andapplythe(deterministic)k-meansclusteringmethoddirectlytoit.TheUK-means
clustering approach is very similar to the K-means clustering approach, except that we use the
expecteddistancefromthedata’suncertaintyregiontotherepresentativeofthecandidateclusterto
whichitisassigned.Itwasshownin[16],thattheuseofexpecteddistanceshasclearadvantages
over an approach which uses deterministic clustering algorithms over representative data points.
ThisapproachisreferredtoastheUK-meansalgorithm.
A key challenge is the computation of the expected distances between the data points and the
centroids for the k-means algorithm. A natural technique for computing these expected distances
istouseMonte-carlosampling,inwhichsamplesforthedatapointsareusedinordertocompute
theuncertaindistances.Anothertechniqueistocreatediscretebucketsfromboththedistributions
andcomputetheexpecteddistancesbyapairwiseweightedaveragefromdifferentpairsofbuckets.
Thus,ifonepdfisdiscretizedintom buckets,andanotherpdfisdiscretizedintom buckets,such
1 2
anapproachwouldrequirem ·m distancecomputations.TheMonte-Carloapproachcanbevery
1 2
expensive because a large number of samples may be required in order to compute the distances
accurately. Similarly, a large number of discrete buckets may be required in order to compute the
pairwisedistancesaccurately.Theworkin[16]usesapurelybrute-forceversionoftheUK-means
algorithm in which no optimization or pruning of the distance computations is performed. This
versioncanbeimpractical,especiallyifahighlevelofaccuracyisrequiredintheclusteringprocess.
Clearly,somekindofpruningisrequiredinordertoimprovetheefficiencyoftheapproach.
Theworkin[48]improvesontheworkof[16],anddesignsaprunedversionoftheUK-means
algorithm. The idea here is to use branch-and-bound techniques in order to minimize the number
ofexpecteddistancecomputationsbetweendatapointsandclusterrepresentatives.Thebroadidea
is that once an upper bound on the minimum distance of a particular data point to some cluster
ASurveyofUncertainDataClusteringAlgorithms 461
representative has been quantified, it is necessary to perform the computation between this point
andanotherclusterrepresentative,ifitcanbeprovedthatthecorrespondingdistanceisgreaterthan
thisbound.Inordertocomputethebounds,theminimumboundingrectanglefortherepresentative
pointforaclusterregioniscomputed.Theuncertaindatapointalsorepresentsaregionoverwhich
the object may be distributed. For each representative cluster, its minimum bounding rectangle is
usedtocomputethefollowingtwoquantitieswithrespecttotheuncertaindatapoint:
(cid:129) The minimum limit on the expected distance between the MBR of the representative point
andtheuncertainregionforthedatapointitself.
(cid:129) The maximum limit on the expected distance between the MBR of the representative point
andtheuncertainregionforthedatapointitself.
These upper and lower bound computations are facilitated by the use of the Minimum Bounding
Rectanglesinconjunctionwiththetriangleinequality.Wenotethataclusterrepresentativecanbe
pruned, if its maximum limit is less than the minimum limit for some other representative. The
approachis[48]constructsak-dtreeontheclusterrepresentativesinordertopromoteanorderly
pruning strategy and minimize the number of representatives which need to be accessed. It was
shown in [48] that such an approach significantly improves the pruning efficiency over the brute
forcealgorithm.
18.4.2 TheCK-MeansAlgorithm
While the work in [16] claims that UK-means provides qualitatively superior results to deter-
ministic clustering, the work in [45] shows that the model utilized by the UK-means is actually
equivalenttodeterministicclustering,byreplacingeachuncertaindatapointbyitsexpectedvalue.
Thus,theUK-meansapproachactuallyturnsouttobeequivalent todeterministicclustering.This
contradictstheclaimin[16]thattheUK-meansalgorithmprovidessuperiorresultstoadeterminis-
ticclusteringmethodwhichreplacesuncertaindatapointswiththeircentroids.Wefurthernotethat
mostofthecomputationalcomplexityiscreatedbytherunningtimerequiredforexpecteddistance
calculations.Ontheotherhand,deterministicdistancecomputationsareextremelyefficient,andare
almostalwayssuperiortoanymethodwhichisbasedonexpecteddistancecomputations,whether
ornotpruningisused.
TheUK-meansalgorithmaimstooptimizethemeansquareexpecteddistanceabouteachcluster
centroid.Akeystepisthecomputationoftheexpectedsquaredistanceofanuncertaindatapoint
X withaclustercentroidY,wherethelatterisapproximatedasadeterministicentity.LetY bethe
i
centroidofacluster,andX ...X bethesetofdatapointsinthecluster.Then,theexpectedmean
1 r
squaredistanceofdatapointX aboutY isgivenbyE[||X −Y||2].Then,ifY isapproximatedasa
i i
deterministicentity,thenwecanshowthefollowing:
Lemma18.4.1 LetX beanuncertaindatapoint,andY beadeterministicpoint.Letc =E[X]and
i i i
var(X)representthesumofthevariancesofthepdfsinX overalldimensions.Then,wehave:
i i
E[||X −Y||2]=E[||X||2]−||c||2+||c −Y||2
i i i i
=var(X)+||c −Y||2
i i
Wewillprovideaproofofageneralizedversionofthislemmaslightlylater(Lemma18.4.2).We
further note that the value of E[||X||2]−||c||2 is equal to the variance of the uncertain data point
i i
X (summed over all dimensions). The term ||c −Y||2 is equal to the deterministic distance of
i i
theY to the centroid of the uncertain data point X. Therefore, the expected square distance of an
i
uncertaindatapointX tothecentroidY isgivenbythesquaresumofitsdeterministicdistanceand
i
462 DataClustering: AlgorithmsandApplications
the variance of the data point X. The variance of the data point is not dependent on the value of
i
Y.Therefore,whilecomputingtheexpectedsquaredistancetothedifferentcentroidsfortheUK-
meansalgorithm,itsufficestocomputethedeterministicdistancetothecentroidc ofX insteadof
i i
computingtheexpectedsquaredistance.ThismeansthatbyreplacingeachuncertaindatapointX
i
withitscentroid,theUK-meanscanbereplicatedexactlywithanefficientdeterministicalgorithm.
It is important to note that the equivalence of the UK-means method to a deterministic algo-
rithm is based on the approximation of treating each cluster centroid (in intermediate steps) as a
deterministicentity.Inpractice,someofthedimensionsmaybemuchmoreuncertainthanothers
intheclusteringprocessovermostofthedatapoints.Thisisespeciallythecasewhendifferentdi-
mensionsarecollectedusingcollectiontechniqueswithdifferentfidelity.Insuchcases,thecluster
centroidsshouldnotbetreatedasdeterministicentities.Someofthestreamingmethodsforuncer-
taindataclusteringsuchasthosediscussedin[5]alsotreattheclustercentroidsasuncertainentities
inordertoenablemoreaccuratecomputations.Inthosecase,suchdeterministicapproximationsare
notpossible.Anothermethod,whichtreatsclustercentroidsasuncertainentitieswaslaterproposed
independentlyin[30].Theworkonclusteringstreams,whiletreatingcentroidsasuncertainentities
willbediscussedinalatersectionofthischapter.
18.4.3 ClusteringUncertainDatawithVoronoiDiagrams
The work in [48] uses minimum bounding boxes of the uncertain objects in order to compute
distance bounds for effective pruning. However, the use of minimax pruning can sometimes be
quiterestrictiveinefficientlycharacterizingtheuncertainobject,whichmayhavearbitraryshape.
An approach which is based on voronoi diagrams, also improves the UK-means algorithms by
computingthevoronoidiagramsofthecurrentsetofclusterrepresentatives[39].Eachcellinthis
voronoi diagram is associated with a cluster representative. We note that each cell in this voronoi
diagramhasthepropertythatanypointinthiscellisclosertotheclusterrepresentativeforthatcell,
thananyotherrepresentative.Therefore,iftheMBRofanuncertainobjectliescompletelyinsidea
cell,thenitisnotnecessarytocomputeitsdistancetoanyotherclusterrepresentatives.Similarly,
for any pair of cluster representatives, the perpendicular bisector between the two is a hyperplane
whichisequidistantfromthetworepresentativesandiseasilyderivablefromthevoronoidiagram.
IntheeventthattheMBRofanuncertainobjectliescompletelyononesideofthebisector,wecan
deduce thatoneoftheclusterrepresentativesofclosertotheuncertainobjectthantheother.This
allowsustopruneoftherepresentatives.
Asin[48],thisworkisfocusedonpruningthenumberofexpecteddistancecomputations.Ithas
beenshownin[39]thatthepruningpowerofthevoronoimethodisgreaterthantheminmaxmethod
proposedin[48].However,theworkin[39]doesnotcompareitsefficiencyresultstothosein[45],
whichisbasedontheequivalenceofUK-meanstoadeterministicalgorithm,anddoesnotrequire
any expected distance computations at all. It would seem to us, that any deterministic method for
k-means clustering (as proposed in the reduction of [45]) should be much more efficient than a
method based on pruning the number of expected distance computations, no matter how effective
thepruningmethodologymightbe.
18.4.4 ApproximationAlgorithmsforClusteringUncertainData
Recently,techniqueshavebeendesignedforapproximationalgorithmsforuncertainclustering
in [20]. The work in [20] discusses extensions of the k-mean and k-median version of the prob-
lems.Bi-criteriaalgorithmsaredesignedforeachofthesecases.Onealgorithmachievesa(1+ε)-
approximationtothebestuncertaink-centerswiththeuseofO(k·ε−1·log2(n))centers.Thesecond
algorithmpicks2kcentersandachievesaconstant-factorapproximation.
A key approach proposed in the paper [20] is the use of a transformation from the uncertain
casetoaweightedversionofthedeterministiccase.Wenotethatsolutionstotheweightedversion
ASurveyofUncertainDataClusteringAlgorithms 463
ofthedeterministicclusteringproblemarewellknown,andrequireonlyapolynomialblow-upin
the problem size. The key assumption in solving the weighted deterministic case is that the ratio
of the largest to smallest weights is polynomial. This assumption is assumed to be maintained in
the transformation. This approach can be used in order to solve both the uncertain k-means and
k-medianversionoftheproblemwiththeafore-mentionedapproximationguarantees.Wereferthe
readerto[20,28]fordetailsofthesealgorithms.
18.4.5 SpeedingupDistanceComputations
We note that there are two main ways in which the complexity of distance computations in
a k-means algorithm can be reduced. The first is by using a variety of pruning tricks, which cuts
downonthenumberofdistancecomputationsbetweendatapointsandclusterrepresentatives.The
secondisbyspeedinguptheexpecteddistancecomputationitself.Thiskindofapproachcanespe-
ciallybeusefulwherethepruningeffectivenessofatechniquesuchasthatproposedin[48]isnot
guaranteed.Therefore,anaturalquestionarisesaswhetheronecanspeeduptheuncertaindistance
computations,whichcausetheperformancebottleneckinthesemethods.
Theworkin[54]designsmethodsforspeedingupdistancecomputationsfortheclusteringpro-
cess.WenotethatsuchfastdistancecomputationscanbeveryeffectivenotonlyfortheUK-means
algorithm,butforanyclusteringtechniquewhichisdependentonexpecteddistancecomputations.
Theworkin[54]proposesanumberofmethodsforperformingdistancecomputationsbetweenun-
certainobjects,whichprovidedifferenttradeoffsbetweeneffectivenessandefficiency.Specifically,
forapairofuncertainobjectsX andY,thefollowingmethodscanbeusedinordertocomputethe
distancesbetweenthem:
(cid:129) Certain Representation: Each uncertain object can be replaced by a certain object, corre-
spondingtotheexpectedvaluesofitsattributes.Thedistancesbetweentheseobjectscanbe
computedinastraightforwardway.Whilethisapproachisveryefficient,itprovidesverypoor
accuracy.
(cid:129) Sampling:Itispossibletorepeatedlysampleboththeobjectsforpairsofinstantiationsand
compute the distances between them. The average of these computed distances can be re-
ported as the expected value. However, such an approach may require a large number of
samplesinordertoprovideahighqualityapproximation.
(cid:129) ProbabilityHistograms:Eachuncertainobjectcanbeapproximatedbyasetofbins,which
correspondstoitsprobabilityhistogram.Then,foreverypairofbinsbetweenthetwoobjects,
the probability of that instantiation and the distance between the average values of those
binsiscomputed.Theweightedaverageoverallpairsofbinsisreported.Suchanapproach
can still be quite inefficient in many scenarios, where a large number of bins is required to
representtheprobabilityhistogrameffectively.
(cid:129) Gaussian Mixture Modeling with Sample Clustering: Each uncertain object can be ap-
proximated with a mixture of gaussians. Specifically, we sample each uncertain object with
theuseofitspdf,andthenclusterthesesampleswithdeterministick-meansclustering.Each
of these clusters can be fit into a gaussian model. Then, the pairwise weighted average dis-
tancesbetweeneachofthecomponentsofthemixturecanbecomputed.
(cid:129) SingleGaussianModeling:Itturnsoutthatitisnotnecessarytousemultiplecomponentsin
themixturemodelfortheapproximationprocess.Infact,itsufficestouseasinglecomponent
forthemixture.
The last result is actually not very surprising in light of Lemma 18.4.1. In fact, the Gaussian as-
sumption is not required at all, and it can be shown that the distance between a pair of uncertain
464 DataClustering: AlgorithmsandApplications
objects(forwhichthepdfsareindependentofoneanother),canbeexpressedpurelyasafunction
oftheirmeansandvariances.Therefore,weproposethefollowing(slight)generalizationofLemma
18.4.1.
Lemma18.4.2 Let X andY be two uncertain data points, with means c and d respectively. Let
i i i i
thesumofthevariancesacrossalldimensionsofthesepointsbevar(X)andvar(Y)respectively.
i i
Then,wehave:
E[||X −Y||2]=||c −d||2+var(X)+var(Y) (18.2)
i i i i i i
Proof:Wecanexpandthetermwithintheexpectationonthelefthandsideasfollows:
E[||X −Y||2]=E[||(X −c)+(c −d)+(d −Y)||2] (18.3)
i i i i i i i i
Wefurthernotethateachofthethreeexpressionswithintheroundbracketsontherighthandside
arestatisticallyindependentofoneanother.Thismeansthattheircovariancesarezero.Furthermore,
theexpectedvaluesof(X −c)and(d −Y)areboth0.Thiscanbeusedtoshowthattheexpectation
i i i i
oftheproductofanypairoftermswithintheroundbracketsontherighthandsideofEquation18.3
is0.Thisimpliesthatwecanre-writetherighthandsideasfollows:
E[||X −Y||2]=E[||X −c||2]+(c −d)2+E[||d −Y||2] (18.4)
i i i i i i i i
ThefirsttermintheRHSoftheaboveexpressionisvar(X)andthelasttermisvar(Y).Theresult
i i
follows.
The afore-mentioned results suggest that it is possible to compute the distances between pairs
of uncertain objects very efficiently, as long as the uncertainty in different objects are statistically
independent. Another observation is that these computations do not require knowledge of the full
probability density function of the probabilistic records, but can be made to work with the more
modest assumption about the standard error var(·) of the underlying uncertainty. This is a more
reasonable assumption for many applications. Such standard errors are included as a natural part
ofthemeasurementprocess,thoughthefullprobabilitydensityfunctionsarerarelyavailable.This
alsosuggeststhatalotofworkonpruningthenumberofexpecteddistancecomputationsmaynot
bequiteascriticaltoefficientclustering,ashasbeensuggestedintheliterature.
18.5 ClusteringUncertainDataStreams
Inmanyapplicationssuchassensordata,thedatamayhaveuncertainty,becauseoferrorsinthe
readingsoftheunderlyingsensors.Thismayresultinuncertainstreamsofdata.Uncertainstreams
poseofspecialchallengebecauseofthedualcomplexityofhighvolumeanddatauncertainty.As
we have seen in earlier sections, efficiency is a primary concern in the computation of expected
distances,whenworkingwithprobabilitydensityfunctionsofdatapoints.Therefore,itisdesirable
toworkwithsimplerdescriptionsoftheunderlyinguncertainty.Thiswillreduceboththeunderlying
data volume and complexity of stream computations. In recent years, a number of methods have
specificallybeenproposedforclusteringuncertaindatastreams.
18.5.1 TheUMicroAlgorithm
In this section, we will introduceUMicro, the Uncertain MICROclustering algorithm for data
streams.Weassumethatwehaveadatastreamwhichcontainsddimensions.Theactualrecordsin