Anomaly Classification with the Anti-Profile Support Vector Machine

0.42 MB·English

by  Wikum Dinalankara

Anomaly Classification with the Anti-Profile Support
Vector Machine
WikumDinalankara
CenterforBioinformaticsandComputationalBiology
DepartmentofComputerScience
3
UniversityofMaryland
1
CollegePark,MD20742
0
[email protected]
2

n
a He´ctorCorradaBravo
J CenterforBioinformaticsandComputationalBiology

5 DepartmentofComputerScience
1 UniversityofMaryland
CollegePark,MD20742

] [email protected]
L
M
.
t Abstract
a
t
s
[ We introduce the anti-profile Support Vector Machine (apSVM) as a novel al-

gorithm to address the anomaly classification problem, an extension of anomaly
1
detectionwherethegoalistodistinguishdatasamplesfromanumberofanoma-
v
lousandheterogeneousclassesbasedontheirpatternofdeviationfromanormal
4
stableclass. Weshowthatunderheterogeneityassumptionsdefinedherethatthe
1
5 apSVMcanbesolvedasthedualofastandardSVMwithanindirectkernelthat
3 measuressimilarityofanomaloussamplesthroughsimilaritytothestablenormal
. class. We characterize this indirect kernel as the inner product in a Reproduc-
1
ingKernelHilbertSpacebetweenrepresentersthatareprojectedtothesubspace
0
spanned by the representers of the normal samples. We show by simulation and
3
1 applicationtocancergenomicsdatasetsthattheanti-profileSVMproducesclas-
: sifiers that are more accurate and stable than the standard SVM in the anomaly
v
classificationsetting.
i
X
r
a 1 Introduction
Thetaskofanomaly, oroutlier, detection[12,8,1]istoidentifydatasamplesthatdeviatesignifi-
cantlyfromaclassforwhichtrainingsamplesareavailable.Weexploreanomalyclassificationasan
extensiontothissetting,wherethegoalistodistinguishdatasamplesfromanumberofanomalous
and heterogeneous classes based on their pattern of deviation from a normal stable class. Specifi-
cally, presentedwithsamplesfromanormalclass, alongwithsamplesfrom2ormoreanomalous
classes, wewanttotrainaclassifiertodistinguishsamplesfromtheanomalousclasses. Sincethe
anomalousclassesareheterogeneoususingdeviationfromthenormalclassasthebasisofclassifi-
cationinsteadofbuildingaclassifierfortheanomalousclassesthatignoressamplesfromthenormal
classmayleadtoclassifiersandresultsthataremorestableandreproducible.
Themotivationforexploringthislearningsettingisfromrecentresultsincancergenomics[3]. In
particular,itwasshownthathyper-variabilityincertaingenomicmeasurements(DNAmethylation
andgeneexpression)inspecificregionsisastablecancermarkacrossmanytumortypes. Further-
more,thishyper-variabilityincreasesduringstagesofcancerprogression.Thisledustothequestion
1
Figure1:(A)PrincipalcomponentanalysisofDNAmethylationdata[3]. Variabilityinmethylation
measurementsincreasesfromnormaltobenignlesions(adenoma)tomalignantlesions(cancer).The
heterogeneityoftheadenomaandcanceranomaloussamplesisthedefiningfeatureoftheanomaly
classificationsetting. (B)and(C)illustrationoftheheterogeneityassumptioninDefinition1. For
bothalinearandradialbasisfunctionkernel, themagnitudeofeigenvaluesofthekernelmatrixis
largerfortheanomalousclasses.
ofhowtodistinguishsamplesfromdifferentstagesinthepresenceofhyper-variability. Inessence,
how to distinguish samples from different anomalous classes (given by cancer progression stage)
basedondeviationfromawell-definednormalclass(measurementsfromnon-canceroussamples).
We introduce the anti-profile Support Vector Machine (apSVM) as a novel algorithm suitable for
the anomaly classification task. It is based on the idea of only using the stable normal class to
define basis functions over which the classifier is defined. We show that the dual of the apSVM
optimizationproblemisthesameasthedualofthestandardSVMwithamodifiedkernelfunction.
We then show that this modified kernel function has general properties that ensure better stability
thanthestandardSVMintheanomalyclassificationtask.
The paper is organized as follows: we first present the anomaly classification setting in detail; we
nextdescribetheAnti-ProfileSupportVectorMachine(apSVM),andshowthatthedualoftheopti-
mizationproblemdefinedbyitisequivalenttothedualofthestandardSVMwithaspecifickernel
modification;wenextshowthatthiskernelmodificationleadsdirectlytoatheoreticalstatementof
thestabilityoftheapSVMcomparedtothestandardSVMintheanomalyclassificationsetting;we
nextshowsimulationresultsdescribingtheperformanceandstabilityoftheapSVM;andfinally,we
presentresultsfromcancergenomicsshowingthebenefitofapproachingclassificationproblemsin
thisareafromtheanomalyclassificationpointofview.
2 Theanomalyclassificationproblem
We present the anomaly classification problem in the binary case, with two anomalous classes.
Assume we are given training samples in Rp from three classes: m datapoints from normal class
Z, and n training datapoints as pairs (cid:104)x ,y (cid:105),...,(cid:104)x ,y (cid:105) with labels y ∈ {−1,1} indicating
1 1 n n i
membership of x in one of two anomalous classes A− and A+. Furthermore, we assume that
i
theanomalousclassesareheterogeneouswithrespecttonormalclassZ. Figure1aillustratesthis
learningsettingforDNAmethylationdata[3](see5fordetailsonthisaspectofcancerepigenetics).
Itisatwo-dimensionalembedding(usingPCA)ofDNAmethylationdatafornormalcolontissues
alongwithbenigngrowths(adenomas)andcancerousgrowths(tumor). Variabilityinthesespecific
measurementsincreasesfromnormaltoadenomatotumor. Wewouldliketobuildstableandrobust
classifiersthatdistinguishbenigngrowthsfromtumors.
2
Nextweseektoformalizetheheterogeneityassumptionoftheanomalyclassificationproblem. In-
tuitively, the heterogeneity assumption we make is that given random samples of the same size
from the normal class and from the anomalous classes, in expectation, the sample covariance of
the anomalous samples is always larger than the covariance of the normal samples. We state our
assumption in the case of Reproducing Kernel Hilbert Spaces (RKHS) since we will use this ma-
chinery throughout the paper [10, 14]. Recall that in a Bayesian interpretation of this setting, the
kernelfunctionassociatedwithaRKHSservesasthecovarianceofaGaussianpointprocess.
Definition 1 (Heterogeneity Assumption). Let H be a Reproducing Kernel Hilbert Space with
associatedkernelfunctionk. LetKm andKm bethekernelmatricesresultingfromevaluatingthe
Z A
kernelfunctionforasampleofsizemofpointsinthenormalandanomalousclassesrespectively.
Theheterogeneityassumptionisthatforeveryintegerm,thereexists(cid:15)∈R,where0<(cid:15)<1such
thatEdetKZm <(cid:15).
detKm
A
Figures 1b and c show that the heterogeneity assumption is satisfied in the DNA methylation data
forbothlinearandradialbasisfunctionkernels.Eachfigureshowsthemagnitudeoftheeigenvalues
of the resulting kernel matrices. The magnitude of the eigenvalues in both cases is larger for the
anomalousclasses.
Theheterogeneityassumptiongivesusahinttoconstructclassifiersthatdealwiththeheterogeneity
oftheanomalousclasses.InSection3.4weshowthatheterogeneityhasanimpactonrobustnessand
stabilityofclassifiersbuiltfromtrainingsetsoftheanomalousclasses. Ourgoalistousesamples
fromthestablenormalclasstocreateclassifiersthatarerobust. Wedescribetheanti-profileSVM
asanextensiontoSupportVectorMachinesthataccomplishesthisgoal.
3 Theanti-profileSVM
Support Vector Machines(SVMs) are one of the primary machine learning tools used for classi-
fication. SVMs operate by learning the maximum-margin separator between two groups of data
provided at training time. Any new observation provided to the SVM is classified by determining
whichsideoftheseparatorthenewobservationliesin. AnimportantadvantageofSVMsisthatby
applyingthekerneltrick,itispossibletofindahyperplaneinahigherdimensionalspacewherethe
twogivenclassesarelinearlyseparable,evenwhentheyarenotlinearlyseparableintheiroriginal
featurespaces,andbyvirtueofthekerneltrickthiscomputationcanbeperformedatnosignificant
cost. Whileprimarilydesignedforbinaryclassification,SVMshavebeenextendedformanyother
problems,suchasmulti-classclassificationandfunctionestimation.
3.1 TheSVMasweighthedvotingofbasisfunctions
HerewereviewSVMsfromafunctionapproximationperspective[14]: considerasetofnobser-
vations,eachobservationbeingdrawnfromX ×Y,whereX ∈ Rp,andY ∈ {−1,1}. Herepis
the number of features in each observation, or the dimensionality of the feature space. Thus each
observation consists of a pair (cid:104)x ,y (cid:105), x ∈ Rp and y ∈ {−1,1}, for i = 1..n; here y indicates
i i i i i
which of the two classes the observation belongs to. If we introduce a new observation x(cid:48) which
needstobeclassified,thentheclassificationproblemamountstocomparingx(cid:48)totheexistingsetof
pointsandcombiningthecomparisonstomakeadecision.
Tomakethecomparisonsbetweenobservations,wemakeuseofasimilarityfunction. Letk(x ,x )
i j
be a positive-definite similarity function which compares two points x ,x ∈ Rp. Weighing the
i j
similarityofthenewobservationtoeachexistingobservation,thedifferenceofthesumofweighted
similaritiesforthetwogroupswillprovidethenecessaryclassification:g(x)=d+(cid:80)n c k(x,x ).
i=1 i i
Herec ≥0 ∀iistheweightassociatedwitheachpoint,anddisabiasterm. Classificationisthen
i
basedonthesignoftheexpansion: f(x)=sgn[g(x)].
UsuallyinSVMsfunctionk isfurtherassumedtohavethereproducingpropertyinaReproducing
Kernel Hilbert Space H associated with k: (cid:104)f,k(x,·)(cid:105) = f(x) for all f ∈ H, and in particular
H
(cid:104)k(x,·),k(y,·)(cid:105) = k(x,y). Inthiscase,thebasisfunctionsintheclassifiercorrespondtorepre-
H
sentersk(x,·). InthestandardSVM,therepresentersofalltrainingpointsarepotentiallyusedas
basisfunctionsintheclassifier,buteffectivelyonlyasmallnumberofrepresentersareusedasbasis
functions, namely the Support Vectors. However, for a given problem, we may choose a different
3
setofpointsforthederivationofthesetofbasisfunctions; thebasisfunctionsdeterminehowthe
similaritiesaremeasuredforanewpoint.
3.2 TheAnti-ProfileSVMoptimizationproblem
Thecoreideaintheanti-rofileSVM(apSVM)istomakeuseofthischaracterizationoftheSupport
VectorMachineasalinearexpansionofbasisfunctionsdefinedbyrepresentersoftrainingsamples.
Inordertoaddresstheheterogeneityassumptionunderlyingtheanomalyclassificationproblemwe
definebasisfunctionsonlyusingsamplesfromthestablenormalclass.
Formally, we restrict the set of functions available to define the subspace of H spanned by the
representersofsamplesz ,...,z fromnormalclassZ: f(x)=d+(cid:80)m c k(z ,x). Toestimate
1 m i=1 i i
coefficientsc inthebasisexpansionweapplytheusualregularizedriskfunctionalbasedonhinge
i
loss
n
1 (cid:88) λ
R (f)= (1−y f(x )) + (cid:107)h(cid:107)2 ,
reg n i i + 2 H
j=1
where (·) = max(0,·), f(x) is defined as f(x) = d + h(x), and λ > 0 is a regularization
+
parameter. Bythereproducingkernelproperty,wehaveinthiscasethat(cid:107)h(cid:107)2 =c(cid:48)K cwhereK
H n n
isthekernelmatrixdefinedonthemnormalsamples.
Theminimizeroftheempiricalriskfunctionalisgivenbythesolutionofaquadraticoptimization
problem, similartothestandardSVM,butwithtwokernelmatricesused: K , definedinthepre-
n
vious paragraph, and K , which contains the evaluation of kernel function k between anomalous
s
samplesx ,...,x andnormalsamplesz ,...,z :
1 n 1 m
min eTξ+ nλcTK c (1)
2 n
d,c,ξ
s.t. Y(K c+de)+ξ ≥e,ξ ≥0
s
Here we use slack variables ξ = (ξ ,ξ ,...,ξ )(cid:48), denote the unit vector of size n as e, and define
1 2 n
matrixY asthediagonalmatrixsuchthatY =y .
ii i
3.3 SolvingtheapSVMoptimizationproblem
TheLagrangianofproblem1isgivenby
nλ
L(c,d,ξ,α,β)=eTξ+ cTK c−αT [Y(K c+de)+ξ−e]−βTξ
2 n s
whereα =(α ,...,α )T andβ =(β ,...,β )T aretheLagrangianmultipliers. Minimizing
n×1 1 n n×1 1 s
withrespecttoz,candd,wefindthattheWolfedualofproblem1is
max eTα− 1 αTYK˜Yα (2)
α 2nλ
s.t. 0≤α≤e,eTYα=0
whereK˜ = K K−1KT. HereweassumeK−1 representsapseudo-inverseinthecasewhereK
s n s n n
isnotpositivedefinite.
ForastandardSVM,theobjectiveoftheWolfedualiseTα− 1 αTYKYα,withKthekernelma-
2nλ
trixthetrainingdatapoints. ThusthedualproblemoftheapSVMhasthesameformasthestandard
SVMdualproblemwiththeexceptionthatkernelmatrixK isreplacedbyinducedkernelmatrixK˜
intheapSVM.KernelmatrixK˜ essentiallyrepresentsanindirectkernelbetweenanomaloussam-
plesinducedbythesetofbasisfunctionsdeterminedbythesamplesfromthenormalclass. Since
the essential form of the SVM solution is unchanged by the modification, this provides the addi-
tionaladvantagethatthemodifiedSVMcanbesolvedbythesametoolsthatsolvearegularSVM,
4
butwithadifferentkernelmatrixprovided. Forourparticularproblemdomain,weusetheindirect
kerneltorepresentdeviationfromtheprofileofnormalsamples,andthusrefertothisclassifieras
theanti-profileSVM.
3.4 Characterizingtheindirectkernel
We saw above that the apSVM can be solved as a standard SVM with induced kernel K˜ =
K K−1KT. Inthissectionwecharacterizethisindirectkernel,andstateageneralresultthateluci-
s n s
dateshowtheapSVMcanproduceclassifiersthataremorerobustandreproduciblethatastandard
SVMinthissetting.
Proposition1. LetP bethelinearoperatorthatprojectsrepresentersk(x,.) ∈ H tothespace
Z
spannedbytherepresentersofthemnormalsamplesoftheanomalyclassificationproblem.Induced
kernelk˜satisfiesk˜(x,y)=k(P k(x,.),P k(y,.)).
Z Z
Proof. ProjectionP k(x,.)isdefinedasP k(x,.)=(cid:80)m βˆk(z ,.)where
Z Z i=1 i i
βˆ = argmin1(cid:107)k(x,.)−(cid:88)βk(z ,.)(cid:107)2
β 2 i H
i
1 (cid:88) (cid:88)
= argmβin2(cid:104)k(x,.)− βk(z,.),k(x,.)− βk(zi,.)(cid:105)H
i i
 
1 (cid:88) (cid:88)
= argmβin2 (cid:104)k(zi,.),k(zj,.)(cid:105)H − (cid:104)k(x,.),k(zi,.)(cid:105)H
i,j i
= argmin1(cid:0)βTK β−kT β(cid:1), (3)
β 2 n zx
wherek isthevectorwithelementiequaltok(z ,x). From(3)wegetβˆ= K−1k . Therefore
zx i n zx
(cid:104)PZk(x,.),PZk(y,.)(cid:105)H =kzTxKn−1kzy =k˜(x,y).
This proposition states that the indirect kernel is the inner product in Reproducing Kernel Hilbert
Space H between the representers of anomalous samples projected to the space spanned by the
representersofnormalsamples. BytheheterogeneityassumptionofDefinition1,thespacespanned
byanysubsetofanomaloussampleswillbesmalleraftertheprojection. Inparticular,thesmallest
sphere enclosing the projected representers will be smaller, and from results such as the Vapnik-
Chapellesupportvectorspanrule[13],classifiersbuiltfromthisprojectionwillbemorerobustand
stable.
4 SimulationStudy
WefirstpresentsimulationresultsthatshowthattheapSVMobtainsbetteraccuracyintheanomaly
classificationsettingwhileprovidingstableandrobustclassifiers. Wegeneratedsamplesfromthree
normaldistributionsasfollows:ifA+andA−aretheanomalousclassesthatweneedtodistinguish,
and Z is the normal class, then for a given feature we draw datapoints from distributions Z =
N(0,σ2 ),A− = N(0,σ2 ) and A+ = N(0,σ2 ). To simulate our problem setting, we set
N A− A+
σ2 <σ2 <σ2 .
Z A− A+
ResultshavebeenobtainedfromtestswrittenonR(version2.14)withRpackageskernlab(version
0.9-14)[7]andsvmpath(version0.952). Thesvmpathtoolprovidesafittingfortheentirepathof
theSVMsolutiontoamodelatlittleadditionalcomputationalcost[4]. Usingtheresultingfit,the
SVMclassificationsforanygivencostparametercanbeobtained. Forourexperiments,thetesting
setaccuracywascomputedforeachvalueofcostalongtheregularizationpath,andthebestaccuracy
possiblewasobtained;tieswerebrokenbyconsideringtheoptionwiththeleastnumberofsupport
vectors used. Note that a small ridge parameter (1e-10) was used in the svmpath method to avoid
singularitiesincomputations.
5
Figure 2: (A) Accuracy results in simulated anomaly classification data. The anti-profile SVM
achieves better accuracy than the standard SVM. (B) Stability results in simulated data. The anti-
profileSVMusesasmallerproportionoftrainingpointsassupportvectors. SVMsthatusefewer
supportvectorsaremorerobustandstable.
Each training set contained 20 samples from each of A− and A+ classes, while each testing set
contained5samplesfromeachclass;20samplesfromclassZ wereusedfortheanti-profileSVM.
For a given number of features, each test was run 10 times and the mean accuracy computed. To
estimatethehyperparameterfortheradialbasiskernel,theinverseofthemeandistancebetween5
normaland5anomaloussamples(chosenrandomly)wasused.
Figure2ashowstheaccuracyofastandardSVMandtheapSVMusinganRBFkernelforsimulated
datawithσ =1,σ =2,σ =4. Witharadialbasiskernel,theanti-profileSVMwasableto
Z A− A+
achievebetterclassificationthantheregularSVM.
Wecharacterizethestabilityofaclassifierusingtheproportionoftrainingsamplesthatareselected
assupportvectors.Classifiersthatuseasmallproportionofpointsassupportvectorsaremorerobust
andstabletostochasticityinthesamplingprocess. ThemoresupportvectorsusedbyanSVM,the
morelikelyitisthattheclassificationboundarywillchangewithanychangesinthetrainingdata.
Henceaboundarythatisdefinedbyonlyafewsupportvectorswillresultinamorerobust,reliable
SVM.Figure2bshowsthatinthesimulationstudytheapSVMusedfewersupportvectorsthanthe
standardSVMwhileobtainingbetteraccuracy.
5 Applicationtocancergenomics
Themotivationforthisworkisfromrecentstudiesofepigeneticmechanismsofcancer. Epigenet-
icsisthestudyofmechanismsbywhichtheexpressionlevelofagene(i.e. thedegreetowhicha
genecanexertit’sinfluence)canbemodifiedwithoutanychangeintheunderlyingDNAsequence.
RecentresultsshowthatcertainchangesinDNAmethylationarecloselyassociatedwiththeoccur-
renceofmultiplecancertypes[3]. Inparticular, theexistenceofhighly-variableDNA-methylated
regionsincancerascomparedtonormals(i.e. healthytissue)hasbeenshown. Furthermore, these
highly-variable regions are associated with tissue differentiation, and are present across multiple
cancertypes.Anotherimportantobservationmadethereisthatadenomas,whicharebenigntumors,
showintermediatelevelsofhyper-variabilityinthesameDNA-methylatedregionsascomparedto
cancerandnormals.
Thispresentsaninterestingmachinelearningproblem:distinguishingbetweencancerandadenoma
based on the hyper-variability of their methylation levels with respect to normal samples? A suc-
cessful tool that can classify between the two groups can have far-reaching benefits in the area of
personalizedmedicineanddiagnostics.Sincethetwoclassesareessentiallydifferentiatedbythede-
greeofvariabilitytheyexhibitwithrespecttonormals,forourpurposewecanabstracttheproblem
tothesettingwepresenthereasanomalyclassification.
6
Figure3: ClassificationresultsforDNAmethylationincancer[3]. WhilebothastandardSVMand
theanti-profileSVMachievesimilaraccuracyusinganRBFkernel(A),theanti-profileSVMuses
muchfewersupportvectors.
5.1 Methylationdataresults
WestudytheperformanceoftheapSVMinadatasetofDNAmethylationmeasurementsobtained
forcolontissuefrom25healthysamples,19adenomasamples,and16cancersamples,for384spe-
cificpositionsinthehumangenome[3].Asmentionedpreviously,thecancersamplesexhibithigher
variancethanhealthysamples, withadenomasamplesshowinganintermediatelevelofvariability
(Figure 1). We used the same classification methods mentioned in the previous section, but with
multipleruns,foreachrunrandomlychoosing80%oftumorsamplesfortrainingandtheremaining
fortesting. Figure3showstheresultsobtainedusingaradialbasiskernel. Whiletheindirectkernel
performseitheratthesamelevelormarginallybetterthantheregularkernel,thenanti-profileSVM
usesmuchlesssupportvectorsthanthestandardSVM,thusprovidingamuchmorerobustclassifier.
5.2 Expressiondataresults
We further applied our method to gene expression data obtained with a clinical experiment on
adrenocortical cancer[2]. The datacontains expressionlevels for54675 probesets, for 10 healthy
samples,22adenomasamples,and32cancersamples. Thedatashowsthesamepatternwithregard
tohyper-variabilityasthemethylationdata. Usingthesamemethodsasbefore,theresultsobtained
usingalinearkernelareshowninFigure4.Forfeatureselection,thefeatureswererankedaccording
tologvar(Carcinoma) andforagivennumbernasthenumberoffeaturestobeused,nfeatureswith
var(Adenoma)
the highest variance ratio were selected. While both the standard SVM and the apSVM provided
almostperfectclassification,thereisasignificantdifferenceinthenumberofsupportvectorsused,
with the indirect kernel requiring much fewer support vectors and hence providing a more stable
classifier.
6 Discussion
We have introduced the anti-profile Support Vector Machine as a novel algorithm to address the
anomaly classification problem. We have shown that under the assumption that the classes we
are trying to distinguish with a classifier are heterogeneous with respect to a third stable class,
we can define a Support Vector Machine based on an indirect kernel using the stable class. We
haveshownthatthedualoftheapSVMoptimizationproblemisequivalenttothatofthestandard
SVMwiththeadditionofanindirectkernelthatmeasuressimilarityofanomaloussamplesthrough
similaritytothestablenormalclass. Furthermore,wehavecharacterizedthisindirectkernelasthe
innerproductinaReproducingKernelHilbertSpacebetweenrepresentersthatareprojectedtothe
subspacespannedbytherepresentersofthenormalsamples. ThisledtotheresultthattheapSVM
will learn classifiers that are more robust and stable than a standard SVM in this learning setting.
Wehaveshownbysimulationandapplicationtocancergenomicsdatasetsthattheanti-profileSVM
does in fact produce classifiers that are more accurate and stable than the standard SVM in this
setting.
7
Figure4: Classificationresultsforgeneexpressionincancer[2]. SimilartoFigure3,theaccuracy
of both the standard and anti-profile SVM is similar (in this case almost perfect testset accuracy
is achieved by both classifiers). However, the anti-profile SVM again uses fewer support vectors,
leadingtoclassifiersthataremorerobustandstable.
Whilethemotivationandexamplesprovidedherearebasedoncancergenomicsweexpectthatthe
anomalyclassificationsettingisapplicabletootherareas. Inparticular,wehavestartedlookingat
theareaofstatisticaldebuggingasasuitableapplication[16].
Thecharacterizationoftheindirectkernelthroughprojectiontothenormalsubspacealsosuggests
other possible classifiers suitable to this task. For instance, by defining a margin based on the
projectiondistancedirectly.Furthermore,connectionstokernelmethodsforquantileestimation[11]
willbeinterestingtoexplore.
Anotherdirectionofinterestingresearchwouldbetofurthersolidifythestabilitycharacterizationwe
provideinSection3.4. Forinstance,byexploringtherelationshiptootherleave-one-outbounds[9,
6,15,5],andthespanruleforkernelquantileestimation[10].
References
[1] VarunChandola, ArindamBanerjee, andVipinKumar. Anomalydetection: Asurvey. ACM
Comput. Surv., 41(3):15:1–15:58, July 2009. ISSN 0360-0300. doi: 10.1145/1541880.
1541882. URLhttp://doi.acm.org/10.1145/1541880.1541882.
[2] ThomasJGiordano,RorkKuick,TobiasElse,PaulGGauger,MichelleVinco,JulianeBauers-
feld,DonitaSanders,DafyddGThomas,GerardDoherty,andGaryHammer. Molecularclas-
sification and prognostication of adrenocortical tumors by transcriptome profiling. Clinical
cancerresearch: anofficialjournaloftheAmericanAssociationforCancerResearch,15(2):
668–676,January2009.
[3] KasperDanielHansen, WinstonTimp, He´ctorCorradaBravo, SarvenSabunciyan, Benjamin
Langmead,OliverGMcDonald,BoWen,HaoWu,YunLiu,DinhDiep,EirikurBriem,Kun
Zhang, Rafael A Irizarry, and Andrew P Feinberg. Increased methylation variation in epige-
neticdomainsacrosscancertypes. NatureGenetics,43(8):768–775,August2011.
[4] TrevorHastie,SaharonRosset,RobertTibshirani,andJiZhu. TheEntireRegularizationPath
for the Support Vector Machine. The Journal of Machine Learning Research, 5:1391–1415,
December2004.
[5] T.JaakkolaandD.Haussler. Probabilistickernelregressionmodels. Proceedingsofthe1999
ConferenceonAIandStatistics,1999.
[6] T.Joachims. EstimatingthegeneralizationperformanceofaSVMefficiently. Proceedingsof
theInternationalConferenceonMachineLearning,2000.
[7] Alexandros Karatzoglou, Alex Smola, Kurt Hornik, and Achim Zeileis. kernlab – an S4
package for kernel methods in R. Journal of Statistical Software, 11(9):1–20, 2004. URL
http://www.jstatsoft.org/v11/i09/.
8
[8] LarryM.ManevitzandMalikYousef. One-classsvmsfordocumentclassification. J.Mach.
Learn. Res., 2:139–154, March 2002. ISSN 1532-4435. URL http://dl.acm.org/
citation.cfm?id=944790.944808.
[9] M. Opper and O. Winther. Gaussian Processes for Classification: Mean-Field Algorithms.
NeuralComputation,12:2655–2684,2000.
[10] B.ScholkopfandA.J.Smola. LearningwithKernels. MITPressCambridge,Mass,2002.
[11] B Scho¨lkopf, J C Platt, J Shawe-Taylor, A J Smola, and R C Williamson. Estimating the
supportofahigh-dimensionaldistribution. Neuralcomputation,13(7):1443–1471,July2001.
[12] IngoSteinwart,DonHush,andClintScovel.Aclassificationframeworkforanomalydetection.
J. Mach. Learn. Res., 6:211–232, December 2005. ISSN 1532-4435. URL http://dl.
acm.org/citation.cfm?id=1046920.1058109.
[13] V.VapnikandO.Chapelle.BoundsonErrorExpectationforSupportVectorMachines.Neural
Computation,12:2013–2036,2000.
[14] G. Wahba. Support vector machines, reproducing kernel Hilbert spaces, and randomized
GACV. Advancesinkernelmethods: supportvectorlearning,pages69–88,1999.
[15] G.Wahba,Y.Lin,Y.Lee,andH.Zhang.OntherelationbetweentheGACVandJoachims’ξα
methodfortuningsupportvectormachines,withextensionstothenonstandardcase.Technical
report, Technical Report 1039, Statistics Department University of Wisconsin, Madison WI,
2001,2001.
[16] AXZheng,MIJordan,andBLiblit. Statisticaldebuggingofsampledprograms. Advancesin
NeuralInformationProcessingSystems,16,2003.
9