Table Of ContentEVALBUIAOTIINNFGOSROMURACTEICSSOF VARIABILITY IN PATHWAY Vol.00no.002011
Doc-Start Pages1–9
PROFILING
Annalisa Barla(1)∗, Samantha Riccadonna(2)∗, Salvatore Masecchia(1),
Margherita Squillario(1), Michele Filosi(2,3), Giuseppe Jurman(2) and Cesare
Furlanello(2)†
2
1 1DISI,UniversityofGenoa,viaDodecaneso35,I-16146Genova,Italy.
0 2FBK,viaSommarive18,I-38123Povo(Trento),Italy.
2 3CIBIO,UniversityofTrento,viaDelleRegole101,I-38123Mattarello(Trento),Italy
n
ReceivedonXXXXX;revisedonXXXXX;acceptedonXXXXX
a
J
6
AssociateEditor:XXXXXXX
1
]
N
ABSTRACT featureranking/selection algorithm, theenrichment procedure, the
M
Motivation: A bioinformatics platform is introduced aimed at inference method and the networks comparison function. Each of
. identifying models of disease-specific pathways, as well as a set these components is a potential source of variability, as shown in
o
of network measures that can quantify changes in terms of global thecaseofbiomarkersfrommicroarrays(TheMAQCConsortium,
i
b structureorsinglelinkdisruptions.Theapproachintegratesanetwork 2010). Considerable efforts have been directed to tackle the
- comparison framework with machine learning molecular profiling. problem of poor reproducibility of biomarker signatures derived
q
TheplatformincludesdifferenttoolscombinedinoneOpenSource fromhigh-throughput-omicsdata(TheMAQCConsortium,2010),
[
pipeline, supporting reproducibility of the analysis. We describe addressing the issues of selection bias (AmbroiseandMcLachlan,
1 here the computational pipeline and explore the main sources of 2002;Furlanelloetal.,2003)andmorerecentlyofpervasivebatch
v variabilitythatcanaffecttheresults,namelytheclassifier,thefeature effects (Leeketal., 2010). We argue that it is now urgent to
6
ranking/selectionalgorithm,theenrichmentprocedure,theinference adopt a similar approach for network medicine studies. Stability
1
methodandthenetworkscomparisonfunction. (and thus reproducibility) in this class of studies is still an
2
Results: The proposed pipeline is tested on a microarray dataset open problem (Barallaetal., 2009). Underdeterminacy is a major
3
of late stage Parkinsons’ Disease patients together with healty issue (DeSmetandMarchal, 2010), as the ratio between network
.
1 controls. Choosing different machine learning approaches we get dimension (number of nodes) and the number of available
0 low pathway profiling overlapping in terms of common enriched measurementstoinferinteractionsplaysakeyroleforthestability
2
elements.Nevertheless,theyidentifydifferentbutequallymeaningful of the reconstructed structure. Furthermore, the most interesting
1
biologicalaspectsofthesameprocess,suggestingtheintegrationof applications are based on inferring networks topology and wiring
:
v informationacrossdifferentmethodsasthebestoverallstrategy. fromhigh-throughputnoisymeasurements(Heetal.,2009).
i Availability: Alltheelementsoftheproposedpipelineareavailable Despiteitscommonuseeveninbiologicalcontexts(SharanandIdeker,
X
asOpenSourceSoftware:availabilitydetailsareprovidedinthemain 2006), the problem of quantitatively comparing networks (e.g.,
r
a text. using a metric instead of evaluating network properties) is a still
Contact:[email protected] an open issue in many scientific disciplines. The central problem
is of course which network metrics should be used to evaluate
stability, whether focusing on local changes or global structural
changes.AsdiscussedinJurmanetal.(2011),theclassicdistances
1 INTRODUCTION
in the edit family focus only on the portions of the network
We present a computational framework for the study of
interested by the differences in the presence/absence of matching
reproducibilityinnetworkmedicinestudies(Barabasietal.,2011).
links. Spectral distances - based on the list of eigenvalues of the
Networks, molecular pathways in particular, are increasingly
Laplacianmatrixoftheunderlyinggraph-areinsteadparticularly
looked at as a better organized and more rich version of gene
effective for studying global structures. In particular, the Ipsen-
signatures.However,highvariabilitycanbeinjectedbythedifferent
Mikhailov (IpsenandMikhailov, 2002) distance was found robust
methods that are typically used in system biology to define a
inawiderangeofsituations(Jurmanetal.,2011).However,global
cellular wiring diagram at diverse levels of organization, from
distancescanbetrickedbyisomorphorclosetoisomorphgraphs.
transcriptomicstosignalling,ofthefunctionaldesign.Forexample,
InJurmanetal.(2012),bothapproachesareimprovedbyproposing
to identify the link between changes in graph structures and
aglocalmeasurewhichcombinesaspectraldistancewithatypical
disease, we choose and combine in a workflow a classifier, the
Hamming local editing component. In thispaper we use thisnew
tooltoquantifyhowstabilityofnetworkreconstructionismodified
∗Theseauthorsequallycontributedtothework inpracticebythedifferentinferenceandenrichmentmethods.
†towhomcorrespondenceshouldbeaddressed
(cid:13)c OxfordUniversityPress2011. 1
Barlaetal
Pathway enrichment methods arewidely used inbioinformatics properDataAnalysisProtocol,whichwillensureaccurateandreproducible
analysis,forexampletoassesstherelevanceofbiomarkerlistsoras results (TheMAQCConsortium, 2010). Theprediction model M is built
afirststepinnetwork analysis. Theenrichment stepisperformed by using two different algorithms for classification and feature ranking.
usingthefunctional informationstoredintheKyotoEncyclopedia Themorerecent one is the ℓ1ℓ2 regularization with double optimization,
capableofselectingsubsetsofdiscriminative genes.Thealgorithmcanbe
of Genes and Genomes (KEGG) database (KanehisaandGoto,
tunedtogiveaminimalsetofdiscriminativegenesorlargersetsincluding
2000) and in the Gene Ontology (GO) database (Ashburneretal.,
correlated genes and it is based on the optimization principle presented
2000). The reconstruction of molecular pathways from high-
inZouandHastie(2005).Theimplementation usedconsistsoftwostages
throughput data is then based on the theory of complex networks
(DeMoletal.,2009)anditiscastinnestedloopsof10-foldcross-validation
(e.g. Strogatz (2001); Newman (2003); Boccalettietal. (2006); (Barlaetal., 2008). The first stage identifies the minimal set of relevant
Newman (2010); Buchananetal. (2010)) and, in particular, in variables (in terms of prediction error), while, starting from the minimal
the reconstruction algorithms for inferring networks topology and list,thesecondoneselectsthefamilyof(almostcompletely)nestedlistsof
wiringfromdata(Heetal.,2009). relevantvariablesforincreasingvaluesoflinearcorrelation. Asalternative
Thestabilityanalysisisappliedtonetworksidentifyingcommon choice we consider Liblinear, a linear Support Vector Machine (SVM)
andspecificnetworksubstructuresthatcouldbebasicallyassociated classifierspecificallydesignedforlargedatasets(millionsofinstancesand
features) (Fanetal., 2008). In particular, the classical dual optimization
todisease. Theoverall goal of thepipelineistoidentifyandrank
problem with L2-SVM loss function is solved with a coordinate descent
the pathways reflecting major changes between two conditions,
or during a disease evolution. We start from a profiling phase method. Forourexperimentweadopttheℓ2-regularized penaltytermand
the module of the weights for ranking purposes within a 100 × 3-fold
based on classifiers and feature selection modules organized
cross validation schema. Webuild amodelforincreasing feature sublists
in terms of experimental procedure by Data Analysis Protocol where the feature ranking is defined according to the importance for the
(TheMAQCConsortium, 2010), obtaining a ranked list of genes classifier.Wechoosethemodel,andthusthetoprankedfeatures,providing
withthe highest discriminative power. Classificationtasks aswell a balance between the accuracy of the classifier and the stability of the
as quantitative phenotype targets can be considered by using signature(TheMAQCConsortium,2010).Thus,theoutputofthisfirststep
different machine learning methods inthis firstphase. Alternative isagenesignatureg1,...,gk(oneforeachmodelM)containingthekmost
methods are made available as components of one Open Source discriminantfeatures,rankedaccordingtheirfrequencyscore.
PathwayEnrichment
pipelinesystem.Theproblemofunderdeterminacyintheinference
Thesuccessive enrichment phasederives alistofrelevant pathways from
procedure is avoided by focusing only on subnetworks, and the
the discriminant features, moving the focus of the analysis from single
relevance of the studied pathways for the disease is judged in
genes to functionally related pathways. As outlined in the review by
terms of discriminative relevance for the underlying classification
Huangetal. (2009), in the last 10 years the gene-annotation enrichment
problem. analysis field has been growing rapidly and several bioinformatics tools
Asatestbedfortheglocalstabilityanalysis,wecomparenetwork have been designed for this task. Huangetal. (2009) provide a unique
structures on a collection of microarray data for patients affected categorization of these enrichment tools in three major categories based
by Parkinson’s disease (PD), together with a cohort of controls on the underlying algorithm: singular enrichment analysis (SEA), gene
(Zhangetal., 2005b). PD is a neurodegenerative disorder that setenrichmentanalysis(GSEA),andmodularenrichmentanalysis(MEA).
impairsthemotorskillsattheonsetandthecognitiveandthespeech We choose one representative E for each class for our comparison
functionssuccessively. Thegoalofthistaskistoidentifythemost referring as sources of information D both to the KEGG, to explore
knowninformationonmolecularinteractionnetworks, andGO,toexplore
relevant dirsupted pathways and genes in late stage PD. On this
functionalcharacterizationandbiologicalannotation.Inthefirstcategorywe
dataset, we show that choosing different profiling approaches we
chooseWebGestalt(WG),anonlinegenesetanalysistoolkit(Zhangetal.,
getlowoverlappingintermsofcommonenrichedpathwaysfound.
2005a)taking as input a list of relevant genes/probesets. The enrichment
Despitethisvariability,ifweconsiderthemostdisruptedpathways, analysis is performed in KEGG and GO identifying the most relevant
theirglocaldistances(betweencaseandcontrolnetworks)sharea pathwaysandontologiesinthesignatures. WGadoptsthehypergeometric
common distribution assessing equal meaningfulness to pathways testtoevaluatefunctionalcategoryenrichmentandperformsamultipletest
foundstartingfromdifferentapproaches. adjustment (the default method is the one from BenjaminiandHochberg
(1995)).Theusermaychoosedifferentsignificancelevelsandtheminimum
number of genes belonging to the selected functional groups. GSEA
(Subramanianetal.,2005)isourrepresentative ofthesecondclass.Itfirst
2 MATERIALSANDMETHODS performs a correlation analysis between the features and the phenotype
obtaining a ranked list of features. Secondly it determines whether the
The machine learning pipeline adopted in this paper has been originally membersofgivengenesetsarerandomlydistributed intherankedlistof
introduced in (Barlaetal., 2011). As shown in Figure 1, it handles features obtained above, orprimarily foundatthe toporbottom. Weuse
case/controltranscriptiondatathroughfourmainsteps,fromaprofilingtask theprerankedanalysistool, feedingtherankedlistsofgenesproducedby
totheidentification ofdiscriminantpathways. Thepipelineisindependent the profiling phase directly to the enrichment step of GSEA. To avoid a
fromthealgorithmsused:herewedescribeeachstepandtheimplementation miscalculationoftheenrichmentscoreES,weprovideasinputthecomplete
adopted for the following experiments evaluating the impact of different list ofvariables (not just the selected ones), assigning to the not-selected
enrichmentmethodsinpathwayprofiling. a zero score. Note that GSEA calcultates enrichment scores that reflect
Formally, wearegiven acollection ofnsubjects, each described bya thedegreetowhichapathwayisoverrepresented atthetoporthebottom
p-dimensional vector x of measurements. Each sample is also associated ofthe ranked list. Inour analysis weconsidered only pathways enriched
withaphenotypical label, e.g. y ={1,−1}, assigning ittoaclass (ina withthetopofthelist. Finally, thetoolintheMEAclassisthePathways
classificationtask).Thedatasetisthereforerepresentedbyan×pexpression and Literature Strainer (PaLS) (Alibe´setal., 2008), which takes a list or
datamatrixX,wherep≫n,andacorrespondinglabelsvectorY. asetoflists ofgenes (orprotein identifiers) andshows whichones share
FeatureSelectionStep thesameGOterms orKEGGpathways, following acriterion basedona
The matrix X is used to feed the profiling part of the pipeline within a
2
Evaluatingsourcesofvariabilityinpathwayprofiling
Fig.1. Generalschemeoftheanalysispipeline,withtheindicationofthealgorithmsandtoolsusedinthePDdatasetapplication(lowerboxes).
threshold t set by the user. The tool provides as output those functional informationscoresforallpossiblepairswithinthecorrespondingnetwork
groupsthataresharedatleastbythet%oftheselectedgenes.PaLSisaimed context (i.e. all the pairs including either the regulator or the target).
ateasingthebiologicalinterpretationofresultsfromstudiesofdifferential RegnANNisanewlydefinedmethodforinferringgeneregulatorynetworks
expressionandgeneselection,withoutassigninganystatisticalsignificance basedonanensembleoffeed-forwardmultilayer perceptrons. Correlation
to the final output. Applying the above mentioned pathway enrichment isusedtodefinegeneinteractions. Foreachgeneaone-to-manyregressor
techniques, weretrieveforeachgenegithecorrespondingwholepathway istrainedusingthetranscriptiondatatolearntherelationshipbetweenthe
pi = {h1,...,ht},wherethegeneshj 6= ginotnecessarilybelongtothe geneandalltheothergenesofthenetwork.Theinteractionamonggenesare
originalsignatureg1,...,gk.Extendingtheanalysistoallthehjgenesofthe estimatedindependently andtheoverallnetworkisobtainedbyjoiningall
pathwayallowsustoexplorefunctionalinteractionsthatwouldotherwiseget theneighborhoods.Summarizing,weobtainareal-valuedadjacencymatrix
lost. as output of the subnetwork inference step for each dataset X, for each
SubnetworkInference class y, for each model M, foreach enrichment tool E, foreach source
Foreachpathway,networksareinferredseparatelyondatafromthedifferent ofinformationD,foreachpathwaypi,andforeachsubnetworkinference
classes. Thesubnetworkinferencephaserequirestoreconstructanetwork algorithmN.Wethusneedtoquantitatively evaluatenetworkdifferences,
Npi,y onthepathway pi byusingthesteadystateexpressiondataofthe i.e.usingametricinsteadofevaluatingnetworkproperties.
samplesofeachclassy.Thenetworkinferenceprocedureislimitedtothe SubnetworkDistanceandStability
sole genes belonging to the pathway pi in order to avoid the problem of Amongthepossiblechoicesalreadyavailableinliterature,wefocusontwo
intrinsicunderdeterminacyofthetask.Asanadditionalcautionagainstthis ofthemostcommondistancefamilies:thesetofedit-likedistancesandthe
problem, in the following experiments we limit the analysis to pathways spectraldistances.Thefunctionsintheformerfamilyquantitativelyevaluate
havingmorethan4nodes andlessthan1000nodes. Thepipeline allows the differences between two networks (with the same number of nodes)
torunanalysis inparallel withdifferent methods andthus toevaluate the in terms of minimum number of edit operations (with possibly different
variability along the whole pipeline. We adopt four different subnetwork costs)transforming onenetwork into theother, i.e. deletion andinsertion
reconstructionalgorithmsN:theWeightedGeneCo-ExpressionNetworks oflinks, whilespectral measuresrelies onfunctions oftheeigenvalues of
(WGCN)algorithm(Horvath,2011),theAlgorithmfortheReconstruction oneoftheconnectivity matrices oftheunderlying graph. Asdiscussedin
of Accurate Cellular Networks (ARACNE) (Margolinetal., 2006), the Jurmanetal.(2011),thedrawbackofmanyclassicaldistances(suchasthose
ContextLikelihoodofRelatedness(CLR)approach(Faithetal.,2007),and oftheeditfamily) islocality, that isfocusing onlyontheportions ofthe
theReverseEngineeringGeneNetworksusingArtificialNeuralNetworks networkinterested bythedifferences inthepresence/absence ofmatching
(RegnANN)(Grimaldietal.,2011).Inthiswork,weappliedWGCNA,CLR links.Spectraldistancescanovercomethisproblemconsideringtheglobal
andARACNEtoanalyzethepathwayidentifiedinthePathwayEnrichment structureofthecomparedtopologies. Withinthem,weconsidertheIpsen-
step,whileRegnANNwasused,asanalternativealgorithm,toreconstruct Mikhailovǫdistance: originallyintroducedinIpsenandMikhailov(2002)
interestingdisruptedpathwaysandtocompareitsresultswithresultsfrom asatoolfornetworkreconstructionfromitsLaplacianspectrum,ithasbeen
methodsmentionedabove.WGCNAisbasedontheideaofusing(afunction proventobethemostrobustinawiderangeofsituationsbyJurmanetal.
of) the absolute correlation between the expression of a couple of genes (2011). We are also aware that spectral measures are not flawless: they
across thesamples todefinealink between them. ARACNEis amethod cannotdistinguishisomorphicorisospectralgraphs,whichcancorrespond
forinferring networks from the transcription level (Margolinetal., 2006) toquitedifferentconditionswithinthebiologicacontext.Wethusintroduce
to the metabolic level (Nemenmanetal., 2007). Beside it was originally theglocaldistanceφasapossiblesolutionagainstbothissues:φisdefined
designedforhandlingthecomplexityofregulatorynetworksinmammalian astheproductmetricoftheHammingdistanceH(asrepresentative ofthe
cells,itisabletoaddressawiderrangeofnetworkdeconvolutionproblems. edit-familiy) andtheǫdistance. Fullmathematical details areavailable in
Thisinformation-theoretic algorithm removesthevastmajorityofindirect Jurmanetal.(2012).
candidateinteractionsinferredbyco-expressionmethodsbyusingthedata Relyingonthedistancesǫandφ,weevaluatenetworkscorrespondingto
processing inequality property (CoverandThomas, 1991). CLR belongs thesamepathwayfordifferentclasses,i.e.allthepairs(Npi,+1,Npi,−1)
to the relevance networks class of algorithms and is employed for the andrankthepathwaysthemselvesfromthemosttotheleastchangingacross
identification of transcriptional regulatory interactions (Faithetal., 2007). classes.
Inparticular,interactionsbetweentranscriptionfactorsandgenetargetsare Moreover,weattachedtoeachnetworkaquantitativemeasureofstability
scored by using the mutual information between the corresponding gene with respect to data subsampling, in order to evaluate the reliability of
expression levels coupled with an adaptive background correction step. inferred topologies. Inparticular, foreach Npi,y, weextracted arandom
Indeedthemostprobableregulator-targetinteractionsarechosencomparing subsampling(ofafractionrofXlabelledasy)onwhichthecorresponding
themutualinformationscoreversusthe”background”distributionofmutual Npi,y willbereconstructed. Repeatingmtimesthesubsampling/inferring
3
Barlaetal
Table 1. Summary of pathways retrieved in the Table 2. Summary of common most disrupted
pathwayenrichmentstep.Thenumbersinbracketsrefer pathways(φ≥0.05).
to the pathways considered for the network inference
step.
M D |∩(AllE)| |∩(EWG,PaLS)|
M D WG GSEEA PaLS ℓ1ℓ2 KGEGOG 10 2127
GO 0 5
Liblinear
GO 114(92) 7(7) 381(331) KEGG 0 21
ℓ1ℓ2 KEGG 43(43) 2(2) 71 (71)
GO 83(45) 0(0) 404(356)
Liblinear
KEGG 56(55) 1(1) 77 (77)
boostrapConfidenceInterval: (0.78;0.83)) coupledwithastability
of0.70.Thelistshave119commongenes.
The number of enriched pathways greatly varied depending on
procedure, a set of m nets will be generated for each Npi,y. Then all theselectionandenrichmenttools.Withℓ ℓ ,wefoundgloballyfor
mutual (cid:0)m2(cid:1) = m(m2−1) distances are computed, and for each set of GOandKEGG,157,452and9pathways1as2significantlyenriched,
m graphs we build the corresponding distance histogram. In particular,
for WG, PaLS and GSEA respectively. Similarly, for Liblinear,
for our experiments we set m = 20 and r = 2. Mean and variance
3 theidentified pathways were: 139, 481 and 1. Table1 reports the
oftheconstructedhistogramswillquantitatively assessthestabilityofthe
detailedresultsformodelM,enrichmentE anddatabaseD.
subnetworkinferredfromthewholedataset:thelowerthevalues,thehigher
thestabilityintermsofrobustnesstodataperturbation(subsampling). If we consider the ℓ1ℓ2 selection method and the enrichment
Datadescriptionandpreprocessing performedwithintheGO,wemaynotethatnocommonGOterms
The presented approach is applied to PD data originally introduced in were selected across enrichment methods. A significant overlap
Zhangetal. (2005b) and publicly available at Gene Expression Omnibus of results was found only between WG and PaLS, with 30 GO
(GEO),withaccessionnumberGSE20292.Thebiologicalsamplesconsist common terms. Similar considerations may be drawn with the
ofwholesubstantianigratissuein11PDpatientsand18healthycontrols. resultsfromtheLiblinearfeatureselectionmethod.WithintheGO
Expressions were measured on Affymetrix HG-U133A platform. We
enrichment we did not identify any common GO term among the
perform the data normalization on the raw data with the rma algorithm
threeenrichment tools. Considering only WGand PaLS,wewere
of the R Bioconductor affy package with a custom CDF (downloaded
abletoselect12commonGOterms.
from BrainArray: http://brainarray.mbni.med.umich.edu) adopting Entrez
If we consider the ℓ ℓ selection method and the enrichment
identifiers. 1 2
SoftwareAvailability performed within KEGG, two common pathways are identified
ThePythonimplementationofℓ1ℓ2regularizationwithdoubleoptimization across enrichment methods. A significant overlap of results was
isavailable at http://slipguru.disi.unige.it/Software/L1L2Py. Liblinear was found between WG and PaLS, with 43 common pathways. For
originally developed by the Machine Learning Group at the National Liblinear,onlyonecommonpathwaywasselectedamongthethree
Taiwan University and it is now available within the Python mlpy enrichmenttools.Asignificantoverlapofresultswasfoundbetween
library (http://mlpy.fbk.eu). We adopt the l2rl2losssvcdual WGandPaLS,with55commonpathways.
solver, with C = 10e−4. WG is available as a web application at
Following the pipeline, we also performed a comparison of the
http://bioinfo.vanderbilt.edu/webgestalt.GSEAisavailableeitherasaweb
three network reconstruction methods. We considered the most
applicationoraJavastand-alonetoolathttp://www.broadinstitute.org/gsea.
disruptednetworks,keepingfortheanalysisthosepathwaysthathad
PaLSisavailable onlineathttp://pals.bioinfo.cnio.es asawebapplication.
For three of the network reconstruction algorithms, we adopted the R aglocaldistancegreaterorequaltothechosenthresholdτ =0.05.
Bioconductorimplementation:theWGCNApackageforWGCN,andMiNET Thechoiceofsuchthresholdwasmadeconsideringthedistribution
(Mutual Information NETworks package) for ARACNE and CLR. In of glocal distances φ for the methods M. For instance, if we
particular, wesettheWGCNAsoftthresholding exponent to5, while we consider the Liblinear selection method and the KEGG database,
keepthedefaultvaluefortheARACNEdataprocessinginequalitytolerance we have a cumulative distribution as depicted in Figure 2(a). The
parameter (Meyeretal., 2008). Moreover, the ARACNE implementation threshold τ is set to 0.05 and allows retaining at least 50% of
requires all the features to have non-zero variance on each class and for pathways. The plot in Figure 2(b) represents the glocal distances
consistencypurposesweappliedthisinallexperiments.RegnANNisinstead
distribution for all enrichment methods E with respect to the two
availablefromhttp://sourceforge.net/projects/regnann.Itisimplementedin
components of the glocal distance: the Ipsen distance ǫ and the
Candrelies onGPGPUprogrammingparadigm forimproving efficiency.
HammingdistanceH.Theredcurvedlinerepresentsthethreshold
Theglocal distance φisavailable uponrequesttotheauthors eitherasR
scriptorPythonscript.ThecomputationoftheIpsen-Mikhailovdistanceǫ τ inthisspace. Theplot inFigure2(c)isdetailedfor subnetwork
isincludedascomponentoftheglocalscript. inferencemethodN.
After retaining the most distant pathways, we performed a
comparison of common terms for fixed selection method M and
3 RESULTSANDDISCUSSION
databaseD. TheresultsarereportedinTable2. InTables3and4
The feature selection results varied accordingly to the chosen wereport the most disrupted GO terms and KEGG pathways that
method: ℓ ℓ identified 458 discriminant genes associated to an haveaglocaldistanceφgreaterorequaltothechosenthresholdτ.
1 2
averagepredictionperformanceof80.8%,whilewithLiblinearwe AsanexampleofaselectedpathwaywithinKEGG,thenetworks
selectedthetop-500genesassociatedtoanaccuracyof80%(95% (thresholded at edge weight 0.1 for graphic purposes) inferred
4
Evaluatingsourcesofvariabilityinpathwayprofiling
Table3. SummaryofmostdisruptedGOtermscommonbetweenWGandPaLS,fordifferentmodelsM.EachGOtermisassociatedtoaglocaldistance
φ≥0.05forallsubnetworkreconstructionalgorithmsN.GOtermsaresortedaccordingdecreasingaverageφ.BoldfontsrepresenttheGOtermssharedby
modelM.
ℓ1ℓ2 Liblinear
ID Termname ID Termname
GO:0005739 Mitochondrion GO:0042127 Regulationofcellproliferation
GO:0031966 Mitochondrialmembrane GO:0005783 Endoplasmicreticulum
GO:0005743 Mitochondrialinnermembrane GO:0015629 Actincytoskeleton
GO:0042802 Identicalproteinbinding GO:0006469 Negativeregulationofproteinkinaseactivity
GO:0007018 Microtubule-basedmovement GO:0005747 MitochondrialrespiratorychaincomplexI
GO:0046961 Proton-transportingATPaseactivity,rotationalmechanism
GO:0005753 Mitochondrialproton-transportingATPsynthasecomplex
GO:0000502 Proteasomecomplex
GO:0015986 ATPsynthesiscoupledprotontransport
GO:0045202 Synapse
GO:0048487 Beta-tubulinbinding
GO:0042734 Presynapticmembrane
GO:0005747 MitochondrialrespiratorychaincomplexI
GO:0006120 Mitochondrialelectrontransport,NADHtoubiquinone
GO:0015078 Hydrogeniontransmembranetransporteractivity
GO:0015992 Protontransport
GO:0005874 Microtubule
5566555036556660078008552103558585637384935595563551293P605a45r29k0846in583s40o20n467 134 767−43 70945240864157404 61−64 W6241G7773C11N274A72 19(t30h7226=19030.35572)19725099741033289293880439261288116844191192729460673132 00..2350 Aracne 050C1L4R WGCNA 555666550350656607808055231058553658483753959563515259305605450291084465 83−40 20G46S73E4767243079242948246 75c40la1646s26s14 +777131 1−27 4R27e91g032762n91A03N5327N91052747990133289932804832926818181464991211790264763312
0.20
5566555036556660078008552103555858363748395559536552193P605a4r5k29i0846n583s40o20n4 67−341767 4−37 904254846075140416 −46 26W1477G731C127N4A2791 (03t27h6291=03053.72519)725099741033289293880439261288116844191192729460673132 f 000...011505 555665635056500667088055231085556385847355935936525159306505450291084465 83−40 G2046S73E4762743079242948264 57c04la6164s62s41 −777131 1−27 4R72e19g307226n19A30N3572N19750297491033829329848039268218114864911291794206736321
0.00
(a) (b) (c)
Control PD Control PD Control PD
Fig.3. (a)NetworksinferredbyWGCNAalgorithm fortheALSKEGGpathwayforPDpatients(above)andcontrols(below), onthesamepathwayfor
differentinferencealgorithm.(b)WGCNAisthemethodshowingthehigheststabilityonthetwoclasses.(c)SamepathwayreconstructedwithRegnANN.
by WGCNA (together with the corresponding stability) on the of this step by choosing two approaches within the regularization
AmyotrophicLateralSclerosisKEGGpathway (ALS- 05014) are methods family. Both classifiers adopt a ℓ -regularization penalty
2
displayed in Figure 3. We also plot the inferred network by the term, combined with different loss functions and, for ℓ ℓ with
1 2
RegnANNalgorithm.Similarly,inFigure4weplotthePathogenic another regularization term. We used similar but not equal model
E. coli infection KEGG pathway, reconstructed by WGCNA, its selectionprotocols.Bothguaranteethattheresultsarenotaffected
stability plot, and the corresponding inferred networks by the byselection-bias.Inthiswork,themainsourceofvariabilitywasthe
RegnANNalgorithm. choiceofthegeneenrichmentmodule.Therefore,theexperimenter
Discussion must be careful in choosing one method or another and in using
The variability in the results, as expected, strongly depends on it compliantly with the experimental design. For instance, GSEA
the method of choice. For feature selection, the nature of the wasdesigned for estimatingthesignificance levelsbyconsidering
method is key. In the proposed pipeline we limited the impact separatelythepositively andnegativelyscoringgene setswithina
5
Barlaetal
Table4. SummaryofmostdisruptedKEGGpathwayscommonbetweenWGandPaLS,fordifferentmodelsM.Eachpathwayisassociatedtoaglocal
distanceφ ≥ 0.05forallsubnetworkreconstructionalgorithmsN.KEGGpathwaysaresortedaccordingdecreasingaverageφ.Boldfontsrepresentthe
KEGGpathwayssharedbymodelM.
ℓ1ℓ2 Liblinear
ID Pathwayname ID Pathwayname
01100 Metabolicpathway 04630 Jak-STATsignalingpathway
05130 PathogenicEscherichiacoliinfection 01100 Metabolicpathway
04910 Insulinsignalingpathway 05130 PathogenicEscherichiacoliinfection
00310 Lysinedegradation 04623 CytosolicDNA-sensingpathway
04140 Regulationofautophagy 00330 Arginineandprolinemetabolism
03050 Proteasome 04910 Insulinsignalingpathway
00230 Purinemetabolism 05212 Pancreaticcancer
05014 Amyotrophiclateralsclerosis* 03030 DNAreplication
00980 MetabolismofxenobioticsbycytochromeP450 05213 Endometrialcancer
00620 Pyruvatemetabolism 04660 Tcellreceptorsignalingpathway
05213 Endometrialcancer 04310 Wntsignalingpathway
00270 Cysteineandmethioninemetabolism 05210 Colorectalcancer
00240 Pyrimidinemetabolism 04912 GnRHsignalingpathway
05120 EpithelialcellsignalinginHelicobacterpyloriinfection 05332 Graft-versus-hostdisease
05110 Vibriocholeraeinfection 04520 Adherensjunction
00020 Citratecycle(TCAcycle) 04621 NOD-likereceptorsignalingpathway
00562 Inositolphosphatemetabolism 04370 VEGFsignalingpathway
00600 Sphingolipidmetabolism 04662 Bcellreceptorsignalingpathway
05218 Melanoma 04722 Neurotrophinsignalingpathway
00010 Glycolysis/Gluconeogenesis 05214 Glioma
00051 Fructoseandmannosemetabolism 04330 Notchsignalingpathway
04722 Neurotrophinsignalingpathway
*Note:ThisistheonlyselectedpathwaysharedacrossallenrichmentmethodsE.
listofgenesselectedwithfiltermethodsbasedonclassicalstatistical biologicalfeaturesinparticularatthemithocondriallevel.Moreover
tests. It is worth noting that, if one uses the preranked option, at the phenotypic level the skeletal muscles of the patients are
as we did, negative regulated groups might not be significant at severelyaffectsinfluencingthemovements.InFigure3itisevident
all (we indeed discarded them). WG uses the Hypergeometrical thatahighnumberofinteractionsareestablishedamongthegenes
test to assess the functional groups but, differently from GSEA, going from the control (below) to the affected (above) pathways.
does not use any significance assessment based on permutation It is also interesting to underline that CYCS (Entrez ID: 54205)
of phenotype labels. PaLS is the simplest methods, being just a one of the hub genes (represented by a red dot in the graph)
measureofoccurrencesofagivendescriptorinthelistofselected within the pathway was identified by ℓ ℓ as discriminant. This
1 2
genes. However, enrichmentmethodsfromdifferentcategoriesare geneishighlyinvolvedinseveralneurodegenerativediseases(e.g.,
complementary and can identify different but equally meaningful PD,Alzheimer’s, Huntington’s)andinpathwaysrelatedtocancer.
biological aspects of the same process. Thus, the integration of Furthermoreitsproteinisknowntofunctionsasacentralcomponent
informationacrossdifferentmethodsisthebeststrategy. oftheelectron transport chain inmitochondria andtobeinvolved
Moreover,theassessmentofthereconstructiondistancebetween in initiation of apoptosis, known cause of the neurons loss in
case and control version of the same pathways help in providing PD.AcrossvariableselectionalgorithmsM, fivehighlydisrupted
a quantitative focus on the key pathway involved in the process. pathwayswerefoundassharedbetweentwoofthethreeenrichment
Theuseofadistancemixingtheeffectsofstructuralchangeswith methods(seeTable4,bolditems).Inparticular,werepresentedin
thoseduetothedifferencesinrewiringmoreover warrantsamore Figure4thecorrespondinginferrednetworks. Tofurtherhighlight
informative view on the difference assessment itself. The limited the different outcomes occurring from the same dataset when
effect of different feature selection methods is confirmed by the diverseinferencemethodsareemployed,wereconstructedtheALS
plotsinFigure5. andPathogenicE.coliinfectionbytheRegnANNalgorithm,which
For ℓ ℓ , the only most disrupted pathway shared by the three tends to spot also second order correlation among the network
1 2
enrichmenttoolsEandthethreereconstructionmethodsN isALS. nodes,seeFigures3and4.
Thispathway is relevant in this context because, like PD, ALS is Two genes in the E. coli infection pathway were selected both
another neurodegenerative disease therefore they share significant by ℓ ℓ and Liblinear, namely ABL1 (Entrez ID:71) and TUBB6
1 2
6
Evaluatingsourcesofvariabilityinpathwayprofiling
67005579966513960770507180144728627097771847627983400833P7047a74r7k5733i4n503s35o49n78 2145 76−29 5083564821113306024 2−8730 1W3807G6823C840N4107A814 4(6t981h1940=775095.51028)091397080633786192101903823819117919100649001197809009595099293 00..2350 Aracne 051C3L0R WGCNA 670057996561396500570770114828747026777981742689730048330704577147353703435 0−35 49G7S824E57622905823694821213 c06l24a2873s01s38 07+62831804 −14 70R814e46g98119n470A75N958012N907391680037316890121930828311911979010469119007800909509995923
0.20
6700557996651396077050718014472862709777187426789340083P370a47r74k75i733n4503s35o49n7 8−241576 2−95 8036548211133060242 873−01 3W8076G823840C41N078A144 6(981t1h9407=75095.1028509)1397080633786192101903823819117919100649001197809009595099293 f 000...011505 6750076599616539007507701148284770627797741826897030483037054717437503374 305−53 G947S82E457262905823694281231 c60l422a8370s13s80 7−62831804 −1470 8R144e69g8119n407A7595N1802N091379086033781962011938028131991170419061197090009508909995923
0.00
(a) (b) (c)
Control PD Control PD Control PD
Fig.4. (a)NetworksinferredbyWGCNAalgorithmforthePathogenicE.coliinfectionKEGGpathwayforPDpatients(above)andcontrols(below),onthe
samepathwayfordifferentinferencealgorithm.(b)WGCNAisthemethodshowingthehigheststabilityonthetwoclasses.(c)Samepathwayreconstructed
withRegnANN.
(Entrez ID: 84617). ABL1 seems to play a relevant role as hub ACKNOWLEDGEMENT
both in the WGCNA and in the RegnANN networks. ABL1 is a The authors at FBK want to thank Shamar Droghetti for his help
protooncogene that encodes protein tyrosine kinase that has been withtheenrichmentwebinterfaces.
implicated in processes of cell differentiation, cell division, cell
adhesion, and stressresponse. It wasalsofound tobe responsible Funding: The authors at DISI acknowledge funding by the
of apoposis in human brain microvascular endothelial cells. In Compagnia di San Paolo funded Project Modelli e metodi
Figure 6 we note that pathways with high number of genes are computazionali nello studio della fisiologia e patologia di reti
similarintermoflocaldistance,insteadawiderrangeofvariability molecolari di controllo in ambito oncologico. The authors at
isfoundlookingatthespectraldistance.Theredlinein6(b)divides FBK acknowledge funding by the European Union FP7 Project
the 2 cluster. Pathway targets beyond and within the red line are HiperDARTandbythePATfundedProjectENVIROCHANGE.
representedinthecumulativehistogramin6(a). Pathwaysbeyond
the threshold are equally distributed and they represent a wider
range of targets, instead pathways within the threshold show a REFERENCES
smallernumberoftargets6(a)ontheright.
Alibe´s,A.,Can˜ada,A.,andD´ıaz-Uriarte,R.(2008).PaLS:filteringcommonliterature,
biologicaltermsandpathwayinformation.NucleicAcidsRes,36(WebServerissue),
W364–W367.
Ambroise,C.andMcLachlan,G.(2002).Selectionbiasingeneextractiononthebasis
ofmicroarraygene-expressiondata.PNAS,99(10),6562–6566.
Ashburner,M.,Ball,C.A.,Blake,J.A.,Botstein,D.,Butler,H.,Cherry,J.M.,Davis,
A.P.,K.,D.,Dwight,S.S.,Eppig,J.T.,Harris,M.A.,Hill,D.P.,Issel-Tarver,
4 CONCLUSION
L.,Kasarskis,A.,Lewis,S.,Matese,J.C.,Richardson,J.E.,Ringwald,M.,Rubin,
Moving from gene profiling towards pathway profiling can be an G.M.,andSherlock,G.(2000). Geneontology:toolfortheunificationofbiology.
effectivesolutiontoovercometheproblemofthepooroverlapping thegeneontologyconsortium.NatureGenetics,25(1),25–9.
Barabasi,A.L.,Gulbahce,N.,andLoscalzo,J.(2011).Networkmedicine:anetwork-
in -omics signatures. Nonetheless, the path from translating a
basedapproachtohumandisease.NatureReviewGenetics,12,56–68.
discriminant gene panel intoacoherent set of functionally related Baralla,A.,Mentzen,W.,anddelaFuente,A.(2009).InferringGeneNetworks:Dream
genesetsincludesanumberofstepseachcontributingininjecting orNightmare?Ann.N.Y.Acad.Sci.,1158,246–256.
variability in the process. To reduce the overall impact of such Barla,A.,Mosci,S.,Rosasco,L.,andVerri,A.(2008). Amethodforrobustvariable
selectionwithsignificanceassessment.ProceedingsofESANN2008.
variability, it is thus critical that, whenever possible, the correct
Barla,A.,Jurman,G.,Visintainer,R.,Filosi,M.,Riccadonna,S.,andFurlanello,C.
tool for each single step is adopted, accurately focussing on the
(2011). Amachinelearningpipelinefordiscriminantpathwaysidentification. In
desiredtarget tobe investigated. Thismainly holds for thechoice CIBB2011,pages1–10.ISBN:9788890643705.
of the most suitable enrichment tool and biological knowledge Benjamini,Y.andHochberg,Y.(1995).Controllingthefalsediscoveryrate:apractical
database, and, to a lower extent, to the inference method for andpowerfulapproachtomultipletesting. JournaloftheRoyalStatisticalSociety,
SeriesB.
the newtork reconstruction: all these ingredients are planned
Boccaletti,S.,Latora,V.,Moreno,Y.,Chavez,M.,andHwang,D.-U.(2006).Complex
for different objectives, and their use on other situations may networks:Structureanddynamics.PhysicsReports,424(4–5),175–308.
result misleading. As a final observation and a possible future Buchanan,M.,Caldarelli,G.,DeLosRios,P.,Rao,F.,andVendruscolo,M.,editors
developmenttoexplore,theemerginginstabilitycanbetackledby (2010).NetworksinCellBiology.CambridgeUniversityPress.
Cover,T.andThomas,J.(1991).ElementsofInformationTheory.Wiley.
obtainingthefunctionalgroupsidentificationastheresultofaprior
DeMol, C., Mosci, S., Traskine,M., andVerri, A.(2009). Aregularizedmethod
knowledgeinjectioninthelearningphase,ratherthanaprocedurea
forselectingnestedgroupsofrelevantgenesfrommicroarraydata. Journalof
posteriori(Zycinskietal.,2011,2012). ComputationalBiology,page8.
7
Barlaetal
0.00.20.40.60.8 0.00.20.40.60.8
GSEA PALS WebGestalt GSEA PALS WebGestalt
0.8 A A A 0.8 A A A
0.6 N N N 0.6 N N N
C C C C C C
0.4 G G G 0.4 G G G
0.2 W W W 0.2 W W W
0.0 GSEA PALS WebGestalt 0.0 GSEA PALS WebGestalt
0.8 0.8
R R R 0.6 R R R 0.6
L L L L L L
e C C C 0.4 e C C C 0.4
0.2 0.2
GSEA PALS WebGestalt 0.0 GSEA PALS WebGestalt 0.0
0.8 0.8
e e e e e e
0.6 n n n 0.6 n n n
c c c c c c
0.4 a a a 0.4 a a a
0.2 Ar Ar Ar 0.2 Ar Ar Ar
0.0 0.0
0.00.20.40.60.8 0.00.20.40.60.8 0.00.20.40.60.8 0.00.20.40.60.8
H H
a:ℓ ℓ andKEGG b:LiblinearandKEGG
1 2
0.00.20.40.60.8 0.00.20.40.60.8
GSEA PALS WebGestalt GSEA PALS WebGestalt
0.8 A A A 0.8 A A A
0.6 N N N 0.6 N N N
C C C C C C
0.4 G G G 0.4 G G G
0.2 W W W 0.2 W W W
0.0 GSEA PALS WebGestalt 0.0 GSEA PALS WebGestalt
0.8 0.8
R R R 0.6 R R R 0.6
L L L L L L
e C C C 0.4 e C C C 0.4
0.2 0.2
GSEA PALS WebGestalt 0.0 GSEA PALS WebGestalt 0.0
0.8 0.8
e e e e e e
0.6 n n n 0.6 n n n
c c c c c c
0.4 a a a 0.4 a a a
0.2 Ar Ar Ar 0.2 Ar Ar Ar
0.0 0.0
0.00.20.40.60.8 0.00.20.40.60.8 0.00.20.40.60.8 0.00.20.40.60.8
H H
c:ℓ ℓ andGO d:LiblinearandGO
1 2
Fig.5. PlotsofHammingvs.Ipsendistances(Hvs.ǫ)forallpossiblecombinationsofM,D,EandN.Inouranalysisweconsideredtheglocaldistanceφ,
definedasthenormalizedproductofHandǫ.
DeSmet,R.andMarchal,K.(2010). Advantagesandlimitationsofcurrentnetwork perspectives.J.Biotechnol.,144(3),190–203.
inferencemethods.NatureReviewMicrobiology,8,717–729. Horvath,S.(2011).WeightedNetworkAnalysis:ApplicationsinGenomicsandSystems
Faith, J., Hayete, B., Thaden, J., Mogno, I., Wierzbowski, J., Cottarel, G., Kasif, Biology.Springer.
S., Collins, J., andGardner, T.(2007). Large-ScaleMappingandValidationof Huang,D.,Sherman,B.,andLempicki,R.(2009). Bioinformaticsenrichmenttools:
Escherichiacoli Transcriptional Regulation from a Compendium of Expression pathstowardthecomprehensivefunctionalanalysisoflargegenelists.NucleicAcids
Profiles.PLoSBiol.,5(1),e8. Res,37(1),1–13.
Fan,R.-E.,Chang,K.-W.,Hsieh,C.-J.,Wang,X.-R.,andLin,C.-J.(2008). Liblinear: Ipsen,M.andMikhailov,A.(2002). Evolutionaryreconstructionofnetworks. Phys.
Alibraryforlargelinearclassification. JournalofMachineLearningResearch,9, Rev.E,66(4),046109.
1871–1874. Jurman, G., Visintainer, R.,andFurlanello, C.(2011). Anintroductiontospectral
Furlanello,C.,Serafini,M.,Merler,S.,andJurman,G.(2003). Entropy-basedgene distancesinnetworks.InProc.WIRN2010,pages227–234.
rankingwithoutselectionbiasforthepredictiveclassificationofmicroarraydata. Jurman,G.,Visintainer,R.,Riccadonna,S.,Filosi,M.,andFurlanello,C.(2012). A
BMCBioinformatics. glocaldistancefornetworkcomparison.arXiv:submit/0397475[math.CO].
Grimaldi,M.,Visintainer,R.,andJurman,G.(2011). Regnann:Reverseengineering Kanehisa,M.andGoto,S.(2000).KEGG:kyotoencyclopediaofgenesandgenomes.
genenetworksusingartificialneuralnetworks.PLoSONE,6(12),e28646. NucleicAcidsRes,28(1),27–30.
He,F.,Balling,R.,andZeng,A.-P.(2009).Reverseengineeringandverificationofgene Leek,J.,Scharpf,R.,CorradaBravo,H.,Simcha, D.,Langmead,B.,Johnson,W.,
networks:Principles,assumptions,andlimitationsofpresentmethodsandfuture Geman, D.,Baggerly, K.,andIrizarry, R.(2010). Tacklingthewidespreadand
8
Evaluatingsourcesofvariabilityinpathwayprofiling
criticalimpactofbatcheffectsinhigh-throughputdata.NatRevGenet,11,733–739.
Margolin,A.,Nemenman,I.,Basso,K.,Wiggins,C.,Stolovitzky,G.,Dalla-Favera,
R.,andCalifano,A.(2006).ARACNE:analgorithmforthereconstructionofgene
regulatorynetworksinamammaliancellularcontext.BMCBioinform.,7(7),S7.
Meyer,P.,Lafitte,F.,andBontempi,G.(2008). minet:AR/BioconductorPackage
for Inferring Large TranscriptionalNetworks UsingMutualInformation. BMC
Bioinform.,9(1),461. 20
al
NemReencomnasntr,uIc.,tiEosncoolfaM,Ge.t,aHbolalivcaNceektw,Wor.k,sUfnrkoemfeHr,iPg.h,-UTnhkreofuegrh,Cpu.,taMndetWabaollli,tMeP.r(o2fi00li7n)g. of Tot15
e
Data.Ann.N.Y.Acad.Sci.,1115,102–115. ag10
Newman,M.(2003).TheStructureandFunctionofComplexNetworks.SIAMReview, ent
45,167–256. Perc 5
Newman,M.(2010).Networks:AnIntroduction.OxfordUniversityPress.
0
Sharan, R.andIdeker, T.(2006). Modelingcellularmachinerythroughbiological
0.0 0.1 0.2 0.3 0.4
networkcomparison.NatureBiotechnology,24(4),427–433. f
Strogatz,S.(2001).Exploringcomplexnetworks.Nature,410,268–276.
Subramanian,A.,Tamayo,P.,Mootha,V.K.,Mukherjee,S.,Ebert,B.L.,Gillette,
M.A.,Paulovich,A.,Pomeroy,S.L.,Golub,T.R.,Lander,E.S.,andMesirov,J.P.
(2005).Genesetenrichmentanalysis:Aknowledge-basedapproachforinterpreting
genome-wideexpressionprofiles.PNAS,102(43),15545–15550.
The MAQC Consortium (2010). The MAQC-II Project: A comprehensive study (a)
of common practices for the development and validation of microarray-based
predictivemodels.NatureBiotechnology,28(8),827–838.
Zhang,B.,Kirov,S.,andSnoddy,J.(2005a). WebGestalt:anintegratedsystemfor
exploringgenesetsinvariousbiologicalcontexts.NucleicAcidsRes,33(WebServer
issue),W741–8.
Zhang,Y.,James,M.,Middleton,F.A.,andDavis,R.L.(2005b). Transcriptional
Aracne CLR WGCNA
analysisofmultiplebrainregionsinparkinson’sdiseasesupportstheinvolvement 0.050.100.15
ofspecificproteinprocessing, energymetabolism, andsignalingpathways, and GSEA PALS WebGestalt
suggestsnoveldiseasemechanisms. AmJMedGenetBNeuropsychiatrGenet, 0.6
137B(1),5–16. 0.5
Zou,H.andHastie,T.(2005).Regularizationandvariableselectionviatheelasticnet. 0.4
J.R.Statist.Soc.B. e 0.3
0.2
Zycinski,G.,Barla,A.,andVerri,A.(2011). Svs:Dataandknowledgeintegrationin
0.1
computationalbiology. EngineeringinMedicineandBiologySociety,EMBC,2011
AnnualInternationalConferenceoftheIEEE,pages6474–6478. 0.050.100.15 0.050.100.15
Zycinski, G., Squillario, M., Barla, A., Sanavia, T., Verri, A., andDiCamillo, B. H
(2012). Discriminantfunctionalgenegroupsidentificationwithmachinelearning
andpriorknowledge.Submitted.
(b)
0.05 0.15
GSEA PALS WebGestalt
0.6
0.5 A A A
0.4 N N N
0.3 C C C
0.2 G G G
W W W
0.1
GSEA PALS WebGestalt 0.6
0.5
R R R 0.4
L L L 0.3
e C C C 0.2
0.1
0.6 GSEA PALS WebGestalt
0.5
e e e
0.4 n n n
0.3 c c c
a a a
0.2 Ar Ar Ar
0.1
0.05 0.15 0.05 0.15
H
(c)
Fig.2. DetaileddistanceplotforLiblinearandKEGG(seeFigure5b).The
histogramplotin(a)represents thecumulative histogramforalldistances
acrossenrichmentmethodsEandsubnetworkinferencealgorithmsN.The
threshold τ is set to retain at least 50% of pathways. In (b) a plot of
Hammingvs. Ipsendistances (Hvs. ǫ)forallpossiblecombinations ofE9
andN,whichisdetailedin(c).
Barlaetal
0 200 400 600 800100012001400
beyond within
35
30
Percent of Total 11220505
5
0
0 200 400 600 800100012001400
Number of genes
(a)
0.00 0.10 0.20
GSEA PALS WebGestalt
0.4
0.3
e 0.2
0.1
0.0
0.00 0.10 0.20 0.00 0.10 0.20
H
(b)
Fig.6. (a)Pathwaytargetcumulativehistogram.(b)HammingversusIpsen
(Hvs.ǫ)distance,andthresholdingofhighpopulatedpathways.
10
