Table Of ContentImproving power in
genome-wide association studies:
7
weights tip the scale
0
0
2
n Kathryn Roeder, B Devlin, and Larry Wasserman
a
J
3
]
T
S
.
h
Department of Statistics (KR, LW)
t
a
m Carnegie Mellon University
[ Pittsburgh,PA
1 Department of Psychiatry (BD)
v
4 University of Pittsburgh
0
Pittsburgh,PA
1
1
0
7
0
/
h
t
a
m RunningTitle: Weightsin genome-wideassociationstudies
:
v
i
X
r
a
Addressforcorrespondenceand reprints:
KathrynRoeder,DepartmentofStatistics,CarnegieMellonUniversity,5000ForbesAvenue,Pitts-
burgh,PA 15213. E-mail: [email protected]
Abstract
Genome-wideassociationanalysishasgeneratedmuchdiscussionabouthowtopreservepower
1
todetectsignalsdespitethedetrimentaleffectofmultipletestingonpower. Wedevelopaweighted
multiple testing procedure that facilitates the input of prior information in the form of groupings
of tests. For each group a weight is estimated from the observed test statistics within the group.
Differentially weighting groups improves the power to detect signals in likely groupings. The
advantageofthegrouped-weightingconcept,overfixedweightsbasedonpriorinformation,isthat
it often leads to an increase in power even if many of the groupings are not correlated with the
signal. Being data dependent, the procedure is remarkably robust to poor choices in groupings.
Power is typically improved if one (or more) of the groups clusters multiple tests with signals,
yet little power is lost when the groupings are totally random. If there is no apparent signal in a
group, relative to a group that appears to have several tests with signals, the former group will be
down-weighted relative to the latter. If no groups show apparent signals, then the weights will be
approximatelyequal. The only restriction on the procedure is that the numberof groups be small,
relativeto thetotalnumberoftestsperformed.
KeyWords: Bonferronicorrection,Genome-wideassociationanalysis,Multipletesting,Weighted
p-values.
2
Thorough testing for association between genetic variation and a complex disease typically
requires scanning large numbers of genetic polymorphisms. In a multiple testing situation, such
as a whole genome association scan, the null hypothesis is rejected for any test that achieves a
p-valueless than a predetermined threshold. To account for the greater risk of false positives,this
threshold is more stringent as the number of tests conducted increases. To bolster power, recent
statistical methods suggest up-weighting and down-weighting of hypotheses, based on prior like-
lihood of association with the phenotype (Genovese et al. 2006, Roeder et al. 2006). Weighted
proceduresmultiplythethresholdbytheweightw,foreachtest,raisingthethresholdwhenw > 1
and lowering it if w < 1. To control the overall rate of false positives, a budget must be imposed
on the weighting scheme. Large weights must be balanced with small weights, so that the aver-
age weight is one. These investigations reveal that if the weights are informative, the procedure
improves power considerably, but, if the weights are uninformative, the loss in power is usually
small. Surprisingly, aside from this budget requirement, any set of non-negative weights is valid
(Genoveseetal. 2006). Whiledesirableinsomerespects,thisflexibilitymakesitdifficulttoselect
weightsforaparticularanalysis.
The type of prior information readily available to investigators is often non-specific. For in-
stance, SNPs might naturally be grouped, based on features that make various candidates more
promising for this disease under investigation. For a brain-disorder phenotype we might cross-
classify SNPs by categorical variables such as those displayed in Table I. The SNPs in seem
1
G
most promising, a priori, while those in seem least promising. Those in and are more
4 2 3
G G G
promisingthanthosein ,butsomewhatambiguous. Itiseasytoimagineadditionalvariablesthat
4
G
furtherpartitiontheSNPsintovariousclassesthathelptoseparatethemorepromisingSNPsfrom
the others. While this type of information lends itself to grouping SNPs, it does not lead directly
to weights for the groups. Indeed it might not even be to possible to choose a natural ordering of
the groups. What is needed is a way to use the data to determine the weights, once the groups are
formed.
3
Functional Non-Functional
Brain expressed
1 2
G G
Non-Brain expressed
3 4
G G
TableI.
Untilrecently,methodsforweightedmultiple-testingrequired that priorweightsbedeveloped
independently of the data under investigation (Genovese et al. 2006, Roeder et al. 2006). In this
article we ask the following questions: if the weights are to be applied to tests grouped by prior
information, what choice of weights will optimize the average power of the genetic association
study? Howcanweestimatetheseweightsfromthedatatoachievegreaterpowerwithoutaffecting
controlofthefamily-wiseerror rate?
Methods
Consider m hypotheses corresponding to standardized test statistics T = (T1,...,Tm). The p-
values associated with the tests are (P1,...,Pm). We assume Tj is approximately normally dis-
tributedwithnon-centralityparameterξ ,orthetestsareχ2 distributedwithnon-centralityparam-
j
eter ξ2. When using a Bonferroni correction for m tests, the threshold for rejection is achieved if
j
thep-valueP α/m. TheweightedBonferroni procedureof Genovese,Roeder and Wasserman
j
≤
(2005)is as follows. Specify nonnegativeweightsw = (w1,...,wm) and reject hypothesisHj if
P α
j = j : j . (1)
∈ R w ≤ m
(cid:26) j (cid:27)
As long as m−1 w = 1, this procedure controls family-wiseerror rate at level α. For a test of
j j
ξ = 0 vs. ξ = 0P,thepowerofasingleweightedtestis
j j
6
−1 αwj −1 αwj
π(ξ ,w ) = Φ Φ ξ +Φ Φ +ξ ,
j j j j
2m − 2m
(cid:16) (cid:16) (cid:17) (cid:17) (cid:16) (cid:16) (cid:17) (cid:17)
where Φ(t) is the upper tail probability of a standard normal cumulative distribution function.
Whenthealternativehypothesisistrue,weightingincreasesthepowerwhenw > 1anddecreases
j
4
thepowerwhen wj < 1. We call π(ξj,wj) theper-hypothesispower. Forsignals(ξ1,...,ξm) and
weights(w1,...,wm)theaveragepower is
m
1
π(θ,w) = π(ξ ,w ).
j j
m1
j=1
X
The optimal weight vector w = (w1,...,wm) that maximizes the average power subject to
w 0 andm−1 m w = 1 is (Wassermanand Roeder 2006)
j ≥ j=1 j
P
m ξ c
w(ξ ) = Φ | j| + , (2)
j
α 2 ξ
(cid:18) | j|(cid:19)
wherec istheconstantthatsatisfies thebudgetcriterion onweights
m
1
w(ξ ) = 1. (3)
j
m
j=1
X
The optimal weights vary with the signal strength in a non-monotonicmanner (Figure 1). For
anyparticularsample,cadjuststheweightstosatisfythebudgetconstraintonweights. Insodoing,
itshiftsthemodeoftheweightfunctionfromlefttorightdependingonthenumberofsmall,versus
large, signalsobserved.
The optimal weight function has an interesting effect on the rejection threshold. This choice
ofweights results in athreshold for rejection that varies smoothlywith thesignal strength. Figure
2 plots the rejection threshold log (αw /m), calculated for the data displayed in Figure 1,
10 j
−
as a function of the signal strength and contrasts it with the rejection threshold of a Bonferroni
corrected test log (α/m). From Figures 1-2 it is evident why an optimally weighted test has
10
−
greater powerthan a non-weighted test. The weighted-thresholdis less stringentfor signals in the
midrange,andmorestringentforbothlargeandsmallsignals. Consequently,ifthesignalislikely
tobe verystrongorvery weak, thetest isdown-weighted(weightless thanone). In practice, little
power is lost by this tradeoff. For small signals the chance of rejecting the hypothesis is minimal
with orwithout weights. Forlarge signalsthe p-valueis likelyto cross the thresholdregardless of
theweight. Larger weightsarefocused in themidrangeto helpto revealsignalsthatare marginal.
5
Clearlyξ isnotknown,soitmustbeestimatedtoutilizethisweightfunction. Anaturalchoice
j
istobuildonthetwostageexperimentaldesign(SatagopanandElstonRC2003;Wangetal. 2006)
and split the data into subsets, using one subset to estimateξ , and hence w(ξ ), and the second to
i i
conductaweightedtestofthehypothesis(Rubinetal. 2006). Thisapproachwouldarisenaturally
in an association test conducted in stages. It does lead to a gain in power relative to unweighted
testingof stage2 data; however,it is notbetter than simplyusingthefull data set withoutweights
for the analysis (Rubin et al. 2006; Wasserman and Roeder 2006). These results are corroborated
bySkoletal. (2005)inarelatedcontext. Theyshowedthatitisbettertousestages1and2jointly,
ratherthanusingstage2 as anindependentreplicationofstage1.
To gain a strong advantage with data-based weights, prior information is needed. One option
is to order the tests (Rubin et al. 2006), but with a large number of tests this can be challenging.
Another option is to group tests that are likely to have a signal, based on prior knowledge, as
follows:
1. Partition the tests into subsets 1,..., K, with the k’th group containing rk elements, en-
G G
suringthatr isat least10-20.
k
2. CalculatethesamplemeanY andvarianceS2 fortheteststatisticsin each group.
k k
3. Label thei’th test in group k, T . At best onlya fraction of theelements in each group will
ik
have a signal, hence we assume that for i = 1,...,r the distributionof the test statistics is
k
approximatedby amixturemodel
T (1 π )N(0,1)+π N(ξ ,1)
ik k k k
∼ −
or
2 2 2
T (1 π )χ (0)+π χ (ξ )
ik ∼ − k 1 k 1 k
whereξ isthesignalsizeforthosetestswithasignal inthek’th group. (Thisis an approx-
k
imationbecausethesignalislikelytovary across tests.)
6
4. Estimate(π ,ξ ) usingthemethodofmomentsestimator. Forthenormalmodelthisis
k k
2 2 2
π = Y /(Y +S 1), ξ = Y /π ,
k k k k − k k k
b
providedπ > 1/r ;othberwiseξ = 0.
k k k
b
b
For the χ2 model ξ2 is a root of the quadratic equation x2 bx + 1 = 0 where b = (S2
− k −
1)/(Y 1)+Y 5. Ifbothrootsare negative,ξ2 = 0;otherwise, π = (Y 1)/ξ2.
k − k −b k k k − k
b b
5. Foreachofthek groups,constructweightsw(ξ ). Then,toaccountfborexcessivevariability
k
intheweights,induced byvariabilityin ξ , smooththeweightsbytakinga
k b
b −1
w = 0.95w(ξ )+0.05K w(ξ ).
k k k
k
X
b b
Renorm weights if necessabry to ensure the weights sum to m. Each test in group k receives
theweightw .
k
Thisweightingbschemereliesondata-basedestimatorsoftheoptimalweights,butwithaparti-
tionofthedatasufficientlycrudetopreservethecontroloffamily-wiseerrorrate. Theapproachis
an exampleofthe“sieveprinciple”. Moreformallythisresultis statedinthefollowingTheorem.
Theorem. Let b = 1 √r . If m w = m, then (1) controls family-wise error at level
m m k k j=1 j R
α+O(b ). Proofisin tPheAppendixP.
m
b
Thisresultestablishescontroloffamily-wiseerrorat levelα, asymptotically,provided
√r
b = k k 0, as m .
m
r → → ∞
P k k
The inflation term in the error rate isPnear zero under a number of circumstances. Loosely speak-
ing, the requirement is that each group contains a sufficient number of elements to permit valid
estimation of ξ . For instance, if each group has the same number of elements r = r, then
k k
{ }
b 7
b = 1/√r, which goes to zero, provided the number of groups grows more slowly than the
m
numberoftestsperformed. Likewise,b 0 ifmax √r /min r 0.
m k k
→ { } { } →
Figure 3 illustrates how w(ξ ) varies with ξ and the samplevariances (weight is proportional
k k
to the diameter of the circle). Notice that weight increases as a function of the signal until it
b b
becomesfairly largeand thendeclines.
Results
To simulate a large scale study of association, we generate test statistics from m = 10,000 tests
with m1 = 50 and 100 tests having a signal (ξi > 0) and m0 = m m1 following the null
−
hypothesis. These choices were made to simulate the second stage of a two-stage genome-wide
association study, with about 1/3-1% of the initial SNPs tested at stage 2. In the proximity of a
causal SNP, clusters of tests tend to exhibit a signal. We simulate the data as if 5-10 additional
SNPs were in the proximity of each causal SNP. Thus, if 10-20 actual causal variants are present
in the genome, approximately 50 to 100 tests might be associated with the phenotype at varying
levelsofintensity.
Thesimulatedsignalstrengthsvaryover5levels(ξ1,...,ξ5) = ξ0 (1,1.5,2,2.5,3)withm1/5
×
realizations of each of the 5 levels of signals. The m simulated tests are grouped into categories
1,..., K with the groupings formed to convey various levels of informativeness. Let ξik be the
G G
signal of the i’th element in group k, ξ¯ be the mean in group k, and ξ¯ be the mean of the whole
.k ..
set,respectively. Theinformationin apriorgroupingissummarizedby theR2
¯ 2
(ξ ξ )
R2 = 1 k i ik − .k .
− (ξ ξ¯ )2
PkPi ik − ..
The 10,000 tests are grouped into 10 catePgorPies. We start the process by dividingthe m0 tests
that do not havea signalrandomlyinto 5 equal sized groupings, 1,..., 5. Nowm1 tests remain
G G
toconstitutetheremaining5categories, 6,..., 10. Wecreatetheidealpartitionofthesetestsby
G G
placing all tests with a common value of ξ in the same category. Next, to create more realistic
j
8
groupings,wemovesometestsfromcategories1-5into6-10andviceversa. Specifically,wemove
afractionp0 ofthem0 nullteststocategories6-10,anddistributethemevenly. Likewisewemove
a fraction p1 of the m1 tests with ξ > 0 to categories 1-5, and distribute them evenly. By varying
(p0,p1) we obtain various levels of informativeness of the groupings, reflecting priors of various
value.
To see the effect of including null loci in the same grouping as the SNPs with true effects, we
fix (ξ0 = 2,p1 = 0,m1 = 100) and vary p0. Setting p0 = 0.5 (0.1) increases the elements of
groups 6-10 to 1,010 (218), but only 20 are true alternatives. For p0 = 0.01, 0.1, 0.25, and 0.5 we
find a difference in power(weighted minustheunweighted procedure) of 14, 5, 0, and -3 percent,
respectively. So, forp0 > 0.25 thereisalossinpower, butitis relativelysmall.
Next we explore the effect of failing to place the true effects in the more promisingcategories
(6-10). To do so, we fix (ξ0 = 2,p0 = .1,m1 = 100) and vary p1. For p1 = 0.05. 0.1, 0.5, and
0.9, we find a difference in power of 7, 3, 2, -5 and -2 percent, respectively. Even when 90% of
the true alternatives are grouped with large numbers of nulls in groups 1-5, the loss in power is
relativelysmall. Anotherinterestingfeatureisthata50%swapleadstoagreaterlossinpowerthan
a 90% swap. The latter occurs because weights are approximately constant across groups when
the alternatives are scattered nearly at random. When half of the alternatives are in the promising
groups, these categories are up-weighted at the expense of the other categories. This balance can
lead toanet lossinpower, relativeto theunweightedtest.
Figure 4 displays the difference in power as a function of R2. The proportion of null tests in
cells1-5, and alternativetestsincells 6-10 varies: p0 [0.01 0.5]and p1 [0.01 0.95]. From
∈ − ∈ −
these simulations we see that, provided p0 < 0.5 and p1 > 0.1, the weighted method is generally
more powerful than the unweighted method (plot symbol “o”). Two exception occur; both have
R2 less than 2% of the variability in signal. For R2 near 0 the loss in power from poorly selected
groupings is modest. Deviations in p1 from ideal have a greater impact than deviations of p0
(plot symbol “⋆” vs. “+”). This asymmetry is expected because groups (1-5) contain many more
9
elements than groups 6-10. Consequently signals can be swamped by nulls in these groupings.
Finally we tried mixingthe variouslevelsof true alternativesξ0 (1,1.5,2,2.5,3)among groups
×
6-10and foundthat thishada negligibleeffect onthepower(resultsnot shown).
Discussion
Wholegenomeanalysishas generated much discussionaboutpower, theeffect ofmultipletesting
onpower,andvariousmultistageexperimentaldesigns(e.g.,Wangetal. 2006). Weinvestigatethe
performanceofaweightingschemethatallowsfortheinputofweakpriorinformation,intheform
ofgroupingsoftests,toimprovepowerinlargescaleinvestigationsofassociation. Themethodcan
be applied at any stage of an experiment. The beauty of the grouped-weighting concept is that it
islikelytolead toan increase inpower,providedmultipletestswithsignalsare clusteredtogether
inone(ormore)ofthegroups. Littlepowerislostwhen manygroupscontainnotruesignal. This
remarkable robustness is achieved because the procedure uses the observed test statistics in the
groupingtodeterminetheweight. Ifthereisnoapparentsignal,thegroupwillbedown-weighted.
The only restriction on the procedure is that the number of groups be small, relative to the total
numberoftestsperformed.
Using groupingsand weights to interpret the manytests conducted in a large scale association
studyhaspotential,regardlessofpowerlostwhenweightsarepoorlychosen. TypicallysomeSNPs
arefavoredduetoknowledgegleanedfromtheliteratureandpriorinvestigations. Whenseemingly
random SNPs produce smaller p-values than the favored candidates, one is baffled about how to
handle the situation. Moreover, it often happens that promisingcandidate SNPs do produce small
p-values, but these p-values might not be small enough to cross the significance threshold when
a Bonferroni correction is applied. After the huge investment of a whole genome scan it would
be foolhardy not to pursue both (i) SNPs that produce tiny p-values and (ii) SNPs that produce
respectablep-valuesthatwouldhavebeensignificanthadaformalweightingschemebeenutilized
10
