Factorial and Unbalanced ANOVA

70 Pages·2017·0.31 MB·English

by  Nathaniel E. Helwig

Factorial and Unbalanced Analysis of Variance
Nathaniel E. Helwig
AssistantProfessorofPsychologyandStatistics
UniversityofMinnesota(TwinCities)
Updated04-Jan-2017
NathanielE.Helwig (UofMinnesota) Factorial&UnbalancedAnalysisofVariance Updated04-Jan-2017 : Slide1
Copyright
Copyright © 2017 by Nathaniel E. Helwig
NathanielE.Helwig (UofMinnesota) Factorial&UnbalancedAnalysisofVariance Updated04-Jan-2017 : Slide2
Outline of Notes
1) Balanced Two-Way ANOVA: 2) Balanced Three-Way ANOVA:
Model Form & Assumptions Model Form & Estimation
Least-Squares Estimation Hypertension Example (pt 3)
Basic Inference
Hypertension Example (pt 1)
3) Unbalanced ANOVA Models:
Multiple Comparisons
Overview of problem
Hypertension Example (pt 2)
Types of sums-of-squares
Hypertension Example (pt 4)
NathanielE.Helwig (UofMinnesota) Factorial&UnbalancedAnalysisofVariance Updated04-Jan-2017 : Slide3
BalancedTwo-WayANOVA
Balanced Two-Way ANOVA
NathanielE.Helwig (UofMinnesota) Factorial&UnbalancedAnalysisofVariance Updated04-Jan-2017 : Slide4
BalancedTwo-WayANOVA ModelForm&Assumptions
Two-Way ANOVA Model (cell means form)
The Two-Way Analysis of Variance (ANOVA) model has the form
y = µ +e
ijk jk ijk
for i ∈ {1,...,n }, j ∈ {1,...,a}, and k ∈ {1,...,b} where
jk
y ∈ R is real-valued response for i-th subject in factor cell (j,k)
ijk
µ ∈ R is real-valued population mean for factor cell (j,k)
jk
e ∼iid N(0,σ2) is a Gaussian error term
ijk
n is number of subjects in cell (j,k) and n = (cid:80)a (cid:80)b n
jk j=1 k=1 jk
(note: n = n ∀j,k in balanced two-way ANOVA)
jk ∗
a and b are number of levels for first and second factors
Implies that y i∼nd N(µ ,σ2).
ijk jk
NathanielE.Helwig (UofMinnesota) Factorial&UnbalancedAnalysisofVariance Updated04-Jan-2017 : Slide5
BalancedTwo-WayANOVA ModelForm&Assumptions
Two-Way ANOVA Model (effect coding)
Using effect coding, the mean for factor cell (j,k) has the form
µ = µ+α +β +(αβ)
jk j k jk
for j ∈ {1,...,a} and k ∈ {1,...,b} where
µ is overall population mean
α is main effect of first factor such that (cid:80)a α = 0
j j=1 j
β is main effect of second factor such that (cid:80)b β = 0
k k=1 k
(αβ) is interaction effect such that (cid:80)a (αβ) = 0 ∀k and
jk j=1 jk
(cid:80)b (αβ) = 0 ∀j
k=1 jk
Set (αβ) = 0 ∀j,k to fit an additive model.
jk
NathanielE.Helwig (UofMinnesota) Factorial&UnbalancedAnalysisofVariance Updated04-Jan-2017 : Slide6
BalancedTwo-WayANOVA ModelForm&Assumptions
Two-Way ANOVA Model (matrix form)
In matrix form, the two-way ANOVA model is y = Xb+e where
1 x11···x1(a−1) z11···z1(b−1) x11z11···x1(a−1)z11 ··· x11z1(b−1)···x1(a−1)z1(b−1)
1 x21···x2(a−1) z21···z2(b−1) x21z21···x2(a−1)z21 ··· x21z2(b−1)···x2(a−1)z2(b−1)
X=1... x3...1·.·.·.x3(a−...1) z3...1·.·.·.z3(b−...1) x31z...31··.·.x.3(a−1...)z31 ·.·..· x31z3(b−...1)·.·.·.x3(a−1)...z3(b−1)
1 xn1···xn(a−1) zn1···zn(b−1) xn1zn1···xn(a−1)zn1 ··· xn1zn(b−1)···xn(a−1)zn(b−1)
b= (cid:0)µ α1···αa−1 β1···βb−1 (αβ)11 ··· (αβ)(a−1)1 ··· (αβ)1(b−1) ··· (αβ)(a−1)(b−1)(cid:1)(cid:48)
where X has 1+(a−1)+(b−1)+(a−1)(b−1) = ab columns

1 if i-th observation is in j-th level of first factor

x = −1 if i-th observation is in a-th level of first factor
ij
 0 otherwise

1 if i-th observation is in k-th level of second factor

z = −1 if i-th observation is in b-th level of second factor
ik
 0 otherwise
i ∈ {1,...,n} and additional subscripts on y and e are dropped
Implies that y ∼ N(Xb,σ2I ).
n
NathanielE.Helwig (UofMinnesota) Factorial&UnbalancedAnalysisofVariance Updated04-Jan-2017 : Slide7
BalancedTwo-WayANOVA ModelForm&Assumptions
Two-Way ANOVA Model (assumptions)
The fundamental assumptions of the two-way ANOVA model are:
1 xij, zik and yi are observed random variables (known constants)
2 ei ∼iid N(0,σ2) is an unobserved random variable
3 µjk are unknown constants
4 (yi|xij,zik) i∼nd N(µjk,σ2)
note: homogeneity of variance
Interpretation of µ depends on model form
jk
Additive: µ = µ+α +β
jk j k
Interaction: µ = µ+α +β +(αβ)
jk j k jk
NathanielE.Helwig (UofMinnesota) Factorial&UnbalancedAnalysisofVariance Updated04-Jan-2017 : Slide8
BalancedTwo-WayANOVA Least-SquaresEstimation
Ordinary Least-Squares
We want to find the effect estimates (i.e., µˆ, αˆ , βˆ , and (αˆβ) terms)
j k jk
that minimize the ordinary least squares criterion
b a njk
(cid:88)(cid:88)(cid:88)
SSE = (y −µ−α −β −(αβ) )2
ijk j k jk
k=1j=1 i=1
If n = n ∀j,k the least-squares estimates have the form
jk ∗
µˆ= 1 (cid:80)b (cid:80)a (cid:80)n∗ y =y¯
abn k=1 j=1 i=1 ijk ··
∗
(cid:18) (cid:19)
αˆ = 1 (cid:80)b (cid:80)n∗ y −µˆ=y¯ −y¯
j bn k=1 i=1 ijk j· ··
∗
(cid:18) (cid:19)
βˆ = 1 (cid:80)a (cid:80)n∗ y −µˆ=y¯ −y¯
k an j=1 i=1 ijk ·k ··
∗
(cid:18) (cid:19)
(αˆβ) = 1 (cid:80)n∗ y −µˆ−αˆ −βˆ =y¯ −y¯ −y¯ +y¯
jk n i=1 ijk j k jk j· ·k ··
∗
which implies that yˆijk = y¯jk for all (i,j,k). Proof
NathanielE.Helwig (UofMinnesota) Factorial&UnbalancedAnalysisofVariance Updated04-Jan-2017 : Slide9
BalancedTwo-WayANOVA Least-SquaresEstimation
Fitted Values and Residuals
Form of fitted values depends on fit model:
Additive: µˆ = y¯ +y¯ −y¯
jk j· ·k ··
Interaction: µˆ = y¯
jk jk
Residuals have the form
eˆ = y −µˆ
ijk ijk jk
where form of µˆ depends on fit model (additive versus interaction).
jk
NathanielE.Helwig (UofMinnesota) Factorial&UnbalancedAnalysisofVariance Updated04-Jan-2017 : Slide10