ANOVA Model and Matrix Computations

14 Pages·2001·0.05 MB·English

ANOVA
Model and Matrix Computations
Notation
The following notation is used throughout this chapter unless otherwise stated:
N Number of cases
F Number of factors
CN Number of covariates
k Number of levels of factor i
i
Y Value of the dependent variable for case k
k
Z Value of the jth covariate for case k
jk
w Weight for case k
k
W Sum of weights of all cases
The Model
A linear model with covariates can be written in matrix notation as
Y=Xb +ZC+e (1)
where
Y N· 1 vector of values of the dependent variable
1 6
X Design matrix N· p of rank q< p
1 6
b Vector of parameters p· 1
1 6
Z Matrix of covariates N· CN
1 6
C Vector of covariate coefficients CN· 1
1 6
e Vector of error terms N· 1
1
2 ANOVA
Constraints
To reparametrize equation (1) to a full rank model, a set of non-estimable
conditions is needed. The constraint imposed on non-regression models is that all
parameters involving level 1 of any factor are set to zero.
For regression model, the constraints are that the analysis of variance
parameters estimates for each main effect and each order of interactions sum to
zero. The interaction must also sum to zero over each level of subscripts.
For a standard two way ANOVA model with the main effects a and b , and
i j
interaction parameter g , the constraints can be expressed as
ij
a =b =g =g =0 non- regression
1 1 1j i1

a • =b •g= ig• = •j =0 regression
•
where indicates summation.
Computation of Matrices
X¢X
Non-regression Model
The X¢X matrix contains the sum of weights of the cases that contain a particular
combination of parameters. All parameters that involve level 1 of any of the factors
are excluded from the matrix. For a two-way design with k =2 and k =3, the
1 2
symmetric matrix would look like the following:
a b b g g
2 2 3 22 23
a 2 N2• N22 N23 N22 N23
b 2 N•2 0 N22 0
b 3 N•3 0 N23
g N 0
22 22
g N
23 23
The elements Ni• or N•j on the diagonal are the sums of weights of cases that
have level i of a or level j of b. Off-diagonal elements are sums of weights of cases
cross-classified by parameter combinations. Thus, N•3 is the sum of weights of
ANOVA 3
cases in level 3 of main effect b , while N is the sum of weights of cases with
3 22
a and b .
2 2
Regression Model
A row of the design matrix X is formed for each case. The row is generated as
follows:
If a case belongs to one of the 2 to k levels of factor i, a code of 1 is placed in
i
the column corresponding to the level and 0 in all other k - 1 columns associated
i
with factor i. If the case belongs in the first level of factor i, - 1 is placed in all the
k - 1 columns associated with factor i. This is repeated for each factor. The entries
i
for the interaction terms are obtained as products of the entries in the corresponding
m1ai6n effect columns. This vector of dummy variables for a case will be denoted as
d i ,i=1,K,NC, where NC is the number of columns in the reparametrized
design matrix. Af1te6r t1he6 vector d is generated for case k, the ijth cell of X¢X is
incremented by d i d j w , where i=1,K,NC and j‡ i.
k
Checking and Adjustment for the Mean
After all cases have been processed, the diagonal entries of X¢X are examined.
Rows and columns corresponding to zero diagonals are deleted and the number of
levels of a factor is reduced accordingly. If a factor has only one level, the analysis
will be terminated with a message. If the first specified level of a factor is missing,
the first non-empty level will be deleted from the matrix for non-regression model.
For regression designs, the first level cannot be missing. All entries of X¢X are
subsequently adjusted for means.
The highest order of interactions in the model can be selected. This will affect
the generation of X¢X. If none of these options is chosen, the program will
generate the highest order of interactions allowed by the number of factors. If sub-
matrices corresponding to main effects or interactions in the reparametrized model
are not of full rank, a message is printed and the order of the model is reduced
accordingly.
Cross-Product Matrices for Continuous Variables
Provisional means algorithm are used to compute the adjusted-for-the-means cross-
product matrices.
4 ANOVA
Matrix of Covariates Z¢Z
The covariance of covariates m and l after case k has been processed is
(cid:25) (cid:28)(cid:25) (cid:28)
(cid:26) k (cid:29)(cid:26) k (cid:29)
˚ ˚
wk(cid:27)(cid:26)WkZlk - wjZlj(cid:30)(cid:29)(cid:27)(cid:26)WkZmk - wjZmj(cid:30)(cid:29)
0 5 0 5
Z(cid:135)Z k =Z(cid:135)Z k-1 + j=1 j=1
ml ml
WkWk-1
where W is the sum of weights of the first k cases.
k
The Vector Z¢Y
The covariance between the mth covariate and the dependent variable after case k
has been processed is
(cid:25) (cid:28)(cid:25) (cid:28)
(cid:26)(cid:26) (cid:229) k (cid:29)(cid:29)(cid:26)(cid:26) (cid:229) k (cid:29)(cid:29)
w (cid:27)W Y - w Y (cid:30)(cid:27)W Z - w Z (cid:30)
0 5 0 5 k k k j j k mk j mj
Z¢Y k =Z¢Y k- 1+ j=1 j=1
m m W W
k k- 1
The Scalar Y¢Y
The corrected sum of squares for the dependent variable after case k has been
processed is
(cid:25) (cid:28)
2
(cid:26) k (cid:29)
(cid:229)
wk(cid:27)(cid:26)WkYk - wjYj(cid:30)(cid:29)
1 6 1 6
j=1
Y¢Y k = Y¢ Y k- +1
WkWk- 1
The Vector X¢Y
X¢Y is a vector with NC rows. The ith element is
N
(cid:229)
X¢Y= Y w d ,
i k k k
k=1
ANOVA 5
where, for non-regression model, d =1 if case k has the factor combination in
k 1 6
column i of X¢X; d =0 otherwise. For regression model, d =d i , where
1 6 k k
d i is the dummy variable for column i of case k. The final entries are adjusted for
the mean.
Matrix X¢Z
The (i, m)th entry is
N
(cid:229)
X¢Z = Z w d
im mk k k
k=1
where d has been defined previously. The final entries are adjusted for the mean.
k
Computation of ANOVA Sum of Squares
The full rank model with covariates
Y=Xb +ZC+e
can also be expressed as
Y=X b +X b +ZC+e
k k m m
where X and b are partitioned as
(cid:31) "
b #
X= X |X and b= ! k $.
k m
b
m
6 ANOVA
The normal equations are then
(cid:31) "(cid:31) " (cid:31) "
Z¢Z Z¢ X Z¢ X # C$ # Z¢Y #
! X¢kZ X¢kXkk X¢ kXmm$##! b$$k $##=! X¢kY$## (2)
X¢ Z X¢ X X¢ X b X¢ Y
m m k m m m m
The normal equations for any reduced model can be obtained by excluding those
entries from equation (2) corresponding to terms that do not appear in the reduced
model.
Thus, for the model excluding b ,
m
Y=X b +ZC+e
k k
the solution to the normal equation is:
(cid:31) " (cid:31) " (cid:31) "
C~ # Z¢Z Z¢ X #- 1 Z¢Y #
!b~ $=!X¢ Z X¢ Xk $ !X¢ Y$ (3)
k k k k k
The sum of squares due to fitting the complete model (explained SS) is
(cid:31) "
Z¢Y #
1 6
$ $ $ # $ $ $
R C,bk,bm = C¢,b¢k,b¢ m ! X¢kY$#=C¢Z¢ Y+ b¢ kX¢ k+Y ¢ bm¢ XmY
X¢ Y
m
For the reduced model, it is
(cid:31) "
1 6 ~ ~ Z¢Y # ~ ~
R C,b = C¢,b¢ ! $=C¢Z¢ Y+ b¢ X¢ Y
k k X¢ Y k k
k
The residual 1(unexplain6ed) sum of squares for the complete model is
RSS =Y¢Y- R C,b ,b and similarly for the reduced model. The total sum of
k m
squares is Y¢Y. The reduction in the sum of squares due t1o includin6g bm in a
model that already includes b and C will be denoted as R b |C,b . This can
k m k
also be expressed as
1 6 1 6 1 6
R b |C,b = R C,b ,b - R C,b
m k k m k
ANOVA 7
1 6
There are several ways to compute R b |C,b . The sum of squares due to the
m k
full model, as well as the sum of squares due to the reduced model, can each be
calculated, and the difference obtained (Method 1).
1 6
R b |C,b =C$¢Z¢ Y+b$¢ X¢ Y+b$¢ X¢ Y- C~¢Z¢ Y- b~¢ X¢ Y
m k k k m m k k
A sometimes computationally more efficient procedure is to calculate
1 6
R b |C,b =b$¢ T- 1b$
m k m m m
$
where b are the estimates obtained from fitting the full model and T is the
m m
partition of the inverse matrix corresponding to b (Method 2).
m
(cid:31) " (cid:31) "
Z¢Z Z ¢ X Z¢ X #- 1 T T T #
k m c ck cm
# #
! X¢kZ X¢kXk X¢ kXm$# =! Tkc Tk Tkm$#
X¢ Z X¢ X X¢ X T T T
m m k m m mc mk m
Model and Options
Notation
Let b be partitioned as
(cid:31) "
m #
1
#
M
(cid:31) " #
M# m #
b=! $= F #
D d #
1
M #
! $#
dF- 1
8 ANOVA
where
M Vector of main effect coefficients
m Vector of coefficients for main effect i
i
16
i M excluding m
m i
Mi* M including only m1 through mi- 1
D Vector of interaction coefficients
d Vector of kth order interaction coefficients
k
d Vector of coefficients for the ith of the kth order interactions
k
i
1 6
k D excluding d
D k
Dk* D including only d1 through dk- 1
16
i d excluding d
dk k ki
C Vector of covariate coefficients
ci Covariate coefficient
16
i C excluding c
C i
Ci* C including only c1 through ci- 1
Models
Different types of sums of squares can be calculated in ANOVA.
Sum of Squares for Type of Effects
Covariates Main Effects Interactions
1 6 1 6
4 9
Experimental and R C R M|C R d |C,M,Dk*
Hierarchical k
1 6 1 6
4 9
Covariates with R C,M R C,M R d |C,M,Dk*
Main Effects k
1 6 1 6
4 9
Covariates after R C|M R M R d |C,M,Dk*
Main Effects k
1 6 1 6
4 9
Regression R C|M,D R M|C,D R d |C,M,Dk*
k
ANOVA 9
All sums of squar2es1 are c6a7lculated as described in the introduction. Reductions in
sums of squares R A|B are computed using Method 1. Since all cross-product
matrices have been corrected for the mean, all sums of squares are adjusted for the
mean.
Sums of Squares Within Effects
Covariates Main Effects Interactions
(cid:25) 16(cid:28) (cid:25) 16(cid:28) (cid:25) 16(cid:28)
DExepfaeurilmt ental R(cid:27)ci|Ci (cid:30) R(cid:27)mi|C,M i (cid:30) R(cid:27)(cid:26)dki|C,M,Dk* ,dki (cid:30)(cid:29)
(cid:25) (cid:28) (cid:25) (cid:28)
16 16
Covariates with (cid:27) i (cid:30) (cid:27) i (cid:30) same as default
R c |M,C R m |C,M
Main Effects i i
(cid:25) (cid:28) (cid:25) (cid:28)
16 16
Covariates after (cid:27) i (cid:30) (cid:27) i (cid:30) same as default
R c |M,C R m |M
Main Effects i i
(cid:25) 16 (cid:28) (cid:25) 16 (cid:28) (cid:25) 3 8(cid:28)
Regression R(cid:27)ci|M,Ci ,D(cid:30) R(cid:27)mi|M i ,C,D(cid:30) R(cid:27)(cid:26)dki|C,M,D ki (cid:30)(cid:29)
4 9 4 9
Hierarchical R c |Ci* R m |C,Mi* same as default
i i
4 9 4 9
Hierarchical and R c |Ci* ,M R m |Mi* same as default
Covariates with i i
Main Effects or
Hierarchical and
Covariates after
Main Effects
Reductions in sums of squares are calculated using Method 2, except for
specifications involving the Hierarchical approach. For these, Method 1 is used. All
sums of squares are adjusted for the mean.
Degrees of Freedom
Main Effects
F 1 6
(cid:229)
df = k - 1
M i
i=1
10 ANOVA
Main Effect i
1 6
k - 1
i
Covariates
df =CN
c
Covariate i
1
Interactions d
r
df = number of linearly independent columns corresponding to interaction d in
r r
X¢X
Interactions d
r
i
df = number of independent columns corresponding to interaction d in X¢X
r
i
Model
F- 1
(cid:229)
df =df +df + df
Model M c r
r=1