Misuses of Regression Approaches to ANOVA

14 Pages·2007·0.17 MB·English

DOCUMENT RESUME
TM 820 668
ED 222 522
Willson, Victor L.
AUTHOR
ANOVA and
Misuses of Regression Approaches to
TITLE
ANCOVA.
Feb 82
PUB DATE
Annual Meeting of the
14p.; Paper presented at the
NOTE
Association (Austin,
Southwest Bducatiofial Research
TX, February 11, 1982).
MF01/PC01 Plus Postage.
EDRS PRICE
of Variance; Error
*Analysis of Covariance; *Analysis
DESCRIPTORS
Mathematical Models;
of Measurement; Generalization;
Problems;
*Multiple Regression Analysis; *Research
Analysis;
Research Utilization; Stptistical
Statistical Significance
ABSTRACT
regression models in
The current state of usage of
and
designs is empirically examined,
analysis of variance (ANOVA)
presented.
statistical errors made in usage are
examples of several
all predictors
general linear model are that
The assumptions of the
fixed with no
of measurement and are
are known.without error
in the population, errors are
replication or sample variation;
with variance, and errors are
normally distributed independently
rules for construction of the
independent of all predictors. The
levels
to hold just as if the
ANOVA allow the expected mean squares
Covariance
randomly sampled. Analysis of
of each factor had been
design,
of xegression analysis with
(ANCOVA) combines the elements
The homogeneity of regression
albeit in a restricted manner.
point.
restriction from the design view
coefficients is a parameter
level is
associated with a given covariate
The regression weight
approaches to ANOVA and ANCOVA assume a
discussed. Most regression
oversight
design specifications. A major
fixed factor model under all
contributed to the current lack of
of expected mean squares has
from the design
of generalizability warranted
concern for the level
interaction models are examined.
specification. Aptitude treatment
(Author/CM)
***********************************************************************
best that can be made
Reproductions supplied by EDRS are the
from the original doCument.
***********************************************************************
4,
ci
(NJ
LCN
r
Misuses of Regression Approaches
to ANOVA and ANCOVA
Victor L. Willson
Texas ASIM University
U.S. DEPARTMENT OF EDUCATION
NATIONAL INSTITUTE OF EDUCATION
EDUCATIONAL RESOURCES INFORMATION
CENTER (ERIC)
XThis document has been reproduced as
received from the person or organization
originating it.
Mimi changes have been made to improve
reproduction quality.
Points of view or opinions stated in this docu-
NIE
ment do not necessarily represent official
position or Policy
"PERMISSION TO REPRODUCE THIS
MATERIAL HAS BEEN GRANTED BY
L
1.10/1
V
TO THE EDUCATIONAL RESOURCES
INFORMATION CENTER (ERIC)."
Paper presented at the Southwest Educational Research Association
Annual Meeting, Austin, Texas, February 11, 1982.
1
ANOVA and ANCOVA
Misuses of Regression Approaches to
of education graduate students
The instruction of several generations
logi-
based on Lindquist (1953) and his
in research design statistics was
(1968), Glass and Stanley (1970)
Winer (1962, 1971), Kirk
cal succession:
(ANOVA) using Fisherian
All stressed analysis of variance
and others.
eMPhasis on re-§reS--
squares.' There waS -ii-dry little
partition of sums of
(1973) and
of Ward and Jennings'
sion approaches until the appearance
which use regression models ex-
Kerlinger and Pedhazur's (1973) texts,
increased use of regres-
These tests have apparently promoted
clusivel).
,
Willson (1980)
in the last several years.
sion models in ANOVA situations
Journal,
American Educational Research
reviewed ten years' research in the
Since
of regression approaches.
from 1969 to 1978, and found little use
extensively used, as will be re-
1978, however the technique has been
to examine empirically the
It is the purpose of this paper
ported here.
and to list
regression models in ANOVA designs
current state of usage of
made in this usage.
by example several statistical errors
of the general
It is worth reviewing assumptions
Regression Assumptions.
will be referenced in light of cur-
linear model, for these assumptions
assumption as
Darlington' (1968) has summarized the
rent practice.
follows:
without error of measurement;
All predictors Xi are known
1.
with no replication or sample
All predictors Xi are fixed
2.
variation;
2
normally distributed independently
In the population errors are
3.
with variancelre;
Errors are independent of all Xi.
4.
and Glass
CornefieldandTukey (1956) and Millman
Design Assumptions.
of the ANOVA table for the
(1967) presented the rules for construction
drawn_with._equal, probability
design in which a levels of factor A are
levels,
possible
from N
possible levels, b levels of'Factor B
from N
B
A
Random factors
the design.
and sd forth for each factor in
c from N
Nr
In
Nci, and fixed factors defined for r =
are defined for any q <
at random from
of the design n elements are *awn
. q
each cell abc
.
.
This is the urn
in the population of elements.
a possible Nabc...q
The expected mean
(1956; p. 917):
sampling model of Cornfield and Tukey
each factor had been
this model just as if the levels of
squares hold for
randomly sampled.
elements of
Analysis-of covariance combines the
ANCOVA Assumptions.
From
albeit in a restrictive manner.
regresSion analysis with design,
without error,
of view the covariate must be known
a regression point
-1 in so-
level (these are coded with 1, 0, or
as must the treatment
homogeneity of re-
The so-called assumption of
called dummy coding).
the
just a parameter restriction from
gression coefficients is really
effect at
There could be a different covariate
design point of view.
either fixed
The covariate can be considered
each level of a covariate.
1000) or
100 mg increments from 0 to
(drug dosage maintenance levels in
sampled
increments from 0 to 1000, randomly
random (drug levels in 10 mg
3
associated with
The regression weight
groups).
in stratified 100 mg
commonly
for each level; more
level could be different
a given covariate
levels, hence the homo-
identical for all covariate
it is assumed to be
form of a general
which is merely a restrictive
geneity of coefficients,
1973).
linear model (Ward & Jennings,
interaction term cap
covariate treatment
_
The addition of a so-called
_
X(C-1)
interaction with (T-1)
terms as addition of an
be thought of in design
and C the
T is the number of treatments
degrees of freedom where
most analyses to
This term is reduced in
number of covariate levels.
Each group has a dif-
parameter perlgroup.
1-1 degrees of freedom, one
commonly encountered in
This model is most
ferent regression slope.
of
aptitude-treatment interaction models
edUcational research in the
(Cronbach & Snow, 1977).
Cronbach and his co-workers
most
The approach to ANCOVA taken in
for ANCOVA.
Expected Mean Squares
covariates as random factors
Kirk, 1968) is to treat
texts (Winer, 1971;
for the usual ANOVA.
from sums of squares
whose variances are removed
conditional on the
squares for ANOVA are
In effect the expected-mean
(Winer, 1971; p. 770).
so written
covariates and the expectations are
after fit of the
that the residual mean square
What is quite clear is
the covariate or co=
appropriate error term for
full model is not the
theory have
Those who use regression
model.
variates under the usual
with and without
difference between mean squares
rather casually used the
covariates'
residual as the test of
covariate divided by mean square
expectation table for a
Table 1 shows the mean square
significance.
4
assumptions, most
factor design under usual
single covariate-single
the
treatment effects are conditional on
important of which is that
covariate adjustment.
Insert Table 1 About Het'e
It was noted earlier that regres-
,Unreliability of Regression Variables.
error.-
the predictors to-be known:without
--.
sion analysis modelsassume
Glass,
violated.
present the assumption is
When error of measurement is
violatioh
reviewed research on this
Peckham and Sanders (1972) have
effects of unrelia-
Rogosa (1977) has examined the
for ANOVA models.
interval
ANCOVA models for confidence
bility on interaction terms in
technique.
estimation using the Johnson-Neyman
results
that unreliability does affect
Glass et al (1972) concluded
adjust
They recommend procedures to
of ANCOVA in unpredictable ways.
the F-statistic.
literatures on unreliable predictors
Rogosa (1977) reviewed the
(non-homogeneous covariate
(covariates) for within-group regression
rate may be changed.
Power is reduced and the Type I error
slopes).
(covariates), and
complex for two predictors
The situation is even more
be even more severe.
distortion of error rates can
Glass et al (1972) indicate
Variables.
Stochastic Regression/Covariate
not fixed
when a covariate or predictor is
there is no serious difficulty
for ANCOVA models
Rogosa (1977) found no effect
random,
but
regression slopes.
with within-group, nonhomogeneous
0
5
Method
research journals
from three educational
All data based studies
Educa-
The journals were American
1979 to 1981.
were surveyed for years
Psychology, and Educational
Journal of Educational
tional Research Journal,
designs with
Studies using ANOVA-type
Evaluation and Policy Analysis.
Expected mean square
analyses were examined.
regression-type statistical
using the proced4res
where.possible for each. design
tables were constructed
empirical results
comparison was made with
A
(1967).
of Millman and Glass
ANCOVA model used
Assumptions of the ANOVA and
reported for each table.
Of special
noted.
actual practice and discrepancies
were compared with
of regression slopes
misspecification, nohhomogeneity
interest were model
Model misspecificat:on is
unreliable covariates.
for covariates, and
predictor as fixed (or
specification of a factor or
defined as incorrect
clearly
the author's later generalization
random) when common practice or
slopes
Nonhomogeneity of regression
specification.
point to the opposite
interaction which was
of covariate-treatment
refers to the possibility
to the presence of un-
Unreliability of covariates refers
never tested.
socioeconomic
intelligence, achievement, or
reliable covariates, typically
treatment groups)
Differential reliability may exist across
measures.
which was not.examined.
Results
there were 29
published between 1979 and 1981
From all articles
ANOVA
regression apProach to
(none in EEPA) that used a
in AERJ and JEP
ANOVAs that
straightforward fixed factor
Six of these were
or ANCOVA.
eleven treated
Of the remaining 23
assumptions:
met all usual ANOVA
13
7
factors as fixed that are either usually treated as random or treated
//
//
about the population from whic
a factor as fixed and then generalized
Factors thus treated included teachers, classrooms,
it was drawn.
students, and school buildings.
tests of homogeneity of regression
Eight of the studies made no
slopes in ANCOVA models.- Since many of the remaining studie mere
test/vas the major
aptitute-treatment studies in which this interaction
thrust of the study, the failure to test was significa7t in the remainder.
9NCOVA tested for
It should be noted that few studies using straight
homogeneity either.
Insert Table 2 About Her,/
d covariates known to be un-
Of the twenty-three studies eighteen
Fourteen made no attempt to :50-ist for unequal reliabilities
reliable.
use of generalizability theory
extens%6e
across groups, which four made
to explore facet generalizability.
In.all cases where mixed mØls should have been used (but one) the
In some cases it
residual was used as the error term for all F-tests.
tables under the appro-
was possible to reconstruct expected mean square
would
s apparent that numerous F statistics
priate mixed rodel.
It
change from significande to nonsignificance or vice-versa, changing
Even this first step reanalysis did
interpretations in some instances.
quasi-F's under
not pursue hierarchical pooling procedures on construct
It is possible to say that some studies need reinterpretation.
all models.
7
Discussion
and ANCOVA published in two
Most regression approaches to ANOVA
three years assume a fixed factor
major education journals in the last
This procedure seems unnecessarily
model under all design specifications.
using a computer package
thoughtless, although the ease of computation
GLMIliay have contributed_ to it._ When one_
such as SAS with its PROC
design texts (Kerlinger
most commonly used regression-approach
reviews the
is no
1973; Cohen & Cohen, 1975) there
& Pedhazur, 1973; Ward & Jennin9s,
This major oversight clearly
mention of expected mean squares in them.
for the level of general-
has contributed to the current'lack of concern
Cohen and
specification of a study.
izability warranted from the design
fixed, however.
Cohen (1975) do attempt to place all designs as
studies involved aptitude-
It is interesting that so many of the
ATI models always involve.covariates, the
treatment interaction (ATI).
The interactions form the major
aptitudes, and factors, the treatments.
study with the exception of Martin,
tests of interest, and yet in not one
model specifically examined.
Veldman, and Anderson (1980) was a mixed
these studies the ATI's expected
Given the random factors that accompany
complex interactions of the random
mean-squares surely involve more
This aspect needs immediate address,
factors with aptitude covariate.
today.
given the high inierest in ATI research
8
Table 1
ANCOVA,
Expected Mean Square Table for
(Fixed or Random)
One Covariate, One Factor
Variance
(No C.ovariate)
Model II
(Covariate)
Model
I
6e,x
A
GAR
(re
A
jjx
-cr-)2c
Source
of
Variation
1
n
A
1
B2
1
X
1
Eh(
1
1
E ,x
(E)tcji
yii
yii = 1)+0(i +
+ eii
c)(x1 -px)
+
=
Model
yii -
p+oei +
(EIX)ij
c3(x1 -px) =
X)]2
( y,
= [CoARR
(fe - 1)
cs;
fe
(1 -
ew2)
=
for error
fe = degrees of freedom