Lecture 6 - ANOVA

33 Pages·2009·0.12 MB·English

ANOVA
Dr. Frank Wood
Frank Wood, [email protected] Linear Regression Models Lecture 6, Slide 1
ANOVA
• ANOVA is nothing new but is instead a way of
organizing the parts of linear regression so as
to make easy inference recipes.
• Will return to ANOVA when discussing
multiple regression and other types of linear
statistical models.
Frank Wood, [email protected] Linear Regression Models Lecture 6, Slide 2
Partitioning Total Sum of Squares
• “The ANOVA approach is based on the
partitioning of sums of squares and degrees
of freedom associated with the response
variable Y”
• We start with the observed deviations of Y
i
¯
around the observed mean Y
¯
Y − Y
i
Frank Wood, [email protected] Linear Regression Models Lecture 6, Slide 3
Partitioning of Total Deviations
SSR
SSTO SSE
Frank Wood, [email protected] Linear Regression Models Lecture 6, Slide 4
Measure of Total Variation
• The measure of total variation is denoted by
¯
2
SST O = (Y − Y )
i
(cid:1)
• SSTO stands for total sum of squares
• If all Y ’s are the same, SSTO = 0
i
• The greater the variation of the Y ’s the
i
greater SSTO
Frank Wood, [email protected] Linear Regression Models Lecture 6, Slide 5
Variation after predictor effect
• The measure of variation of the Y ’s that is still
i
present when the predictor variable X is taken
into account is the sum of the squared
deviations
ˆ
2
SSE = (Y − Y )
i i
(cid:1)
• SSE denotes error sum of squares
Frank Wood, [email protected] Linear Regression Models Lecture 6, Slide 6
Regression Sum of Squares
• The difference between SSTO and SSE is
SSR
ˆ ¯
2
SSR = (Y − Y )
i
(cid:1)
• SSR stands for regression sum of squares
Frank Wood, [email protected] Linear Regression Models Lecture 6, Slide 7
Partitioning of Sum of Squares
¯ ˆ ¯ ˆ
Y − Y = Y − Y + Y − Y
i i i i
Total Deviation Deviation
deviation of fitted around
regression fitted
value regression
around mean line
Frank Wood, [email protected] Linear Regression Models Lecture 6, Slide 8
Remarkable Property
• The sums of the same deviations squared
has the same property!
¯ ˆ ¯ ˆ
2 2 2
(Y − Y ) = (Y − Y ) + (Y − Y )
i i i i
or SSTO = SSR + SSE
• Proof:
Frank Wood, [email protected] Linear Regression Models Lecture 6, Slide 9
Remarkable Property
¯ ˆ ¯ ˆ
2 2 2
• Proof: (Y − Y ) = (Y − Y ) + (Y − Y )
i i i i
¯ 2 ˆ ¯ ˆ 2
(Y − Y ) = [(Y − Y ) + (Y − Y )]
i (cid:2) i i i
ˆ ¯ 2 ˆ 2 ˆ ¯ ˆ
= [(Y − Y ) + (Y − Y ) + 2(Y − Y )(Y − Y )]
(cid:2) i i i i i i
ˆ ¯ 2 ˆ 2 ˆ ¯ ˆ
= (Y − Y ) + (Y − Y ) + 2 (Y − Y )(Y − Y )
(cid:2) i (cid:2) i i (cid:2) i i i
but
ˆ ¯ ˆ ˆ ˆ ¯ ˆ
(Y − Y )(Y − Y ) = Y (Y − Y ) − Y (Y − Y ) = 0
i i i i i i i i
(cid:1) (cid:1) (cid:1)
By properties previously demonstrated
Frank Wood, [email protected] Linear Regression Models Lecture 6, Slide 10