Lec 3: Model Adequacy Checking

30 Pages·2011·0.77 MB·English

by  Ying Li

Lec 3: Model Adequacy Checking
Ying Li
November 16, 2011
YingLi Lec3:ModelAdequacyChecking
Model validation
Model validation is a very important step in the model
building procedure. (one of the most overlooked)
A high R2 value does not guarantee that the model fits the
data well.
Use of a model that does not fit the data well can not provide
good answers to the underlying scientific questions.
YingLi Lec3:ModelAdequacyChecking
An interesting example: Ascombe dataset
1011 l l 89 l l l l l l l
y1 789 l l l l l l y2 67 l l
6 l 5 l
5 l 4
4 l 3 l
4 6 8 10 12 14 4 6 8 10 12 14
x1 x2
l l
12 12
10 10
y3 l y4 ll
68 l l l l l l l l l 68 llllllll
4 6 8 10 12 14 8 10 12 14 16 18
x3 x4
YingLi Lec3:ModelAdequacyChecking
1011 l l 89 l l l l l l l
y1 456789 l l l l l l Rl_sabql==u30a..r05e00=000l11.667 y2 34567 l l l l R_sabq==u30a..r05e00=00011.667
4 6 8 10 12 14 4 6 8 10 12 14
x1 x2
l l
12 a=3.0001 12
b=0.5001
10 R_square=0.667 10
y3 l y4 ll
68 l l l l l l l l l 68 llllllll R_sabq==u30a..05re00=00011.667
4 6 8 10 12 14 8 10 12 14 16 18
x3 x4
YingLi Lec3:ModelAdequacyChecking
Main Tool: Graphical residual Analysis
Different types of plots of residuals (histgram, Plot of residuals
in time sequence, plot of residuals versus fitted values) provide
information on the adequacy of different aspects of the model.
Graphical methods have an advantage over numerical
methods in model validation
graphical methods: a broad range of complex aspects
numerical methods: narrowly focused on a particular aspect(a
number)
YingLi Lec3:ModelAdequacyChecking
Why use residuals?
If the model fit to the data were correct, the residuals would
approximate the random errors.
If the residuals appear to behave as the assumptions of the
error it suggests the model fit the data well.
Otherwise the model fits the data poorly.(non-normality,
dependency, heteroscedasticity).
YingLi Lec3:ModelAdequacyChecking
Two concepts
Homogeneity (Homoscedasticity) : In statistics, a
sequence or a vector of random variables is homoscedastic if
all random variables in the sequence or vector have the same
finite variance. This is also known as homogeneity of variance.
Heteroscedasticity: a collection of random variables is
heteroscedastic, or heteroscedastic, if there are
sub-populations that have different variabilities than others
YingLi Lec3:ModelAdequacyChecking
The assumptions for ANOVA
y = µ+τ +(cid:15)
ij i ij
Assumptions of the errors:
(cid:15) are normally distributed
ij
(cid:15) are independent
ij
(cid:15) has mean zero and constant variance σ2.
ij
YingLi Lec3:ModelAdequacyChecking
Check the normality
Normal probability plot.
YingLi Lec3:ModelAdequacyChecking
Check the normality
Normal probability plot.
YingLi Lec3:ModelAdequacyChecking