Multivariate Statistics: A practical approach

306 Pages·1988·6.1 MB·English

by  Bernhard Flury

Multivariate Statistics
A practical approach
OTHER STATISTICS TEXTS FROM
CHAPMAN AND HALL
Applied Statistics
D.R. Cox and E.l. Snell
The Analysis of Time Series
C. Chatfield
Decision Analysis: A Bayesian Approach
IQ. Smith
Statistics for Technology
C. Chatfield
Introduction to Multivariate Analysis
C. Chatfield and A.l. Collins
Introduction to Optimization Methods and their Application in Statistics
B.S. Everitt
An Introduction to Statistical ModeUing
A.l. Dobson
Multivariate Analysis of Variance and Repeated Measures
c.c.
D.l. Hand and Taylor
Multivariate Statistical Methods - a primer
Bryan F. Manly
Statistical Methods in Agriculture and Experimental Biology
R. Mead and R.N. Curnow
Elements of Simulation
D.l.T. Morgan
Essential Statistics
D.G. Rees
Applied Statistics: A Handbook of BMDP Analyses
E.J. Snell
Intermediate Statistical Methods
G.B. Wetherill
Foundations of Statistics
D.G. Rees
Probability: Methods and Measurement
A. O'Hagan
Elementary Applications of Probability Theory
H.C. Tuckwell
Further information on the complete range of Chapman and Hall
statistics books is available from the publishers.
Multivariate Statistics
A practical approach
Bernhard Flury
and
Hans Riedwyl
London New York
CHAPMAN AND HALL
First published in 1988 by Chapman and Hall Ltd
11 New Fetter Lane, London EC4P 4EE
Published in the USA by Chapman and Hall
29 West 35th Street, New York NY 10001
© 1988 B. Flury and H. Riedwyl
Softcover reprint of the hardcover 1st edition 1988
ISBN-13:978-94-010-7041-6
This title is available in both hardbound and paperback editions. The
paperback edition is sold subject to the condition that it shall not, by way of
trade or otherwise, be lent, resold, hired out, or otherwise circulated without
the publisher's prior consent in any form of binding or cover other than that in
which it is published and without a similar condition including this condition
being imposed on the subsequent purchaser.
All rights reserved. No part of this book may be reprinted, or reproduced or
utilized in any form or by any electronic, mechanical or other means, now
known or hereafter invented, including photocopying and recording, or in any
information storage and retrieval system, without permission in writing from
the publisher.
British Library Cataloguing in Publication Data
Flury, Bernhard
Multivariate statistics: a practical
approach.
1. Multivariate analysis
I. Title II. Riedwyl, Hans
519.5'35 QA278
ISBN-13:978-94-010-7041-6 e-ISBN-13:978-94-009-1217-5
DOl: 10.1007/978-94-009-1217-5
Library of Congress Cataloging-in-Publication Data
Flury, Bernhard, 1951-
Multivariate statistics.
Rev. translation of: Angewandte multivariate
Statistik.
Bibliography: p.
Includes index.
1. Multivariate analysis. I. Riedwyl, Hans.
II. Title. .
QA278.F58813 1988 519.5'35 87-18405
ISBN-13:978-94-010 -7041-6
Contents
Preface ix
1 The data 1
Discussion 9
2 Univariate plots and descriptive statistics 11
Discussion 16
Further study 18
3 Scatterplot, correlation and covariance 19
Discussion 36
Further study 37
4 Face plots 38
Discussion 52
Further study 52
5 Multiple linear regression 54
5.1 Introductory remarks 54
5.2 The model of multiple linear regression 54
5.3 Least squares estimation 56
5.4 Residual analysis 58
5.5 Model building, analysis of variance 59
5.6 The overall test of significance 61
5.7 Coefficient of determination and multiple correlation 62
5.8 Tests of partial hypotheses 63
5.9 Standard errors of the regression coefficients 64
5.10 Selection of a subset of regressors 64
Discussion 69
Further study 74
6 Linear combinations 75
6.1 Introduction 75
vi Contents
6.2 A special linear combination 75
6.3 Linear combinations of two variables 77
6.4 Linear combinations of several variables 81
6.5 Mean and standard deviation of linear combinations 84
Discussion 86
Further study 86
7 Linear discriminant analysis for two groups 88
7.1 Introduction 88
7.2 Multivariate standard distance 88
7.3 Relationship between discriminant analysis and multiple
linear regression 94
7.4 Testing hypotheses about the discriminant function 96
7.5 Screening a discriminant function 99
7.6 Further uses of the coefficient of determination 104
7.7 Classification of observations 106
Discussion 114
Further study 123
Examples 123
8 Identification analysis 136
8.1 Introduction 136
8.2 Identification analysis as a special case of discriminant
analysis 137
8.3 More about standard distance 138
8.4 Identification of a bank note 142
8.5 Analysis of outliers 144
Discussion 149
Further study 150
Examples 152
9 Specification analysis 156
9.1 Standard distance between a sample and a hypothetical
mean vector 156
9.2 Specification analysis of the bank notes 159
9.3 Confidence regions for a mean vector 165
9.4 A more general model 168
9.5 Specification faces 170
Further study 174
Examples 174
10 Principal component analysis 181
10.1 Introduction 181
Contents Vll
10.2 Principal components of two variables 184
10.3 Properties of principal components in the multidimensional case 190
10.4 Principal component analysis of the genuine bank notes 192
10.5 The singular case 197
10.6 Principal components, standard distance, and the
multivariate normal distribution 198
10.7 Standard errors of the principal component coefficients and
related problems 201
10.8 Principal component analysis of several groups 205
Discussion 210
Further study 212
Examples 213
11 Comparing the covariance structures of two groups 234
11.1 Introduction 234
11.2 The bivariate case 235
11.3 The multivariate case 237
11.4 Comparison of the covariance matrices of the genuine and forged
bank notes 243
11.5 Partial statistics for the analysis of Y and Y 245
max min
11.6 Stepwise analysis of Y and Y 248
max min
11. 7 Relationships to standard distance and principal component
analysis 252
11.8 Critical values of the distribution of F max and F min 256
Discussion 257
Further study 258
Examples 258
12 Exercises 263
12.1 Exercises based on the bank note data 263
12.2 Additional exercises 270
13 Mathematical appendix 276
13.1 Introduction and preliminaries 276
13.2 Data matrix, mean vector, covariance and correlation 277
13.3 Multiple linear regression 277
13.4 Linear combinations 279
13.5 Multivariate standard distance and the linear discriminant
function 280
13.6 Principal component analysis 282
13.7 Comparison of two covariance matrices 286
References 288
Index 293
Preface
During the last twenty years multivariate statistical methods have become
increasingly popular among scientists in various fields. The theory had already
made great progress in previous decades and routine applications of
multivariate methods followed with the advent of fast computers. Nowadays
statistical software packages perform in seconds what used to take weeks of
tedious calculations.
Although this is certainly a welcome development, we find, on the other
hand, that many users of statistical packages are not too sure of what they
are doing, and this is especially true for multivariate statistical methods.
Many researchers have heard about such techniques and feel intuitively that
multivariate methods could be useful for their own work, but they haven't
mastered the usual mathematical prerequisites. This book tries to fill the gap
by explaining - in words and graphs - some basic concepts and selected
methods of multivariate statistical analysis.
Why another book? Are the existing books on applied multivariate statistics
all obsolete? No, some of them are up to date and, indeed, quite good.
However, we think that this introduction distinguishes itselffrom any existing
text in three ways. First, we illustrate the basic concepts mainly with graphical
tools, hence the large number of figures. Second, all techniques, with just
one exception, are illustrated by the same numerical data. Being familiar
with the data will help the reader to understand what insights multivariate
methods can otTer. Third, we have avoided mathematical notation as much
as possible. While we are well aware of the fact that this is a risky business,
since avoiding mathematical language inevitably implies some loss of
accuracy, we feel that it is possible to understand the basic ideas of multivariate
analysis without mastering matrix algebra. Of course, we do not wish to
discourage anybody from learning multivariate statistics the mathematical
way. Indeed, many ideas and concepts can be stated in fewer words using
concise mathematical language, but in order to appreciate the advantages
of a more abstract approach the reader needs to be familiar with matrix
theory, which most non-mathematicians tend to think of as a difficult subject.
This book has grown out of its German predecessor Angewandte
x Preface
multivariate Statistik, published in 1983 by Gustav Fischer Verlag, Stuttgart.
It is not just a translation, however; many new ideas and didactic tools have
been introduced, one chapter has been removed, and another one has been
added. The chapter that was deleted was the one on factor analysis, which
was criticized by some reviewers of the German text. Although it is probably
impossible to write about factor analysis without provoking criticism, we
felt that we were not sufficiently expert in this field, and that it was probably
better to omit this topic altogether.
Some remarks are in order concerning Chapter 5, which deals with multiple
linear regression. This chapter differs from the others in having a more
compressed presentation. It was indeed conceived as a restatement of linear
regression rather than a first introduction, and assumes that the reader is
familiar at least with simple linear regression. We felt that it wasn't possible
to omit this chapter, since a basic understanding of linear regression is
extremely helpful for understanding the linear discriminant function and
related concepts.
We do not claim that our book treats the various multivariate methods
comprehensively - on the contrary, we purposely limited the presentation to
those ideas and techniques that seemed most important to us, and tried to
explain those as carefully as possible. Our selection of topics is obviously
biased towards our own research areas, but this is what we feel most
competent to write about.
Many chapters are followed by one or several additional examples and
by a discussion. The questions stem mostly from the participants of several
courses on applied multivariate analysis that we taught from 1980 to 1985.
These questions gave us an opportunity to discuss additional important
methods or ideas that didn't fit into the main text. The length of the discussions
essentially reflects the number of times the respective chapter has been taught
so far.
Some readers of the German predecessor of this book have asked us to
supplement it with detailed instructions for the use of existing statistical
software such as BMDP, SAS, or SPSS. We have resisted the temptation to
introduce such instructions for two reasons. First, we would like the
information in this book to remain correct for, say, fifteen years. Most
instructions for the use of currently existing programs will probably be
obsolete by that time. Second, we think that statistical software should do
what textbooks propose, whereas the current situation is often just the
opposite: students think of statistics as the output produced by some computer
program.
Understanding this book requires relatively few prerequisites: the reader
should master basic algebra and be familiar with the basic methods of
univariate statistics. Knowing linear regression, at least for the case of a
single regressor, will be very helpful, however. It may be a good idea to do