Foundations and Applications of Statistics: An Introduction Using R

640 Pages·2011·7.02 MB·English

by  Randall Pruim

The UNDERPGRuAreD UanATdE A p TpEliXeTdS
SERIES
Sally
Foundations
and Applications
of Statistics
An Introduction Using R
Randall Pruim
American Mathematical Society
Foundations
and Applications
of Statistics
An Introduction Using R

T h e UNDERGRADUATE TEXTS • 13
SERIES
Pure and Applied
Sally
Foundations
and Applications
of Statistics
An Introduction Using R
Randall Pruim
American Mathematical Society
Providence, Rhode Island
EDITORIAL COMMITTEE
Paul J. Sally, Jr. (Chair) Joseph Silverman
Francis Su Susan Tolman
2010 Mathematics Subject Classification. Primary 62–01; Secondary 60–01.
For additional informationand updates on this book, visit
www.ams.org/bookpages/amstext-13
Library of Congress Cataloging-in-Publication Data
Pruim,RandallJ.
Foundationsandapplicationsofstatistics: anintroductionusingR/RandallPruim.
p.cm. —(Pureandappliedundergraduatetexts;v.13)
Includesbibliographicalreferencesandindex.
ISBN978-0-8218-5233-0(alk.paper)
1.Mathematicalstatistics—Dataprocessing. 2.R(Computerprogramlanguage) I.Title.
QA276.45.R3P78 2010
519.50285—dc22
2010041197
Copying and reprinting. Individual readers of this publication, and nonprofit libraries
actingforthem,arepermittedtomakefairuseofthematerial,suchastocopyachapterforuse
in teaching or research. Permission is granted to quote brief passages from this publication in
reviews,providedthecustomaryacknowledgmentofthesourceisgiven.
Republication,systematiccopying,ormultiplereproductionofanymaterialinthispublication
is permitted only under license from the American Mathematical Society. Requests for such
permissionshouldbeaddressedtotheAcquisitionsDepartment,AmericanMathematicalSociety,
201 Charles Street, Providence, Rhode Island 02904-2294 USA. Requests can also be made by
[e-mail].
(cid:2)c 2011byRandallPruim. Allrightsreserved.
PrintedintheUnitedStatesofAmerica.
(cid:2)∞ Thepaperusedinthisbookisacid-freeandfallswithintheguidelines
establishedtoensurepermanenceanddurability.
VisittheAMShomepageathttp://www.ams.org/
10987654321 161514131211
Contents
Preface ix
What Is Statistics? xv
Chapter 1. Summarizing Data 1
1.1. Data in R 1
1.2. Graphical and Numerical Summaries of Univariate Data 4
1.3. Graphical and Numerical Summaries of Multivariate Data 17
1.4. Summary 20
Exercises 21
Chapter 2. Probability and Random Variables 27
2.1. Introduction to Probability 28
2.2. Additional Probability Rules and Counting Methods 33
2.3. Discrete Distributions 49
2.4. Hypothesis Tests and p-Values 56
2.5. Mean and Variance of a Discrete Random Variable 65
2.6. Joint Distributions 72
2.7. Other Discrete Distributions 80
2.8. Summary 92
Exercises 97
Chapter 3. Continuous Distributions 113
3.1. pdfs and cdfs 113
3.2. Mean and Variance 124
3.3. Higher Moments 126
3.4. Other Continuous Distributions 133
v
vi Contents
3.5. Kernel Density Estimation 145
3.6. Quantile-Quantile Plots 150
3.7. Joint Distributions 155
3.8. Summary 163
Exercises 166
Chapter 4. Parameter Estimation and Testing 175
4.1. Statistical Models 175
4.2. Fitting Models by the Method of Moments 177
4.3. Estimators and Sampling Distributions 183
4.4. Limit Theorems 191
4.5. Inference for the Mean (Variance Known) 200
4.6. Estimating Variance 208
4.7. Inference for the Mean (Variance Unknown) 212
4.8. Confidence Intervals for a Proportion 223
4.9. Paired Tests 225
4.10. Developing New Tests 227
4.11. Summary 234
Exercises 238
Chapter 5. Likelihood-Based Statistics 251
5.1. Maximum Likelihood Estimators 251
5.2. Likelihood Ratio Tests 266
5.3. Confidence Intervals 274
5.4. Goodness of Fit Testing 277
5.5. Inference for Two-Way Tables 288
5.6. Rating and Ranking Based on Pairwise Comparisons 297
5.7. Bayesian Inference 304
5.8. Summary 312
Exercises 315
Chapter 6. Introduction to Linear Models 323
6.1. The Linear Model Framework 324
6.2. Simple Linear Regression 333
6.3. Inference for Simple Linear Regression 341
6.4. Regression Diagnostics 353
6.5. Transformations in Linear Regression 362
6.6. Categorical Predictors 369
6.7. Categorical Response (Logistic Regression) 377
6.8. Simulating Linear Models to Check Robustness 384
Contents vii
6.9. Summary 387
Exercises 391
Chapter 7. More Linear Models 399
7.1. Additive Models 399
7.2. Assessing the Quality of a Model 413
7.3. One-Way ANOVA 423
7.4. Two-Way ANOVA 455
7.5. Interaction and Higher Order Terms 470
7.6. Model Selection 474
7.7. More Examples 482
7.8. Permutation Tests and Linear Models 493
7.9. Summary 496
Exercises 499
Appendix A. A Brief Introduction to R 507
A.1. Getting Up and Running 507
A.2. Working with Data 513
A.3. Lattice Graphics in R 527
A.4. Functions in R 535
A.5. Some Extras in the fastR Package 542
A.6. More R Topics 544
Exercises 546
Appendix B. Some Mathematical Preliminaries 549
B.1. Sets 550
B.2. Functions 552
B.3. Sums and Products 553
Exercises 555
Appendix C. Geometry and Linear Algebra Review 559
C.1. Vectors, Spans, and Bases 559
C.2. Dot Products and Projections 563
C.3. Orthonormal Bases 567
C.4. Matrices 569
Exercises 575
Appendix D. Review of Chapters 1–4 579
D.1. R Infrastructure 579
D.2. Data 580
D.3. Probability Basics 581
D.4. Probability Toolkit 581
viii Contents
D.5. Inference 582
D.6. Important Distributions 583
Exercises 583
Hints, Answers, and Solutions to Selected Exercises 587
Bibliography 599
Index to R Functions, Packages, and Data Sets 605
Index 609
Preface
Intended Audience
As the title suggests, this book is intended as an introduction to both the foun-
dations and applications of statistics. It is an introduction in the sense that it
does not assume a prior statistics course. But it is not introductory in the sense
of being suitable for students who have had nothing more than the usual high
schoolmathematicspreparation. Thetargetaudienceisundergraduatestudentsat
the equivalent of the junior or senior year at a college or university in the United
States.
Students should have had courses in differential and integral calculus, but not
much more is required in terms of mathematical background. In fact, most of my
students have had at least another course or two by the time they take this course,
but the only courses that they have all had is the calculus sequence. The majority
of my students are not mathematics majors. I have had students from biology,
chemistry, computer science, economics, engineering, and psychology, and I have
tried to write a book that is interesting, understandable, and useful to students
with a wide range of backgrounds and career goals.
This book is suitable for what is often a two-semester sequence in “mathe-
matical statistics”, but it is different in some important ways from many of the
books written for such a course. I was trained as a mathematician first, and the
book is clearly mathematical at some points, but the emphasis is on the statistics.
Mathematics and computation are brought in where they are useful tools. The
result is a book that stretches my students in different directions at different times
– sometimes statistically, sometimes mathematically, sometimes computationally.
The Approach Used in This Book
Features of this book that help distinguish it from other books available for such a
course include the following:
ix