Probability and Statistical Inference Volumen 2.

188 Pages·1985·14.439 MB·English

by  J. G. Kalbfleisch

J. G. Kalbfleisch
Probability and
Statistical Inference
Volume 2: Statistical Inference
Second Edition
With 27 Illustrations
Springer-Verlag
New York Berlin Heidelberg Tokyo
Springer Texts in Statistics
Advisors:
Stephen Fienberg Ingram Olkin
J.G. Kalbfleisch
DUenpivaerrtmsiteyn to of fW Satatetirslotioc s and Actuarial Science Preface
Waterloo, Ontario, N2L 3G 1 -
Canada
Editorial Board
Stephen Fienberg Ingram Olkin
Department of Statistics Department of Statistics
Carnegie-Mellon University Stanford University
Pittsburgh, PA 15213 Stanford, CA 94305
U.S.A. U.S.A.
This book is in two volumes, and is intended as a text for introductory courses
AMS Classification: 62-01 ipgnho aopsdrio zbbeasa bcakilpgiptryloi ucaanntddi o nsintsa tafirsnetdisc hlsmo gaatin c tahcle ap lsrcieunclcouinsp dlei sso rsr uatthfhfiirecdri etynheta anfr o mur namitvhoeesrmst iatoytif c latehlv eet hlm.e oIartty ee.r miaA l
Library of Congress Cataloging in Publication Data presented. Several starred sections have been included as supplementary material.
Kalbfleisch, J.G. Nearly 900 problems and exercises of varying difficulty are given, and Appendix
I. PCT(IISn.rio cotpPnllbreutriae.ond bnbgeIitIsaels. ibr: ti ySintlv eeid.at xreiniteIxesd.ss e . is.P sn .tr aos2ttib.a sattMiibsctiaailclti hstiey)n m-fvear.te icn2a.c le S.s ttaattiissttiiccasl. inference. Aeaemnx dTcpa ohtaoinencrt aagdfliiae rnsnmdsti -zevafatnoohtislorouw dnmese x retafso om ( rtCot phdh leaeae bsp, c1otter9uhir7tbes 9 ion 1dnge-iesd8 c-ai)ttun hidsdoiesr nidam.ol snoaS snfwo itompihtfuhee l m iapnst.rdei oncebgtpia oetbnnhidesli metyhn. at m vIrteao indsbd eesolesimm na inrlvdeaa wrrw iiraniitbt htcle eonmsn ataaetnnhndd t
QA273.K27 1985 519.5'4 85-12580 conditional probability. Many new exercises have been added.
TP©Prrh ooe1bb 9afa7ibbr9sii ltl iibtteyyyd iSaatnnipoddrni n SSgwtteaaarttsi-is sVtptiieuccrabalalll igsII nnhNffeeeedrrw eeinn nY ccteeow rIIko I ( Iv(UnUonclni.uvi:mv eeresrssi,ti etexxt)t ). beepxasrtostIienipmsn e satrfhitotviieroee snlts yh epo.ec r fooCa nbenhdslaae tlipymvmtsoesialrs,ut s iam aos9nn e id na p( niCtrndhoht eeca Jre1pOpd9t reu7edrrt9seea ssteic9,do r-inia1tb in6oeod) nf ,t. hi dnpCeat rthroauoa.bsd peauTt bechoireli fiss1 t 1ytmhc otemah vteeloenirrdk iaedeaglllisi seh ch oauaprossre dso be buesfas eufbennrdi elc riqtteaiuyovse ninsa tehcnidyned
A©Ywoli ltr 1hkr9o i8gu15ht0 t0wbs1 yr0r ie,tS stepUenrr.iv Snpeg.deAer.r m.- NVisoes rilpoaangr tf Nrooefmw t h SiYsp orbrikon ogIkenr c-m.V aeyr labge, t1ra7n5s lFaitfetdh oArv reenpureo,d uNceewd Yino arkn,y Nfoewnn cpm1or3ain mtafeinarddireai nll1y c4o e ntp oi r nteetfsesreertnivqntau glmes,n .eac Ctnyhdh o addcpasott naefo.rn re1 Tca2htn deai etl ylswiczkriietnihblgi eh esdoa aotretldais e tusrra n omtdifoea s rti esgtrthnaieait iflas icotsiasncun emcisse pt,i utmwisoeaintdth i oo tafnop n.p uoClnirhcmifaayatpi lotitetnhyrses,
TPryipnetesde t abnyd Hbo: uCnhda rblye sRw.oRr.t hD &o nCnoel.l eLyt da.n, dH Suodndse,r sHfiaerlrdi,s oEnnbgulragn,d .V irginia. awtiitohn .e mChpahpatseisr 1o5n ctohnes iidmeprso rstuafnfcicei eonft cstoartriesctitclys amndo dceolnlidnigti otnhea l etxesptesr,i manedn Ctahl aspittue r
Printed in the United States of America. 16 presents some additional topics in statistical inference.
9 8 7 6 5 4 3 2 I The content of volume two is unusual for an introductory text. The importance
IISSBBNN 03--358470--9966118833--66 SSpprriinnggeerr--VVeerrllaagg BNeewrli nY Horeki dBeelbrleirng HNeeidwe lYbeorrgk TTookkyyoo ofofr tdheer ipvrio 1bga bsuiliittayb lme oedsteilm iast eesm, pinhtaesrvizaelds,, aanndd tgesetns efrraolm t etchhen ilqikueelsih oaroed pfruenscetniotend.
Preface
1 atptAh mhfrufTleaeeloosa schs Imintaoicn eeolgiha1ng,i rm tlaust iiIscin v el tenuaahihsteqsgsenle a. a un dovtsttemtoh iemeinto eot ee cnil ansotfete hvtto uaas eoorirton,dsmn di dffd w so atm edf orlhirdpe nuoitt cracthcmyueeivhhespssr o io peieotiniorns rhdnyp gefte> be t r oattod.eddhh tfbteai,eet hoclet epaynirsasma p tpilaraoooespttn fi.fnte cdht h-t aAuaehtafrhlros enaadwier ndrs ool icl,mtydrb aeee entsa stei eottiofrveicrt fn vars c igtrlteoaaouai.d elu nd Itrmrgensyoesn ui cneigmontais snpifg oo mie iepahnfs .iptit a cb izmpsvmFaoetlenoaa oia cecttrklaihar eis critet tnyoiwclmioesce rinv ctaastafhrsseltrni i laetcc,ctempq oaaerrup ln,aia c ela nstiIobn htcy famecedao t irtiitsrpei ssr postuceteh einutascaciiststnnsnie,is a. g d l Contents of Volume 2
RftEftroferoBtogrSx otos omheLre pemJImiseec aoeo aIti xauc nnirhrtmSwoTpti hc,gnhoa rnrhea im laferess ibg oW lstrh,(aotplrir s6h sena fafi bttta lttu onmhit8eehonGclh d yf eo kyateeuor c ihk sl odtrlof aaao l iu aTnmnpaetB tSpotidMrankpteoim e a ob cr nLt sSSelmomhlediet,.coiuesn dyuesrit nLr1ta .iteBsaR,scr9tcya gi iai7ooontLnokIooel4lnndnnad o mml)a aec nTP.ttlT .aFeehdod arHgtnaeoo o bfrut bFomofrinl Ikeelmehria, ey9sssa s nnsu sh 7p f de aowotlfe9efrar onorrrru is s dre,mffrsuJp eo d tFopebis SerBiscrtR p.tyetsuiRimsako sooieudt o Dr.inblipefMStsosn.eo.,nesr . g criari,tati Hoc scbocncpt ina K oaadaaet ,r len lt rtaedDo ,aocms ytp nnh ,hrA r(ir .dnes1of eeVg iosF9dpaccrUior5rurroiah lialcn2clnh.niinu ile)ei tvtd;kls1 rty teTor p updae(rY maor3isnrnrnbanaaierad tgdltflJpiny ee,to.a r ys ts sennoo o aF,do des eIfniiF tf,ux ,nt h Wdo i.ccItaDeoR hfleel anl Mila l.T,sLytl,Smv e e e,aiaei. rn1tadbdnd,leI9tt iod ,lri eac w6etoa srnaai 6Vri uoindIalyf)sgIn dol;h .r CPHreAfPaTceE R 9 v
Likelihood Methods
J.G. Kalbfleisch 999... 321 RCTheolema tbMivineei tnhLgoi kdIe noldifhe oMpoeadnx diemnut mE xLpiekreimliheonotsd 133
999 ..654 CILneivknaesrloiiahrnioncoged info rL iCfeotnimtine uEouxsp eMrimodeenltss 321257
9. 7 Normal Approximations 37
9.8 Newton's MetlioJ 40
Review Problems 46
51
CHAPTER 10
Two-Parameter Likelihoods
53
!11J100000.....5 423l ANMRM oeaaDlrxxamoitimimsavelueu -R mAmLep isRLkppreeiookllinaxehtsiloimievho eaoEd to xiLoad.iann kmEdse splCitlhieomo noatdtoi ounr Maps 76650513
1100..76* SAonm Eex Damerpivlea tfiroonms Leaming Theory 8734
10.8* Multi-Parameter Likelihoods 88
92
ix
Contents
viii Contents
285
15.2 Properties .of Sufficient Statistics
CHAPTER 11 15.3 Exact Significance Levels and Coverage Probabilities 289
Frequency Properties 96 15.4 Choosing the Reference Set 296
15.5 Conditional Tests for Composite Hypotheses 300
11. I Sampling Distributions 97 15.6 Some Examples of Conditional Tests 305
11. 2 Coverage Probability 102
11.3 Chi-Square Approximations 107
11.4 Confidence Intervals 113 CHAPTER 16 314
11.5 Results for Two-Parameter Models 120 Topics in Statistical Inference
11.6* Expected Information and Planning Experiments 124 16.1 * The Fiducial Argument 314
11. 7* Bias 129 16.2* Bayesian Methods 321
326
16.3* Prediction
CHAPTER 12 16.4* Inferences from Predictive Distributions 330
Tests of Significance 134 16.5* Testing a True Hypothesis 334
12. I Introduction 134
12.2 Likelihood Ratio Tests for Simple Hypotheses 141 APPENDIX A
12.3 Likelihood Ratio Tests for Composite Hypotheses 149 Answers to Selected Problems 337
12 .4 Tests for Binomial Probabilities 156
12.5 Tests for Multinomial Probabilities 160 APPENDIX B
12.6 Tests for Independence in Contingency Tables 170 Tables 347
I 2. 7 Cause and Effect 179
357
12.8 Testing for Marginal Homogeneity 182 Index
12.9 Significance Regions 186
12.10* Power 190
CHAPTER 13
Analysis of Normal Measurements 196
13. I Introduction 196
13.2 Statistical Methods 200
13. 3 The One-Sample Model 206
13.4 The Two-Sample Model 212
13. 5 The Straight Line Model 220
13.6 The Straight Line Model (Continued) 229
13. 7 Analysis of Paired Measurements 234
Review Problems 240
CHAPTER 14
Normal Linear Models 242
14.1 Matrix Notation 242
14.2 Parameter Estimates 247
14.3 Testing Hypotheses in Linear Models 252
14.4 More on Tests and Confidence Intervals 260
14.5 Checking the Model 267
14.6* Derivations 274
CHAPTER 15
Sufficient Statistics and Conditional Tests 277
15. I The Sufficiency Principle 277
Contents of Volume 1
Preface
CHAPTER I
Introduction
I. I Probability and Statistics
1. 2 Observed Frequencies and Histograms
1.3 Probability Models
1.4 Expected Frequencies
CHAPTER 2
Equi-Probable Outcomes
2.1 Combinatorial Symbols
2.2 Random Sampling Without Replacement
2.3 The Hypergeometric Distribution
2.4 Random Sampling With Replacement
2.5 The Binomial Distribution
2.6* Occupancy Problems
2.7* The Theory of Runs
2.8* Symmetric Random Walks
CHAPTER 3
The Calculus of Probability
3.1 Unions and lntersections of Events
3.2 Independent Experiments and Product Models
3.3 Independent Events ·
xii Contents Contents Xlll
3.4 Conditional Probability 7. 3 Transformations of Normal Variates
3.5 Some Conditional Probability Examples 7.4 * The Bivariate Normal Distribution
3.6 Bayes's Theorem 7. 5* Conditional Distributions and Regression
3. 7* Union of n Events
Review Problems CHAPTER 8
Generating Functions
CHAPTER 4 8. 1* Preliminary Results
Discrete Variates
8.2* Probability Generating Functions
4.1 Definitions and Notation 8.3* Moment and Cumulant Generating Functions
4.2 Waiting Time Problems 8.4* Applications
4.3 The Poisson Distribution 8.5* Bivariate Generating Functions
4.4 The Poisson Process
4.5 Bivariate Distributions APPENDIX A
4. 6 Independent Variates Answers to Selected Problems
4.7 The Multinomial Distribution
Review Problems
APPENDIX B
Tables
CHAPTER 5
Mean and Variance
Index
5. 1 Mathematical Expectation
5.2 Moments; the Mean and Variance
5. 3 Some Examples
5.4 Covariance and Correlation
5. 5 Variances of Sums and Linear Combinations
5.6* Indicator Variables
5. 7* Conditional Expectation
Review Problems
CHAPTER 6
Continuous Variates
6.1 Definitions and Notation
6.2 Uniform and Exponential Distributions
6.3* Transformations Based on the Probability Integral
6.4* Lifetime Distributions
6.5* Waiting Times in a Poisson Process
6.6 The Normal Distribution
6. 7 The Central Limit Theorem
6.8 Some Normal Approximations
6.9 The Chi-Square Distribution
6.10 The F and t Distributions
Review Problems
CHAPTER 7
Bivariate Continuous Distributions
7. 1 Definitions and Notation
7.2 Change of Variables
CHAPTER 9
Likelihood Methods
The first volume dealt with probability models, and with mathematical
methods for handling and describing them. Several of the simplest discrete
and continuous probability models were considered in detail. This volume is
concerned with applications of probability models in problems of data
analysis and interpretation.
One important use of probability models is to provide simple mathemat
ical descriptions of large bodies of data. For instance, we might describe a set
of 1000 blood pressure measurements as being like a sample of 1000
independent values from a normal distribution whose mean µ and variance
a2 are estimated from the data. This model gives a concise description of the
data, and from it we can easily calculate the approximate proportion of blood
pressure measurements which lie in any particular range. The accuracy of
such calculations will, of course, depend upon how well the normal distri
bution model fits the data.
We shall be concerned primarily with applications of probability models in
problems of statistical inference, where it is desired to draw general
conclusions based on a limited amount of data. For instance, tests might be
run to determine the length of life of an aircraft component prior to failure
from metal fatigue. Such tests are typically very expensive and time
consuming, and hence only a few specimens can be examined. Based on the
small amount of data obtained, one would attempt to draw conclusions
about similar components which had not been tested. The link between the
observed sample and the remaining components is provided by the proba
bility model. The data are used to check the adequacy of the model and to
estimate any unknown parameters which it involves. General statements
concerning this type of component are then based on the model.
A limited amount of data can be misleading, and therefore any general
2 9. Likelihood Methods 9.1 The Method of Maximum Likelihood 3
conclusions drawn will be subject to uncertainty. Measurement of the extent 9.1 The Method of Maximum Likelihood
of this uncertainty is an important part of the problem. An estimate is of little
value unless we know how accurate it is likely to be. Suppose that a probability model has been formulated for an experiment, and
In statistical inference problems, we usually start with a set of data, and that it involves a single unknown parameter 8. The experiment is performed
with some information about the way in which the data were collected. We and some data are obtained. We wish to use the data to estimate the value of
then attempt to formulate a probability model for the experiment which gave 8. More generally, we wish to determine which of the possible values of 8 are
rise to the data. Examination of the data, and· of other similar data sets, can plausible or likely in the light of the observations. .
be very useful at this stage. It is important to treat the data set in context, and The observed data can be regarded as an event E in the sample space for
to take full advantage of what is already known from other similar the probability model. The probability of event E can be determined from the
applications. model, and in general it will be a function of the unknown parameter, P(E; 8).
Usually the probability model will involve one or more unknown para The maximum likelihood estimate (MLE) of 8 is the value of 8 which
meters which must be estimated from the data. We have already encountered maximizes P(E; 8). The MLE of 8 is usually denoted by 0. It is the parameter
this problem on several occasions, and have used the observed sample mean value which best explains the data E in the sense that it maximizes the
as an estimate of the mean of a Poisson or exponential distribution. probability of E under the model..
Intuitively, this is a reasonable thing to do, but intuition may fail us in more
complicated situations. EXAMPLE 9.1.1. Suppose that we wish to estimate 8, the proportion of people
Section 9.1 introduces the method of maximum likelihood, which provides with tuberculosis in a large homogeneous population. To do this, we
a routine procedure for obtaining estimates of unknown parameters. Section randomly select n individuals for testing, and find that x of them have the
2 considers the problem of estimating an unknown parameter 8 on the basis disease. Since the population is large and homogeneous, we assume that the n
of data from two independent experiments. Section 3 shows how the relative individuals tested are independent, and that each has probability 8 of having
likelihood function may be used to rank possible values of 8 according to tuberculosis. The probability of the observed event (data) is then
their plausibilities.
P(E; 8) = P(x out of n have tuberculosis)
Section 4 describes likelihood methods when the probability model is
continuous. The special case of censoring in lifetime experiments is consi
(9.1.1)
dered in Section 5. Section 6 discusses the invariance property of likelihood
methods, and Section 7 describes a normal approximation to the log relative
likelihood function. The use of Newton's method in finding maximum where 0::;; 8::;; 1. The maximum likelihood estimate 8 is the value of 8 which
likelihood estimates and likelihood intervals is illustrated in Section 8. x
maximizes (9.1.1). We shall show later that (9.1.1) is maximized for 8 = - , and
In this chapter it is assumed that the probability model involves only one n
unknown parameter. Likelihood methods for the estimation of two or more so the MLE of 8 is e= :_. To maximize the probability of the data we
unknown parameters are described in Chapter 10. Some theoretical pro- . n
perties of these estimation procedures are considered in Chapter 11. x
Chapter 12 introduces tests of significance, which are used to investigate estimate 8, the proportion of diseased persons in the population, by - , the
n
whether various hypotheses of interest are consistent with the data. Several
proportion of diseased persons in the sample.
applications of significance tests to frequency data are given.
Traditionally, the normal distribution has played a very important role in
statistical applications. Chapters 13 and 14 develop estimation procedures The Likelihood and Log Likelihood Functions
and significance tests for a variety of situations where measurements are
assumed to be independent and normally distributed. Finally, Chapters 15
and 16 deal with some more advanced topics in statistical inference. Note that the constant factor (:) will have no effect on the maximization of
(9.1.1) over 8. To simplify expressions, we shall generally omit such constants
and consider only the part of P(E; 8) which involves 8.
The likelihood function of 8 is defined as follows:
L(8) = c · P(E; 8). (9.1.2)
4
9. Likelihood Methods \, 9.1 The Method of Maximum Likelihood 5
e8Hx, eparrleet hscso iiuso gnah nf yoit r p mLolas,8iyt) ,i bvaeen cad o fsnuusnbtcastneioqt nuw eointfht trrheeess updleatcstat w.t oiWl l8 e;n otchht aodtoe ispse,e cnc d its o un pooobt ntaa tifhnue na cs tspiieomcnip foilcef lTikheel ihieofoodrm fautniocnti ofnu nwctiitohn re..s1p(e8c) t ist o m8:i nus the second derivative of the log
choice made.
Usually P(E; 8) and L(8) are products of terms, and it will be more .~(8) = -/"(8) = -S'(8) = - d;~~). (9.1.5)
cloognavreinthiemn to tfo L w ork with logarit1h(8m) s=. lTohge L l(o8g) . likelihood function is the na(9t.u1r.3a)l aNnoT tihen teet hsrevatat lnn o eofi ftrh peeaorls sSvi(ba8ll)ue e nvsoa, rlsu .ue.1cs(h 8o )af ds8.e; [i pOs e,c nald]lls e idno ntt hhteeh epe axcrahamomipec.letee ora fsb pcoa vicnee ,. (a9U.n1sd.u2 a)t.lh ley fQi rsist
Note that, by (9.1.2), /(8) = c' +log P(E; 8) sTaenhcdeo nns,ed ci fod 8nerd ii svd aeatrniiv veian twtievirleils o bro efp /no(8ei)ng taw toiitvfh en ' ra,e tst ph8ee =c ft i8rt.os t T 8dh eeuxrsiivs uta natidtv eaerl l w tihniletls eebr ieoc orz enprdooii tniaotnsnd os ftw hnee.
where c' =log c is not a function of 8. have
lf1Pou(8(gnET). c ;lht iTi8eko)h e.n mluT iashahn oxetdoih m dvet ha ufleuMmu nleo Lc lgotiEi kfol ie8nkBl .iew h liisohh iooctdhho e dem sfvtauiaxmnlicumatetii eozo neBfs. U 8iPss (uwEtha;h eli8l cy)v h wia tl imuilsle ae axaolissmfoi e 8ismz tew astxoh i itwmchheoi z rmelki kaLwex(8ilitim)hh ioatznoheddes mTWoae x fiitnmhdeu n8m , v wheera isfdy e,b teeberymn cifnoheue cnthkdei. n Srgo( Bot)ht s=e o0sf;i gthne om.f.a f.(x.B1i(m)8 >)u moo. r l ikoethliehrowoids ee,q uthataito na Sr(e8(l)9a .=t1i v.06e.)
In some simple examples, the maximum likelihood equation S(8) = 0 can
tEiXmAeMs PthLeE e9x.1p.1re s(scioonnt ifnoure Pd)(E. ;T 8h) ei nl i(k9e.1l.i1h)o, owdh efuren cc tmioany odfe p8e ins da onny nc aonndst xa nbtu ct 9sbi.e8t u).as otilovnesd, iat lwgeilbl rbaeic naellcye sstoa ryy iteol ds olav ef othrmis uelqau afotiro ne .n uImn ermicoarlely c(oseme pSleiccatitoend
not on 8. Since the aim in choosing c is to simplify the expression, a natural Situations do arise in which e cannot be found by solving the maximum
choice is c = 1 /(:).and then likelihood equation S(8) = 0. For instance S(8) need not be zero if the overall
L(B) = B"(l - er-" for 0:::;: 8:::;: I. i' mExaaxmimpulems 9.o1f.1 /(a8n) do 9c.1c.u2r)s. Tohne saa mboe uisn dtraurey ifo 8f ist hree stpraicrtaemd ettoe ra dspisaccree ten s e(ts eoef
The log likelihood function is now I possible values such as the integers (see Problems 9.1.7 and 9.1.11).
/(8) = x log 8 + (n-x)log(l - 8) for 0:::;: 8:::; I. EXAMPLE 9.1.1 (continued). For this example, the score and information
functions are
The MLE Bis the value of 8 which maximizes /(8).
d/(8) x n - x
S(8) = de = B - 1 - 8 for0<8< l;
The Score and Information Functions
dS(8) x n - x
5(8) = - d8 = 82 + (1 - 8)2 for 0 < 8 <I.
Tofo 8 e. vTahluisa ctea n8, uwseu anlleye db eto d looncea tbey t hdeif fmeraexnitmiautimng o /f( 8/()8 w) oitvhe rr easlpl epcots tsoib 8le, sveatltuinegs For 1 :::; x:::; n - I, the maximum likelihood equation S(8) = 0 has a unique
pthreo cdeedruirvea tmivieg het qyuiealld tao r ezleartoiv, e amnidn ismoulvmin ogr fpoori n8t. oIft i nifsl epxoiosns iibnlset etahda to ft thhies solution 8 =:n:. Since .1(8) > 0 at 8 = :n: , the likelihood function has a relative
fmoauxnidm, upmer hdaepssir ebdy. cThheucsk iint gi st hnaetc ethssea rsye ctoon dv edriefryi vtahtaivt ea ism naexgiamtiuvme. has been maximum at 8 = :n: . Furthermore, since L(B) = 0 for 8 = 0 and for 8 = 1, we
likTelhihe osocdo rfeu nfuctnicotnio wn itSh( 8r)e sips ecdte ftion e8d: to be the first derivative of the log have found the overall maxi. mum; and thus e= -xn .
If x = 0, the equation S(8) = 0 has no solution, and the maximum occurs on
S(B) = /'(8) = d~~); (9.1.4) a boundary of the parameter space Q = (0, 1]. In this case we have
P(E; 8) = (1 -' 8)" for 0 s 8:::; 1,