Table Of ContentAn Intermediate Course in Probability
Allan Gut
An Intermediate Course
in Probability
~ Springer
Alian Gut
Department of Mathematics
Uppsala University
S-75106 Uppsala
Sweden
With 5 figures.
Mathematics Subject Classification (1991): 60-01
Library of Congress Cataloging-in-Publication Data
Out, Alian, 1944-
An intermediate course in probability/A lian Out.
p cm.
Includes bibliographical references (pp. 253-256) and index.
ISBN 978-0-387-94507-1 ISBN 978-1-4757-2431-8 (eBook)
DOI 10.1007/978-1-4757-2431-8
1. Probabilities. 1. Title.
QA273.08685 1995
519.2-dc20 95-11995
Printed on acid-free paper.
© 1995 Springer Science+Business Media New York
Original1y published by Springer Science+Business Media, Inc. in 1995
All rights reserved. This work may not be translated or copied in whole or in part without the written
permis sion of the publisher, Springer Science+Business Media, LLC.
except for brief excerpts in connection with reviews or scholarly analysis. Use in
connection with any form of information storage and retrieval, electronic adaptation, computer
software, or by similar or dissimilar methodo1ogy now known or hereafter developed is forbidden.
The use in this publication of trade names, trademarks, service marks and similar terms, even if they
are not identified as such, is not to be taken as an expression of opinion as to whether or not they are
subject to proprietary rights.
98765
springeronline.com
Preface
The purpose of this book is to provide the reader with a solid background
and understanding of the basic results and methods in probability the
ory before entering into more advanced courses (in probability and/or
statistics). The presentation is fairly thorough and detailed with many
solved examples. Several examples are solved with different methods in
order to illustrate their different levels of sophistication, their pros, and
their cons. The motivation for this style of exposition is that experi
ence has proved that the hard part in courses of this kind usually in the
application of the results and methods; to know how, when, and where
to apply what; and then, technically, to solve a given problem once one
knows how to proceed. Exercises are spread out along the way, and every
chapter ends with a large selection of problems.
Chapters I through VI focus on some central areas of what might
be called pure probability theory: multivariate random variables, condi
tioning, transforms, order variables, the multivariate normal distribution,
and convergence. A final chapter is devoted to the Poisson process be
cause of its fundamental role in the theory of stochastic processes, but
also because it provides an excellent application of the results and meth
ods acquired earlier in the book. As an extra bonus, several facts about
this process, which are frequently more or less taken for granted, are
thereby properly verified. The book concludes with three appendixes:
In the first we provide some suggestions for further reading and in the
second we provide a list of abbreviations and useful facts concerning
some standard distributions. The third appendix contains answers to
the problems given at the end of each chapter.
The level of the book is between the first undergraduate course in
probability and the first graduate course. In particular, no knowledge of
measure theory is assumed. The prerequisites (beyond a first course in
probability) are basic analysis and some linear algebra.
vi Preface
Chapter V is, essentially, a revision of a handout by professor Carl
Gustav Esseen. I am most grateful to him for allowing me to include the
material in the book.
The readability of a book is not only a function of its content and
how (well) the material is presented. Very important are layout, fonts,
and other aesthetical aspects. My heartfelt thanks to Anders Vretblad
for his ideas, views, and suggestions, for his design and creation of the
allan. sty file, and for his otherwise most generous help.
I am also very grateful to Svante Janson for providing me with
various index-making devices and to Lennart Norell for creating Fig
ure 111.6.1. Ola Hossjer and Pontus Andersson have gone through the
manuscript with great care at different stages in a search for misprints,
slips, and other obscurities; I wish to thank them so much for everyone
of their discoveries as well as for many other remarks (unfortunately,
I am responsible for possible remaining inadvertencies). I also wish to
thank my students from a second course in probability theory in Uppsala
and Jan Ohlin and his students from a similar course at the Stockholm
University for sending me a list of corrections on an earlier version of
this book.
Finally, I wish to thank Svante Janson and Dietrich von Rosen for
several helpful suggestions and moral support, and Martin Gilchrist of
Springer-Verlag for the care and understanding he has shown me and my
manuscript.
Up psala, Sweden Allan Gut
Contents
Preface............................................... v
Notation and Symbols .... , .. .. . . .. . . . . . . . . . . . . . . . . .. . . . xi
Introduction ......................................... 1
1. Models.. . .. . . .. . . . . . . . . . . . . . . . . . . . . .. . . .. . .. . . . . . . . 1
2. The Probability Space ............................... 3
3. Independence and Conditional Probabilities ............ 5
4. Random Variables ................................... 6
5. Expectation, Variance, and Moments .................. 8
6. Joint Distributions and Independence .................. 9
7. Sums of Random Variables, Covariance,
and Correlation ...................................... 10
8. Limit Theorems. . . .. . .. . . . . . . .. ... . . .. . . .. . .. . . .. . .. 12
9. Stochastic Processes ................................. 13
10. The Contents of the Book ............................ 13
I. Multivariate Random Variables ...................... 17
1. Introduction ........................................ 17
2. Functions of Random Variables ....................... 21
3. Problems ........................................... 27
II. Conditioning ...................................... 32
1. Conditional Distributions ............................ 32
2. Conditional Expectation and Conditional Variance .... . . . 34
3. Distributions with Random Parameters ................ 40
4. The Bayesian Approach .............................. 45
5. Regression and Prediction ............................ 49
6. Martingales ........................................ 53
7. Problems. . . . . . . . . . . . . . . . .. . . .. . . . . . . . . . .. . .. . . .. . . . 55
III. Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
1. Introduction ........................................ 60
2. The Probability Generating Function .................. 62
viii Contents
3. The Moment Generating Function ..................... 66
4. The Characteristic Function .......................... 73
5. Sums of a Random Number of Random Variables ....... 80
6. Branching Processes ................................. 87
7. Distributions with Random Parameters ................ 94
8. Problems ........................................... 96
IV. Order Statistics ................................... 102
1. One-Dimensional Results ............................. 102
2. The Joint Distribution of the Extremes ................ 107
3. The Joint Distribution of the Order Statistic. . . . . . . . . . .. 111
4. Problems... . . . . .. .. . . . . . .. . . . . . .. . . .. . . .. . . . . . .. . .. 115
V. The Multivariate Normal Distribution ................ 119
1. Preliminaries from Linear Algebra ..................... 119
2. The Covariance Matrix .............................. 122
3. A First Definition ................................... 123
4. The Characteristic Function: Another Definition ........ 125
5. The Density: A Third Definition ...................... 127
6. Conditional Distributions ............................ 129
7. Independence ....................................... 133
8. Linear Transformations .............................. 135
9. Quadratic Forms and Cochran's Theorem .............. 139
10. Problems ........................................... 143
VI. Convergence 149
1. Definitions 149
2. Uniqueness ........................................ . 152
3. Relations Between the Convergence Concepts .......... . 155
4. The Borel-Cantelli Lemmas and Complete Convergence .. 163
5. Convergence via Transforms ......................... . 169
6. The Law of Large Numbers and
the Central Limit Theorem .......................... . 172
7. Convergence of Sums of Sequences of Random Variables .. 177
8. Problems .......................................... . 186
VII. The Poisson Process. . . . . . . . .. . . . . . . . .. . . . . . . . . . .. 195
1. Introduction and Definitions .......................... 195
2. Restarted Poisson Processes .......................... 207
3. Conditioning on the Number of
Occurrences in an Interval ............................ 215
4. Conditioning on Occurrence Times .................... 220
5. Several Independent Poisson Processes ................. 222
Contents ix
6. Thinning of Poisson Processes ........................ 231
7. The Compound Poisson Process ....................... 236
8. Some Further Generalizations and Remarks ............. 238
9. Problems.... . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . .. 246
Appendixes ........... " . . . . . . . .. . . . . . . . . . . . . . . . . . . .. 253
1. Suggestions for Further Reading ....................... 253
2. Some Distributions and Their Characteristics ........... 257
3. Answers to Problems ................................ 263
Index ............................................... 271
Notation and Symbols
sample space
w elementary event
:F collection of events
I{A} indicator function of (the set) A
#{A} number of elements in (cardinality of)
(the set) A
complement of the set A
probability of A
X,Y,Z, ... random variables
F(x), Fx(x) distribution function (of X)
XEF X has distribution (function) F
C(Fx) the continuity set of Fx
p(x), px(x) probability function (of X)
I(x), Ix(x) density (function) (of X)
~(x) standard normal distribution function
4J(x) standard normal density (function)
Be(p) Bernoulli distribution
(J(r, s) beta distribution
Bin(n,p) binomial distribution
C(m,a) Cauchy distribution
x
2(n) chi-square distribution
o(a)
one-point distribution
Exp(a) exponential distribution
F(m,n) (Fisher's) F~distribution
Fs(p) first success distribution
r(p,a) gamma distribution
Ge(p) geometric distribution
H(N,n,p) hypergeometric distribution
L(a)
Laplace distribution
LN(Il,0'2) log-normal distribution
N(Il,0'2) normal distribution
