Probability and Measure

638 Pages·2012·38.78 MB·English

by  Patrick Billingsley

Probability
and Measure
Anniversary Edition
Patrick Billingsley
Probability
and Measure
WILEY SERIES IN PROBABILITY AND STATISTICS
Established by WALTER A. SHEWHART and SAMUEL S. WILKS
Editors: David J. Balding, Noel A. C. Cressie, Garrett M. Fitzmaurice,
Harvey Goldstein, Iain M. Johnstone, Geert Molenberghs, David Scott,
Adrian F. M. Smith, Ruey S. Tsay, Sanford Weisberg
Editors Emeriti: Vic Barnett, J. Stuart Hunter, Joseph B. Kadane, JozefL. Teugels
A complete list of the titles in this series appears at the end of this volume.
PROBABILITY
AND MEASURE
Anniversary Edition
Patrick Billingsley
University of Chicago
©WILEY
A John Wiley & Sons, Inc., Publication
Cover photo depicts the sun setting lightly on Camps Bay, Capetown South Africa by Margaret Quigley
([e-mail]).
Copyright © 1979, 1986, 1995, 2012 by John Wiley & Sons, Inc. All rights reserved
Published by John Wiley & Sons, Inc., Hoboken, New Jersey
Published simultaneously in Canada
Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in
preparing this book, they make no representations or warranties with respect to the accuracy or completeness of
the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a
particular purpose. No warranty may be created or extended by sales representatives or written sales materials.
The advice and strategies contained herein may not be suitable for your situation. You should consult with a
professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other
commercial damages, including but not limited to special, incidental, consequential, or other damages.
For general information on our other products and services or for technical support, please contact our Customer
Care Department within the United States at [phone], outside the United States at [phone] or fax
[phone].
Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be
available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com.
Library of Congress Cataloging-in-Publication Data:
Billingsley, Patrick
Probability and measure / Patrick Billingsley.—Anniversary ed.
p. cm.—(Wiley series in probability and mathematical
statistics. Probability and mathematical statistics)
“A Wiley-Interscience publication.”
Includes bibliographical references and index.
ISBN 0-471-00710-2 (3rd. ed.)
ISBN 978-1-118-12237-2 (anniversary ed.)
1. Probabilities. 2. Measure theory. I. Title. II. Series
QA273.B575 1995
519.2—dc20
Printed in the United States of America
10 98765432 1
12,571 P*, 39, 49 &193
2^,571 P(P|A), 54 F„ =» F, 202, 349, 4
0, 571 lim sup„ A„, 55 S°°, 28, 331
(a, b], 572 lim infnA„, 55 AA, 43
1a , 571 lim„ A„, 55 ■ .£, 43
|/|,2 i.o., 56 A*, 46
P, 2, 23 //, 305 *, 283
JR1, 572 Xn ^ p X , 75,286,.
J"nipA, 6 Rk, 575 Ilf II, 264
s„(w),6 [X = x], 72 \[f\\p, 256
N, 8, 380 a (X), 73, 272 ZF, 256
[xj, 572 jU, 78, 169, 273 Pn =>■ B, 350, 402
sgn x, 572 F[X], 81,291 Xn => X ,352,402
A — B, 571 Var[X], 83, 293 X„ =>■ o, 354
Ac, 571 £„[/], 93 3A, 573
A AB,571 sc(a),99 (p(t), 365
A C B,571 t, 106, 141, 493, 540 Bn ~>v B935
-set, 21 Pij, 118 Fs, 440
21 S, 117 rfF(x), 242
o (,s/), 23 a,, 118 X x Y,245
23 7T|, 131 BC x °>/, 246
38, 23 M(s), 154, 296, 303 /xxv, 247
(Q,^,P),25 9Ak, 167 Fac, 440
Ant A, 571 ,5/ n J20, 168 v « ix, 449
A„ | A, 571 x„ t *, 169, 572 dv/dn, 449
X, 27, 45, 177 * = 169 vs, 451
A, 572 B*, 174 Vac, 451
v, 572 175 P[A||.A], 455, 457
/W ), 35 Ai, 177 P[A||Xf,f € F], 460
P„(A),37 Afe, 181 F[X||// ], 473
D(A), 37 F, 185, 187 Pr, 515
Si, 37 A aF, 186 r^T,5i6
P*, 39, 49 T~lA', 572 VF„531
Contents
FOREWORD xi
PREFACE xiii
Patrick Billingsley 1925-2011
XV
Chapter 1
PROBABILITY i
1. BOREL’S NORMAL NUMBER THEOREM, 1
The Unit Interval—The Weak Law of Large
Numbers—The Strong Law of Large Numbers—Strong Law
Versus Weak—Length—The Measure Theory of Diophantine
Approximation*
2. PROBABILITY MEASURES, 18
Spaces —Assigning Probabilities—Classes of Sets—Probability
Measures—Lebesgue Measure on the Unit Interval—Sequence
Space*—Constructing cx-Fields*
3. EXISTENCE AND EXTENSION, 39
Construction of the Extension—Uniqueness and the
Theorem—Monotone Classes—Lebesgue Measure on the Unit
Interval—Completeness—Nonmeasurable Sets—Two
Impossibility Theorems*
4. DENUMERABLE PROBABILITIES, 53
General Formulas—Limit Sets—Independent
Events—Su bfields—The Borel-Cantell i
Lemmas—The Zero-One Law
5. SIMPLE RANDOM VARIABLES, 72
Definition—Convergence of Random
Variables—Independence—Existence of Independent
Sequences—Expected Value—Inequalities
‘Asterisks indicate topics that may be omitted on a first reading.
v
VI CONTENTS
6. THE LAW OF LARGE NUMBERS, 90
The Strong Law—The Weak Law—Bernstein’s
Theorem—A Refinement of the Second Borel-Cantelli
Lemma
7. GAMBLING SYSTEMS, 98
Gambler’s Ruin—Selection Systems—Gambling Policies—Bold
Play*—Timid Play*
8. MARKOV CHAINS, 117
Definitions—Higher-Order Transitions—An Existence
Theorem—Transience and Persistence—Another Criterion for
Persistence—Stationary Distributions—Exponential
Convergence*—Optimal Stopping*
9. LARGE DEVIATIONS AND THE LAW
OF THE ITERATED LOGARITHM, 154
Moment Generating Functions—Large Deviations—Chernoff’s
Theorem*—The Law of the Iterated Logarithm
Chapter 2
MEASURE 167
10. GENERAL MEASURES, 167
Classes of Sets—Conventions Involving
oo—Measures—Uniqueness
11. OUTER MEASURE, 174
Outer Measure—Extension—An Approximation Theorem
12. MEASURES IN EUCLIDEAN SPACE, 181
Lebesgue Measure—Regularity—Specifying Measures on the
Line—Specifying Measures in F^—Strange Euclidean Sets*
13. MEASURABLE FUNCTIONS AND MAPPINGS, 192
Measurable Mappings—Mappings into F^—Limits and
Measurability—Transformations of Measures
14. DISTRIBUTION FUNCTIONS, 198
Distribution Functions—Exponential Distributions—Weak
Convergence—Convergence of Types* —Extremal
Distributions*
CONTENTS VI
Chapter
INTEGRATION 211
15. THE INTEGRAL, 211
Definition—Nonnegative Functions—Uniqueness
16. PROPERTIES OF THE INTEGRAL, 218
Equalities and Inequalities—Integration to the Limit—Integration
over Sets—Densities—Change of Variable—Uniform
Integrability—Complex Functions
17. THE INTEGRAL WITH RESPECT TO LEBESGUE MEASURE, 234
The Lebesgue Integral on the Line—The Riemann
Integral—The Fundamental Theorem of Calculus—Change of
Variable—The Lebesgue Integral in Stieltjes Integrals
18. PRODUCT MEASURE AND FUBINI’S THEOREM, 245
Product Spaces—Product Measure—Fubini’s
Theorem—Integration by Parts—Products of Higher Order
19. THE L.P SPACES*, 256
Definitions—Completeness and Separability—Conjugate
Spaces—Weak Compactness—Some Decision
Theory—The Space L2— An Estimation Problem
4
Chapter
RANDOM VARIABLES AND EXPECTED VALUES 271
20. RANDOM VARIABLES AND DISTRIBUTIONS, 271
Random Variables and Vectors—
Subfields—Distributions—Multidimensional
Distributions—Independence—Sequences of Random
Variables—Convolution—Convergence in
Probability—The Glivenko-Cantelli Theorem*
21. EXPECTED VALUES, 291
Expected Value as Integral—Expected Values
and Limits—Expected Values and
Distributions—Moments—Inequalities—Joint
Integrals—Independence and Expected Value—Moment
Generating Functions
VI CONTENTS
22. SUMS OF INDEPENDENT RANDOM VARIABLES, 300
The Strong Law of Large Numbers—The Weak Law
and Moment Generating Functions—Kolmogorov’s Zero-One
Law—Maximal Inequalities—Convergence of Random
Series—Random Taylor Series*
23. THE POISSON PROCESS, 316
Characterization of the Exponential Distribution—The Poisson
Process—The Poisson Approximation—Other Characterizations
of the Poisson Process—Stochastic
Processes
24. THE ERGODIC THEOREM*, 330
Measure-Preserving Transformations—Ergodicity—Ergodicity of
Rotations—Proof of the Ergodic Theorem—The
Continued-Fraction T ransformation — Diophantine
Approximation
Chapter 5
CONVERGENCE OF DISTRIBUTIONS 349
25. WEAK CONVERGENCE, 349
Definitions—Uniform Distribution Modulo 1*—Convergence
in Distribution—Convergence in Probability—Fundamental
Theorems—Helly’s Theorem —Integration to the Limit
26. CHARACTERISTIC FUNCTIONS, 365
Definition—Moments and Derivatives—
Independence—Inversion and the Uniqueness Theorem—The
Continuity Theorem —Fourier Series*
27. THE CENTRAL LIMIT THEOREM, 380
Identically Distributed Summands—The Lindeberg
and Lyapounov Theorems—Dependent Variables*
28. INFINITELY DIVISIBLE DISTRIBUTIONS*, 394
Vague Convergence—The Possible Limits—Characterizing
the Limit
29. LIMIT THEOREMS IN Rk, 402
The Basic Theorems—Characteristic Functions—Normal
Distributions in Rk—The Central Limit Theorem