Table Of ContentSTATISTICS
• ©
& s <•
WILLIAM F. STOUT
Almost Sure Convergence
Probability and Mathematical Statistics
A Series of Monographs and Textbooks
Editors Z. W. Birnbaum E. Lukács
University of Washington Bowling Green State University
, ,
Seattle Washington Bowling Green Ohio
1. Thomas Ferguson. Mathematical Statistics: A Decision Theoretic Approach. 1967
2. Howard Tucker. A Graduate Course in Probability. 1967
3. K. R. Parthasarathy. Probability Measures on Metric Spaces. 1967
4. P. Revesz. The Laws of Large Numbers. 1968
5. H. P. McKean, Jr. Stochastic Integrals. 1969
6. B. V. Gnedenko, Yu. K. Belyayev, and A. D. Solovyev. Mathematical Methods of
Reliability Theory. 1969
7. Demetrios A. Kappos. Probability Algebras and Stochastic Spaces. 1969
8. Ivan N. Pesin. Classical and Modern Integration Theories. 1970
9. S. Vajda. Probabilistic Programming. 1972
10. Sheldon M. Ross. Introduction to Probability Models. 1972
11. Robert B. Ash. Real Analysis and Probability. 1972
12. V. V. Fedorov. Theory of Optimal Experiments. 1972
13. K. V. Mardia. Statistics of Directional Data. 1972
14. H. Dym and H. P. McKean. Fourier Series and Integrals. 1972
15. Tatsuo Kawata. Fourier Analysis in Probability Theory. 1972
16. Fritz Oberhettinger. Fourier Transforms of Distributions and Their Inverses: A
Collection of Tables. 1973
17. Paul Erdos and Joel Spencer. Probabilistic Methods in Combinatorics. 1973
18. K. Sarkadi and I. Vincze. Mathematical Methods of Statistical Quality Control. 1973
19. Michael R. Anderberg. Cluster Analysis for Applications. 1973
20. W. Hengartner and R. Theodorescu. Concentration Functions. 1973
21. Kai Lai Chung. A Course in Probability Theory, Second Edition. 1974
22. L. H. Koopmans. The Spectral Analysis of Time Series. 1974
23. L. E. Maistrov. Probability Theory: A Historical Sketch. 1974
24. William F. Stout. Almost Sure Convergence. 1974
In Preparation
Z. Govindarajulu. Sequential Statistical Procedures
E. J. McShane. Stochastic Calculus and Stochastic Models
Roger Cuppens. Decomposition of Multivariate Probabilities
Almost Sure
Convergence
WILLIAM F. STOUT
Department of Mathematics
University of Illinois at Urbana-Champaign
Urbana, Illinois
1974
ACADEMIC PRESS New York San Francisco London
,
A Subsidiary of Harcourt Brace Jovanovich Publishers
Copyright © 1974, by Academic Press, Inc.
ALL RIGHTS RESERVED.
NO PART OF THIS PUBLICATION MAY BE REPRODUCED OR
TRANSMITTED IN ANY FORM OR BY ANY MEANS, ELECTRONIC
OR MECHANICAL, INCLUDING PHOTOCOPY, RECORDING, OR ANY
INFORMATION STORAGE AND RETRIEVAL SYSTEM, WITHOUT
PERMISSION IN WRITING FROM THE PUBLISHER.
ACADEMIC PRESS, INC.
Ill Fifth Avenue, New York, New York 10003
United Kingdom Edition published by
ACADEMIC PRESS, INC. (LONDON) LTD.
24/28 Oval Road, London NW1
Library of Congress Cataloging in Publication Data
Stout, William F
Almost sure convergence.
(Probability and mathematical statistics)
Bibliography: p.
1. Random variables. 2. Sequences (Mathematics)
3. Partial sums (Series) I. Title.
QA273.5.S76 519.2'4 73-2075
ISBN 0-12-672750-3
AMS(MOS) 1970 Subject Classifications: 60F15, 60G45, 60G50, 60-02
PRINTED IN THE UNITED STATES OF AMERICA
Contents
Preface........................................................................................................................................ vii
Glossary of Symbols and Conventions............................................................................. ix
Chapter 1 Introduction
1.1. Delineation of the Subject.......................................................................................... 1
1.2. A Brief Chapter by Chapter Outline of Topics Covered.............................. 3
1.3. Methodology................................................................................................................... 7
1.4. Applications to Fields outside Probability............................................................ 8
Chapter 2 Pointwise Convergence of Partial Sums
2.1. Application of the Borel-Cantelli Lemma and the Chebyshev Inequality
to Almost Sure Convergence............................................................................. 10
2.2. Almost Sure Convergence of Subsequences of {Sn, n > 1} when the Are
Orthogonal................................................................................................................... 14
2.3. The Rademacher-Menchoff Fundamental Convergence Theorem................. 17
2.4. Convergence of {Sn, n > 1} Assuming Only Moment Restrictions. . . . 23
2.5. Martingale Difference Sequences............................................................................. 30
2.6. Preservation of the Martingale and Submartingale Structure...................... 36
2.7. The Upcrossings Inequality and Basic Convergence Theorem for Submartin
gales ............................................................................................................................ 40
2.8. Applications of the Basic Submartingale Convergence Theorem................. 44
2.9. Sums of Squares and Martingale Transforms................................................... 70
2.10. Convergence of Martingale Transforms Satisfying a Local Regularity Condi
tion ................................................................................................................................ 80
2.11. Global Regularity ....................................................................................................... 91
2.12. Convergence of Partial Sums of Independent Random Variables . . . . 93
2.13. Centering at Medians, the Levy-Type Inequality, and Symmetrization. . . 100
v
VI CONTENTS
Chapter 3 Stability and Moments
3.1. Introduction......................................................................................................................114
3.2. Stability Results for Independent Identically Distributed Random Variables 116
3.3. Stability Results for Martingales...............................................................................137
3.4. Stability Results for Independent Random Variables.......................................158
3.5. Stability Results for Stationary Random Variables.................................................167
3.6. Application of Stationarity to Stability Results for Markov Sequences. . . 185
3.7. Stability Results for {Sn, n > 1} Assuming Only Moment Restrictions 200
Chapter 4 Stability of Weighted Sums of Random Variables
4.1. Independent Identically Distributed Xk .......................................................................219
4.2. Other Dependence Structures.................................................... 234
Chapter 5 Exponential Inequalities, the Law of the Iterated
Logarithm, and Related Results
5.1. Introduction...........................................................................................................................255
5.2. The Kolmogorov-Prokhorov Exponential Inequalities with Applications. . 260
5.3. An Almost Sure Invariance Principle with Applications........................................280
5.4. A Survey of Further Results Related to the Law of the Iterated Logarithm 298
Chapter 6 Recurrence of {Sn, n > 1} and Related Results
6.1. Recurrence of {Sn,n > 1} in the Case of Independent Identically Dis
tributed X i..................................................................' .................................................331
6.2. Generalizations of the Results of Section 6.1 to Other Stochastic Structures 352
6.3. Recurrence of {Sn/na9n > 1} in the Case of Independent Identically Dis
tributed X i...........................................................................................................................358
References...................................................................................................................................367
Index 375
Preface
Almost sure behavior of partial sums of random variables, the subject
of this book, has enjoyed both a rich classical period and a recent resurgence
of research activity. This is nicely illustrated by the law of the iterated
logarithm for partial sums, the subject of Chapter 5: Attempts to sharpen
Borel’s [1909] strong law of large numbers culminated in Khintchine’s
[1924] law of the iterated logarithm for Bernoulli random variables. The
Kolmogorov [1929] and the Hartman-Wintner [1941] extensions of Khint
chine’s result to large classes of independent random variables were mile
stones in the classical theory. Levy’s (1937, see [1954]) law of the iterated
logarithm for martingales, an important class of dependent random vari
ables, was another major advance in the classical theory.
The modern period for the law of the iterated logarithm was started by
Strassen ([1964], [1965], [1966]) with his discovery of almost sure invari
ance principles, his deep functional law of the iterated logarithm, and his
converse to the Hartman-Wintner law of the iterated logarithm (this last
occurring, remarkably, a quarter century after the Hartman-Wintner result).
As Chapter 5 indicates, one of the characteristics of the modern period
has been an emphasis on laws of the iterated logarithm for dependent
random variables.
Because of the rich history and current interest in the law of the iterated
logarithm and in other areas of almost sure behavior, it seems desirable to
have a monograph which treats almost sure behavior in a systematic and
unified manner. This book presents such a treatment of the law of the
iterated logarithm and of four other major varieties of almost sure behavior:
almost sure convergence of partial sums (Chapter 2), almost sure stability
of partial sums (Chapter 3), almost sure stability of weighted partial sums
(Chapter 4), and recurrence of partial sums (Chapter 6).
Subdivision into major topics within chapters is usually done on the
basis of dependence structure. For example, the law of the iterated logarithm
vii
viii PREFACE
for partial sums of independent random variables, which is developed in
Sections 5.1-5.3 of Chapter 5, is such a major topic. The treatment of
each major topic starts with more elementary and usually classical theorems
and proceeds with more advanced and often recently established theorems.
Since the book’s viewpoint is probabilistic, certain mathematically
important topics, although logically fitting under the umbrella of almost
sure behavior, are omitted because they are not probabilistic in nature.
Examples of such excluded topics are the almost everywhere convergence
of Fourier series and the pointwise ergodic theory of operators in Lv spaces.
Topics of probabilistic interest have often been excluded as well, thereby
keeping the book from being too long.
The book was developed from a course on almost sure behavior given
by the author. This course was given as a sequel to the basic graduate level
probability course at the University of Illinois. The book contains ample
material for a one semester course. It assumes familiarity with basic real
analysis and basic measure theory (such as provided by Royden’s “Real
Analysis”) and with basic measure theoretic probability (such as provided
by Ash [1972, Chapters 5-8] or Chung [1968]). The book should also prove
useful for independent study, either to be read systematically or to be used
as a reference.
A glossary of symbols and conventions is provided for the reader’s
convenience. This should be especially helpful when the book is used as a
reference rather than systematically read. Results are numbered by chapter
and section. For instance, Example 2.8.11 is the eleventh of the examples in
Section 8 of Chapter 2. Chapters are somewhat independent; hence each
may be read without extensive reference to previous chapters. Exercises
are included to help the reader develop a working familiarity with the
subject and to provide additional information on the subject. Numerous
references are given to stimulate further reading.
I especially thank R. J. Tomkins for reading the entire manuscript,
making many valuable suggestions, and spotting numerous errors. I thank
R. B. Ash, J. L. Doob, and W. Philipp for reading portions of the manuscript
and making valuable suggestions. Carolyn Bloemker’s excellent typing was
greatly appreciated.
The writing of this book was partially supported by the National Sci
ence Foundation.
Glossary of Symbols
and Conventions
(A p) Underlying probability space
,
EX E(X) Expectation of X
{Xui> \} Basic sequence of random variables
{Sn,n > 1} sB = SJL, Xi
Ac Complement of A
A = В P[A n Bc] + P[AC n B] = 0, unless explicitly stated
otherwise
A A В (A D Bc) u (Ac n B)
А с В P[A n Bc] = 0
A implies В A c B
ап^ а | a | = oo not allowed, unless explicitly stated otherwise
On A, ... A implies ...
)
co\(X, Y Covariance of X and Y
var( Y) Variance of Y
Rn «-dimensional Euclidean space
R One-dimensional Euclidean space
Roo Infinite-dimensional Euclidean space
Borel sets of Rn
&00 Borel sets of R^
a € A} o field generated by {Ya, a e A}
&oo &{Xl9X%,...}
Occurrence See parenthetical remark, p. 1
a.s. Almost surely
, n > 1} adapted to ^ increasing o fields, c: Yn is ^ measurable
{У»,я> 1}
{Уп,$*,п > 1} adapted Same as above
stochastic sequence
{!0 ,Q)
1(A) Indicator function of event A
i.o. Infinitely often
An i.o. f)£iU n=kAn
fi(X) Median of X
x+
max(X, 0)
IX
