Table Of ContentSpringer Series in Statistics
Advisors:
P. Bickel, P. Diggle, S. Fienberg, K. Krickeberg,
I. Olkin, N. Wermuth, S. Zeger
Springer Science+Business Media, LLC
Springer Series in Statistics
Andersen/Borgan/Gil/IKeiding: Statistical Models Based on Counting Processes.
Atkinson/Riani: Robust Diagnostic Regression Analysis.
Berger: Statistical Decision Theory and Bayesian Analysis, 2nd edition.
BorglGroenen: Modem Multidimensional Scaling: Theory and Applications
BrockwelllDavis: Time Series: Theory and Methods, 2nd edition.
Chan/Tong: Chaos: A Statistical Perspective.
Chen/ShaoIIbrahim: Monte Carlo Methods in Bayesian Computation.
David/Edwards: Annotated Readings in the History of Statistics.
DevroyelLugosi: Combinatorial Methods in Density Estimation.
Efromovich: Nonparametric Curve Estimation: Methods, Theory, and Applications.
EggermontlLaRiccia: Maximum Penalized Likelihood Estimation, Volume I:
Density Estimation.
FahrmeirlTutz: Multivariate Statistical Modelling Based on Generalized Linear
Models, 2nd edition.
Fan/Yao: Nonlinear Time Series: Nonparametric and Parametric Methods.
Farebrother: Fitting Linear Relationships: A History of the Calculus of Observations
1750-1900.
Federer: Statistical Design and Analysis for Intercropping Experiments, Volume I:
Two Crops.
Federer: Statistical Design and Analysis for Intercropping Experiments, Volume II:
Three or More Crops.
GhoshiRamamoorthi: Bayesian Nonparametrics.
GlaziNauslWallenstein: Scan Statistics.
Good: Permutation Tests: A Practical Guide to Resampling Methods for Testing
Hypotheses, 2nd edition.
Gourieroux: ARCH Models and Financial Applications.
Gu: Smoothing Spline ANOVA Models.
Gyorj'zlKohlerlKrzyzak/ Walk: A Distribution-Free Theory of Nonparametric
Regression.
Haberman: Advanced Statistics, Volume I: Description of Populations.
Hall: The Bootstrap and Edgeworth Expansion.
Hardie: Smoothing Techniques: With Implementation in S.
Harrell: Regression Modeling Strategies: With Applications to Linear Models,
Logistic Regression, and Survival Analysis
Hart: Nonparametric Smoothing and Lack-of-Fit Tests.
HastieiTibshiranilFriedman: The Elements of Statistical Leaming: Data Mining,
Inference, and Prediction
HedayatiSloanelStujken: Orthogonal Arrays: Theory and Applications.
Heyde: Quasi-Likelihood and its Application: A General Approach to Optimal
Parameter Estimation.
HuetlBouvierlGruetlJolivet: Statistical Tools for Nonlinear Regression: A Practical
Guide with S-PLUS Examples.
Ibrahim/Chen/Sinha: Bayesian Survival Analysis.
Jolliffe: Principal Component Analysis.
(continued after index)
S.N. Lahiri
Resampling Methods
for Dependent Data
With 25 Illustrations
, Springer
S.N. Lahiri
Department of Statistics
Iowa State University
Ames, IA 50011-1212
USA
Library of Congress Cataloging-in-Publication Data
Labiri, S.N.
Resampling metbods for dependent data / S.N. Labiri.
p. cm. - (Springer series in statistics)
Includes bibliographical references and index.
ISBN 978-1-4419-1848-2 ISBN 978-1-4757-3803-2 (eBook)
DOI 10.1007/978-1-4757-3803-2
1. Resampling (Statistics) 1. Title. II. Series.
QA278.8.L344 2003
519.5'2-dc21 2003045455
ISBN 978-1-4419-1848-2 Printed on acid-free paper.
© 2003 Springer Science+Business Media New York
Originally published by Springer-Verlag New York, Inc. in 2003
Softcover reprint ofthe hardcover Ist edition 2003
AlI rights reserved. This work may not be translated or copied in whole or in part without tthhee
written permission of tbe publisher Springer Science+Business Media, LLC,
except for brief excerpts in connection with reviews or scholarly analysis. Use
in connection with any form of information storage and retrieval, electronic adaptation, computer
software, or by similar or dis similar methodology now known or hereafter developed is forbidden.
The use in this publication of trade names, trademarks, service marks, and similar terms, even iiff
they are not identified as such, is not to be taken as an expression of opinion as to whether or nnoott
they are subject to proprietary rights.
987 6 5 4 3 2 l SPIN 10922705
Typesetting: Pages created by the author using a Springer TEX macro package.
www.springer-ny.com
To my parents
Preface
This is a book on bootstrap and related resampling methods for temporal
and spatial data exhibiting various forms of dependence. Like the resam
pling methods for independent data, these methods provide tools for sta
tistical analysis of dependent data without requiring stringent structural
assumptions. This is an important aspect of the resampling methods in the
dependent case, as the problem of model misspecification is more preva
lent under dependence and traditional statistical methods are often very
sensitive to deviations from model assumptions. Following the tremendous
success of Efron's (1979) bootstrap to provide answers to many complex
problems involving independent data and following Singh's (1981) example
on the inadequacy of the method under dependence, there have been several
attempts in the literature to extend the bootstrap method to the dependent
case. A breakthrough was achieved when resampling of single observations
was replaced with block resampling, an idea that was put forward by Hall
(1985), Carlstein (1986), Kiinsch (1989), Liu and Singh (1992), and others
in various forms and in different inference problems. There has been a vig
orous development in the area of res amp ling methods for dependent data
since then and it is still an area of active research. This book describes
various aspects of the theory and methodology of resampling methods for
dependent data developed over the last two decades.
There are mainly two target audiences for the book, with the level of
exposition of the relevant parts tailored to each audience. The first five
chapters of the book are written in a pedantic way, giving full details of
the proofs of the theoretical results and step-by-step instructions for im
plementation of the methodology. This part of the book, together with
VIII Preface
selected material from the later chapters, can be used as a text for a grad
uate level course. For the first part, familiarity with only basic concepts of
theoretical Statistics is assumed. In particular, no prior exposure to Time
Series is needed. The second part of the book (Chapters 6-12) is written
in the form of a research monograph, with frequent reference to the litera
ture for the proofs and for further ramification of the topics covered. This
part is primarily intended for researchers in Statistics and Econometrics,
who are interested in learning about the recent advances in this area, or
interested in applying the methodology in their own research. A third po
tential audience is the practitioners, who may go over the descriptions of
the resampling methods and the worked out numerical examples, but skip
the proofs and other technical discussions. Many of the results presented in
the book are from preprints of papers and are yet to appear in a published
medium. Furthermore, some (potential) open problems have been pointed
out.
Chapter 1 gives a brief description of the "bootstrap principle" and ad
vocates resampling methods, at a heuristic level, as general methods for
estimating what are called "level-2" (and "higher-level") parameters in the
book. Chapter 2 sketches the historical development of bootstrap methods
since Efron's (1979) seminal work and describes various types of bootstrap
methods that have been proposed in the context of dependent (temporal)
data. Chapter 3 establishes consistency of various block bootstrap meth
ods for estimating the variance and the distribution function of the sample
mean. Chapter 4 extends these results to general classes of statistics, in
cluding M-estimators and differentiable statistical functionals, and gives
a number of numerical examples. Chapter 5 starts with a numerical com
parison of different block bootstrap methods and follows it up with some
theoretical results. Chapter 6 deals with Edgeworth expansions and second
order properties of block bootstrap methods for normalized and studentized
statistics under dependence. Chapter 7 addresses the important problem
of selecting the optimal block size empirically. Chapter 8 treats bootstrap
based on independent and identically distributed innovations in popular
time series models, such as the autoregressive processes. Chapter 9 deals
with the frequency domain bootstrap. Chapter 10 describes properties of
block bootstrap and subsampling methods for a class of long-range depen
dent processes. Chapter 11 treats two special topics - viz., extremums of
dependent random variables and sums of heavy-tailed dependent random
variables. As in the independent case, here the block bootstrap fails if the
resample size equals the sample size. A description of the random limit is
given in these problems, but the proofs are omitted. Chapter 12 consid
ers resampling methods for spatial data under different spatial sampling
designs. It also treats the problem of spatial prediction using resampling
methods. A list of important definitions and technical results are given in
Appendix A, which a reader may consult to refresh his or her memory.
Preface IX
I am grateful to my colleagues, coauthors, and teachers, A. Bose, K.B.
Athreya, G.J. Bahu, N. Cressie, A. C. Davison, P. Hall, J. Horowitz, D.
Isaacson, B. Y. Jing, H. Koul, D. Politis, and A. Young for their interest,
encouragement, and constructive suggestions at various stages of writing
the hook. Special thanks are due to K. Furukawa for help with the nu
merical examples and to D. Nordman for carefully going over parts of the
manuscript. I also thank J. Fukuchi, Y. D. Lee, S. Sun, and J. Zhu who
have enriched my research on the topic as students at various time points.
I thank my wife for her moral support and understanding. Many thanks go
to Sharon Shepard for converting my scrihhlings into a typed manuscript
with extraordinary accuracy and consistency. I also thank Springer's Ed
itor, John Kimmel, for his patience and good humor over the long time
period of this project. I gratefully acknowledge the continuous support of
the National Science Foundation for my research work in this area.
Contents
1 Scope of Resampling Methods for Dependent Data 1
1.1 The Bootstrap Principle 1
1.2 Examples . . . . 7
1.3 Concluding Remarks 12
1.4 Notation..... 13
2 Bootstrap Methods 17
2.1 Introduction...................... 17
2.2 IID Bootstrap.. .................. 17
2.3 Inadequacy of IID Bootstrap for Dependent Data. 21
2.4 Bootstrap Based on IID Innovations 23
2.5 Moving Block Bootstrap . . . . . 25
2.6 Nonoverlapping Block Bootstrap 30
2.7 Generalized Block Bootstrap .. 31
2.7.1 Circular Block Bootstrap 33
2.7.2 Stationary Block Bootstrap 34
2.8 Subsampling ........... 37
2.9 Transformation-Based Bootstrap 40
2.10 Sieve Bootstrap. ........ 41
3 Properties of Block Bootstrap Methods for the Sample
Mean 45
3.1 Introduction..................... 45
3.2 Consistency of MBB, NBB, CBB: Sample Mean. 47
XII Contents
3.2.1 Consistency of Bootstrap Variance Estimators 48
3.2.2 Consistency of Distribution Function Estimators 54
3.3 Consistency of the SB: Sample Mean . . . . . . . . . . . 57
3.3.1 Consistency of SB Variance Estimators ..... 57
3.3.2 Consistency of SB Distribution Function Estimators 63
4 Extensions and Examples 73
4.1 Introduction ...... . 73
4.2 Smooth Functions of Means 73
4.3 M-Estimators ....... . 81
4.4 Differentiable Functionals . 90
4.4.1 Bootstrapping the Empirical Process. 92
4.4.2 Consistency of the MBB for Differentiable
Statistical Functionals 94
4.5 Examples . . . . . . . . . . . . . . . ... 99
5 Comparison of Block Bootstrap Methods 115
5.1 Introduction ........ . 115
5.2 Empirical Comparisons. . . 116
5.3 The Theoretical Framework 118
5.4 Expansions for the MSEs . 120
5.5 Theoretical Comparisons. . 123
5.5.1 Asymptotic Efficiency 123
5.5.2 Comparison at Optimal Block Lengths. 124
5.6 Concluding Remarks . . . . . . . . . . . . . . . 126
5.7 Proofs....................... 127
5.7.1 Proofs of Theorems 5.1-5.2 for the MBB, the NBB,
and the CBB . . . . . . . . . . . . . . 128
5.7.2 Proofs of Theorems 5.1-5.2 for the SB 135
6 Second-Order Properties 145
6.1 Introduction ...... . 145
6.2 Edgeworth Expansions for the Mean Under Independence 147
6.3 Edgeworth Expansions for the Mean Under Dependence 154
6.4 Expansions for Functions of Sample Means ..... . 160
6.4.1 Expansions Under the Smooth Function Model
Under Independence . . . . . . . . . . . . . 160
6.4.2 Expansions for Normalized and Studentized
Statistics Under Independence ........... . 163
6.4.3 Expansions for Normalized Statistics Under
Dependence . . . . . . . . . . . . . . . . . . 164
6.4.4 Expansions for Studentized Statistics Under
Dependence . . . . . . . . . . . . . . . . . . . 166
6.5 Second-Order Properties of Block Bootstrap Methods 168
