Markov Chains and Mixing Times

465 Pages·2017·11.12 MB·English

by  David A. Levin, Yuval Peres, Elizabeth L. Wilmer, James G. Propp, David B. Wilson

Markov Chains
and Mixing Times
Second Edition
David A. Levin
Yuval Peres
With contributions by
Elizabeth L. Wilmer
AMERICAN MATHEMATICAL SOCIETY
10.1090/mbk/107
Markov Chains
and Mixing Times
Second Edition
David A. Levin
University of Oregon
Yuval Peres
Microsoft Research
With contributions by
Elizabeth L. Wilmer
With a chapter on “Coupling from the Past” by
James G. Propp and David B. Wilson
AMERICAN MATHEMATICAL SOCIETY
Providence, Rhode Island
2010 Mathematics Subject Classification. Primary 60J10, 60J27, 60B15, 60C05, 65C05,
60K35, 68W20, 68U20, 82C22.
FRONT COVER: The figure on the bottom left of the front cover, courtesy of
David B. Wilson, is a uniformly random lozenge tiling of a hexagon (see Section
25.2). The figure on the bottom center, also from David B. Wilson, is a random
sample of an Ising model at its critical temperature (see Sections 3.3.5 and 25.2)
with mixed boundary conditions. The figure on the bottom right, courtesy of Eyal
Lubetzky, is a portion of an expander graph (see Section 13.6).
For additional informationand updates on this book, visit
www.ams.org/bookpages/mbk-107
Library of Congress Cataloging-in-Publication Data
Names: Levin, David Asher, 1971- | Peres, Y. (Yuval) | Wilmer, Elizabeth L. (Elizabeth Lee),
1970-|Propp,James,1960-|Wilson,DavidB.(DavidBruce)
Title: Markov chains and mixing times / David A. Levin, Yuval Peres ; with contributions by
ElizabethL.Wilmer.
Description: Secondedition. |Providence,RhodeIsland: AmericanMathematicalSociety,[2017]
| “With a chapter on Coupling from the past, by James G. Propp and David B. Wilson.” |
Includesbibliographicalreferencesandindexes.
Identifiers: LCCN2017017451|ISBN9781470429621(alk. paper)
Subjects: LCSH: Markov processes–Textbooks. | Distribution (Probability theory)–Textbooks.
| AMS: Probability theory and stochastic processes – Markov processes – Markov chains
(discrete-time Markov processes on discrete state spaces). msc | Probability theory and sto-
chastic processes – Markov processes – Continuous-time Markov processes on discrete state
spaces. msc | Probability theory and stochastic processes – Probability theory on algebraic
andtopologicalstructures–Probabilitymeasuresongroupsorsemigroups,Fouriertransforms,
factorization. msc | Probability theory and stochastic processes – Combinatorial probability
– Combinatorial probability. msc | Numerical analysis – Probabilistic methods, simulation
and stochastic differential equations – Monte Carlo methods. msc | Probability theory and
stochastic processes – Special processes – Interacting random processes; statistical mechan-
ics type models; percolation theory. msc | Computer science – Algorithms – Randomized
algorithms. msc | Computer science – Computing methodologies and applications – Simula-
tion. msc|Statisticalmechanics,structureofmatter–Time-dependentstatisticalmechanics
(dynamicandnonequilibrium)–Interactingparticlesystems. msc
Classification: LCCQA274.7.L482017|DDC519.2/33–dc23LCrecordavailableathttps://lccn.
loc.gov/2017017451
Copying and reprinting. Individual readers of this publication, and nonprofit libraries
acting for them, are permitted to make fair use of the material, such as to copy select pages for
useinteachingorresearch. Permissionisgrantedtoquotebriefpassagesfromthispublicationin
reviews,providedthecustomaryacknowledgmentofthesourceisgiven.
Republication,systematiccopying,ormultiplereproductionofanymaterialinthispublication
is permitted only under license from the American Mathematical Society. Permissions to reuse
portions of AMS publication content are handled by Copyright Clearance Center’s RightsLink(cid:2)
service. Formoreinformation,pleasevisit: http://www.ams.org/rightslink.
Sendreq[e-mail].
Excludedfromtheseprovisionsismaterialforwhichtheauthorholdscopyright. Insuchcases,
requestsforpermissiontoreuseorreprintmaterialshouldbeaddresseddirectlytotheauthor(s).
Copyrightownershipisindicatedonthecopyrightpage,oronthelowerright-handcornerofthe
firstpageofeacharticlewithinproceedingsvolumes.
Secondedition(cid:2)c 2017bytheauthors. Allrightsreserved.
Firstedition(cid:2)c 2009bytheauthors. Allrightsreserved.
PrintedintheUnitedStatesofAmerica.
(cid:2)∞ Thepaperusedinthisbookisacid-freeandfallswithintheguidelines
establishedtoensurepermanenceanddurability.
VisittheAMShomepageathttp://www.ams.org/
10987654321 171615141312
Contents
Preface ix
Preface to the Second Edition ix
Preface to the First Edition ix
Overview xi
For the Reader xii
For the Instructor xiii
For the Expert xiv
Acknowledgements xvi
Part I: Basic Methods and Examples 1
Chapter 1. Introduction to Finite Markov Chains 2
1.1. Markov Chains 2
1.2. Random Mapping Representation 5
1.3. Irreducibility and Aperiodicity 7
1.4. Random Walks on Graphs 8
1.5. Stationary Distributions 9
1.6. Reversibility and Time Reversals 13
1.7. Classifying the States of a Markov Chain* 15
Exercises 17
Notes 19
Chapter 2. Classical (and Useful) Markov Chains 21
2.1. Gambler’s Ruin 21
2.2. Coupon Collecting 22
2.3. The Hypercube and the Ehrenfest Urn Model 23
2.4. The P´olya Urn Model 25
2.5. Birth-and-Death Chains 26
2.6. Random Walks on Groups 27
2.7. Random Walks on Z and Reflection Principles 30
Exercises 34
Notes 35
Chapter 3. Markov Chain Monte Carlo: Metropolis and Glauber Chains 38
3.1. Introduction 38
3.2. Metropolis Chains 38
3.3. Glauber Dynamics 41
Exercises 45
Notes 45
iii
iv CONTENTS
Chapter 4. Introduction to Markov Chain Mixing 47
4.1. Total Variation Distance 47
4.2. Coupling and Total Variation Distance 49
4.3. The Convergence Theorem 52
4.4. Standardizing Distance from Stationarity 53
4.5. Mixing Time 54
4.6. Mixing and Time Reversal 55
4.7. (cid:2)p Distance and Mixing 56
Exercises 57
Notes 58
Chapter 5. Coupling 60
5.1. Definition 60
5.2. Bounding Total Variation Distance 61
5.3. Examples 62
5.4. Grand Couplings 69
Exercises 73
Notes 73
Chapter 6. Strong Stationary Times 75
6.1. Top-to-Random Shuffle 75
6.2. Markov Chains with Filtrations 76
6.3. Stationary Times 77
6.4. Strong Stationary Times and Bounding Distance 78
6.5. Examples 81
6.6. Stationary Times and Ces`aro Mixing Time 83
6.7. Optimal Strong Stationary Times* 84
Exercises 85
Notes 86
Chapter 7. Lower Bounds on Mixing Times 87
7.1. Counting and Diameter Bounds 87
7.2. Bottleneck Ratio 88
7.3. Distinguishing Statistics 91
7.4. Examples 95
Exercises 97
Notes 98
Chapter 8. The Symmetric Group and Shuffling Cards 99
8.1. The Symmetric Group 99
8.2. Random Transpositions 101
8.3. Riffle Shuffles 106
Exercises 109
Notes 112
Chapter 9. Random Walks on Networks 115
9.1. Networks and Reversible Markov Chains 115
9.2. Harmonic Functions 116
9.3. Voltages and Current Flows 117
CONTENTS v
9.4. Effective Resistance 118
9.5. Escape Probabilities on a Square 123
Exercises 125
Notes 126
Chapter 10. Hitting Times 127
10.1. Definition 127
10.2. Random Target Times 128
10.3. Commute Time 130
10.4. Hitting Times on Trees 133
10.5. Hitting Times for Eulerian Graphs 135
10.6. Hitting Times for the Torus 136
10.7. Bounding Mixing Times via Hitting Times 139
10.8. Mixing for the Walk on Two Glued Graphs 143
Exercises 145
Notes 147
Chapter 11. Cover Times 149
11.1. Definitions 149
11.2. The Matthews Method 149
11.3. Applications of the Matthews Method 151
11.4. Spanning Tree Bound for Cover Time 153
11.5. Waiting for All Patterns in Coin Tossing 155
Exercises 157
Notes 157
Chapter 12. Eigenvalues 160
12.1. The Spectral Representation of a Reversible Transition Matrix 160
12.2. The Relaxation Time 162
12.3. Eigenvalues and Eigenfunctions of Some Simple Random Walks 164
12.4. Product Chains 168
12.5. Spectral Formula for the Target Time 171
12.6. An (cid:2)2 Bound 171
12.7. Time Averages 172
Exercises 176
Notes 177
Part II: The Plot Thickens 179
Chapter 13. Eigenfunctions and Comparison of Chains 180
13.1. Bounds on Spectral Gap via Contractions 180
13.2. The Dirichlet Form and the Bottleneck Ratio 181
13.3. Simple Comparison of Markov Chains 185
13.4. The Path Method 188
13.5. Wilson’s Method for Lower Bounds 193
13.6. Expander Graphs* 196
Exercises 198
Notes 199
vi CONTENTS
Chapter 14. The Transportation Metric and Path Coupling 201
14.1. The Transportation Metric 201
14.2. Path Coupling 203
14.3. Rapid Mixing for Colorings 207
14.4. Approximate Counting 209
Exercises 213
Notes 214
Chapter 15. The Ising Model 216
15.1. Fast Mixing at High Temperature 216
15.2. The Complete Graph 219
15.3. The Cycle 220
15.4. The Tree 221
15.5. Block Dynamics 224
15.6. Lower Bound for the Ising Model on Square* 227
Exercises 229
Notes 230
Chapter 16. From Shuffling Cards to Shuffling Genes 233
16.1. Random Adjacent Transpositions 233
16.2. Shuffling Genes 237
Exercises 241
Notes 242
Chapter 17. Martingales and Evolving Sets 244
17.1. Definition and Examples 244
17.2. Optional Stopping Theorem 245
17.3. Applications 247
17.4. Evolving Sets 250
17.5. A General Bound on Return Probabilities 255
17.6. Harmonic Functions and the Doob h-Transform 257
17.7. Strong Stationary Times from Evolving Sets 258
Exercises 260
Notes 260
Chapter 18. The Cutoff Phenomenon 262
18.1. Definition 262
18.2. Examples of Cutoff 263
18.3. A Necessary Condition for Cutoff 268
18.4. Separation Cutoff 269
Exercises 270
Notes 270
Chapter 19. Lamplighter Walks 273
19.1. Introduction 273
19.2. Relaxation Time Bounds 274
19.3. Mixing Time Bounds 276
19.4. Examples 278
Exercises 278
Notes 279
CONTENTS vii
Chapter 20. Continuous-Time Chains* 281
20.1. Definitions 281
20.2. Continuous-Time Mixing 283
20.3. Spectral Gap 285
20.4. Product Chains 286
Exercises 290
Notes 291
Chapter 21. Countable State Space Chains* 292
21.1. Recurrence and Transience 292
21.2. Infinite Networks 294
21.3. Positive Recurrence and Convergence 296
21.4. Null Recurrence and Convergence 301
21.5. Bounds on Return Probabilities 302
Exercises 303
Notes 305
Chapter 22. Monotone Chains 306
22.1. Introduction 306
22.2. Stochastic Domination 307
22.3. Definition and Examples of Monotone Markov Chains 309
22.4. Positive Correlations 310
22.5. The Second Eigenfunction 314
22.6. Censoring Inequality 315
22.7. Lower Bound on d¯ 320
22.8. Proof of Strassen’s Theorem 321
Exercises 322
Notes 323
Chapter 23. The Exclusion Process 324
23.1. Introduction 324
23.2. Mixing Time of k-Exclusion on the n-Path 329
23.3. Biased Exclusion 330
Exercises 334
Notes 334
Chapter 24. Ces`aro Mixing Time, Stationary Times, and Hitting
Large Sets 336
24.1. Introduction 336
24.2. Equivalence of t , t , and t for Reversible Chains 338
stop Ces G
24.3. Halting States and Mean-Optimal Stopping Times 340
24.4. Regularity Properties of Geometric Mixing Times* 341
24.5. Equivalence of t and t 342
G H
24.6. Upward Skip-Free Chains 344
24.7. t (α) Are Comparable for α≤1/2 345
H
24.8. An Upper Bound on t 346
rel
24.9. Application to Robustness of Mixing 346
Exercises 347
Notes 348
viii CONTENTS
Chapter 25. Coupling from the Past 349
25.1. Introduction 349
25.2. Monotone CFTP 350
25.3. Perfect Sampling via Coupling from the Past 355
25.4. The Hardcore Model 356
25.5. Random State of an Unknown Markov Chain 358
Exercise 359
Notes 359
Chapter 26. Open Problems 360
26.1. The Ising Model 360
26.2. Cutoff 361
26.3. Other Problems 361
26.4. Update: Previously Open Problems 362
Appendix A. Background Material 365
A.1. Probability Spaces and Random Variables 365
A.2. Conditional Expectation 371
A.3. Strong Markov Property 374
A.4. Metric Spaces 375
A.5. Linear Algebra 376
A.6. Miscellaneous 376
Exercise 376
Appendix B. Introduction to Simulation 377
B.1. What Is Simulation? 377
B.2. Von Neumann Unbiasing* 378
B.3. Simulating Discrete Distributions and Sampling 379
B.4. Inverse Distribution Function Method 380
B.5. Acceptance-Rejection Sampling 380
B.6. Simulating Normal Random Variables 383
B.7. Sampling from the Simplex 384
B.8. About Random Numbers 384
B.9. Sampling from Large Sets* 385
Exercises 388
Notes 391
Appendix C. Ergodic Theorem 392
C.1. Ergodic Theorem* 392
Exercise 393
Appendix D. Solutions to Selected Exercises 394
Bibliography 425
Notation Index 439
Index 441
Preface
Preface to the Second Edition
Sincethepublicationofthefirstedition,thefieldofmixingtimeshascontinued
to enjoy rapid expansion. In particular, many of the open problems posed in the
first edition have been solved. The book has been used in courses at numerous
universities, motivating us to update it.
In the eight years since the first edition appeared, we have made corrections
andimprovementsthroughoutthebook. Weaddedthreenewchapters: Chapter22
on monotone chains, Chapter 23 on the exclusion process, and Chapter 24, which
relates mixing times and hitting time parameters to stationary stopping times.
Chapter 4 now includes an introduction to mixing times in (cid:2)p, which reappear in
Chapter 10. The latter chapter has several new topics, including estimates for hit-
tingtimesontreesand Euleriandigraphs. Abound for cover timesusing spanning
trees has been added to Chapter 11, which also now includes a general bound on
cover times for regular graphs. The exposition in Chapter 6 and Chapter 17 now
employs filtrations rather than relying on the random mapping representation. To
reflectthekeydevelopmentssincethefirstedition, especiallybreakthroughsonthe
Ising model and the cutoff phenomenon, the Notes at the end of chapters and the
open problems have been updated.
We thank the many careful readers who sent us comments and corrections:
Anselm Adelmann, Amitabha Bagchi, Nathanael Berestycki, Olena Bormashenko,
Krzysztof Burdzy, Gerandy Brito, Darcy Camargo, Varsha Dani, Sukhada Fad-
navis, Tertuliano Franco, Alan Frieze, Reza Gheissari, Jonathan Hermon, Ander
Holroyd, Kenneth Hu, John Jiang, Svante Janson, Melvin Kianmanesh Rad, Yin
Tat Lee, Zhongyang Li, Eyal Lubetzky, Abbas Mehrabian, R. Misturini, L. Mor-
gado, Asaf Nachmias, Fedja Nazarov, Joe Neeman, Ross Pinsky, Anthony Quas,
Miklos Racz, Dinah Shender, N. J. A. Sloane, Jeff Steif, Izabella Stuhl, Jan Swart,
RyokichiTanaka,DanielWu,andZhenZhu. WeareparticularlygratefultoDaniel
Jerison,PawelPralat,andPerlaSousi,whosentuslonglistsofinsightfulcomments.
Preface to the First Edition
Markov first studied the stochastic processes that came to be named after him
in 1906. Approximately a century later, there is an active and diverse interdisci-
plinarycommunityofresearchersusingMarkovchainsincomputerscience,physics,
statistics, bioinformatics, engineering, and many other areas.
The classical theory of Markov chains studied fixed chains, and the goal was
to estimate the rate of convergence to stationarity of the distribution at time t, as
t → ∞. In the past two decades, as interest in chains with large state spaces has
increased, a different asymptotic analysis has emerged. Some target distance to
ix