Table Of ContentPrinted in the USA
Universitext
Mario Lefebvre
Applied Stochastic
Processes
^ Springer
Mario Lefebvre
Departement de mathematiques
et de genie industriel
Ecole Polytechnique de Montreal, Quebec
C.R 6079, succ. Centre-ville
Montreal H3C 3A7
Canada
[email protected]
Mathematics Subject Classification (2000): 60-01, 60Gxx
Library of Congress Control Number: 2006935530
ISBN-10:0-387-3417M e-ISBN-10:0-387-48976-2
ISBN-13: 978-0-387-34171-2 e-ISBN-13: 978-0-387-48976-6
Printed on acid-free paper.
©2007 Springer Science+Business Media LLC
All rights reserved. This work may not be translated or copied in whole or in part
without the written permission of the publisher (Springer Science+Business Media LLC,
233 Spring Street, New York, NY 10013, U.S.A.), except for brief excerpts in connection
with reviews or scholarly analysis. Use in connection with any form of information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar
methodology now known or hereafter developed is forbidden.
The use in this publication of trade names, trademarks, service marks and similar terms,
even if they are not identified as such, is not to be taken as an expression of opinion as
to whether or not they are subject to proprietary rights.
9 87654321
springer.com (TXQ)
Life ^s most important questions are, for the most part,
nothing but probability problems.
Pierre Simon de Laplace
Preface
This book is based on the lecture notes that I have been using since 1988
for the course entitled Processus stochastiques at the Ecole Polytechnique de
Montreal. This course is mostly taken by students in electrical engineering
and applied mathematics, notably in operations research, who are generally
at the master's degree level. Therefore, we take for granted that the reader
is familiar with elementary probability theory. However, in order to write a
self-contained book, the first chapter of the text presents the basic results in
probability.
This book aims at providing the readers with a reference that covers the
most important subjects in the field of stochastic processes and that is ac
cessible to students who don't necessarily have a sound theoretical knowledge
of mathematics. Indeed, we don't insist very much in this volume on rigor
ous proofs of the theoretical results; rather, we spend much more time on
applications of these results.
After the review of elementary probability theory in Chapter 1, the remain
der of this chapter is devoted to random variables and vectors. In particular,
we cover the notion of conditional expectation, which is very useful in the
sequel.
The main characteristics of stochastic processes are given in Chapter 2.
Important properties, such as the concept of independent and stationary in
crements, are defined in Section 2.1. Next, Sections 2.2 and 2.3 deal with
ergodicity and stationarity, respectively. The chapter ends with a section on
Gaussian and Markovian processes.
Chapter 3 is the longest in this book. It covers the cases of both discrete-
time and continuous-time Markov chains. We treat the problem of calculating
the limiting probabilities of the chains in detail. Branching processes and birth
and death processes are two of the particular cases considered. The chapter
contains nearly 100 exercises at its end.
The Wiener process is the main subject of Chapter 4. Various processes
based on the Wiener process are presented as well. In particular, there are sub
sections on models such as the geometric Brownian motion, which is very im-
viii Preface
portant in financial mathematics, and the Ornstein-Uhlenbeck process. White
noise is defined, and first-passage problems are discussed in the last section
of the chapter.
In Chapter 5, the Poisson process, which is probably the most important
stochastic process for students in telecommunications, is studied in detail.
Several generalizations of this process, including nonhomogeneous Poisson
processes and renewal processes, can be found in this chapter.
Finally, Chapter 6 is concerned with the theory of queues. The models
with a single server and those with at least two servers are treated separately.
In general, we limit ourselves to the case of exponential models, in which both
the times between successive customers and the service times are exponential
random variables. This chapter then becomes an application of Chapter 3
(and 5).
In addition to the examples presented in the theory, the book contains ap
proximately 350 exercises, many of which are multiple-part problems. These
exercises are all problems given in exams or homework and were mostly cre
ated for these exams or homework. The answers to the even-numbered prob
lems are given in Appendix B.
Finally, it is my pleasure to thank Vaishali Damle, Julie Park, and Eliza
beth Loew from Springer for their work on this book.
Mario Lefebvre
Montreal, November 2005
Contents
Preface vii
List of Tables xi
List of Figures xiii
1 Review of Probability Theory 1
1.1 Elementary probability 1
1.2 Random variables 8
1.3 Random vectors 21
1.4 Exercises 34
2 Stochastic Processes 47
2.1 Introduction and definitions 47
2.2 Stationarity 52
2.3 Ergodicity 55
2.4 Gaussian and Markovian processes 58
2.5 Exercises 65
3 Markov Chains 73
3.1 Introduction 73
3.2 Discrete-time Markov chains 77
3.2.1 Definitions and notations 77
3.2.2 Properties 85
3.2.3 Limiting probabilities 94
3.2.4 Absorption problems 100
3.2.5 Branching processes 104
3.3 Continuous-time Markov chains 109
3.3.1 Exponential and gamma distributions 109
3.3.2 Continuous-time Markov chains 121
3.3.3 Calculation of the transition function Pij{t) 124
X Contents
3.3.4 Particular processes 129
3.3.5 Limiting probabilities and balance equations 138
3.4 Exercises 143
4 Diffusion Processes 173
4.1 The Wiener process 173
4.2 Diffusion processes 181
4.2.1 Brownian motion with drift 183
4.2.2 Geometric Brownian motion 185
4.2.3 Integrated Brownian motion 191
4.2.4 Brownian bridge 196
4.2.5 The Ornstein-Uhlenbeck process 199
4.2.6 The Bessel process 204
4.3 White noise 207
4.4 First-passage problems 214
4.5 Exercises 222
5 Poisson Processes 231
5.1 The Poisson process 231
5.1.1 The telegraph signal 248
5.2 Nonhomogeneous Poisson processes 250
5.3 Compound Poisson processes 254
5.4 Doubly stochastic Poisson processes 258
5.5 Filtered Poisson processes 264
5.6 Renewal processes 267
5.6.1 Limit theorems 278
5.6.2 Regenerative processes 284
5.7 Exercises 289
6 Queueing Theory 315
6.1 Introduction 315
6.2 Queues with a single server 319
6.2.1 The model M/M/1 319
6.2.2 The model M/M/l/c 327
6.3 Queues with many servers 332
6.3.1 The model M/M/s 332
6.3.2 The model M/M/s/c and loss systems 336
6.3.3 Networks of queues 342
6.4 Exercises 346
Appendix A: Statistical Tables 357
Appendix B: Answers to Even-Numbered Exercises 363
References 375
Index 377
List of Tables
1.1 Means, Variances, and Characteristic Functions of the Main
Random Variables 19
A.l. Distribution Function of the Binomial Distribution 358
A.2. Distribution Function of the Poisson Distribution 360
A.3. Distribution Function of the N(0,1) Distribution 361
