APM 541

92 Pages·2013·1.67 MB·English

by  Jay Taylor

APM 541: Stochastic Modelling in Biology
Discrete-time Markov Chains
Jay Taylor
Fall 2013
JayTaylor (ASU) APM541 Fall2013 1/92
;
Outline
1 Motivation
2 Markov Processes
3 Markov Chains: Basic Properties
4 Asymptotics: Class Structure
5 Asymptotics: Hitting Times and Probabilities
6 Asymptotics: Stationary Distributions
7 Time Reversal
8 Hidden Markov Models
JayTaylor (ASU) APM541 Fall2013 2/92
Motivation
Stochastic Processes
Definition
A stochastic process is simply a collection of random variables {X :t ∈T} indexed by
t
some set T, where all of the variables are defined on the same underlying probability
space. We say that the process is E-valued if all of the variables take values in the set
E. We say that the process is a discrete-time stochastic process if the index set T is a
discrete subset of R, e.g., T ⊂Z, or a continuous-time stochastic process if the index
set is an interval T =(a,b)⊂R.
Interpretation: Usually, T ⊂R is a subset of the real line and we think of the X as
t
specifying the state of some random system at time t ∈T. Thus the stochastic process
as a whole describes the noisy or random evolution of the system over time.
JayTaylor (ASU) APM541 Fall2013 3/92
Motivation
To motivate our consideration of Markov processes, let us compare two models of
density-dependent population regulation in a finite population of an asexual (or
hermaphroditic) species. In both models, we will assume that reproduction is seasonal
and we will let X denote the number of individuals alive at the beginning of year t.
t
Model 1 will make the following assumptions:
Individuals become reproductively mature in the year following their birth, i.e., an
individual born in year t will begin to reproduce in year t+1.
Conditional on the current population size X , the number of surviving offspring
t
born to an individual alive at time t is Poisson-distributed with mean
rexp(−γ X ). Furthermore, the numbers of offspring born to different individuals
b t
are independent of one another.
Following reproduction, each individual dies, independently of the rest, with
probability 1−(1−µ)exp(−γ X ).
d t
JayTaylor (ASU) APM541 Fall2013 4/92
Motivation
Assume that the individuals alive at the beginning of time t are labeled i =1,··· ,X
t
and let η denote the number of offspring born to the i’th individual in that season.
i,t
Since η ,··· ,η are independent Poisson-distributed random variables, it follows
1,t Xt,t
that if we condition on X =n, then the total number of offspring born at time t is also
t
Poisson-distributed, with mean nre−γbn:
n
η ≡Xη ∼ Poisson`nre−γbn´.
t i,t
i=1
Furthermore, under this same condition, the number of surviving adults S is
t
binomially-distributed with parameters n and (1−µ)e−γdn:
S ∼ Binomial`n,(1−µ)e−γdn´.
t
Combining these two observations, it follows that the number of individuals alive at the
beginning of time t+1 is
X =S +η .
t+1 t t
JayTaylor (ASU) APM541 Fall2013 5/92
Motivation
This process can be simulated using the following procedure:
1 Ask the user to input values for the parameters r,µ,γb,γd as well as the initial
number of individuals X alive at time t =0.
0
2 Given that Xt =n, generate independent random variables
ηt ∼ Poisson`nre−γbn´ and St ∼ Binomial`n,(1−µ)e−γdn´.
3 Set Xt+1 =St+ηt.
4 Increase t to t+1 and return to step 2.
Several features of this model contribute to the ease with which the process can be
simulated. One is that we are able to directly sample the total number of individuals
born in each season, rather than having to sample each of the individual offspring
numbers separately. Equally importantly, todeterminethe numberof individuals alive at
time t+1, we only need to know the number alive at time t.
JayTaylor (ASU) APM541 Fall2013 6/92
Motivation
In the second model, we will assume that the density of the population at time t affects
not only the survival and reproduction of adults alive at that time, but also the
developmentofanyjuvenilesthatareborninthatyear. Specifically,wewillassumethat
density has a lasting negative effect on the future reproduction and survival of such
juveniles, as described below:
Supposethatanindividualisτ yearsold. ConditionalonthepopulationdensityX
t
in the present and on the population density X in the year when that individual
t−τ
was born, the number of surviving offspring born to that individual at time t is
Poisson-distributed with mean
re−γbXte−λbXt−τ.
Following reproduction, the probability that this individual dies is
1−µe−γdXte−λdXt−τ.
Apart from these modifications, we will assume that Model 2 coincides with Model 1.
JayTaylor (ASU) APM541 Fall2013 7/92
Motivation
As innocuous as these modifications may seem, they make the task of simulating Model
2 much more computationally demanding than Model 1. Specifically, we must deal with
the following complications:
Because reproduction and survival now depend on the density of the population in
each individual’s natal year, we must keep track of the ages of each individual in
the population so that we can determine when they were born. This also
necessitates sampling Poisson and binomially-distributed random variables for
every age class rather than for the population as a whole.
Moreseriously,wemustknowthepopulationdensitynotonlyatthepresentt,but
potentially at every time 0,1,2,···t−1 in the past. In other words, the entire
history of the population is needed to be able to propagate the process forward
from time t to t+1.
The history-dependence exhibited by Model 2 is a major obstacle both to analytical and
Monte Carlo investigations. Although this may sometimes be unavoidable, it is usually
preferable to begin with a model that can be propagated forward using only the present
state of the system.
JayTaylor (ASU) APM541 Fall2013 8/92
MarkovProcesses
Stochastic models that evolve in a manner that depends only on the current state are
known as Markov processes. The formal definition is given below.
Definition
AstochasticprocessX =(X :n≥0)withvaluesinasetE issaidtobeadiscretetime
n
Markov process if for every n≥0 and every set of values x ,x ,··· ,x ∈E, we have
0 1 n
P(X ∈A|X =x ,X =x ,··· ,X =x )=P(X ∈A|X =x ),
n+1 0 0 1 1 n n n+1 n n
whenever A is a subset of E such that {X ∈A} is an event. In this case, the
n+1
functions defined by
p (x,A)=P(X ∈A|X =x)
n n+1 n
are called the one-step transition probabilities of X. If the functions p (x,A) do not
n
depend on n, i.e., if there is a function p such that
p(x,A)=P(X ∈A|X =x)
n+1 n
for every n≥0, then we say that X is a time-homogeneous Markov process with
transition function p. Otherwise, X is said to be time-inhomogeneous.
JayTaylor (ASU) APM541 Fall2013 9/92
MarkovProcesses
Remarks:
Markovprocessesaresometimessaidtolack memory inthesensethatifweknow
the current state of the process X =x, then the values assumed by the process at
t
futuretimes,sayX =y,donotdependonhowtheprocessarrivedatstatex at
t+s
timet. Inotherwords, theprocessisonlyawareofitscurrentlocationandforgets
how it arrived there.
Equivalently, it can be shown that a Markov process has the following property,
which is known as the Markov property. If we condition on the event {X =x},
n
then the variables (X :k ≥1) are independent of the variables (X ;k ≥1),
n+k n−k
i.e., the future is conditionally independent of the past given the present.
Although time-inhomogeneous processes have useful applications, here we will
assume that all of our processes are time-homogeneous. Indeed, in some cases, we
can convert a time-inhomogeneous process to one that is time-homogeneous by
explicitly modeling the variables that are responsible for the changing transition
probabilities.
JayTaylor (ASU) APM541 Fall2013 10/92