The exponential family of Markov chains and its information geometry

0.12 MB·

by  Hiroshi Nagaoka

The28thSymposiumonInformationTheory
andItsApplications(SITA2005)
Onna,Okinawa,Japan,Nov.20–23,2005
The exponential family of Markov chains
and its information geometry
Hiroshi Nagaoka ∗
Abstract— We introduce a new definiton of exponential where X is a finite set and ν is the counting measure,
family of Markov chains, and show that many characteris- whereas the essence of the arguments can be extended
ticpropertiesoftheusualexponentialfamilyofprobability to the generalcase by putting proper reguralitycondi-
7 distributionsareproperlyextendedtoMarkov chains. The tions.
1 method of information geometry is effectively applied to
Several attempts have been made so far to extend
0
our framework, which enables us to characterize the diver-
2 the definition of exponential familes to Markove pro-
gence rate of Markov chain from a differential geometric
cesses or more general stochastic processes (e.g. [3, 4,
n viewpoint.
a 5]). Although they share the common criterion that
Keywords— exponential family, Markov chain, Markov
J an exponential family should have a finite dimensional
process, information geometry
2 (exactorasymptotic)sufficientstatistic,thedefinitions
2 giventherearediverse,reflectingtheirrespectiveback-
1 Introduction
grounds. The authors of such papers as [9, 10, 11] had
]
T A d-dimensional parametric family essentially the same concept on exponentialfamilies of
I Markov chains as ours when they said that the total-
. p θ =(θ1,...,θd)∈Θ , Θ⊂Rd
s θ ity of strictly positive Markov chains of a fixed order
c
[ of proba(cid:8)bility density functions (cid:9)on a measure space is asymptotically regarded as an exponential family in
an approximate sense. We refine the concept so that
(X,B,ν) is called an exponential family with a nat-
1 thegeneraldefinitionofexponentialfamiliesofMarkov
v uralparameterθ =(θi)(e.g.[2])whenthereexistd+1
kernels is given without appealing to asymptotic set-
9 R-valuedfunctionsC,F ,...,F onX andanR-valued
1 d tings,althoughthesignificanceofthedefinitionismade
1 function ψ on Θ such that
1 clear through some asymptotic arguments. This non-
6 d asymptotic treatment enables us to develop the infor-
0 logp (x)=C(x)+ θiF (x)−ψ(θ). (1) mationgeometryofMarkovchainsinatransparentand
θ i
.
1 i=1 systematic way.
X
0 This article is essentially based on the technicalre-
7 The notion of exponential family is very important in port [1]. The proofs of the theorems are omitted for
1 various fields such as the theory of statistical infer-
want of space.
:
v ence (parameter estimation, hypothesis testing, etc.),
i large deviations, information theory, etc, and a dual-
X 2 Definition and an example
istic viewpoint of the information geometry ([6, 7, 8])
r works effectively to understand the structure of expo- Let X be a finite set with |X| ≥ 2. We denote
a
nential families in a unified and elegant manner. The the totality of positive probability distributions on X
present paper is aimed at demonstrating that many by P = P(X). A Markov kernel (transition matri-
characteristicpropertiesofexponentialfamilies (1) are ces) on X is a map w : X2 → [0,1] ((x,y) 7→ w(y|x))
preserved by families of Markov kernel densities of the satisfying y∈X w(y|x) = 1 for every x ∈ X. Sup-
form pose that we are given a subset E of X2 = X × X
P
for which the directed graph (X,E) is strongly con-
d nected; i.e., for any (x,y)∈X2 there exists a sequence
logw (y|x)=C(x,y)+ θiF (x,y)
θ i (x ,x ),(x ,x ),...,(x ,x ) in E such that x =
1 2 2 3 n−1 n 1
Xi=1 x,xn =y,n≥2. LetthetotalityofirreducibleMarkov
+K (y)−K (x)−ψ(θ). (2)
θ θ kernels w on X such that {(x,y) w(y|x)>0} = E be
denoted by W = W(X,E). This includes the set of
Inparticular,weshowthatthefamilyisduallyflatjust
strictly positive Markovkernels W(X,X2) as a special
asisthecasefor(1),whichenablesustoapplythegen-
case.
eraltheoryforduallyflatspacestoinvestigationofthe
Fromthe irreducibility,eachw∈W hastheunique
information-geometricalstructure of the family. In or-
stationary distribution in P, which is denoted by p .
der to avoid being involeved with functional analytic w
We introduce the notaion
arguments and to concentrate upon geometric and al-
gebraicaspects,wemostlyconfineourselvestothecase p(n)(x ,...,x )=p (x )w(x |x )···w(x |x )
w 1 n w 1 2 1 n n−1
∗ Graduate School of Information Systems, The University of
Electro-Communications. 1-5-1, Chouhugaoka, Chouhu-shi, (p(w1)(x) = pw(x) and p(w2)(x,y) = pw(x)w(y|x) in par-
Tokyo,182-8585, Japan. (e-mail: [email protected]) ticular), which will be used in later sections.
A d-dimensional parametric family Afunctionf ∈F issaidtobeshift-invariantiff(+,·)=
f(·,+), and the totality of shift-invariant functions is
M = wθ θ =(θ1,...,θd)∈Rd ⊂W denoted by FS = FS(X,E). A function ∈ F is said
tobe anti-shift-invariant ifthereexistsafunctionκon
(cid:8) (cid:9)
is called an exponential family (or a full exponential
X such that f(x,y) = κ(y)−κ(x) for any (x,y) ∈ E,
family following the terminology of [2]) of Markovker-
and the totality of anit-shift-invariant functions is de-
nels on (X,E) with a natural (or canonical) parameter noted by FA = FA(X,E). The linear subspaces FS
θ, when there exist functions C,F1,...,Fd : X → R, and FA of F are the orthogonal complements of each
K : X ×Rd → R ((x,θ) 7→ K (x)) and ψ : Rd → R
θ otherwithrespecttotheinnerproductonF definedby
suchthatequation(2)holdsforeveryθ ∈Rd andevery
hf ,f i= f (x,y)f (x,y), which follows from
1 2 (x,y)∈E 1 2
(x,y)∈E.
Whenafamilyofinitialdistributions,say q θ ∈Rd , P
θ f(x,y)(κ(y)−κ(x))
isspecified,thefamilyofjointprobabilitydistributions
for xn = (x ,...,x ) satisfying (x ,x ) ∈(cid:8)E (∀t) is (cid:9) (xX,y)∈E
1 n t t+1
determined by = (f(+,x)−f(x,+))κ(x).
x∈X
X
q(n)(xn)=q (x )w (x |x )···w (x |x ), (3)
θ θ 1 θ 2 1 θ n n−1 We thus have the direct sum decompositon F =FS⊕
for which we have from (2) FA. In addition, from the assumption that (X,E) is
strongly connected, the necessary and sufficient condi-
n−1 tionfortwofunctionsκ ,κ todefinethesameelement
logqθ(n)(xn)=logqθ(x1)+ C(xt,xt+1) of FA is that κ1−κ2 i1s co2nstant on X. This leads to
Xt=1n dimFA = |X|−1 and dimFS = dimF −dimFA =
d |E|−|X|+1.
+ θiF(x ,x )−ψ(θ) +K (x )−K (x )
t t+1 θ n θ 1 Let
Xi=1 o
n−1 d n−1 F+ =F+(X,E)d=ef f f :E →R+ ⊂F,
= C(x ,x )+ θi F(x ,x )
t t+1 t t+1
t=1 i=1 t=1 where R+ = (0,∞). Then(cid:8)we have the fo(cid:9)llowing the-
X X X
−(n−1)ψ(θ)+O(1) (4) orem, which is a direct consequence of the Perron-
Frobenius theorem for irreducible nonnegative matri-
This shows that the family q(n) θ ∈Rd is asymp- ces.
θ
totically an exponential faminly in the usuaol sense. Theorem 1 For any f ∈ F+, there exist w ∈ W,
Z >0 and γ :X →R+ such that
Example 1 For any w∈W =W(X,X2), we have
1 γ(y)
|X|−1|X|−1 ∀(x,y)∈E, w(y|x)= f(x,y). (6)
w(j|i)w(0|j) Z γ(x)
logw(y|x)= log δ (x,y)
i,j
w(0|i)w(0|0)
i=0 j=1 Here w and Z are unique, and γ is unique up to a
X X
+logw(0|x)−logw(0|y)+logw(0|0), (5) constant factor.
Denoting the correspondence between f and w in
where δ (x,y) = 1 if (i,j) = (x,y) and = 0 oth-
i,j (6) by w =Γ(f), we define the mapping Γ:F+ →W.
erwise. We can easily verify this identity by respec-
Note that W ⊂F+ and Γ(w)=w for any w ∈W.
tively calculating the RHS for the two cases y 6=0 and
Since exp(f(x,y)) defines an element of F+ for ev-
y =0. This showsthatW is anexponentialfamilywith
ery f, we obtain a mapping
dimW =|X|·(|W|−1).
def
∆ = Γ◦exp :F →W
3 Affine structures
f 7→Γ(exp(f)).
The gist of the definition will be clarified by a fun-
damental relationbetween the affine structure offunc- Notingthatw =∆(f)=Γ(exp(f))iswrittenasw(y|x)=
tions on E and the Markov kernels on E as shown be- exp(f(x,y))γ(y)/Zγ(x) from (6), we have
low. LetF =F(X,E)bethetotalityoffunctionsonE,
whichisa|E|-dimensionallinearspace. Anelementf of logw(y|x)=f(x,y)+κ(y)−κ(x)−ψ, (7)
F is sometimes identified with its extension to a func-
tiononX2bylettingf(x,y)=0forany(x,y)∈X2\E. where κ(x) = logγ(x) and ψ = logZ. This means
that logw ≡ f mod FA ⊕ R, where a real constant
For a f ∈F we define the functions f(+,·)andf(·,+)
is identified with the corresponding constant function
on X by
in F. Moreover, ∆ gives a diffeomorphism from the
f(+,x)= f(y,x) and f(x,+)= f(x,y). quotient linear space F/(FA⊕R) to W. Now we have
the following theorem.
y∈X y∈X
X X
Theorem 2 A subset M of W is an exponential fam- 5 Expectation parameters
ily if and only if there exists an affine subspace V of For an exponential family M = w θ ∈Rd of
θ
F/(FA⊕R) for which M = ∆(V) d=ef {∆(f) f ∈V}, the form (2), we define
(cid:8) (cid:9)
andwehavedimM =dimV. Moreover,thecorrespon-
η (θ)d=ef p(2)(x,y)F (x,y). (12)
dencebetweenexponentialfamilesandaffinesubspaces i wθ i
is one-to-one. (xX,y)∈E
From this theorem, we have the following corollar- Thenη =(η1,...,ηd)formsanothercoordinatesystem
ies. for M, which we call the expectation parameter corre-
sponding to the representation (2).
Corollary 1 W itself is an exponential family of di-
mension |E|−|X|. Theorem 4 We have:
Remember that in Example 1 we have verified the η =∂ ψ, (13)
i i
above fact for the case of complete graph E =X2 by a
∂ η =g , (14)
i j ij
little complicated calculation.
∂iθj =gij, (15)
Corollary 2 The 1-dimensional exponential family
{wt t∈R}passingthroughgiventwokernelsw0,w1 ∈ [gij]=[gij]−1, (16)
W is written in the form wt = Γ(w1tw01−t), or equiva- where ∂i = ∂/∂η , and [gij] denotes the Fisher infor-
i
lently
mationmatrixwithrespecttothedualparameter(η ).
i
w (y|x)= 1 γt(y) {w (y|x)}t{w (y|x)}1−t. (8) Moreover,if we define ϕ= iθiηi−ψ, we have
t 1 0
Zt γt(x) θi =P∂iϕ, (17)
A 1-dimensional exponential family is called an e- gij =∂i∂jϕ. (18)
geodesic.
Corollary 3 AsubsetM ofW is anexponentialfam- 6 A dually flat structure
ily if and only if for any two points w and w in M For an arbitrary d-dimensional parametric family
0 1
the e-geodesic Γ(wtw1−t) t∈R lies in M. M = w θ =(θi)∈Θ ⊂ W, the exponential con-
1 0 θ
nection (or e-connection for short) ∇(e) and the mix-
(cid:8) (cid:9) (cid:8) (cid:9)
4 Fisher information ture connection (or m-connection for short) ∇(m) are
For an arbitrary d-dimensional parametric family defined as follows.
M = wθ θ =(θi)∈Θ ⊂ W, where Θ is an open Γ(e) =g(∇(e)∂ ,∂ )
subsetofRd,theFisherinformationmatrixG =[g (θ)]: ij,k ∂i j k
θ ij
(cid:8) (cid:9)
d×d(withrespectto the parameter(θi))isdefinedby = ∂i∂jlogwθ(y|x)∂kpθ(x,y), (19)
def (xX,y)∈E
g (θ) = p (x,y){∂ logw (y|x)}{∂ logw (y|x)},
ij θ i θ j θ Γ(m) =g(∇(m)∂ ,∂ )
(xX,y)∈E ij,k ∂i j k
(9) = ∂ ∂ p (x,y)∂ logw (y|x), (20)
i j θ k θ
where pθ(x,y) = p(w2θ)(x,y) = pwθ(x)wθ(y|x) and ∂i = (xX,y)∈E
∂/∂θi. Thisdefinitioniscommonlyusedbecauseofthe
where g is the Fisher metric. Then ∇(e) and ∇(m) are
following fact. Let {q θ ∈Θ} be an arbitrary family
θ
dual with respect to g in the sense that Xg(Y,Z) =
of probability distributions on X (possibly being inde-
(e) (m)
pendent of θ) and consider the joint distributions g(∇X Y,Z)+g(Y,∇X Y) holds for any vector fields
X,Y,Z.
qθ(n)(xn)=qθ(x1)wθ(x2|x1)···wθ(xn|xn−1). Theorem 5 For an exponential family M =
w θ ∈Rd of the form (2), both ∇(e) and ∇(m) are
Letting G(n) =[g(n)(θ)] be the Fisher informationma- θ
θ ij flat, and [θi] and [η ] are affine coordinate systems of
i
trix (in the usual sense) of the family q(n) θ ∈Θ , (cid:8)these connec(cid:9)tions, respectively.
θ
we have n o Theorem 6 Suppose thatM is anexponentialfamily
1
g (θ)= lim g(n)(θ). (10) andN isasubmanifoldofM. ThenN isanexponential
ij n→∞n ij family if and only if N is auto-parallel with respect to
From the information-geometric viewpoint, the Fisher the e-connection of M.
informationisregardedasaRiemannianmetricgthrough
Let
therelationg(∂ ,∂ )=g andiscalledtheFishermet-
i j ij
ric. P(2)S =P(E)∩FS
Theorem 3 WhenM = w θ ∈Rd isanexponen-
θ
= p(2) ∈P(E) ∀y, p(2)(x,y)= p(2)(y,x) .
tial family of the form (2), the Fisher information ma-
(cid:8) (cid:9) ( )
trix withrespectto the naturalparameter(θi)is given Xx Xx
by Then w 7→ p(2) gives a diffeomorphism from W to
w
gij(θ)=∂i∂jψ(θ). (11) P(2)S.
Theorem 7 Them-connectionofW isthenaturalflat References
connection induced from the convexity of P(2)S. In [1] H. Nagaoka, “Exponential families of Markov
particular,the m-geodesic (i.e., the auto-parallelcurve chains and their information geometry,” IS Tech-
withrespecttothem-connection)connectinggiventwo nical Reports, UEC-IS-2003-6, Univ. of Electro-
points w and w in X is represented as Communications, 2003.
0 1
[2] O.E. Barndorff-Nielsen, Information and Expo-
p(2) =tp(2)+(1−t)p(2). (21) nentialFamiliesinStatisticalTheory,Wiley,1978.
wt w1 w0
[3] P. D. Feigin, “Conditional exponential families
and a representation theorem for asymptotic in-
7 Canonical Divergence
ference,”Ann. Statist.,vol.9,no.3,597–603,1981.
Let M be an exponential family of the form (2).
[4] U. Ku¨chler and M. Sørensen, “On exponential
Then the canonical divergence D : M ×M → R with
familiesofMarkovprocesses,”J.Statist.Planning
respect to the dually flat structure (g,∇(m),∇(e)) is
and Inference, vol.66, 3-19, 1998.
defined by
[5] U.Ku¨chlerandM.Sørensen,ExponentialFamilies
of Stochastic Processes, Springer-Verlag,1997.
d
D(w |w )=ϕ(w )+ψ(w )− η (w )θi(w ). (22) [6] S. Amari, Differential-Geometrical Methods
1 2 1 2 i 1 2
in Statistics (Lecture Notes in Statistics 28),
i=1
X Springer-Verlag,1985.
ThedivergenceD isalsocharacterizedbythefollowing [7] S.AmariandH.Nagaoka,Methods ofInformation
property: let γ be an m-geodsic such that γ(1) = w Geometry, AMS&Oxford Univ. Press, 2000.
1
and γ(0)=w , and δ be an e-geodsic suchthat δ(1)= [8] H. Nagaoka and S. Amari, “Differential geometry
2
w andδ(0)=w ,whichintersectatw . Thenwehave of smooth families of probability distributions”,
3 2 2
METR 82-7, Univ. of Tokyo, 1982.
D(w |w )+D(w |w )−D(w |w )=g(γ˙(0),δ˙(0)), [9] H. Ito and S. Amari, “Geometry of information
1 2 2 3 1 3
(23) sources,” Proceedings of the 11th Symposium on
where the RHS means the inner product between the Information Theory and Its Applications (SITA
tangentvectorsofγ andδ attheintersectingpointw . ’88), 57–60,1988 (in Japanese).
2
[10] N. Merhav, “The estimation of the model order
Theorem 8 D is represented as
in exponential families,” IEEE Trans. on Inform.
Theory, vol.35, no.5, 1109–1114,1989.
w (y|x)
D(w |w )= p(2)(x,y)log 1 . (24) [11] S. Amari, “Information geometry on hierarchy
1 2 w1 w (y|x)
2 of probability distributions,” IEEE Trans. on In-
(xX,y)∈E
form. Theory, vol.47, no.5, 1701–1711,2001.
This is nothing but the divergence rate of Markov
chains. Actually, we have
1
D(w |w )= lim D(q(n)|q(n)), (25)
1 2 n→∞n 1 2
(n)
where q , i=1,2 are defined as
i
q(n)(x ,...,x )=q (x )w(x |x )···w(x |x )
i 1 n i 1 2 1 n n−1
by arbitrary distributions q on X.
i
8 Remaining subjects
The following subjects can also be treated in the
present framework or its obvious extension.
• Application to the large deviation theory.
• Some variant of Cram´er-Rao inequality, and an
estimation-theoreticcharacterizationofexponen-
tial families.
• Extension to higher-order Markov chains, and a
hierarchyofexponentialfamiles: P =W ⊂W =
0
W ⊂W ⊂···.
1 2
• Extension to general measureble spaces, and au-
toregressive models as an example.