Finite-length Analysis on Tail probability for Markov Chain and Application to Simple Hypothesis Testing

0.33 MB·

by  Shun Watanabe

Finite-length Analysis on Tail
probability for Markov Chain and
5 Application to Simple Hypothesis
1
0 Testing
2

r
a Shun Watanabe1 and Masahito Hayashi2
M 1Department of Information Science and Intelligent Systems, University of
Tokushima, Japan,
5
and Institute for System Research, University of Maryland, College Park.
1
2Graduate School of Mathematics, Nagoya University, Japan,

] and Centre for Quantum Technologies, National University of Singapore,
T
Singapore.
S
.
h
Abstract: Usingterminologies of informationgeometry, wederive upper
t
a and lower bounds of the tail probability of the sample mean. Employing
m these bounds, we obtain upper and lower bounds of the minimum error
probability of the 2nd kind of error under the exponential constraint for
[ theerrorprobabilityofthe1stkindoferrorinasimplehypothesistesting

forafinite-lengthMarkovchain,whichyieldstheHoeffdingtypebound.For
2
thesederivations,wederiveupperandlowerboundsofcumulantgenerating
v
functionforMarkovchain.Asabyproduct,weobtainanothersimpleproof
1
ofcentrallimittheorem forMarkovchain.
0
8 Primary62M02,62F03.
3 Keywordsandphrases:simplehypothesistesting,tailprobability,finite-
. lengthMarkovchain,informationgeometry,relativeentropy,relativeR´enyi
1 entropy.
0
4
1
1. Introduction
:
v
i Markov chain is a natural model for probability distribution with stochastic
X
correlation. Under this model, we often focus on the sample mean of n obser-
r
a vations, and discuss the cumulant generating function and the tail probability.
Manyexistingstudiesinvestigatedtheirasymptoticbehaviors[3,5,6,7,28].For
example,the papers [5, 6, 7] showedthe centrallimit theorem,i.e., they proved
thatthedifferencebetweenthesamplemeanandtheexpectationasymptotically
obeystheGaussiandistribution.DemboandZeitouni[3]derivedtheasymptotic
cumulant generating function and the large deviation bound by using its Leg-
endre transform. Further, other existing studies [1, 2] investigated the simple
hypothesistestingforMarkovchains.TheyderivedtheHoeffdingbound[25]for
two Markov chains, i.e., the exponentially decreasing rate of the second error
probability under the exponential constraint for the first error probability. In
the independently and identically distributed (i.i.d.) case, as the generalization
1
S. Watanabe and M. Hayashi/Finite-length Analysis for Markov Chains 2
of Stein’s lemma. Strassen [17] derived the asymptotic expansion of the expo-
nentialdecreasingrateoftheseconderrorprobabilityuptotheorder√n,under
the constantconstraintforthe firsterrorprobability,whosequantum extension
was recently done by the papers in [10, 14].
Indeed,itisnotdifficulttogiveaboundwhenitisnotsotightoritscompu-
tationisnotsoeasy.Here,weshouldmentionaproperrequirementforabetter
finite-length bound as follows.
(1) Asymptotic tightness.For example,in the case of the tail probability,the
bound can recover one of the following in the limit n ;
→∞
(T1) Central limit theorem [5, 6, 7]
(T2) Moderate deviation [28, 5]
(T3) Large deviation [3, 5]
(2) Computability. The bound should have less computational complexity,
e.g., O(1), O(n) or O(nlogn). For example, we call the bound O(1)-
computable when its computation complexity is O(1).
Inthei.i.d.case,itisknownthattheMarkovinequalityderivesanupperbound
of the tail probability that attains the asymptotic tightness in the sense of
(T2) and (T3) and is called Chernoff bound [3, 29]. However, even in the i.i.d.
case, there is no O(1)-computable finite-length lower bound that attains the
asymptotic tightness in the sense of (T2) nor (T3). The Berry-Essen theorem
givesupperandlowerO(1)-computableboundsofthetailprobabilitythatattain
the asymptotic tightness in the sense of (T1) in the i.i.d. case (see e.g., [26]).
The paper [27, Theorem 2] extended the Berry-Essen theorem to the Markov
chain,andgavesimilarupperandlowerO(1)-computableboundsfortheMarkov
chain.
In the case of simple hypothesis testing, three kinds of the asymptotic tight-
ness are characterizedas follows.
(H1) Constant constraint for the first error probability ǫ=const.
(H2) Moderate deviation type constraint for the first error probability ǫ =
e n1−2tr with t (0,1).
− ∈ 2
(H3) Large deviation type constraint for the first error probability ǫ = e nr
−
(Hoeffding bound [1, 2]).
Inthe i.i.d.case(including the quantumcase),the paper[10]derivedlowerand
upper O(1)-computable finite-length bounds for the second error probability
thatattainthe asymptotictightnessinthe senseof(H1).Also,itisnotdifficult
to derive an upper O(1)-computable finite-length bound for the second error
probability that attains the asymptotic tightness in the sense of (H2) nor (H3).
However, no study addressed a lower O(1)-computable finite-length bound for
the second error probability that attains the asymptotic tightness in the sense
of (H2) nor (H3) even in the i.i.d. case.
This paper derives the finite-length bounds for the above topics satisfying
the above requirement. Firstly, we derive upper and lower bounds of the cu-
mulant generating function when n observations are given. We show that these
S. Watanabe and M. Hayashi/Finite-length Analysis for Markov Chains 3
limitsrecovertheasymptoticcumulantgeneratingfunction[3].Usingourevalu-
ationofthecumulantgeneratingfunction,wealsoderiveupperandlowerO(1)-
computablebounds ofthe tailprobabilitythatattainsthe asymptotic tightness
in the sense of (T2) and (T3) in the Markov chain as well as in the i.i.d. case.
Ouranalysiscoversthe samplemeanoftwo-inputfunctionslikeg(X ,X )as
k+1 k
wellasthe simple samplemean n Xi.Asabyproduct,employingthe evalu-
i=1 n
ationofthecumulantgeneratingfunction,wesimplyreproducethecentrallimit
P
theorem [5, 6, 7]. Indeed, since we address a general function g(X ,X ), our
k+1 k
evaluations can be applied to the sample mean of the hidden Markov random
variables.
For the simple hypothesis testing, this paper derives the lower and upper
O(1)-computable bounds of the second error probability under the same con-
straintwithfiniteobservationswhoselimitsrecovertheasymptoticbound(H3)[1,
2] and the asymptotic bound (H2). For describing these finite-length bounds,
we employ the notations given by the transition matrix version of information
geometry, i.e., the relative entropy (Kullback Leibler divergence), the relative
R´enyi entropy, exponential family, natural parameter, and expectation param-
eter [2, 9, 15]. Further, employing the Markov version of the Berry-Essentheo-
rem[27,Theorem2],wealsoobtainanothertypeO(1)-computablefinite-length
bound,whichderivestheasymptoticbound(H1)asageneralizationoftheresult
by Strassen [17].
Indeed, there are two ways to define a transition matrix version of exponen-
tial family. We employ the definition by [2, 9, 15], which is different from the
definitionby[18,19,20,21,22,23,24].Theexponentialfamilytobeusedplays
anessentialrole in ourderivation.Thatis, the exponentialfamily enables us to
discusssimple hypothesis testing andthe parameterestimation[15]ina unified
manner.Theobtainedboundsareusedfortheevaluationsofseveralinformation
theoretical problems [16].
The remaining of this paper is organizedas follows. Section 2 gives the brief
summary of obtained results. In Section 3, we review an exponential family
of transition matrices [2, 9, 15] in the one-parameter case. In Section 4, we
characterizeLegendre transformof the potential function. In Section6, we give
usefulupperandlowerboundsofthecumulantgeneratingfunction.InSection7,
wegivea simple alternativeproofofthe centrallimittheoremofMarkovchain.
In Section 8, we also give useful upper and lower bounds of the tail probability
with finite observation, which produces the large deviation bound of the tail
probability.InSection9,usingthesebounds,wederiveupperandlowerbounds
of the second error probability of simple hypothesis testing, which yields the
Hoeffding type bounds.
2. Summary of results
Here,wepreparenotationsanddefinitions.ForagiventransitionmatrixW over
,wedefineW n(x ,x ,...,x x¯):=W(x x )W(x x ) W(x x¯)
× n n 1 1 n n 1 n 1 n 2 1
aXnd Wn(xx¯) = − W n(|x,x ,...,|x −x¯). For a−g|ive−n d·is·t·ributio|n
| xn−1,...,x1 × n−1 1|
P
S. Watanabe and M. Hayashi/Finite-length Analysis for Markov Chains 4
P on and a transition matrix V from to , we define V P(y,x) :=
X X Y ×
V(y x)P(x). and VP(y):= V P(y,x).
| x ×
A non-negative matrix W is called irreducible when for each x,x¯ , there
existsanaturalnumbernsuPchthatWn(xx¯)>0[34].Anirreducible∈mXatrixW
|
iscalledergodicwhentherearenoinputx¯andnointegern suchthatWn(x¯x¯)=
′
|
0unlessnisdivisiblebyn [34].ItisknownthattheoutputdistributionofWnP
′
converges to the stationary distribution of W for a given ergodic transition
matrix W [11, 34].
2.1. Cumulant generating function
Assume that the random variable X obeys the Markov process with the tran-
n
sition matrix W(xx¯). In this paper, for a two-input function g(x,x¯), we fo-
cus on the random| variable gn(Xn+1) := n g(X ,X ), and Xn+1 :=
i=1 i+1 i
(X ,...,X ). This is because a two-input function g(x,x¯) is closely related
n+1 1
P
toanexponentialfamilyoftransitionmatrices.Indeed,thesimplesamplemean
can be treated in the formulation by choosing g(x,x¯) as x or x¯. Here, when we
choose a general function g(x), gn(Xn+1) = n g(X ) is the sample mean
i=1 i+1
ofthehiddenMarkovrandomvariable.So,ourresultscanappliedtothehidden
P
Markov random case.
We denote the Perron-Frobenius eigenvalue of W(xx¯)eθg(x,x¯) by λ and de-
θ
|
fine the potential function φ(θ) := logλ . Then, we focus on the cumulant
θ
generating function φ (θ) := logE[eθgn(Xn+1)], where E denotes the expecta-
n
tion. We will define functions δ(θ) and δ(θ) in Section 6 so that δ(θ) 0 and
→
δ(θ) 0 as θ 0. Then, we will evaluate φ (θ) as
n
→ →
nφ(θ)+δ(θ) φ (θ) nφ(θ)+δ(θ). (2.1)
n
≤ ≤
2.2. Tail probability
Given an irreducible and ergodic transition matrix W, we will evaluate the tail
probabilityofthe randomvariablegn(Xn+1) by using the one-parameterexpo-
nanendtiDal1+fasm(WilyθkWWθθ¯)gievxepnlainin[e1d5,inSeScetcitoinon3]3a.nads ftohlelorwelsa.tFivoereanntyroap>iesED[g(]W, wθekWwθi¯l)l
show
logP g˜n(Xn+1) na nD(W W ) δ(θ), (2.2)
− { ≥ }≥ φ′−1(a)k 0 −
where φ′−1(a) is the inverse function of φ′(θ) = ddφθ(θ), i.e., ddφθ(φ′−1(a)) = a.
Conversely,we will show
logP g˜n(Xn+1) na
− { ≥ }
1
inf nD (W W )+ [δ((1+s)θ) δ(θ)]
1+s θ 0
≤ s>0 k s −
θ>φ′−1(a)
1+slog 1 e−nD(Wφ′−1(a)kWθ)+δ(φ′−1(a))−δ(θ) . (2.3)
− s −
(cid:16) (cid:17)
S. Watanabe and M. Hayashi/Finite-length Analysis for Markov Chains 5
Similarly, for a<E[g], we will show
logP g˜n(Xn+1) na nD(W W ) δ(θ). (2.4)
− { ≤ }≥ φ′−1(a)k 0 −
Conversely,we will show
logP g˜n(Xn+1) na
− { ≤ }
1
inf nD (W W )+ [δ((1+s)θ) δ(θ)]
1+s θ 0
≤ s>0 k s −
θ<φ′−1(a)
1+slog 1 e−nD(Wφ′−1(a)kWθ)+δ(φ′−1(a))−δ(θ) . (2.5)
− s −
(cid:16) (cid:17)
2.3. Simple hypothesis testing
NanodwW, w1ne−c1o×nsPid1e.rTthheen,hwypeoctohnessiidsetresting with the two hypotheses W0n−1 ×P0
βǫ(W1n−1×P1kW0n−1×P0)
:=Sminn+1{1−W0n−1×P0(S)|W1n−1×P1(S)≤ǫ}. (2.6)
⊂X
Using the one-parameterexponential family W of transition matrices with the
θ
generator g(x,x¯):= logW1(x|x¯), the cumulant generating function φ(θ) defined
W0(x|x¯)
by g(x,x¯), we will show that
n( θr φ(θ)) δ(θ)
sup − − −
1 θ
0 θ 1
≤ ≤ −
≤−logβe−nr(W1×n×P1kW0×n×P0)
1
inf nD (W W )+ (δ((1+s)θ) (1+s)δ(θ))
1+s θ 0
≤s>0,θ (θˆ(r),1) k s −
∈
1+slog(1 2e−nD(Wθˆ(r)kWθ)−δ(θ)+(1−1θ−)δθˆ((θrˆ)(r))), (2.7)
− s −
wherethe functions θˆ(r) isgiveninSection4.We willalsoasymptoticallychar-
acterize βǫ(W1n−1×P1kW0n−1×P0) with a fixed ǫ.
3. Geometric structure for transition matrices
Inthissection,wereviewthedefinitionandthepropertiesoftheone-parameter
exponential family of transition matrices [2, 9] by following the logical order of
[15,Section4]althoughalargepartofresultsforexponentialfamilyoftransition
matrices were obtained by Nagaoka [9] and Nakagawa and Kanaya [2]. This is
because the logical order of [15, Section 4] is more suitable for the context of
this paper. These relations are explained in [15, Remarks 3.5, 4.12, and 4.14].
Notethatthedefinitionofexponentialfamilyinthispaperisdifferentfromthat
by the papers [18, 19, 20, 21, 22, 23, 24] as explained in [15, Remark 4.13].
S. Watanabe and M. Hayashi/Finite-length Analysis for Markov Chains 6
3.1. Preparations
For the definition and the properties of the one-parameter exponential family
of transition matrices, we prepare the following things.
Lemma 3.1. [15, Lemma 3.1] Consider an irreducible and ergodic transition
matrix W over and a real-valued function g on . Then, we define the
X X ×X
support 2 := (x,x¯) 2 W(xx¯) > 0 . Define φ(θ) as the logarithm of the
XW { ∈ X | | }
Perron-Frobenius eigenvalue of the matrix:
W˜ (xx¯):=W(xx¯)eθg(x,x¯). (3.1)
θ
| |
Then, the function φ(θ) is convex. Further, the following conditions are equiva-
lent.
(1) No real-valued function f on satisfies that g(x,x¯)=f(x) f(x¯)+c for
any (x,x¯) 2 with a constXant c R. −
∈XW ∈
(2) The function φ(θ) is strictly convex, i.e., d2φ(θ)>0 for any θ.
dθ2
(3) d2φ(θ) >0.
dθ2 |θ=0
Using Lemma 3.1, given two distinct ergodic transition matrices W and V
with the same support, we define the relative entropy and the relative R´enyi
entropies. For this purpose, we denote the logarithm of the Perron-Frobenius
eigenvalue of the matrix W(xx¯)1+sV(xx¯) s by ϕ(1+s). Then, we define
−
| |
dϕ ϕ(1+s)
D(W V):= (1), D (W V):= . (3.2)
1+s
k ds k s
Note that the limit lim D (W V) equals D(W V). Since W and V are
s 0 1+s
distinct, the function lo→gW(x|x¯) satkisfies the conditikon for the function g in
V(xx¯)
|
Lemma 3.1. Hence, the function s sD (W V) is strictly convex, which
1+s
7→ k
implies that sD (W V) < (1 s¯)0+ ss¯D (W V) for 0 < s < s¯. Since a
1+s k − s s¯ 1+s¯ k
similar relation holds for 0 > s > s¯, the relative R´enyi entropy D (W V) is
1+s
k
strictly monotone increasing with respect to s.
3.2. Exponential family
Now, we focus on a transition matrix W(xx¯) from to and a real-valued
| X X
function g on satisfying the condition in Lemma 3.1. In the following,
X ×X
we assume that the function g satisfies condition in Lemma 3.1. Then, we will
define the matrix W (xx¯) from to for θ by following steps below.For this
θ
purpose, we define the m| atrix W˜X(xx¯X) from to by
θ
| X X
W˜ (xx¯):=W(xx¯)eθg(x,x¯). (3.3)
θ
| |
UsingthePerron-Frobeniuseigenvalueλ ofW˜ ,wedefinethepotentialfunction
θ θ
φ(θ):=logλ . (3.4)
θ
S. Watanabe and M. Hayashi/Finite-length Analysis for Markov Chains 7
Due to Lemma 3.1, the second derivative d2φ is strictly positive. Hence, the
dθ2
potential function φ(θ) is strictly convex. In the following, using the strictly
convex function φ(θ), we define a one-parameter exponential family for transi-
tion matrices.
Note that, since the value W˜ (xx¯) generally depends on x¯, we cannot
x θ |
makeatransitionmatrixbysimply multiplyingaconstantwiththematrixW˜ .
P θ
To make a transitionmatrix from the matrix W˜ , we recallthat a non-negative
θ
matrixV from to isatransitionmatrixifandonlyifthevector(1,...,1)T is
X X
aneigenvectorofthetransposeVT.Inordertoresolvethisproblem,wefocuson
thestructureofthematrixW˜ .WedenotethePerron-Frobeniuseigenvectorsof
θ
W˜ anditstransposeW˜T byP˜ andPˆ .SincetheirreducibilityofW guarantees
θ θ θ θ
the irreducibilityofP˜ ,the relationPˆ (x)>0holds.Accordingto[2, 9, 5, 15]1,
θ θ
we define the matrix W (xx¯) as
θ
|
Wθ(x|x¯):=λ−θ1Pˆθ(x)W˜θ(x|x¯)Pˆθ(x¯)−1. (3.5)
ThematrixW (xx¯)isatransitionmatrixbecausevector(1,...,1)T isaneigen-
θ
|
vector of the transpose WT. In the following, we call the family of transition
θ
matrices := W an exponential family of transition matrices with the gen-
θ
E { }
erator g.
Using the potential function φ(θ), we explain several concepts for transition
matrices based on Lemma 3.1, formally. We call the parameter θ the natural
parameter,andtheparameterη(θ):=φ(θ)= dφ(θ)theexpectationparameter.
′ dθ
For η, we define the inverse function φ′−1(η) of φ′ as
1
φ′(φ′− (η))=η. (3.6)
Then,wedefine theFisherinformationforthe naturalparameterbythe second
derivative d2φ(θ).TheFisherinformationfortheexpectationparameterisgiven
dθ2
as d2φ(θ) 1.
dθ2 −
Lemma 3.2. [15, Lemma 4.4] The relative entropy and the relative R´enyi en-
tropies between two transition matrices Wθ and Wθ¯ are characterized as
dφ
D(WθkWθ¯)=(θ−θ¯)dθ(θ)−φ(θ)+φ(θ¯) (3.7)
φ((1+s)θ sθ¯) (1+s)φ(θ)+sφ(θ¯)
D1+s(WθkWθ¯)= − −s . (3.8)
In the following, E denotes the expectation with respect to the joint dis-
W
tributionwhen the conditionaldistributionis givenby the transitionmatrix W
and the input distribution is given by the stationary distribution of W. Then,
for a generator g and a real number a, we define as
g,a
M
:= W E g(X,X )=a . (3.9)
g,a W ′
M { | }
A transition matrix version of the Pythagoreantheorem [4] holds as follows.
1Appendix of [15] explains detailed relation the papers [2, 9, 5, 15] for an exponential
familyoftransitionmatrices.
S. Watanabe and M. Hayashi/Finite-length Analysis for Markov Chains 8
Theorem 3.3. [2, Lemma 5],[15, Corollary 4.8] For a transition matrix V, a
generator g, and a real number a, we define
V := argmin D(W V). (3.10)
∗
k
W∈Mg,a
(1) Any transition matrix W satisfies
g,a
∈M
D(W V)=D(W V∗)+D(V∗ V). (3.11)
k k k
(2) The transition matrix V is the intersection of the set and the expo-
∗ g,a
M
nential family generated by g containing V.
Due to Lemma 3.2, the Fisher information d2φ(θ ) can be characterized by
dθ2 0
the limits of the relative entropy and relative R´enyi entropy as follows.
Lemma 3.4. Under the limit δ 0, we have
→
1 1 1d2φ
lim D(W W )= lim D(W W )= (θ ), (3.12)
δ 0δ2 θ0+δk θ0 δ 0δ2 θ0k θ0+δ 2dθ2 0
→ →
1 1
lim D (W W )= lim D (W W )
δ 0δ2 1+s θ0+δk θ0 δ 0δ2 1+s θ0k θ0+δ
→ →
1+sd2φ
= (θ ). (3.13)
2 dθ2 0
4. Relation with Legendre transform
Given two irreducible and ergodic transition matrices W and V, we choose the
exponentialfamily W with the generatorg(x,x¯):=logV(xx¯) logW(xx¯) so
θ
| − |
that W = W and W = V. In fact, an arbitrary exponential family W can
0 1 θ
be writtenasthe aboveformbychoosingtwoirreducibleandergodictransition
matrices as W :=W and V :=W . The Legendre transform sup [θa φ(θ)]
of the convex functio0n φ can be ch1aracterizedas follows. θ≥0 −
Lemma 4.1. When a> D(W V),
− k
φ((1+s)θ) (1+s)φ(θ)
inf D (W W )= inf −
1+s θ 0
s>0 k s>0 s
θ>φ′−1(a) θ>φ′−1(a)
1 1
=φ′− (a)a−φ(φ′− (a))=sup[θa−φ(θ)]=D(Wφ′−1(a)kW0). (4.1)
θ 0
≥
Similarly, when a< D(W V),
− k
φ((1+s)θ) (1+s)φ(θ)
1 1
inf − =φ′− (a)a φ(φ′− (a))
s>0 s −
θ<φ′−1(a)
=sup[θa φ(θ)]=D(W W ).
− φ′−1(a)k 0
θ 0
≤
S. Watanabe and M. Hayashi/Finite-length Analysis for Markov Chains 9
Proof. Sincethefunctionθ φ(θ)isconvex,thefunctions φ((1+s)θ)−(1+s)φ(θ)
7→ 7→ s
is monotone increasing due to Lemma C.1. Hence, infs>0 φ((1+s)θ)−s(1+s)φ(θ) =
θddφθ(θ)−φ(θ) for θ >φ′−1(a). Thus,
φ((1+s)θ) (1+s)φ(θ) dφ
inf − = inf θ (θ) φ(θ).
s>0 s θ>φ′−1(a) dθ −
θ>φ′−1(a)
Since the function θ φ(θ) is convex, the function θ θdφ(θ) φ(θ) is
7→ 7→ dθ −
monotone increasing for θ 0. Therefore,
≥
dφ
1 1
inf θ (θ) φ(θ)=φ′− (a)a φ(φ′− (a))=sup[θa φ(θ)],
θ>φ′−1(a) dθ − − θ 0 −
≥
wherethesecondequationfollowsfromtheconvexityofφ(θ).Thefinalequation
in (4.1) is shown by D(Wφ′−1(a)kW0)=φ′−1(a)a−φ(φ′−1(a)).
Now, for an arbitrary convex function φ and r > 0, we define the function
θˆ(r)=θˆ[φ](r) as the smaller solution of the equation
dφ
(θ 1) (θ) φ(θ)=D(W W )=r (4.2)
θ 1
− dθ − k
with respect to θ. Hence, due to the convexity of φ, we have
inf D (W W )
1+s θ 0
s>0,θ (0,θˆ(r)) k
∈
1
= inf [φ((1+s)θ) (1+s)φ(θ)]
s>0,θ (0,θˆ(r))s −
∈
1 dφ
=inf [φ((1+s)θˆ(r)) (1+s)φ(θˆ(r))]=θˆ(r) (θˆ(r)) φ(θˆ(r))
s>0s − dθ −
r+φ(θˆ(r)) θˆ(r)r φ(θˆ(r))
= θˆ(r) φ(θˆ(r))= − −
− 1 θˆ(r) − 1 θˆ(r)
− −
(a) θr φ(θ) θ( r+D1 θ(W0 W1))
= sup − − = sup − − k , (4.3)
1 θ 1 θ
0 θ<1 0 θ<1
≤ − ≤ −
which implies the following lemma. Here, (a) can be derived as follows. Due
to Lemma C.1, the maximum can be attained when (4.2) holds, i.e., θ = θˆ(r).
Hence, we have (a).
Lemma 4.2. When 0 r D(W W ),
0 1
≤ ≤ k
θr φ(θ) θ( r+D (W W ))
1 θ 0 1
sup − − = sup − − k
1 θ 1 θ
0 θ 1 0 θ 1
≤ ≤ − ≤ ≤ −
= inf D (W W )=D(W W )= min D(W W ).
s>0,θ∈(0,θˆ(r)) 1+s θk 0 θˆ(r)k 0 W:D(WkW1)≤r k 0
Here, when φ(θ) is regardas a function of δ := θ , i.e., is described as f(δ),
1 θ 1−θ
sup0≤θ≤1 −θr1−−φθ(−θ) isgivenastheLegendretransform−off,i.e,sup0≤θ≤1δr−f(δ)
S. Watanabe and M. Hayashi/Finite-length Analysis for Markov Chains 10
Proof. The first and second equations follow from (4.3). The third equation
followsfrom(4.3)andtherelationD(W W )=θˆ(r)dφ(θˆ(r)) φ(θˆ(r)).Now,
θˆ(r)k 0 dθ −
we show the final equation. We choose W satisfying that D(W W ) r. We
1
k ≤
also choose a such that W , which is defined in Theorem 3.3. Then, we
g,a
∈M
denotetheintersectionofthesetMg,a andtheexponentialfamily{Wθ}byWθ¯.
Theorem 3.3 implies that D(Wθ¯kW1)≤r and D(Wθ¯kW0)≤D(WkW0). Thus,
we obtain
min D(W W )= min D(W W ). (4.4)
0 θ 0
W:D(WkW1)≤r k θ:D(WθkW1)≤r k
Duetothecondition0 r D(W W ),theabovevalueequalsD(W W ).
≤ ≤ 1k 0 θˆ(r)k 0
5. Information processing inequality
Now, we introduce a condition for a transition matrix as follows. A transition
matrixW on iscallednon-hiddenfor whenW (xx):= W(x,y x,y )
X×Y X X | ′ y | ′ ′
does not depend on y . When the Markovchain for X and Y g∈enYeratedby
′
∈Y P
W and W satisfies the above condition, the sequence for X is also a Markov
chain, not a hidden Markov chain. This is the reason of the name of “non-
hidden”.Asatransitionmatrixversionofinformationprocessinginequality,we
have the following theorem.
Theorem 5.1. When two transition matrices W and V on are non-
X ×Y
hidden for , the following hold for s ( 1,0) (0, ):
X ∈ − ∪ ∞
D(W V) D(W V ), D (W V) D (W V ). (5.1)
X X 1+s 1+s X X
k ≥ k k ≥ k
Proof. For s > 0, let λ and b = (b ) be the Perron-Frobenius eigenvalue
x,y
andeigenvectorofthematrixW(x,y x,y )1+sV(x,y x,y ) s.Sincethereverse
′ ′ ′ ′ −
| |
Ho¨lder inequality implies
W(x,y x,y )1+sV(x,y x,y ) s
′ ′ ′ ′ −
| |
y
X
( W(x,y x,y )(1+s)/(1+s))1+s( V(x,y x,y ) s/( s)) s
′ ′ ′ ′ − − −
≥ | |
y y
X X
=W (xx)1+sV (xx) s,
X ′ X ′ −
| |
the number c = b satisfies
x y x,y
P
λc = λb = W(x,y x,y )1+sV(x,y x,y ) sb
x x,y ′ ′ ′ ′ − x′,y′
| |
y y x′,y′
X XX
W (xx)1+sV (xx) sb = W (xx)1+sV (xx) sc ,
X ′ X ′ − x′,y′ X ′ X ′ − x′
≥ | | | |
x′,y′ x′
X X