Table Of ContentGENERALIZED ARITHMETIC AND GEOMETRIC MEAN
DIVERGENCE MEASURE AND THEIR STATISTICAL ASPECTS
INDER JEET TANEJA
5 Abstract. Using Blackwell’sdefinitionofcomparingtwoexperiments,a comparisonis
0
made with generalized AG - divergence measure having one and two scalar parameters.
0
Connection of generalized AG - divergence measure with Fisher measure of information
2
is also presented. A unified generalization of AG - divergence and Jensen-Shannon
n
a divergence measures is also presented.
J
9
1
1. Introduction
]
T
S Several measures have been introduced in the literature on information theory and
.
h statistics as measures of information. The most famous in the literature of statistics is
t
a Fisher [9] measure of information. It measures the amount of information supplied by
m
data about an unknown parameter θ. The most commonly used in information theory is
[
the Shannon [18] entropy. It gives the amount of uncertainty concerning the outcome of
1
an experiment. Kullback and Leibler [13] introduced a measure associated with two dis-
v
7 tributions of an experiment. It expresses the amount of information supplied by the data
9
for discriminating among the distribution. As a symmetric measure, Jeffreys-Kullback-
2
1 Leibler J-divergence is commonly used. R´enyi [16] generalized both Shannon entropy and
0
Kullback-Leibler relative information by introducing a scalar parameter. Burbea and
5
0 Rao [4], [5] and Taneja [21], [22] have proposed various alternative ways to generalize the
/
h Jeffreys-Kullback-Leibler J-divergence. The proposed measures of Burbe and Rao [4], [5]
t
a involve one parameter. Measures proposed by author [22] involve two scalar parameters.
m
Let E = {X,S ,P ;θ ∈ Θ} denote a statistical experiment in which a random vari-
X X θ
:
v able or random vector X defined on some sample space S is to be observed and the
X
i
X distribution P of X depends on the parameter θ whose values are unknown and lie in
θ
r some parameter space Θ. We shall assume that there exists a generalized probability
a
density function p(x|θ) for the distribution P with respect to σ−finite measure µ. Let
θ
also Ξ denote the class of all prior distributions on the parameter space Θ. Given a prior
distribution ξ ∈ Ξ, let p(x) denote the corresponding marginal generalized probability
density function (gpdf)
p(x) = p(x|θ)dξ(θ).
ZΘ
2000 Mathematics Subject Classification. 94A17; 62B10.
Key words and phrases. Comparisonofexperiments; Fisher measureofinformation; GeneralizedAG–
divergence; JS–divergence.
1
2 INDER JEET TANEJA
Similarly, if we have two prior distributions ξ , ξ ∈ Ξ, the corresponding marginal
1 2
gpdf’s are
p (x) = p(x|θ)dξ (θ), i = 1,2.
i i
ZΘ
In this context, the relative information, the J–divergence, the Jensen-Shannon diver-
gence, and the arithmetic and geometric mean divergence measures are given as follows:
• Relative information (Kullback and Leibler [13])
p (x)
(1) K(ξ ;ξ ||X) = p (x)ln 1 dµ.
1 2 1
p (x)
Z (cid:18) 2 (cid:19)
• J – divergence (Jeffreys [11], Kullback and Leibler [13])
p (x) p (x)
(2) J(ξ ;ξ ||X) = p (x)ln 1 dµ+ p (x)ln 2 dµ.
1 2 1 p (x) 2 p (x)
Z (cid:18) 2 (cid:19) Z (cid:18) 1 (cid:19)
• Jensen–Shannon divergence (Sibson [19], Burbea and Rao [4], [5])
1
(3) I(ξ ;ξ ||X) = [p (x)lnp (x)+p (x)lnp (x)]
1 2 1 1 2 2
2
Z
p (x)+p (x) p (x)+p (x)
− 1 2 ln 1 2 dµ.
2 2
Z (cid:18) (cid:19) (cid:18) (cid:19)
• AG – divergence (Taneja [22])
p (x)+p (x) p (x)+p (x)
(4) T(ξ ;ξ ||X) = 1 2 ln 1 2 dµ.
1 2
Z (cid:18) 2 (cid:19) 2 p1(x)p2(x)!
p
The three divergence measures given above can be written in terms of Kullback-Leibler
relative information as
(5) J(ξ ;ξ ||X) = K(ξ ;ξ ||X)+K(ξ ;ξ ||X),
1 2 1 2 2 1
1 ξ +ξ ξ +ξ
(6) I(ξ ;ξ ||X) = K(ξ ; 1 2||X)+K(ξ ; 1 2||X)
1 2 1 2
2 2 2
(cid:20) (cid:21)
and
1 ξ +ξ ξ +ξ
(7) T(ξ ;ξ ||X) = K( 1 2;ξ ||X)+K( 1 2;ξ ||X) .
1 2 1 2
2 2 2
(cid:20) (cid:21)
Moreover we have the following equality holding among the three divergence measures
1
(8) I(ξ ;ξ ||X)+T(ξ ;ξ ||X) = J(ξ ;ξ ||X).
1 2 2 1 1 2
4
AG – DIVERGENCE MEASURES 3
Recently, author [24] proved an interesting inequality among these three divergence
measures:
1
(9) I(ξ ;ξ ||X) 6 J(ξ ;ξ ||X) 6 T(ξ ;ξ ||X),
1 2 2 1 1 2
8
where all the probability distributions involved are positive.
In view of (8) the inequalities (9) can be extended as follows:
1 1
(10) I(ξ ;ξ ||X) 6 J(ξ ;ξ ||X) 6 T(ξ ;ξ ||X) 6 J(ξ ;ξ ||X).
1 2 2 1 1 2 2 1
8 4
Based on above notations, the Csisz´ar [6] φ−divergence is given by
p (x)
(11) C (ξ ;ξ ||X) = p (x)φ 1 dµ,
φ 1 2 2 p (x)
Z (cid:18) 2 (cid:19)
X
where the function φ is arbitrary convex function defined in the interval (0,∞). In order
to avoid meaningless expressions, the functions φ satisfy some conventional conditions
given in [6].
In this paper we shall present two parameter generalizations of the AG – divergence.
Also we shall present one parametric unified generalization of the measures (3) and (4).
For two parametric generalization of the measures (2) and (3) refer to Taneja [22]. Also
refer on line book by author [23]. Here, in this paper we shall make connections of gen-
eralized AG – divergence measures with Fisher measure of information. The comparison
of experiments is also studied applying Blackwell’s [2] approach.
2. Unified (r,s)−Arithmetic and Geometric Mean Divergence Measures
In this section, we shall present two different ways of generalizing the AG – divergence
measure (4). Before it we shall give two parametric unified (r,s)−generalization [22] of
the relative information:
Ks(ξ ;ξ ||X), r 6= 1, s 6= 1
r 1 2
(12) Ks(ξ ;ξ ||X) = K1s(ξ1;ξ2||X), r = 1, s 6= 1
r 1 2 K1(ξ ;ξ ||X), r 6= 1, s = 1
r 1 2
K(ξ ;ξ ||X), r = 1, s = 1
1 2
for all r > 0 and −∞ < s < ∞, where
s−1
r−1
Ks(ξ ;ξ ||X) = (s−1) 1 p (x)rp (x)1 rdµ −1 , r 6= 1, s 6= 1
r 1 2 − 1 2 −
" #
(cid:18)Z (cid:19)
Ks(ξ ;ξ ||X) = (s−1) 1 e(s 1)K(ξ1;ξ2 X) −1 , s 6= 1
1 1 2 − − ||
and (cid:2) (cid:3)
K1(ξ ;ξ ||X) = (r −1) 1ln p (x)rp (x)1 rdµ , r 6= 1, s 6= 1.
r 1 2 − 1 2 −
(cid:18)Z (cid:19)
4 INDER JEET TANEJA
2.1. First Generalizations. In (7) replace K(ξ ;ξ ||X) by Ks(ξ ;ξ ||X), we get
1 2 r 1 2
1 ξ +ξ ξ +ξ
(13) 1T s(ξ ;ξ ||X) = Ks( 1 2;ξ ||X)+Ks( 1 2;ξ ||X)
r 1 2 2 r 2 1 r 2 2
(cid:20) (cid:21)
1Ts(ξ ;ξ ||X), r 6= 1, s 6= 1
r 1 2
= 1T1s(ξ1;ξ2||X), r = 1, s 6= 1
1T1(ξ ;ξ ||X), r 6= 1, s = 1
r 1 2
T(ξ ;ξ ||X), r = 1, s = 1
1 2
where
1 p (x)+p (x) r rs−−11
1Ts(ξ ;ξ ||X) = (s−1) 1 1 2 p (x)1 rdx
r 1 2 − 2 2 1 −
(cid:20)(cid:26)Z (cid:18) (cid:19) (cid:21)
p (x)+p (x) r rs−−11
+ 1 2 p (x)1 rdµ −2 , r 6= 1, s 6= 1
2 −
2
)
(cid:20)Z (cid:18) (cid:19) (cid:21)
1T1s(ξ1;ξ2||X) = 21(s−1)−1 e(s−1)K(ξ1+2ξ2;ξ1||X) +e(s−1)K(ξ1+2ξ2;ξ2||X) −2 , s 6= 1
h i
and
1 p (x)+p (x) r
T1(ξ ;ξ ||X) = (r −1) 1 ln 1 2 p (x)1 rdµ
r 1 2 − 2 2 1 −
(cid:20) (cid:18)Z (cid:18) (cid:19) (cid:19)
p (x)+p (x) r
+ ln 1 2 p (x)1 rdµ , r 6= 1
2 −
2
(cid:18)Z (cid:18) (cid:19) (cid:19)(cid:21)
for all r > 0 and −∞ < s < ∞
2.2. Second Generalizations. In particular, when r = s in (13), we get
(14) 1Ts(ξ ;ξ ||X) = (s−1) 1·
s 1 2 −
p (x)+p (x) s p (x)1 s +p (x)1 s
· 1 2 1 − 2 − dµ−1 ,
2 2
(cid:20)Z (cid:18) (cid:19) (cid:18) (cid:19) (cid:21)
for all s 6= 1, s > 0.
We shall use the expression (14) to give the alternative generalizations of AG – diver-
gence measure. This unified way is given by
2Ts(ξ ;ξ ||X), r 6= 1, s 6= 1
r 1 2
(15) 2T s(ξ ;ξ ||X) = 2T1s(ξ1;ξ2||X), r = 1, s 6= 1
r 1 2 2T1(ξ ;ξ ||X), r 6= 1, s = 1
r 1 2
T(ξ ;ξ ||X), r = 1, s = 1
1 2
AG – DIVERGENCE MEASURES 5
for all r > 0 and −∞ < s < ∞, where
2Ts(ξ ;ξ ||X) = (s−1) 1·
r 1 2 −
p (x)+p (x) r p (x)1 r +p (x)1 r rs−−11
· 1 2 1 − 2 − dµ −1 ,
2 2
( )
(cid:20)Z (cid:18) (cid:19) (cid:18) (cid:19) (cid:21)
2Ts(ξ ;ξ ||X) = (s−1) 1 e(s 1)T(ξ1;ξ2 X) −1 ,
1 1 2 − − ||
and (cid:2) (cid:3)
2T1(ξ ;ξ ||X) = (r −1) 1·
r 1 2 −
p (x)+p (x) r p (x)1 r +p (x)1 r
·ln 1 2 1 − 2 − dµ ,
2 2
(cid:26)Z (cid:18) (cid:19) (cid:18) (cid:19) (cid:27)
for all r > 0, r 6= 1, s 6= 1.
In particular, we have
1T s(ξ ;ξ ||X) = 2Ts(ξ ;ξ ||X).
s 1 2 s 1 2
2.3. Composition Relations. We observe that the measures αTs(ξ ;ξ ||X) (α = 1,2)
r 1 2
are continuous with respect to the parameters r and s. This allows us to write them in
the following simplified way
(16) αT s(ξ ;ξ ||X) = CE{αTs(ξ ;ξ ||X)|r > 0, r 6= 1, s 6= 1}, α = 1,2,
r 1 2 r 1 2
where “CE” stands for “continuous extension” with respect to r and s.
Also we can write
(17) 1T s(ξ ;ξ ||X) = N 1T 1(ξ ;ξ ||X)
r 1 2 s r 1 2
and (cid:0) (cid:1)
ξ +ξ ξ +ξ
(18) 2T s(ξ ;ξ ||X) = N K1 1 2;ξ ||X +N K1 1 2;ξ ||X ,
r 1 2 s r 2 1 s r 2 2
(cid:18) (cid:18) (cid:19)(cid:19) (cid:18) (cid:18) (cid:19)(cid:19)
where N : (0,∞) → R(reals) is given by
s
(s−1) 1 e(s 1)x −1 , s 6= 1
− −
(19) N (x) =
s
(x, (cid:2) (cid:3) s = 1
Proposition 1. The measure N (x) given above has the following properties:
s
(i) N (x) > 0 with equality iff x = 0;
s
(ii) N (x) is an increasing function of x;
s
(iii) N (x) is an increasing function of s;
s
(iv) N (x) is strictly convex function of x for s > 1;
s
(v) N (x) is strictly concave function of x for s < 1.
s
6 INDER JEET TANEJA
2.4. Alternative Generalizations. We see that the measure (14) is considered for
s > 0. It is required for the non-negativity of the measure. We can rewrite it in little
different way, where we don’t require this condition. This form is given by
(20) ITs(ξ1;ξ2||X) = [s(s−1)]−1·
p (x)+p (x) s p (x)1 s +p (x)1 s
· 1 2 1 − 2 − dµ−1 ,
2 2
(cid:20)Z (cid:18) (cid:19) (cid:18) (cid:19) (cid:21)
where s 6= 0,1
The measure (20) admits the following limiting cases:
limIT (ξ ;ξ ||X) = I(ξ ;ξ ||X)
s 1 2 1 2
s 0
→
and
limIT (ξ ;ξ ||X) = T(ξ ;ξ ||X),
s 1 2 1 2
s 1
→
where I(ξ ;ξ ||X) and T(ξ ;ξ ||X) are as given by (3) and (4) respectively.
1 2 1 2
In view of these limiting cases, we re-write the measure (20) in the following unified
way
IT (ξ ;ξ ||X), s 6= 0,1
s 1 2
(21) IT (ξ ;ξ ||X) = I(ξ ;ξ ||X), s = 0
s 1 2 1 2
T(ξ ;ξ ||X), s = 1
1 2
3. Relationship with Csisza´r φ−Divergence
We can relate the above generalizations of the AG – divergence measure with the well
known Csisz´ar φ−divergence. It is given as follows:
1
1Trs(ξ1;ξ2||X) = 2 ηs φ(ξ1;ξ2||X)r−11 +ηs φ∗(ξ1;ξ2||X)r−11
h (cid:16) (cid:17) (cid:16) (cid:17)i
and
2Trs(ξ1;ξ2||X) = ηs φ−(ξ1;ξ2||X)r−11 ,
where (cid:16) (cid:17)
(s−1) 1[ys 1−1], s 6= 1
− −
η (y) =
s
(y, s = 1
and φ(ξ ;ξ ||X), φ (ξ ;ξ ||X) and φ (ξ ;ξ ||X) are the φ−divergences of ξ , ξ in the
1 2 ∗ 1 2 − 1 2 1 2
notations of Vajda [26] with
1+x r
φ(x) = ,
2
(cid:18) (cid:19)
1
φ (x) = xφ
∗
x
(cid:18) (cid:19)
and
1
φ (x) = [φ(x)+φ (x)].
− ∗
2
AG – DIVERGENCE MEASURES 7
We can also write the measure (21) in terms of Csisz´ar φ−divergence as follows:
p (x)
(22) IT (ξ ;ξ ||X) = p (x)φ 1 dµ,
s 1 2 Z 2 ITs(cid:18)p2(x)(cid:19)
X
where
[2s(s−1)]−1 (x1−s +1) x+21 s −(x+1) , s 6= 0,1
(23) φ s(x) = x2 lnx+ x+21(cid:2)ln x+21 , (cid:0) (cid:1) (cid:3) s = 0 ,
IT
x+1 ln x+1 , s = 1
(cid:0) (cid:1) (cid:0) (cid:1)
2 2√x
for all x ∈ (0,∞). (cid:0) (cid:1) (cid:16) (cid:17)
Proposition 2. For all r > 0 and −∞ < s < ∞, we have
(i) αT s(ξ ;ξ ||X) > 0 (α = 1,2);
r 1 2
6 2T s(ξ ;ξ ||X), s 6 r
(ii) 1T s(ξ ;ξ ||X) r 1 2
r 1 2 (> 2Trs(ξ1;ξ2||X), s > r
(iii) IT (ξ ;ξ ||X) > 0.
s 1 2
4. Divergence Measures and Sufficiency of Experiments
Blackwell [2] definition of comparison of experiments states that experiment E is suf-
X
ficient for experiment E , denoted by E (cid:23) E , if there exists a stochastic transformation
Y X Y
of X to a random variable Z(X) such that for each θ ∈ Θ the random variable Z(X)
and Y have identical distributions. By E = {Y,S ,Q ;θ ∈ Θ} we shall denote a second
Y Y θ
statistical experiment for which there exists a gpdf g(y|θ) for the distribution Q with
respect to a σ−finite measure µ. According to this definition, if E (cid:23) E , then there
X Y
exists a nonnegative function h satisfying (DeGroot [7])
(24) g(y|θ) = h(y|x)f(x|θ)dµ
Z
X
and
h(y|x)dυ = 1.
Z
X
Changing the order of integration in (24), we get
(25) g (y) = h(y|x)f (x)dµ, i = 1,2.
i i
Z
X
Let I be any measure of information contained in an experiment. If E (cid:23) E implies
X Y
that I > I , then we say that E is as informative as E . This approach is successfully
X Y X Y
carried out by Lindley [14] for Shannon entropy. Goel and DeGroot [10] applied it for
Kullback andLeibler [13]relative information. Ferentinos andPapaioannou[8]appliedfor
α−order generalization of Kullback and Leibler relative information and generalizations
of Fisher measure of information. Author [20] extended it to different generalizations of
J–divergence measure having two scalar parameters. For the I – divergence measure and
their two parametricgeneralizations refer toTaneja et al. [25]. Here ouraimisto compare
8 INDER JEET TANEJA
experiments for the unified (r,s)−AG – divergences given by (13) and (15). Results are
also extended for the measure (21).
Theorem 1. If E (cid:23) E , then αTs(ξ ;ξ ||X) > αTs(ξ ;ξ ||Y) (α = 1,2) for every ξ ,
X Y r 1 2 r 1 2 1
ξ ∈ Ξ, for all r > 0 and −∞ < s < ∞.
2
Proof. Since E (cid:23) E , there exists a function h satisfying (24) and (25), then we can
X Y
write
g (y)+g (y) r
(26) 1 2 g (y)1 r
1 −
2
(cid:18) (cid:19)
= h(y|x) f1(x)+f2(x) dµ r h(y|x)f (x)dµ 1−r.
1
2
(cid:20)Z (cid:18) (cid:19) (cid:21) (cid:20)Z (cid:21)
X X
Applying H¨older’s inequality on the right side of (26), we get
g (y)+g (y) r
(27) 1 2 g (y)1 r
1 −
2
(cid:18) (cid:19)
r
> h(y|x) f1(x)+2f2(x) [h(y|x)f1(x)]1−rdµ, 0 < r < 1
X
R h (cid:16) (cid:17)i
6 h(y|x) f1(x)+2f2(x) r[h(y|x)f1(x)]1−rdµ, r > 1
X
R h (cid:16) (cid:17)i
Hence
g (y)+g (y) r
(28) (g1(y))1−r 1 2 dυ
2
Z (cid:18) (cid:19)
Y
r
> (f1(x))1−r f1(x)+2f2(x) dµ, 0 < r < 1
X
R (cid:16) (cid:17)
6 (f1(x))1−r f1(x)+2f2(x) rdµ, r > 1
X
R (cid:16) (cid:17)
As sign s 1 = sign(s − 1) for r > 1 and sign s 1 6= sign(s − 1) for 0 < r < 1,
r−1 r−1
where sign(x−) = 1 if x > 0 and sign(x) = −1 if x < 0,−then from (28) one gets
(cid:0) (cid:1) (cid:0) (cid:1)
1 g (y)+g (y) r sr−−11
(29) s−1 (g1(y))1−r 1 2 2 dυ
(cid:20)Z (cid:18) (cid:19) (cid:21)
Y
1 f (x)+f (x) r rs−−11
6 s−1 (f1(x))1−r 1 2 2 dµ ,
(cid:20)Z (cid:18) (cid:19) (cid:21)
X
for all r 6= 1, s 6= 1, r > 0.
AG – DIVERGENCE MEASURES 9
Similarly, we can obtain
1 g (y)+g (y) r sr−−11
(30) s−1 (g2(y))1−r 1 2 2 dυ
(cid:20)Z (cid:18) (cid:19) (cid:21)
Y
1 f (x)+f (x) r rs−−11
6 (f2(x))1−r 1 2 dµ ,
s−1 2
(cid:20)Z (cid:18) (cid:19) (cid:21)
X
for all r 6= 1, s 6= 1, r > 0
Adding (29) and (30), subtracting 2(s−1) 1 (s 6= 1) and then dividing by 2, we get
−
1Ts(ξ ;ξ ||X) > 1Ts(ξ ;ξ ||Y), r 6= 1, s 6= 1, r > 0.
r 1 2 r 1 2
Since the unified measure 1Ts(ξ ;ξ ||X) given in (13) is a continuous extension of
r 1 2
1Ts(ξ ;ξ ||X) for the real parameters r and s we can immediately conclude that
r 1 2
1T s(ξ ;ξ ||X) > 1T s(ξ ;ξ ||Y), r > 0,
r 1 2 r 1 2
whenever E (cid:23) E .
X Y
Let us prove now the second part. Since E (cid:23) E , there exist a function h satisfying
X Y
(24) and (25), then we can write
g (y)+g (y) r g (y)1 r +g (y)1 r
1 2 1 − 2 −
(31)
2 2
(cid:18) (cid:19) (cid:18) (cid:19)
= 1 h(y|x)f (x)dµ 1−r h(y|x) f1(x)+f2(x) dµ r
1
2 2
(cid:20)Z (cid:21) (cid:20)Z (cid:18) (cid:19) (cid:21)
X X
+ 1 h(y|x)f (x)dµ 1−r h(y|x) f1(x)+f2(x) dµ r.
2
2 2
(cid:20)Z (cid:21) (cid:20)Z (cid:18) (cid:19) (cid:21)
X X
Applying H¨older’s inequality in (31), integrating over Y, and using the fact that
h(y|x)dυ = 1, we get
Y
(R32) g1(y)+g2(y) r g1(y)1−r +g2(y)1−r dυ
2 2
Z (cid:18) (cid:19) (cid:18) (cid:19)
Y
> f1(x)+f2(x) r f1(x)1−r+f2(x)1−r dµ, 0 < r < 1
2 2
X
R (cid:16) (cid:17) (cid:16) (cid:17) .
6 f1(x)+f2(x) r f1(x)1−r+f2(x)1−r dµ, r > 1
2 2
X
As sign rs−11 =signR(s(cid:16)−1) for r (cid:17)> 1(cid:16)and sign rs−11 (cid:17)6= sign(s−1) for 0 < r < 1, we
have − −
(cid:0) (cid:1) (cid:0) (cid:1)
1 g (y)+g (y) r g (y)1 r +g (y)1 r rs−−11
(33) 1 2 1 − 2 − dυ
s−1 2 2
(cid:20)Z (cid:18) (cid:19) (cid:18) (cid:19) (cid:21)
Y
1 f (x)+f (x) r f (x)1 r +f (x)1 r rs−−11
6 1 2 1 − 2 − dµ .
s−1 2 2
(cid:20)Z (cid:18) (cid:19) (cid:18) (cid:19) (cid:21)
X
10 INDER JEET TANEJA
Subtracting (s−1) 1 (s 6= 1) on both sides of (33), we get
−
2T(ξ ,ξ ||X) > 2T(ξ ,ξ ||Y), r 6= 1, s 6= 1, r > 0,
1 2 1 2
and consequently, we have
2T s(ξ ,ξ ||X) > 2T s(ξ ,ξ ||Y), r > 0,
r 1 2 r 1 2
whenever E (cid:23) E . (cid:3)
X Y
Theorem 2. If E (cid:23) E , then IT (ξ ;ξ ||X) > IT (ξ ;ξ ||Y) for every ξ , ξ ∈ Ξ, for
X Y s 1 2 s 1 2 1 2
all s ∈ (−∞,∞).
The proof of the above theorem is based on the following lemmas.
Lemma 1. (Joint convexity). If φ : (0,∞) → R be convex, then C (ξ ;ξ ||X) jointly
φ 1 2
convex for every ξ , ξ ∈ Ξ. Moreover if φ(1) = 0, then C (ξ ;ξ ||X) > 0.
1 2 φ 1 2
Lemma 2. If E (cid:23) E , then C (ξ ;ξ ||X) > C (ξ ;ξ ||Y) for every ξ , ξ ∈ Ξ, provided
X Y φ 1 2 φ 1 2 1 2
φ is convex.
Lemma 1 is due to Csisz´ar [6] and Lemma 2 is due to Ferentinos and Papaioannou [8].
Proof. of Theorem 1. In view of Lemmas 1 and 2, it is sufficient to prove the convexity
of the function φ (x) given by (18). It is in view of the following derivatives:
s
IT
s(x+1)1−s+(1 s)(xs+1)(x+1)−s
2x − 2 2 , s 6= 0,1
2s(s 1)
(34) φ′ (x) = 1 1−x 1 −l−nx−2ln 2 , s = 0
ITs 4 − x+1
1 lnx+ln 2 , s = 1
2 (cid:2) x+1 (cid:0) (cid:1)(cid:3)
and (cid:2) (cid:0) (cid:1)(cid:3)
81 (xs−2 +1) x+21 −s−1, s 6= 0,1
(35) φ′′ s(x) = 41 x2x(2x++11) ,(cid:0) (cid:1) s = 0 .
IT
(cid:16)1 , (cid:17) s = 1
2x(x+1)
Thus we have φ (x) > 0 for all x > 0, and hence, φ (x) is convex for all x > 0.
′I′Ts ITs
Also, we have φ (1) = 0. In view of this we can say that I– & T– divergence of type
s
IT
s given by (21) is nonnegative and convex in the pair of probability distributions P and
Q. (cid:3)
5. φ−Divergence and Fisher Information Matrix
Consider a family M = {P ,θ ∈ Θ} of probability measures on a measurable space
θ
(X,A) dominated by a finite or σ−finite measure µ. The parameter space Θ can either
be an open subset of the real line or an open subset of n−dimensional Euclidean space
Rk. Let f(x,θ) = dPθ. Let Γ = {f(x,θ)|x ∈ X, θ ∈ Θ}.
dµ
