Admissible Estimator of the Eigenvalues of Variance-Covariance Matrix for Multivariate Normal Distributions--Detailed Proof--

0.3 MB·English

by  Yo Sheena

Admissible Estimator of the Eigenvalues of
Variance-Covariance Matrix for Multivariate Normal
9
Distributions
0
0
2

g Yo Sheena∗ and Akimichi Takemura†
u
A
August 2009

2
1

]
T
S Abstract
.
h
An admissible estimator of the eigenvalues of the variance-covariance matrix
t
a is given for multivariate normal distributions with respect to the scale-invariant
m
squared error loss.
[

1 AMS(2000) Subject Classification: Primary 62C15; Secondary 62F10
v
Key words and phrases: covariance matrix, Wishart distribution, squared error loss, Kar-
1
0 lin’s method
7
1
.
8 1 Introduction
0
9
0 The variance-covariance matrix of a multivariate normal distribution is usually estimated
:
v by the sample variance-covariance matrix, which is distributed as Wishart distribution.
i
X Let S be distributed according to Wishart distribution W (ν,Σ), where p (≥ 2) is the
p
r dimension, ν (≥ p) is the degree of freedom, and Σ is the variance-covariance matrix of
a
the original multivariate normal distribution.
In many situations of multivariate analysis, such as principle component analysis,
canonical correlation analysis, we need to estimate the eigenvalues of Σ rather than Σ
itself. Also, many test statistics in multivariate analysis have distributions determined
solely by the eigenvalues of Σ because of their invariance property under some natural
transformations.
For the estimation of the eigenvalues of Σ, the corresponding sample eigenvalues of
S are usually used, but their distribution is quite complicated and makes it difficult
to obtain mathematically clear results. Especially in a decision theoretic approach we
encounter difficulty since we essentially need the calculation of the risk (the expectation
∗Department of Economics, Shinshu University
†Graduate School of Information Science and Technology, University of Tokyo
1
of a loss) with respect to the distribution of the eigenvalues for finite degrees of freedom
ν. Mainly because of this difficulty, there exist only a few literature which directly deal
with the estimation of the eigenvalues from the standpoint of the decision theory. Dey
(1988) and Jin (1993) derive estimators which dominate the traditional estimators under
the (non-scale-invariant) quadratic loss function. In view of the decision theory, one of
the important tasks is to derive an admissible estimator, but it has been an unsolved
problem so far. The aim of this paper is the derivation of an admissible estimator. For
the proof of admissibility, we adopted the method of Gosh and Singh (1968), in which
they proved the admissibility of an estimator for the reciprocal of the scale parameter of
Gamma distributions using “Karlin’s method” (Karlin (1958)).
Here we formally state the framework. Let λ ≥ ... ≥ λ > 0 denote the eigenvalues
1 p
of Σ, while l ≥ ... ≥ l > 0 are the eigenvalues of S. As is well known, the distribution
1 p
of l = (l ,...,l ) depends only on λ = (λ ,...,λ ). For an estimator
1 p 1 p
ψ(l) = (ψ (l),...,ψ (l)),
1 p
we measure the loss by the scale-invariant squared error loss function
p p
(ψ (l)−λ )2/λ2 = (ψ (l)/λ −1)2. (1)
i i i i i
i=1 i=1
X X
2 Main Result
Before stating the main result as a theorem, we introduce some notation. For a vector
x = (x ,...,x ) and a set of powers α = (α ,...,α ) the monomial xα1 ...xαp is denoted
1 p 1 p 1 p
by xα. If α = α = ··· = α is common, we denote the monomial by xα. Let H =
1 p
(h ) denote a p-dimensional orthogonal matrix. The group of p dimensional orthogonal
ij
matrices is denoted by O(p) and µ is the invariant probability measure on O(p). Since
we mainly work with the reciprocal of the population eigenvalue, t = λ−1 (j = 1,...,p),
j j
more often than λ itself, we define the following notation for convenience.
j
Tb = {t = (t ,...,t ) | (0 ≤) a < t ≤ ··· ≤ t < b (≤ ∞)},
a 1 p 1 p
G(l) = l(ν−p−1)/2 (l −l ),
i j
i<j
Y
p p
1
F(t|l) = exp − t l h2 dµ(H),
2 i j ij
ZO(p) i=1 j=1 !
XX
∂F(t|l)
∂ F(t|l) = , i = 1,...,p.
i
∂t
i
The density function f(l|t) of l is given by
f(l|t) = K tν/2 G(l)F(t|l), (2)
where K is a constant (not depending on l and t). Our main result is given as follows.
2
Theorem 1 For 1 ≤ i ≤ p, let
∂ F(t|l) tν/2−1 t2dt
ν −1 T∞ i i
ψ∗(l) = − +1 0 .
i 2 F(t|l) tν/2−1 t2dt
R T∞ i
(cid:16) (cid:17) 0
R
The estimator ψ∗(l) = (ψ∗(l),...,ψ∗(l)) is admissible with respect to the loss function
1 p
(1).
Proof of this theorem is given in Section 4.
Notice that ψ∗(l)(1 ≤ i ≤ p) can be rewritten as
i
p
ψ∗(l) = τ (l)l , (3)
i ij j
j=1
X
where
h2 exp −1 p p t l h2 tν/2−1 t2 dµ(H)dt
T∞ O(p) ij 2 s=1 k=1 s k sk i
(ν +2)τ (l) = 0 (4)
ij R T∞R O(p)exp −(cid:0)21 Pps=1 Ppk=1tslkh2sk (cid:1)tν/2−1 t2i dµ(H)dt
0
R R (cid:0) P P (cid:1)
It is easily seen that τ (l) (1 ≤ i,j ≤ p) is scale-invariant, that is, τ (cl) = τ (l) for any
ij ij ij
positive constant c. Furthermore τ ’s are nonnegative and
ij
p
1
τ (l) = ,
ij
ν +2
j=1
X
since h2 = 1. This means that ψ∗(l) = (ψ∗(l),...,ψ∗(l)) is an estimator which
j ij 1 p
shrinks l/(ν +2).
P
ψ∗(l) (1 ≤ i ≤ p) has another useful expression;
i
p
ψ∗(l) = τ˜ (l) l (5)
i ij i
(cid:18)j=1 (cid:19)
X
where
l
τ˜ (l) = τ (l) j (1 ≤ i,j ≤ p).
ij ij
l
i
τ˜ (l) is again bounded and scale-invariant. (Lemma 8 in Section 4.)
ij
3 Some simulation studies
In this section, we report a small-scale simulation result which illustrates the behavior
of the admissible estimator ψ∗(l) = (ψ∗(l),...,ψ∗(l)) compared to the simple estimator
1 p
ϕ∗(l) = l/(ν +2).
3
Table 1: p = 2
ν = 5 ν = 20
λ risk of ψ∗ risk of ϕ∗ risk of ψ∗ risk of ϕ∗
(1.0,1.0) 0.623 0.785 0.186 0.266
(1.0,0.9) 0.605 0.756 0.170 0.230
(1.0,0.8) 0.557 0.678 0.150 0.193
(1.0,0.7) 0.550 0.636 0.161 0.188
(1.0,0.6) 0.565 0.653 0.169 0.183
(1.0,0.5) 0.571 0.634 0.186 0.182
(1.0,0.4) 0.559 0.606 0.189 0.175
(1.0,0.3) 0.605 0.643 0.202 0.183
(1.0,0.2) 0.591 0.614 0.189 0.180
(1.0,0.1) 0.608 0.627 0.192 0.189
Using the variable transformation,
t = r u,t = r u,...,t = r u,t = u
1 1 2 2 p−1 p−1 p
we can easily notice that (4) equals
h2 (1 p p r l h2 )−(pν/2+2)rν/2−1 ·····rν/2−1r2 dµ(H)dr
R1 O(p) ij 2 s=1 k=1 s k sk 1 p−1 i
0 , (6)
R R (1 Pp Pp r l h2 )−(pν/2+2)rν/2−1 ·····rν/2−1r2 dµ(H)dr
R1 O(p) 2 s=1 k=1 s k sk 1 p−1 i
0
where r =R (rR,...,rP) anPd R1 = {r | 0 < r < ··· < r < 1}. For given p,ν,l, we
1 p−1 0 1 p−1
calculated τ (l) (1 ≤ i,j ≤ p) using 100 random points uniformly distributed respectively
ij
on O(p) and R1.
0
We carried out a simulation for p = 2 and p = 3. In each case, ν equals 5 or 20 and
several patterns of population eigenvalues λ are given. We used 1000 Wishart random
matrices for the risk calculation for each p,ν,λ. Table 1 and 2 show the simulation
results. We notice that if the population eigenvalues are close to one another, then the
estimator ψ∗ performs better than ϕ∗, while as population eigenvalues get dispersed,
the risk of ψ∗ rapidly increases. This indicates the admissibility of ψ∗ is acquired by
the good performance when population eigenvalues are equal at the expense of the poor
performance when they are scattered.
4 Proofs
In this section we give a proof of Theorem 1. Since the proof is long and complicated,
first we give an outline of the proof for readability. Then we give a full proof in a series
of lemmas. Long proofs of some lemmas are given in Appendix.
4
Table 2: p = 3
ν = 5 ν = 20
λ risk of ψ∗ risk of ϕ∗ risk of ψ∗ risk of ϕ∗
(1.0,1.0,1.0) 0.860 1.472 0.254 0.525
(1,1/2,1/4) 0.730 1.071 0.326 0.286
(1,1/3,1/9) 0.859 1.018 0.726 0.291
(1.0,1.0,0.5) 0.766 1.286 0.243 0.363
(1.0,0.5,0.5) 0.761 1.076 0.263 0.337
(1.0,1.0,0.2) 0.933 1.244 0.527 0.377
(1.0,0.2,0.2) 0.979 1.101 0.348 0.355
(1.0,1.0,0.1) 2.073 1.213 1.480 0.362
(1.0,0.1,0.1) 1.838 1.148 0.756 0.354
An outline of the proof. Assume that some estimator ψ(l) = (ψ (l),...,ψ (l)) dom-
1 p
inates ψ∗(l) = (ψ∗(l),...,ψ∗(l)). Then for all t ∈ T∞,
1 p 0
p p
t2 (ψ (l)−t−1)2f(l|t)dl ≤ t2 (ψ∗(l)−t−1)2f(l|t)dl, (7)
i i i i i i
i=1 ZL i=1 ZL
X X
where L = {l|l ≥ ··· ≥ l > 0}. The right side of (7), the risk of ψ∗(l), is always finite
1 p
((18) of Lemma 1). Together with this finiteness, (7) leads to the inequality
p p
T (t) ≤ 2 t2 (ψ∗(l)−ψ (l))(ψ∗(l)−t−1)f(l|t)dl, (8)
i i i i i i
i=1 i=1 ZL
X X
where
T (t) = T (t ,...,t ) = t2 (ψ (l)−ψ∗(l))2f(l|t)dl. (9)
i i 1 p i i i
ZL
We also denote
T (a;b) = T (a,...,a,b,...,b). (10)
im i
m p−m
We will show that (8) implies pi=1Ti(t)|≡{z0 a}nd|h{eznc}e ψ(l) is almost surely equal to
ψ∗(l)onL. Weintegratethebothsidesof (8)w.r.t.themeasuret−1dt = ( p t−1)dt ...dt
P j=1 j 1 p
over Tb (0 < a < b < ∞). Then we have
a Q
p p
T (t) ≤ 2 (ψ∗(l)−ψ (l)) (ψ∗(l)−t−1)t2f(l|t)t−1dtdl. (11)
i i i i i i
i=1 ZTba i=1 ZL ZTba
X X
The interchange of the integrals is guaranteed by (19) in Lemma 1.
5
Let
∂ F(t|l) tν/2−1 t2dt
T∞ i i
H (l;a,b) = 0 F(t|l) tν/2−1 t2dt− ∂ F(t|l)tν/2−1t2dt
i F(t|l) tν/2−1 t2dt i i i
R T∞0 i ZTba ZTba
ν
= −R +1 ψ∗(l) F(t|l) tν/2−1 t2dt− ∂ F(t|l)tν/2−1t2dt. (12)
2 i i i i
Tb Tb
(cid:16) (cid:17) Z a Z a
Then each integral of the right-hand side of (11) is decomposed as follows;
(ψ∗(l)−ψ (l)) (ψ∗(l)−t−1)t2f(l|t)t−1dtdl = R (a,b)+I (a,b),
i i i i i i i
ZL ZTba
where
ν −1
R (a,b) = −K +1 (ψ∗(l)−ψ (l))G(l)H (l;a,b)dl,
i 2 i i i
(cid:16) (cid:17) ZL
ν −1
I (a,b) = −K +1 (ψ∗(l)−ψ (l))G(l)
i 2 i i
(cid:16) (cid:17) ZL
ν
× ∂ F(t|l) tν/2−1 t2dt+ +1 F(t|l) tν/2−1 t dt dl.
i i 2 i
(cid:20)ZTba ZTba (cid:16) (cid:17) (cid:21)
I (a,b) is bounded by the integral I˜(a,b) defined as
i i
ν −1
I˜(a,b) = K +1 |ψ∗(l)−ψ (l)|G(l)
i 2 i i
(cid:16) (cid:17) ZL
ν
× ∂ F(t|l)tν/2−1t2dt+ +1 F(t|l)tν/2−1t dt dl. (13)
i i 2 i
(cid:12)ZTba ZTba(cid:16) (cid:17) (cid:12)
(cid:12) (cid:12)
(cid:12) (cid:12)
Lemma 4 says tha(cid:12)t there exist constants cim (i = 1,...,p, m = p − 1,(cid:12)p) which are
independent of a,b and satisfy
p
I˜(a,b) ≤ c T1/2(a;b).
i im im
m=p−1
X
Consequently with c = 2max c we have
i,m im
p p p p
T (t)t−1dt ≤ c T1/2(a;b)+2 R (a,b) (14)
i im i
Tb
i=1 Z a i=1 m=p−1 i=1
X X X X
If we substitute r−1 and r (r ≥ 1) respectively into a and b in (14), we have
p p p p
T (t)t−1dt ≤ c T1/2(r−1;r)+2 R (r−1,r), (15)
i im i
i=1 ZT(r) i=1 m=p−1 i=1
X X X X
6
Lemma 1 Lemma 12
Theorem 1
Lemma 4 Lemma 11 Lemma 10
Lemma 3 Lemma 5 Lemma 8 Lemma 9
Lemma 2 Lemma 7
Lemma 6
Figure 1: Relations among lemmas and Theorem 1
where T(r) = Tr . By Lemma 10 there exists a constant M such that
r−1
T (r−1;r) ≤ M, 1 ≤ ∀i ≤ p, 0 ≤ ∀m ≤ p, ∀r ≥ 1.
im
Since
lim R (r−1,r) = 0 (16)
i
r→∞
by Lemma 11, the continuity of R (r−1,r) implies that R (r−1,r) is also bounded on the
i i
region r ≥ 1 for each i. Therefore the left-hand side of (15) is bounded and hence the
increasing sequence lim T (t)t−1dt converges for each i. This means
r→∞ T(r) i
R
T (t)t−1dt < ∞. (17)
i
T∞
Z 0
By Lemma 12, the inequalities (15), (16) and (17) imply
T (t) = 0, a.e. in T∞, 1 ≤ i ≤ p.
i 0
Hence ψ (l) = ψ∗(l) a.e. on L, 1 ≤ i ≤ p. (End of outline.)
i i
The following lemmas (see Figure 1 for their relation to Theorem 1) constitute a full
proof of Theorem 1.
In the following E [·] denotes the expected value w.r.t. the distribution of the eigen-
t
values of Wishart matrix S with ν degrees of freedom and the population eigenvalues
(t−1,...,t−1). We often use the inequality (x−y)2 ≤ 2x2 + 2y2, x,y ∈ R, to bound an
1 p
integral from above. diag(a ,...,a ) denotes a diagonal matrix with diagonal elements
1 p
a ,...,a . If A ,...,A are square matrices of appropriate sizes diag(A ,...,A ) denotes
1 p 1 k 1 k
a block-diagonal matrix.
7
Lemma 1
p
t2 (ψ∗(l)−t−1)2f(l|t)dl < ∞ (18)
i i i
i=1 ZL
X
p
|ψ∗(l)−ψ (l)||ψ∗(l)−t−1|f(l|t)dl t−1t2dt < ∞ (19)
i i i i i
i=1 ZTba ZL
X
Proof. The integral in (18) can be written as E [ p (ψ∗(l) − t−1)2t2]. It suffices to
t i=1 i i i
prove E [(ψ∗(l))2] < ∞ for each i. From (3) we have (ν +2)ψ∗(l) ≤ p l = trS and
t i P i j=1 j
E [(trS)2] < ∞.
t
P
Now we will prove (19). The integral can be written as
p
t2E [|ψ∗(l)−ψ (l)||ψ∗(l)−t−1|]t−1dt.
i t i i i i
Tb
Z a i=1
X
Since the closure of Tb is a compact region and the integrand is continuous in t on the
a
closure of Tb, it suffices to prove p E [|ψ∗(l)−ψ (l)||ψ∗(l)−t−1|] < ∞. By Cauchy-
a i=1 t i i i i
Schwartz inequality, the following relationship holds.
P
p p
E [|ψ∗(l)−ψ (l)||ψ∗(l)−t−1|] ≤ E [(ψ∗(l)−t−1)2 +|ψ (l)−t−1||ψ∗(l)−t−1|]
t i i i i t i i i i i i
i=1 i=1
X X
p
≤ E [(ψ∗(l)−t−1)2]
t i i
i=1
X
p
+ E [(ψ∗(l)−t−1)2] 1/2 E [(ψ (l)−t−1)2] 1/2
t i i t i i
i=1
X(cid:8) (cid:9) (cid:8) (cid:9)
p
≤ 2E [(ψ∗(l)−t−1)2].
t i i
i=1
X
The last inequality holds since ψ(l) dominates ψ∗(l).
Lemma 2 Let α > 0, β > 0 and b > a > 0. Then
b [α+1] (α)
exp(−xβ)xαdx ≤ jaα−j exp(−aβ),
βj+1
Za j=0
X
where [x] is the largest integer that is not larger than x and (α) = α(α−1)···(α−j+1)
j
is the falling factorial.
8
Proof. Note that (α) ≥ 0 for 0 ≤ j ≤ [α+1] and (α) ≤ 0. By integration by parts
j [α+2]
b 1 α b
exp(−xβ)xαdx = (aαexp(−aβ)−bαexp(−bβ))+ exp(−xβ)xα−1dx
β β
Za Za
= ...
[α+1]
(α)
= j aα−jexp(−aβ)−bα−j exp(−bβ)
βj+1
j=0
X (cid:0) (cid:1)
(α) b
+ [α+2] exp(−xβ)xα−[α+2]dx
β[α+2]
Za
[α+1]
(α)
≤ jaα−jexp(−aβ).
βj+1
j=0
X
Lemma 3 Let x = (x ,...,x ), X = {x|(0 <)a ≤ x ≤ ··· ≤ x ≤ b(< ∞)}, α > 0,
1 p 1 p i
β > 0, i = 1,...,p. Then
i
p
xα exp − β x dx
i i
ZX i=1 !
X
is bounded by a linear combination of finite terms each of which has the form
p
βγ aPpi=1(αi+γi+1) exp −a βi ,
!
i=1
X
with some integer vector γ = (γ ,...,γ ). The coefficients of the linear combination are
1 p
positive and independent of a, b, β (i = 1,...,p).
i
Proof. By enlarging the region of integral to the direct product [a,b]p, we have
p b b
xα exp − β x dx ≤ xα1 exp(−β x )dx ×···× xαp exp(−β x )dx .
i i 1 1 1 1 p p p p
ZX i=1 ! Za Za
X
Applying Lemma 1 to each term on the right-hand side, we obtain the lemma.
Lemma 4 Let I˜(a,b) be defined as in (13). Then for 1 ≤ i ≤ p,
i
p
I˜(a,b) ≤ c T1/2(a;b)
i im im
m=p−1
X
with some constants c (m = p−1,p) which are independent of a,b.
im
9
Proof.
ν −1
I˜(a,b) = K +1 |ψ∗(l)−ψ (l)|G(l)
i 2 i i
(cid:16) (cid:17) ZL
× tν/2−1 F(t|l)tν/2+1 ti=ti+1 dt dl, (20)
j i ˆi
(cid:12)(cid:12)ZTi(cid:18)Yj6=i (cid:19)h iti=ti−1 (cid:12)(cid:12)
(cid:12) (cid:12)
where (cid:12) (cid:12)
(cid:12) (cid:12)
t = (t ,...,t ,t ,...,t ),
ˆi 1 i−1 i+1 p
Ti = {t |t (≡ a) < t ≤ ··· ≤ t ≤ t ≤ ··· ≤ t < t (≡ b)}.
ˆi 0 1 i−1 i+1 p p+1
(20) is bounded by I˜ +I˜ , where
i1 i2
K
I˜ = |ψ∗(l)−ψ (l)|G(l) tν/2−1 F(t|l) tν/2+1dt dl
i1 ν/2+1 i i j i+1 ˆi
ZL ZTi(cid:18)j6=i (cid:19) (cid:12)ti=ti+1
Y (cid:12)
K (cid:12)
I˜ = |ψ∗(l)−ψ (l)|G(l) tν/2−1 F(t|l)(cid:12) tν/2+1dt dl.
i2 ν/2+1 i i j i−1 ˆi
ZL ZTi(cid:18)j6=i (cid:19) (cid:12)ti=ti−1
Y (cid:12)
(cid:12)
First we prove the lemma for the case i 6= 1,p. Let y = p(cid:12) l h2 . Then the inner
s j=1 j sj
integrals of I˜ and I˜ are rewritten respectively as
i1 i2 P
1
tν/2−1 tν exp − t y +t (y +y ) dt dµ(H) (21)
j i+1 2 s s i+1 i i+1 ˆi
ZO(p)ZTi j6=i,i+1 s6=i,i+1 !!
Y X
1
tν/2−1 tν exp − t y +t (y +y ) dt dµ(H). (22)
j i−1 2 s s i−1 i i−1 ˆi
ZO(p)ZTi j6=i,i−1 s6=i,i−1 !!
Y X
If we use Lemma 3, the inner integrals of (21) and (22) are seen to be bounded by linear
combinations (whose coefficients arenonnegative andindependent ofa,b,y (s = 1,...,p))
s
of such terms as
a
aP1≤s6=i≤p(α˜s+γs+1) y˜sγsexp −2 y˜s ,
!
s6=i 1≤s6=i≤p
Y X
where for (21),
α˜ = ν/2−1, y˜ = y , if 1 ≤ s 6= i,i+1 ≤ p, (23)
s s s
α˜ = ν, y˜ = y +y , if s = i+1.
s s i i+1
and for (22),
α˜ = ν/2−1, y˜ = y , if 1 ≤ s 6= i,i−1 ≤ p, (24)
s s s
α˜ = ν, y˜ = y +y , if s = i−1.
s s i i−1
10