A new subtraction-free formula for lower bounds of the minimal singular value of an upper bidiagonal matrix

0.14 MB·

by  Takumi Yamashita

A new subtraction-free formula for lower bounds
of the minimal singular value of an upper bidiag-
onal matrix
TakumiYamashita1, KinjiKimura2 and YusakuYamamoto3
4
1 Abstract
0
2
Tracesofinversepowersofapositivedefinitesymmetrictridiagonalma-

n
trixgivelowerboundsoftheminimalsingularvalueofanupperbidiag-
a
J onal matrix. In a preceding work, a formula for the traces which gives

0 thediagonalentries oftheinversepowersispresented. Inthispaper, we
1 presentanotherformulawhichgivesthetracesbasedonaquitedifferent

] ideafromtheoneintheprecedingwork. Anefficientimplementationof
A
theformulaforpracticeis alsopresented.
N
.
h
t 1 Introduction
a
m
Alowerboundoftheminimalsingularvalueofamatrixhashistorically
[
beeninvestigatedforestimationofanupperboundoftheconditionnum-
1
berofamatrix. Asanotherapplication,suchalowerboundforanupper
v
0 bidiagonal matrix may be used to accelerate convergence of iteration in
5
some singular value computing algorithms [1, 3, 8, 9]. In the standard
3
2 procedure for computing the singular values, one first reduces the in-
1. put matrix to an upper bidiagonal matrix by orthogonal transformations
0 and then computes the singular values of the obtained upper bidiagonal
4
matrix by some iterative algorithms. The iterative algorithms referred
1
: aboveuse a techniquecalled the shift of origin. This techniquerequires
v
i aquantitycalledashift. Alowerboundoftheminimalsingularvalueof
X
theupperbidiagonalmatrixcan beused todeterminethisquantity.
r
a Several lowerbounds of the minimalsingular valueof a matrix have
been proposed. For example, see [2, 4, 5, 7, 10, 14]. For an upper
bidiagonal matrix B, where all the diagonal and the upper subdiagonal
entries are positive, the traces Tr((BB ) M) (M = 1,2,...) give lower
⊤ −
bounds of the minimal singular value of B. For example, the following
128-20Kojogaoka,Otsu,Shiga520-0821Japan,Tel.:[phone],Fax:[phone],e-mail:
[email protected]
2GraduateSchoolofInformatics,KyotoUniversity,Yoshida-Honmachi, Sakyo-ku,Kyoto,Kyoto606-
8501Japan,e-mail:[email protected]
3GraduateSchoolofInformatics andEngineering, TheUniversity ofElectro-Communications, 1-5-1,
Chofugaoka,Chofu,Tokyo182-8585,Japan,e-mail:[email protected]
1
twoquantities
̺ = (Tr((BB⊤)−1))−12
and
1 N
υ = ,
sTr((BB⊤)−1) · vuuuuuut1+ s(N −1) N · (TTrr((((BBBB⊤⊤))−−12)))2 −1!
where N is the matrix size of B, are such lower bounds. For details, see
[8] by von Matt. Forcomputationof thetraces of (BB ) 1 and (BB ) 2,
⊤ − ⊤ −
vonMatt[8]alsopresentedamethodtocomputethediagonalentriesof
theseinverses. Ontheotherhand,Kimuraetal. [6]presentedasequence
oflowerboundsoftheminimalsingularvalueofB. Theselowerbounds
θ (B) (M = 1,2,...) are given with the traces J (B) = Tr((B B) M) =
M M ⊤ −
Tr((BB ) M)as
⊤ −
θM(B) = (JM(B))−21M, M = 1,2,....
It holds ̺ = θ (B). They increase monotonically and converge to the
1
minimal singular value σ (B) of B as M goes to infinity [6, Theorem
min
3.1], thatis,
θ (B) < θ (B) < < σ (B),
1 2 min
···
lim θ (B) = σ (B).
M min
M
→∞
Kimuraet al. [6] also presented a formulafor computationofthe traces
ofJ (B)foranarbitrarypositiveintegerM. Thisformulagivesthediag-
M
onalentries oftheinversepowers(B B) M and(BB ) M (M = 1,2,...)
⊤ − ⊤ −
in a form of recurrence relation. In [12], Yamashita et al. derived an-
otherformulaforthesediagonalentriesstartingfromtheformulain[6].
While theformula in [6] includes subtraction in it in the case of M 2,
≥
theformulain[12]consistsofonlyaddition,multiplicationanddivision
among positive quantities. Namely, the formula in [12] is “subtraction-
free”. This property clearly excludes any possibility of cancellation er-
ror.
In this paper, we present another formula for computation of the
traces J (B) (M = 1,2,...). This formula is also subtraction-free. We
M
derivetheformulawithanideawhichisquitedifferentfromthatin[12].
We do not aim to obtain the diagonal entries of (B B) M or (BB ) M
⊤ − ⊤ −
(M = 1,2,...) in the derivation. Instead, equations on the determinant
and the entries of A λI, where A is B B or BB , λ is a parameter and
⊤ ⊤
−
I is the unit matrix, are considered. The new formula is obtained by
2
differentiating these equations with respect to the parameter λ repeat-
edly. Computational cost for the traces are also discussed. Moreover,
an implementationfor the trace J (B) which is useful in practice is pre-
2
sented. This implementation has the following merits compared with
thatin [12].
Thenumberofoperationsissmallercomparedwiththeimplemen-
•
tationin [12].
Only one “loop” is required while the implementation in [12] re-
•
quirestwoloops.
No“array” is necessary,in contrasttotheimplementationin [12].
•
This paper is organized as follows. In Section 2, the new formula
is derived. In Section 3, computational cost for the new formula is dis-
cussed. In Section 4, an efficient implementation for the trace J (B) is
2
presented. Section5 isdevotedforconcludingremarks.
2 Derivation of the formula for the traces
Let us consider an N N real upper bidiagonal matrix B, where all the
×
diagonal and the upper subdiagonal entries are positive. In this sec-
tion, we derive the new formula for the traces J (B) = Tr((B B) p) =
p ⊤ −
Tr((BB ) p) for an arbitrary positive integer p in a form of recurrence
⊤ −
relation. From these traces, lower bounds of the minimal singularvalue
of B are obtained. In the context of singular value computation, we can
assume the positivity of the diagonal and the upper subdiagonal entries
of B without loss of generality [1]. We present two recurrence relations
inSections2.1and2.2. Theideastoderivatetherecurrencerelationsare
quitedifferent fromthosein [12].
Hereafter, we fix somenotations. Let Bbe
√q √e
1 1
√q √e
2 2
whereBq=> 0 for i = 1,.....,.N a√nd.q.N.e−1> 0√√efoqNrN−1i =,1,...,N 1. Let I(b1e)
i i
theN N unitmatrix. Weusetheconvention k = 0if j−> k. Letλbe
× i=j
aparameter.
P
3
2.1 Derivation-typeI
Inthissubsection,weactuallyderivetheformula. Lettheeigenvaluesof
B Bbeλ ,...,λ . Foranarbitrarypositiveinteger p,theeigenvaluesof
⊤ 1 N
(B B) p are λ p,...,λ p. Then, the summation N λ p is the trace of
⊤ − −1 −N i=1 −i
(B B) p. We derive a formula to compute this summation. The matrix
⊤ −
P
B Bisgivenas
⊤
q √q e
1 1 1
√q e q +e ...
B B =  1 1 2 1 .
It can b⊤ereadily verifiedt.h.a.t we√obqtNa.−i.1n.etNh−e1fol√qloNqwN+i−n1egeNN−m−11atrix
q q e
1 1 1
1 q +e ...
A =  2 1 
by similarity trans.f.o.rmat.i1.o.n. qqTNNh−+e1neeN,N−−t1h1ematrices A and B⊤B have the
same eigenvalues. A key point of this derivation is to express the de-
terminant of A λI in two ways. As the first way, the determinant is
−
expressedas
N
det(A λI) = (λ λ). (2)
i
− −
i=1
Y
Forthesecond way,let usconsiderdecompositionofthematrix
q λ q e
1 1 1
−
1 q +e λ ...
A λI =  2 1 − 
intothe−matrixproductexpres.s.e.das .1.. qNq+N−e1Ne−N1−−1 λ 
qˆ(0) 1 eˆ(0)
1 1
1 qˆ(0) 1 ...
whereAqˆ−(0)λfIo=r i= 1,....2.,.N a.1n..d eˆqˆ(0(N)0)fori = 1,...,.N.. eˆ1(N10−a)1ref,unctions of
λ. Thesei functionsare repeatedlyi differentiated in th−e discussionshown
4
below. The superscript (0) indicates that the function has not been dif-
ferentiated yet. Comparison of the diagonal and the upper subdiagonal
entries of A λI gives
−
q +e λ = qˆ(0) +eˆ(0), i = 1,...,N, (3)
i i−1 − i i−1
qe = qˆ(0)eˆ(0), i = 1,...,N 1, (4)
i i i i −
where e(0) = 0 and eˆ(0) = 0. Then, thefunctions qˆ(0) for i = 1,...,N and
0 0 i
eˆ(0) fori = 1,...,N 1areobtainedbythefollowingrecurrencerelation
i −
qˆ(0) = q λ,
1 1 −
qe
eˆ(0) = i i, i = 1,...,N 1,
i qˆ(0) −
i
qˆ(0) = q +e λ eˆ(0), i = 2,...,N.
i i i−1 − − i−1
Thus,thesecondexpressionofthedeterminantof A λI isgivenas
−
N
det(A λI) = qˆ(0). (5)
− i
i=1
Y
By (2)and (5), wehave
N N
(λ λ) = qˆ(0). (6)
i − i
i=1 i=1
Y Y
We differentiate this equation (6). The result of differentiation of the
left-hand sideof(6)is
N N N
1 1
(λ λ) = det(A λI). (7)
j
−λ λ − − λ λ −
i=1  i − j=1   i=1 i − 
BeforXe differentiatYion of the right-haXnd-side of(6), we introduce func-
tionsqˆ(p) ofλ defined by
i
dpqˆ(0)
qˆ(p) = i (8)
i dλp
fori = 1,...,N and p = 1,2,.... Theresult ofdifferentiationis
N Nj=1qˆ(j0) qˆ(1) = N qˆ(i1) det(A λI). (9)
qˆ(0) · i qˆ(0) −
i=1 Q i i=1 i 
FromX(7)and (9), wederiveX 
N 1 N qˆ(1)
= i . (10)
i=1 λi −λ i=1 −qˆ(i0)
X X 
5
Then, by substituting λ = 0 into the left-hand-side of (10), we have
the summation N λ 1 which is equal to the trace of (B B) 1. For i =
i=1 −i ⊤ −
1,...,N and p = 2,3,..., itholdsthat
P
N N
d 1 1
= (p 1) . (11)
dλ (λ λ)p 1 − (λ λ)p
i=1 i − − i=1 i −
X X
Thisrelationshipimpliesthatwehavethesummation N (λ λ) p for
p = 2,3,... by differentiating (10) repeatedly. From tih=i1s siu−mm−ation,
weobtainthesummation N λ p whichisequaltothePtraceof(B B) p
i=1 −i ⊤ −
by substitution of λ = 0. To make handling of the right-hand-side of
(10) easier, let us introduPce functions Hˆ(p) of λ for i = 1,...,N and
i
p = 1,2,... defined by
qˆ(1)
i , p = 1,
Hˆ(p) =  −qˆ(i0) (12)
i  dHˆdi(λp−1), p = 2,3,....
Wecan readily verifythat itholdsthat
N N
1
(p 1)! = Hˆ(p) (13)
− (λ λ)p i
i=1 i − i=1
X X
for p = 1,2,... by differentiating (10) repeatedly and taking care of
(10),(11)and(12). Thus,thetraceof(B B) pisobtainedbysubstituting
⊤ −
λ = 0into(13). Letusintroduceconstants H(p) fori = 1,...,N and p =
i
1,2,... definedby H(p) = Hˆ(p) . Thetraceof(B B) p isexpressedas
i i λ=0 ⊤ −
N(cid:12)
1 (cid:12)
Tr((B B) p) = (cid:12) H(p), p = 1,2,.... (14)
⊤ − (p 1)! i
− i=1
X
Thus,wecanobtainthesetracesiftheconstantsH(p)fori = 1,...,N and
i
p = 1,2,... are obtained by somemeans. The relationship(14) implies
that a formula for H(p) is required. We derive a recurrence relation for
i
Hˆ(p). A recurrence relation for H(p) is obtained by substitution of λ = 0
i i
intotherecurrencerelationforHˆ(p). OnthefunctionsHˆ(p),thefollowing
i i
lemmaholds.
Lemma 2.1
Let functionshˆ(p) ofλfori = 1,...,N and p = 1,2,... bedefined by
i
qˆ(p)
hˆ(p) = i . (15)
i −qˆ(0)
i
6
Fori = 1,...,N, it holds
Hˆ(1) = hˆ(1), (16)
i i
p 1
−
Hˆ(p) = hˆ(p) + C hˆ(k)Hˆ(p k), p = 2,3,.... (17)
i i p 1 k i i −
−
k=1
X
Proof.
Inthisproof, leti = 1,...,N.
Substituting p = 1 into(15)andcomparingwith(12), wehave(16).
Wegivethederivativeofhˆ(r) forr = 1,2,.... It holds
i
dhˆ(r) qˆ(r+1) qˆ(r)qˆ(1)
i = i + i i , r = 1,2,...
dλ − qˆ(0) (qˆ(0))2
i i
from (8)and(15). Then, itholdsthat
dhˆ(r)
i = hˆ(r+1) +hˆ(1)hˆ(r), r = 1,2,... (18)
dλ i i i
from (15). Using Hˆ(1) = hˆ(1) in(16), wehaveanotherform
i i
dhˆ(r)
i = hˆ(r+1) +Hˆ(1)hˆ(r), r = 1,2,.... (19)
dλ i i i
Weusemathematicalinductionfor proof.
We write the definition of Hˆ(p) for p = 2,3,... again. The definition
i
is
dHˆ(p 1)
−
Hˆ(p) = i . (20)
i dλ
We derive (17) for p = 2. Differentiating (16) and using (19) and
(20), weobtain
Hˆ(2) = hˆ(2) +hˆ(1)Hˆ(1). (21)
i i i i
Thus,(17)holdsfor p = 2.
Hereafter,letrbeanintegersuchthatr 2inthisproof. Assumethat
(17) holds for p = 2,...,r. We consider d≥ifferentiation of the function
Hˆ(r). Differentiating(17)for p = r and using(18)and (20), wederive
i
r 1
Hˆ(r+1) = hˆ(r+1)+hˆ(1)hˆ(r)+ − C hˆ(k+1) +hˆ(1)hˆ(k) Hˆ(r k) +hˆ(k)Hˆ(r+1 k) .
i i i i r 1 k i i i i − i i −
−
k=1
X (cid:16)(cid:16) (cid:17) (cid:17)
(22)
7
Sinceit holds
r 1
hˆ(1)hˆ(r) + − C hˆ(1)hˆ(k)Hˆ(r k) = hˆ(1)Hˆ(r)
i i r 1 k i i i − i i
−
k=1
X
from theassumption,(22)isrewrittenas
r 1 r 1
Hˆ(r+1) = hˆ(r+1)+hˆ(1)Hˆ(r)+ − C hˆ(k+1)Hˆ(r k)+ − C hˆ(k)Hˆ(r+1 k). (23)
i i i i r 1 k i i − r 1 k i i −
− −
k=1 k=1
X X
Werearrangethethirdtermintheright-hand-sideof(23). Sinceitholds
that
r 1 r
− r−1Ckhˆi(k+1)Hˆi(r−k) = r−1Ck′−1hˆi(k′)Hˆi(r+1−k′),
k=1 k=2
X X′
wehave
r 1 r 1
− C hˆ(k+1)Hˆ(r k) = hˆ(r)Hˆ(1) + − C hˆ(k)Hˆ(r+1 k). (24)
r 1 k i i − i i r 1 k 1 i i −
− − −
k=1 k=2
X X
Thefourthterm intheright-hand-sideof(23)is rewrittenas
r 1 r 1
− C hˆ(k)Hˆ(r+1 k) = (r 1)hˆ(1)Hˆ(r) + − C hˆ(k)Hˆ(r+1 k). (25)
r−1 k i i − − i i r−1 k i i −
k=1 k=2
X X
From (23), (24)and (25), wederive
r 1
Hˆ(r+1) = hˆ(r+1)+hˆ(r)Hˆ(1)+ − ( C + C )hˆ(k)Hˆ(r+1 k)+rhˆ(1)Hˆ(r). (26)
i i i i r 1 k 1 r 1 k i i − i i
− − −
k=2
X
Itcan readilybeverified thatthesummationofthecombinationsin(26)
is
C + C = C (27)
r 1 k 1 r 1 k r k
− − −
inthecaseofr 3. Inthecaseofr = 2,thesummationofthethirdterm
≥
intheright-hand-sideof(26)is zero. Then, wefinallyobtain
r
Hˆ(r+1) = hˆ(r+1) + C hˆ(k)Hˆ(r+1 k).
i i r k i i −
k=1
X
Thus,(17)holdsfor p = r+1. (cid:3)
ByLemma2.1,weobtainarecurrencerelationforHˆ(p). However,we
i
have not obtained a recurrence relation for the functions hˆ(p). We show
i
the followinglemmawhich gives a method to computehˆ(p) in a form of
i
arecurrence relation.
8
Lemma 2.2
Thefunctionshˆ(p) fori = 1,...,N and p = 1,2,... satisfythefollowing
i
recurrence relation. For p = 1, therecurrence relationis
1
hˆ(1) = , (28)
1 qˆ(0)
1
eˆ(0)hˆ(1) +1
hˆ(1) = i 1 i 1 , i = 2,...,N. (29)
− −
i qˆ(0)
i
Fori = 1 and p = 2,3,..., therecurrence relationis
hˆ(p) = 0. (30)
1
Fori = 2,...,N and p = 2,3,..., therecurrence relationis
eˆ(0) p−2
hˆ(p) = i 1 hˆ(p) + phˆ(1)hˆ(p 1) + C hˆ(k)hˆ(p k). (31)
i qˆ(i−0) i−1 i−1 i−−1 k=1 p k i−1 i −
(cid:16) (cid:17) X
Proof.
In this proof, another key point of the derivation of the recurrence
relation is applied. The key point is as follows. We differentiate the
relationships q + e λ = qˆ(0) + eˆ(0) for i = 1,...,N shown in (3)
and qe = qˆ(0)eˆi(0) foi−r1i−= 1,...,iN 1i−s1hown in (4). Then, we derive a
i i i i −
recurrence relation whichthefunctionshˆ(p) satisfy.
i
We introduce functions eˆ(p) of λ for i = 0,1,...,N 1 and p =
i −
1,2,... defined by
dpeˆ(0)
eˆ(p) = i .
i dλp
Differentiating(4)repeatedly, wehave
p
C qˆ(k)eˆ(p k) = 0, i = 1,...N 1, p = 1,2,....
p k i i − −
k=0
X
Solvingthisequationforeˆ(p), weobtain
i
p
eˆ(p) = C hˆ(k)eˆ(p k), i = 1,...N 1, p = 1,2,... (32)
i p k i i − −
k=1
X
sincehˆ(k) = qˆ(k)/qˆ(0) fromthedefinition.
Wesihow−thai t (2i8)and(29)hold. Differentiating(3), wehave
qˆ(1) +eˆ(1) = 1, i = 1,...,N. (33)
i i 1 −
−
9
Then, hˆ(1) = qˆ(1)/qˆ(0) isexpressedas
i − i i
eˆ(1) +1
hˆ(1) = i 1 , i = 1,...,N. (34)
−
i qˆ(0)
i
Substituting p = 1into(32), wehave
eˆ(1) = hˆ(1)eˆ(0), i = 1,...,N 1. (35)
i i i −
From (34), (35)and eˆ(0) = 0,weimmediatelyobtain(28)and (29).
0
Hereafter, let p = 2,3,... in this proof. We show that the relation-
ships(30)and(31)hold. Differentiating(33) repeatedly,weobtain
qˆ(p) +eˆ(p) = 0, i = 1,...,N. (36)
i i 1
−
From this relationship and the definition hˆ(p) = qˆ(p)/qˆ(0), the functions
i − i i
hˆ(p) for p = 2,3,... are
i
eˆ(p)
hˆ(p) = i 1, i = 1,...,N. (37)
−
i qˆ(0)
i
We show that (30) holds. It holds that eˆ(p) = 0 since eˆ(0) = 0. Then,
0 0
substitutingi = 1 into(37), wehave(30).
We show that (31) holds. Hereafter, let i = 2,...N in this proof.
Substituting(32) into(37), weobtain
p
1
hˆ(p) = C hˆ(k)eˆ(p k). (38)
i qˆ(i0) k=1 p k i−1 i−−1
X
It followsfrom (35) that
C hˆ(p 1)eˆ(1) = peˆ(0)hˆ(p 1)hˆ(1). (39)
p p 1 i −1 i 1 i 1 i −1 i 1
− − − − − −
From (38)and(39), it holds
p 2
1 −
hˆ(p) = hˆ(p)eˆ(0) + peˆ(0)hˆ(p 1)hˆ(1) + C hˆ(k)eˆ(p k) . (40)
i qˆ(i0)  i−1 i−1 i−1 i−−1 i−1 k=1 p k i−1 i−−1 
Then, weobtain(31)from (37)and (40). (cid:3)X 
Remark 2.3
From Lemmas 2.1 and 2.2, the functions Hˆ(1) satisfy the following re-
i
currence relation
1
Hˆ(1) = ,
1 qˆ(0)
1
eˆ(0)Hˆ(1) +1
Hˆ(1) = i 1 i 1 , i = 2,...,N.
− −
i qˆ(0)
i
10