Table Of Content1
On The Positive Definiteness of Polarity
Coincidence Correlation Coefficient Matrix
F. Haddadi, Student Member, IEEE, M. M. Nayebi, Senior Member, IEEE, and M. R. Aref
Abstract—Polarity coincidence correlator (PCC), when where
used to estimate the covariance matrix on an element-by- +1 : x≥ 0
sgn(x) = (2)
element basis, may not yielda positive semi-definite (PSD) (cid:26) −1 : x < 0
9
estimate. Devlin et al. [1], claimed that element-wise PCC
0 Using (1), an estimate of r from N iid observations
0 is not guaranteed to be PSD in dimensions p>3 for real
x ,y ,i = 1,...,N is given by
2 signals.However,nojustificationorproofwasavailableon i i
thisissue.Inthisletter,itisprovedthatforrealsignalswith N
n π 1
a p≤3 and for complex signals with p≤2, a PSD estimate rˆ= sin sxisyi . (3)
2N
J is guaranteed. Counterexamples are presented for higher (cid:16) Xi=1 (cid:17)
9 dimensions which yield invalid covariance estimates.
wheres = sgn(x ). In the complexcase,wecandefine
2 xi i
Index Terms—Polarity coincidence correlator, element- the complex sign function as sgn (x) , sgn(ℜ[x]) +
c
] wise covariance estimate, positive semi-definite. jsgn(ℑ[x]), whereℜ[x] andℑ[x] are realand imaginary
T
parts ofx, respectively.In the Appendix,it is shownthat
I
.
cs I. INTRODUCTION AND PRELIMINARIES ℜ[r]= sin πE ℜ[sgn (x)sgn∗(y)]
[ (cid:16)4 c c (cid:17)
1 POanLdAnRonITpYaracmoientcriicdeensctiemcaotorrreolaftobriv(aPrCiaCte)cisorarerloabtiuosnt ℑ[r]= sin π4E(cid:8)ℑ[sgnc(x)sgn∗c(y)](cid:9) (4)
v (cid:16) (cid:8) (cid:9)(cid:17)
8 [1], [2]. It is also a fast and low-cost estimator for where (·)∗ denotescomplex conjugate.Similar to (3), an
4 applications with extraordinary computational complex- estimate for the complex case is obtained by replacing
6
ity. Radio astronomy is an instance in which PCC is by expectation with the average as
4
far the most favorable correlator [3].
.
1 N
Several researchers have investigated the statistical π 1
0 rˆ = sin [s s +s s ]
R xiR yiR xiI yiI
9 error of PCC as an estimate of bivariate correlation (cid:16)4N Xi=1 (cid:17)
0 [4], [5]. In multivariate case, using PCC to estimate
N
: π 1
v elements of the covariance matrix does not guarantee rˆ = sin [s s −s s ] (5)
I xiI yiR xiR yiI
Xi a PSD matrix estimator [1], [6]. Devlin et al. [1, Sec. (cid:16)4N Xi=1 (cid:17)
4.4], referring to a personal communication, claim that
r where (·) and (·) denote real and imaginary parts,
a R I
element-wisePCC(or”quadrantcorrelation”),mayyield
respectively.
aninvalidcovarianceestimateforp > 3andrealsignals.
In thisletter, weprovethatforrealsignalswith p ≤ 3,
II. MAIN RESULT
and complex signals with p ≤ 2, PCC estimate is PSD.
For higher dimensions, counterexamples are presented Let Rp×p be the covariance matrix of p random
signals with unit diagonal elements and off-diagonal
which yield invalid covariance estimates.
elements r : i,j = 1,··· ,p. For p = 2 case, a valid
Let x and y be two zero-mean real random variables ij
correlation estimate should satisfy |rˆ| ≤ 1. For the real
with correlation coefficient r distributed with elliptical
case of (3), |rˆ| = |sin(·)| ≤ 1. For the complex case,
symmetry. It is well known that [6]:
regarding (5) define α and β such that rˆ = sin(α) and
R
π rˆ = sin(β). Then
I
r = sin E{sgn(x)sgn(y)} (1)
2
(cid:16) (cid:17) N
π
α+β = [s (s −s )+s (s +s )]
xiR yiR yiI xiI yiR yiI
This work was supported by Advanced Communication Re- 4N
Xi=1
search Institute (ACRI), Sharif University of Technology, Tehran,
(6)
Iran. Authors are with the Department of Electrical Engineering,
and it can be easily checked that the argument of sum-
Sharif University of Technology, Tehran, Iran (e-mails: farzanhad-
[email protected], [email protected], and [email protected]). mationin (6)belongsto{±2}. Thisyieldsα+β ≤ π2.In
2
TABLE I
x Na1 COUNTEREXAMPLESFORREALANDCOMPLEXDATA
y Na2 Real Case Complex Case
Na3 sx + + + + sx ++ ++
z s + + − −
y s ++ −+
s + + + − y
z
Fig. 1. Polarity coincidence diagram of x,y,z. Black strips sw + + − + sz ++ −−
denote the packed positions of polarity coincidences of each signal
with signal x. The strips lengths Na are the number of polarity
i
coincidences.
strip of z is in the left corner, and minimum when it is
in the right corner. After some calculations, the range of
the samemanner, we canshowthat± α±β ≤ π2 which rs23 is found as
gives |α|+|β| ≤ π. Now it is straightforward to see that
|rˆ|2 = sin2(α)+si2n2(β) ≤ sin2(|α|)+sin2(π−|α|) = 1. |rs12+rs13|−1 ≤ rs23 ≤ 1−|rs12−rs13|. (9)
2
Forp = 3andrealsignals,wecalculatethevalidrange It should be noted that the effect of finite N is the
of the elements of a 3×3 covariance matrix. Then we quantization of the accessible values. Now, it can be
show that PCC estimate lies in this range. readily verified that
π
sin (1−|rs12−rs13|) =
A. Valid Range of Covariance 2
(cid:16) (cid:17)
Let R∈ R3×3 be a covariance matrix with unit diag- r12r13+ 1−r122 1−r123 (10)
onal elements. Valid range of r23 should be calculated q(cid:0) (cid:1)(cid:0) (cid:1)
and
when r12,r13 ∈ [−1,+1] are fixed. It can be readily
π
shown that |R| ≥0 implies that sin (|rs12 +rs13|−1) =
2
(cid:16) (cid:17)
2 2
|r23 −r12r13|≤ q 1−r12 1−r13 . (7) r12r13− 1−r122 1−r123 . (11)
(cid:0) (cid:1)(cid:0) (cid:1) q
(cid:0) (cid:1)(cid:0) (cid:1)
B. PCC Covariance Estimate Therefore, rˆ23 = sin π2 rs23 satisfies (7). This, besides
|rˆ12|< 1and|rˆ13| <(cid:0)1canb(cid:1)eusedtoshowthat|Rˆ| ≥0
Assume random sign sequences s ,s ,s with length
x y z (as in (7)) and the assertion is proved that for p = 3 and
N. Consider the positions of polarity coincidence with
real data, PCC estimate is a valid covariance matrix.
s as black positions or ”+” and elsewhere as white
x
or ”−”. Obviously all of the positions in s is ”+”
x
III. COUNTEREXAMPLES
and (s ,s ) have four states of {++,+−,−+,−−}.
yi zi
Since the permutation of the samples does not affect the Inthissection,somecounterexamplesarepresentedto
estimate in (3), put the samples of s ,s ,s from left show that PCC covariance estimate is not guaranteed to
x y z
in the order of {+ + −,+ + +,+ − +,+ − −} as in be PSD in dimensions p > 3 for real signals and p > 2
Fig. 1. Thenanyrandomsignsequencesofs , s ands for complex signals. In real data case with p = 4 and
x y z
can be replaced by the model in Fig. 1 with appropriate number of observations N = 4, the real sign sequences
strip lengths Nai (with a1 = 1) and relative positions of in Table I results in an invalid covarianceestimate. After
strips. simple computations, we will have rs12 = rs34 = 0
Let Rs be the covariance matrix of sx,sy,sz with and rs13 = rs14 = rs23 = rs24 = 0.5. The covariance
elements rsik,i,k = 1,2,3. The maximumof rs12 = +1 estimate will be
occurs in a2 = 1 and the minimum of rs12 = −1 in 1 0 0.7 0.7
a(N2 =−N0.a2In)]f=ac2t,a2rs−121,=inN1otPherNi=w1osrxdissyi = N1[Na2 − Rˆ1 = 00.7 01.7 01.7 00.7
0.7 0.7 0 1
1+rs1i
a = . (8)
i 2 with eigenvalues [−0.4,1,1,2.4]. Then Rˆ1, with a neg-
rs12 and rs13 are determined by the values of a2 and ative eigenvalue, is not a valid covariance matrix.
a3, rsii = 1, and the possible range of rs23 should We can augment this example to give a counter-
be calculated. rs23 depends on the number of polarity example for dimension p = 5. Repeat each sign twice
coincidences of y and z which is maximum when the to have four signals with number of observations 2N.
3
Note that the covariance matrix does not change. Now, A. MIMO Communication Systems
add a new signal with alternating sign in each sample.
Multiple antenna systems are an integral part of the
The covariance estimate will be
most new wireless communication systems increasing
user and data capacity (e.g. UMTS/W-CDMA, 802.11n
Rˆ = Rˆ1 0 . WLAN, 60 GHz WPAN). Multiple antennascanprovide
aug (cid:20) 0T 1 (cid:21) diversity gain and/or better antenna gain through beam-
forming in base station and/or handset. Beamformers
where 0 is the 4×1 vector of zeros. As a consequence (e.g. conventional or Capon) usually utilize an estimate
of the structure of Rˆaug, eigenvalues of Rˆ1 are also of the array covariance matrix [8], that may be obtained
eigenvalues of Rˆ . Therefore, Rˆ is an invalid co- using PCC as a power-savingestimator. It is well known
aug aug
variance matrix. This procedure can continue to produce that performance improvement due to diversity gain re-
counterexamplesfor higherdimensionsin realdata case. ducesas the number of antennasincreases.This, besides
In case of complex signals, p = 3 and N = 2, a spacelimitonthehandsetandcouplingphenomenahave
counterexampleis givenin TableI,where ”−+”denotes resulted in the prevalence of MIMO systems with very
−1+j. The resulting estimate is few (usually 2 to 4) antennas [9], [10].
1 0.7−j0.7 0 B. Blind Source Separation (BSS)
Rˆ = 0.7+j0.7 1 0.7−j0.7 BSS has found numerous potential applications in
0 0.7+j0.7 1 the field of audio signal processing [11]. An array of
microphones is used to gather multiple signal mixtures
with eigenvalues[−0.4,1,2.4] which make Rˆ an invalid and diverse methods are used to extract signals from
covariance matrix. Augmentation of the complex signal these observations. A large class of BSS methods use
set for higher dimensions is similar to the real case, real-valued inter-sensor covariances with different time
except that the new added signal alternates between lags to estimate the mixing matrix and desired signals
”++” and ”−−”. (e.g. SOBI [12], JADE [13]). This also includes in-
put signals whitening as a preprocessing that converts
the convolutive source separation problem to a simpler
IV. APPLICATIONS OF THE RESULTS independent component analysis (ICA) problem. This
family of two-step algorithms is known as AMUSE
In this section, we discuss the practical usefulness of
(Algorithm for MUltiple Source Extraction). PCC, as
the main results of this letter which focuses on low
a fast correlator, can make real-time operation more
number of sensors. In the signal processing context,
feasible in these methods. For realistic situations where
covariance estimation often arises in the multi-sensor
we have fewer sensors than sources, underdetermined
applicationswhereparametersofinterestarefunctionsof
methodsare proposed[14]. Manymethodsare presented
the true data covariance matrix. Although PCC estimate
for the special case of 2 sensors and multiple sources
of the covariance matrix exhibits attractive features such
(e.g. DUET [15], and [16]), and also quite few sensors
as robustness and extremely low complexity, it cannot
are common to many realizations of the methods [12],
be guaranteed to be PSD in the applications with large
[14].
number of sensors.
Selection of the number of sensors in an application APPENDIX
depends on both nature of the problem and practical COMPLEX PCC
limitations. In theory, more sensors always results in
Let x,y be two zero-mean, unit-variance, and circu-
a better estimate, as proved in many cases such as
larlysymmetriccomplexrandomvariableswithindepen-
directionfindingthroughexaminationoftheCramer-Rao
dent real and imaginary parts. To prove (4), we expand
bounds [7]. In practice, complexity issues usually limit
the expectation as
the number of sensors. Large arrays are used whenever
∗
E{sgn (x)sgn (y)} = E [sgn(x y )+sgn(x y )]
performance be of the main importance regardless of c c R R I I
the cost. In such cases as DOA estimation in military +j[s(cid:8)gn(x y )−sgn(x y )] .(12)
I R R I
environments (radar and sonar), thousands of sensors ∗ (cid:9)
Furthermore, E{xy } = r implies that
are not uncommon. Nevertheless, most low-cost civil
applications use very few sensors. In the following, we E{xRyR+xIyI}= rR
consider some of these applications. E{x y −x y } = r . (13)
I R R I I
4
Circular symmetry of x and y yields [15] O.YilmazandS.Rickard,”Blindseparationofspeechmixtures
viatime-frequencymasking”,IEEETransSignalProc.,vol.52,
E{xRyR} = E{xIyI} = rR/2 No. 7, pp. 1830-1847, July 2004.
[16] P.Comon, ”Blind identificationand source separation in 2×3
E{x y }= −E{x y } = r /2 (14)
I R R I I under-determinedmixtures”,IEEETrans.SignalProc.,vol.52,
and E{x2} = E{x2} = E{y2} = E{y2} = 1 . Then No. 1,pp. 11-12, Jan. 2004.
R I R I 2
the correlation coefficients will be
Cor(x ,y )= Cor(x ,y ) = r
R R I I R
Cor(x ,y ) = −Cor(x ,y ) = r . (15)
I R R I I
Substituting (15) and (1) into (12) gives
∗ −1 −1
E{sgn (x)sgn (y)} = 4/π[sin (r )+jsin (r )](16)
c c R I
which implies (4).
REFERENCES
[1] S. J. Devlin, R. Gnanadesikan, and J. R. Kettenring, ”Robust
estimation and outlier detection with correlation coefficients”,
Biometrika, vol. 62, No. 3, pp. 531-545, March 1975.
[2] S. S. Wolff, J. B. Thomas, and T. R. Williams, ”The polarity-
coincidence correlator: anonparametric detection device”, IRE
Trans. Information Theory, vol. 8, No. 1, pp. 5-9, Jan. 1962.
[3] P.C.Egau,”Correlationsystemsinradioastronomyandrelated
fields”,IEEProc.,vol.131, Pt.F,No.1,pp. 32-39,Feb.1984.
[4] K. J. Gabriel, ”Comparison of three correlation coefficient
estimators for gaussian stationary processes”, IEEE Trans.
Acoustic,Speech,SignalProc.,vol.ASSP-31,No.4,pp.1023-
1025, Aug. 1983.
[5] G. Jacovitti and R. Cusani, ”Performance of normalized co-
rrelationestimatorsforcomplexprocesses”,IEEETrans.Signal
Proc., vol. 40, No. 1, pp. 114-128, Jan. 1992.
[6] S.Visuri,V. Koivunen, and H. Oja, ”Signand rank covariance
matrices”,J.Stat.Plan.Infer.,vol.91,No.2,pp.557-575,Feb.
2000.
[7] P. Stoica and A. Nehorai, ”MUSIC, maximum likelihood and
Cramer-Rao bound”, IEEE Trans. Acoustic Speech Sig. Proc.,
vol 37, No. 5, pp. 720-741, May 1989.
[8] B.D.VanVeenandK.M.Buckley,”Beamforming: aversatile
approach tospatial filtering”,IEEEASSP Mag., pp. 4-24, Apr.
1988.
[9] H. T. Hui, ”Practical dual-helical antenna array for diversity-
MIMO receiving antennas on mobile handsets”, IEE Proc.
Microw. Antennas Propag., vol. 152, No. 5, pp. 367-372, Oct.
2005.
[10] D. Browne, M. Manteghi, M. P. Fitz, and Y. Rahmat-samii,
”Experiments with compact antenna arrays for MIMO radio
communications”, IEEE Trans. Ant. Propag., vol. 54, No. 11,
pp. 3239-3259, Nov. 2006.
[11] K.Torkkola, ”Blindseparation foraudiosignals- arewethere
yet?”, in Proc. Workshop on Independent Component Analysis
andBlindSignalSeparation,Aussois,France,Jan.11-15,1999.
[12] A. Belouchrani, K. Abed-Meraim, J.-F. Cardoso, and E.
Moulines, ”A blind source separation technique using second-
order statistics”, IEEE Trans. Signal Proc., vol. 45, No. 2, pp.
434-444, Feb. 1997.
[13] J.F.Cardoso,”High-order contrastforindependent component
analysis”, Neural Computation, vol. 11, No. 1, pp. 157-192,
1999.
[14] S. Winter,W. Kellermann, H. Sawada, and S. Makino, ”MAP-
based underdetermined blind source separation of convolutive
mixturesbyhierarchicalclusteringandl1-normminimization”,
EURASIP J. Advances Signal Proc.,pp. 1-12, 2007.
