Measurement-driven Quality Assessment of Nonlinear Systems by Exponential Replacement

0.11 MB·

by  Manuel Stein

1
Measurement-driven Quality Assessment of
Nonlinear Systems by Exponential Replacement
Manuel Stein, Josef A. Nossek and Kurt Barbe´∗
Institute for Circuit Theory and Signal Processing (NWS), Technische Universita¨t Mu¨nchen, Germany
∗Research Team Stochastics (STOX), Dept. Mathematics (DWIS), Vrije Universiteit Brussel, Belgium
Email: [email protected], [email protected], [email protected]
6
1 Abstract—Wediscusstheproblemhowtodeterminethequality Ifthe analyticalformof thesystem modelp(z;θ)is known
0 of a nonlinear system with respect to a measurement task. but to complicated to work with, one can resort to an direct
2 Due to amplification, filtering, quantization and internal noise
approximation of the distribution function like performed
sources physical measurement equipment in general exhibits a
n in [1] for the Rician model. When the nonlinear output is
nonlinear and random input-to-output behaviour. This usually
a makes it impossible to accurately describe the underlying sta- a parametric but deterministic transformation f(x;θ) of an
J
tistical system model. When the individual operations are all intermediate variable x ∈ R with known statistical model
9
known and deterministic, one can resort to approximations of p(x;θ), the input-to-output relation z = f(x;θ) can be
2
the input-to-output function. The problem becomes challenging
represented by a Taylor expansion [2]. If the input to the

when the processing chain is not exactly known or contains
] nonlinearsystem is Gaussian, the modelcan be approximated
nonlinear random effects. Then one has to approximate the
T
output distribution in an empirical way. Here we show that through a Bussgang decomposition [3], where the idea is
I
. by measuring the first two sample moments of an arbitrary to use the best linear approximation of the nonlinearity and
s set of output transformations in a calibrated setup, the output to characterize the second moment of the resulting error. In
c
distribution of the actual system can be approximated by an
[ the situation where the output model is completely unknown
equivalent exponential family distribution. This method has the
one has to use empirical methods like histograms [4]. In the
1 propertythattheresultingapproximationofthestatisticalsystem
v model is guaranteed to be pessimistic in an estimation theoretic case that the system parameter only modulates the output
7 sense. We show this by proving that an equivalent exponential mean and leaves the variance constant, it is possible to use
1 familydistributioningeneralexhibitsalowerFisherinformation a pessimistic replacement strategy by assuming an equivalent
1 measure than the original system model. Withvarious examples Gaussianmodel.Thisbecomespossibleasforsuchasituation
8 and a model matching step we demonstrate how this estimation
the Gaussian model is the least favorable distribution [5], [6].
0 theoreticaspect can beexploited inpracticeinordertoobtaina
. conservative measurement-driven qualityassessment method for The method of exponential replacement and subsequent
1 nonlinear measurement systems. model matching that is presented on the following pages can
0
beinterpretedasageneralizationoftheGaussianreplacement
6 IndexTerms—nonlinearsystems,Fisherinformation,Crame´r-
1 Raolowerbound,exponentialfamily,Salehmodel,Ricianmodel, strategy.This replacementcan be extendedto any distribution
: regression, Wiener system, measurement uncertainty of the exponentialtype,where the gatheredmeasurementsare
v
used to estimate the sufficient statistics and model matching
i
X I. MOTIVATION is used to determine the natural parameters of the selected
r The characterization of nonlinear systems and the develop- exponential distribution. By using the Cauchy-Schwarz in-
a
mentofappropriateprocessingalgorithmsformsaproblemfor equality we show that such an replacement p˜(z;θ) in general
engineering tasks like signal processing or system identifica- exhibits a lower Fisher information measure than the original
tionwhiletheincreasingdemandonlow-cost,energy-efficient system model p(z;θ). We discuss how to match the expo-
and fast measurement devices makes it inevitable to operate nential replacement model p˜(z;θ) in an estimation theoretic
such systems outside their linear regime. In order to provide sense by maximizing its Fisher information. The result of
high processing accuracy under these circumstances, it is im- the presented procedure is an approximation of the statistical
portanttoinvestigatenonlinearmodelsandtodevelopgeneric systemmodelwhichcanbeobtainedinameasurement-driven
approaches and methods for such kind of problems. The key way and is guaranteed to be conservative with respect to
challenge for efficient solutions to nonlinear measurement the possible accuracy that can be obtained by gathering data
problems lies in the fact that access to an appropriate model from p(z;θ) to measure the parameter θ without bias. In
p(z;θ) is required. In theory, this probabilistic description order to demonstrate the useful character of the exponential
establishes the statistical relationship of the system parameter replacement,weshowhowtoevaluatetheestimationtheoretic
θ ∈R and the outputmeasurementz ∈R. This allows to for- qualityofthreenonlinearsystemsbycalibratedmeasurements
mulate efficient signal processing algorithms and to describe at the system output. We use the Saleh model [7] to simulate
the achievable performance by analytical tools. However, in an amplification device with saturation effects and unknown
practicethe system modelp(z;θ) is rarelyknownexactlyand input power level. As a second example we use the Rician
therefore has to be established by numerical calculations or model with unknown distance parameter, which is of interest
approximated by physical measurements. in wireless communications [8] and biomedical applications
2
likefMRI[9].Finally,wediscussapolynomialinput-to-output is obtained. Within this factorization w (θ) is the l-th natural
l
relation of cubic order with unknown regression parameter parameter, t (z) is the associated sufficient statistic, λ(θ) is
l
which exhibits the properties of a Wiener type of nonlinear the log-normalizer and κ(z) is a carrier measure. The log-
system [10]. likelihood function of an exponential family is given by
L
II. PROBLEM FORMULATION
lnp(z;θ)= w (θ)t (z)−λ(θ)+κ(z), (5)
l l
Consider the problem of measuring the system parameter
l=1
X
θ by taking measurements at the output z of a system. The
while the score function attains the structure
system is represented by a parametrized probability density
or mass function p(z;θ), with random variable z ∈ Z and a ∂lnp(z;θ) L ∂w(θ) ∂λ(θ)
deterministic system parameter θ ∈ Θ. Z ⊂ R is the support ∂θ = ∂lθ tl(z)− ∂θ . (6)
of the random output z and Θ ⊂ R the parameter space of Xl=1
θ. Throughout our discussion we assume that all integrands
are absolutely integrable on the support Z. All density or B. Exponential Replacement
massfunctionsp(z;θ)exhibitregularityandaredifferentiable We substitute the original system p(z;θ) by p˜(z;θ) and
with respect to the parameter θ. For such systems the Fisher assume that the score function of the replacement model
information measure is defined by [11] p˜(z;θ) exhibits an exponential factorization
∂lnp(z;θ) 2 L
F(θ)= p(z;θ)dz. (1) ∂lnp˜(z;θ)
ZZ(cid:18) ∂θ (cid:19) ∂θ = βl(θ)φl(z)−α(θ), (7)
For any unbiased estimator Xl=1
where β (θ) and α(θ) are weighting factors which depend
Ez;θ θˆ(z)−θ =0 (2) on θ. Thle functionsφl(z) are arbitrary transformationsof the
h i output data z.
using N independent samples, here denoted by z ∈ RN, the
accuracy is limited by the classical Crame´r-Rao lower bound
(CRLB) [12], [13] C. Fisher Information Bound
E θˆ(z)−θ 2 ≥ 1 . (3) It can be shown that the approximation (7) provides a
z;θ NF(θ) pessimisticsystemmodelinanestimationtheoreticsense.This
h(cid:0) (cid:1) i result is obtained by invoking the Cauchy-Schwarz inequality
Note that if N is sufficiently large and θˆ(z) is an efficient
with the exponential score function (7).
estimator, the statement (3) holds with equality. Therefore,
Theorem 1 (Fisher Information Bound): For a probability
the Fisher information measure (1) provides a mathematical
densityormassfunctionp(z;θ),asetofLoutputtransforma-
tool in order to evaluate the asymptotic estimation theoretic
tionsφ(z)andarbitraryweightingfunctionsβ (θ), theFisher
l l
quality of the model p(z;θ). The higher the actual value of
information measure is lower bounded by
the information measure F(θ), the higher the accuracy that
can be expected while estimating θˆ(z). However, calculation F(θ)≥
of (1) requires an analytic description of the log-likelihood
L 2
function lnp(z;θ) and its derivative ∂ln∂pθ(z;θ), referred to as βl(θ)∂Ez;θ∂[θφl(z)]
the score function. Further, the computation of the integral in (cid:16)l=1 (cid:17) . (8)
L P 2 L 2
(1) can be demanding. This motivates the problem of finding E β (θ)φ (z) − β (θ)E [φ (z)]
z;θ l l l z;θ l
approximations for F(θ) which has found our attention for (cid:20)(cid:16)l=1 (cid:17) (cid:21) (cid:16)l=1 (cid:17)
the design of quantized signal processing systems [14] and ProofP: see Appendix A. P
allowstodeduceapessimisticbuthighlytractablereplacement
p˜(z;θ) for the original model p(z;θ) [6].
D. Model Matching
III. EXPONENTIALREPLACEMENT It is possible to optimize the weighting factors β (θ) such
l
Our approach follows the idea of replacing the original that the right hand side of the information bound (8) is
system p(z;θ) by an equivalent distribution p˜(z;θ) which maximized.Inordertoobtainacompactproblemformulation,
belongs to the exponential family. we define the transformation vector
T
φ(z)= φ (z) φ (z) ... φ (z) . (9)
1 2 L
A. Exponential Family
An exponentialfamily with a single parameter θ is a set of The corresponding(cid:2)weighting functions are sum(cid:3)marized by
probabilitydensityormassfunctions,whichcanbefactorized T
β(θ)= β (θ) β (θ) ... β (θ) . (10)
such that the form 1 2 L
L Further, we define(cid:2)the mean of the transformati(cid:3)on vector
p(z;θ)=exp w(θ)t (z)−λ(θ)+κ(z) (4)
l l
l=1 ! µφ(θ)=Ez;θ[φ(z)] (11)
X
3
and its covariance IV. QUALITY OF NONLINEARSYSTEMS
R (θ)=E φ(z)φT(z) −µ (θ)µT(θ). (12) In order to demonstrate the application of the information
φ z;θ φ φ
bound (22) for systems with unknown or difficult analytic
h i
This allows to reformulate the individual parts of (8) by model description p(z;θ), we use three nonlinear examples.
For each of them we use L=7 and
L ∂E [φ (z)] ∂µ (θ)
βl(θ) z;θ∂θ l =βT(θ) ∂φθ (13) φ(z)= z z2 z3 z4 |z| ln|z| ln2|z| T (23)
l=1
X
with the aim(cid:2)to obtain a measurement-driven app(cid:3)roximation
and
of the Fisher information based on the considered output
L L
2 2 transformations.Our choice(23) is motivatedby the common
E β (θ)φ (z) − β (θ)E [φ (z)] =
z;θ l l l z;θ l
" #
(cid:16)Xl=1 (cid:17) (cid:16)Xl=1 (cid:17) TABLEI
=βT(θ)R (θ)β(θ). (14) EXPONENTIALFAMILYDISTRIBUTIONS
φ
With (13) and (14) it is possible to state an alternative form Distribution t1(z) t2(z)
of the Fisher information bound (8) Bernoulli z -
βT(θ)∂µφ(θ) ∂µφ(θ) Tβ(θ) Binomial z -
∂θ ∂θ Chi-squared lnz -
F(θ)≥ . (15)
βT(θ)R(cid:16) (θ)β(θ(cid:17)) Dirichlet lnz -
φ
Exponential z -
With the substitution Gaussian z z2
β(θ)=R−12(θ)β′(θ), (16) Gamma lnz z
φ Laplace (zero-mean) |z| -
the tightest bound (15) is found by solving the quadratic Log-Normal lnz ln2z
maximization problem Poisson z -
max β′T(θ) R−12(θ)∂µφ(θ) ∂µφ(θ) TR−12(θ) β′(θ) distributions within the exponential family (see Tab. I) and
β′(θ)∈RL (cid:18) φ ∂θ (cid:18) ∂θ (cid:19) φ (cid:19) their sufficient statistics. We expect that intractable practical
s.t. β′T(θ)β′(θ)=1 (17) distributions may be some mixture of these common proba-
bility laws. In such a case one can use a linear combination
with respect to β′(θ). An optimization problem with the
of the sufficient statistics in order to reach a proper bound.
structure of (17) is solved by
With N = 109 realizations for each calibrated setup of θ,
β′⋆(θ)=d (θ), (18) the required mean and covariance of the transformations are
1
approximated by their sample mean
where d (θ) is the eigenvector of the matrix
1
N
1
D(θ)=R−12(θ)∂µφ(θ) ∂µφ(θ) TR−12(θ), (19) µφ(θ)≈ N φ(zn) (24)
φ ∂θ ∂θ φ n=1
(cid:18) (cid:19) X
and
corresponding to the principal eigenvalue ζ (θ) of D(θ). As
1
here the matrix D(θ) hasrank one,it is possible to show that 1 N
R (θ)≈ φ(z )φT(z )
−1 ∂µ (θ) φ N n n
d1(θ)= ∂µRφ(θφ)2(TθR)−1∂φ(θθ)∂µφ(θ) (20) − (cid:16)1 nXN=1φ(z ) 1 N(cid:17) φ(z ) T. (25)
∂θ φ ∂θ N n N n
and q(cid:0) (cid:1) (cid:16) nX=1 (cid:17)(cid:16) nX=1 (cid:17)
After numerically approximatingthe derivative of µ (θ) it is
∂µ (θ) T ∂µ (θ) φ
ζ (θ)= φ R−1(θ) φ . (21) possibletocalculateapessimisticapproximationoftheFisher
1 ∂θ φ ∂θ information measure by the lower bound (22) and to state a
(cid:18) (cid:19)
conservative version of the CRLB (3).
Theorem 2 (Matched Fisher Information Bound): For a
probability density or mass function p(z;θ) and any set of L
A. Saleh model
deterministicoutputtransformationsφ(z),withthedefinitions
The first example is the Saleh model
(11) and (12), the Fisher information measure F(θ) is lower
ax
bounded by
z = , (26)
1+bx2
∂µ (θ) T ∂µ (θ)
F(θ)≥ φ R−1(θ) φ . (22) where a,b ∈ R are model parameters. This nonlinear model
∂θ φ ∂θ
(cid:18) (cid:19) was originally developed in order to characterize amplitude
Proof:Followsfromthefactthatwiththesolutionof(17) saturationeffectsintraveling-wave-tubeamplifiers[7].Weuse
the right hand side of (15) obtains the value given in (21). the developed approximation (22) in order to investigate the
4
quality of (26) with respect to the problem of measuring the 0
system input power. To this end, we assume that the input x
of the nonlinear model follows a Gaussian distribution, i.e.,
−5
x∼N(0,θ) with θ >0. Fig. 1 shows the information loss
1 ∂µ (θ) T ∂µ (θ) θ)
χSM(θ)= Fx(θ)(cid:18) ∂φθ (cid:19) R−φ1(θ) ∂φθ (27) χ(RM −10
g
in dB with the setup a = 2.1587,b = 1.1517. Note that the lo −15
Fisher information measure with respect to θ at the system 10
input x is F (θ) = 1 . It can be observed that the Saleh
x 2θ2
−20
0
−25
0 0.5 1 1.5 2
−2
θ
θ() −4 Fig.2. InformationLoss-Rician model
M
S
χ
g C. Cubic polynominal model
o
l −6
10 Finally we consider a regression problem and use our
approximation(22)inordertodeterminethesmallestpossible
−8 mean-squareerror (MSE) for the regression parameter θ. The
considered system model is a cubic polynomial
−10 z =θx3+x. (30)
0 1 2 3 4
We assume a Gaussian input x ∼ N(a,b) and calculate the
θ
pessimistic CRLB with respect to the parameter θ
Fig.1. Information Loss-Salehmodel(a=2.1587,b=1.1517) ∂µ (θ) T ∂µ (θ) −1
CRLB (θ)= φ R−1(θ) φ . (31)
CM ∂θ φ ∂θ
model (26), under the considered choice of a and b, performs (cid:18) (cid:19) !
good for small and large input power values θ. For unit input Fig.3showstheconservativeapproximationofthenormalized
variance a substantial information loss, in comparison to the MSE
input, is present at the output of the amplifier.
CRLB (θ)
NRMSE (θ)= CM (32)
CM θ2
r
B. Rician model for different values of the regression parameter θ and input
signal setups a,b. It becomes visible that the configuration of
As a second example we investigate a Rician model
the input signal plays a significant role for the estimation of
θ. A higher variance and an input mean different from zero
z = x2+x2, (28)
1 2 will lead to better performance with an unbiased estimator of
q the regression parameter θ.
where x ∼ N(θcos(a),1) and x ∼ N(θsin(a),1). Such
1 2
a model is popular in wireless communications in order to
V. CONCLUSION
describe the strength of the line-of-sight (LOS) propagation
We have presented a conservative framework which allows
in relation to multi-path channels [1], [8]. Further, one finds
to determine the quality of nonlinear stochastic systems in
such models in biomedical signal processing for the charac-
a measurement-driven way. By replacing the original system
terization of brain scan images with fMRI [9]. Fig. 2 shows
with an equivalentcounterpartwhich belongsto the exponen-
the information loss of the Rician system
tial family, we showed that it is possible to derive a lower
1 ∂µ (θ) T ∂µ (θ) bound for the Fisher information measure. All entities which
χRM(θ)= F (θ) ∂φθ R−φ1(θ) ∂φθ (29) are required to determine the adequate replacement model
x (cid:18) (cid:19) can be obtained by calibrated measurements of the system
in dB. Note, that the Fisher informationmeasure with respect output.This allows to analyze the estimation theoreticquality
toθ underdirectaccesstobothinputsx andx isF (θ)=1. of measurement systems with complicated or unknown an-
1 2 x
It becomes visible that phase informationis extremely impor- alytical input-to-output relationship. With different examples
tantforsmall valuesof the distanceparameterθ. Thiskindof we have demonstratedthat the result is of interest for various
information is discarded by the Rician model (28). applications in signal processing and system identification.
5
15 a generic lower bound for the Fisher information measure
a=0,b=1
2
a=1,b=1 f(z;θ)g(z;θ)p(z;θ)dz
Z
a=0,b=2 F(θ)≥ (cid:16)R Zg2(z;θ)p(z;θ)dz (cid:17)
a=2,b=2
) 10 2
θ(M R βl(θ)∂E[∂φθl(z)]
C =
E E β (θ)φ (z)(cid:16)2P−2α(θ) β ((cid:17)θ)E[φ (z)]+α2(θ)
S l l l l
M
R h(cid:0)P (cid:1) i P (38)
N 5
is obtained. As the optimization problem
x⋆ =argmaxh(x) (39)
x∈R
with the objective function
0
0.2 0.4 0.6 0.8 1 a
h(x)= (40)
θ b−2xc+x2
hasa uniquesolution x⋆ =c, the strongestformof the bound
Fig.3. PessimisticCRLB-Cubicpolynomial model
(38) is obtained by setting
L
APPENDIX A α(θ)= βl(θ)Ez;θ[φl(z)]. (41)
FISHERINFORMATION BOUND Xl=1
Using (41) in (38) results in (8).
Proof: Define
∂lnp(z;θ)
f(z;θ)= , (33) REFERENCES
∂θ
[1] S. Medawar, P. Handel, P. Zetterberg, “Approximate maximum likeli-
and hoodestimation ofRician K-factor andinvestigation ofurbanwireless
measurements,” IEEE Trans. Wireless Commun., vol. 12, no. 6, pp.
L
2545–2555, June2013.
g(z;θ)= β (θ)φ (z)−α(θ), (34)
l l [2] G.W.Oehlert, “AnoteontheDelta method”, Amer.Statisticiany, vol.
l=1 46,no.1,pp.27–29,Feb.1992
X
[3] H.E.Rowe,“Memorylessnonlinearities withGaussianinputs:Elemen-
where βl(θ) and α(θ) are weighting factors, free to design. tary results,” Bell Syst. Tech. J., vol. 61, no. 7, pp. 1519–1525, Sept.
The two functions (33) and (34) have the properties 1982.
[4] K.Barbe´,L.Gonzales Fuentes,L.Barford,L.Lauwers,“Aguaranteed
blind and automatic probability density estimation of raw measure-
f(z;θ)g(z;θ)p(z;θ)dz =
ments,” IEEE Trans. Instrum. Meas., vol. 63, no. 9, pp. 2120–2128,
ZZ Sept.2014.
L ∂p(z;θ) ∂p(z;θ) [5] P.Stoica, P.Babu, “The Gaussian data assumption leads to the largest
= β (θ) φ (z)dz−α(θ) dz Crame`r-Raobound,”IEEESignalProcess.Mag.,vol.28,pp.132–133,
l l
Z ∂θ Z ∂θ May2011.
l=1 Z Z
X [6] M. Stein, A. Mezghani, J. A. Nossek, “A lower bound for the Fisher
L ∂E [φ (z)] informationmeasure,”IEEESignalProcess.Letters,vol.21,no.7,pp.
= βl(θ) z;θ l (35) 796–799,July2014.
∂θ
[7] A.A.M.Saleh,“Frequency-independent andfrequency-dependentnon-
l=1
X linear models ofTWTamplifiers,” IEEETrans.Commun.,vol.29,no.
and 11,pp.1715–1720, Nov.1981.
[8] C. Tepedelenlioglu, A. Abdi, G. B. Giannakis, “The Ricean K factor:
L 2
estimation andperformance analysis,” IEEETrans.Wireless Commun.,
g2(z;θ)p(z;θ)dz = βl(θ)φl(z) p(z;θ)dz vol.2,no.4,pp.799–810,July2003.
ZZ ZZ(cid:18)l=1 (cid:19) [9] K.Barbe´, W.VanMoer,L.Lauwers,“Functional Magnetic Resonance
X Imaging:Animprovedshortrecordsignalmodel,”IEEETrans.Instrum.
L
Meas.,vol.60,no.5,pp.1724–1731,2011.
−2α(θ) β (θ) φ (z)p(z;θ)dz+α2(θ)
l l [10] E.-W.Bai,“Frequency domainidentification ofWienermodels”, Auto-
l=1 ZZ matica, vol.39,no.9,,pp.1521–1530,Sept.2003
X
L 2 L [11] S. M. Kay, Fundamentals of Statistical Signal Processing: Estimation
=E β (θ)φ (z) −2α(θ) β (θ)E [φ (z)] Theory.UpperSaddlerRiver, NJ:Prentice Hall,1993.
z;θ l l l z;θ l [12] C. R. Rao, “Information and accuracy attainable in the estimation of
"(cid:18)Xl=1 (cid:19) # Xl=1 statistical parameters,” Bull. Calcutta Math. Soc., vol. 37, no. 3, pp.
+α2(θ). (36) 81–91,1945.
[13] H.Crame´r,MathematicalMethodsofStatistics.Princeton,NJ:Princeton
In conjunction with the Cauchy-Schwarz inequality Univ.Press,1946.
[14] M. Stein, S. Theiler, J. A. Nossek, “Overdemodulation for high-
2 performance receivers withlow-resolution ADC,”IEEEWireless Com-
Zf(z;θ)g(z;θ)p(z;θ)dz mun.Letters, vol.4,no.2,pp.169–172,2015.
f2(z;θ)p(z;θ)dz ≥ ,
ZZ (cid:16)R Zg2(z;θ)p(z;θ)dz (cid:17)
(37)
R