RVP-FLMS : A Robust Variable Power Fractional LMS Algorithm

0.12 MB·

by  Jawwad Ahmad

7
1
0
2

n
a
J

0
2

]
C
Pre-print version
O
. of
h
t
a ”RVP-FLMS : A Robust Variable Power Fractional LMS Algorithm”
m
by
[

1 Jawwad Ahmad, Muhammad Usman, Shujaat Khan, Imran Naseem and
v
7 Hassan Jamil Syed
7
6 submitted to
5
0 IEEE International Conference on Control System, Computing and
.
1
Engineering (ICCSCE). IEEE, 2016.
0
7
1
:
v
i
X
r
a
RVP-FLMS : A Robust Variable Power Fractional
LMS Algorithm
Jawwad Ahmad∗, Muhammad Usman∗, Shujaat Khan∗†, Imran Naseem‡ and Hassan Jamil Syed§
∗Faculty of Engineering Science and Technology (FEST),
Iqra University, Defence View,
Karachi-75500,Pakistan.
Email: {jawwad, musman, shujaat}@iqra.edu.pk
†Department of Bio and Brain Engineering,
Korea Advanced Institute of Science and Technology (KAIST),
335 Gwahangno, Yuseong-gu, Daejeon 305-701, Korea.
Email: [email protected]
‡College of Engineering,
Karachi Institute of Economics and Technology,
Korangi Creek, Karachi 75190, Pakistan.
Email: [email protected]
§Faculty of Computer Science and Information Technology,
University of Malaya, Malaysia.
Email: [email protected]
Abstract—In this paper, we propose an adaptive framework tical operatorsof FOC, researchershave shown an inclination
for the variable power of the fractional least mean square towardstheimplementationofFOCtoobservethesolutionof
(FLMS) algorithm. The proposed algorithm named as robust
the problems that were solved using the traditional calculus
variable power FLMS (RVP-FLMS) dynamically adapts the
[2].
fractional power of the FLMS to achieve high convergence rate
with low steady state error. For the evaluation purpose, the For the problem of path tracking in autonomous vehicles
problems of system identification and channel equalization are FOC was used in [3]. FOC was used in [4] to give an
considered. The experiments clearly show that the proposed experimental proof of a theoretical model proposed for the
approachachievesbetterconvergencerateandlowersteady-state
pulse propagation through porous medium. FOC has turned
errorcomparedtotheFLMS.TheMATLABcodefortherelated
outtobeaconvenienttoolintheapplicationsofviscoelasticity
simulation is available online at https://goo.gl/dGTGmP.
[5], image processing [6], edge detection [7], control systems
Index Terms— Least mean square (LMS), fractional calculus, and many others [8]. Implementation of FOC in the field of
plant identification, channel equalization, fractional LMS (FLMS), signal processing has been suggested and opted by a number
robust variable power FLMS(RVP-FLMS),robust variable stepsize of researchers. In this context, the utilization of FOC in least
(RVSS), high convergence, low steady state error, adaptive filter.
mean square (LMS) adaptive filtering is being particularly
focused.
I. INTRODUCTION
FOC based least mean square algorithm was proposed in
The conventional integer value integration and differenti- [9] as FLMS. The algorithm was implemented on system
ation is undoubtedly a fundamental tool of calculus being identification problem and the results were compared with
utilizedbyprofessionalstodeterminethesolutionofnumerous LMS. Several variants of FLMS algorithm have been pro-
problems in various fields. On the contrary fractional order posed showing better performance [10], [11], [12]. In this
calculus (FOC) deals with the derivative and integral calcu- paper a contemporarymodification of the FLMS algorithm is
lated with the fractional power. Derivative with the fractional proposed. The proposal is to use a variable fractional power
power is certainly a paradox which introduced a new aspect rather than a fixed one, the concept of which is inspired by
in mathematics.Thoughthe conceptsof FOC were developed variable learning rate least mean square algorithm (RVSS-
earlier in the 16th centuryby Liouville,Reimann and Leibniz LMS) [13]. The objective is to use variable power of the
[1],tendencyofitsutilizationingenericareasbecamenotable fractional derivative. The proposed algorithm is evaluated on
inthepastdecade.Withthematurationoftheoreticalandprac- theproblemofequalizationandplantidentification,theresults
v
∂J(n) ∂
of which are compared with the conventional LMS. The rest According to chain rule and J(n) are
of the paper is structured as follows: A brief literature review ∂wk (cid:18)∂wk(cid:19)
defined as :
of modified least mean algorithms is discussed in section
II. The details of proposed algorithm is discussed in section
∂J(n) ∂J(n)∂e(n)∂y(n)
III followed by the experimental setup in section IV. The
= (3)
conclusion of the paper is drawn in section V. ∂wk ∂e ∂y ∂wk
v v
II. LEASTMEAN SQUARE ∂ ∂J(n)∂e(n) ∂
J(n)= y(n) (4)
(cid:18)∂w (cid:19) ∂e ∂y (cid:18)∂w (cid:19)
Least mean square algorithm is an adaptive filter that k k
belongstothegroupofstochasticgradientmethods.LMShas solving equation (3) result in equation (5)
been extensively adopted to counter a number of problems in ∂J(n)
=−e(n)x(n) (5)
the area of signal processing.A considerableamountof work
∂w
k
is done towards the mitigation of the drawbacks in LMS.
Using the Rieman-Lioville fractional derivative method
Researchers have used several techniques to optimize the
t
performanceofLMS[10],[14],[15],[16],[17].Thetheoryof 1 d
FOC was applied to the LMS algorithm and a fractionalleast (Dvf)(t)= ( )n (t−τ)n−v−1f(τ)dτ (6)
Γ dt Z
meansquare(FLMS)wasproposedin[17].Thealgorithmwas 0
testedfortheestimationofinputnonlinearcontrolautoregres-
Γ(1+α)
sive (INCAR) models and the output resulted in better con- Dv(t−a)α = (t−a)α−v (7)
vergence rates when compared to volterra least mean square Γ(1+α+v)
(VLMS)andkernelleast meansquare(KLMS).Themodified equation (4) result in equation (8)
fractional least mean square (MFLMS) was proposed in [11].
The algorithm was designed to improve the computational ∂ v w(1−v)(n)
complexity involved in FLMS due to the complex gamma J(n)=−e(n)x(n) k (8)
(cid:18)∂w (cid:19) Γ(2−v)
function.MFLMShasthereforeshownimprovedresultscom- k
Using equation (5) and (8) the weight update equation (1)
pared to the LMS and FLMS. In another work [12] the
becomes :
adaptive weight gain parameters are incorporated. Gradient-
based approach is implemented on variable learning scheme
that changedthe natureof the orderof fractionalderivativein
w (n+1)=w (n)+µe(n)x(n)
k k
MFLMSfromfixedto adaptive.Theyproposedadaptivestep-
w1−v(n)
size modified fractional least mean square (AMFLMS) and +µ e(n)x(n) k (9)
f
presentedimprovementbycomparingtheiralgorithmwiththe Γ(2−v)
LMS, FLMS and MFLMS. For time varying fractionalpower the v can be replaced by
In this research we propose to utilize the conceptof robust v(n) :
variablestepsize(RVSS)[13]forthevariablefractionalpower
of FLMSalgorithm.Theproposedschemeis robustand com- wk(n+1)=wk(n)+µe(n)x(n)
putationallylessexpensive.Fortheperformanceevaluationwe w1−v(n)(n)
+µ e(n)x(n) k (10)
considertheproblemofsystemidentificationandequalization. f
Γ(2−v(n))
The results are compared with the FLMS algorithm.
The update rule for the time varying fractional power v
using the RVSS-LMS based method is defined as [13] :
III. PROPOSED RVP FLMS
v(n+1)=βv(n)+γp2(n) (11)
In [9] the weight update equation for FLMS is given as :
where (0 < β < 1), (γ > 0), p(n) is the average error
∂J(n) ∂ v energy correlation and v(n+1) is set to vmin or vmax
w (n+1)=w (n)−µ −µ J(n) (1)
k k f
∂w (cid:18)∂w (cid:19)
k k p(n)=αp(n−1)+(1−α)e(n)e(n−1) (12)
where wk(n) is the weight of the kth tap at nth iteration, The positive constant α (0 < α < 1) is a weighting
J(n) is the cost function defined in equation (2), v is the parameter that governs the averaging time constant. It is in
fractional power of derivative, µ and µf are the step sizes. actual the forgetting factor. The limits on v(n+1) is given
by:
J(n)= 1e(n)2 = 1(d(n)−y(n))2 (2) vmax if v(n+1)>vmax
2 2 v(n+1)= vmin if v(n+1)<vmin (13)
 v(n+1) otherwise
where e(n) is the instantaneous error between the desired
output d(n) and the estimated output y(n) at nth iteration. where v >v >0.
max min
IV. SIMULATIONSETUP AND RESULTS
FLMS (10dB)
For the evaluation of the proposed method the problem of 0 RVP−FLMS (10dB)
B)
system identification and equalization are considered. E (d −5
S
M
A. System Identification
−10
Toestimatethemathematicalmodelofasystem,thesystem
0 10 20 30 40 50 60 70 80 90 100
identification method is used. Literature including [9], [14], Iterations
[17], [18], [19] show that the adaptive techniques produce 5
FLMS (20dB)
good performance in the application of system identification. 0 RVP−FLMS (20dB)
Tcoonsaisdseerssatlhineeparersfyosrtmeman,csehoowfnthienpFriogpuorese1d:RVP-FLMS, we MSE (dB)−−105
−15
−20

0 10 20 30 40 50 60 70 80 90 100
y(t) = a x(t) + a x(t − 1) + a x(t − 2) + n(t) (14) Iterations
1 2 3
Equation(14)showsthemathematicalmodelofthesystem,
wherex(t)istheinputandy(t)istheoutputofthesystem,the Fig.2. Meansquareerror(MSE)curveforsystemidentification problem
disturbancemodeln(t)isassumedtobeN(0,σ2),a sdepicts
d i
the polynomial coefficients representing number of zeros in
FLMS takes 1.57 seconds. The experimentclearly shows that
the system. For the experiment, x(t) is taken to be a binary
the proposedrobustapproachdynamicallyadaptsthe learning
phaseshiftkeying(BPSK)modulated500randomlygenerated
ratetoachievetheminimumsteadystateerrorinlessernumber
samples. In Figure 1, the impulse response of the system is
of iteration.
p(t)whileyˆ(t)isapproximateoutput,pˆ(t)istheapproximate
impulse responseand b(t) is the errorof estimation. The sim- B. Equalization
ulation parametersselected for the experimentsare: a1 =0.9, The equalizer is used to nullify the effect of noise and
a2 =0.3, a3 =−0.1. The experiments are performed on two distortioncausedbythechannel.Adaptivelearningtechniques
noise levels with the SNR values of 10 dB and 20 dB. show good results in this context [20], [21], [22], [23]. To
There are 3 number of taps for the model structure. The evaluate the effectiveness of the proposed RVP-FLMS, we
weights values were initialised to zero. For the FLMS and
consider a linear filter, shown in Figure 3:
RVP-FLMS value of the µ and µ was set to 1×10−4 and
f
the initial value of the fractional power v is taken to be 0.5.
For the proposedRVP-FLMS the valuesof α, β and γ are all
set to 0.9, 0.99, and 0.9 respectively. The value of v and
max
v are chosen to be 0.5 and 1 respectively.
min
Theexperimentareperformedontwonoiselevelsthemean
square error (MSE) curves are shown in Figure 2. For the
SNR values of 10 dB and 20 dB the RVP-FLMS shows the
best performance.The MSE of −10.44 dB and −20.20 dB is
achievedin around45 and 50 iterationsrespectively.Whereas
Fig.3. Equalization usingadaptive learning algorithm.
the FLMS converges to the MSE value of −10.43 dB and
−20.17 dB in around 80 and 90 iterations respectively.
To compare the time complexity of the suggested method
with FLMS, we investigated the training time for 100 itera-
q(t) = a y(t) + a y(t − 1) + a y(t − 2) (15)
1 2 3
tions. The proposedmethodutilizes1.85secondswhereasthe
Equation (15) shows the mathematical model of the linear
filter,wherey(t)istheinputandq(t)istheoutputofthefilter.
a s depicts the polynomial coefficients which are actually
i
the number of zeros in the system. For the purpose of this
experiment the same plant defined in section (IV-A) is used.
In Figure 3, the impulse response of the filter is p(t)
while the disturbancemodeln(t) is supposedto be N(0,σ2).
d
yˆ(t) is the approximate input, x(t) is the approximate noisy
outputande(t)istheerrorofestimation.Theexperimentsare
performedon two noise levels with the SNR values of 10 dB
and 20 dB.
In the equalizer structure, 3 number of taps were chosen.
Fig.1. Systemidentification usingadaptive learning algorithm. Theweightsvalueswereinitialisedtozero.FortheFLMSand
5 [2] R.E.Gutie´rrez,J.M.Rosa´rio,andJ.TenreiroMachado,“Fractionalor-
FLMS (10dB)
RVP−FLMS (10dB) dercalculus:basicconceptsandengineeringapplications,”Mathematical
B) 0 ProblemsinEngineering, vol.2010,2010.
E (d [3] J.I.Sua´rez,B.M.Vinagre,A.Caldero´n,C.Monje,andY.Chen,“Using
MS −5 fractional calculus for lateral and longitudinal control of autonomous
vehicles,” in International Conference on Computer Aided Systems
Theory. Springer, 2003,pp.337–348.
−10
0 10 20 30 40 50 60 70 80 90 100 [4] Z.Fellah,C.Depollier,andM.Fellah,“Applicationoffractionalcalculus
Iterations
tothesoundwavespropagationinrigidporousmaterials:validationvia
5 ultrasonic measurements,” ActaAcustica united with Acustica, vol. 88,
FLMS (20dB) no.1,pp.34–39,2002.
0 RVP−FLMS (20dB)
[5] E. Soczkiewicz, “Application of fractional calculus in the theory of
E (dB) −5 viscoelasticity,” Molecular and Quantum Acoustics, vol. 23, pp. 397–
S−10 404,2002.
M
[6] B. Mathieu, P. Melchior, A. Oustaloup, and C. Ceyral, “Fractional
−15
differentiation for edge detection,” Signal Processing, vol. 83, no. 11,
−200 10 20 30 40 50 60 70 80 90 100 pp.2421–2432, 2003.
Iterations [7] L. Debnath, “Recent applications of fractional calculus to science and
engineering,” International Journal of Mathematics and Mathematical
Sciences, vol.2003,no.54,pp.3413–3442, 2003.
Fig.4. Meansquareerror(MSE)curveforequalization problem [8] M.DalirandM.Bashour,“Applicationsoffractionalcalculus,”Applied
Mathematical Sciences, vol.4,no.21,pp.1021–1032, 2010.
[9] R. M. A. Zahoor and I. M. Qureshi, “A modified least mean square
RVP-FLMS value of the µ and µ was set to 1×10−4 and algorithm using fractional derivative and its application to system
f identification,” EuropeanJournal ofScientific Research,vol.35,no.1,
the initial value of the fractional power v is taken to be 0.5. pp.14–21,2009.
For the proposedRVP-FLMS the valuesof α, β and γ are all [10] Y. Tan, Z. He, and B. Tian, “A novel generalization of modified lms
set to 0.9, 0.99, and 0.9 respectively. The value of v and algorithmtofractionalorder,”SignalProcessingLetters,IEEE,vol.22,
max no.9,pp.1244–1248,2015.
vmin are chosen to be 0.5 and 1 respectively. [11] B. Shoaib, I. M. Qureshi, Ihsanulhaq, and Shafqatullah, “A modified
Theexperimentareperformedontwonoiselevelsthemean fractional least mean square algorithm for chaotic and nonstationary
time series prediction,” Chinese Physics B, vol. 23, no. 3, p. 030502,
square error (MSE) curves are shown in Figure 4. For the
2014.
SNR values of 10 dB and 20 dB the RVP-FLMS shows the [12] B.Shoaib,I.M.Qureshi,Shafqatullah,andIhsanulhaq,“Adaptivestep-
best performance. The MSE of −9.96 dB and −16.33 dB is size modified fractional least mean square algorithm for chaotic time
seriesprediction,” Chinese PhysicsB,vol.23,no.5,p.050503,2014.
achievedin around40 and 50 iterationsrespectively.Whereas
[13] T. Aboulnasr and K. Mayyas, “A robust variable step-size lms-type
the FLMS converges to the MSE value of −9.58 dB and algorithm: analysis and simulations,” IEEE Transactions on signal
−16.28 dB in around 70 and 90 iterations respectively. processing, vol.45,no.3,pp.631–639,1997.
[14] M. A. Z. Raja and N. I. Chaudhary, “Two-stage fractional least mean
To compare the time complexity of the suggested method
square identification algorithm for parameter estimation of carma sys-
with FLMS, we investigated the training time for 100 itera- tems,”SignalProcessing,vol.107,pp.327–339, 2015.
tions.Thesuggestedmethodrequires1.92secondswhereasthe [15] A.Khalili,A.Rastegarnia,andS.Sanei,“Quantizedaugmentedcomplex
least-mean square algorithm: Derivation and performance analysis,”
FLMS takes 1.63 seconds. The experimentclearly shows that
SignalProcessing,vol.121,pp.54–59,2016.
the proposedrobustapproachdynamicallyadaptsthe learning [16] A. Khalili, A. Rastegarnia, W. M. Bazzi, and Z. Yang, “Derivation
ratetoachievetheminimumsteadystateerrorinlessernumber and analysis of incremental augmented complex least mean square
algorithm,” SignalProcessing,IET,vol.9,no.4,pp.312–319,2015.
of iteration.
[17] N. I. Chaudhary, M. A. Z. Raja, J. A. Khan, and M. S. Aslam,
“Identification of input nonlinear control autoregressive systems using
V. CONCLUSION fractionalsignalprocessingapproach,”TheScientificWorldJournal,vol.
2013,2013.
In this research an adaptive least mean square algorithm
[18] S. Ciochin, C. Paleologu, and J. Benesty, “An optimized {NLMS}
based on fractional derivative is proposed. In particular, the algorithm for system identification,” Signal Processing, vol. 118, pp.
fractional power of the FLMS is made variable using the 115–121,2016.
[19] S.Khan,I.Naseem,R.Togneri,andM.Bennamoun,“Anoveladaptive
concept of robust variable step size. The performance of the
kernel for the rbf neural networks,” Circuits, Systems, and Signal
proposed approach is evaluated on system identification and Processing,pp.1–15,2016.
equalization problems. The results of the proposed algorithm [20] S. Chen, B. Mulgrew, and P. M. Grant, “A clustering technique for
digitalcommunications channelequalization usingradialbasisfunction
are compared to the FLMS. The proposed algorithm attains
networks,” IEEE Transactions on neural networks, vol. 4, no. 4, pp.
better convergencerate and steady-state error and is therefore 570–590,1993.
found to be superior to the FLMS algorithm. [21] M. V. Clark, “Adaptive frequency-domain equalization and diversity
combining for broadband wireless communications,” IEEE Journal on
SelectedAreasinCommunications,vol.16,no.8,pp.1385–1395,1998.
ACKNOWLEDGMENT [22] S.Liu,G.Shen,andH.Tian,“A60-ghzrofsystememployingvariable
step size lms equalizer with fast convergence speed,” in Optical Fiber
The research is partially supported by the National ICT
Communication Conference. Optical Society of America, 2016, pp.
R&D Fund (http://www.ictrdf.org.pk) Th2A–18.
[23] S.SoniandA.Agarwal,“Analysisandsynthesisofadaptiveequalization
REFERENCES techniques under various modulation techniques,” in Recent Develop-
mentsinControl,AutomationandPowerEngineering(RDCAPE),2015
International Conference on. IEEE,2015,pp.102–106.
[1] J. Liouville, Me´moire sur le changement de la variable inde´pendante,
danslecalcul desdiffe´rentielles aindices quelconques, 1835.