Table Of ContentParameters Optimizing of Late Reverberation
Spectrum Estimator
Arkadiy Prodeus1
1. Acoustic and Acoustoelectronic Department, National Technical University of Ukraine (KPI), UKRAINE, Kyiv, Pobeda Avenue
37, E-mail: [email protected]
Abstract - Enhancement of speech distorted by The reverberant signal y(t) results from the
reverberation is issue of the day. Before suppression of convolution of the anechoic speech signal x(t) and the
late reverberation by spectral subtraction or
causal time-invariant Acoustic Impulse Response (AIR)
frequency correction techniques, it is necessary to
h(t):
estimate the spectrum of the late reverberation.
Recommendations for optimizing of late reverberation ∞
y(t)= ∫h(v)x(t−v)dv=x(t)⊗h(t),
spectrum estimation had been proposed in the paper.
Keywords - Speech enhancement, reverberation 0
suppression, late reverberation spectrum, optimizing. were ⊗ is convolution symbol.
When selecting regions corresponding to early and late
I. INTRODUCTION reflections in AIR h(t)
Problem of speech dereverberation in automatic speech h(t), 0≤t≤Tl; h(t+Tl), t≥0;
recognition (ASR) and communication systems is issue of hi(t)=0, др.t, hl(t)=0, др.t,
the day [1-4]. This problem was especially actively
reverberation action can be described as
investigated in the last decade due to the rapid
y(t)=h (t)⊗x(t)+r(t), (1)
development of mobile communications. It was found i
that late reverberation is main detrimental factor which where r(t)=hl(t)⊗x(t) is component due to late
may be interpreted as kind of noise. Unfortunately, strong reverberation; T is time, corresponding to boundary
l
non-stationarity of late reverberation makes ineffective
between early and late reflections (see Fig. 1).
traditional techniques of noise suppression such as
Because of stochastic character of late reverberation,
spectral subtraction or frequency correction, because
it’s action can be interpreted as kind of noise.
these techniques are designed for stationary or slow non-
Unfortunately, strong non-stationarity of late
stationary noise.
reverberation makes ineffective traditional techniques of
At the same time it was found that late reverberation
noise suppression [5], because these techniques are
power spectrum may be relatively easily estimated when
designed for stationary or slow non-stationary noise. At
Polack’s statistical reverberation model is chosen [3].
the same time, influence of early reflections, described
Formulas for such estimation had been developed, but
with convolution of signal x(t) and AIR h (t), may be
these formulas contain some parameters, which values i
compensated in ASR systems by standard techniques,
were not optimized. Firstly, parameter T value, which is
l such as, for example, mean cepstral normalization [2].
time corresponding to boundary between early reflections
and late reverberation, was not optimized. The problem is
that the boundary is blurred: we find T ≈30...60 ms in
l
[1] and 40…100 ms in [4]. But these values were
experimentally obtained when problems of speech
intelligibility and musical clarity were investigated, and it
isn’t evident that the same values will be good for ASR
and communication systems. Secondly, there was not
optimized averaging procedure when estimating of late
reverberation spectrum.
The objective of the paper is developing of
recommendations on optimizing of late reverberation Fig. 1 Room AIR structure
spectrum estimation in sense of maximizing of ASR It can be shown that late reverberation suppression
accuracy and speech quality. procedure may be realized almost by the same remedies
which are usually using for noise suppression. The only
II. LATE REVERBERATION SPECTRUM distinction consists in estimation of late reverberation
ESTIMATION spectrum instead of noise spectrum.
Proceedings of the Xth International Conference "Perspective Technologies and Methods in MEMs Design"
(MEMSTECH 2014). - June 2014, Lviv, Ukraine. – P. 100-103.
Correction in frequency field is one of the most spread It was also proposed upper-bound of constant ηd(k)
z
approaches to noise suppression:
λˆ12(l,k)=G(l,k)λ12(l,k), (0≤ηdz(k)<1):
x y
1
where λy(l,k) is power spectrum of l-th signal y(t) ηdz(k)=1+2δ(k)R F . (7)
s
frame at frequency f =kF /N ; F is sampling
k s fft s
III. EXPERIMENTAL RESULTS
frequency; N is FFT parameter; k is number of
fft
Clean speech signals (single words) were recorded in
ˆ
frequency sample; λx(l,k) is power spectrum estimator anechoic room and had been used for ASR system
of l-th frame of signal x(t) for k-th frequency sample; training. Parameters of digitized sounds were: sampling
rate 22050 Hz, linear quantization 16 bit. Reverberated
G(l,k) is correction filter gain for l-th signal y(t) frame
signals had been simulated by convolving of clear speech
for k-th frequency sample.
and AIRs of three rooms with time reverberation 0.74 s,
When using logMMSE method [5] for noise
0.89 s and 1.1 s.
suppression, filter gain is
VoiceBox [6] routine “ssubmmse.m” designed to
ξ(l,k) 1 ∞ e−t reduce the noise was modified in accordance with Eqs.
G(l,k)= exp ∫ dt, (4)-(7) for getting dereverberator as ASR system
1+ξ(l,k) 2 t
v(l,k) preprocessor. Frame length 32 ms with 50% overlapping
ξ(l,k) and Hamming window were used for signal processing.
v(l,k)= γ(l,k),
Reverberation time was estimated by applying
1+ξ(l,k)
Schroeder’s method [4] to a bandpass filtered versions of
where ξ(l,k)=λ (l,k) λ (l,k) is prior signal-to-noise
x n a d
the AIRs. Moreover, it was taken η (k)=0,5⋅η (k).
ratio (SNR); γ(l,k)=λ (l,k) λ (l,k) - posterior SNR; y y
y n
Toolkit HTK [7] had been used for ASR system
λ (l,k) - power spectrum of l-th noisen(t) frame at
n simulation. Training of ASR system had been made with
frequency fk. usage of 269 samples of 27 words of clean speech saved
When modifying scheme of noise suppression, which for two speakers-women. Reverberated discrete speech
is made in accordance with (2)-(4), we can substitute late signal (with 0.2…0.5 s pauses between single words) was
reverberation spectrum λ (l,k) assessment instead of used as test signal, and there were presented, in testing,
r
all 27 words used in training. There were 27 phonemes of
noise spectrum λ (l,k) assessment.
n Ukrainian language in phoneme vocabulary and there had
For distances between speech source and microphone,
been used 39 MFCC_0_D_A coefficients when ASR
which are more then critical distance D [1], late
c simulating.
reverberation power spectrum λ (l,k) may be calculated Quantitative evaluation of dereverberation performance
r
by spectrum λ (l,k) of signal y(t) [3, 4]: had been made by means of objective measures, such as
y
ASR accuracy and speech quality. ASR accuracy assessed
λ (l,k)=e−2δ(k)Tl ⋅λ (l−N ,k), (2) using the indicator:
r y l
N−D−S−I
where Nl =TlFs/R; R denotes the frame rate in Acc%= ×100%,
N
samples of the short-time Fourier transform (STFT);
where N is the total number of labels in the reference
δ(k)=2ln10 T (k); T (k) is reverberation time.
60 60
transcriptions; D is the number of deletion errors; S is
To enhance the accuracy of the spectrum estimation
the number of substitution errors; I is the number of
λ (l,k) it was proposed averaging in [3]:
y insertion errors. Indicator PESQ had been used for speech
ˆ ˆ 2 quality assessment [8, 9].
λ (l,k)=η λ (l−1,k)+(1−η )Y(l,k) , (3)
y z y z
It was shown in [10] for 2-nd averaging type that
where ηz is averaging parameter (0≤ηz <1) and optimal, in sense of Acc% maximum, T value lies in the
l
Y(l,k) is STFT of signal y(t). We will name below the interval 100…200 ms. More uncertain is situation with
averaging as “1-st averaging type”. PESQ(T ) dependency, where, in two of three cases, the
l
More complicated averaging (named below as “2-nd speech quality decreased with increasing T values, and
l
averaging type”) algorithm was proposed in [4]:
only one case was observed with weakly pronounced
ˆ ˆ 2
λy(l,k)=ηz(k)λy(l−1,k)+(1−ηz(k))Y(l,k) , (4) maximum at Tl ≈200...240 ms.
ηz(k)=ηdz(k), Y(l,k)2 ≤λˆy(l−1,k); (5) parIanm tehtiesr sp avpaelur,e sa tatenmd pat lstoo tdoe fcinoem pmaorere 1e-xsta catnlyd t2h-ensde
ηaz(k), востальныхслучаях, averaging types, had been made.
Graphics of Acc%(T ) for different η values,
0≤ηa(k)<ηd(k). (6) l z
z z calculated in accordance with Eqs. (4)-(7) are shown in
Proceedings of the Xth International Conference "Perspective Technologies and Methods in MEMs Design"
(MEMSTECH 2014). - June 2014, Lviv, Ukraine. – P. 100-103.
Figs. 2a-2d. But here η parameter (it’s values are provides recognition accuracy, which is 5% better then
z
denoted as “etsd” values) was adopted as independent of one for ηz(k).
k, as in the case of Eq. (3). It can be seen that all
Averaged recognition accuracy
Acc%(Tl) curves have maximums for Tl ≈90K110 ms
80,0
and ηz ≈0,6K0,75. Average dependency (Fig. 2d) is the 70,0
60,0
most informative, so we will use averaged results below. %50,0 etzd = 0,67
Recognition accuracy for T60=0,74 s Acc3400,,00 eettzzdd == 0o,ld75
20,0
10,0
90 0,0
80
70 etzd = 0 16 48 80 112 144 176 208 240
% 5600 etzd = 0,2 Tl, ms
Acc 40 etzd = 0,5
30 etzd = 0,666 Fig. 3 Comparison of cases η (k) and η (k)=const
20 z z
100 eettzzdd == 00,,875 When studying Acc%(T) for ηd =0.5 and ηa =kηd
l z z z
32 64 96 128 160 192 224 etzd = 0,833 (k =0,1; 0,5; 0,9), we can see that Acc% is independent
Tl, ms
of ηa (Fig. 4).
a z
Averaged recognition accuracy
Recognition accuracy for T60=0,89 s
70,0
70 60,0
5600 etzd = 0 %4500,,00 k = 0,1
Acc% 3400 eettzzdd == 00,,25 Acc2300,,00 kk == 00,,59
20 etzd = 0,666 10,0
10 etzd = 0,75 0,0
016 48 80 112 144 176 208 240 eettzzdd == 00,,8833 16 48 80 112 Tl, 1m44s 176 208 240
Tl, ms
Fig. 4 Comparison of cases ηa =kηd for Acc%(T)
b z z l
Important conclusion from the results can be made.
Recognition accuracy for T60=1,1 s Simple 1-st averaging type procedure is potentially more
effective compare to 2-nd averaging type procedure,
80
70 when proper averaging parameter η value is chosen.
60 etzd = 0 z
Acc% 345000 eettzzdd == 00,,25 c omAvpelircaagteedd chaPraEcStQer( T(Fl)ig . 5d).e pendencies have more
etzd = 0,666
20
10 etzd = 0,75 Averaged speech quality
0 etzd = 0,8
16 48 80 112 144 176 208 240 etzd = 0,833 2,350
Tl, ms 2,300 etzd = 0
2,250 etzd = 0,2
c ESQ22,,125000 eettzzdd == 00,,567
P2,100 etzd = 0,75
Averaged recognition accuracy 2,050 etzd = 0,8
2,000
etzd = 0,83
80,0 1,950
6700,,00 eettzzdd == 00,2 16 48 80 112 144 176 208 240
50,0 etzd = 0,5 Tl, ms
%
Acc3400,,00 eettzzdd == 00,,6775 a
20,0 etzd = 0,8
10,0 etzd = 0,83 Averaged speech quality
0,0
16 48 80 112 144 176 208 240 2,350
2,300
Tl, ms
2,250 etzd = old
d SQ2,200 etzd = 0,5
PE2,150 etzd = 0,67
Fig. 2 Averaged Acc%(Tl) curves 2,100 etzd = 0,75
2,050
Results of comparing the best curves of Fig. 2d with
2,000
similar curve, calculated for ηz(k), are shown in Fig. 3. 16 48 80 112 144 176 208 240
Tl, ms
The case of ηz(k) is denoted as “etzd = old”. It can be
b
seen that proper choice of ηz(k)=const≈0.66K0.75
Fig. 5 Averaged PESQ(T) curves
l
Proceedings of the Xth International Conference "Perspective Technologies and Methods in MEMs Design"
(MEMSTECH 2014). - June 2014, Lviv, Ukraine. – P. 100-103.
Maximum of PESQ(Tl) occurs for ηdz ≈0.5K0.66, times 0.74...1.1 s, which are typical for laboratories and
classrooms, this value is close 100 ms.
but the PESQ(T)curves have not local extremes. Local
l REFERENCES
extremes of PESQ(Tl) occurs when ηdz ≈0K0.2 for [1] P. Naylor and N. Gaubitch, Speech Dereverberation.
Tl ≈50K80 ms (Fig. 5a). Springer, 2010, 399 p.
Results of comparing the best curves of Fig. 5a with [2] T. Yoshioka et al, “Making Mashine Understand Us in
similar curve, calculated for η (k), are shown in Fig. 5b. Reverberant Rooms,” IEEE Signal Processing
z
The case of η (k) is denoted “etzd = old”, as before. It is Magazine, vol.29, No.6, pp.114-126, Nov. 2012.
z
[3] K. Lebart at al, “A new method based on spectral
evident that choice of η (k)=0.5 provides speech
z subtraction for speech dereverberation,” Acta
quality, which is aloud better then one for ηz(k). Acoustica, vol.87, No.3, pp.359–366, 2001.
When studying PESQ(T) for ηd =0.5 and [4] E. Habets, Single- and Multi-Microphone Speech
l z
Dereverberation using Spectral Enhancement: dissert.
ηa =kηd, we can see that PESQ(T) is independent of
z z l … Doctor of Philosophy Electrical Engineering,
ηa (Fig. 6). Technische Universiteit Eindhoven, 2007.
z
[5] Y. Ephraim and D. Malah, “Speech enhancement
Averaged speech quality using a minimum mean square error Log-spectral
2,350 amplitude estimator,” IEEE Trans. Acoust., Speech,
2,300 Signal Processing, vol.ASSP-33, pp.443-445, 1985.
k = 0,1
Q2,250 k = 0,3 [6] M. Brooks, VOICEBOX: Speech Processing Toolbox
ES2,200 k = 0,5 for MATLAB. Imperial College London, Electrical
P2,150 k = 0,7
2,100 k = 0,9 Engineering Department, 2014.
[7] S. Young et al, The HTK Book. Cambridge:
2,050
16 48 80 112 144 176 208 240 University Engineering Department, 2009.
Tl, ms [8] P. Loizou, Speech enhancement: Theory and Practice.
Boca Raton: CRC Press, 2007.
Fig. 6 Comparison of cases ηa =kηd for PESQ(T)
z z l [9] J. Beerends, E. Larsen, N. Iyer and J. van Vugt,
“Measurement of speech intelligibility based on the
IV. DISCUSSION
PESQ approach,” in Proc. Int. Conf. Measurement of
It was shown experimentally that simple averaging type Speech and Audio Quality in Networks (MESAQIN),
procedure, based on Eq. (3), is potentially more effective Prague, Czech Republic, June 2004.
compare to averaging type procedure, based on Eqs. (4)- [10] A. Prodeus and O. Ladoshko, “On existance of
(7), when proper averaging parameter ηz value is optimal boundary value between early reflections and
chosen. late reverberation,” in Electronics and
At 32 ms frames, overlapping by 50%, local extremes Nanotechnology: IEEE 34th International Scientific
of dependencies Acc%(T ) and PESQ(T) were Conference. Nation. Tech. Univ. of Ukraine, Kyiv,
l l
Ukraine, 15-18 April 2014.
identified within range 0≤η ≤0.83. Thus, the Acc%
z
maximum was reached at Tl ≈90K110 ms and V. CONCLUSION
ηz ≈0.67K0.75. The PESQ(Tl) maximum was It was shown in the paper that simple averaging
procedure can be used when late reverberation power
achieved at η ≈0.5 and T ≈16 ms, but PESQ(T)
z l l
spectrum is estimated. Empirical relation to determine the
hasn’t local maximum at η ≈0.5.
z optimal averaging parameter η value for any
z
Summarizing the results, it can be proposed an
(reasonable) frame length T and frame shift time
empirical relation to determine the optimal parameter frame
values for any (reasonable) frame length T and frame T values had been proposed.
frame shift
shift time T values: Detailed experimental studies of dependencies
shift
Acc%(T ) and PESQ(T ) were realized. It was shown
T −T l l
η = aver frame , that optimal, in sense of Acc% maximum, T value is
z T −T +T l
aver frame shift
close 100 ms. PESQ(T) maximum was achieved at
where it’s necessary to take Taver =60K80 ms to η ≈0.5 and T ≈16 mls, but PESQ(T) hasn’t local
maximize Acc%(T ), and T ≈50 ms to maximize z l l
l aver maximum at η ≈0.5 and is similar to a monotonically
PESQ(T). z
l decreasing function.
More detailed, comparing with [7], the functional
dependences Acc%(T ) had been got, which helped to
l
clarify the optimal parameter T value. For reverberation
l
Proceedings of the Xth International Conference "Perspective Technologies and Methods in MEMs Design"
(MEMSTECH 2014). - June 2014, Lviv, Ukraine. – P. 100-103.
