vata: an improved personal computer-based vowel articulation

94 Pages·2004·0.66 MB·English

by  A. Matthew Zimmer

VATA: AN IMPROVED PERSONAL COMPUTER-BASED VOWEL
ARTICULATION TRAINING AID

by

Andrew Matthew Zimmer
B.A. May 1992, University of Virginia
B.S. May 1997, Old Dominion University

A Thesis Submitted to the Faculty of
Old Dominion University in Partial Fulfillment of the
Requirement for the Degree of

MASTER OF SCIENCE
ELECTRICAL ENGINEERING

OLD DOMINION UNIVERSITY
May 2002

Approved by:

______________________________
Stephen A. Zahorian (Director)

______________________________
Vijayan K. Asari (Member)

______________________________
Ravindra P. Joshi (Member)
ABSTRACT
VATA: AN IMPROVED PERSONAL COMPUTER-BASED VOWEL
ARTICULATION TRAINING AID

Andrew Matthew Zimmer
Old Dominion University, 2002
Director: Dr. Stephen A. Zahorian

This thesis discusses work accomplished on the further
development of a vowel articulation training aid for persons with
hearing impairments. The system produces visual feedback about the
quality of articulation for ten American English monophthong vowel
phonemes. A large database of vowel recordings has been collected and
used to improve and test the recognition rates of artificial neural
networks used by the training aid. A maximum-likelihood classifier has
been introduced into the system to improve performance by reducing the
likelihood of incorrect feedback produced by the displays. A series of
experiments has been performed to examine effects of tuning parameters,
and of varying the amount and type of speech data used to train the
system. Much of the program code for the speech display system has been
revised to use a more modular structure, which in turn provides for
easier maintenance and testing. Current system features and operational
details are discussed in an appendix.

Members of Advisory Committee: Dr. Vijayan K. Asari
Dr. Ravindra P. Joshi
iii

© 2002 A. Matthew Zimmer. All Rights Reserved.
iv

To Mom, Dad, Bretta and Ben.
Thanks for all of your patience and support.
v
ACKNOWLEDGMENTS

I would like to thank Dr. Stephen Zahorian for the opportunity to
work on this project and in the Speech Communication Lab, and for his
infinite patience during the writing of this thesis.

I would also like to express my gratitude to those who donated
their voices to the speech database, and especially to those helped
with arrangements for speech database collection:

Ms. Sarah Balcom
Ms. Joyce Kiser
The Zahorian Family
Ms. Diedre Henriques
Mr. Thomas Hudgins
Ms. Arlene Ingram

Finally, I would like to acknowledge the National Science Foundation
who partially supported this work under grant# BES 9977260
vi
TABLE OF CONTENTS
Page
LIST OF TABLES..................................................... viii
LIST OF FIGURES...................................................... ix
LIST OF EQUATIONS.................................................... xi
INTRODUCTION TO VATA.................................................. 1
PURPOSE......................................................... 1
DESCRIPTION..................................................... 1
SYSTEM OPERATION OVERVIEW....................................... 4
OVERVIEW OF PROCESSING STEPS.................................... 5
DATA MANAGEMENT AND REAL TIME OPERATIONAL CONSIDERATIONS....... 10
THESIS GOALS AND SUMMARY....................................... 11
SPEECH DATABASE COLLECTION........................................... 13
INTRODUCTION................................................... 13
DESCRIPTION OF RECORDING EQUIPMENT............................. 16
DATA REVIEW AND EVALUATION..................................... 18
NOISE AND FEATURE STABILITY.......................................... 20
BACKGROUND..................................................... 20
SOUND CARDS AS A POSSIBLE SOURCE OF NOISE...................... 21
TEST SIGNALS................................................... 22
SOUND CARD TESTING............................................. 24
SOUND CARD FREQUENCY RESPONSE AND NOISE TESTS.................. 25
SPECTRAL AND FEATURE STABILITY TESTS WITH TEST SQUARE WAVES.... 34
SUMMARY........................................................ 48
MAXIMUM LIKELIHOOD CLASSIFIER........................................ 50
INTRODUCTION................................................... 50
GENERAL DESCRIPTION OF THE MLC................................. 51
MLC IN THE VATA SYSTEM......................................... 53
PERFORMANCE EVALUATION............................................... 57
INTRODUCTION................................................... 57
TRAINING DATABASE AND HIDDEN LAYER SIZE VARIATION.............. 57
SAMPLING RATE EFFECTS.......................................... 63
LOW-PASS AND HIGH-PASS FILTERING EFFECTS....................... 65
SYSTEM TRAINING RESULTS........................................ 67
SUMMARY........................................................ 68
CONCLUSIONS.......................................................... 69
REFERENCES........................................................... 71
APPENDIX I: DESCRIPTION OF CURRENT VATA SYSTEM....................... 73
BRIEF HISTORY OF VATA.......................................... 73
NEWEST FEATURES................................................ 73
APPENDIX II: DATABASE SUMMARY........................................ 77
ORGANIZATION................................................... 77
vii
APPENDIX III: VATA FILES SUMMARY..................................... 82
VITA................................................................. 83
viii
LIST OF TABLES
Table Page

1. Major programs comprising the VATA system.......................... 5
2. Breakdown of speech database totals by speaker group and recording
type.............................................................. 16
3. Computers and sound cards used to record speech samples for database
................................................................. 17
4. Computers and sound cards used for sound card testing............. 24
5. Training and test recognition rates for different combinations of
11kHz and 22kHz sampled data...................................... 65
6. Training recognition rates when all available data is used for
training.......................................................... 67
7. Vowel/CVC database summary (as of Ver. 2.1)....................... 79
ix
LIST OF FIGURES
Figure Page

1. Bargraph display response for correct pronunciation of /ee/........ 2
2. Ellipse display response for correct pronunciation of /ee/......... 3
3. VATA block diagram and signal processing steps..................... 6
4. Feature calculation detail......................................... 8
5. Expected neural network recognition rate trend as a function of
training data set size............................................ 14
6. Observed recognition rate behavior as a function of training set
size for a small training database................................ 14
7. Setup for sound card tests........................................ 26
8. Average noise spectra of soundcards, no input signal.............. 29
9. Average frequency spectra of sound cards in soundproof room,
microphone on..................................................... 30
10. Average frequency spectra of sound cards in laboratory, microphone
on................................................................ 31
11. Frequency response of sound cards to white noise signal (line
input)............................................................ 33
12. Frequency response of sound cards to white noise signal (mic input)
................................................................. 34
13. Spectral and feature stability of synchronous and asynchronous
square waves, direct signal processing (no sound cards)........... 36
14. Spectral and feature stability of asynchronous square wave signal
using sound cards................................................. 37
15. Spectral and feature stability for male speaker /ah/, /ee/ and /ue/
tokens, bypassing soundcard....................................... 40
16. Spectral and feature stability for female speaker /ah/, /ee/ and
/ue/ tokens, bypassing soundcard.................................. 41
17. Spectral and feature stability of female speaker /ah/ token using
two sound cards................................................... 42
18. Spectral and feature stability of male speaker /ah/ token using two
sound cards....................................................... 43
19. Spectral and feature stability of female speaker /ee/ token using
two sound cards................................................... 44
x
20. Spectral and feature stability of male speaker /ee/ token using two
sound cards....................................................... 45
21. Spectral and feature stability of female speaker /ue/ token using
two sound cards................................................... 46
22. Spectral and feature stability of male speaker /ue/ token using two
sound cards....................................................... 47
23. False acceptance/rejection rates as a function of decision threshold
value α, male speaker case........................................ 55
24. False acceptance/rejection rates as a function of decision threshold
value α, female speaker case...................................... 55
25. False acceptance/rejection rates as a function of decision threshold
value α, combined male/female speaker case........................ 56
26. Recognition rate vs. training set size for neural networks with 5,
25 and 100 nodes in hidden the layer (male speakers).............. 60
27. Recognition rate vs. training set size for neural networks with 5,
25 and 100 nodes in hidden the layer (female speakers)............ 61
28. Recognition rate vs. training set size for neural networks with 5,
25 and 100 nodes in hidden the layer (male and female speakers)... 62
29. Recognition rate for lowpass filtered speech as the bandwidth
increases (lowpass cutoff frequency increases).................... 66
30. Recognition rate for highpass filtered speech as the bandwidth
decreases (highpass cutoff frequency increases)................... 67