Table Of ContentDATA-DRIVEN TECHNIQUES IN SPEECH SYNTHESIS
Data-Driven Techniques
in Speech Synthesis
Edited by
R.I. Damper
University ofS outhampton
......
"
SPRINGER-SCIENCE+BUSINESS MEDIA, B.V.
A C.I.P. Catalogue record for this book is available from the Library of Congress.
ISBN 978-1-4419-4733-8 ISBN 978-1-4757-3413-3 (eBook)
DOI 10.1007/978-1-4757-3413-3
Printed on acid-free paper
AlI Rights Reserved
© 2001 Springer Science+Business Media Dordrecht
Originally published by Kluwer Academic Publishers in 2001
Softcover reprint ofthe hardcover Ist edition 2001
No part of the material protected by this copyright notice may be reproduced or
utilized in any form or by any means, electronic or mechanical,
including photocopying, recording or by any information storage and
retrieval system, without written permission from the copyright owner.
This book is dedicated to the memories of Frank Fallside (1931-1993) and
John Holmes (1929-1999), two pioneers of the science and technology of
speech synthesis.
Contents
Preface xiii
List of Contributors xv
Learning About Speech from Data: Beyond NETtalk
Robert I. Damper
1.1 Introduction 2
1.2 Architecture of a TIS System 4
1.3 Automatic Pronunciation Generation 6
1.3.1 The Nature of the Problem 8
1.3.2 Rules: The Traditional Solution 9
1.3.3 Letter-Phoneme Alignment 11
1.3.4 NETtalk: A Data-Driven Solution 11
Input coding 12
Output coding 13
Hidden units 13
Performance 13
Discussion 14
1.3.5 NETSpeak-a Re-implementation of NETtalk 15
Input and output codings 15
Generalisation 16
Common versus uncommon words 16
1.3.6 Pronunciation by Analogy 16
1.3.7 Evaluation 19
1.4 Prosody 21
1.5 The Synthesis Module 23
1.6 Conclusion 25
2
Constructing High-Accuracy Letter-to-Phoneme Rules with Machine 27
Learning
Ghulum Bakiri and Thomas G. Dietterich
2.1 Introduction 27
2.2 The NETtalk Approach 28
2.2.1 The NETtalk Dictionary 28
2.2.2 Inputs and Outputs 30
2.2.3 Training and Classification 31
2.3 High-Performance ML Approach 31
2.3.1 Learning Algorithm 32
2.3.2 Output Representation 34
2.3.3 Input Representation 37
vii
viii
2.4 Evaluation of Pronunciations 40
2.4.1 Methods 40
2.4.2 Results 41
2.4.3 Discussion 43
2.5 Conclusions 44
3
Analogy, the Corpus and Pronunciation 45
Kirk P. H. Sullivan
3.1 Introduction 45
3.2 Why Adopt a Psychological Approach? 47
3.3 The Corpus as a Resource 49
3.4 The Sullivan and Damper Model 50
3.4.1 Analogy in the Phonemic Domain 51
Flexible GPC rules 52
Phoneme-lattice matching 52
Ranking and thresholding 54
3.4.2 Analogy in the Orthographic Domain 54
3.5 Parallels with Optimality Theory 55
3.6 Implementation 56
3.6.1 Scoring Pronunciations 56
Method 1 57
Method 2 58
Confidence ratings 59
3.6.2 Treatment of Word Boundaries 59
3.7 Corpora 59
3.7.1 Corpus Construction 59
3.7.2 The Deep/Shallow Continuum 60
3.8 Performance Evaluation 61
3.8.1 English 62
3.8.2 German 64
3.8.3 Maori 65
3.8.4 Impact of Implementational Choices 66
Preference values 66
Treatment of word boundaries 66
Corpus construction 67
Position on the deep/shallow continuum 67
3.9 Future Challenges 68
4
A Hierarchical Lexical Representation for Pronunciation Generation 71
Helen Meng
4.1 Introduction 71
4.2 Previous Work 72
4.3 Hierarchical Lexical Representation 74
4.4 Generation Algorithm 75
4.4.1 Training Procedure 76
4.4.2 Testing Procedure 79
4.4.3 Efficient Search Algorithm 80
Contents ix
4.5 Evaluation Criteria 81
4.6 Results on Letter-to-Sound Generation 83
4.7 Error Analyses 83
4.8 Evaluating the Hierarchical Representation 85
4.8.1 Investigating the Hierarchy 85
4.8.2 Comparison with a Single-Layer Approach 87
4.9 Discussions and Future Work 89
5
English Letter-Phoneme Conversion by Stochastic Transducers 91
Robert W. P. Luk and Roben I. Damper
5.1 Introduction 91
5.2 Modelling Transduction 93
5.2.1 History 93
5.2.2 Stochastic Grammars and Translation 95
Permutational equivalence 95
Alignment and translation as derivation 96
Automata for alignment and translation 97
Stochastic alignment and translation 99
5.3 Stochastic Finite-State Transducers 100
5.3.1 Transducer Topology 100
5.3.2 Statistical Modelling 101
5.4 Inference of Letter-Phoneme Correspondences 103
5.4.1 Stochastic Alignment 104
5.4.2 Diagonal Movement Segmentation 105
5.4.3 Projection Segmentation 107
5.5 Translation 108
5.5.1 Determinisation 109
5.5.2 Computing Translation and Alignment 111
5.5.3 Efficiency of Transduction 113
Computational complexity 113
Measurement model 114
5.6 Results 116
5.6.1 Performance Measures 118
5.6.2 Training and Test Data 118
5.6.3 Performance Comparison 119
5.6.4 Errors 121
5.7 Conclusions 122
6
Selection of Multiphone Synthesis Units and Grapheme-to-Phoneme 125
Transcription using Variable-Length Modeling of Strings
Sabine Deligne and Franfois Yvon and Frederic Bimbot
6.1 Introduction 126
6.2 Multigram Model 127
6.2.1 Formulation of the Model 127
6.2.2 Estimation of Multigram Parameters 128
6.2.3 Joint Multigram Model 130
Formulation of the model 130
x
Estimation of the joint multigram parameters 131
6.3 Multiphone Units for Speech Synthesis 131
6.3.1 Some Issues in Concatenative Synthesis 131
6.3.2 Application of the n-Multigram Model 133
6.3.3 Experiments on the Malecot Database 133
Evaluation protocol 134
Results 134
6.3.4 Comments and Perspectives 137
6.4 Learning Letter-to-Sound Correspondences 137
6.4.1 Background 137
6.4.2 Adaptation of the Multigram Model 139
6.4.3 Experiments on the BDLEX Database 140
Evaluation protocol 140
Results 141
Decoding with a syntagmatic model 143
6.4.4 Discussion 144
6.5 General Discussion and Perspectives 146
7
TREETALK: Memory-Based Word Phonemisation 149
Walter Daelemans and Antal van den Bosch
7.1 Introduction 149
7.2 Memory-Based Phonemisation 151
7.2.1 Phonemisation with IGTREE 153
Automatic alignment 154
Lexicon compression 155
Generalising from the compressed lexicon 156
7.2.2 Experiments with I G TREE 158
7.3 TRIBL and TREETALK 161
7.3.1 The TRIBL Learning Method 161
7.3.2 Experiments with TREETALK 162
7.4 Modularity and Linguistic Representations 164
7.4.1 Experiments 166
M-A-G-Y-S 166
M-G-S 168
7.4.2 Comparing the Modular Systems 169
7.5 Conclusion 172
8
Learnable Phonetic Representations in a Connectionist TIS System - 175
I: Text to Phonetics
Andrew D. Cohen
8.1 Introduction 175
8.2 Problem Background 178
8.3 Data Inputs and Outputs to Module Ml 180
8.3.1 Dictionary 180
8.3.2 Encoding of Inputs and Outputs 181
8.4 Detailed Architecture of the Text-to-Phonetics Module 182
Contents xi
8.4.1 SOMNetwork 183
8.4.2 MLPNetwork 183
8.4.3 AANetwork 183
8.5 Model Selection 184
8.5.1 Bias and Variance 184
8.5.2 Generalisation and Error Measure 186
8.5.3 Multiple Networks 187
8.5.4 Training the AA Network 189
8.5.5 Optimal Number of Correction Cycles 189
8.6 Results 191
8.6.1 Results on the Individual Networks 191
8.6.2 Results with Multiple Networks 192
8.7 Conclusions and Further Work 196
9
Using the Tilt Intonation Model: A Data-Driven Approach 199
Alan W. Black and Kurt E. DusterhoJf and Paul A. Taylor
9.1 Background 199
9.2 Tilt Intonation Model 200
9.2.1 Intonational Events 200
9.2.2 Event Classification 201
Locating events 201
RFC analysis 201
Tilt analysis 202
9.2.3 Intonational Events 203
Full event formalism 203
Segmental formalism 204
9.2.4 Generation of fo Contours 204
9.2.5 Key Features of Tilt 205
9.3 Training Tilt Models 205
9.3.1 Speech Database 205
9.3.2 Extracting Training Data 206
9.3.3 CART Models 207
9.3.4 Measure of Accuracy 208
9.4 Experiments and Results 208
9.4.1 Three Experiments 209
Experiment 1 209
Experiment 2 209
Experiment 3 210
9.4.2 Comparison and Discussion of Results 210
9.4.3 Context Features Revisited 212
9.5 Conclusion 213
10
Estimation of Parameters for the Klatt Synthesizer from a Speech Database 215
John Coleman and Andrew Slater
10.1 Introduction 215
10.2 Global Parameter Settings 219
10.2.1 Gain 220
xii
10.2.2 Number of Formants 220
10.2.3 Cascade-Paral1ellAll-Parallel Configuration 220
10.2.4 Sampling Rate 220
10.2.5 Frame Rate 221
10.3 Synthesis of Vowels, Diphthongs and Glides 221
10.3.1 Estimation of Parameters 221
10.3.2 Synthesis of Vocoids 223
10.4 Stop Consonants (and Voiceless Vowels) 225
10.4.1 Estimation of Noise Source Parameters 225
10.4.2 Estimation of Resonator Amplitudes 228
10.4.3 Observations on the Time-Course of Stop Consonants 230
10.5 Estimation of Fricative Parameters 231
10.5.1 Estimation of Noise Source Amplitudes 232
10.5.2 Observations on the Time-Course of Fricatives 234
10.6 Other Sounds 235
10.6.1 Affricates 235
10.6.2 Nasals and Liquids 235
10.7 Application: A Database of English Monosyllables 236
10.7.1 Recordings 237
10.7.2 Parameter Prototypes 237
10.8 Conclusion 238
11
Training Accent and Phrasing Assignment on Large Corpora 239
Julia Hirschberg
11.1 Introduction 239
11.2 Intonational Model 241
11.3 Classification and Regression Trees 241
11.4 Predicting Pitch Accent Placement 243
11.4.1 Speech Corpora for Testing and Training 245
11.4.2 Variables for Predicting Accent Location 246
11.4.3 The Rule-Based System 250
11.4.4 Automatic Classification of Pitch Accent 256
11.4.5 Discussion of Pitch Accent Prediction 263
11.5 Predicting Phrase Boundary Location 264
11.5.1 Phrasing Prediction from Labeled Speech Corpora 265
11.5.2 Phrasing Prediction Trained on Labeled Text 271
11.6 Conclusion 273
12
Learnable Phonetic Representations in a Connectionist TTS System - 275
II: Phonetics to Speech
Andrew D. Cohen
12.1 Introduction 275
12.2 Architecture of Phonetics-to-Speech Module 277
12.3 Training and Alignment 278
12.4 Phonetics-to-Speech Results 280
12.5 Conclusions and Further Work 281
