Table Of ContentSpringer Series in Information Sciences 3
Editor: Manfred R. Schroeder
Springer Series in Information Sciences
Editors: King-sun Fu Thomas S. Huang Manfred R. Schroeder
Volume I Content-Addressable Memories
By T. Kohonen
Volume 2 Fast Fourier Transform and Convolution Algorithms
By H.J. Nussbaumer 2nd Edition
Volume 3 Pitch Determination of Speech Signals Algorithms and Devices
By W. Hess
Volume 4 Pattern Analysis
By H. Niemann
Volume 5 Image Sequence Analysis
Editor: T. S. Huang
Volume 6 Picture Engineering
Editors: King-sun Fu and T. L. Kunii
Volume 7 Number Theory in Science and Communication
With Applications in Cryptography, Physics, Biology
and Digital Information
By M. R. Schroeder
Volume 8 Self-Organization and Associative Memory
By T. Kohonen
Volume 9 An Introduction to Digital Picture Processing
By L.P. Yaroslavsky
Volume 10 Probability, Statistical Optics, and Data Testing
A Problem Solving Approach
By B. Roy Frieden
Volume 11 Physical and Biological Processing of Images
Editors: O. J. Braddick and A. C. Sleigh
Wolfgang Hess
Pitch Determination
of Speech Signals
Algorithms and Devices
With 254 Figures
Springer- ¥erlag
Berlin Heidelberg New York Tokyo 1983
Dr.-Ing.-habil. Wolfgang Hess
Lehrstuhl flir Datenverarbeitung im Institut flir Informationstechnik,
Technische Universitat Miinchen, ArcisstraBe 21,
D-8000 Miinchen 2, Fed. Rep. of Germany
Series Editors:
Professor King-sun Fu
School of Electrical Engineering, Purdue University,
West Lafayette, IN 47907, USA
Professor Thomas S. Huang
Department of Electrical Engineering and Coordinated Science Laboratory,
University of lllinois, Urbana, IL 61801, USA
Professor Dr. Manfred R Schroeder
Drittes Physikalisches Institut, Universitat Gottingen, BiirgerstraBe 42-44,
D-3400 Gottingen, Fed. Rep. of Germany
ISBN-13:978-3-642-81928-5 e-ISBN-13:978-3-642-81926-1
DOl: 10.1007/978-3-642-81926-1
Library of Congress Cataloging in Publication Data. Hess, W. (Wolfgang), 1940-. Pitch determination of speech signals.
(Springer series in information sciences; v. 3). Bibliography: p. Includes index. l. Speech processing systems. I. Title.
II. Series. TK7882.S65H47 1983 621.389'598 83-450
This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned,
specifically those of translation, reprinting, reuse of illustrations, broadcasting, reproduction by photocopying machine
or similar means, and storage in data banks. Under § 54 of the German Copyright Law where copies are made for
other than private use, a fee is payable to "Verwertungsgesellschaft Wort", Munich.
© by Springer-Verlag Berlin Heidelberg 1983
Softcover reprint of the hardcover 1st edition 1983
The use of registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific
statement, that such names are exempt from the relevant protective laws and regulations and therefore free for
general use.
2153/3130-543210
To my wife Helga
and to my children Patrizia and Eurydice
for their love, encouragement, and support
Preface
Pitch (i.e., fundamental frequency FO and fundamental period TO) occupies a
key position in the acoustic speech signal. The prosodic information of an
utterance is predominantly determined by this parameter. The ear is more
sensitive to changes of fundamental frequency than to changes of other speech
signal parameters by an order of magnitude. The quality of vocoded speech is
essentially influenced by the quality and faultlessness of the pitch measure
ment. Hence the importance of this parameter necessitates using good and
reliable measurement methods.
At first glance the task looks simple: one just has to detect the funda
mental frequency or period of a quasi-periodic signal. For a number of
reasons, however, the task of pitch determination has to be counted among
the most difficult problems in speech analysis.
1) In principle, speech is a nonstationary process; the momentary position
of the vocal tract may change abruptly at any time. This leads to drastic
variations in the temporal structure of the signal, even between subsequent
pitch periods, and assuming a quasi-periodic signal is often far from realistic.
2) Due to the flexibility of the human vocal tract and the wide variety of
voices, there exist a multitude of possible temporal structures. Narrow-band
formants at low harmonics (especially at the second or third harmonic) are an
additional source of difficulty.
3) For an arbitrary speech signal uttered by an unknown speaker, the
fundamental frequency can vary over a range of almost four octaves (50 to
800 Hz).
4) The glottal excitation signal itself is not always regular. Even under
normal conditions, i.e., when the voice is neither hoarse nor pathologic, the
glottal waveform exhibits occasional irregularities. In addition, the voice may
temporarily fall into vocal fry, which is a nonpathologic mode of voice
excitation with irregular intervals between subsequent glottal pulses.
5) Additional problems arise in speech communication systems, where the
signal is often distorted or band limited.
Literally hundreds of pitch-determination methods and algorithms have been
developed. None of them work perfectly for every voice, application, and
environmental condition. A great number of these methods, however, operate
rather well when the external conditions are specified more precisely.
VllI Preface
This book presents a detailed survey of the principal methods of pitch
determination as well as a discussion of individual solutions. The problem of
pitch determination is looked at in three different ways.
l) The problem is discussed from the point of view of the process of
speech communication. Speech is a means of human communication; it is an
acoustic signal generated by a human speaker in order to be perceived by a
human listener. We can thus look at the speech signal from the point of view
of speech production as well as of speech perception, or we can regard it as
an ordinary acoustic signal without taking into account its origin or destina
tion. In doing so, we arrive at different representations of the parameter
pitch, such as rate of vocal-fold vibration, elapsed time between adjacent
glottal pulses, fundamental frequency and period, or perceived (spectral or
virtual) pitch. For a stationary periodic signal all these definitions are uniquely
related; the speech signal, however, which is time variant, causes them to
diverge. Temporary failure of a particular method is thus often due to a
mismatch between the underlying definition of pitch and the signal. The
question of defining the parameter pitch constitutes the main criterion for
categorizing pitch determination methods; the primary categorization (Sect.2.6)
separates time-domain pitch determination methods (Chap.6), which process the
signal period by period, and short-term analysis methods (Chap.8), which deal
with the signal in a more global way. The various definitions of the parameter
pitch are explicitly formulated in Section 9.1.1. To better understand how
pitch is produced and how it is perceived, Chaps.3 and If provide a short
introduction to the mechanisms of the voice source and to pitch perception.
2) The problem is discussed from the point of view of the respective
application. There are four main areas of application for pitch determination
algorithms: a) speech communication; b) phonetics, linguistics, and musicology;
c) education; and d) phoniatrics and medicine. With respect to the problem of
pitch determination this means that the requirements on the respective algo
rithms are rather diverse, if not contradictory. The problem is briefly discus
sed in Sect. 1.2; a more detailed discussion follows in Section 9.1f. Prospective
readers of the present book, if they come from all these areas of application,
thus have rather different scientific backgrounds. On the other hand, describing
all the methods of pitch determination in a unified way can only be done in
terms of digital signal processiTlfT. A great number of algorithms cannot run
other than digitally; however, any method realized in analog technology can
be implemented in a digital way as wei!. Hence this book contains a short
tutorial introduction to digital signal processing (Chap.2); more details on
short-term analysis and spectral transformations are added later (Sect.8.l).
3) The problem is discussed from a functional point of view. Each pitch
determination algorithm is subdivided into the three functional blocks prepro
cessor, basic extractor, and postprocessor (Sect.2.6). For each of these blocks
there exist a number of specific methods which may improve the performance
when they are combined, but which may also be mutually exclusive. Arranging
the pitch determination methods according to this criterion forms an alterna
tive to categorization according to l) and is discussed in Section 9.1.
Preface IX
Voicing determination is a necessary prerequisite for pitch determination,
i.e., it is necessary to locate the segments in the signal where pitch is
present. Although the tasks of voicing and pitch determination can be combined
in one algorithm, they are two separate tasks. Voicing determination is not
the main topic of this book; it is reviewed in Section 5.3.
A survey of instrumental methods of pitch determination, which derive the
information on pitch from the glottal excitation signal without analyzing the
speech signal, completes the text (Sect.5.2). These methods have found wide
application in phoniatrics; yet they can be used in other areas of application
as well.
This book is the outgrowth of a thesis written in order to obtain the venia
legendi in digital signal processing at the Technical University of Munich. I am
indebted to the thesis supervisors, Professor T. Einsele and Professor E. Ter
hardt, for their careful review and constructive criticism of this work. My
special thanks go to Professor Terhardt for his valuable suggestions with
respect to the survey of frequency-domain PDAs (Sect.8.5) and the review of
pitch perception theories (Chap.4). Other colleagues who have been involved
directly with the preparation of this book include Mr. H. Dettweiler who com
piled a part of the bibliography. Dr. M. Cartier and Dr. G. Mercier of CNET
(Lannion, France) as well as Dr. A. Knipper (Moscow) provided me with relevant
Soviet literature on speech processing. Mr. A. Bladon of Oxford University and
Dr. J. Holmes of JSRU (Cheltenham, England) reviewed several chapters and
gave numerous valuable comments especially on Chapter 2. The help and
support of all these colleagues is gratefully acknowledged.
The layout of this book was prepared by means of a text processing system
and a graphic software package developed at the author's laboratory. I wish to
thank Mrs. I. Kunert who edited most of the text. I am also indebted to
Dr. H. Lotsch, Mr. R. Michels, and Dr. M. Wilson of Springer-Verlag; without
their help and patience many of my intentions would not have been realized.
Munich, March 1983 Wolfgang Hess
Contents
1. Introduction ••.••••••••••.•.•••••••••••.•.•.•••••••.•••••••.•..•••••••••••
1.1 Voice Source Parameter Measurement and the Speech Signal .•••••.•••••• 1
1.2 A Short Look at the Areas of Application •••••••.•••••••••••••••••..••• 3
1.3 Organization of the Book ••••••••••••••.•••....•.••.••••••••••••••••••• 6
2. Basic Terminology. A Short Introduction to Digital Signal Processing •••••••.•. 8
2.1 The Simplified Model of Speech Excitation ••••.••.••••••••••.•.••••••••• 8
2.2 Digital Signal Processing 1: Signal Representation........................ 13
2.3 Digital Signal Processing 2: Filters ••••••••••.••••••••••••••••••••••..•• 18
2.4 Time-Variant Systems. The Principle of Short-Term Analysis.............. 24
2.5 Definition of the Task. The Linear Model of Speech Production •••••••••.. 31
2.6 A First Categorization of Pitch Determination Algorithms (PDAs).......... 34
3. The Human Voice Source .••..•.••••••••••••••••••••.•.•••.•••••••••••••••• 38
3.1 Mechanism of Sound Generation at the Larynx........................... 38
3.2 Operational Modes of the Larynx. Registers ••••••.•••••••••••••••••.•••• 48
3.3 The Glottal Source (Excitation) Signal .................................. 51
3.4 The Influence of the Vocal Tract Upon Voice Source Parameters •.••..••• 55
3.5 The Voiceless and the Transient Sources ••••••••••••••••••••••••..•..•.• 59
4. Measuring Range, Accuracy, Pitch Perception ••...••••••••••••••••••••.••••• 63
4.1 The Range of Fundamental Frequency................................... 64
4.2 Pitch Perception. Toward a Redefinition of the Task .••.••••••••.•••••••• 66
1f.2.1 Pitch Perception: Spectral and Virtual Pitch....................... 66
4.2.2 Toward a Redefinition of the Task ............................... 76
4.2.3 Difference Limens for Fundamental-Frequency Change • • • • • • • • • • • • • • 78
4.3 Measurement Accuracy .••••.•••••••••••••.•••••••••••••....•..•••••••• 83
4.4 Representation of the Pitch Information in the Signal •.•.•••••••••••••••• 88
4.5 Calibration and Performance Evaluation of a PDA •••••••••.•.••••••••••• 90
5. Manual and Instrumental Pitch Determination, Voicing Determination •.•.•••••• 92
5.1 Manual Pitch Determination .•...••••••••••.••.•••••••••.••••.•••••••••• 93
5.1.1 Time-Domain Manual Pitch Determination •.••••••••.•.••••••.••••• 94
5.1.2 Frequency-Domain Manual Pitch Determination..................... 98
5.2 Pitch Determination Instruments (PDIs) •••••.•••....••••••..••..••••••••. 103
5.2.1 Clinical Methods for Larynx Inspection •••••••••••••••••••.•••••••• 105
5.2.2 Mechanic PDls ••••••••••••••••••••••••••••.•••••••••••.•.••••••• 107
5.2.3 Electric PDls •••••••.•.••••••••••••••••.•••••••••••••••.•••••••• 116
5.2.4 Ultrasonic PDls •••••••••••••••••••••••••..••..••.•••••••••••••.• 124
5.2.5 Photoelectric PDIs (Transillumination of the Glottis) ••••••••••••••• 127
5.2.6 Comparative Evaluation of PDIs •••••••...•••••.•..••••••••••••••• 129
xu Contents
5.3 Voicing Determination - Selected Examples.............................. 133
5.3.1 Voicing Determination: Parameters ••••••••••••••••••••••••••••••• 136
5.3.2 Voicing Determination - Simple Voicing Determination Algo-
rithms (VDAs); Combined VDA-PDA Systems •••••••••••••••••••••• 138
5.3.3 Multiparameter VDAs. Voicing Determination by Means of Pat-
tern Recognition Methods........................................ 144
5.3.4 Summary and Conclusions •••••••••••••••••••••••••••••••••••••••• 151
6. Time-Domain Pitch Determination .......................................... 152
6.1 Pitch Determination by Fundamental-Harmonic Extraction................. 154
6.1.1 The Basic Extractor ...... ............ ........................... 156
6.1.2 The Simplest Pitch Determination Device - Low-Pass Filter and
Zero (or Threshold) Crossings Analysis Basic Extractor •••••••••••••• -162
6.1.3 Enhancement of the First Harmonic by Nonlinear Means ••••••••••• 166
6.1.4 Manual Preset and Tunable (Adaptive) Filters...................... 173
6.2 The Other Extreme - Temporal Structure Analysis ....................... 181
6.2.1 Envelope Modeling - the Analog Approach......................... 183
6.2.2 Simple Peak Detector and Global Correction •••••••••••••••••••••• 198
6.2.3 Zero Crossings and Excursion Cycles •••••••••••••••••••••••••••••• 202
6.2.4 Mixed-Feature Algorithms •••••••••••••••••••••••••••••••••••••••• 210
6.2.5 Other PDAs That Investigate the Temporal Structure of the
Signal.......................................................... 216
6.3 The Intermediate Device: Temporal Structure Transformation and Sim-
plification •••••••••••••••••••••••••••••••••••••••••••••••••••••••••••• 220
6.3.1 Temporal Structure Simplification by Inverse Filtering.............. 221
6.3.2 The Discontinuity in the Excitation Signal: Event Detection......... 230
6.4 Parallel Processing in Fundamental Period Determination. Multichannel
PDAs ••••••••••••••• •••••••••••••••••••• •••••••• ••••••••••••••• •••••• 242
6.4.1 PDAs with Multichannel Preprocessor Filters •••••••••••••••••••••• 243
6.4.2 PDAs with Several Channels Applying Different Extraction Prin-
ciples .......................................................... 251
6.5 Special-Purpose (High-Accuracy) Time-Domain PDAs •••••••••••••••••••••• 254
6.5.1 Glottal Inverse Filtering ••••••••••••••••••••••••••••••••••••••••• 255
6.5.2 Determining the Instant of Glottal Closure •••••••••••••••••••••••• 269
6.6 The Postprocessor ••••••••••••••••••••••••••••••••••••••••••••••••••••• 277
6.6.1 Time-to-Frequency Conversion; Display............................ 277
6.6.2 FO Determination With Basic Extractor Omitted ••••••••••••••••••• 281
6.6.3 Global Error Correction Routines ••••••••••••••••••••••••••••••••• 284
6.6.4 Smoothing Pitch Contours • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 295
6.7 Final Comments ••••••• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 298
7. Design and Implementation of a TIme-Domain PDA for Undistorted and
Band-Limited Signals ••••••••••••••••••••••- ................................ 302
7.1 The Linear Algorithm .................................................. 302
7.1.1 Prefiltering ..................................................... 303
7.1.2 Measurement and Suppression of Fl .............................. 303
7.1.3 The Basic Extractor.... ............. ............................ 304
7.1.4 Problems with the Formant F2. Implementation of a Multiple
Two-Pulse Filter (TPF) •••••••••••••••••••••••••••••••••••••••••• 307
7.1.5 Phase Relations and Starting Point of the Period .................. 308
7.1.6 Performance of the Algorithm with Respect to Linear Distor-
tions, Especially to Band Limitations .............................. 309
7.2 Band-Limited Signals in Time-Domain PDAs .............................. 310
7.2.1 Concept of the Universal PDA ............................ ....... 311
7.2.2 Once More: Use of Nonlinear Distortion in Time-Domain PDAs •••• 312
