Table Of ContentM obile Speech and Advanced Natural
Language Solutions
A my N eustein ● J udith A. M arkowitz
E ditors
M obile Speech and
Advanced Natural
Language Solutions
E ditors
A my N eustein J udith A. M arkowitz
L ingustic Technology Systems J . Markowitz Consultants
F ort Lee, N J, USA C hicago, IL, U SA
I SBN 978-1-4614-6017-6 I SBN 978-1-4614-6018-3 (eBook)
D OI 10.1007/978-1-4614-6018-3
S pringer New York Heidelberg Dordrecht London
L ibrary of Congress Control Number: 2012954940
© Springer Science+Business Media New York 2 013
T his work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifi c ally the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfi l ms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection
with reviews or scholarly analysis or material supplied specifi c ally for the purpose of being entered and
executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this
publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s
location, in its current version, and permission for use must always be obtained from Springer. Permissions
for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to
prosecution under the respective Copyright Law.
T he use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specifi c statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
W hile the advice and information in this book are believed to be true and accurate at the date of
publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for
any errors or omissions that may be made. The publisher makes no warranty, express or implied, with
respect to the material contained herein.
P rinted on acid-free paper
S pringer is part of Springer Science+Business Media (www.springer.com)
P reface
M obile Speech and Advanced Natural Language Solutions provides a comprehensive
and forward-looking treatment of natural speech in the mobile environment. This
14-chapter anthology brings together lead scientists from Apple, Google, IBM,
AT&T, Yahoo! Research, and other companies, along with academicians, technol-
ogy developers, and market analysts. They analyze the growing markets for mobile
speech, new methodological approaches to the study of natural language, empirical
research fi n dings on natural language and mobility, and future trends in mobile
speech.
T his book is divided into four sections.
T he fi r st section explores the growing markets for mobile speech. It opens
with a challenge to the industry to broaden the discussion about speech in mobile
environments. Looking at speech recognition in factories and warehouses, moni-
toring for home-incarcerated and community-release offenders using speaker
verifi c ation, and speech-enabled robots, this chapter dispels any misconception
that mobile speech is limited to the smartphone. The chapter is followed by a
fascinating look at the evolving role of speech in consumer electronics. It
describes the evolution of speech in mobile devices and envisions the household
products that can respond to voice commands as naturally as they would to “the
fl i ck of a switch.” The section is rounded out by a progressive look at how today’s
automated personal assistant can coalesce into a personal-assistant model that
serves as “the primary user-interface modality” for providing the user with a
“unifi e d technology experience.”
T he second section describes, analyzes, and dissects the methodologies used for
natural-language processing. It proposes innovative methods that would enable
mobile personal assistants and other speech-based mobile technology to better
understand both text-based and spoken language input.
T he section begins with a detailed history of natural language understanding for
both speech and text. That chapter also provides a comparative analysis of the meth-
odologies for extracting meaning from human-language input. The following chap-
ter offers a new natural-language method for mining user-generated content in
mobile applications. The method locates and extracts valuable opinion-related
v
vi Preface
data buried in online postings—and makes such data available to mobile users. The
section concludes with an in-depth analysis of methodological approaches to
machine translation, with an eye toward improvements needed to enable mobile
users to benefi t from multilingual communication.
T he third section is devoted to empirical studies of natural-language technology
for mobile devices. The section opens with an illuminating analysis of two ground-
ing frameworks for designing mobile interfaces capable of engaging in human-like
interaction with users. In doing so, it brings the dream of truly intelligent mobile
personal assistants one step closer to reality. The chapter is followed by an empirical
exploration of the g oogle.com query stream for mobile voice search. It proposes the
construction of “large-scale discriminative N-gram language models” as a means to
achieving signifi c ant gains in recognition performance.
T he next chapter focuses on how to improve noise robustness for mention detec-
tion (MD) in searching. It describes a multi-stage approach involving the augmenta-
tion of an existing statistical MD system. The approach also reduces false alarms
generated by error-fi l led “spurious passages” while, at the same time, maintaining
performance on clean input. The section concludes with a novel study of referential
practice in human interactions with search engines. It examines differences between
searches performed when the user knows the name of the entity for which they are
searching and when they do not.
T he fourth section is the coda to this book. It opens a window into some of the
most exciting new trends in mobile speech.
T he section begins with the presentation of an innovative approach to summariz-
ing opinion-related information for mobile devices. It describes two common tech-
niques—a graphical summarization and review summarization—and offers a hybrid
approach “which combines abstractive and extractive summarization methods to
extract relevant opinions and relative ratings from text documents.” The chapter that
follows shows the utility of natural speech for medical applications used by the US
military. It does so by adding Siri-like features to a VAMTA (Voice-Activated
Medical Tracking Application) that is designed to perform a spectrum of tasks for
military personnel, such as answering questions, making recommendations, and
delegating requests to a set of web services. The next chapter advocates for funda-
mental changes to speech synthesis designed to achieve truly human-like perfor-
mance. It addresses the merits of including cognitive neuroscience, music perception,
and the psychology of language acquisition, in a broadly based, multidisciplinary
approach to spoken-language output.
T he fi n al chapter provides an intriguing look at “super-natural” language dialogs
(SNLD), referring to user interfaces designed around s uper-human technology.
Among the technologies explored are text-to-speech synthesis “that can easily
exceed human capabilities—encompassing a wider pitch range, speaking faster or
slower, and/or pronouncing tongue-twisters…[or] non-speech audio [that] can pro-
vide prompts and feedback more quickly than speech, and can also exploit musical
syntax and semantics.”
T he editors have endeavored to make this book a defi n itive resource on mobile
speech for speech engineers, system developers, linguists, cognitive scientists, and
Preface vii
others interested in utilizing natural-language technology in diverse applications.
This compilation is predicated on the belief that mobile personal assistants, speech-
enabled consumer electronics, talking robots, and other speech-driven mobile
devices will forever change our lives for the better.
Fort Lee, NJ, USA Amy Neustein
Chicago, IL, USA Judith A. Markowitz
C ontents
Part I Growing Markets for Mobile Speech
1 Beyond SIRI: Exploring Spoken Language in Warehouse
Operations, Offender Monitoring and Robotics .................................. 3
Judith A. Markowitz
2 Speech’s Evolving Role in Consumer Electronics…From Toys
to Mobile .................................................................................................. 23
Todd Mozer
3 The Personal-Assistant Model: Unifying the
Technology Experience ........................................................................... 35
William Meisel
Part II Innovations in Natural Language Processing
4 Natural Language Processing: Past, Present and Future .................... 49
Deborah A. Dahl
5 Sequence Package Analysis: A New Natural Language Method
for Mining User-Generated Content for Mobile Uses ......................... 75
Amy Neustein
6 Getting Past the Language Gap: Innovations
in Machine Translation ........................................................................... 103
Rodolfo Delmonte
Part III Empirical Studies of Natural Language and Mobility
7 Natural Language Technology in Mobile Devices:
Two Grounding Frameworks ................................................................. 185
Jerome R. Bellegarda
ix
