Table Of ContentHuman Behavior
Learning and Transfer
Human Behavior
Learning and Transfer
Yangsheng Xu
The Chinese University of Hong Kong
Ka Keung C. Lee
The Chinese University of Hong Kong
Boca Raton London New York
A CRC title, part of the Taylor & Francis imprint, a member of the
Taylor & Francis Group, the academic division of T&F Informa plc.
Published in 2006 by
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2006 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group
No claim to original U.S. Government works
Printed in the United States of America on acid-free paper
10 9 8 7 6 5 4 3 2 1
International Standard Book Number-10: 0-8493-7783-8 (Hardcover)
International Standard Book Number-13: 978-0-8493-7783-9 (Hardcover)
Library of Congress Card Number 2005049923
This book contains information obtained from authentic and highly regarded sources. Reprinted material is
quoted with permission, and sources are indicated. A wide variety of references are listed. Reasonable efforts
have been made to publish reliable data and information, but the author and the publisher cannot assume
responsibility for the validity of all materials or for the consequences of their use.
No part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic,
mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and
recording, or in any information storage or retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.com
(http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC) 222 Rosewood Drive,
Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration
for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate
system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only
for identification and explanation without intent to infringe.
Library of Congress Cataloging-in-Publication Data
Xu, Yangsheng.
Human behavior learning and transfer / Yangsheng Xu, Ka Keung Lee.
p. cm.
ISBN 0-8493-7783-8
1. Learning, Psychology of. I. Lee, Ka Keung. II. Title.
BF318.X8 2006
153.1'54--dc22 2005049923
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
Taylor & Francis Group and the CRC Press Web site at
is the Academic Division of T&F Informa plc. http://www.crcpress.com
To Our Families
Preface
Overthepasttwodecades,rapidadvancesincomputerperformancehavenot
been matched with similar advances in the development of intelligent robots
and systems. Although humans are quite adept at mastering complex and
dynamic skills, we are far less impressive in formalizing our behavior into
algorithmic, machine-codable strategies. Therefore, it has been difficult to
duplicate the types of intelligent skills and actions we witness every day as
humans,inrobotsandothermachines. Thisnotonlylimitsthecapabilitiesof
individual robots,but also the extent to which humans and robots can safely
interact and cooperate with one another. Nevertheless, human actions are
currently our only examples of truly “intelligent” behavior. As such, there
exists a profound need to abstract human skill into computational models,
capable of realistic emulation of dynamic human behavior.
The main focus of this book is the modeling and transfer of human action
and reaction behaviors. For human reaction skills, we apply machine learn-
ing techniques and statistical analysis towards abstracting models of human
behavior. It is our contention that such models can be learnt efficiently to
emulate complex human control behaviors in the feedback loop. For human
actionskills,themethodspresentedherearebasedontechniquesforreducing
the dimensionality of data sets while preserving as much useful information
as possible.
The research in modeling human behaviors from human data or human
demonstration has thus far not addressed a number of important issues, in-
cludingmodelingofcontinuousanddiscontinuousbehaviors,modelvalidation,
model evaluation, model optimization, and efficient transfer of model chara-
teristics. Chapters 2 to 8 of this book will discuss various topics involved
in human reaction skill modeling. We propose and develop an efficient con-
tinuous learning framework for modeling human controlstrategy (HCS) that
combines cascade neural networks and node-decoupled extended Kalman fil-
tering. WethenapplycascadelearningtowardsabstractingHCSmodelsfrom
experimentalcontrolstrategydata(Chapter3). Next,wepresentastochastic,
discontinuousmodelingframeworkformodeling discontinuoushumancontrol
strategies. For validation of behavior model, we formulate a stochastic sim-
ilarity measure, based on hidden Markov model analysis that is capable of
comparing multi-dimensional stochastic control trajectories (Chapter 4).
For model evaluation, we propose and develop performance criteria based
on inherence analysis (Chapter 5). We also propose an iterative optimiza-
tion algorithm, based on simultaneous perturbed stochastic approximation
iii
iv Human Behavior Learning and Transfer
(SPSA), for improving the performance of learnt HCS models (Chapter 6).
InChapter7,wedeveloptwoalgorithmsforHCSmodeltransferring. Oneof
them is based onthe similarity measure and SPSA. The proposedalgorithms
allow us to develop useful apprentice models that nevertheless incorporate
some of the robust aspects of the expert HCS models. In Chapter 8, we will
present a case study in which human navigational skills are transferred to a
wheelchair.
In Chapters 9 to 14, issues related to human action skill modeling will
be presented. In Chapter 10, we will discuss global parametric modeling of
human performance data. Local modeling based on general purpose non-
parametric methods is the focus of Chapter 11.
Chapter 12 presents adaptations to these non-parametric methods for the
specific purpose of modeling human performance, in which we derive a s-
plinesmootherthatcanbuildbest-fittrajectoriesofhumanperformancedata
through phase space.
In Chapters 13 and 14, three case studies in human action modeling are
presented. We will model the actions of human on three different levels,
namely, facial actions, full-body actions, and walking trajectories.
We wish to acknowledge the contributors, who are the past students of
the first author, on whose work our presentation is partly based. Michael
Nechyba’s work laid the foundation of human reaction modeling and the re-
lated work are presented in Chapters 2 to 4. Jingyan Song developed the
methodology for the evaluation, optimization and trasfer of human reaction
behaviors, which form the contents in Chapters 5 to 7. The work presented
in Chapter 8 was mainly performed by Kyle H. N. Chow. Chapters 9 to 12,
which discuss the modeling of human action skills, come primarily from the
research of Christopher Lee. The first author would also like to thank his
students for their supportandfriendshipwhile he workedatCarnegieMellon
University and the Chinese University of Hong Kong.
Our book reviewers have contributed their advices in making critical sug-
gestions. We are also grateful for the effort of the editors and staff at Taylor
and Francis throughout the development of this book.
This book could not have happened without the support from our fund-
ing sources. The research and development work described in this book is
partially supported by the grants from the Research Grants Council of the
Hong Kong Special Administration Region (Projects No. CUHK4138/97E,
CUHK4164/98E, CUHK 4197/00E, CUHK4228/01E, CUHK4317/02E, and
CUHK4163/03E), and by the Hong Kong Innovation and Technology Fund
under grant ITS/140/01.
Yangsheng Xu and Ka Keung Lee
The Chinese University of Hong Kong Spring 2005
Authors
Yangsheng Xu receivedhis Ph.D. fromUniversity of Pennsylvaniain the
area of robotics in 1989. He has been with the Department of Automation
and Computer-Aided Engineering at The Chinese University of Hong Kong
(CUHK) since 1997, and served as department chairman from 1997 to 2004.
Prof. Xu is currently a chair professor in the department and he was a fac-
ulty member at the School of Computer Science, Carnegie Mellon University
(CMU) from 1989 to 1999.
Prof. Xu’s research interests have been in robotics and human interface,
and their applications in service, aerospace, and industry. At first he worked
on designing and controling robots for space operations. He also made con-
tributions in human control strategy modeling and applications in real-time
control. Hismorerecenteffortshavebeenconcentratedonwearableinterface,
intelligent surveillance, and future space systems.
Hehasbeenaprincipalinvestigatorinmorethan30projectsfundedbyboth
governments and industries. Based on his research work, he has fortunately
published over70 papers in journals,130 papers in internationalconferences,
and several book contributions and books. He has been serving or served on
advisory boards or panels in various government agencies and industries in
the United States, Japan, Korea, Hong Kong, and mainland China. He is a
fellow of IEEE, HKIE, and IEAS.
v
