Learning Issues for Intelligent Tutoring Systems

325 Pages·1988·6.275 MB·English

by  Allan Collins, John Seely Brown (auth.), Dr. Heinz Mandl, Dr. Alan Lesgold (eds.)

Cognitive Science
Series Editors:
Marc M. Sebrechts
Gerhard Fischer
Peter M. Fischer
Heinz Mandl Alan Lesgold
Editors
Learning Issues
for Intelligent
Tutoring Systems
With 57 Figures
Springer-Verlag
New York Berlin Heidelberg
London Paris Tokyo
Dr. Heinz Mandl Dr. Alan Lesgold
Deutsches Institut fUr Fernstudien Learning Research and
an der Universitat Tiibingen Development Center
7400 Tiibingen University of Pittsburgh
Federal Republic of Germany Pittsburgh, P A 15260, USA
Library of Congress Cataloging-in-Publication Data
Learning issues for intelligent tutoring systems.
(Cognitive science)
Includes bibliographies and index.
1. Intelligent tutoring systems. I. Mandl, Heinz.
II. Lesgold, Alan M. III. Series.
LBW28.5.L388 1988 371.3'9445 87-23449
© 1988 by Springer-Verlag New York Inc.
Copyright not claimed for chapters by U.S. government employees.
All rights reserved. This work may not be translated or copied in whole or in part without the
written permission of the publisher (Springer-Verlag, 175 Fifth Avenue, New York, NY
10010, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use
in connection with any form of information storage and retrieval, electronic adaptation,
computer software, or by similar or dissimilar methodology now known or hereafter de
veloped is forbidden.
The use of general descriptive names, trade names, trademarks, etc. in this publication,
even if the former are not especially identified, is not to be taken as a sign that such names,
as understood by the Trade Marks and Merchandise Marks Act, may accordingly be used
freely by anyone.
Typeset by Best-set Typesetter Ltd., Hong Kong
9 8 7 6 5 432 1
ISBN-13: 978-0-387-96616-8 e-ISBN-13: 978-1-4684-6350-7
DOl: 10.1007/978-1-4684-6350-7
Preface
Modern tech rio logical societIes, with the demands they place upon
people, make critical the improved teaching of higher-order skills such as
reasoning, critical analysis, synthesis, problem solving, how to learn in
various fields, and how to control and manage one's thinking (Lesgold,
1987). New technologies-particularly the computer-combined with
better understanding of cognitive processes offer possibilities for the
necessary improvements. This book focuses on these possibilities, attend
ing especially to learning issues. The authors of the chapters were members
of a group of cognitive scientists from various disciplines who gathered in
Tiibingen in 1985 to discuss the cognitive processes of learning and how
those processes might be enhanced by intelligent instructional systems.
The promise of educational improvement through computers is not
a new one. Computers have been used in education for almost 30 years.
In the 1960s and 1970s, when computers began to appear in schools, a
variety of computer-assisted instructional programs were developed.
Generally, however, these did not meet the high expectations educators
had for the new medium. Learning via computer was almost always
based on inflexible presentations of didactic material. Moreover, these
programs had only limited capabilities for adaptive diagnosis and feed
back. Two things limited early teaching programs. First, the hardware
then available had minimal memory capacity and computational speed.
Second, the only theory available to guide instructional development was
behavior theory, which poorly matched the cognitive goals of education.
Now both these limitations are being overcome. Recent prototype
computer-based instructional systems are able to conduct an "intelligent"
dialogue with the student and to provide screen-based learning environ
ments that promote considerable self-initiated exploratory activity. These
developments are possible partly because computer hardware is more
powerful and there are better languages available for channeling that
power. Further, the development of productivity-enhancing programming
environments for object-oriented (e.g., Smalltalk and Loops) and rule
based (e.g., OPS5) program design has made it easier for psychologists
VI Preface
and educators to participate directly in software design. Further develop
ments are likely in this growing area of the computer world (Bobrow &
Stefik, 1986). The development of a technology for knowledge engineering
and expert system design is also important. The early expert systems
technology has played an important role in helping the world of education
see what might be possible on a computer, although ultimately other
contributions of artificial intelligence research will probably prove more
relevant.
Finally, and most important, there have been great advances in the
analysis of mental processes and in techniques for representing knowl
edge, including how it is acquired and how it changes over time. Work on
machine learning of semantic networks, rule systems, and "mental"
models (e.g., Michalski, Carbonell, & Mitchell, 1986) has provided useful
insights, and work directly on human learning has flourished. Together,
artificial intelligence and research on human cognition have provided a
basis for a new generation of computer-based teaching programs. Like
the human teacher, these programs attempt to model the knowledge of
the student as learning takes place.
Now we have new expectations of computer-based instructional
systems. They should be able to conduct a flexible and adaptive dialogue
with the student through words or graphic interfaces. Students should be
able to access information in varying forms and from varying viewpoints
as they wish. This information should be presented according to the
knowledge and skill level of the student, and should be grounded in the
core mental models that are needed for the particular domain of instruc
tion. Tasks for the student, and help in completing those tasks, should be
selected based upon a continually updated model of the student's learning
progress.
STRUCTURE OF INTELLIGENT TUTORING SYSTEMS
In this book, our colleagues join us in exploring the issues of learning that
arise in building intelligent tutoring systems, systems that can make
inferences about student knowledge and can interact intelligently with
students based upon individual representations of what those students
know. What do such systems look like? While they vary in architecture,
intelligent tutoring systems tend to have four major components (cf.
Lesgold, 1988, inter alia): the expert knowledge component, the learner
modeling component, the tutorial planning component, and the commu
nication component.
The expert knowledge component comprises the knowledge of experts:
the facts and rules of a particular domain. Expert knowledge is repre
sented in various ways, including semantic networks, frames, and pro
duction systems. It must include not only surface knowledge, but also the
representational ability that is a critical part of expertise: the ability to
Preface VII
construct implicit representational understanding from explicit observa
tions and other information. There are two different ways in which expert
knowledge can be represented in a tutor. In earlier tutors, the knowledge
was present only in "black box" form, which remained opaque to the
learner. For example, the SOPHIE I tutor (Brown, Burton, & Bell, 1975;
Burton & Brown, 1979) could answer any question a learner posed about
electrical measurement values for any point in a complex circuit. How
ever, because it used numerical simulation techniques to derive these test
values, it could not explain to the learner why the values were as reported.
Because such explanations seem very important to the acquisition of
expertise, more recent work has focused on "glass box models" (Goldstein
& Papert, 1977), whose knowledge is represented in a way that more
directly matches human capability, affording richer possibilities for
explanations to the learner.
The learner modeling component refers to the dynamic representation
of the emerging knowledge and skill of the learner. This requires a diag
nostic capability that can deduce the learner's knowledge from his inter
actions with the system as he tries to handle the educational tasks posed to
him. Again, different approaches to this component can be observed in
recent prototype systems and articles. One approach, the overlay model,
represents the learner's knowledge as a subset of the expert's knowledge
(Carr & Goldstein, 1977). An overlay model is useful only when the expert
knowledge is represented in a form that reflects human learning and per
formance capabilities and limitations. For example, SOPHIE I could not
have an overlay model of learner device knowledge because it used a
complex mathematical simulation to generate device information; human
knowledge of the device would not be organized as a system of mathe
matical equations.
An alternative approach is to include deviations from expertise in the
model of the learner, since learners sometimes show incorrect inferences
and systematic errors (Brown & Burton, 1978). Several approaches to
deviation modeling have been taken. For example, following Anderson's
(1983) theory of acquisition, one might represent deviations as incorrectly
generalized or differentiated conditions in production rules. Or one might
focus on the mental model as the unit of knowledge, with possible devia
tions from the expert view (e.g., White & Frederiksen, 1986).
The tutorial planning component is the part of an intelligent instruc
tional system that designs and regulates instructional interactions with the
learner. It is closely linked to the learner modeling component, using
knowledge about the learner and its own tutorial goal structure to decide
which instructional activities will be presented: hints to overcome per
formance impasses, advice and support, explanation, new material to be
taught, different practice tasks, tests to confirm the learner model, etc. The
tutorial component then is the source and orchestrator of pedagogical
interventions.
Vlll Preface
One important distinction among tutorial strategies is that between
didactic and discovery-oriented learning-between being told and learn
ing from experience. The primary advantage of didactic approaches is
that they are strongly goal-oriented. All activity focuses on the system's
instructional goals, because the system initiates and controls activity. In
discovery environments, some learners take a long time to make the dis
coveries that constitute the system's goals. Discovery environments are
seen as having an advantage in that they allow new knowledge to be con
structed in the learner's terms-in terms of the concepts and capabilities
the learner already possesses-from direct concrete experience. Micro
worlds (see, for example, Lesgold, 1987, or Shute, Glaser, & Raghavan,
in press) are one approach to discovery learning, and some recent micro
worlds tend to "guide" the discovery process in an effort to increase the
efficiency of discovery learning. It is important to note that modeling the
learner's knowledge becomes more difficult as his degrees of freedom are
increased.
Finally, the fourth component of intelligent instructional systems is
the communication component, which controls interactions between the
system and the learner. For a variety of reasons, intelligent instructional
systems have tended to use graphic interfaces heavily. Such interfaces
provide greater concreteness in the information provided to the student,
and they make the interaction with the student more "user-friendly" by
substituting pictures and pointing for text and typing. We can expect to
see intelligent instructional systems communicating with the learner pri
marily via graphics, since computer recognition and generation of written
and spoken language, though becoming more feasible, are incomplete
and computationally intensive. From a psychological viewpoint, this
presents a quandary, since there is a much richer body of research on
verbal processing than on the perception of and interaction with graphics.
More work is needed on learning issues in the use of graphics, even though
practical work using graphics flourishes. On the verbal side, Wahlster
(1981) has pointed out relevant questions concerning the design of natural
speech in dialogue systems.
RECENT EFFORTS
During the last few years, several intelligent instructional systems have
been developed as research prototypes (e.g., GUIDON by Clancey, 1983;
SOPHIE by Brown, Burton & de Kleer, 1982; WEST, by Burton &
Brown, 1979; the BUGGYIDEBUGGY series by Burton, 1982; and
Anderson's Lisp Tutor, Anderson, Boyle, & Reiser, 1985). Good over
views can be found in Wenger (1987), Barr & Feigenbaum (1982), and
Sleeman & Brown (1982). Several of the most recent efforts, such as
SMITHTOWN (Shute, Glaser, & Raghavan, in press) place special
emphasis on discovery learning and on the coaching of metacognitive
processes of learning.
Preface ix
Nonetheless, the development of intelligent instructional systems is still
in its infancy, though like many infants it flourishes. What is needed are a
stronger theoretical foundation based on deeper analysis of the acquisi
tion of complex learning-to-learn, conceptual and performance knowl
edge, and a stable level of research funding that can attract more of the
best cognitive science talent to this field. The same might be said for the
entire field of intelligent systems science and technology (Hayes-Roth &
Thorndyke, 1985). The infancy of this field has been very productive,
however, and this book attempts to record what has been learned so far.
The first part of the book deals with fundamentals of learning and in
struction that are especially relevant to the design of intelligent tutoring
systems. It begins with a programmatic chapter by Allan Collins and John
Seely Brown, on learning through reflection. Collins and Brown have
called for a new wave of apprenticeship learning, in which the learner
can reflectively compare his learning and problem-solving strategies and
methods with those of an expert. They have used concrete screen displays
that provide viewpoints on several different aspects of the learning or
problem-solving process simultaneously, "reifying" the cognitive process
ing that has taken place. Having access to these concrete reifications of
his own thinking, the learner is then more able to generalize metacogni
tive strategies from his, or the machine expert's, specific performances.
Kurt VanLehn (Chapter 2) proposes that impasses playa critical role in
the learning of procedural skills. Building from his work on Repair Theory
(Brown & VanLehn, 1980; VanLehn, 1981) VanLehn has developed a
metal eve I architecture for learning. When an impasse in performance
occurs, the student either attempts to repair his performance using weaker
methods or seeks help. When the chosen metaprocess succeeds in resolving
the impasse, the student may then abstract the actions needed to deal with
the kind of situation just encountered (see Rosenbloom & Newell, 1986,
for another view of the abstraction process). To the extent that the student
encounters impasses that he cannot resolve with his weak repair methods,
repairs may tend to generate buggy subprocedures. To the extent that
effective help is sought and received, impasses are the source of new
learning in the VanLehn formulation.
Cognitive diagnosis is central to psychological research, to instruction,
and to the construction of computer systems that adapt to their users'
needs. Ohlsson and Langley, in Chapter 3, describe a diagnostic method
for specifying the learner model for a procedural skill, given instances of
the performance of that skill. The heart of the method is a set of psycho
logically motivated evaluation criteria for choosing between competing
diagnoses. Ohlsson and Langley have implemented their method in a
computer program that performs cognitive diagnosis for the domain of
subtraction. They present selected computational results and discuss the
power and limitations of their method.
Chapter 4, by Ipke Wachsmuth, stresses the problem of situation
specific knowledge that is straightforward in one domain but cannot be
x Preface
applied in related situations. To use domain-specific knowledge in a
broad range of situations, two things appear to be required: (1) a set of
rules that support successful performance in a given subject domain, and
(2) a set of situations of sufficient perceptual variation to span the range
of circumstances in which the learner will be calling upon those rules.
Wachsmuth proposes a computer-based learning model, LAKOS 1,
which models different levels of student mathematical knowledge. Levels
of specificity in the learner's knowledge are typified by certain language
that influences the situations in which those rules will be effective.
Wachsmuth also outlines how his approach can be adapted for use in
intelligent tutoring systems.
The second part of the book deals with issues of learning systems
design. William Clancey (Chapter 5) develops a basis for teaching
students how to direct their own learning. He describes how a model of
active learning is being developed and tested in a knowledge acquisition
procedure for an expert system. The model is derived from modeling
methods that knowledge engineers use to approach new domains and to
acquire practical problem-solving knowledge. In particular, a knowledge
engineer is good at detecting gaps in a knowledge base and asking focused
questions that can lead to improved expert system performance. This
ability stems from domain-general knowledge about problem-solving
procedures and from knowledge of the categories of routine problem
solving and domain and task differences. Clancey examines different
forms of metaknowledge and illustrates how it can be incorporated into
an intelligent tutoring system. He presents a model of learning which
describes how the knowledge engineer detects problem-solving failures
and tracks them back to gaps in domain knowledge, which are then re
formulated as questions to ask teachers.
Alan Lesgold (Chapter 6) presents an architecture for representing
curriculum or goal knowledge in intelligent tutors. It is thus a first step
toward a theory of curriculum to inform the design of intelligent tutoring
and testing systems. He suggests that the knowledge in an intelligent tutor
should be structured in three interconnected levels: (1) curriculum
knowledge, a lattice of lessons or curricular subgoals connected by the
prerequisite relation; (2) a network representation of the knowledge to be
taught, from which explanations and student models can be generated;
and (3) a representation of the enduring characteristics of the student
(such as his metacognitive skills or aptitudes) to which instruction should
be sensitive. To illustrate one way in which such a theory can sharpen
ideas about intelligent tutoring, Lesgold focuses on the concept of pre
requisite. He shows that both (1) the curricular level, and (2) the domain
level of an intelligent system's knowledge base are needed to determine
why a student had trouble with a lesson for which he knew all the pre
requisites.
Gerhard Fischer (Chapter 7) considers the problem of users of powerful
and complex computer systems when they lack the skills needed to use
Preface XI
such systems effectively and efficiently. From a general theory of incre
mental learning processes, he derives processes "which will be the domi
nant way to master systems of broad functionality." This approach is
based on the paradigm of increasingly complex micro worlds (Burton,
Brown, & Fischer, 1983; Fischer, Burton, & Brown, 1978). The theory is
complemented by several applicatio:'"} systems that feature cooperative
"critics" that follow a person's use of a system and intervene when they
have advice or help that might be useful.
A further framework for learning systems design is provided by Norbert
A. Streitz in Chapter 8. In describing human-computer interaction, he dis
tinguishes a content problem of the learner who wants to acquire knowl
edge about a domain from the interaction problem that the learner has in
interacting with a tutoring system. In his approach, Streitz builds upon
Norman's "mental model zoo," distinguishing among a user's mental
model, a system's realization, the psychologist's view, and the designer's
view, and he introduces a formal notation that can help in distinguishing
these viewpoints. He then goes on to discuss how to communicate con
ceptual models of a novel system to a new user, discussing metaphorical
approaches that are as conflict-free as possible.
The third part of the book focuses on conditions that facilitate learning.
Mark Lepper and Ruth Chabay (Chapter 10) argue that motivational
components of tutoring strategies are as important as the cognitive com
ponents, and that truly personalized instruction must adapt to motiva
tional as well as cognitive differences. They illustrate and compare
interactions between computer tutors and learners on the one hand and
between human tutors and learners on the other, discussing four aspects
of tutorial intervention: control, timing, content, and style. The chapter
continues with a discussion of whether empathy might be built into a
computer tutor, whether part of an instructional system's intelligence
might not relate to motivational aims. Some steps toward the develop
ment of empathetic computer tutors are outlined, emphasizing the variety
of issues that arise in motivating children to engage in activities to foster
learning.
A related issue is addressed in Chapter 9, by Peter Fischer and Heinz
Mandl. They address the affective stresses that are placed on a learner by
the feedback that an intelligent tutor might provide, discussing what is
needed to model this aspect of the acquisition process adequately and
reliably. They suggest that some of the stresses can be overcome by trans
forming any kind of covert feedback into deliberate, overt feedback. They
then describe their KAVIS II interactive teaching-learning system, in
which the learner is trained to make intelligent use of the feedback pre
sented and to experience it positively. KAVIS II, like some of the other
systems presented in the book, was created as both a learning and a re
search tool that might efficiently and sensitively detect ongoing cognitive
processes.
The next two contributions have in common an experimental approach