Your Wish Is My Command: Programming by Example (The Morgan Kaufmann Series in Interactive Technologies)

443 Pages·2001·25.705 MB·English

by  Henry Lieberman

Your Wish
Is My Command
Programming by
Example
This Page Intentionally Left Blank
Your Wish
Is My Command
Programming by
Example
Edited by
Henry Lieberman
Media Lab
Massachusetts Institute of Technology
®
M [4
MORGAN KAUFMANN PUBLISHERS
AN IMPRINT OF ACADEMIC PRESS
A Harcourt Science and Technology Company
SAN FRANCISCO SAN DIEGO NEW YORK BOSTON
LONDON SYDNEY TOKYO
Executive Editor Diane D. Cerra
Assistant Developmental Editor Marilyn Alan
Publishing Services Manager Scott Norton
Production Editor Howard Severson
EditoriaIAssistant Mona Buehler
Cover Design Yvo Riezebos
Cover Image Kazuo Kawai / Photonica; Back photo: © 2000 by Webb Chappell
Text Design Rebecca Evans & Associates
Cop)editor Laura Larson
Proofreader Ruth Stevens
Composition & Illustration Technologies 'N Typography
Indexer Steve Rath
Printer Courier Corporation
The following material is reprinted with permission from Communications of the ACM, March 2000, Vol. 43(3):
Smith, D. C., Cypher, A., and Tesler, L. "Novice Programming Comes of Age," 75-81; Myers, B. A., McDaniel, R.,
and Wolber, D. "Intelligence in Demonstrational Interfaces," fig. 3; Repenning, A. and Perrone, C. "Programming
by Analogous Examples," figs. 3, 4, and 5; St. Amant, R., Lieberman, H., Potter, R., and Zettlemoyer, L. "Visual
Generalization in Programming by Example," figs. 3 and 4.
Designations used by companies to distinguish their products are often claimed as trademarks or registered
trademarks. In aH instances where Morgan Kaufmarm Publishers is aware of a claim, the product names appear in
initial capital or all capital letters. Readers, however, should contact the appropriate companies for more com-
plete information regarding trademarks and registration.
ACADEMIC PRESS
A Harcourt Science and Technology Company
525 B Street, Suite 1900, San Diego, CA 92101-4495, USA
h ttp:/ /w ww. acade m icp ress, co m
Academic Press
Harcourt Place, 32 Jamestown Road, London NWl 7BY, United Kingdom
http://www.academicpress.com
Morgan Kaufmann Publishers
340 Pine Street, Sixth Floor, San Francisco, CA 94104-3205, USA
h trp: / /www. m kp. co m
© 2001 by Academic Press
All rights reserved
Printed in the United States of America
05 04 03 02 01 5 4 3 2 1
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any
means--electronic, mechanical, photocopying, recording, or otherwise--without the prior written permission of
the publisher.
Library of Congress Cataloging-in-Publication Data
Your wish is my command : programming by example / Henry Lieberman, editor.
p. cm.
Includes index.
ISBN 1-55860-688-2
1. Computer programming. I. Lieberman, Henry.
QA76.6.Y63 2001
005.2--dc21 00-069638
This book has been printed on acid-free paper.
Foreword
Ben Shneiderman
University of Maryland
Setting an alarm clock is probably the most common form of programming.
Users set a time and then put the clock in alarm mode. Older twelve-hour
mechanical clocks usually had a special alarm hand that could be moved to
the time for the alarm to ring, and then the users turned the alarm switch
on. A nice form of direct manipulation programming--easy to learn and
use.
Direct manipulation is a term I coined in 1981 to describe the visual
world of action in many successful graphical user interfaces such as video
games, air traffic control, and what-you-see-is-what-you-get word proces-
sors. The principles were to
• represent the objects and actions visually,
• replace typing with pointing and dragging,
• provide rapid, incremental and reversible actions, and
• offer immediate and continuous feedback to users.
These principles can lead to interfaces that help novices and experts, pre-
vent or at least reduce errors, and encourage exploration because reversibil-
ity is supported. Designers continue to refine and extend direct manipula-
tion, but critics complain that direct manipulation only works for limited
tasks. They often ignore the possibility of direct manipulation program-
ming, which was part of the original conception (Shneiderman 1982, 1983).
To explore the possibilities, we built a direct manipulation programming
tool in 1984-85 that enables users to create macros for MS DOS. This tool,
Direct Manipulation DOS (DMDOS) (Iseki 1986), enabled users to record
and view their actions, and then store and replay macros. We were moti-
vated by successful macro facilities for Unix, word processors, and spread-
sheets. These early keyboard-oriented systems led us to joke that "those
who ignore history are destined to retype it." We were also inspired by
vi Foreword
innovative programming by demonstration in David Canfield Smith's (1977)
Pygmalion, graphical macro facilities in Dan Halbert's (1984) SmallStar, and
Alan MacDonald's (1982) early call for visual programming. These pioneers
and other innovators believed in the goal of empowering users to create
useful programs, extend existing interfaces, and build small just-in-time
programs that automated daily tasks.
This important volume carries forward the agenda of making direct ma-
nipulation programming (or programming by example, programming by
demonstration, end-user programming, programming in the user interface,
etc.) a reality. While there have been successes in the intervening years,
such as programmable machine tools, visual programming languages, and
a variety of macro-building programs, widespread adoption is still elusive.
Henry Lieberman deserves credit for his long devotion to this topic and for
collecting the diverse strategies and application domains in this volume. He
and the contributors to this volume remind us all of the breadth of opportu-
nities and depth of ambition.
The allure of direct manipulation programming is its capacity to em-
power users, while minimizing learning of programming concepts. Re-
searchers continue to seek simple cognitive models for programming that
are in harmony with the cognitive model of the existing user interface. Just
as the programmable mechanical alarm clock is tied to the familiar model
of clock hands, researchers have wanted to build on the visual nature of
graphical user interfaces.
This fundamental human-computer interaction challenge has inspired a
generation of designers, who have come up with innovative strategies for
supporting iteration, conditionals, parameter passing, modular design, pat-
tern matching, and data representation. This treasure chest of strategies is
likely to pay off in multiple solutions for direct manipulation programming
and related problems. A successful strategy would not only be easy to learn
but also support rapid composition of common programs. Then it would
also be easy to invoke, with comprehensible feedback about successful and
unsuccessful executions.
One strategy represented in this book is to develop software that recog-
nizes familiar patterns of action and infers a useful program. There may be
some opportunities along this path, but I prefer the second path of special
tools for users to create a program, just as they move the special hand of an
alarm clock to set the wake-up time.
A third path, also well represented in this book, is visual programming
languages in which the users set out to write a program, but visually instead
of textually. Visual programming languages may have a simple basis such as
Foreword vii
dragging items from a relational table to a screen-based form to create a re-
port program. More elaborate visual programming languages have graphic
symbols to represent objects, actions, conditionals, loops, and pattern
matching.
A fourth path might be to add history capture environments for every
interface. Unix command line interfaces had a history log that allowed users
to conveniently review and reuse commands. World Wide Web browsers
support history keeping of page visits with relatively easy review and reuse.
Microsoft Word captures a history of actions to support undo operations,
but users cannot review the history or save it. Adobe Photoshop 5.0 added a
nice history feature for graphic designers, demonstrating that even in com-
plex environments, rich history support is possible.
Our current efforts with Simulation Processes in a Learning Environ-
ment have emphasized history keeping, enabling users to review their work,
replay it, annotate it, and send it to peers or mentors for advice (Plaisant
1999). An immediate payoff was that faculty could run the simulation in ex-
emplary or inappropriate ways and store the histories for students to use as
a training aid.
The story of this field and this book is that there is magic and power in
creating programs by direct manipulation activities, as opposed to writing
code. The potential for users to take control of technology, customize their
experiences, and creatively extend their software tools is compelling.
Eighteenth-century scientists, such as Ben Franklin, experimented with
electricity and found its properties quite amazing. Franklin, Michael Fara-
day, ]ames Clerk Maxwell, and others laid the foundation for Thomas Edi-
son's diverse applications, such as refinements of telegraphy, generators,
and electric lighting. This book brings reports from many Franklins, Fara-
days, and Maxwells who are laying the foundation for the Thomas Edisons
still to come. It is difficult to tell which idea will trigger broad dissemination
or whose insight will spark a new industry. However, the excitement is
electric.
References
Halbert, Daniel. 1984. Programming by example. Ph.D. diss. University of California,
Berkeley. (Available as Xerox Report OSD-T8402, Palo Alto, CA, 1984.)
Iseki, O. and B. Shneiderman. 1986. Applying direct manipulation concepts: Direct
Manipulation Disk Operating System (DMDOS). ACM SIGSOFT Software Engi-
neering Notes 11, no. 2 (April): 22-26.
viii Foreword
MacDonald, Alan. 1982. Visual programming. Datamation 28, no. 11 (October): 132-
140.
Plaisant, C., A. Rose, G. Rubloff, R. Salter, and B. Shneiderman. The design of history
mechanisms and their use in collaborative educational simulations. In Proceed-
ings of the Computer Supported Collaborative Learning Conference (December
1999).
Shneiderman, B. The future of interactive systems and the emergence of direct ma-
nipulation. Behaviour and Information Technology 1, no. 3 (1982) 237-256.
1983. Direct manipulation: A step beyond programming languages. IEEE
Computer 16, no. 8 (August 1983): 57-69.
Smith, D. C. 1977. Pygmalion: A computer program to model and stimulate creative
thought. Basel: Birkhauser.
Contents
Foreword V
Ben Shneiderman
Color Plates following page 192
Introduction 1
Henry Lieberman
Chapter 1 Novice Programming Comes of Age 7
David Canfield Smith, Allen Cypher, and Larry Tesler
Abstract 8
1.1 Introduction 8
1.2 Programming without a Textual Programming
Language 9
1.3 Theoretical Foundations 11
1.3.1 Sloman’sA pproach 13
1.3.2 Bruner’s Approach 15
1.4 Empirical Evidence 16
1.5 Conclusion 18
References 19
Chapter 2 Generalizing by Removing Detail: How Any
Program Can Be Createdb y Working with
Examples 21
Ken Kahn
Abstract 22
2.1 Introduction 22
2.2 A Brief Introduction to ToonTalk 24
2.3 An Example of Programming by Example 26
ix