Watch What I Do In March 1992, a programming by demonstration (PBD) workshop was held to give current researchers an opportunity to discuss their work with pioneers of the field. Out of that workshop came the idea for Watch What I Do. Edited by Allen Cypher, a research scientist with the Advanced Technology Group, Apple Computer, Inc., Watch What I Do contains chapters from both current researchers and early pioneers of the field. Most of the chapters were written specifically for this book, some with updates and extensions of existing systems, but a few chapters are reprints or adaptations of previous publication.
Watch What I Do is intended for a wider audience than just researchers active in the field, and the editor and contributing authors have collaborated to achieve this goal successfully. For the by-demonstration novice, the text is surprisingly clear and concise, and offers a breadth of introduction not found elsewhere. For the intermediate reader -- perhaps someone familiar with visual languages or user interface design -- this book provides an expanded understanding of PBD by considering the relationships among and between the different systems, as well as reflecting on the issues brought up about PBD systems as a group. For the by-demonstration expert, Watch What I Do offers a collection of relevant writings, a PBD chronology and a test suite intended to be of immediate use. The test suite is a collection of example tasks and analyses contributed by PBD researchers. These tasks show the broad applicability of PBD systems, and can be used by researchers to evaluate their own systems.
The book is broken into four sections. The first section, Systems, extends for over half the book and presents 18 PBD systems spanning a wide range of applications and techniques. The second section, Components, contains detailed discussions of components common across PBD systems. The third section, Perspectives, discusses some of the broader issues relating to PBD. The Appendices account for the fourth section.
Prior to the Systems section is an excellent introduction to PBD written by the editor, Allen Cypher. The introduction contains a compare-and-contrast look at four current approaches to end-user programming (end-user programming allows the user to customize a software application) -- preferences, scripting languages, macro recorders and programming by demonstration -- and then offers a motivation for PBD systems and their promises. In a nutshell: "The motivation behind PBD is simple and compelling: if a user knows how to perform a task on a computer, that should be sufficient to create a program to perform the task".
PBD systems generalize programs from user actions, and such systems are appealing because programming is done in the same environment in which the user performs an action. However, PBD systems have a challenge to overcome: to create a generalized program from user actions, the system often needs to determine why the user performed a certain action. Watch What I Do describes the various ways that different systems implement the appeal and deal with the challenge of PBD. A brief description of a few chapters follows.
Chapter 1 is a look at David Canfield Smith's Pygmalion, considered to be the first PBD system. Pygmalion was completed in 1975, and the chapter is a glimpse back, revealing groundbreaking ideas from almost 20 years ago. In addition, Smith motivates the benefits of concreteness and interactivity in PBD systems, describes Pygmalion and discusses things he would do differently building Pygmalion today.
Metaphors play a key role in many PBD systems, and an example is Chapter 4's reprint of Finzer and Gould's Rehearsal World. Designed to be used by non-programming educational software designers, Rehearsal World uses the theater metaphor to give abstract programming concepts a concrete, familiar feel.
The teaching metaphor is commonly used in PBD systems. This is not surprising, as the main focus of many PBD systems is the user 'teaching' the computer, via examples, the behavior of the desired program. Tinker, Metamouse and Turvy are examples of systems using this metaphor.
Chapter 6 describes Brad Myers' Peridot, one of the first systems to combine inference with PBD. Peridot's goal was to allow non-programmers to create graphical, highly-interactive user interfaces. The chapter provides a sound base for tackling Peridot's progeny, such as Garnet and its tools, which are described in a later chapter.
The contributions of some of the chapters should be new (and perhaps surprising) to most readers; two examples are Turvy and Triggers. David Maulsby's Turvy is an instructible agent, tested and refined using the "Wizard of Oz" technique -- Turvy is a researcher hidden behind a screen. The chapter describes Turvy and the results and observations of end-user experiments; one suspects, though, that the experiments were a bit more raucous than the chapter lets on.
PBD systems have been successful in limited application domains, and one of the challenges is to extend this success to system-wide, application-independent use. Richard Potter's Triggers attempts this by taking the unconventional approach of operating on low-level data -- pixels on the computer screen. The chapter on Triggers is full of intriguing insights and observations.
Chapter 21, part of the Components section, is about PBD invocation techniques. The authors state that PBD program invocation techniques are a lightly-researched area, and that new invocation techniques offer significant improvement opportunities for PBD systems. Two new invocation techniques are introduced: generalized, event-based invocation and program invocation by demonstration.
Chapter 24, also of the Components section, discusses how voice input can be used to clarify the user's intent. To ascertain user intent, many systems either prompt the user for more information, or prompt for confirmation about the correctness of any inference made. Unfortunately, this prompting is often considered intrusive, as it disrupts the user's concentration. The author suggests that voice commands might be a natural and non-intrusive alternative.
The Perspectives section contains a chapter entitled 'Characterizing PBD Systems', which provides a framework to compare and contrast different PBD systems. I recommend that this chapter be at least scanned prior to reading the Systems section.
Chapter 27, also of the Perspectives section, defines just-in-time programming as implementing a program for a task while the task is in progress. Just-in-time programming is not limited to PBD techniques, but this chapter discusses some of PBD's unique contributions to overcoming the obstacles to effective just-in-time programming.
Watch What I Do contains insights and observations based on new PBD research, and retrospections and extensions of existing PBD systems. It is recommended for those interested in visual languages in general, and by-demonstration programming in particular.
back to ... Watch What I Do