Table of Contents Watch What I Do
The eighteen programming by demonstration systems presented in this section span a wide range of approaches, applications, audiences and user interaction techniques. From systems that monitor high-level user events to systems that monitor the pixels on the computer screen, from systems for hand-held calculators to systems for formatting bibliographies, from systems for programmers to systems for graphic designers, from systems that display LISP code to systems that speak in English -- there is great variety in these implementations. But the common theme of "demonstration" appears in all of these systems, and there are other themes as well, such as recording, instructing, and programming.
As mentioned in the Introduction, macro recorders like QuicKeys and KeyWatch record low-level user actions and play them back. PBD systems that take their inspiration from macro recorders extend this approach in two ways: they work with high-level events, and they generalize the actions to create programs. In general, designers of recording systems strive to maintain the ease of use of macro recorders. They may even hope that their end users do not realize that they are creating programs.
The designers of systems that take their inspiration from the teaching metaphor, on the other hand, are much more willing to be explicit about the fact that their systems are creating programs and making generalizations. They want to encourage the end user to be a good teacher and to provide clear and informative examples. These systems use a variety of techniques to increase the communication between the user and the PBD system.
Systems that take their inspiration from traditional programming are willing to demand some technical expertise from their users. These systems use demonstrational techniques to overcome some of the difficulties of programming, but they may require the user to explicitly add flow of control constructs, such as loops and conditionals.
Tinker (Chapter 2) is for beginning programmers. It is designed around the "teaching" metaphor, and relies on the teacher's ability to provide good examples. Tinker presents a solution to the problems of demonstrating conditionals and constructing recursive programs.
The Predictive Calculator (Chapter 3) is a macro-like system for calculators. It adds the innovation of always monitoring the user's actions -- there is no "record" mode. Even though it uses a rather simple inferencing scheme, it is remarkably successful at detecting patterns. The KeyWatch macro recorder is quite similar to the Predictive Calculator, but without the inferencing capabilities.
Rehearsal World (Chapter 4) extends traditional programming by adding a macro recorder. It replaces writing lines of code with demonstrations, thereby avoiding many of the complications of syntax and vocabulary. It is intended to be used by educational software designers who do not know how to program.
SmallStar (Chapter 5) is one of the classic PBD systems. It introduces the notion of "programming in the user interface". It is the first PBD system designed for the direct-manipulation desktop interface that is now common on personal computers. Halbert introduces the concept of "data descriptions", which is central to understanding how PBD systems refer to objects in users' demonstrations. SmallStar is explicitly designed for a non-programming audience. It includes a scripting language that mixes text and icons.
Peridot (Chapter 6) is an influential system because of its effective use of inferencing. It inspired much of the subsequent use of inferencing in the systems that followed it. It was successful because it deals with a fairly narrow domain -- the creation of user interface objects -- and it takes advantage of built-in knowledge about typical situations in that domain. Another significant feature of Peridot is that it effectively uses PBD to establish graphical constraints between objects. So, for instance, reshaping a rectangle can cause the rectangle centered inside it to be reshaped as well.
Metamouse (Chapter 7) applies programming by demonstration to graphical editing. It is based on the metaphor of teaching, and makes effective use of an "agent" -- Basil the turtle -- as the student who is to be taught by the user. Metamouse has users express their intent by making graphical constructions, such as creating a line segment to indicate how far one object should be from another.
TELS (Chapter 8) is a text editing system that dramatically illustrates how messy the real world actually is. Even tasks that we consider to be highly repetitive are fraught with exceptions and special cases. TELS demonstrates a rather simple approach to this problem that is surprisingly effective.
Eager (Chapter 9) is another PBD system that is based on macro recorders. It handles simple repetitive tasks, and was designed specifically to be easy to use. Eager turns the tables and gives demonstrations to the user, highlighting objects to show what it has inferred.
Garnet (Chapter 10) is a complete User Interface Development Environment that employs demonstrational techniques in some of its tools. It is an extension of Peridot, and meets a wide variety of practical needs. Garnet signals a trend towards hybrid systems. In a hybrid system, programming by demonstration can be used where it is most beneficial, but other techniques -- such as special purpose editors and dialogs -- can be employed to cover for its weaknesses.
Turvy (Chapter 11) is the ultimate example of a PBD system based on the instructional metaphor -- the student is not actually a program, but a person. Turvy investigates the classical hard problem for programming by demonstration: how to infer the user's intent. Since the user is talking to a real person rather than to a computer program, the user can communicate in English, and Turvy can investigate what users would like to be able to tell their PBD system.
Chimera (Chapter 12) exemplifies a class of applications that brings a wider scope to the use of demonstrational techniques. Chimera uses demonstrations to establish constraints between graphical objects and to specify graphical search and replace, as well as to create macros by example.
The Geometer's Sketchpad (Chapter 13) is a commercial application that is being used in schools to teach geometry. It presents a novel approach to end-user programming by considering constructions in an interactive geometry environment as programming. The Geometer's Sketchpad carries on a trend in programming with graphical constraints that began with Ivan Sutherland's Sketchpad program.
Tourmaline (Chapter 14) is another application with several PBD components. Designed for use in text formatting, it has separate components for formatting headings, tables, and bibliographies. It is unusual in that it does not make use of information about how the user formatted the example text. Instead, it just looks at the result and makes inferences from that. Tourmaline achieves success without trace information by taking advantage of domain knowledge.
History-Based Macros by Example (Chapter 15) is a part of the Chimera system described in Chapter 12. This chapter focuses on how the graphical history can be used to create macros. Graphical histories are an important extension to the macro-recording paradigm, since the graphical representation of the steps in a program makes it much easier to "read" the program and modify it. The graphical history mechanism presented here is an important advance. It employs a variety of techniques to make the history more readable than a simple reproduction of small copies of the screen. All developers of systems for programming by demonstration are concerned about how to present programs to the end user, and Chimera's graphical history is one answer.
Mondrian (Chapter 16) allows users to add their own commands to the palette in a graphical editor. It offers a novel extension to the macro-recording paradigm by using synthesized speech to explain its inferences while the user is demonstrating a program. Mondrian uses a storyboard to present programs to the user.
Triggers (Chapter 17) is unique among PBD systems in that it operates on low-level data -- the pixels on the computer screen. By working with pixels, it can access data that is unavailable to other PBD systems. Triggers works with any application on a Macintosh computer.
AIDE (Chapter 18) deals with the issue of providing system-wide, application-independent tools for creating programs by demonstration. While most of the other systems in this book were built as part of one particular application program, AIDE offers an environment so that any application can incorporate programming by demonstration capabilities.
back to ... Table of Contents Watch What I Do