"Watch What I Do" Chapter 9


Programming Repetitive Tasks by Demonstration

Allen Cypher


A significant problem with today's personal computers is that users have to perform many repetitive tasks by hand. If my department decides to have a meeting every Monday, I have to go through my calendar, week by week, and paste the meeting into each Monday. If I decide to reformat a bibliography to use double-quotes instead of italics, I have to change each entry by hand. Although computers are supposed to excel at performing repetitive tasks, most users are unable to take advantage of this capability because they do not know how to program.

Programming by Demonstration is a technique that can potentially solve this problem. Since by definition most of the steps in a repetitive task are the same as in the previous repetition, it should be possible for a computer program to automate the task by recording the steps and replaying them.

Figure 1. In this example, the user makes a list of the subjects of a stack of messages. The user selects (a) and copies the subject of the first message, goes to the list, types "1. " and pastes in the subject (b). Then the user navigates to the second message (c), selects (d) and copies its subject, goes to the list, types "2. ", and pastes in the second subject (e). After the user navigates to the third message, the Eager icon appears (f), indicating that Eager has detected a pattern in the user's actions. Eager does not immediately begin performing actions; rather, it uses green highlighting to "anticipate" what the user is going to do next. In (f), Eager is anticipating that the user will select the subject of the third message, which is correct. The user selects and copies the third subject, goes to the list, and Eager anticipates that the user will type "3. " (g). After the user does this, Eager anticipates that the user will paste (h). The user does the paste and then navigates to the fourth message, which is anticipated in (i). At this point, the user is confident that Eager can perform the task correctly and therefore clicks on the Eager icon (j). A dialog box appears, and the user selects the option "Finish The Task" (k). Eager writes and executes a program which completes the task for the user, and automatically terminates (l).

What Eager Does

Eager offers a solution to the iteration problem for Programming by Demonstration -- the problem of specifying loops by demonstration. Eager is always on, constantly looking for repetitions in actions performed by the user. When it detects an iterative pattern, the "Eager" icon pops up on the screen. As the user continues to perform the task, Eager anticipates each next action by turning menu items, buttons, and text selections green. When Eager is correct, the user's intended selection will appear highlighted in green before it is selected. When Eager is incorrect, the user's choice will not match the highlighted item, and Eager will use the actual choice to revise its program. Once the user is confident that Eager knows how to perform the task correctly, he or she clicks on the Eager icon and Eager writes and executes a program which completes the task automatically. Figure 1 gives an example of Eager assisting a user with a repetitive task.


Uses and Users

Application domain: Application-independent; currently applied to HyperCard

Tasks within the domain: Repetitive tasks

Intended users: Non-programmers

User Interaction

How does the user create, execute and modify programs?
Programs are created by simply performing a repetitive task.
Programs are modified by performing an action other than the one highlighted by Eager.

Feedback about capabilities and inferences:
Eager uses anticipation highlighting to show the user how it has generalized.


Eager constantly compares each new event with the event history. It matches commands if they have the same verb and if each of their arguments follows a pattern . Typical patterns include the days of the week, and a linear sequence of integers.

Types of examples: Eager bases its generalizations on multiple examples

Program constructs: Variables. Iteration: set iteration, iteration with a counter, and iteration until a condition is satisfied. For instance, if a loop creates buttons in a column, the loop will terminate when the bottom of the card is reached.

No nested loops. No conditionals.


Types and sources of information:
A knowledge base of patterns that can be expanded by developers. For example, linear sequences of integers, and common string sequences like days of the week.
Application knowledge bases of features to test when matching command arguments. For instance, HyperCard's Cards match on Names, Numbers, or IDs.


Machine, language, size, date: Macintosh. 6,000 lines of LISP on a microExplorer, ported to Macintosh Common Lisp, then ported to 15,000 lines of C++. The knowledge base describing HyperCard commands is 800 lines long. The modifications to HyperCard to make it recordable and to implement anticipation highlighting are 5,000 lines of Pascal. 1990.

back to ... Table of Contents Watch What I Do