"Watch What I Do" Chapter 9
Programming Repetitive Tasks by Demonstration
A significant problem with today's personal computers is that users have to
perform many repetitive tasks by hand. If my department decides to have a
meeting every Monday, I have to go through my calendar, week by week, and paste
the meeting into each Monday. If I decide to reformat a bibliography to use
double-quotes instead of italics, I have to change each entry by hand. Although
computers are supposed to excel at performing repetitive tasks, most users are
unable to take advantage of this capability because they do not know how to
Programming by Demonstration is a technique that can potentially solve this
problem. Since by definition most of the steps in a repetitive task are the
same as in the previous repetition, it should be possible for a computer
program to automate the task by recording the steps and replaying them.
Figure 1. In this example, the user makes a list of the subjects of
a stack of messages. The user selects (a) and copies the subject of the first
message, goes to the list, types "1. " and pastes in the subject (b). Then the
user navigates to the second message (c), selects (d) and copies its subject,
goes to the list, types "2. ", and pastes in the second subject (e). After the
user navigates to the third message, the Eager icon appears (f), indicating
that Eager has detected a pattern in the user's actions. Eager does not
immediately begin performing actions; rather, it uses green highlighting to
"anticipate" what the user is going to do next. In (f), Eager is anticipating
that the user will select the subject of the third message, which is correct.
The user selects and copies the third subject, goes to the list, and Eager
anticipates that the user will type "3. " (g). After the user does this, Eager
anticipates that the user will paste (h). The user does the paste and then
navigates to the fourth message, which is anticipated in (i). At this point,
the user is confident that Eager can perform the task correctly and therefore
clicks on the Eager icon (j). A dialog box appears, and the user selects the
option "Finish The Task" (k). Eager writes and executes a program which
completes the task for the user, and automatically terminates
What Eager Does
Eager offers a solution to the iteration problem for Programming by
Demonstration -- the problem of specifying loops by demonstration. Eager is
always on, constantly looking for repetitions in actions performed by the user.
When it detects an iterative pattern, the "Eager" icon pops up on the screen.
As the user continues to perform the task, Eager anticipates each next
action by turning menu items, buttons, and text selections green. When Eager is
correct, the user's intended selection will appear highlighted in green before
it is selected. When Eager is incorrect, the user's choice will not match the
highlighted item, and Eager will use the actual choice to revise its program.
Once the user is confident that Eager knows how to perform the task correctly,
he or she clicks on the Eager icon and Eager writes and executes a program
which completes the task automatically. Figure 1 gives an example of Eager
assisting a user with a repetitive task.
Uses and Users
Application domain: Application-independent; currently applied to
Tasks within the domain: Repetitive tasks
Intended users: Non-programmers
How does the user create, execute and modify programs?
Programs are created by simply performing a repetitive task.
Programs are modified by performing an action other than the one highlighted by Eager.
Feedback about capabilities and inferences:
Eager uses anticipation highlighting to show the user how it has generalized.
Eager constantly compares each new event with the event history. It matches commands if they have the same verb and if each of their arguments follows a pattern . Typical patterns include the days of the week, and a linear sequence of integers.
Types of examples: Eager bases its generalizations on multiple
Program constructs: Variables. Iteration: set iteration, iteration with
a counter, and iteration until a condition is satisfied. For instance, if a
loop creates buttons in a column, the loop will terminate when the bottom of
the card is reached.
No nested loops. No conditionals.
Types and sources of information:
A knowledge base of patterns that can be expanded by developers. For example, linear sequences of integers, and common string sequences like days of the week.
Application knowledge bases of features to test when matching command arguments. For instance, HyperCard's Cards match on Names, Numbers, or IDs.
Machine, language, size, date: Macintosh. 6,000 lines of LISP on a
microExplorer, ported to Macintosh Common Lisp, then ported to 15,000 lines of
C++. The knowledge base describing HyperCard commands is 800 lines
long. The modifications to HyperCard to make it recordable and to implement
anticipation highlighting are 5,000 lines of Pascal. 1990.
back to ... Table of Contents Watch What I Do