"Watch What I Do" Chapter 20

Chapter
20

Graphical Representation and Feedback in a PBD System

Francesmary Modugno
and
Brad A. Myers

Introduction

A visual shell or desktop is a direct manipulation interface to an operating system. Examples include the Xerox Star and the Apple Macintosh Finder. Such interfaces were developed to hide the complexity of the underlying operating system from the user. Ideally, these systems should be easy to use yet powerful enough to allow novice and non-expert users to tailor the system. Users should easily be able to write programs to handle repetitive tasks, such as "back up all TEX files in the current folder," and simple transformations, such as "change zip code 15213 to 15213-3890 in all my letters." While existing visual shells are easy to use, they are difficult or impossible to program. How can we provide these programming capabilities in a way that is as easy to use as and consistent with the direct manipulation paradigm?

The PURSUIT visual shell contains a Programming by Demonstration (PBD) interface to help solve this problem. In a PBD system, users execute actions on examples and the system constructs a general program. Such systems enable users to create general procedures without having special programming skills. They are easy to use because users operate the way they normally do in the interface. Unfortunately, they have limitations: they can infer incorrectly; most display no static representation of the inferred program; and many provide no editing capabilities. This makes it difficult for users to know if the system has inferred correctly and to correct any errors. It also makes it difficult to revise or change a program.

To address these problems, PURSUIT introduces a novel graphical representation of the program while it is being written. Programs are represented in a state-based visual language in which data objects, such as files and folders, are represented as icons and operations are represented by the changes they cause to data icons. This chapter discusses this approach.

back to ... Table of Contents Watch What I Do