"Watch What I Do" Chapter 16


Chapter
16

Mondrian:
A Teachable Graphical Editor

Henry Lieberman


Introduction

Mondrian is an object-oriented graphical editor that can learn new graphical procedures through programming by demonstration. A user can demonstrate a sequence of graphical editing commands on a concrete example to illustrate how the new procedure should work. An interface agent records the steps of the procedure in a symbolic form. Machine learning techniques track relationships between graphical objects and dependencies between the interface operations. The agent generalizes a program that can then be used on "analogous" examples. The generalization heuristics set it apart from conventional "macros" that can only repeat an exact sequence of steps. The system represents all operations using "storyboards" which depict examples. By bringing the power of procedural programming to easy-to-use graphical interfaces, we hope to break down the "Berlin Wall" that currently exists between computer users and computer programmers.


Mondrian

Uses and Users

Application domain: Graphical editor

Intended users: Visual thinkers

User Interaction

How does the user create, execute and modify programs?
After selecting the New Command icon, the user demonstrates the procedure on an example. This resulting procedure is displayed as a new icon in the application's palette.

Feedback about capabilities and inferences:
As the user performs the example, Mondrian uses speech to describe its inferences.
The procedure can be viewed as a storyboard.

Inference

Program constructs: Procedure calls

Knowledge

Types and sources of information:
Information about properties of graphical objects.
Heuristics for ranking the significance of relationships between objects.

Implementation

Machine, language, size, date: Macintosh, Common Lisp, Approximately 600KB source, 6 MB image. 1991.

Notable Features


Mondrian's "dominoes" and "storyboards" form a consistent visual language for representing both built-in operations and the results of user demonstrations. It is the only PBD system to use natural language feedback.

back to ... Table of Contents Watch What I Do