Table of Contents Watch What I Do
Typical applications created with Garnet include: drawing programs similar to Macintosh MacDraw, user interfaces for expert systems and other AI applications, box and arrow diagram editors, graphical programming languages, game user interfaces, simulation and process monitoring programs, user interface construction tools, CAD/CAM programs, etc. Garnet is in the public domain and is freely available. As of fall, 1992, over 30 projects around the world are using the system regularly. You can get Garnet by anonymous FTP from a.gp.cs.cmu.edu. Change to the directory /usr/garnet/garnet/ and retrieve the README file for instructions. Or you can send electronic mail to garnet@cs.cmu.edu. Garnet stands for Generating an Amalgam of Realtime, Novel Editors and Toolkits.
One of the important goals of the Garnet project is to allow all aspects of the user interface to be created without conventional programming. In particular, we want to allow the user to draw example pictures to show what the user interface will look like, and then demonstrate how the user interface will respond to inputs from the end user. As a result, demonstrational techniques are widely used in Garnet, mainly in the various higher-level tools. This chapter discusses some of these. Other papers about Garnet discuss the overall design [Myers 90d], the components, the programming style [Myers 92a] [Myers 92e], and there is a complete reference manual [Myers 92b].
(a) (d)
(b)
(c)
Figure
1. The workspace window of Lapidary (b), where a node of a graph
editor is being created, along with the standard commands (a), object menus
(c), and a dialog box for setting constraints on rectangles
(d).In addition, like Peridot, Lapidary supports the construction and use of "widgets" (sometimes called interaction techniques or gadgets) such as menus, scroll bars, buttons and icons. Lapidary therefore supports using a pre-defined library of widgets, and defining a new library with a unique "look and feel." The run-time behavior of all these objects can be specified in a straightforward way using constraints and abstract descriptions of the interactive response to the input devices. Lapidary generalizes from the specific example pictures to allow the graphics and behaviors to be specified by demonstration.
Graphical objects can be created in a number of different ways using Lapidary. As shown in Figure 1, the standard menus provide the usual range of graphical primitives, so objects can be created from scratch.
Sometimes, designers want to use relationships that cannot be created out of these simple choices. In that case, the Custom option is selected, and the designer is allowed to type in an arbitrary Common Lisp expression specifying the constraint using the C32 system (discussed below).
Unlike Peridot, Lapidary currently does not try to infer the graphical constraints. Instead, they must all be specified explicitly using the dialog boxes. With Lapidary, we wanted to concentrate on creating a practical tool that extends the range of interfaces that can be produced, and the constraint inferencing in Peridot was felt to be too risky for the first version. Future Garnet tools will revisit this issue.
In order for the graphical objects to be useful at run-time, the specific constraints must be generalized to work on run-time objects, rather than on the specific example objects used in the editor. For example, in Figure 1, the label on the nodes should change, but still stay centered, as the node is replicated. Thus, the constraints need to be generalized to reference objects indirectly through variables, rather than by using specific object names. To do this, the reference to the object is replaced with an expression that calculates the desired object, and stores it in a special slot. The constraint system then automatically ensures that the constraints change whenever the slot is set with a different object.
It is important to emphasize that Lapidary makes these transformations automatically. The user interface designer never sees any of the code. Even if the designer created custom constraints by typing Lisp code, the references in the expression can be to example objects (selected by pointing at them with the mouse), and the system will convert these references to be general variables where appropriate.
Another way that Lapidary generalizes from the examples is to automatically make copies of objects at run-time. For example, to show the selection in a drawing editor, the designer might draw a single set of selection handles around an example object. However, at run time, multiple objects might be selectable, so Lapidary arranges for the selection handles to be duplicated at run-time if necessary.
Often, there will be a specific object that serves as the feedback for an operation. For example, a reverse-video rectangle might move over the items in the menu to show which is the current selection. In other cases, the objects themselves should change to be the feedback. For example, the currently selected item in a menu might be shown in italics. Another use is to have buttons move to cover their shadows (and therefore look more "3-D"), as in the Motif and Garnet look and feels. In this case, the desired changes can be shown by demonstration. To specify the changes by demonstration, first the designer selects the objects that will change, and then hits a button in the dialogue box. The full current state of the selected objects is remembered. Next, the designer edits the objects in whatever way desired, for example to make the string be italic. Then, another button is hit, and Lapidary creates a constraint that will choose between the two values based on whether the object is selected or not. Changes can be made to as many properties as desired, and correct constraints will be created for all of them.
Figure 2 shows a typical instance of C32. Each column contains a separate object. Rows are labeled with the names of the slots, such as :left, :top, :width, :height, :visible, etc. Since different objects can have different slots, the slot names are repeated in each column. For example, lines have slots for the endpoints (:x1, :y1, :x2, :y2) but rectangles do not. Also, each object's display can be scrolled separately, so each has its own scroll bar. This makes the spreadsheet look somewhat like a multi-pane browser as in Smalltalk.
(a)
(b)
The
spreadsheet cells show the current values of the slots. If a value changes,
then the display will be immediately updated. If the user edits the value in
the spreadsheet cell, the object's slot will be updated. The "F" icon by some
slots in Figure 2 means that the slot value is computed from a formula.
Pressing the mouse on the icon causes the constraint expression to appear in a
different window. The expression itself can be edited by typing or other
techniques.
Once a complex formula is created, it will often be needed in a slightly different form for a different slot or a different object. As an example, suppose the user has constructed a constraint that centers an object horizontally with respect to two other objects. Now, suppose the programmer wants to center the object vertically also. The formula could be copied to the :top slot, but all the slot references need to be changed (:left to :top and :width to :height). Therefore, when a formula is copied, C32 tries to guess whether some slot names should be changed. This uses a few straightforward rules based on the slot names of the source and destination slots. Currently, these rules are hardwired into the code. If it appears that slot names should be changed, the user is queried with a dialog box, and if the answer is OK, then the formula is modified automatically. Since this is a more radical change than the inferred slots discussed in the previous section, it seems prudent to require confirmation.
The intelligent copying and generalizing in C32 helps the user generate correct constraints by example. Without these aids, it is quite common to forget to change one or more of the references when formulas are copied. Generalizing also helps the programmer decrease the size of the code by promoting the reuse of existing formulas.
Demonstrational techniques have been added to Gilt in two places: to infer graphical styles from examples, and to infer transformations of data and dependencies to minimize the number of call-back procedures.
Another problem for interface designers is laying out the widgets in the window. When the designer places widgets with the mouse, they tend to be uneven and look sloppy. Therefore, most builders provide grids and alignment commands. However, these can be clumsy to use, and they do not ensure that different dialog boxes will have a consistent alignment (for example, that the titles are always centered at the top of the window).
To help solve these problems, Gilt introduces the notions of Graphical Tabs and Graphical Styles into an interface builder, which are more completely described in [Hashimoto 92]. These are based on the styles and tabs in text editors such as Microsoft Word. A "graphical tab" is simply a horizontal or vertical position in the graphics window to which objects can be aligned. A "graphical style" is a named set of properties, which can be applied to widgets. The designer can edit a widget so it has the desired properties, select it, and then define a named style based on it. The values of the properties and the positions of the widgets will be associated with that style name. The style can then be applied to other widgets.
Furthermore, Gilt will try to automatically guess when to apply a style, so the designer does not have to. By guessing the appropriate properties and layout, Gilt makes the user interface design process significantly faster, since users can quickly and imprecisely place widgets, and the system will automatically neaten them. Since the inferencing is based on the styles the user has defined, rather than based on global, default rules, as in earlier systems like Peridot and Druid [Singh 90], the inferred properties and positions are more likely to be correct.
These features in Gilt are classified as "demonstrational" because the user defines a style by example on a particular widget, but the style is automatically generalized so it will work on any of a set of widget types.
A graphical style includes a set of widget properties, and optionally some position information as well. To create a new style, the designer modifies a widget to the desired appearance using the conventional property sheets, selects that widget, and then issues the Define Style command. The designer must then type a style name into the Style editing window that will appear. Gilt compares the widget's current properties with the default values for that widget and copies all that are different. Styles can also include position information. For example, a designer might specify that objects with the Main-Title-Style should use a large bold font, and be centered at the top of the window. The position information for styles can either be with respect to a graphical tab stop, or relative to a previously created object.
When inferencing is on, Gilt tries to infer a new style whenever a widget is created or moved. The algorithm looks for styles that affect the same type as the widget, and, if the style has a position component, then it checks how close the widget matches the style's position. The types that styles are associated with include strings, button objects (including radio buttons and check boxes), numeric sliders (including both sliders and scroll bars), text input fields, etc. A list is created of all the styles that match, sorted from most likely to least likely.
Any inferencing system will sometimes guess wrong. Thus, it is important to provide appropriate feedback so the users are confident that they are in control and know what Gilt is doing. In immediate mode, the first style on the style list is immediately applied to the graphics, and the name of the style is shown at the bottom of the style control window (Figure 3). The widget will also jump to the inferred position and change appearance. If the inferred style is not correct, the designer can hit the "Try Again" button, which will remove the guessed style and instead apply the next style in the sorted list. This can be repeated until there are no more styles in the list. The "Undo" button can also be hit to remove the guessed style, and return the widget to its original position and properties. In prompt-first mode, the sorted list of all the inferred styles is presented in a window, with the most likely selected. The designer can select a different style, if necessary, and then hit OK or Cancel. When a style is defined, it immediately becomes a candidate for inferencing. This is very useful when a number of widgets will all be created using the same style.
Styles can be edited in two ways. A property sheet can be displayed which shows the current values of the properties for the style, and this can be edited directly. This property sheet has the same format as the ones for the standard widgets. The positions associated with the style can be edited using the appropriate dialog boxes.
Alternatively, the designer can edit the styles in the same way as they were created: by working on example widgets. Whenever a widget is edited that has already been defined to be of a particular style, Gilt pops up a dialog box asking if the edit should change the style itself. The other alternatives are to make the widget no longer belong to the style, or to cancel the change and return the object to its appearance before the edit was attempted.
In the future, we plan to add the ability to have objects use a particular style with exceptions, but this is a complex problem [Johnson 88]. Some of the issues are whether to copy the attributes or retain the link to the original style, what to do to a style when the style it inherits from is changed, and whether to save the inheritance links in the style files, or write out all the style information to each file.
We have observed that many of the call-back procedures are actually used to filter the values from widgets and connect widgets to each other, rather than to perform real application work. By identifying some common tasks that call-backs are used for, and providing other methods for handling the tasks, we have been able to eliminate the need for most call-backs. The tasks can be classified into the following categories:
Gilt tries to automatically pick the appropriate transformation. There are two techniques used to guess what is appropriate. First, the designer can type an example value into the Resulting Filtered Value field at the bottom of the Exported Value Control window (Figure 4-a). In this case, Gilt will try to guess a transformation that will convert the current unfiltered value into the specified value. If none of the built-in transformations is appropriate, then Gilt creates a case statement. The designer can then operate the widget to put it into different states (and therefore to change the unfiltered value), and type the desired filtered value for each case. This allows arbitrary transformations (e.g., converting the German "Fettdruck" or the French "Gras" to :BOLD). The resulting code for the filter is shown in the Filter Expression window.
The second option is used when the designer enters a procedure into the filter expression, and then selects a widget to supply the value to a parameter of the procedure. Here, Gilt tries to find an appropriate transformation so that the widget value will be filtered into the required type of the parameter. A Value Control window will pop up to confirm each transformation, and also to request the designer to specify the transformation if Gilt cannot infer it.
The user can check that the filter expression is achieving the desired result in two ways. First, the interface can be exercised to test the code. Second, the Filter Expression field shows the Lisp code that is being used. In the future, we will be investigating other techniques for showing the transformations that will be usable by non-programmers. For example, the filter expressions might use normal arithmetic expressions, or we might create a special graphical programming language.
Figure 4. (a) The Gilt window that allows the designer to control how values for widgets are filtered. Many of the fields are filled in by Gilt as the designer demonstrates the desired behavior. The Unfiltered Value shows the value as currently provided by the widget before any filtering. The Filter Expression is the Lisp expression to filter the value. The designer can hit the Use Value of Object button to insert a reference to the value of a selected object. The default filter simply copies the original value. The Resulting Filtered Value field shows the final value after the filtering. This field can be edited to show the transformation for the current widget by example. (b) shows the filter expression after a function has been selected from a menu and the widget references have been filled in. (c) shows the additional windows that appear to confirm the transformations that are inferred for the widgets that are referenced in (b).
(a)
(b)
(c)
For enabling and disabling widgets, similar techniques are used. One of the most common dependencies is to enable and disable widgets based on the values of other widgets. To specify this, the designer can operate a widget to have the appropriate value, then enable or disable the dependent widget, and Gilt will fill in the values for the Change my Enable expression. In trying to guess appropriate control expressions for dependent slots, Gilt knows about check boxes and radio buttons being on or off, text fields being empty or having a value, and numbers being zero or non-zero. In addition, if the Change my Enable window is for a set of selectable items (such as a menu or a panel of buttons), the controlling widget can return a list of values, each element of which controls an item.
All the other properties of widgets can be controlled in the same way as enabling. Widgets can be made to be visible and invisible by bringing up a Change my Visible window. Similar windows control properties such as color and font.
To edit the value of any of the filter expressions for a widget, the designer can simply select the widget and bring up the appropriate Control... or Change my... window. The designer can then edit the text of the expression. Alternatively, if the user demonstrates new transformations, these will replace the existing ones as appropriate.
The demonstrational aspect of Jade is that it will automatically generate the rules that control the layout from examples of the desired picture. An interactive editor is being created that will allow the designer to show the system how the interface should look. This part of the system is still under development.
The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the U.S. Government.
Intended Users: Primarily programmers
Feedback about capabilities and inferences:
Lapidary: Shows the inference in dialog boxes.
C32: Dialog boxes show the inferences, and the user must hit the "OK" button.
Gilt: When inferring a graphical style, the style is immediately applied, so the object's appearance will change. Also, the style name is displayed. For filter expressions, the inferred Lisp code is displayed.
Program constructs: Variables
Table of Contents Watch What I Do