The goal of a graphic designer or multimedia designer is often to create a graphic artifact that serves to communicate some information to a viewer. The designer must encode his or her communication intentions in some structured representation of media objects that reside in a graphical editor.
The problem is that the current generation of graphic editors keeps only the results of design choices, that is, only the arrangement of media elements the designer has chosen. This is fragile, because if any of the assumptions responsible for the current set of graphic choices changes, the design may no longer be valid as a solution to its communication goals. If, for example, the amount of space on the screen or page changes, the correct response to such a change is not simply to stretch or shrink the design, but to change decisions about typeface, aspect ratio or placement of elements. The designer must redesign the artifact "as if" the new situation had been present from the beginning, while preserving as much of the original reasoning [not necessarily the original data!] as possible. This process is tedious and error-prone, and it is here that computer assistance could be of real help to the designer.
Culturally, designers are comfortable with direct manipulation of the visual elements of a design, and not comfortable at all with textual input of abstract concepts. So a possible approach might be to use concrete visual examples as the representation of abstract concepts in the domain of visual design.
1. Example Yellow Pages advertisement
An important component of the representation activity that knowledge engineers do involves establishing the structure of the concept domain. Part-whole hierarchies indicate relationships between components of the design. An inheritance hierarchy links more specific to more general concepts. These are expressed in terms of a textual object-oriented description language, of which a sample appears below.
(defob graphic-ad-component () (ad-component graphic))
((advertiser (a 'graphic-ad-component))
(body (a 'graphic-ad-component))
(information (a 'graphic-ad-component)))
((advertiser (a 'graphic-ad-component))
(address (a 'graphic-ad-component))
(phone-number (a 'graphic-ad-component)))
Writing such descriptions is problematic for graphic designers as a means of communicating the conceptual information in a design. The syntax is difficult to remember, and the connections between the concepts and the graphic elements are not immediately apparent.
Callouts are textual meta-graphic labels introduced into a picture to serve as visual pointers to the conceptual function of graphic elements in a design. In Figure 2, the user annotates the yellow pages ad by attaching graphic labels to each element, using the same operations provided by the graphical editor. Note that labels themselves can be labeled, to produce multilayered structures.Figure 2. A meta-graphic annotation of a Yellow Pages ad
The task of the meta-graphic editor is to build associations between the conceptual structure of the design and the primitive graphic elements, such as lines, text, or bitmaps, and attributes of those elements, such as fonts and color. Figure 3 illustrates the graphic structure for our example ad, seen through an inspector [or browser] interface.
Figure 3. Purely graphic object structure for Yellow Pages
By simply noting the position of the labels and lines in the meta-graphic annotations, a very simple visual parser can produce a hierarchical structure that corresponds to the annotations, and build cross-links between the parts of the conceptual structure and the graphic structures that serve as examples of them. Annotations allow the designer to visually verify that the knowledge structures have a clear correspondence with the graphic structure.
4. The conceptual structure of the Yellow Pages ad
Figure 4 shows the conceptual structure obtained from the meta-graphic annotations.
In the Yellow Pages, another format, -- the listing ad -- contains only text, not graphics, and displays the essential information of the name of the advertiser, address and phone numbers. Typeface and alignment relations indicate the conceptual function of each component, as did placement relations in the display ad format. See Figure 5.Figure 5. Yellow Pages listing ad
To teach the system how to make listing ads from display ads, the designer can transform the Harnum Crane display ad into the format of the listing ad shown above by modifying the properties of the graphic elements. The system records the modification steps, and generalizes them according to the relations specified by the annotations.
6. An interactive change to a typeface
For example, the interactive type style change of Figure 6 to the text element representing the address "869 Woburn St. Wilmington" in the Harnum Crane ad would be represented in the recorded procedure as the code,
(setf (font-style (address display-ad)) :bold) referring to that element by its annotated function in the design rather than by its graphic structure. The recorded procedure would then take a display ad as an argument, and produce an analogous listing ad. We supply a new example by annotating a new ad with a similar conceptual structure (see Figure 7). We can take advantage of our original annotation by copying most of the annotation meta-graphics from the original example, and adjusting them as necessary to fit the new example.
7. Annotation of another display ad
The result of this (Figure 8) is a conceptual structure for the Mass. Heavy Equipment ad analogous to that of the Harnum Crane ad.
8. Conceptual structure of second display ad
The representation of the conceptual structure of the display ad allows the system to automatically generate the corresponding listing ad shown in Figure 9.Figure 9. Automatically generated listing ad
Some care must be taken both in how the user demonstrates the procedure, and what aspects of the graphic operations the system decides to record. If the user centers a graphic element by eye rather than by calling the centering function of the graphic editor, the system will record a move action, and it might not recognize the intent to center the object. Such a recorded procedure might not have the desired centering effect in subsequent examples. Some systems such as Peridot, contain rules for comparing graphic states that would be capable of recognizing centering relations. Similarly, the system must not record the operations too literally, for example by embedding absolute cursor positions in the code.
This paper has presented a scenario of how concrete visual examples can serve as a means for communicating design knowledge in a graphical editor. Annotations serve as the graphical counterpart of variables in programming. Systems that can record user actions and generalize on graphical annotations can go beyond the current generation of graphical editors in providing support for automatically generating design alternatives based on the functionality of components of the design. Programming by demonstration systems can help visually oriented users access the procedural power of computers, which would otherwise require programming in a textual programming language.