James A. Landay

Interactive Sketching for the Early Stages of User Interface Design Degree Type: Ph.D. in Computer Science
Advisor(s): Brad Myers, James Morris
Graduated: December 1996

Abstract:

Current interactive user interface construction tools are often more of a hindrance than a benefit during the early states of interface design. Theses tools are hard to use and they encourage designers and evaluators to focus on the wrong issues, such as color, fonts, and alignment, at this early stage. Most designers prefer instead to sketch early interface ideas on paper. However, designing on paper also has many drawbacks. Paper-based designs are hard to edit and cannot easily be tested with users. This dissertation describes the design, implementation, and evaluation of an interactive sketching tool called SILK that overcomes these problems and combines many of the benefits of paper-based sketching with current electronic tools.

SILK allows designers to quickly sketch an interface using an electronic pad and stylus. SILK preserves the important properties of pencil and paper: a rough drawing can be produced quickly and the medium is flexible. However, unlike a paper sketch, this electronic sketch is interactive and can easily be annotated and modified using editing gestures.

SILK recognizes user interface widgets and other interface elements as they are drawn and gives the designer feedback so that these inferences can be changed. Recognized interface elements have built-in behaviors and thus these elements can be exercised in their sketchy state. For example, the "elevator" in a sketched scrollbar can be dragged up and down. Unfortunately, the behavior of individual widgets is insufficient to test a working interface. SILK's electronic storyboards allow the illustration of the dynamic behavior between interface elements, such as a dialog box appearing when a button is pressed. A designer creates a storyboard by first copying sketched screens to the storyboard window and then drawing transition arrows on the screens. The arrows specify which objects cause transitions to which screens when the end-user clicks on the objects. When the designer is satisfied with this early prototype, the system can transform the sketch into an operational interface using real widgets in a specified look-and-feel.

An evaluation of SILK, using professional and student designers, showed that it was an effective tool for both early creative design and for communicating the resulting design ideas to other members of an engineering team. By supporting the early phases of the interface design life cycle, electronic sketching can both ease the prototyping of user interfaces and improve the interfaces that are eventually produced.

Thesis Committee:
Brad A. Myers (Chair)
James H. Morris (Co-Chair)
Roger B. Dannenberg
Daniel Boyarski (CMU Department of Design)
Thomas P. Moran (Xerox Palo Alto Research Center)

James Morris, Head, Computer Science Department
Raj Reddy Dean, School of Computer Science

Keywords:
User interfaces, design, sketching, storyboarding, gesture recognition, interaction techniques, pen-based computing, SILK