Richard G. McDaniel

Building Whole Applications Using Only Programming-by-Demonstration Degree Type: Ph.D. in Computer Science
Advisor(s): Brad Myers, James Morris
Graduated: May 1999

Abstract:

Present day tools require a developer to learn complex programming languages to build modern interactive software. However, the effort used to create such software such as games, simulations, and educational software would be better spent not in programming the application's logic, but in providing the engaging background, artwork, and gameplay that keeps users interested. Artists and educators who could produce good material for these applications are often unable to program. Thus, providing a tool that does not require programming skill but still allows a wide range of behavior to be created is desirable.

This thesis has developed techniques that allow a developer to build complete applications without using a written programming language. The foundation of these techniques is programming-by-demonstration in which a developer shows the system what to do by presenting examples of the desired behavior. The techniques include innovations both in interaction and inferencing.

The interaction techniques developed in this research are designed to give developers the ability to express the application's behavior with appropriate detail. The developer can draw guide objects in the scene that are hidden to the user. These objects express important relationships and hold the application's data. The developer can also use cards and decks to represent collections of data as well as to provide randomness. The developer can give the system hints by pointing out which objects a behavior relies upon. Also, the techniques include an efficient method for demonstrating examples called nudges. Nudges allow the developer to revise behaviors as problems are discovered, and they allow the developer create negative examples as easily as positive examples.

The inferencing techniques allow a broader range of behavior to be generated automatically than prior PBD systems allowed. By using decision tree learning, the system can automatically infer conditional expressions where the objects that are referenced by a behavior do not have to be affected by that behavior. Another technique based on recursive difference methods allows the system to form and revise arbitrarily long chains of expressions.

These techniques are implemented in a tool called Gamut which has been tested in a usability study to gauge the techniques' effectiveness. The test showed that Gamut can be successfully used by nonprogrammers to build complicated behaviors. Gamut provides a rich medium for expressing a developer's intentions with sufficient inferencing power to create interactive software while requiring minimal programming expertise.

Keywords:
End-user programming, user interface software, application builders, programming-by-example, programming-by-demonstration, inductive learning, Gamut