Nathaniel S. Borenstein The Design and Evaluation of On-line Help Systems Degree Type: Ph.D. in Computer Science Advisor(s): James Morris Graduated: May 1985 Abstract: On-line help is a vital part of nearly every computer system of any significant size, yet it is poorly understood and generally poorly implemented. The primary goal of this research was to discover as much as possible about how on-line help systems should be built. Several results are presented in the thesis. First, the feasibility of easily building more general and powerful help systems than are commonly available is demonstrated. A prototype system, including nearly all the features included in any real-world help system, and integrating them in a manner not found in any such system, was built in just a few months without any spectacular tricks of implementation. The prototype system runs well and quickly, even with a very large database of help information, so that there is simply no reason not to expect practical help systems to live up to its standards. Second, controlled experiments comparing various alternative help systems were conducted. These experiments suggest that the quality of help texts is far more important than the mechanisms by which those texts are accessed. The experiments also suggest that, despite the well-documented fact that people read and comprehend better from printed texts than from computer screens, it is at least possible to compensate for this fact through sophisticated help access mechanisms. Thus, on-line help with no printed manual may be at least as useful as a more traditional manual-based system, and there are reasons to suspect that it can be even better. Additional experiments, using a simulated natural language interface, cast doubt on the usefulness of natural language in a help system. In addition to the concrete results summarized above, the thesis makes several other contributions. A general taxonomy of on-line help systems is developed, and a survey of the literature on help systems relates existing help systems to that taxonomy. Further, the design and evaluation of help systems is considered as an example of the more general problem of designing and evaluating user interfaces. The methodology facilitates the evolution of such interfaces with a minimum of attention to the details of implementation and experimentation. Finally, a potpourri of practical, sometimes anecdotal information likely to be of interest to future help system designers is collected in a practitioner's summary, and related topics ripe for further research are described in a researcher's agenda.
