Content area
Abstract
While traditional computer interfaces based on the mouse and keyboard are ubiquitous, they are ill suited to many common application domains. This is particularly true in education, where recent research suggests that students perform better when instructional interfaces are more similar to work practice. Thus, the goal of our work is to create computational techniques and user interface design principles to enable natural, pen-based tutoring systems that scaffold students in solving problems in the same way they would ordinarily solve them with paper and pencil. In our work, we have focused on interfaces suitable for a "pentop computer," a writing instrument that is used with special dot-patterned paper, and that has an integrated digitizer and embedded processor. A pentop is capable of producing dynamic output in the form of synthesized speech and recorded sound clips.
Accurate shape recognition is an essential foundation for developing pen-based interfaces. We created a trainable, multi-stroke recognizer that is insensitive to orientation, non-uniform scaling, and drawing order. Symbols are represented internally as attributed relational graphs describing both the geometry and topology of the symbols. Symbol recognition is accomplished by finding the definition symbol whose attributed relational graph best matches that of the unknown symbol. We developed five efficient approximate matching techniques to perform the graph matching.
To explore instructional and interface design issues, we created Newton's Pen, a pentop-based statics tutor. This system, which is intended for undergraduate education, scaffolds students in the construction of free body diagrams and equilibrium equations. Newton's Pen employs a finite state machine architecture that effectively models the student's problem-solving progress, thus serving as a convenient means for providing context-sensitive tutorial feedback. User studies suggest that Newton's Pen is an effective teaching tool, and that students are satisfied with the interface.
A key issue in the design of pentop interfaces is how to provide effective feedback to the user. To explore this issue, we developed PaperCAD, a system that enables users to query geometric information from printed CAD drawings. PaperCAD employs two methods of feedback: audio feedback with an adjustable level of conciseness, and a PDA that provides a video display of the portion of the drawing near the pen tip. This system also employs a novel technique that uses a hidden Markov model to correct interpretation errors in handwritten equations. Results of a user study suggest that users are highly satisfied with the interface and prefer it to a traditional WIMP interface.