Vos Data Manager: Providing Immediate Feedback On Teaching Effectiveness

Abstract

New classroom assessment technologies produce large amounts of data. However, providing useful information to instructors, who may not be familiar with coding or analytic methods, remains challenging. For the past several years, the VaNTH (Vanderbilt Northwestern Texas Harvard-MIT) Engineering Research Center for Bioengineering Educational Technologies has employed the VaNTH Observational System (VOS). VOS allows minute-by-minute capture of both student and instructor activities during a classroom session via handheld personal data assistants (PDAs) and has been previously described. However, generating useful information for instructors based on this data is time-consuming, and instructors often wait days or weeks for the appropriate reports to be generated on a case-by-case basis. Furthermore, multiple files resulting from these methods have been difficult to organize and maintain.

The VOS Data Manager has been developed to automate the process of importing, organizing, and analyzing VOS data. The import process checks the underlying data for validity, automatically correcting simple artifacts and identifying more complex ones for manual correction. The entire import process, including artifact correction, takes about five minutes for a one-hour classroom session. Once data has been imported, a variety of reports can be generated within seconds that detail instructional methods, student engagement (or lack thereof), and other classroom activities over a particular session. Additional reports show how one session compares to another, or to an average of other sessions given by the same or different teachers. Instructors can now receive feedback within minutes after a class ends, and researchers can more easily compare data across different sessions, instructors, and/or coders.

The VOS Data Manager runs on any computer equipped with Excel (Microsoft Corp., Redmond, WA). Excel was chosen because of its wide install base, familiarity to most instructors and educational researchers, and the fact that it’s built-in macro language, Visual Basic for Applications, provides a high degree of functionality and compatibility across recent versions of the Windows (Microsoft) and MacOS (Apple Computer, Inc., Mountain View, CA) platforms. This technology facilitates rapid development and dissemination of the VOS Data Manager, allowing VOS researchers to provide immediate and useful feedback to instructors following a classroom session.

Proceedings of the 2004 American Society for Engineering Education Annual Conference & Exposition Copyright © 2004, American Society for Engineering Education