Work-in-Progress: Automation in Undergraduate Classes: Using Technology to Improve Grading Efficiency, Reliability, and Transparency in Large Classes

Abstract

Automation in Undergraduate Classes: Using Technology to Improve Grading Efficiency, Reliability, and Transparency in Large ClassesLarge undergraduate classes offer many challenges relating to scale. This paper describes a suiteof automated computer tools developed to assist with these challenges, specifically those relatingto grading and performance analysis for either individual students or classes as a whole. Whilethe computer tools developed are independent of any Learning Management System (LMS), theycould easily be adapted to operate more closely with an LMS in other academic environments.The suite of tools in question allow for automated digital rubric generation, collection fromstudents, return to students, and most notably, analysis. Features include the ability to condenseseveral files submitted by one student into a single PDF for review, the ability to executesubmitted code in three programming languages (Python 3, MATLAB, and ANSI C) whilecapturing the output into a PDF, and the ability to track error conditions such as late submissionand incorrect file names and automatically assign penalties.Statistical reports are generated for each assignment automatically, providing a window intostudents’ performance and possible areas of concern. Automated warnings alert the teachingteam to potential errors in grading, equity issues (such as one section of the class performingsubstantially better or worse than another) or opportunities for improvement in the academicprocess (such as rethinking the pedagogy relating to specific ideas or areas that prove broadlytroublesome). These reports streamline instructor workflow and allow for much deeper insightsinto student performance than time would normally allow.The suite of tools was implemented using Visual Basic for Applications (VBA), Python 3, andMySQL databases. The implementation of these automated tools was inexpensive and providedmany benefits to the instructors and graders in terms of convenience, time saved, graderaccountability, process reliability, and enabling new diagnostic capabilities. Furthermore, costsavings were realized from reduced grader time and from almost eliminating the use of paper tooffset the cost of developing the tools. This paper presents details on each of the tools developedas a part of this effort, results of the adoption of the tools in a large first-year class, the potentialuses of similar tools in other venues, and avenues for future work and development.