Comparability of Online and Paper/Pencil Mathematics Performance Measures

Abstract

The purpose of this study was to investigate the relationship between student mathematics performance of 4th, 8th, and 11th grade students in Nebraska and the mode of test administration, online and paper-pencil. Schools were allowed to select the mode of test administration for their school with some exceptions for students needing accommodations. This resulted in four test groups, namely students taking the online tests in schools selecting paper or online assessments along with students taking the paper-pencil tests in schools selecting paper or online assessments. Since the students in the study were clustered within schools, the data from the Nebraska State Accountability (NeSA) mathematics assessment (2010) were analyzed using hierarchical linear modeling (HLM) for the three grade level groups. The use of HLM allowed the researcher to adjust for and model the dependence of students clustered within schools. Both school level and student level variables were included in the model to control for sample differences between test modes. Student variables such as gender, students eligible for free and reduced lunch (FRL), students receiving special education services (SPED), English language learners eligible for support (ELL), and the seven ethnicities were incorporated in all three grade level models along with school level percents of FRL, SPED, ELL, and ethnicity. In each of the three grades, the results failed to indicate a significant effect on mathematics performance between students taking paper-pencil tests in schools selecting paper-pencil assessments (P/P) and students taking online tests in schools selecting online assessments (O/O) (p = 0.0518). Significant differences were noted between results for P/P test takers and the results for those students taking paper-pencil tests in schools selecting online assessments (P/O) (p < 0.0001). Likewise significant differences were found in results of students taking P/P tests and the results of students taking online tests in schools selecting paper assessments (O/P) (p < 0.0001). The state policy makers have considered the expansion of online testing throughout the state. The advantages of computer testing with the assurance of comparable performance on both test modes need to be considered before moving forward with online assessments in Nebraska schools. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone [phone]. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]