Non-response bias on Web-based surveys as influenced by the digital divide and participation gap

Abstract

Higher education scholars, policy makers, and administrators know little about the experiences of undergraduate students who matriculate with minimal experience with technology. It is often assumed that all students, particularly traditionally-aged students, have significant experience with, knowledge of, and comfort with technology. Although that assumption is correct for many students, it is false for others. Despite the enormous increase in the use of Web-based assessment surveys and the increasing importance of accurate assessment and accountability data, those efforts may not be collecting adequate and accurate data about and from all students.

This study explores the non-response bias of first-year undergraduate students on a self-administered Web-based survey. First, data were collected with a supplemental survey added to the Beginning College Survey of Student Engagement (BCSSE). K-means clustering was used with this newly constructed Internet Access and Use survey to classify students according to their Internet access and use experiences. Second, demographic data from BCSSE and the Internet access and use data were included in a logistic regression predicting response to the subsequent National Survey of Student Engagement (NSSE).

The Internet Access and Use instrument proved to be a viable way to classify students along lines of their previous Internet access and use experiences. However, that classification played no meaningful role predicting whether students had completed NSSE. Indeed, despite its statistical significance the final logistic regression model using provided little meaningful predictive power.

Generalizing the results of this study to all Web-based surveys of undergraduate college students with random or census sampling indicates that those surveys may not introduce significant non-response bias for students who have had less access to the Internet. This is particularly important since that population is already vulnerable in many ways as being disproportionately composed of first-generation students, underrepresented minority students, and students with lower socioeconomic statuses. This reassures assessment professionals and all higher education stakeholders that cost- and labor-efficient Web-based surveys are capable of collecting data that do not omit the voices of these students.