Content area
Abstract
High satisficing and nonresponse behaviors can be greatly detrimental to survey quality; therefore, web surveys must be designed with great care to maximize response quality. One derivative design is grid-style questions, which are extensively used in web surveys. However, grid questions are usually perceived as complex and difficult. Presenting grid questions at the end of the survey—where respondents are presumably more fatigued and less motivated—is expected to induce more satisficing behaviors and poorer data quality than grid questions located at the beginning of the survey, when respondents are more likely to be fully engaged and motivated. Furthermore, there may exist inconsistencies in the presentation of grid questions across different platforms; a factor that adds to quality non-equivalence. In using smartphones and tablets, respondents tend to exert additional effort to answer grid questions, which is likely to deteriorate the data quality when compared to those using computers.
This research focuses on an experiment designed to explore whether satisficing behaviors and nonresponse behaviors differ on grid questions, depending on: 1) the device type used by respondents, 2) the location of the grid questions within the survey questionnaire, and 3) the interaction between grid location and type of device, while accounting for respondent factors.
To investigate these research questions, this dissertation evaluates the paradata and demographic data, in combination with substantive responses, from a fully randomized experiment conducted on the web component of the Gallup Panel. Evidence from this analysis indicates that the further in the survey the grid questions were located, the following were more likely to occur: increased straight-lining behaviors; reduced response times; more item nonresponse; less differentiated answers; and a higher incidence of breakoff behavior, from respondents. This study further reveals that questions in a grid format displayed poorly on smartphones. The negative effects of grid questions on data quality were exacerbated when those same grid questions were presented on smartphones rather than on computers or tablets. The results suggest that displaying grid questions at the very beginning of the survey and on either computers or tablets is the best way to minimize suboptimal responses on grid questions.
