Paper-and-pencil questionnaires analysis: a new automated technique to reduce analysis time and errors

Abstract

Abstract

Background and Objective Questionnaires are essential tools in many scientific fields, including health and medicine. However, the analysis of paper-and-pencil questionnaires is time consuming, source of errors and expensive, limiting its use in large cohort studies. Computer-based questionnaires might be a valuable alternative but they may introduce bias, especially for sensitive questions, and they require programming skills. The aim of this study is to develop a reliable and adaptable open-source technique (i.e. LightQuest) to automatically analyse various types of scanned paper-and-pencil questionnaires with closed questions, including those with inverted scale.

Methods To evaluate the usefulness of LightQuest, the time needed for 7 experimenters for manually code 10 sets of 4 frequently used questionnaires and the number of errors (i.e. reliability) were compared with the time and errors their made using LightQuest.

Results LightQuest was twice as fast as the manual analysis, even though the time to create the reference model was taken into account (933s vs. 1935s, t(2)=8.81, p<0.001). Without model creation, the reduced analysis time was more pronounced, with an average of 2.77s.question^-1 for the manual technique versus 0.55s.question^-1 for LightQuest (t(2)=22.5, p<0.001). Moreover, during correction of the 5180 questions performed by the 7 experimenters, LightQuest made a total of 2 errors versus 46 with the manual technique (q(2)=4.53, p<0.05).

Conclusion LightQuest demonstrated clear superiority both in terms of time and reliability. The script of this first open-source technique, which does not require programming skills, is downloadable in supplemental data and may become an asset for all studies using questionnaires.

Competing Interest Statement

The authors have declared no competing interest.

*

Abbreviations

AF

Automatic with Feedback

AnF

Automatic with no Feedback

GIGO

Garbage In/Garbage Out

MCQ

multiple choice question

PANAS

Positive and Negative Affect Schedule

POMS

Profile of Mood States

RSES

Rosenberg Self-Esteem Scale

STAI

State-Trait Anxiety Inventory