Introduction
The assessment of population health using health interview surveys is an established method in many countries and is a cornerstone of health reporting, health policies, and health sciences. However, epidemiological studies have shown decreasing response rates since the 1990s [1-3]. The use of mixed-mode health interview surveys offers respondents various data collection modes and can increase the response rate, improve sample composition, and reduce overall costs [3,4]. Currently, there is considerable interest in using Web-based health survey interviews because of lower costs and faster data availability. Web-based surveys are increasingly becoming standard [5], and they are frequently combined with other modes in mixed-mode designs [6]. However, the use of different survey modes may increase the risk of measurement error (mode effects) [5].
Mode effects are systematic distortions caused by different survey modes or interview situations [5]. They often arise when there are large methodological differences in the survey situation (self-administered questionnaire vs interviews) or the communication channel (auditory vs visual) [3]. Such differences are minimal between self-administered paper-based questionnaires (SAQ-paper) and self-administered Web-based questionnaires (SAQ-Web)—both are conducted without an interviewer and both use visual perception. For this reason, these 2 self-administered modes (SAQ-Web and SAQ-paper) are considered mode equivalent [4,7,8]. Mode equivalence is shown if an individual gives the same response to the same question or instrument administered through 2 different modes, leading to the same results [9]. For example, research has shown no differences between the 2 data collection modes in prevalence rates of diseases among adult populations [10,11] or in reported health behaviors among adolescents [12].
However, researchers have discussed mode effects for sensitive topics. Web-based responses are associated with both anonymity and greater individualization. Consequently, SAQ-Web participants are not affected by social desirability; rather, they are less orientated toward social norms. Therefore, SAQ-Web mode yields the most honest reports, especially compared with interview modes [13,14]. Furthermore, differences have been found between the 2 self-administered modes, for example, in political attitudes [15], reporting of sensitive sexual behaviors [16], or adolescent risk behavior [17]. However, there is high consistency of responses across modes, with only a few respondents taking advantage of the greater privacy of the Web mode [16]. Hence, possible mode effects should be investigated before changing or adding modes to existing health surveys. In ongoing longitudinal studies, changing the mode or offering a second mode may risk time-based comparability.
The German Health Interview and Examination Survey for Children and Adolescents (KiGGS) is a nationally representative health interview and examination survey of children and adolescents in Germany [18,19]. It is part of the nationwide health monitoring system administered by the German national public health institute (Robert Koch Institute) [20,21]. KiGGS obtains representative cross-sectional information on German children and adolescents aged 0-17 years at regular intervals. Additionally, based on the first cross-sectional sample (KiGGS baseline; 2003-2006), a KiGGS cohort has been implemented. The baseline respondents are being followed throughout their life course into adulthood [21]. The survey involves physical examinations and tests, as well as laboratory analysis of urine and blood parameters. All the parents and adolescents aged 1117 years completed paper-based questionnaires [20]. The first follow-up, KiGGS Wave 1 (2009-2012), was conducted using telephone interviews of parents and adolescents [22]. KiGGS Wave 2 (2014-2017) involved a health interview and examination, continuing the baseline concept [23]. The aim of the KiGGS survey is to provide current data on population health, health determinants, and the utilization of health care services. In addition, information is gathered about the incidence of disorders as well as trajectories of multiple health indicators throughout the life course. The data are widely used in national health reporting, health policies, and public health research.
When planning population-based (health) studies like KiGGS, the survey design must minimize total survey error [24,25]. In addition to lower data quality owing to measurement errors such as mode effects, the total survey error comprises different kinds of systematic errors—an undervalued sample size leads to imprecise estimates (sampling error) and the composition of the sample might be different from the target population (coverage error) owing to errors in the sampling procedure or because of systematic nonresponse (nonresponse bias). All these aspects were examined in a methodological pilot study as part of the KiGGS Wave 2 pretest. The pilot study aimed to compare 3 mixed-mode survey designs using Web- and paper-based questionnaires with a single-mode SAQ-paper design in terms of response rates, sample composition, data quality, and effort [26]. The study also explored whether estimates of health indicators differed among the survey designs and data collection modes. This study focused only on the second aim of the pilot study and addressed 2 research questions:
Methods
Study Design
The methodological pilot study used a sample of children and adolescents registered in the local resident registries of 20 municipalities in 5 federal states of Germany, covering urban and rural areas as well as the eastern and western regions of the country.
Data were collected using SAQ-Web or SAQ-paper methods. All selected individuals were invited by mail to participate in the study. They were sent a cover letter with the invitation to participate, information about the study and data privacy, and an informed consent form. Depending on the allocated mode, the invitation comprised a username and password for participation through the Web option along with a paper questionnaire for those allocated to the concurrent mixed-mode design, only a paper questionnaire in the single-mode design, or only the access data for the online questionnaire in the sequential mixed-mode design. The SAQ-Web questionnaire was only optimized for desktop computers. A reminder was sent by mail to respondents who had not replied within 3 weeks of the initial invitation. Participants who did not respond to the reminder were telephoned up to 5 times 4 weeks after the initial invitation. As an additional motivation for prospective participants, each parent and adolescent who had completed a questionnaire received a shopping voucher to the value of €10. The methodological pilot study strictly adhered to the data protection regulations set out in the German Federal Data Protection Act. Participation in the study was voluntary. All parents and participating adolescents were informed about the study’s aims and content, as well as data protection, and they provided informed consent. Following the strict data privacy protocol, prospective participants between the ages of 11 and 17 years received their questionnaires only after their parents provided consent.
Different questionnaires were used for different age groups. Main health indicators were included on the health questionnaires for parents of all age groups (0-17 years), and self-report data for main health indicators were obtained from adolescents aged 11-17 years. To reduce the risk of mode effects, the 2 questionnaires were designed to be as similar as possible and contained the same wording for the questions and response categories. On the basis of the unified-mode design [27], the wording and formatting of questions and response categories were standardized. To help participants visually distinguish single-choice questions from multiple-choice questions, all survey modes used the same checkbox design. Single-choice checkboxes were round, whereas multiple-choice checkboxes were rectangular. Additionally, multiple-choice questions included the instruction “Multiple entries are possible.” For filter questions, Web-based questionnaires were optimized with filter skips whenever the perceivability of the questions was not impaired. Plausibility checks and ranges were defined for the Web-based questionnaire. Additionally, soft prompting was programmed into the Web-based questionnaire to reduce item nonresponse. These differences were used to capitalize on the advantages of the Web mode for better data quality, and they were the only mode-specific design differences. Detailed information of the survey design and other technical aspects of the Web-based part of the survey are described in a “Checklist for Reporting Results of Internet E-Surveys” [28] (Multimedia Appendix 1).
As shown in Figure 1, a gross sample of 11,140 children and adolescents was randomly allocated to four survey designs:
There were no statically significant differences in the (gross) sample composition across the 4 design groups in terms of known sample characteristics, such as age, sex, municipality size, region, or respondent citizenship, which were obtained from local registries.
The combined response rate for all survey designs was 38.43% (n=4032), following the internationally used Standard Definitions of Outcome Rates for Surveys of the American Association for Public Opinion Research (AAPOR Response Rate 2) [29]. There were no significant differences in response rates among the concurrent mixed-mode design, the sequential mixed-mode design, and the single-mode control group design. However, there was a significantly lower response rate in the preselect mixed-mode design. Detailed comparisons of response rates, sample compositions, data quality, and efforts among the different survey designs have been published previously [26].
Database
For this study, only survey design groups using the same version of the questionnaire were included, with 3468 completed parent-reported health questionnaires for children and adolescents aged 0-17 years and 1194 questionnaires completed by adolescents aged 11-17 years. A response was defined as one completed health questionnaire from either parents or children. Hence, a valid response did not require both parents and children to complete all requested questionnaires. To answer the first research question regarding mode equivalence across the different survey designs, we compared the single-mode control group with each of the 3 mixed-mode groups. To answer the second research question regarding mode equivalence between the 2 data collection modes, data from all survey designs were pooled (Table 1).
Sociodemographic Characteristics of Responding Parents and Adolescents by Survey Design and Data Collection Mode
Analyzed Sociodemographic Characteristics
The sample compositions of participating parents and adolescents were described by various sociodemographic characteristics separately by survey design and data collection mode. The variables examined included individual adolescent characteristics (age, sex, migration background, and highest level of education reached or aspired); parental characteristics (age, marital status, and participating parent); location (municipality size and region [East vs West Germany]); and household properties (education level and net household income). Household education level was measured using the Comparative Analysis of Social Mobility in Industrial Nations [30]. Household income was assessed using a question on household monthly net income.
Enlarge this image.Figure 1. Study design of the methodological pilot study. SAQ-paper: self-administered paper-based questionnaire; SAQ-Web: self-administered Web-based questionnaire.
Table 1. Cases used in this study.
| Questionnaire type | Survey design | Mode group | |||||||||||
| Single-mode design | Sequential mixed-mode design | Concurrent mixed-mode design | Preselect mixed-mode design | ||||||||||
| SAQ-papera | SAQ-paper | SAQ-Webb | Total | SAQ-paper | SAQ-Web | Total | SAQ-paper | SAQ-Web | Total | SAQ-paper | SAQ-Web | Total | |
| Parent-completed health questionnaires | 970 | 488 | 366 | 854 | 837 | 109 | 946 | 465 | 233 | 698 | 2760 | 708 | 3468 |
| Adolescent-completed health questionnaires | 343 | 117 | 152 | 269 | 244 | 46 | 290 | 191 | 101 | 292 | 895 | 299 | 1194 |
aSAQ-paper: self-administered paper-based questionnaire.
bSAQ-Web: self-administered Web-based questionnaire.
Statistical Methods
Differences between the control group and the different mixed-mode groups and between the 2 data collection modes were tested using chi-squared tests.
Mode Equivalence of Health Indicators Between Survey Designs and Data Collection Modes
A wide range of health status indicators and health behaviors for children and adolescents with high public health relevance were analyzed to identify differences between the mixed-mode designs and the single-mode control group, as well as mode differences between SAQ-paper and SAQ-Web.
Analyzed Indicators of Physical and Mental Health
Lifetime diagnoses of asthma, hay fever, atopic eczema, and attention-deficit hyperactivity disorder (ADHD) were indicated by parents. Recurrent pain during the last 3 months was measured using the adolescents’ self-reports. Self-rated health (SRH) and chronic diseases were evaluated by parental report using the Minimum European Health Module questions [31], modified for children. Adolescents also answered the SRH question. Impairments owing to health problems were evaluated with a question from the Children with Special Health Care Needs Screener, which was answered by parents [32]. To define obesity, body mass index was calculated based on self-reported weight and height for adolescents and parent-reported weight and height for children aged 3-10 years. The body mass index cut-offs used in this study were determined by German norms [33].
Child and adolescent mental health problems were evaluated using the parent- and self-report Strengths and Difficulties Questionnaire (SDQ) [34]. An SDQ total difficulties score was calculated for all children and adolescents. Participants with a borderline or abnormal score (based on German norms) [35] were defined as at risk for emotional and behavioral symptoms. Participants with borderline or abnormal SDQ impact scores were defined as at risk for psychosocial impairment.
Analyzed Indicators of Health Care Utilization
As indicators of health care use, pediatrician and orthodontist visits during the past 12 months for adolescents and parent-reported visits to any doctor for children under 11 years were analyzed [36].
Analyzed Measure of Health-Related Quality of Life
Health-related quality of life (HRQoL) was measured using KIDSCREEN-27 for adolescents aged 11-17 years, with 5 subscores for physical and psychological well-being, relationships with peers and parents, and school well-being. Scores were summed and transformed into t values [37].
Analyzed Health Behaviors
Adolescents reported their current smoking status, water pipe consumption during the past 12 months, second-hand smoke exposure [38], and lifetime consumption and current use of screen-based media. Excessive use of screen-based media was defined as more than 2 hours per day [39]. Harmful alcohol use and binge drinking were defined using responses to the Alcohol Use Disorders Identification Test (AUDIT-C) [40].
Following the recommendation of the World Health Organization [41], healthy physical activity was defined as physical activity for at least 60 min per day. Low physical activity was defined as less than 2 days per week of at least 60 min of activity. All questions on physical activity were answered by adolescents aged 11-17 years.
Statistical Methods
We calculated prevalence rates for dichotomous health indicators and mean values for HRQoL (a scale outcome) by survey design and data collection mode. We compared these values using z or t tests.
Due to the different sample compositions of the SAQ-paper and SAQ-Web groups (see the Results), it was necessary to control for sociodemographic characteristics to identify possible mode effects. Survey modes can differ in selection (different population groups prefer different modes) and measurement (different answers are given by the same person under different modes of administration), so these differences are confounded [42]. Additionally, health status and health behavior differ by sex, education, and other sociodemographic characteristics [43,44]. To eliminate the risk of confounding, we adjusted for sociodemographic characteristics by calculating adjusted prevalence rates using predictive margins [45] based on logistic or linear regression models with sociodemographic factors as covariates. To analyze indicators based on parental reports, we included child attributes (age, sex, and migration background); parental attributes (relationship to the child, age, and marital status); household attributes (education and income); and regional attributes (region and municipality size). Adolescents’ reports were adjusted by child attributes, including the highest level of education completed, as well as household attributes and location. The mode of data collection was another covariate used to identify adjusted prevalence for each mode. Differences were tested using z or t tests.
For the survey design comparison, crude as well as adjusted prevalence rates and mean values were calculated. A statistical test for diversity was conducted between the single-mode control group design and each of the 3 mixed-mode designs. Because the survey design samples did not differ in sociodemographic characteristics (see the Results) and there were only marginal differences between the 2 approaches, only the results for crude prevalence rates or mean values without adjustment for sociodemographic characteristics to simplify the presentation of results are shown here.
Handling of Multiple Testing
In total, we analyzed 12 health indicators using the parental sample and 28 using the adolescent sample. For these health indicators, we tested each mixed-mode survey design against the control group. Additionally, we used 2 other statistical tests to identify differences between the data collection modes, using first the crude values and then the adjusted values.
Regarding the research questions, a sensitive approach to detect possible differences (ie, a higher probability of accepting the null hypothesis) is needed. Therefore, we decided to address the statistical problem of multiple testing by correcting the significance level only for the number of tests performed for each health indicator. This was done only for tests comparing the different survey designs. We used the Bonferroni correction method to neutralize the accumulation of α-error [46], using an adjusted significance level of P < .02 to examine differences between the mixed-mode survey designs and the single-mode control group. For the comparison of data collection modes, a significance level of α=.05 was used.
Results
Sociodemographic Characteristics of Responding Parents and Adolescents in Different Survey Designs and Different Data Collection Modes
Responding Parents
There were no statistically significant differences in sample composition between the mixed-mode survey designs and the single-mode control group for participating parents. However, the sample sociodemographic characteristics differed significantly between data collection modes (Multimedia Appendix 2). Parents responding online were more often married and had higher household education levels, higher incomes, and younger children than those who responded to the SAQ-paper. More fathers responded via the Web-based questionnaire than in the paper-and-pencil group. There were no significant differences in migration background, parental age. P values close to significant level are found for region of residence (P=.08), municipally size (P=.05) or child’s sex (P=.06).
Responding Children and Adolescents
For the responding children and adolescents (aged 11-17 years), there were no statistically significant differences in sociodemographic characteristics between the different survey designs, but adolescents responding online were more often male, had reached or aspired to reach higher levels of education, and were more likely to live in households with higher education and higher income, compared with adolescents who responded to the SAQ-paper (Multimedia Appendix 3).
Mode Equivalence of Health Indicators Between Survey Designs and Data Collection Modes
Physical and Mental Health
The analyzed indicators of physical and mental health status showed no statistically significant differences by survey design or data collection mode (Table 2). Across modes and designs, parents reported the same results for SRH, chronic disease, impairment owing to health problems, lifetime prevalence of diagnosed diseases, obesity, and mental health problems and impairment. Adolescent self-reports showed no statistically significant differences in SRH, mental health problems and impairment, or chronic pain.
Health Care Utilization
No differences in the crude or adjusted prevalence rates were found in adolescent-reported 12-month use of pediatric or orthodontic services (Table 3). The crude prevalence of parent-reported 12-month use of any doctor and of pediatric services (for children under 11 years) differed significantly, with more frequent reports of doctor’s visits in the SAQ-Web group. After adjusting for sociodemographic attributes, this difference disappeared. There were no significant differences between the mixed-mode design groups and the control group for any of the analyzed indicators of health care utilization.
Health-Related Quality of Life
HRQoL, measured using the 5 dimensions of the KIDSCREEN-27 for adolescents, was the only indicator scale analyzed. Independent of adjustment, there were no significant differences between the 2 data collection modes (SAQ-paper and SAQ-Web) for any of the observed dimensions (Table 4). Regarding survey design, better psychological well-being was reported in the concurrent mixed-mode design and better relations with parents were reported in the preselect mixed-mode survey design, compared with the single-mode control group. After correcting the significance level for multiple testing, no differences were found by survey design.
Health Behaviors
The crude prevalence of lifetime alcohol consumption (self-reported by adolescents aged 11-17 years), as well as hazardous consumption and binge drinking (based on AUDIT-C reports), showed significant differences between SAQ-paper and SAQ-Web, with higher levels of alcohol consumption reported by online participants (Table 5). Although the differences in hazardous consumption and binge drinking between the 2 modes of data collection disappeared after controlling for sociodemographic characteristics, significantly more online respondents than paper-and-pencil respondents reported that they had consumed alcohol.
There were no differences in other health behaviors assessed (tobacco consumption, physical activity, and media consumption) by survey design or data collection mode.
Table 2. Physical and mental health status of children and adolescents aged 0-17 years by survey design and data collection mode (prevalence rates).
| Physical and mental health status | Survey designa | Data collection modea | |||||||||||||||||||
| Single-mode design | Sequential MMb design | Concurrent MMb design | Preselect MMb design | SAQ-Webc (crude) | SAQ-paperd (crude) | SAQ-Webc (adjustede) | SAQ-paperd (adjustede) | ||||||||||||||
| n (%) | n (%) | Pf | n (%) | Pf | n (%) | Pf | n (%) | n (%) | P | n (%) | n (%) | P | |||||||||
| General health status | |||||||||||||||||||||
| Self-rated health (very good, good)g | 964 (97.6) | 848 (97.1) | .46 | 942 (97.3) | .71 | 693 (97.3) | .65 | 708 (97.0) | 2739 (97.41) | .60 | 708 (96.7) | 2739 (97.48) | .31 | ||||||||
| Self-rated health (very good, good)h | 338 (90.2) | 267 (92.1) | .41 | 286 (87.1) | .22 | 290 (92.8) | .26 | 299 (90.6) | 881 (90.5) | .93 | 299 (90.1) | 881 (90.6) | .81 | ||||||||
| Chronic disease (yes)g | 962 (10.2) | 849 (10.7) | .71 | 941 (7.4) | .03e | 691 (9.6) | .67 | 708 (10.2) | 2735 (9.25) | .47 | 708 (10.5) | 2735 (9.18) | .31 | ||||||||
| Impairment owing to health problems (yes)g | 956 (3.2) | 851 (4.0) | .39 | 937 (3.4) | .83 | 692 (3.6) | .69 | 708 (4.1) | 2728 (3.41) | .40 | 708 (4.5) | 2728 (3.36) | .19 | ||||||||
| Allergies | |||||||||||||||||||||
| Bronchial asthma (lifetime diagnosis)g | 951 (4.5) | 838 (6.1) | .14 | 928 (4.6) | .91 | 675 (5.2) | .54 | 701 (4.6) | 2691 (5.20) | .48 | 701 (5.0) | 2691 (5.10) | .91 | ||||||||
| Neurodermatitis (lifetime diagnosis)g | 951 (15.0) | 839 (16.0) | .59 | 927 (13.8) | .45 | 678 (17.4) | .20 | 704 (17.0) | 2691 (14.98) | .19 | 704 (17.5) | 2691 (14.95) | .12 | ||||||||
| Hay fever (lifetime diagnosis)g | 955 (11.2) | 831 (11.2) | .99 | 931 (10.7) | .75 | 680 (11.5) | .87 | 702 (11.3) | 2695 (11.09) | .91 | 702 (12.0) | 2695 (10.96) | .50 | ||||||||
| Chronic pain | |||||||||||||||||||||
| Headache (recurrent during the last 3 months)h | 331 (35.6) | 260 (37.7) | .61 | 274 (33.9) | .66 | 282 (31.2) | .24 | 299 (32.1) | 847 (35.4) | .30 | 299 (33.8) | 847 (34.8) | .77 | ||||||||
| Dorsal pain (recurrent during the last 3 months)h | 321 (25.9) | 257 (23.0) | .42 | 265 (24.2) | .64 | 276 (27.5) | .64 | 298 (22.5) | 820 (26.2) | .19 | 298 (23.6) | 820 (25.8) | .45 | ||||||||
| Any pain (recurrent during the last 3 months)h | 343 (84.3) | 265 (83.0) | .68 | 288 (81.9) | .44 | 291 (81.1) | .30 | 294 (82.3) | 892 (82.7) | .87 | 294 (82.6) | 892 (82.6) | .99 | ||||||||
| Mental health problems | |||||||||||||||||||||
| Attention-deficit hyperactivity disorder (lifetime diagnosis)g | 824 (5.1) | 716 (4.1) | .33 | 794 (5.0) | .96 | 593 (3.2) | .07 | 595 (3.5) | 2332 (4.67) | .19 | 595 (3.9) | 2332 (4.56) | .45 | ||||||||
| At risk for emotional and behavioral symptomsg | 788 (13.6) | 695 (15.5) | .29 | 754 (13.7) | .96 | 577 (12.7) | .62 | 598 (13.0) | 2216 (14.12) | .49 | 598 (13.9) | 2216 (13.89) | .97 | ||||||||
| At risk for emotional and behavioral symptomsh | 331 (9.7) | 255 (12.2) | .34 | 278 (14.0) | .10 | 283 (14.1) | .09 | 293 (13.7) | 853 (12.0) | .46 | 293 (13.) | 853 (11.9) | .41 | ||||||||
| At risk for impairment following psychosocial problemsg | 820 (18.5) | 719 (17.4) | .56 | 789 (17.2) | .50 | 592 (16.6) | .33 | 600 (16.3) | 2320 (17.80) | .39 | 600 (17.5) | 2320 (17.48) | .99 | ||||||||
| At risk for impairment following psychosocial problemsh | 339 (16.5) | 263 (19.0) | .43 | 286 (16.4) | .98 | 288 (18.1) | .61 | 298 (19.8) | 877 (16.5) | .21 | 298 (19.5) | 877 (16.6) | .30 | ||||||||
| Obesity | |||||||||||||||||||||
| Obesity of children (aged 0-10 years)g | 401 (2.7) | 407 (2.7) | .97 | 423 (2.4) | .73 | 302 (4.0) | .38 | 370 (1.9) | 1163 (3.18) | .14 | 370 (2.7) | 1163 (3.49) | .46 | ||||||||
| Obesity of adolescents (aged 11-17 years)h | 330 (5.2) | 260 (4.6) | .76 | 275 (4.4) | .65 | 275 (3.6) | .36 | 291 (3.4) | 848 (4.8) | .28 | 291 (4.1) | 848 (4.6) | .78 | ||||||||
aSample sizes are shown in Table 1.
bMM: mixed-mode.
cSAQ-Web: self-administered Web-based questionnaire.
dSAQ-paper: self-administered paper-based questionnaire.
eAdjusted for age of the child (adolescent, parent); sex of the child (adolescent); relationship to the child (parent); household income (adolescent, parent); parental education (adolescent, parent); adolescent education (adolescent); region (adolescent, parent); municipality size (adolescent, parent); and parental marital status (parent).
fTested against single-mode control group.
gProxy-reported by parents of children and adolescents aged 0-17 years.
hSelf-reported by adolescents aged 11-17 years.
Table 3. Health care utilization among children and adolescents aged 0-17 years by survey design and data collection mode (prevalence rates).
| Health care utilization | Survey designa | Data collection modea | ||||||||||||
| Single-mode design | Sequential MMbdesign | Concurrent MMbdesign | Preselect MMbdesign | SAQ-Webc (crude) | SAQ-paperd (crude) | SAQ-Webc (adjustede) | SAQ-paperd (adjustede) | |||||||
| n (%) | n (%) | Pf | n (%) | Pf | n (%) | Pf | n (%) | n (%) | P | n (%) | n (%) | P | ||
| Medical care use | ||||||||||||||
| Any doctor (children aged 0-13 years; past 12 months)g | 970 (91.3) | 804 (92.7) | .31 | 935 (91.8) | .74 | 645 (91.5) | .93 | 594 (95.3) | 2760 (91.05) | < .001 | 594 (92.2) | 2760 (91.69) | .75 | |
| Pediatric services (children aged 0-13 years; past 12 months)g | 970 (70.0) | 803 (71.6) | .46 | 935 (72.0) | .34 | 645 (71.6) | .48 | 593 (80.9) | 2760 (69.16) | < .001 | 593 (70.9) | 2760 (71.47) | .75 | |
| Pediatric services (adolescents aged 14-17 years; past 12 months)h | 205 (35.6) | 147 (35.4) | .96 | 155 (29.0) | .18 | 159 (35.8) | .96 | 164 (34.1) | 502 (34.1) | .99 | 164 (33.8) | 502 (34.2) | .92 | |
| Orthodontic services (adolescents aged 14-17 years; past 12 months)h | 332 (40.4) | 260 (37.3) | .45 | 281 (39.5) | .83 | 288 (44.4) | .31 | 295 (39.7) | 865 (40.7) | .75 | 295 (40.3) | 865 (40.5) | .96 | |
aSample sizes are shown in Table 1.
bMM: mixed-mode.
cSAQ-Web: self-administered Web-based questionnaire.
dSAQ-paper: self-administered paper-based questionnaire.
eAdjusted for age of the child (adolescent, parent); sex of the child (adolescent); relationship to the child (parent); household income (adolescent, parent); parental education (adolescent, parent); adolescent education (adolescent); region (adolescent, parent); municipality size (adolescent, parent); and parental marital status (parent).
fTested against single-mode control group.
gProxy-reported by parents of children and adolescents aged 0-17 years.
hSelf-reported by adolescents aged 11-17 years.
Table 4. Health-related quality of life of adolescents aged 11-17 years by survey design and data collection mode (mean values).
| Health related quality of life | Survey designa | Data collection modea | |||||||||||||||
| Single-mode design | Sequential MMbdesign | Concurrent MMbdesign | Preselect MMbdesign | SAQ-Webc (crude) | SAQ-paperd (crude) | SAQ-Webc (adjustede) | SAQ-paperd (adjustede) | ||||||||||
| n (%) | n (%) | Ph | n (%) | Ph | n (%) | Ph | n (%) | n (%) | P | n (%) | n (%) | P | |||||
| Dimensions | |||||||||||||||||
| Physical well-being (mean)f | 337 (49.9) | 263 (49.9) | .98 | 282 (49.2) | .38 | 290 (49.0) | .21 | 297 (49.5) | 874 (49.5) | .95 | 297 (49.0) | 874 (49.7) | .26 | ||||
| Psychological well-being (mean)f | 336 (51.2) | 265 (50.0) | .14 | 283 (49.6) | .05g | 291 (49.8) | .07 | 297 (49.6) | 877 (50.4) | .20 | 297 (49.3) | 877 (50.5) | .07 | ||||
| Relations with parents (mean)f | 331 (53.5) | 263 (53.0) | .52 | 283 (53.3) | .74 | 290 (52.0) | .04g | 296 (52.5) | 871 (53.1) | .31 | 296 (52.3) | 871 (53.2) | .18 | ||||
| Relations with peers (mean)f | 340 (51.2) | 266 (50.1) | .14 | 288 (50.5) | .37 | 292 (50.1) | .13 | 297 (49.8) | 888 (50.8) | .12 | 297 (49.9) | 888 (50.7) | .18 | ||||
| Well-being in school (mean)f | 335 (51.9) | 265 (51.6) | .66 | 283 (50.8) | .08 | 285 (51.4) | .43 | 294 (51.1) | 873 (51.60) | .31 | 294 (50.9) | 873 (51.70) | .17 | ||||
aSample sizes are shown in Table 1.
bMM: mixed-mode.
cSAQ-paper: self-administered paper-based questionnaire.
dSAQ-Web: self-administered Web-based questionnaire.
eAdjusted for age of the child (adolescent, parent); sex of the child (adolescent); relationship to the child (parent); household income (adolescent, parent); parental education (adolescent, parent); adolescent education (adolescent); region (adolescent, parent); municipality size (adolescent, parent); and parental marital status (parent).
fSelf-reported by adolescents aged 11-17 years.
gNot significant, P value adjusted with Bonferroni correction.
hTested against single-mode control group.
Table 5. Health behaviors of adolescents aged 11-17 years by survey design and data collection mode (prevalence-rates).
| Health related quality of life | Survey designa | Data collection modea | |||||||||||||
| Single-mode design | Sequential MMb design | Concurrent MMb design | Preselect MMb design | SAQ-Webc (crude) | SAQ-paperd (crude) | SAQ-Webc (adjustede) | SAQ-paperd (adjustede) | ||||||||
| n (%) | n (%) | Pi | n (%) | Pi | n (%) | Pi | n (%) | n (%) | P | n (%) | n (%) | P | |||
| Tobacco consumption | |||||||||||||||
| Current smoking status (yes)f | 342 (8.8) | 264 (8.3) | .85 | 288 (11.5) | .27 | 291 (10.7) | .43 | 295 (11.5) | 889 (9.2) | .27 | 295 (11.6) | 889 (9.2) | .25 | ||
| Water pipe consumption (past 12 months, yes)f | 339 (18.0) | 262 (17.9) | .99 | 286 (18.2) | .95 | 291 (16.2) | .54 | 295 (20.3) | 882 (16.7) | .17 | 295 (19.1) | 882 (17.1) | .38 | ||
| Second-hand smoke exposure (yes)f | 308 (13.6) | 239 (15.5) | .55 | 249 (10.8) | .31 | 259 (14.3) | .82 | 261 (11.1) | 793 (14.4) | .16 | 261 (12.9) | 793 (13.8) | .74 | ||
| Alcohol consumption | |||||||||||||||
| Lifetime consumption of alcohol (yes)f | 343 (51.0) | 262 (56.1) | .21 | 290 (50.7) | .93 | 292 (54.8) | .34 | 295 (61.4) | 891 (50.3) | .001 | 295 (60.1) | 891 (50.7) | < .001 | ||
| Hazardous alcohol consumption (based on AUDIT-Cg)f | 328 (12.2) | 255 (12.5) | .90 | 284 (10.6) | .53 | 284 (12.7) | .86 | 293 (17.1) | 857 (10.3) | .01 | 293 (14.9) | 857 (11.9) | .16 | ||
| Binge drinking (based on AUDIT-C)f | 339 (7.7) | 262 (7.7) | .99 | 288 (5.6) | .29 | 291 (8.9) | .57 | 295 (10.5) | 884 (6.5) | .04 | 295 (10.0) | 884 (7.2) | .13 | ||
| Physical activity | |||||||||||||||
| Physical activity consistent with WHOh guidelinesf | 343 (6.1) | 265 (5.3) | .66 | 288 (3.1) | .07 | 292 (3.8) | .17 | 295 (4.1) | 892 (4.8) | .58 | 295 (3.3) | 892 (5.2) | .16 | ||
| Low physical activityf | 343 (14.9) | 265 (12.5) | .39 | 288 (15.3) | .89 | 292 (17.1) | .44 | 295 (13.9) | 892 (15.2) | .57 | 295 (14.5) | 892 (15.0) | .82 | ||
| Currently doing sportsf | 342 (79.5) | 265 (81.9) | .46 | 286 (78.0) | .63 | 291 (77.0) | .44 | 295 (80.3) | 888 (78.6) | .52 | 295 (79.3) | 888 (78.9) | .89 | ||
| Media consumption | |||||||||||||||
| Social media ( > 2 hours/day)f | 341 (20.2) | 262 (19.1) | .72 | 289 (17.3) | .35 | 290 (19.7) | .86 | 295 (20.0) | 886 (18.8) | .67 | 295 (21.3) | 886 (18.4) | .30 | ||
| TV ( > 2 hours/day)f | 341 (42.2) | 262 (41.6) | .88 | 289 (41.2) | .79 | 292 (38.0) | .28 | 294 (39.1) | 889 (41.4) | .49 | 294 (41.5) | 889 (40.6) | .79 | ||
| Game console ( > 2 hours/day)f | 339 (18.9) | 261 (19.9) | .75 | 287 (18.5) | .90 | 290 (12.8) | .03i | 295 (20.0) | 881 (16.7) | .21 | 295 (19.9) | 881 (16.7) | .22 | ||
aSample sizes are shown in Table 1.
bMM: mixed-mode.
cSAQ-Web: self-administered Web-based questionnaire.
dSAQ-paper: self-administered paper-based questionnaire.
eAdjusted for age of the child (adolescent, parent); sex of the child (adolescent); relationship to the child (parent); household income (adolescent, parent); parental education (adolescent, parent); adolescent education (adolescent); region (adolescent, parent); municipality size (adolescent, parent); and parental marital status (parent).
fSelf-reported by adolescents aged 11-17 years.
gAUDIT-C: Alcohol Use Disorders Identification Test.
hWHO: World Health Organization; sample sizes are shown in Table 1.
iNot significant, P value adjusted with Bonferroni correction.
Discussion
Summary
The main aim of this study was to examine the risk of mode effects in a mixed-mode health interview survey for children and adolescents that combined paper-and-pencil questionnaires and Web-based questionnaires. Therefore, we compared prevalence rates and mean values of a broad range of health indicators from 3 alternative mixed-mode designs (all combining paper-and-pencil and Web-based questionnaires) with a single-mode control group (paper-and-pencil only). We also compared results between online respondents and paper-and-pencil respondents regardless of the survey design. First, we examined differences in sociodemographic characteristics by survey design and data collection mode, as it is well documented that sociodemographic characteristics are associated with health status and health behavior [43,44]. Regarding survey design, there were no statistically significant differences in sample composition, prevalence rates, or mean values of the examined health indicators. There were differences in sociodemographic characteristics across the data collection mode groups. After adjusting for these differences, only one of the analyzed health indicators (lifetime alcohol consumption) showed between-group differences. These results indicate that there is limited mode bias in health interview surveys for children and adolescents using a mixed-mode design with Web-based and paper questionnaires.
Sample Composition and Digital Divide
Consistent with previous findings, the sample composition of responding parents and of responding adolescents differed by data collection mode. We confirmed the so-called “digital divide” [47-50]—male adolescents and younger fathers preferred the online mode, a well-known systematic difference [5] between these modes[10,49,51-54]. Additionally, SAQ-Web respondents had higher household incomes [15,49,55] and higher household education levels [10,49,54-57]. Despite these differences, and differences in online response rates between the mixed-mode survey designs, there were no statistically significant differences in sample composition between the paper-and-pencil single-mode control group and the 3 mixed-mode groups. To control for the influence of sociodemographic on health indicators, we adjusted for sociodemographic characteristics by first calculating crude prevalence rates. Then, the analysis was complemented with adjusted prevalence rates or adjusted mean values using predictive margins to identify possible mode effects. Comparisons between the mixed-mode survey designs and the single-mode control group were made using only the crude prevalence rates. Using this approach, hardly any statistically significant differences by data collection mode or by survey design were found for the analyzed health indicators.
Health Status and Health Care Utilization
Prevalence rates of health complaints, such as diagnosed allergies, diagnosed ADHD, obesity, and chronic pain, were equivalent between the modes, as previous studies of adults [9,11,53,58] and adolescents [12] have shown. A population-based Norwegian study found higher asthma prevalence rates among online respondents; this was interpreted as possible nonresponse bias and not as a mode effect because there were no differences in the prevalence rates for any other condition [59]. A literature review by Hox et al showed that after controlling for selection, small mode effects do appear, most often distinguishing between modes that involve interviewers (face-to-face, telephone) and modes that do not (mail, Web) [42].
We found similar prevalence rates for SRH, chronic diseases, and impairment owing to health problems between SAQ-paper and SAQ-Web respondents. The 2 previous studies examining these health indicators among adults in general [11] and among older adults [10] also found no differences between these 2 data collection modes. Another study of adults interpreted the higher SRH found among online respondents compared with paper-based respondents as an expression of different sample characteristics linked to the digital divide era [49], or a case of better-situated people with better health using Web-based questionnaires, and not as a mode effect. We cannot say whether this holds true for the KiGGS methodological pilot study, because we controlled for most characteristics linked to the preference for online participation, such as region of residence and education or income.
For mental and psychosocial problems, we calculated risk groups for emotional and behavioral problems and for impairment owing to psychosocial problems based on SDQ scores [34]. Both parent- and adolescent-reported scores were equivalent across the examined modes. Several other studies have postulated the comparability of measurement results between these 2 self-administered modes for other standardized mental health questionnaires (eg, depression or anxiety) [12,58,60,61].
In their review of 55 studies investigating 79 instruments, Campbell et al [9] found measurement equivalence for electronic- and paper-based patient outcomes and concluded that standardized instruments can generally be used electronically without measurement effects. In our study, we also found comparable results for standardized instruments (the SDQ and AUDIT-C), as well as for self-reported HRQoL (KIDSCREEN-27). No existing studies have compared these particular instruments, but previous studies have compared the Short Form Health Survey-36, a frequently used standardized HRQoL instrument for adults, and found measurement equivalence [9,58,62-64].
All reports of health care utilization were equivalent between the self-administered modes; this is consistent with prior empirical results, including studies of adult vaccination use [11], adolescent health care use [12], and multiple health care quality indicators [56]. The greater use of pediatric services (and of any doctor) before adjustment for sociodemographic characteristics may be explained by the younger age of children in the online group—in Germany, all children are invited to undergo regular health screening examinations (U3-U9 examinations) from early childhood until the age of 5 years, with a well-established system of reminders and reporting.
Health Behaviors
Most of the analyzed adolescent health behaviors (current smoking, 12-month water pipe consumption, second-hand smoke exposure, physical activity, and screen-based media use) showed comparable results and no differences between the 2 modes. These results are consistent with the results of other studies on adolescents [12,65].
Considering alcohol consumption, the crude and adjusted prevalence rates for lifetime consumption were significantly higher among SAQ-Web-responding adolescents. After adjusting for sociodemographic characteristics, the difference decreased but could not be explained by the sociodemographic differences between the 2 groups of respondents. The prevalence of hazardous consumption and binge drinking were comparable between data collection modes after controlling for sample composition.
Most previous studies have reported no statistically significant differences in alcohol consumption among adolescents or young adults by these 2 data collection modes [12,66]. However, research comparing sensitive health behaviors is inconsistent. Some studies have found higher adult binge drinking [53] and higher adolescent alcohol consumption [17] in online reports, whereas others have found no difference in sensitive health behaviors in general for college students [67,68] and young adults [69].
The higher rate of reported lifetime alcohol consumption among SAQ-Web-responding adolescents, in the absence of frequently reported hazardous consumption or binge drinking, may be interpreted in multiple ways. For example, this may be a result of different sample properties, such as SAQ-Web-preferring adolescents being more likely to experiment with alcohol consumption. However, it is also possible that this result is a mode effect based on the assumption of identical alcohol consumption in both groups. Web-based questionnaires afford greater privacy because there is no risk of parents checking the responses. Another possible explanation is the lower social orientation in the Internet mode [13]. Both these explanations assume that Web-based questionnaires are more likely to elicit honest reports, but the similar results between the 2 mode groups for reported harmful alcohol consumption after adjustment contradict this assumption. Taken together, the results for alcohol consumption suggest that lifetime consumption should be used with caution as a health indicator in a mixed-mode design. Hazardous consumption and binge drinking are better indicators because they exhibit mode equivalence and have greater public health relevance than lifetime consumption, which is measured by a single question asking whether the respondent has ever consumed alcohol.
Main Result
Other empirical comparisons of measurement results between different mixed-mode survey designs are rare. In accord with one other result for the adult population [70], all of the analyzed health indicators for children and adolescents showed comparable results, with no statistically significant differences between the single-mode control group and the 3 mixed-mode groups. Additionally, sociodemographic characteristics did not differ by survey design for parents or adolescents. Regarding measurement comparability, any of the tested mixed-mode health interview survey designs, which offer both Web-based and paper questionnaires, could be used for children and adolescents.
Strengths and Limitations
The strengths of the methodological pilot study are the randomized study design, the population-based sample, and the inclusion of a single-mode control group as a reference to interpret the results. However, there are also some limitations, predominantly the relatively small size of the net samples of the analyzed groups. Each survey design had a relatively low number of cases, so interpretations of the results based on the net samples must be made with caution. Possible differences across the 4 survey designs or between the 2 data collection modes could have been overlooked because of a lack of statistical power, particularly regarding the need for correction for multiple testing. Other limitations concern the external validity of the results; the study was conducted in a German setting using register-based samples of children and adolescents, so the results are difficult to generalize to other countries, settings, or populations.
Conclusions
Our results are consistent with those of most previous studies. We found comparable results between the 2 self-administered modes (SAQ-Web and SAQ-paper) for almost all analyzed health indicators, except for lifetime consumption of alcohol among adolescents aged 11-17 years. Thus, no differences were found between the single-mode control group design and 3 mixed-mode survey designs that combined the 2 data collection modes.
These results suggest that it is possible to measure health indicators for children and adolescents using a mixed-mode design combining SAQ-Web and SAQ-paper methods, with a low risk of mode effects and high comparability across different mixed-mode survey designs combining these 2 data collection modes [4]. The implementation of a Web-based option in the existing paper-based interview surveys of children and adolescents has a low risk of changed measurement values caused by the mixed-mode survey design.
Acknowledgments
This study was funded by the Robert Koch Institute and the German Federal Ministry of Health within the German Health Monitoring System. The authors thank Diane Williams, PhD, from Edanz Group for editing a draft of this manuscript.
Conflicts of Interest
None declared.
Multimedia Appendix 1
Checklist for Reporting Results of Internet E-Surveys (CHERRIES).
Galea S, Tracy M. Participation rates in epidemiologic studies. Ann Epidemiol 2007 Sep;17(9):643-653.
Bowling A. Mode of questionnaire administration can have serious effects on data quality. J Public Health (Oxf) 2005 Sep;27(3):281-291.
Revilla M. Quality in unimode and mixed-mode designs: a multitrait-multimethod approach. Surv Res Methods 2010;4(3):151-164
De Leeuw E, Hox J. Internet surveys as part of a mixed mode design. In: Das M, Ester P, Kaczmirek L, editors. Social and Behavioral Research and the Internet: Advances in Applied Methods and Research Strategies. New York: Taylor & Francis Group; 2011:45-76.
Hox JJ, De Leeuw ED, Zijlmans EA. Measurement equivalence in mixed mode surveys. Front Psychol 2015;6:87
Vannieuwenhuyze J, Loosveldt G, Molenberghs G. A method for evaluating mode effects in mixed-mode surveys. Public Opin Q 2011 Feb 09;74(5):1027-1045.
Couper MP. The future of modes of data collection. Public Opin Q 2011 Dec 15;75(5):889-908.
Rutherford C, Costa D, Mercieca-Bebber R, Rice H, Gabb L, King M. Mode of administration does not cause bias in patient-reported outcome results: a meta-analysis. Qual Life Res 2016 Mar 3;25(3):559-574.
Campbell N, Ali F, Finlay AY, Salek SS. Equivalence of electronic and paper-based patient-reported outcome measures. Qual Life Res 2015 Aug;24(8):1949-1961.
de Bernardo DH, Curtis A. Using online and paper surveys: the effectiveness of mixed-mode methodology for populations over 50. Res Aging 2013;35(2):220-240.
Hoebel J, von der Lippe E, Lange C, Ziese T. Mode differences in a mixed-mode health interview survey among adults. Arch Public Health 2014 Dec 22;72(1):46
Mangunkusumo RT, Moorman PW, Van Den Berg-de Ruiter AE, Van Der Lei J, De Koning HJ, Raat H. Internet-administered adolescent health questionnaires compared with a paper version in a randomized study. J Adolesc Health 2005 Jan;36(1):70.e1-70.e6.
Taddicken M. Methodeneffekte von Web-Befragungen: Soziale Erwünschtheit vs. Soziale Entkontextualisierung. In: Weichbold M, Bacher J, Wolf C, editors. Umfrageforschung: Herausforderungen und Grenzen. Wiesbaden: VS Verlag für Sozialwissenschaften; 2009:85-104.
van Gelder MM, Bretveld RW, Roeleveld N. Web-based questionnaires: the future in epidemiology? Am J Epidemiol 2010 Dec 01;172(11):1292-1298.
Rookey BD, Hanway S, Dillman DA. Does a probability-based household panel benefit from assignment to postal response as an alternative to Internet-only? Public Opin Q 2008 Dec 09;72(5):962-984.
Burkill S, Copas A, Couper MP, Clifton S, Prah P, Datta J, et al. Using the web to collect data on sensitive behaviours: a study looking at mode effects on the British national survey of sexual attitudes and lifestyles. PLoS One 2016 Feb 11;11(2):e0147983
Paperny DM, Aono JY, Lehman RM, Hammar SL, Risser J. Computer-assisted detection and intervention in adolescent high-risk health behaviors. J Pediatr 1990 Mar;116(3):456-462.
Kurth BM, Kamtsiuris P, Hölling H, Schlaud M, Dölle R, Ellert U, et al. The challenge of comprehensively mapping children's health in a nation-wide health survey: design of the German KiGGS-Study. BMC Public Health 2008 Jun 04;8:196
Hölling H, Schlack R, Kamtsiuris P, Butschalowsky H, Schlaud M, Kurth BM. [The KiGGS study. Nationwide representative longitudinal and cross-sectional study on the health of children and adolescents within the framework of health monitoring at the Robert Koch Institute]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 2012 Jun 7;55(6-7):836-842.
Kurth BM. [Das RKI-Gesundheitsmonitoring - was es enthält und wie es genutzt werden kann]. Public Health Forum 2012;20(3):4.e1-4.e3.
Kurth BM, Kamtsiuris P, Hölling H, Mauz E. Strategien des Robert Koch-Instituts zum Monitoring der Gesundheit von in Deutschland lebenden Kindern und Jugendlichen. Kinder- und Jugendmedizin 2016;16(3):176-183.
Robert Koch-Institut. Rki.de. 2011. KiGGS - Kinder- und Jugendgesundheitsstudie Welle 1. Projektbeschreibung URL: https://www.rki.de/DE/Content/Gesundheitsmonitoring/Gesundheitsberichterstattung/GBEDownloadsB/KiGGS_welle1.pdf?__blob=publicationFile [accessed 2018-01-30] [WebCite Cache]
Mauz E, Gößwald A, Kamtsiuris P, Hoffmann R, Lange M, von Schenck U, et al. New data for action. Data collection for KiGGS Wave 2 has been completed. J Health Monit 2017;2(3):2-27
Biemer PP. Total survey error: design, implementation, and evaluation. Public Opin Q 2011 Feb 09;74(5):817-848.
Groves RM, Lyberg L. Total survey error: past, present, and future. Public Opin Q 2011 Feb 09;74(5):849-879.
Robert Koch-Institut. Berlin: Robert Koch-Institut; 2017. Methodische Studie zur Durchführung von Mixed-Mode-Befragungen zur Gesundheit von Kindern und Jugendlichen (Pilotstudie KiGGS Welle 2) URL: https://www.rki.de/DE/Content/Gesundheitsmonitoring/Gesundheitsberichterstattung/GBEDownloadsB/Pilotstudie-KiGGS-Welle2.pdf?__blob=publicationFile [accessed 2018-01-27] [WebCite Cache]
Dillman DA, Smyth JD, Christian LM. Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method. 4th Edition. New York: John Wiley & Sons; 2014.
Eysenbach G. Improving the quality of web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res 2004 Dec 29;6(3):e34
American Association for Public Opinion Research (AAPOR). Esomar. 2011. Standard definitions: final dispositions of case codes and outcome rates for surveys (revised 2011) URL: https://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/ESOMAR_Standard-Definitions-Final-Dispositions-of-Case-Codes-and-Outcome-Rates-for-Surveys.pdf [accessed 2018-01-30] [WebCite Cache]
Brauns H, Scherer S, Steinmann S. The CASMIN Educational Classification in International Comparative Research. In: Hoffmeyer-Zlotnik JHP, Wolf C. editors. Advances in Cross-National Comparison: A European Working Book for Demographic and Socio-Economic Variables. Boston, MA: Springer US; 2003:221-244.
Cox B, van Oyen H, Cambois E, Jagger C, le Roy S, Robine JM, et al. The reliability of the Minimum European Health Module. Int J Public Health 2009 Jan;54(2):55-60.
Bethell CD, Read D, Stein RE, Blumberg SJ, Wells N, Newacheck PW. Identifying children with special health care needs: development and evaluation of a short screening instrument. Ambul Pediatr 2002;2(1):38-48.
Kromeyer-Hauschild K, Wabitsch M, Kunze D, Geller F, Geiß H, Hesse V, et al. Perzentile für den Body-mass-Index für das Kindes- und Jugendalter unter Heranziehung verschiedener deutscher Stichproben. Monatsschr Kinderheilkd 2001 Aug 1;149(8):807-818
Goodman R. The extended version of the Strengths and Difficulties Questionnaire as a guide to child psychiatric caseness and consequent burden. J Child Psychol Psychiatry 1999 Jul;40(5):791-799.
Woerner W, Becker A, Rothenberger A. Normative data and scale properties of the German parent SDQ. Eur Child Adolesc Psychiatry 2004 Jul;13 Suppl 2:II3-I10.
Rattay P, Starker A, Domanska O, Butschalowsky H, Gutsche J, Kamtsiuris P, KiGGS Study Group. [Trends in the utilization of outpatient medical care in childhood and adolescence: results of the KiGGS study - a comparison of baseline and first follow up (KiGGS Wave 1)]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 2014 Jul;57(7):878-891.
The Kidscreen Group Europe (editors). The KIDSCREEN Questionnaires: Quality of Life Questionnaires for Children and Adolescents. Lengerich, Germany: Pabst Science Publishers; 2006.
Lampert T, Kuntz B, KiGGS Study Group. [Tobacco and alcohol consumption among 11- to 17-year-old adolescents: results of the KiGGS study: first follow-up (KiGGS Wave 1)]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 2014 Jul;57(7):830-839.
Manz K, Schlack R, Poethko-Müller C, Mensink G, Finger J, Lampert T, KiGGS Study Group. [Physical activity and electronic media use in children and adolescents: results of the KiGGS study: first follow-up (KiGGS wave 1)]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 2014 Jul;57(7):840-848.
Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA. The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Alcohol Use Disorders Identification Test. Arch Intern Med 1998 Sep 14;158(16):1789-1795.
World Health Organization. Geneve: WHO; 2010. Global recommendations on physical activity for health URL: http://www.who.int/dietphysicalactivity/publications/9789241599979/en/ [accessed 2018-01-27] [WebCite Cache]
Hox J, De LE, Klausch T. Mixed mode research: issues in design and analysis. In: Biemer PP, de Leeuw E, Eckman S, Edwards B, Kreuter F, Lyberg LE, et al, editors. Total Survey Error in Practice: Improving Quality in the Era of Big Data. New Yorck: Wiley; 2017:511-530.
Abuladze L, Kunder N, Lang K, Vaask S. Associations between self-rated health and health behaviour among older adults in Estonia: a cross-sectional analysis. Br Med J Open 2017 Jun 09;7(6):e013257
Link BG, Phelan JC. Understanding sociodemographic differences in health--the role of fundamental social causes. Am J Public Health 1996 Apr;86(4):471-473.
Graubard BI, Korn EL. Predictive margins with survey data. Biometrics 1999 Jun;55(2):652-659.
Bland JM, Altman DG. Multiple significance tests: the Bonferroni method. Br Med J 1995 Jan 21;310(6973):170
Estacio EV, Whittle R, Protheroe J. The digital divide: examining socio-demographic factors associated with health literacy, access and use of internet to seek health information. J Health Psychol 2017 Feb 01:1359105317695429.
Kontos E, Blake KD, Chou WS, Prestin A. Predictors of eHealth usage: insights on the digital divide from the Health Information National Trends Survey 2012. J Med Internet Res 2014 Jul;16(7):e172
Shim JM, Shin E, Johnson TP. Self-rated health assessed by web versus mail modes in a mixed mode survey: the digital divide effect and the genuine survey mode effect. Med Care 2013 Sep;51(9):774-781.
Latulippe K, Hamel C, Giroux D. Social health inequalities and eHealth: a literature review with qualitative synthesis of theoretical and empirical studies. J Med Internet Res 2017 Apr 27;19(4):e136
Börkan B. The mode effect in mixed-mode surveys mail and web surveys. SSCR 2010;28(3):371-380
Dolnicar S, Laesser C, Matus K. Online versus paper format effects in tourism surveys. J Travel Res 2009;47(3):295-316.
Link MW, Mokdad AH. Alternative modes for health surveillance surveys: an experiment with web, mail, and telephone. Epidemiology 2005 Sep;16(5):701-704.
Callas PW, Solomon LJ, Hughes JR, Livingston AE. The influence of response mode on study results: offering cigarette smokers a choice of postal or online completion of a survey. J Med Internet Res 2010 Oct 21;12(4):e46
Messer B, Edwards M, Dillman D. Determinants of item nonresponse to web and mail respondents in three address-based mixed-mode surveys of the general public. Surv Pract 2012;5(2):1-27
Zuidgeest M, Hendriks M, Koopman L, Spreeuwenberg P, Rademakers J. A comparison of a postal survey and mixed-mode survey using a questionnaire on patients' experiences with breast care. J Med Internet Res 2011 Sep 27;13(3):e68
Smith AB, King M, Butow P, Olver I. A comparison of data quality and practicality of online versus postal questionnaires in a sample of testicular cancer survivors. Psychooncology 2013 Jan;22(1):233-237.
Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review. Value Health 2008;11(2):322-333
Brøgger J, Nystad W, Cappelen I, Bakke P. No increase in response rate by adding a web response option to a postal population survey: a randomized trial. J Med Internet Res 2007 Dec 31;9(5):e40
Austin DW, Carlbring P, Richards JC, Andersson G. Internet administration of three commonly used questionnaires in panic research: equivalence to paper administration in Australian and Swedish samples of people with panic disorder. Int J Test 2006 Mar;6(1):25-39.
Zhang X, Kuchinke L, Woud ML, Velten J, Margraf J. Survey method matters: online/offline questionnaires and face-to-face or telephone interviews differ. Comput Human Behav 2017 Jun;71:172-180.
Basnov M, Kongsved SM, Bech P, Hjollund NH. Reliability of short form-36 in an Internet- and a pen-and-paper version. Inform Health Soc Care 2009 Jan;34(1):53-58.
Broering JM, Paciorek A, Carroll PR, Wilson LS, Litwin MS, Miaskowski C. Measurement equivalence using a mixed-mode approach to administer health-related quality of life instruments. Qual Life Res 2014 Mar;23(2):495-508.
Kongsved SM, Basnov M, Holm-Christensen K, Hjollund NH. Response rate and completeness of questionnaires: a randomized study of Internet versus paper-and-pencil versions. J Med Internet Res 2007 Sep 30;9(3):e25
Wyrick DL, Bond L. Reducing sensitive survey response bias in research on adolescents: a comparison of web-based and paper-and-pencil administration. Am J Health Promot 2011 May;25(5):349-352.
McCabe SE, Diez A, Boyd CJ, Nelson TF, Weitzman ER. Comparing web and mail responses in a mixed mode survey in college alcohol use research. Addict Behav 2006 Sep;31(9):1619-1627
Pealer LN, Weiler RM, Pigg RM, Miller D, Dorman SM. The feasibility of a web-based surveillance system to collect health risk behavior data from college students. Health Educ Behav 2001 Oct;28(5):547-559.
Jun MK. Websm. Indiana: Indiana University; 2005. Effects of survey mode, gender, and perceived sensitivity on the quality of data regarding sensitive health behaviors URL: http://www.websm.org/db/12/11689/rec/ [accessed 2018-01-27] [WebCite Cache]
McMorris BJ, Petrie RS, Catalano RF, Fleming CB, Haggerty KP, Abbott RD. Use of web and in-person survey modes to gather data from young adults on sex and drug use: an evaluation of cost, time, and survey error based on a randomized mixed-mode design. Eval Rev 2009 Apr;33(2):138-158
Schilling R, Hoebel J, Müters S, Lange C. Edoc.rki.de. Berlin: RKI; 2015. Pilotstudie zur Durchführung von Mixed-Mode-Gesundheitsbefragungen in der Erwachsenenbevölkerung (Projektstudie GEDA 2.0). Beiträge zur Gesundheitsberichterstattung des Bundes URL: http://edoc.rki.de/documents/rki_fv/rex2KXJHIPpM/PDF/226m8YvEr7dPM.pdf [accessed 2018-01-27] [WebCite Cache]
© 2018. This work is licensed under https://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Hoffmann, Robert
; Houben, Robin
; Krause, Laura
; Kamtsiuris, Panagiotis
; Gößwald, Antje