Reliability of educational attainment of survey respondents: an overlooked barrier to comparability?

Abstract

Educational attainment is vital in social science research for analysing socioeconomic inequalities, labour market outcomes, and health disparities. Harmonisation schemes such as the International Standard Classification of Education (ISCED) and its survey-specific adaptation EDULVLB aim to standardise educational classifications across countries, enabling international comparability. Despite their widespread use, concerns persist regarding the reliability of these harmonised measures, particularly at the individual level and across different survey modes. This study evaluates the reliability of harmonised educational attainment measurements using test-retest data from Estonia, Slovenia, and the United Kingdom. Respondents’ answers from the face-to-face European Social Survey Round 8 (2016) and the online CRONOS Panel Wave 6 (2018) were analysed, with reliability coefficients estimated for both the one-digit ISCED and the more detailed EDULVLB classification. The results reveal notable individual-level inconsistencies, especially in the United Kingdom, challenging assumptions of high reliability in harmonised education data. Inconsistencies were most common between adjacent educational levels, suggesting difficulties distinguishing similar qualifications. Device effects were also observed, with smartphone users displaying lower consistency than computers or tablets users. While mode effects could not be fully disentangled from measurement error, the findings underscore the need for systematic reliability assessments and improved instrument design to ensure the comparability and validity of educational measures in cross-national survey research.