Content area
Full text
Abstract
International large-scale assessments excel at robust cross-national comparisons, and detailed insight into students learning progressions emerges when paired with complementary diagnostic approaches. This study employed an Al-enhanced approach-the Collective Intelligence Model for Education (CIME)-to transform item-level responses from students in Ireland who participated in the PISA 2022 mathematics assessment into multidimensional diagnostic profiles. The profiles highlighted pronounced strengths in chance and probability and identified areas for continued development in geometric relationships and measurement. Students demonstrated proficiency in working with data and established mathematical representations, and showed developing proficiency in devising solution strategies, formalising complex situations, and applying mathematical models to structure and analyse real-world contexts. The findings indicate that Al-enhanced profiling yields fine-grained, policy-relevant diagnostics that complement headline scores and inform curriculum planning and system-level improvement.
1 Beyond traditional assessment approaches
International large-scale assessments: from comparison to diagnosis
International large-scale assessments (ILSAs), most prominently the OECD's Programme for International Student Assessment (PISA) and the IEA's Trends in International Mathematics and Science Study (TIMSS), have become influential instruments in policymaking and international benchmarks of educational quality. Their comparative indicators, such as mathematics scores, are widely disseminated and are often treated by policymakers as evidence of system performance and a basis for developing reform agendas.
ILSAs publish not only reports on performance but also public-use files (PUFs) containing item-level responses, background questionnaire data (OECD, n.d.[1]), calibrated item parameters, and methodological documentation (OECD, 2017[2]; OECD, 2023[3]. These resources enable researchers to investigate relationships between performance and contextual factors in depth. The principal focuses of published work include inequalities associated with socioeconomic status (SES), immigrant status and language background, gender, and school-level factors (Hopfenbeck et al., 2018[4]
Governments frequently use these data and reports to develop strategies aimed at improving effectiveness and equity, and the OECD's dissemination of PISA results has contributed to a transnational policy space in which data serve as a common reference point for reform (Grek, 2009[5]; Martens and Niemann, 2013[6]. However, headline scores are rarely sufficient on their own. Knowing whether a country/economy sits above or below the international mean may satisfy accountability demands, yet it sheds little light on what learners find difficult or why particular misconceptions persist (Fullan, 2016[7].
Reliance on broad composite scores risks reducing complex proficiencies and educational processes...