Assessing multimodal reading comprehension in STEM education: a Rasch model approach

Abstract

This case study investigates the English reading comprehension proficiency of STEM (science, technology, engineering, and mathematics) students from three Indonesian universities using a Rasch model analysis. An online reading test consisting of 5 multimodal digital texts—integrating both verbal and visual elements—and 20 items in multiple-choice and short-answer formats was administered. The Rasch model was used to analyze item difficulty and participant ability estimates, providing a nuanced understanding of learner performance. The results revealed a wide range of item difficulty, from − 4 to 4 logits, with Item 10 identified as the most challenging and Item 9 as the easiest. This pattern demonstrates that cross-modal integration and inferential reasoning pose significant cognitive challenges for L2 readers in technical contexts. Survey responses showed that students perceived visual elements as supportive, yet this perception was weakly correlated with performance, indicating a metacognitive gap in processing multimodal information. Technical vocabulary was cited as a moderate challenge (M = 3.8), reinforcing the importance of disciplinary literacy instruction. These findings demonstrate how the integration of multimodal literacy theory with Rasch analysis can enhance assessment practices in multilingual STEM education contexts, providing a framework for developing more targeted pedagogical interventions and equitable assessment strategies.