Content area
This case study investigates the English reading comprehension proficiency of STEM (science, technology, engineering, and mathematics) students from three Indonesian universities using a Rasch model analysis. An online reading test consisting of 5 multimodal digital texts—integrating both verbal and visual elements—and 20 items in multiple-choice and short-answer formats was administered. The Rasch model was used to analyze item difficulty and participant ability estimates, providing a nuanced understanding of learner performance. The results revealed a wide range of item difficulty, from − 4 to 4 logits, with Item 10 identified as the most challenging and Item 9 as the easiest. This pattern demonstrates that cross-modal integration and inferential reasoning pose significant cognitive challenges for L2 readers in technical contexts. Survey responses showed that students perceived visual elements as supportive, yet this perception was weakly correlated with performance, indicating a metacognitive gap in processing multimodal information. Technical vocabulary was cited as a moderate challenge (M = 3.8), reinforcing the importance of disciplinary literacy instruction. These findings demonstrate how the integration of multimodal literacy theory with Rasch analysis can enhance assessment practices in multilingual STEM education contexts, providing a framework for developing more targeted pedagogical interventions and equitable assessment strategies.
Introduction
The intersection of STEM education, multimodal literacy, and language assessment presents unique challenges and opportunities for educational research and practice. As STEM disciplines increasingly rely on complex information presented through multiple modes of communication, students must develop sophisticated literacy skills that extend beyond traditional text comprehension. This need becomes particularly acute in multilingual educational contexts, where students must navigate both disciplinary content and linguistic demands simultaneously.
In Indonesia, STEM literacy development faces distinct challenges. These difficulties are concerning given that proficient reading abilities are foundational for success in STEM disciplines (Acquah, 2023; Arlinwibowo, 2020; Barnard-Brak, 2017; Perfetti, 1999).While STEM integration has extended from primary to secondary levels (Ardwiyanti et al., 2021; Khotimah et al., 2021), it remains limited at the university level, creating a critical gap in higher education. Indonesian students consistently rank below international and ASEAN-OECD averages in reading proficiency on PISA assessments, with performance trailing countries such as Peru and Brazil (Patria, 2021; Safari et al., 2020). These barriers intensify in higher education, where STEM students must simultaneously acquire disciplinary content and develop language skills in English as a second or foreign language (Kurniasih et al., 2023). Recent findings confirm that Indonesian university students struggle with critical reading and academic writing, due in part to limited exposure to multimodal texts, unfamiliar test formats, and minimal integration of English in STEM instruction (Isma & Nur, 2023; Suryaman, 2015).
This study addresses these challenges by integrating three theoretical perspectives: multimodal literacy theory, Rasch measurement theory, and second language reading theory. Together, these frameworks provide a comprehensive lens for understanding and assessing multimodal reading comprehension in multilingual STEM contexts. The purpose of this case study is to examine the English reading proficiency of Indonesian university STEM students using a multimodal digital reading test, investigating the following:
What reading proficiency levels and difficulty patterns are revealed by Rasch analysis?
What cognitive challenges do multimodal STEM texts present for L2 readers?
How do students’ subjective perceptions of text difficulty relate to their actual performance?
The findings contribute to educational theory and practice by demonstrating how the integration of these theoretical perspectives can enhance assessment practices in multilingual STEM education and provide guidance for designing more effective and equitable instructional interventions.
Literature review
An integrated theoretical framework
This study integrates three theoretical perspectives to provide a comprehensive framework for understanding and assessing multimodal reading comprehension in STEM contexts. First, multimodal literacy theory (Kress & van Leeuwen, 2006; Jewitt, 2008) provides a foundation for understanding how meaning is constructed through the interaction of different semiotic resources—verbal, visual, spatial, and gestural. This perspective recognizes that comprehension in STEM disciplines requires readers to navigate and synthesize information presented across multiple modes, each with its own “grammar” and representational logic.
Second, Rasch measurement theory (Bond & Fox, 2015; Wright, 1977) offers a robust psychometric approach for analyzing the interaction between reader abilities and task demands. Unlike classical test theory, the Rasch model enables the simultaneous estimation of item difficulty and person ability on a single logit scale, providing fine-grained insights into the specific cognitive challenges posed by different aspects of multimodal texts. This approach is particularly valuable for identifying threshold concepts in literacy development—points where qualitative shifts in comprehension strategies occur.
Third, second language reading theory (Bernhardt, 2011; Grabe, 2009) illuminates the additional cognitive demands faced by multilingual readers. Models such as Bernhardt’s compensatory model and Cummins’ threshold hypothesis explain how limited language proficiency can constrain the cognitive resources available for higher-order comprehension processes, particularly when dealing with disciplinary content. This perspective helps explain why certain multimodal integration tasks may pose disproportionate challenges for L2 readers in STEM contexts.
The integration of these three theoretical strands offers a powerful lens for examining how Indonesian STEM students navigate the complex demands of multimodal reading in English. By analyzing performance patterns through this integrated framework, we can identify specific threshold points where comprehension breaks down and design targeted interventions to support students at these critical junctures.
Multimodal reading in STEM education
STEM education serves diverse stakeholders and requires multifaceted educational approaches that bridge theory and practice (Evans et al., 2021). The cognitive demands of reading in STEM disciplines are particularly complex, requiring students to navigate multiple representational systems simultaneously—verbal text, mathematical notation, visual diagrams, and dynamic models (Alarcón et al., 2023).
Research on multimedia learning provides insight into how students process multimodal information in STEM contexts. Mayer’s (2001) cognitive theory of multimedia learning proposes that learners have separate channels for processing verbal and visual information, with meaningful learning occurring when students select relevant words and images, organize them into coherent verbal and pictorial representations, and integrate these representations with each other and with prior knowledge. This model helps explain why some multimodal integration tasks may be particularly challenging, especially when working memory resources are constrained by linguistic processing demands.
The creation and use of multimodal resources are particularly effective in enhancing students’ multiliteracies and understanding of diverse semiotic resources (Yeh, 2018). Bodén et al. (2023) emphasize the importance of using multimodal tools in instruction, finding that students respond more positively to materials that combine text with dynamic visuals, such as animated diagrams or videos. When aligned with curricular content, such multimedia resources enhance visualization, active learning, and problem-solving—hallmarks of holistic and inquiry-based instruction (Biria & Mehrabi, 2014; Danielsson & Selander, 2016; Dahlström, 2022).
Multimodal text design for L2 readers
For second-language readers, multimodal texts present both opportunities and challenges. Visual elements can provide additional scaffolding for comprehension when linguistic resources are limited (Mayer & Sims, 1994). However, the cognitive demands of integrating information across multiple modes may tax already-constrained processing resources, potentially leading to cognitive overload (Sweller, 1988). This tension is particularly salient in STEM education, where technical content adds another layer of complexity.
Bernhardt’s (2011) compensatory model of second-language reading provides a useful framework for understanding these dynamics. The model posits that L2 readers must allocate cognitive resources across three dimensions: L1 reading skills transfer, L2 language knowledge, and background knowledge. When processing STEM texts in a second language, students face triple demands: applying general reading strategies, decoding English vocabulary and syntax, and grappling with unfamiliar technical concepts. This “triple deficit” appears particularly pronounced when students encounter tasks requiring cross-modal integration and inferential reasoning.
The threshold hypothesis proposed by Cummins (2008) further suggests that second-language readers must reach a certain level of language proficiency before they can engage effectively in higher-order thinking in that language. This explains why L2 readers might successfully complete simple information location tasks but struggle with more complex inferential tasks that require cross-textual synthesis or the integration of verbal and visual information.
Recent studies highlight the effectiveness of multimodal approaches in supporting English language learners. Multimodal digital videos have been shown to improve students’ language skills, engagement, and motivation in online learning environments (Susanti et al., 2022). Cahyaningati and Lestari (2018) demonstrated that preprinted multimodal texts significantly enhanced Indonesian students’ English reading proficiency compared to traditional linear texts. These findings suggest that with appropriate scaffolding, multimodal texts can support rather than hinder L2 reading development.
Rasch modeling for multimodal assessment
The Rasch model offers distinct advantages for assessing multimodal reading comprehension, particularly in multilingual contexts. As a psychometric approach, it provides a mathematical framework for simultaneously estimating person ability and item difficulty parameters on a single interval scale (Wright, 1977). This allows for precise identification of threshold points in reading development—moments where qualitative shifts in comprehension strategies occur.
Unlike classical test theory approaches, which treat all items as equally difficult, the Rasch model accounts for varying levels of difficulty across different types of comprehension tasks. This is particularly valuable for multimodal assessment, where the cognitive demands of integrating information across different semiotic modes may vary substantially. By converting raw scores to logit measures, the Rasch model provides a more nuanced understanding of the specific challenges posed by different aspects of multimodal texts.
Recent applications of Rasch modeling in language assessment have demonstrated its utility for understanding the cognitive processes underlying reading comprehension. Chang and Ying (2009) used Rasch analysis to identify threshold concepts in second-language reading development, while Wilson and Moore (2011) applied the model to explore the relationship between text features and item difficulty. These studies highlight the potential of Rasch modeling for illuminating the specific challenges faced by L2 readers when navigating multimodal texts.
The application of Rasch modeling to multimodal assessment is still an emerging field, with few studies explicitly addressing the intersection of multimodality, second-language reading, and STEM content. This study addresses this gap by applying Rasch analysis to examine how Indonesian STEM students engage with multimodal texts in English, identifying specific threshold points in comprehension and the text features that contribute to increased cognitive demands.
Method
Research design
This study employs a case study approach to investigate multimodal reading comprehension among Indonesian STEM students. Following Yin’s (2018) methodological framework, this investigation focuses on a bounded system—Indonesian university STEM education—examining the real-life phenomenon of multimodal reading within its contemporary context. This design allows for an in-depth examination of how students navigate the complex demands of integrating verbal and visual information in disciplinary texts, particularly when reading in a second language.
The case study approach is well-suited to the integrated theoretical framework guiding this research. It provides flexibility to examine the interplay between multimodal literacy processes (through varied text types and visual elements), second-language reading dynamics (through student performance and perceptions), and psychometric properties (through Rasch analysis of item responses).
The Rasch measurement model was selected as the primary analytical tool due to its unique advantages over classical test theory approaches. The Rasch model allows for simultaneous estimation of item difficulty and person ability parameters on a single scale, providing a more nuanced understanding of the relationship between student proficiency and specific multimodal comprehension tasks. This alignment between the theoretical framework and methodological approach strengthens the study’s internal coherence and analytical power.
Participants
This study involved 150 STEM students from three Indonesian universities, selected based on their academic reputation, diversity of student populations, and established STEM programs. Participants were fifth-semester undergraduate students who had completed compulsory English courses but were not yet engaged in their final projects. This selection criterion ensured that participants had sufficient language proficiency to engage with English texts while still developing their disciplinary reading skills—a critical juncture in their academic development.
Eligibility was determined by enrollment in STEM courses requiring intensive engagement with disciplinary texts and viewing a set of English critical reading videos on YouTube as a prerequisite for the reading comprehension test. Each institution contributed 50 participants, resulting in a total sample of 150 respondents. Efforts were made to ensure demographic diversity in gender, academic discipline, and regional background.
Ethical considerations
The study involved obtaining informed consent from all participants, who were provided with a detailed explanation of the study’s objectives, confidentiality measures, and intended use of data. Ethical approval was secured from the appropriate institutional review boards at each participating university, aligning with established guidelines for human subject research.
To maintain participant anonymity, identifying information was removed or replaced with anonymized codes before data analysis. Responses were stored securely in password-protected digital files, with access restricted to authorized researchers. Data confidentiality was ensured by reporting only aggregated findings, minimizing the risk of individual identification.
Instruments
Multimodal reading comprehension test
A multimodal English reading comprehension test was designed specifically for this study, drawing on principles from our integrated theoretical framework. The test consisted of 5 digital texts accompanied by 20 comprehension items. Text selection and design were informed by multimodal literacy theory, with careful attention to the relationship between verbal and visual elements and the cognitive demands of integrating information across modes.
The texts, as presented in Table 1, incorporated various visual elements, including diagrams, screenshots of websites, and scientific illustrations, mimicking authentic digital formats that STEM students encounter in academic contexts. Content domains were selected to represent the range of disciplinary reading tasks relevant to engineering education, including scientific discourse (texts on cosmic rays and Antarctic climate), informational/explanatory content (texts on Bitcoin), historical/artistic analysis (text on van Gogh), and technical explanation (text on lock-picking mechanisms).
Table 1. Text characteristics
Text no | Topic | Word count | Number of items | Visual |
|---|---|---|---|---|
1 | Science (astronomy): cosmic rays | 120 | 2 | YouTube page diagram, flow charts |
2 | Science (oceanography): Antarctic climate | 166 | 3 | Popular science website: maps, graphs |
3 | Technology: Bitcoin | 5 | Popular science website: screenshots, infographics | |
Text A | 247 | |||
Text B | 247 | |||
4 | Art technology: van Gogh | 78 | 2 | Website page of science magazine: paintings, timeline |
5 | How technology works: lock-picking mechanism | Technical illustration of a lock | ||
Section 1 | 203 | 3 | ||
Section 2 | 261 | 3 | ||
Section 3 | 159 | 2 |
The test items were designed to assess three core subskills of reading as defined by OECD (2019): locating information, understanding ideas and information, and evaluating textual content. This alignment with international assessment frameworks enhances the test’s construct validity. Items were presented in both multiple-choice and closed constructed-response formats, providing varied assessment of comprehension processes as depicted in Table 2.
Table 2. Test item characteristics
Cognitive aspects | Question Intent | Item Number | Total |
|---|---|---|---|
Locate Information | Identify explicit information | 1, 5, 9, 13, 17 | 5 |
Scan for specific details | 2, 6, 10, 14, 18 | 5 | |
Understand | Integrate information details | 3, 7, 11, 15, 19 | 5 |
Draw inferences across models | 4, 8, 12, 16, 20 | 5 |
Post-test survey
To complement the quantitative analysis provided by the Rasch model, a post-test survey was administered to all participants. This methodological triangulation approach aligns with the integrated theoretical framework by providing insight into students’ subjective experience of multimodal processing alongside objective performance measures.
The survey consisted of two main components: (1) assessment of verbal elements and (2) assessment of visual elements, with respondents rating various aspects on a 5-point Likert scale from 1 (very easy) to 5 (very difficult). For each text, students rated four verbal elements (word choice/vocabulary, grammar, organization of sentences and paragraphs, and text structure/logic) and four visual elements (layout of verbal text and images, background color, font characteristics, and design elements including images and tables).
This instrument was specifically designed to capture metacognitive aspects of multimodal reading—students’ awareness of their own comprehension processes and the perceived difficulty of integrating different semiotic resources. The survey was completed immediately after the reading test to ensure freshness of experience and accurate reflection on the reading process.
Procedure
Data collection was conducted sequentially at each university through a systematic process designed to ensure consistency and reliability. Participants completed the reading comprehension test individually, without direct interaction with other test-takers, to ensure independent responses. A standardized 30-min time limit was imposed for the 20-item test, with items presented in a fixed numerical sequence from 1 to 20. The sequential presentation format prevented participants from skipping items, ensuring complete engagement with each text before proceeding to subsequent sections.
The data collection procedure incorporated elements from all three theoretical perspectives guiding the study. From a multimodal literacy perspective, participants engaged with varied text types incorporating different relationships between verbal and visual elements. From a second-language reading perspective, the timed nature of the test created conditions where resource limitations might affect performance on more complex tasks. From a Rasch measurement perspective, standardized administration supported valid comparison of item difficulty and person ability parameters.
Data analysis
The Rasch measurement model served as the primary analytical framework for this study, aligning with the psychometric component of our integrated theoretical approach. This model was employed to analyze the reading proficiency of participants and the psychometric properties of the test items, particularly their difficulty parameters. The Rasch model offers significant advantages over classical test theory, especially in its ability to simultaneously assess item difficulty (bi) and person ability (θ) on a single logit scale.
Prior to interpreting the results, several validation analyses were performed to confirm that the basic assumptions of the Rasch model were satisfied:
Unidimensionality was verified using item reliability and separation values in the primary component analysis of standardized residuals.
Local independence was checked by calculating residual correlations between items, ensuring values remained below ± 0.3 to avoid excessive local dependence.
Fit statistics were examined, with mean-squared (MNSQ) values within the ideal range of 0.7 to 1.3 indicating appropriate model fit.
The Rasch analysis enabled examination of several key aspects of multimodal reading comprehension:
Item difficulty patterns: Analysis of difficulty parameters (bi) revealed which aspects of multimodal reading presented the greatest challenges for participants.
Person ability distribution: Analysis of ability parameters (θ) showed the range of reading proficiency within the sample.
Item-person map: This visualization placed both items and persons on the same measurement scale, illustrating the relationship between student abilities and task demands.
Item characteristic curves: These probabilistic functions showed how the likelihood of correct response changed across different ability levels.
To complement the Rasch analysis, descriptive statistics were applied to the survey data, calculating mean ratings for verbal and visual elements across different texts. Correlation analyses were performed to examine the relationship between perceived difficulty and actual performance, providing insight into students’ metacognitive awareness.
Results
The Rasch model analysis of multimodal reading performance revealed distinct patterns in item difficulty, student ability distribution, and the relationship between text features and comprehension challenges. These findings illuminate the cognitive processes underlying successful engagement with multimodal STEM texts, particularly for second-language readers.
Rasch analysis of item difficulty and student ability
The Rasch analysis revealed a wide range of item difficulties, providing a nuanced picture of the cognitive demands posed by different aspects of multimodal reading comprehension. Figure 1 depicts the item information curves (IICs), which represent each question’s information function across different ability levels.
[See PDF for image]
Fig. 1
Item information curves (source: current research)
The ability axis in Fig. 1 is essential for understanding how different items function across the proficiency spectrum. Items with peaks near zero provide optimal measurement for students with average reading ability, while items with peaks to the right offer more precise measurement for advanced readers.
Figure 2 provides a complementary visualization through a Wright Map, which places both items and persons on the same measurement scale. The item difficulty parameters range from − 4 to 4 logits, indicating substantial variation in the cognitive demands of different questions.
[See PDF for image]
Fig. 2
Wright map of items test (source: current research)
As shown in Fig. 2, the distribution of item difficulties reveals a hierarchical structure of reading comprehension skills. Items positioned higher on the map require more advanced processing capabilities, while those positioned lower represent more foundational skills. This pattern aligns with our theoretical framework’s integration of multimodal literacy and second-language reading theories, which predict that tasks requiring cross-modal integration and inferential processing will present greater challenges, particularly for L2 readers.
Table 3 presents the difficulty parameters for all 20 items, arranged by item number. The difficulty values range from − 1.24 (question 9) to + 4.15 (question 10), revealing striking differences in item challenge levels. Fifteen of the 20 items had difficulty values greater than or equal to 0, and 7 items exceeded + 1, indicating that the test presented substantial cognitive demands for many participants.
Table 3. Item difficulty parameters
Index | Item text | Difficulty | Index | Item text | Difficulty |
|---|---|---|---|---|---|
1 | Quest 1 | 0.7697 | 11 | Quest 11 | 1.8252 |
2 | Quest 2 | 0.9757 | 12 | Quest 12 | 0.7697 |
3 | Quest 3 | 0.6712 | 13 | Quest 13 | − 0.3328 |
4 | Quest 4 | 1.2764 | 14 | Quest 14 | 1.2368 |
5 | Quest 5 | 0.5434 | 15 | Quest 15 | 1.8252 |
6 | Quest 6 | 0.4188 | 16 | Quest 16 | 0.7366 |
7 | Quest 7 | − 0.0031 | 17 | Quest 17 | 1.3580 |
8 | Quest 8 | 0.7037 | 18 | Quest 18 | 2.5651 |
9 | Quest 9 | − 1.2387 | 19 | Quest 19 | − 0.2421 |
10 | Quest 10 | 4.1459 | 20 | Quest 20 | − 0.2722 |
The Rasch model’s fit statistics, summarized in Table 4, confirm the psychometric quality of the assessment. The high item reliability (0.919) indicates consistent differentiation in item difficulty, while the acceptable separation index (1.026) suggests that the items successfully distinguish between different levels of reading proficiency.
Table 4. Item analysis based on Rasch model
Measurement | Value |
|---|---|
Mean | 0.746 |
Standard deviation (SD) | 1.846 |
Max | 3.819 |
Min | − 1.876 |
Real RMSE | 1.799 |
Model RMSE | 1.799 |
True SD | 1.846 |
Separation (real) | 1.026 |
Separation (model) | 1.026 |
Item reliability | 0.919 |
The Rasch analysis revealed specific patterns in how multimodal text features related to item difficulty, providing insight into the cognitive processes involved in integrating verbal and visual information. By examining the items with extreme difficulty parameters (Items 9, 10, and 18), we can identify key threshold points in multimodal reading comprehension.
Item 9 (the easiest item, − 1.24 logits) required students to locate explicitly stated information about Bitcoin’s value. This item involved straightforward text scanning and memory-based processing, without requiring cross-modal integration or inference.
In stark contrast, Item 10 demonstrated exceptional difficulty (4.15 logits), requiring students to draw inferences from two separate Bitcoin texts. This item’s extreme difficulty parameter represents a critical threshold in the reading comprehension hierarchy, where processing demands shift from individual text comprehension to cross-textual synthesis.
Item 18 represented another significant threshold in difficulty (2.57 logits), requiring students to process technical information across verbal and visual modes in the lock-picking text. This item exemplified the challenges of cross-modal integration, as students needed to map verbal descriptions onto mechanical components in a diagram.
Detailed analysis of these items’ expected score patterns, as shown in Figs. 3 and 4, illustrates how the probability of correct response varied across different ability levels.
[See PDF for image]
Fig. 3
Expected scores for question number 10 (source: current research)
[See PDF for image]
Fig. 4
Expected scores for question number 18 (source: current research)
Further analysis of item difficulty patterns revealed a hierarchical structure of multimodal reading processes, as presented in Table 5. This hierarchy progresses from simple information location tasks through intramodal integration, cross-modal integration, and multimodal inference, culminating in cross-textual synthesis.
Table 5. The test’s inference types complexity
Cognitive process | Description | Example item | Difficulty range |
|---|---|---|---|
Unimodal information location | Locating explicitly stated information in a single mode | 9, 13 | − 1.24 to − 0.33 |
Intramodal integration | Connecting information within the same mode | 1, 2, 3 | 0.67 to 0.98 |
Cross-modal integration | Connecting information across verbal and visual modes | 4, 14, 17 | 1.24 to 1.36 |
Multimodal inference | Drawing conclusions based on integration of modes | 11, 15, 18 | 1.83 to 2.57 |
Cross-textual synthesis | Integrating and reconciling information across texts | 10 | 4.15 |
This hierarchy provides a cognitive processing model that explains the observed difficulty patterns and offers a framework for understanding the development of multimodal reading proficiency. It suggests that successful engagement with STEM texts requires readers to master the progression of increasingly complex integration strategies, with cross-modal integration and cross-textual synthesis representing significant cognitive thresholds.
Student perceptions of text difficulty: survey results
The post-test survey provided valuable insight into students’ subjective experience of multimodal texts, complementing the objective performance data from the Rasch analysis. Table 6 presents students’ ratings of verbal elements across all texts, revealing distinct patterns in perceived difficulty.
Table 6. Mean ratings of verbal elements (1 = very easy, 5 = very difficult)
Text | Word choice & vocabulary | Grammar | Organization of sentence & paragraphs | Test structure & logic | Overall mean |
|---|---|---|---|---|---|
1: Cosmic rays | 3.9 | 3.2 | 3.1 | 3.7 | 3.5 |
2: Antarctic climate | 3.7 | 3.0 | 2.9 | 3.5 | 3.3 |
3A: Bitcoin | 3.8 | 3.1 | 3.0 | 3.6 | 3.4 |
3B: Bitcoin | 3.9 | 3.2 | 3.1 | 3.6 | 3.5 |
4: van Gogh | 3.2 | 2.8 | 2.7 | 3.0 | 2.9 |
5: Lock-picking | 4.3 | 3.4 | 3.2 | 3.9 | 3.7 |
Mean across texts | 3.8 | 3.1 | 3.0 | 3.6 | 3.4 |
Across all texts, students identified word choice/vocabulary (M = 3.8) and text structure/logic (M = 3.6) as the most challenging verbal elements, while organization of sentences and paragraphs (M = 3.0) was perceived as the least challenging. This pattern held consistently across different texts, though absolute difficulty ratings varied. Text 5 (lock-picking mechanism) received the highest difficulty ratings across all verbal categories (M = 3.7), while Text 4 (van Gogh) received the lowest (M = 2.9).
Of particular interest is Text 3 (Bitcoin, parts A and B), which contained Item 10—identified by Rasch analysis as the most difficult item in the test. Despite the extreme difficulty of Item 10 (4.15 logits), students rated the verbal elements of Text 3 as only moderately difficult (M = 3.4–3.5). This discrepancy suggests a metacognitive gap in students’ awareness of the challenges posed by cross-textual synthesis tasks.
Table 7 presents students’ ratings of visual elements across all texts, providing insight into their perceived support value for comprehension.
Table 7. Mean ratings of visual elements (1 = very unsupportive, 5 = very supportive)
Text | Layout | Background color | Font | Design elements | Overall mean |
|---|---|---|---|---|---|
1: Cosmic rays | 3.7 | 3.2 | 3.5 | 3.8 | 3.6 |
2: Antarctic climate | 4.0 | 3.5 | 3.7 | 4.1 | 3.8 |
3A: Bitcoin | 4.1 | 3.4 | 3.8 | 4.3 | 3.9 |
4: van Gogh | 4.3 | 3.7 | 4.0 | 4.5 | 4.1 |
5: Lock-picking | 4.2 | 3.6 | 3.9 | 4.4 | 4.0 |
Mean across texts | 4.1 | 3.5 | 3.8 | 4.2 | 3.9 |
Visual elements were generally perceived as supportive of comprehension, with design elements (including diagrams, images, and tables) receiving the highest ratings (M = 4.2), followed closely by layout (M = 4.1). Background color received the lowest supportiveness ratings (M = 3.5). Text 4 (van Gogh) and Text 5 (lock-picking mechanism) received the highest visual supportiveness ratings (M = 4.0–4.1), while Text 1 (cosmic rays) received the lowest (M = 3.6).
A particularly notable finding emerged for Text 5 (lock-picking mechanism), which contained Item 18—identified by Rasch analysis as the second most difficult item. Despite the considerable difficulty of Item 18 (2.57 logits), students rated the visual elements of Text 5 as highly supportive (M = 4.0). This paradoxical finding further suggests a metacognitive gap in students’ awareness of the challenges involved in cross-modal integration.
Correlation analysis in Fig. 5 revealed a moderate positive correlation (r = 0.63, p < 0.05) between perceived verbal difficulty and Rasch-derived item difficulty. However, several notable outliers were observed, particularly for Items 10 and 18, where actual difficulty far exceeded perceived difficulty. This finding suggests that students were not fully aware of the cognitive challenges posed by these items, especially those requiring cross-textual synthesis or complex multimodal integration.
[See PDF for image]
Fig. 5
Mean perceived verbal difficulty ratings and Rasch difficulty parameters
In contrast, the correlation between perceived visual supportiveness and item difficulty was weak and non-significant (r = − 0.28, p > 0.05), suggesting that students’ subjective assessment of visual aids did not reliably predict their performance on related questions. This finding aligns with research on multimedia learning, which indicates that learners may overestimate the support provided by visual elements when the cognitive demands of integration exceed available processing resources.
Discussion
The findings of this study demonstrate the value of integrating multimodal literacy theory, Rasch measurement principles, and second-language reading research to understand the cognitive demands of reading in STEM contexts. When interpreted through this integrated lens, the results reveal how the interaction between text features, linguistic processing, and cognitive resources shapes comprehension outcomes for Indonesian STEM students engaging with English language materials.
Cognitive dimensions of multimodal reading
The Rasch analysis revealed a clear hierarchy of cognitive processes involved in multimodal reading comprehension, progressing from simple to complex operations. This hierarchy provides a framework for understanding the developmental progression of reading skills and the specific challenges faced by L2 readers in STEM contexts.
At the foundational level, unimodal information location—exemplified by Item 9 (− 1.24 logits)—involves straightforward retrieval of explicitly stated information from a single text segment. This process relies primarily on what van den Broek et al. (2005) term “memory-based processing,” where readers identify and extract information without complex integration or inference. Most participants demonstrated proficiency at this level, suggesting that basic information location represents a baseline skill that can serve as a confidence-building foundation for more complex tasks.
At the intermediate level, intramodal integration—exemplified by Items 1–3 (0.67–0.98 logits)—requires readers to connect information across different sections of verbal text. This process involves working memory resources to maintain multiple text segments simultaneously and identify relationships between them. While more challenging than simple information location, most participants demonstrated reasonable proficiency at this level, suggesting that intramodal integration represents a developmental step that many students have mastered.
Cross-modal integration—exemplified by Items 4, 14, and 17 (1.24–1.36 logits)—represents a significant increase in cognitive complexity. This process requires what Mayer and Sims (1994) describe as “referential processing,” where readers establish connections between corresponding representations in different modalities. The challenge of this process is evident in the higher difficulty parameters for these items, suggesting that cross-modal integration represents a threshold concept in multimodal literacy development.
Multimodal inference—exemplified by Items 11, 15, and 18 (1.83–2.57 logits)—requires readers to go beyond the explicitly stated information to draw conclusions based on the integration of verbal and visual elements. This process engages in what Kintsch (1998) terms “constructive processing,” where readers build mental models that incorporate both textual information and their own knowledge. The high difficulty parameters for these items, particularly Item 18 (2.57 logits), indicate that multimodal inference represents a significant cognitive threshold for many L2 readers.
At the most advanced level, cross-textual synthesis—exemplified by Item 10 (4.15 logits)—requires readers to integrate and reconcile information across multiple texts with potentially different perspectives or emphases. This process involves what Perfetti and Stafura (2014) describe as “situation model building across texts,” where readers construct a coherent mental representation that encompasses multiple sources. The exceptional difficulty of Item 10 suggests that cross-textual synthesis represents a particularly challenging threshold for L2 readers in STEM contexts.
This cognitive hierarchy provides valuable insight into the development of multimodal reading proficiency. It suggests that successful engagement with STEM texts requires readers to master the progression of increasingly complex integration strategies, with cross-modal integration and cross-textual synthesis representing significant cognitive thresholds. For L2 readers, these thresholds may be particularly challenging due to the competition for cognitive resources between language processing and higher-order integration tasks.
The metacognitive dimension of these processes adds another layer of complexity. The survey results revealed that students often underestimated the difficulty of the most challenging items, particularly those requiring cross-modal integration or cross-textual synthesis. This “illusion of comprehension” (Glenberg et al., 1982) suggests that students may not be fully aware of the cognitive demands imposed by these tasks or may overestimate their own processing capabilities. This metacognitive gap has important implications for instruction, as students who do not recognize comprehension challenges are unlikely to deploy appropriate strategies to address them.
Educational implications
The integrated theoretical framework and empirical findings of this study offer significant implications for educational practice in multilingual STEM contexts. Understanding the cognitive hierarchies and threshold concepts in multimodal reading comprehension enables more effective instructional interventions and assessment strategies.
Scaffolded instruction
The hierarchy of reading processes indicates a need for scaffolded instruction that progressively develops integration skills, beginning with unimodal information location and gradually introducing more complex cross-modal and cross-textual activities. This aligns with Vygotsky’s zone of proximal development concept, targeting skills just beyond students’ independent capability but achievable with support.
Instructional sequences should begin with explicit modeling of cross-modal connections, gradually transitioning to guided practice and independent application. For example, teachers might first demonstrate how verbal descriptions relate to specific visual elements and then guide students in identifying similar connections in new texts, before finally asking students to independently integrate information across modes.
Metacognitive strategy instruction
The survey results revealing metacognitive gaps emphasize the importance of explicit instruction in comprehension monitoring and strategy use. STEM students would benefit from metacognitive training in self-questioning, comprehension verification, and awareness of cross-modal connections. As Pintrich (2002) notes, metacognitive awareness is crucial for complex learning tasks where automatic processing proves insufficient.
Specific strategies might include “think-aloud” protocols where students verbalize their integration processes, annotation techniques that explicitly map connections between verbal and visual elements, and structured reflection on comprehension breakdowns and their solutions. Such approaches help students develop the metacognitive tools needed to navigate complex multimodal texts more effectively.
Visual literacy development
The weak correlation between perceived visual support and actual performance suggests a need for targeted visual literacy instruction. Though students viewed visual elements as supportive, their ability to extract and synthesize information from these elements appears limited.
Explicit instruction in interpreting disciplinary visuals and integrating across representational modes aligns with Ainsworth’s (2006) research on multiple representations, emphasizing the importance of understanding the complementary functions of different representational forms. Such instruction might include analyzing the specific conventions of STEM visuals (e.g., the meaning of different types of arrows or symbols in diagrams) and practicing the translation of information between verbal and visual modes.
Disciplinary literacy instruction
Students’ moderate difficulty with technical vocabulary (M = 3.8) highlights the importance of disciplinary literacy instruction addressing domain-specific terminology. Shanahan and Shanahan (2008) emphasize that advanced literacy development requires attention to the specialized vocabulary and representational conventions of specific disciplines.
For STEM education, this suggests the need for integrated approaches that develop technical vocabulary alongside conceptual understanding. Content and language instructors should collaborate to identify key terminology, provide multiple exposures in meaningful contexts, and explicitly teach the relationships between technical terms and their visual representations.
Language scaffolding for L2 readers
For second-language readers, providing language scaffolding allows sufficient cognitive resource allocation to technical and conceptual demands. Techniques like pre-teaching vocabulary, using graphic organizers, and providing bilingual glossaries can reduce linguistic load and free resources for higher-order comprehension.
This approach aligns with Cummins’ (2008) threshold hypothesis by supporting students in reaching the linguistic threshold necessary for engaging with cognitively demanding academic tasks. The substantial gap between basic information location tasks and complex integration tasks observed in our Rasch analysis suggests that such scaffolding is particularly important for tasks requiring cross-modal integration or cross-textual synthesis.
Assessment design considerations
This study’s findings have important implications for reading assessment design in multilingual STEM contexts. The Rasch analysis revealed item difficulty patterns that can inform more valid and informative assessment instruments.
Balanced assessment design
The wide range of item difficulties highlights the importance of including items measuring different levels in the cognitive hierarchy. A balanced assessment should span from simple information location to cross-modal integration and cross-textual synthesis, allowing for a comprehensive evaluation of reading proficiency. This aligns with Wilson’s (2005) concept of constructing maps, which emphasizes the importance of measuring across the full range of developmental progression.
Future assessments should include items representing each level of the cognitive hierarchy identified in this study, with multiple items at each level to ensure reliable measurement. This approach provides a more nuanced picture of student proficiency than assessments focused primarily on basic comprehension or fact retrieval.
Calibrated progression in item difficulty
The substantial gap between Item 10 (4.15 logits) and Item 18 (2.57 logits) suggests the need for a more calibrated progression in item difficulty. Including items with more gradual steps in difficulty would offer more precise measurement of the development of cross-modal and cross-textual integration skills, providing more detailed diagnostic information about specific strengths and challenges.
For example, future assessments might include items requiring partial integration across texts before introducing tasks demanding complete synthesis or items requiring integration of less complex visual elements before introducing more sophisticated diagrams or graphs.
Control for domain knowledge effects
The findings regarding domain familiarity and technical vocabulary underscore the importance of controlling for the effects of content knowledge. As Bachman and Palmer (2010) argue, it is important to distinguish between construct-relevant variance and construct-irrelevant variance in assessment. Difficulty stemming from unfamiliarity with specific technical content may represent construct-irrelevant variance that confounds the measurement of reading processes.
Future assessments should select content domains with similar levels of familiarity across the target population or explicitly measure domain knowledge as a separate variable. This allows for more accurate attribution of performance differences to reading skills rather than background knowledge.
Process-oriented assessment techniques
The discrepancy between students’ perceptions and actual performance suggests the value of including process-oriented assessment techniques such as retrospective verbal protocols or process tracing measures. Such approaches provide insight into the cognitive processes employed during item response and can help identify specific points where comprehension breaks down.
These techniques might include eye-tracking studies to examine how students navigate between verbal and visual elements, think-aloud protocols to reveal integration strategies, or structured retrospective interviews to explore students’ awareness of their own comprehension processes.
Linguistic accommodations for L2 readers
For L2 readers, incorporating appropriate linguistic accommodations in assessment design offers potential benefits. Providing definitions for technical terms or allowing dictionary use for technical vocabulary may reduce construct-irrelevant variance stemming from vocabulary limitations. This aligns with what Abedi (2004) describes as “accommodation that reduce construct-irrelevant factors without giving undue advantage.”
However, such accommodations should be carefully designed to maintain the construct validity of the assessment. The goal is not to remove all linguistic challenges but to ensure that performance differences reflect reading comprehension skills rather than simply vocabulary knowledge.
Limitations and future research
While this study provides valuable insights into multimodal reading comprehension in STEM contexts, several limitations must be acknowledged.
Methodological limitations
The case study approach, while offering in-depth examination of multimodal reading processes, lacks a control group comparing performance on traditional linear texts. Without this comparison, we cannot draw definitive causal conclusions about the effectiveness of multimodal interventions or the specific advantages of the Rasch model over classical test theory approaches. This reflects the exploratory nature of the study, which aimed to examine cognitive demands and diagnostic potential within an authentic educational context rather than conduct a controlled experiment.
The relatively homogeneous sample of fifth-semester STEM students from three Indonesian universities also limits the generalizability of findings. The selection criteria ensured participants had sufficient language proficiency to engage with English texts but may have excluded students with lower proficiency levels who face even greater challenges with multimodal integration.
Assessment limitations
While the Rasch analysis revealed valuable patterns in item difficulty, it does not directly measure how differences in domain knowledge affect performance. The varying technical content across texts may have introduced construct-irrelevant variance that influenced item difficulty parameters. Future studies should explicitly assess domain knowledge as a separate variable to analyze interactions between content knowledge, linguistic proficiency, and multimodal processing.
The primary quantitative approach offered limited insight into individual reading strategies and metacognitive processes. While the survey provided some information about students’ perceptions, it did not capture the specific strategies they employed when navigating challenging integration tasks.
Future research directions
Several promising directions for future research emerge from this study, each addressing different aspects of multimodal reading comprehension in STEM contexts. Most immediately, quasi-experimental designs comparing student performance across multimodal and traditional reading conditions would provide more definitive evidence about the effectiveness of multimodal approaches for L2 readers in STEM contexts, addressing the current study’s limitation regarding causal conclusions. To understand developmental trajectories, longitudinal studies tracking the development of multimodal reading skills over time would provide valuable insight into how students’ progress through the cognitive hierarchy identified in our findings and how specific instructional interventions influence this development.
Complementing this developmental focus, cross-population studies investigating how multimodal reading comprehension patterns vary across academic levels, disciplines, and linguistic contexts would enhance the generalizability of findings and clarify how different factors interact to influence performance. For deeper understanding of cognitive processes, process-oriented research using qualitative methods such as think-aloud protocols, eye-tracking studies, or retrospective interviews would provide insight into the specific strategies students use when navigating multimodal reading challenges and how these strategies differ across proficiency levels. Finally, intervention studies testing the effectiveness of specific instructional approaches based on the cognitive hierarchy identified in this research would provide valuable guidance for educational practice, with experimental or quasi-experimental designs comparing traditional approaches with scaffolded instruction targeting cross-modal integration and metacognitive strategy use generating more definitive evidence about effective pedagogical practices for multilingual STEM students.
Conclusion
This study aimed to fill a significant gap in STEM education by investigating how Indonesian university students engage with multimodal English texts that combine verbal and visual elements. By integrating multimodal literacy theory, Rasch measurement principles, and second-language reading research, we developed a comprehensive framework for understanding the cognitive processes underlying successful comprehension in multilingual STEM contexts.
Summary of key findings
The Rasch model analysis revealed a clear hierarchy of cognitive processes in multimodal reading comprehension, progressing from simple information location through increasingly complex integration tasks. Most significantly, the results identified two critical threshold concepts that posed disproportionate challenges for students: cross-modal integration (connecting verbal and visual information) and cross-textual synthesis (integrating information across multiple texts). The extreme difficulty of these tasks—evident in the high difficulty parameters for Items 10 (4.15 logits) and 18 (2.57 logits)—suggests that they represent qualitative shifts in the cognitive processes required for successful comprehension.
The survey data revealed important metacognitive dimensions of multimodal reading, particularly students’ tendency to underestimate the difficulty of the most challenging integration tasks. Despite the extreme difficulty of Items 10 and 18, students rated the corresponding texts as only moderately difficult, suggesting limited awareness of the cognitive demands imposed by cross-modal and cross-textual processing. This metacognitive gap has important implications for instruction, as students who do not recognize comprehension challenges are unlikely to deploy appropriate strategies to address them.
The relationship between item features and difficulty patterns provided insight into the specific factors that contribute to comprehension challenges in multimodal STEM texts. Technical vocabulary emerged as a significant hurdle (rated as moderately difficult at M = 3.8), while the cognitive demands of integrating information across different representational modes created additional processing burdens. For L2 readers, these challenges were compounded by linguistic processing demands that competed for limited cognitive resources, particularly on tasks requiring sophisticated integration and inference.
Theoretical contributions
This study makes several significant theoretical contributions to our understanding of multimodal reading comprehension in multilingual STEM contexts. First, it demonstrates the value of integrating perspectives from multimodal literacy theory, Rasch measurement, and second-language reading research to develop a more comprehensive framework for understanding reading processes in disciplinary contexts.
Building on this integrated approach, the study identifies a hierarchical model of cognitive processes involved in multimodal reading comprehension, providing a framework for understanding developmental progression in reading skills and specific threshold concepts where comprehension strategies undergo qualitative shifts. Additionally, the research illuminates the metacognitive dimensions of multimodal reading, particularly the gap between perceived difficulty and actual performance on complex integration tasks, suggesting limitations in students’ awareness of their own comprehension processes. Finally, it extends our understanding of the compensatory model of second-language reading (Bernhardt, 2011) by examining how the specific demands of multimodal integration interact with linguistic processing constraints in STEM contexts, thereby offering new insights into the unique challenges faced by L2 readers when navigating technically demanding multimodal texts.
Practical implications
The findings of this study have significant implications for educational practice in multilingual STEM contexts, suggesting five key areas for instructional development. First, educators should implement scaffolded instruction that progressively develops integration skills, beginning with simple information location tasks and gradually introducing more complex cross-modal and cross-textual activities. This progression should be supported by metacognitive strategy instruction that develops students’ awareness of comprehension processes and equips them with strategies for monitoring and repairing comprehension breakdowns, particularly when navigating complex multimodal texts.
Simultaneously, visual literacy development must explicitly teach the conventions of disciplinary visuals and provide structured practice in integrating information across verbal and visual modes. These approaches should be complemented by disciplinary literacy instruction that addresses domain-specific vocabulary and representational conventions, developing technical language alongside conceptual understanding. Finally, for second-language learners, language scaffolding that reduces linguistic load on complex integration tasks is essential, allowing cognitive resources to be allocated to higher-order comprehension processes. Together, these instructional strategies provide a comprehensive framework for developing more effective approaches to multimodal literacy in multilingual STEM education contexts. These practical implications provide guidance for developing more effective instructional approaches and assessment practices in multilingual STEM education, supporting students in developing the advanced literacy skills needed for success in these disciplines.
In an increasingly globalized and technologically mediated educational landscape, where students must navigate complex multimodal information across linguistic and cultural boundaries, such integrated approaches to understanding and supporting reading development become ever more essential. This study offers a step toward more comprehensive models of disciplinary literacy that can inform both research and practice in multilingual STEM education.
Acknowledgements
The authors gratefully acknowledge the financial support provided by the Program Penelitian dan Pengabdian Masyarakat (PPMI) ITB 2023 scheme. We extend our sincere gratitude to Lembaga Penelitian dan Pengabdian Masyarakat (LPPM) Institut Teknologi Bandung and the Faculty of Arts and Design (FSRD) ITB for their assistance throughout this research. We also thank the Gulf University for Science and Technology for covering the Article Processing Charge (APC) for this publication.
Clinical trial number
Not applicable.
Authors’ contributions
• N.K.: The primary author of the majority of this manuscript, and she also designed the research from the earliest stage • S.R: Responsible for the development of the videos for English learning materials and for assisting the main author in preparing the questions for the English reading test. • H.R: Responsible for the design of the illustration and visualization of the reading texts and questions. . P.R: Responsible for the discussion regarding STEM Education. • I.K.N: Involved in the discussion of multimodality • E.F.H: Conducted the statistical analysis.
Funding
The authors received funding for this research from the program Penelitian dan Pengabdian Masyarakat (PPMI) ITB 2023 scheme, and therefore, we would like to express our gratitude to Lembaga Penelitian dan Pengabdian Masyarakat (LPPM) Institut Teknologi Bandung and Faculty of Arts and Design (FSRD) ITB.
Data availability
No datasets were generated or analysed during the current study.
Declarations
Ethics approval and consent to participate
This study was conducted in accordance with the Declaration of Helsinki. The research protocol was reviewed and approved by institutional review boards at each participating university, aligning with established guidelines for human subject research. Written informed consent was obtained from all participants prior to their involvement in the study, ensuring they were fully aware of the study’s purpose, procedures, potential risks, and their right to withdraw at any time without penalty.
Competing interests
The authors declare no competing interests.
Abbreviations
Item information curves
Linear text
First language
Second language
Mean square
The Programme for International Student Assessment
Science, technology, engineering, and mathematics
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
Abedi, J. The No Child Left Behind Act and English language learners: Assessment and accountability issues. Educational Researcher; 2004; 33,
Acquah, A., Odame, J., Akushia Antie, E., & Nkum, E. (2023). Book review on language, literacy and learning in STEM education. Open Journal for Studies in Linguistics
Ainsworth, S. DeFT: A conceptual framework for considering learning with multiple representations. Learning and Instruction; 2006; 16,
Alarcón, IV; Anwar, S; Atiq, Z. How multi-modal approaches support engineering and computing education research. Australasian Journal of Engineering Education; 2023; 28,
Ardwiyanti, D., Prasetyo, Z.K., & Wilujeng, I. (2021). STEM research trends in Indonesia: A systematic literature review.
Arlinwibowo, J; Retnawati, H; Kartowagiran, B; Kassymova, GK. Distance learning policy in Indonesia for facing pandemic COVID-19: School reaction and lesson plans. Journal of Theoretical and Applied Information Technology; 2020; 98, pp. 2828-2838.
Bachman, LF; Palmer, AS. Language assessment in practice; 2010; Oxford University Press:
Barnard-Brak, L; Stevens, T; Ritter, WA. Reading and mathematics are equally important to science achievement: Results from nationally-representative data. Learning and Individual Differences; 2017; 58, pp. 1-9. [DOI: https://dx.doi.org/10.1016/j.lindif.2017.07.001]
Bernhardt, EB. Understanding advanced second-language reading. Routledge. Biria, R., & Mehrabi Boshrabadi, A. (2014). The efficacy of multimodal vs. print-based texts for teaching reading comprehension skills to Iranian high school third graders. International Journal of Language Learning and Applied Linguistics World; 2011; 5, pp. 365-380.
Biria, R; Mehrabi Boshrabadi, A. The efficacy of multimodal vs. print-based texts for teaching reading comprehension skills to Iranian high school third graders. International Journal of Language Learning and Applied Linguistics World; 2014; 5, pp. 365-380.
Bodén, U; Stenliden, L; Nissen, J. The construction of interactive and multimodal reading in school—A performative, collaborative, and dynamic reading. Journal of Visual Literacy; 2023; [DOI: https://dx.doi.org/10.1080/1051144X.2023.2168395]
Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences (3rd ed.). Routledge.
Cahyaningati, DT; Lestari, LA. The use of multimodal text in enhancing engineering students reading skill. International Journal of Language Education; 2018; 2,
Chang, HH; Ying, Z. Nonlinear sequential designs for logistic item response theory models with applications to computerized adaptive tests. The Annals of Statistics; 2009; 37,
Cummins, J. (2008). BICS and CALP: Empirical and theoretical status of the distinction. In B. Street spsampsps N. H. Hornberger (Eds.), Encyclopedia of language and education (2nd ed., Vol. 2, pp. 71–83).
Dahlström, H. Students as digital multimodal text designers: A study of resources, affordances, and experiences. British Journal of Educational Technology; 2022; 53,
Danielsson, K., & Selander, S (2016). Reading multimodal texts for learning – A model for cultivating multimodal literacy. Designs for Learning, 8(1), 25–36, https://doi.org/10.16993/dfl.72
Evans, C; Kandiko Howson, C; Forsythe, A; Edwards, C. What constitutes high quality higher education pedagogical research?. Assessment & Evaluation in Higher Education; 2021; 46,
Glenberg, AM; Wilkinson, AC; Epstein, W. The illusion of knowing: Failure in the self-assessment of comprehension. Memory & Cognition; 1982; 10,
Grabe, W. (2009). Reading in a second language: Moving from theory to practice. Cambridge University Press.
Isma, A., & Nur, S. (2023). Examining the reading performance and problems among students in Indonesian higher education. SELTICS.
Jewitt, C. Multimodality and literacy in the classroom. Review of Research in Education; 2008; 32, pp. 241-267. [DOI: https://dx.doi.org/10.3102/0091732X07310586]
Khotimah, R.P., Adnan, M., Ahmad, C.N., & Murtiyasa, B. (2021). Science, mathematics, engineering, and mathematics (STEM) education in Indonesia: A literature review. Journal of Physics: Conference Series, 1776.
Kintsch, W. (1998). Comprehension: A paradigm for cognition. Cambridge University Press.
Kress, G; Van Leeuwen, T. Reading images: The grammar of visual design (Edisi ke-2); 2006; Routledge: [DOI: https://dx.doi.org/10.4324/9780203619728]
Kurniasih, N., Emilia, E., & Sujatna, E.T.S. (2023). Using item analysis to evaluate a teacher-made PISA-like reading test model. International Journal of Language Testing. 13 (1), 188–205. https://doi.org/10.22034/IJLT.2023.345275.1169
Mayer, RE. Multimedia learning; 2001; Cambridge University Press: [DOI: https://dx.doi.org/10.1017/CBO9781139164603]
Mayer, RE; Sims, VK. For whom is a picture worth a thousand words? Extensions of a dual-coding theory of multimedia learning. Journal of Educational Psychology; 1994; 86,
OECD. PISA 2018 Assessment and Analytical Framework. OECD Publishing; 2019; [DOI: https://dx.doi.org/10.1787/b25efab8-en]
Patria, R. R. (2021). Why Indonesian students struggle in reading test? Proceedings of the International Conference on Educational Assessment and Policy (ICEAP 2020), 29–40. Atlantis Press. https://doi.org/10.2991/assehr.k.210423.060
Perfetti, CA. Brown, CM; Hagoort, P. Comprehending written language: A blueprint of the reader. The neurocognition of language; 1999; Oxford University Press: pp. 167-208.
Pintrich, PR. The role of metacognitive knowledge in learning, teaching, and assessing. Theory into Practice; 2002; 41,
Safari, S.A., Asrijanty, A., Rahmawati, R., & Prakoso, B.H. (2020). The analysis of PISA test 2018 on student perception of reading objective, how is Indonesian students? Proceedings of the 17th International Conference on Cognition and Exploratory Learning in the Digital Age (CELDA 2020).
Shanahan, T; Shanahan, C. Teaching disciplinary literacy to adolescents: Rethinking content-area literacy. Harvard Educational Review; 2008; 78,
Suryaman, M. (2015). Analisis hasil belajar peserta didik dalam literasi membaca melalui studi internasional (PIRLS) 2011. Litera, 14.
Susanti, A., Presdyasmara, C.A., Dewi, F.P., & Wardani, Y. (2022). Developing students’ English skill through digital video as multimodal for young learners in online learning. Proceedings of the International Joint Conference on Arts and Humanities 2021 (IJCAH 2021).
Sweller, J. Cognitive load during problem solving: Effects on learning. Cognitive Science; 1988; 12,
van den Broek, P; Rapp, DN; Kendeou, P. Integrating memory-based and constructionist processes in accounts of reading comprehension. Discourse Processes; 2005; 39,
Wilson, M. (2005) Constructing measures: An item response modeling approach. Lawrence Erlbaum Associates
Wilson, M; Moore, S. Building out a measurement model to incorporate complexities of testing in the language domain. Language Testing; 2011; 28,
Wright, BD. Solving measurement problems with the Rasch model. Journal of Educational Measurement; 1977; 14,
Yeh, H. Exploring the perceived benefits of the process of multimodal video making in developing multiliteracies. Language Learning & Technology; 2018; 22, pp. 28-37. [DOI: https://dx.doi.org/10.64152/10125/44642]
Yin, R.K. (2018) Case study research and applications: Design and methods (6th ed.). SAGE Publications
© The Author(s) 2025. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.