Effectiveness of Structured Teacher Adaptations to an Online Content Literacy Intervention for Third Graders: A Randomized Controlled Trial during COVID-19. EdWorkingPaper No. 24-1001

Abstract

Scaling up evidence-based educational interventions to improve student outcomes presents challenges, especially when adapting to new contexts while maintaining fidelity. Structured teacher adaptations that integrate the strengths of experimental science (high fidelity) and improvement science (high adaptation) offer a viable solution to bridge the research-practice divide. This preregistered study was designed to examine the effectiveness of structured teacher adaptations to a Tier 1 content literacy intervention on Grade 3 students' (N = 1,914) engagement in asynchronous digital app and print-based reading activities, synchronous student-teacher interactions during the Zoom-delivered synchronous lessons, and student literacy outcomes. Using a multisite, cluster randomized trial design, a total of 95 teachers and their students in 26 K-5 elementary schools were randomly assigned to either a core treatment that emphasized fidelity of implementation or a structured teacher adaptations condition (i.e., Adaptive Treatment). In the structured teacher adaptations condition, teachers participated in Team-Based Learning activities that tightly coupled knowledge acquisition and application focused on improving student engagement during the COVID-19 pandemic when instruction was fully remote. Students in the adaptations condition outperformed students in the core treatment condition on measures of science reading comprehension (ES = 0.07) and science background knowledge (ES = 0.09). Implementation analyses indicated that student engagement with digital app activities and synchronous lessons was also higher in the Adaptive Treatment. Findings suggest that structured adaptations represent an asset-based approach that empowers teachers with both research knowledge and team-based practical knowledge to enhance student engagement and literacy outcomes.