1. Introduction
Recent years have seen a growing need for educational reform in undergraduate STEM education. Discipline-Based Educational Research (DBER) demonstrates that traditional instructor-centered approaches are less effective than active learning methods, where students engage more thoroughly with the subject matter (Theobald et al., 2020). A comprehensive meta-analysis by Scott Lewis highlights the positive impacts of Evidence-Based Instructional Practices (EBIPs), including Cooperative Learning, Collaborative Learning, Problem-Based Learning (PBL), Process-Oriented Guided Inquiry Learning (POGIL), Peer-Led Team Learning (PLTL), and Flipped Classrooms (Rahman & Lewis, 2020). The study demonstrates that these approaches lead to notable improvements in student academic performance, with medium to large learning gains compared to traditional teaching methods. Recent calls have thus emphasized the importance of the transition from traditional lecture-based teaching to a more iterative, evidence-based, student-centered approach (Auerbach & Schussler, 2017; Kuh, 2001). However, many faculty members remain resistant to this transition due to the perception that active learning demands substantial class time, potentially limiting the coverage of course content (Michael, 2007; Shadle et al., 2017). The adoption of a flipped classroom model can help reduce cognitive load by allowing pre-class learning and thus the in-class time can be better utilized for more active learning exercises to reinforce and apply the learned concepts, enhancing higher-order thinking (Eichler, 2022). Over the past 10 years, several studies have shown the positive effects of the hybrid/flipped classroom learning on student learning outcomes relative to the traditional instructor-centered approach (Casselman et al., 2020).
While active learning is generally linked to improved course grades and retention, it has been recently reported that it does not necessarily lead to higher-order thinking on its own (Cooper et al., 2024). It is important to consider “what is being taught” (content emphasis) in addition to “how it is taught” (pedagogical approach). A recent quasi-experimental study by Stowe and co-workers revealed that the incorporation of interactive elements, such as clicker questions and small-group discussions, into lectures resulted in no significant improvement in student performance compared to the “teaching as usual” groups (Schwarz et al., 2024). The authors infer that an instructional emphasis on core disciplinary concepts (e.g., energy, bonding interactions) and engaging students in scientific practices, such as modeling and constructing explanations, contributed to students’ ability to engage with complex chemical concepts, resulting in no significant difference in outcomes between the two groups. This study suggests that simply increasing interactivity without modifying other aspects of the learning environment, such as assessment design and instructional emphasis, may not be enough to improve student outcomes. This can be partly attributed to the fact that student knowledge is often characterized by disconnected or weakly linked concepts, which are activated based on superficial similarities rather than a deep understanding of underlying principles (diSessa, 1993). Learning science involves transforming these isolated elements into a coherent and systematic understanding and complex concepts should be scaffolded and built around core disciplinary ideas to foster higher-order thinking and reinforce understanding. The three-dimensional learning (3DL) framework is an effective approach to implement higher-order thinking by guiding students to develop knowledge around the core disciplinary ideas, apply that knowledge through scientific practices, and connect it to crosscutting concepts across different scientific fields (Laverty et al., 2016).
Pre-class learning in flipped classrooms allows students to engage with the foundational content at their own pace, thereby improving their preparedness for in-class active learning. This approach helps students reinforce their foundational knowledge, creating a solid base for constructing new ideas and integrating it more coherently within their existing knowledge framework. Pre-class learning also ensures that in-class active learning explorations remain within each student’s zone of proximal development, addressing any variability in their background knowledge and fostering a more equitable learning environment. This is especially valuable in large enrollment gateway courses like general chemistry with students from diverse STEM majors. Building on these principles, the ongoing focus of our research group is to use the flipped classroom to promote more meaningful learning. Flipped classroom structures have been recognized as a potential approach to reduce the cognitive load compared to traditional lecture-based content delivery. Yet, the initial stages of concept development in flipped classroom modules are insufficiently supported by the typical pre-class materials. This gap likely stems from students primarily engaging in passive activities, such as listening or note-taking, in the online learning environment. However, concept development aligns closely with Constructivist learning theories, which emphasize that knowledge is actively constructed through critical thinking, problem-solving, and collaborative learning rather than passively absorbed. Considering the critical need to integrate concept development more intentionally into the flipped classroom, we previously implemented in-class concept development activities prior to the standard flipped classroom learning modules, which resulted in both short-term and intermediate-term performance gains in a large classroom setting (Wu et al., 2021). Four of these activities in the study integrated Phet interactive simulations to engage students in the exploration phase of the learning cycle.
Interactive visualization tools like Phet simulations have proven to be uniquely transformative for STEM education by allowing students to explore the concepts through hands-on simulated experimentation, and build understanding through meaningful, contextualized experiences. These platforms offer multiple dynamic representations to make the invisible visible and support the inquiry process through multiple trials with rapid feedback. Engaging with these simulations enables users to build their own mental models and deepen their scientific comprehension, creating a meaningful and enjoyable learning experience (Wieman et al., 2008). A review of 31 studies showed that PhET simulations significantly enhanced students’ conceptual understanding in physics (Banda & Nzabahimana, 2021). Another pretest–post-test experimental study investigated the application of PhET virtual simulations in teaching motion kinematics to biology students, reporting a significant enhancement in their cognitive learning outcomes (Verawati et al., 2022). The inclusion of simulation-based activities in classes has yielded positive results across STEM disciples. However, it is challenging to implement such activities in large enrollment classes, particularly in introductory courses with over 100 students, and they are time-consuming to integrate into the curriculum. Addressing this, recent studies explored the use of PhET simulations in flipped classroom modules, where students independently engage with the simulations outside of class to explore and develop their understanding of concepts (Stang et al., 2016). These studies demonstrated an enhancement in student understanding and interest in learning chemistry with the incorporation of PhET simulations. Nevertheless, subsequent studies revealed the need for higher-level scaffolding to meaningfully engage students with the simulation in out-of-class environments (Adams et al., 2015). Despite appropriate scaffolding, learners may still experience difficulty in identifying key features or may misinterpret elements of complex simulations (Rutten et al., 2012). This led to the incorporation of screencasts to guide students through the simulations (Herrington & Sweeder, 2024). Use of screencasts helped instructors to focus students’ attention on key features of the simulation, thus reducing the cognitive load and addressing potential misinterpretations of the simulation’s content or functionality. While the out-of-class use of simulations has demonstrated positive effects on student learning, this environment primarily supports concept exploration. It limits opportunities for peer discussions, collaboration, and immediate feedback from instructors. In the present study, we leverage the co-curricular discussion group sections (recitations) to incorporate collaborative concept development activities into the curriculum. Simulation-based concept development activities were introduced into discussion group sections to promote active collaboration and pre-class learning. The impact of these activities on students’ conceptual learning was evaluated using a two-group quasi-experimental repeated-measures post-test design. Immediate and intermediate-term concept retention was assessed through post-activity assessments administered after the activities and final exam assessments conducted at the end of the term.
1.1. Theoretical Frameworks
Students often find it difficult to develop a comprehensive understanding of chemistry concepts. Johnstone proposed that one reason science is challenging for students is that it requires multilevel thought, and a coherent knowledge in chemistry requires understanding at the three levels: macroscopic (the observables), sub-microscopic (the particles, atoms, ions, molecules), and symbolic (the chemical equations) (Johnstone, 1991). It is especially challenging for students to visualize or imagine the unseen sub-microscopic level, particularly as this ability depends on their spatial reasoning skills. Animations and simulations have been proven to help students develop mental models that improve their understanding of the atomic and molecular phenomena. By providing dynamic visualizations, these tools allow students to connect microscopic concepts to macroscopic levels and formulate the corresponding symbolic equations. Using PhET (Physics Education Technology) and AACT (American Association of Chemistry Teachers) simulations to visualize the sub-microscopic level, we aimed to foster concept development within Johnstone’s triangle, encouraging students to engage with all three ‘levels’ simultaneously.
Pre-class learning typically involves students passively watching videos, taking notes, or independently participating in activities. According to the Constructivist approach, knowledge is actively built by the learner through their experiences and social interactions. Therefore, critical thinking, peer discussions, and collaborative learning are integral to the process of knowledge creation, alongside content exploration. Our concept development activities are thus specifically designed to foster collaborative learning and critical thinking. The smaller class sizes in the discussion sections fostered group work and facilitated meaningful interactions among students and with the instructor.
Although the Constructivist framework is effective in fostering deep understanding, students exploring ideas on their own may misinterpret and retain misunderstandings of key conceptual ideas across different disciplines. According to Ausubel’s theory of learning, meaningful learning occurs when new ideas are integrated into a large existing framework of knowledge in a sensible and coherent manner. Therefore, proper scaffolding of the activity is essential to promote meaningful learning. In this study, through adequate scaffolding, students are prompted to use the models or dynamic representations from the simulation as scientific practices along with the disciplinary core ideas to explain the observed phenomena. This closely aligns with the principles of the 3DL, although our activities are strictly not developed on this framework.
By integrating interactive simulations with Constructivist and Ausubel’s theories, we designed scaffolded concept development activities to promote conceptual understanding and support pre-class learning. These activities were implemented in the discussion sections to actively engage students, encourage peer collaboration, and provide opportunities for immediate instructor feedback. We hypothesize that providing visual representations of molecular-level phenomena through interactive simulations combined with opportunities for collaboration and adequate scaffolding around core disciplinary ideas can promote greater conceptual learning and retention compared to traditional instructor-centered methods.
1.2. Research Questions
The concept development activities were implemented in four discussion group sections. The primary goal was to utilize simulation-based activities to promote concept development. Another important aim was to demonstrate the leverage of co-curricular discussion group sessions to implement collaborative pre-class learning activities in large enrollment courses. To better understand the impact of these activities and identify potential areas for improvement, the following research questions were examined: What is the impact of simulation-based concept development activities on student conceptual learning relative to the traditional instructor-centered approach? What is the impact of simulation-based concept development activities on immediate and intermediate-term retention of conceptual learning?
2. Materials and Methods
This study was conducted in a second-quarter general chemistry course (CHEM 001B) taken by all science majors at a large research-intensive public university that is designated as a Hispanic and Asian American and Native American Pacific Islander Serving Institution (HSI & AANAPISI) during spring 2024. The course met over 10 weeks, twice each week for 80 min (total enrollment = 253 students), and associated discussion/recitation sections met once each week for 50 min. The in-class activities were administered in the weekly discussion sections. A two-group, quasi-experimental, repeated-measures, post-test design was used to evaluate the impact of the collaborative concept development activities. The treatment and teaching-as-usual conditions were administered across eight different discussion group/recitation classroom sections, which were all embedded within the same lecture course. The total 253 students of CHEM001B were distributed among those eight discussion group sections, with each section consisted of 30–32 students. Three important concept areas in second-term general chemistry, phases and phase changes, colligative properties, and chemical kinetics, were chosen for the concept development activities. Phases and chemical kinetics activities were designed based on the available PhET simulations, and the colligative properties activity based on an AACT simulation. The students in the experimental group were involved in collaborative concept development activities on the above-mentioned topics, whereas the control group had instructor-led lectures. Both the groups participated in a learning assessment conducted after the activity or the instruction. A final exam assessment was also conducted at the end of the term. The scores from the assessments were used to measure the immediate and intermediate-term retention of the concepts between the two groups. Each teaching group was facilitated by a separate teaching assistant (TA), with one TA instructing the control groups and the other TA instructing the experimental groups. The instruction in the discussion sections preceded the lectures on the three chosen topics.
2.1. Structure of Instruction in the Discussion Sections
Students in the experimental group were informed two days in advance about the learning activity and were advised to bring laptops or compatible devices, with an instructor-prepared guide and a link to the simulations provided through the learning management system. They were also informed about the on-campus laptop lending services. Following the introduction and presentation of learning goals, these students were provided with hard copies of the activities and given 25–30 min to complete them. Students worked collaboratively in groups of 3–4 to complete the guided enquiry activity by exploring the interactive simulations, while the instructor circulated around the classroom to answer any questions. This was followed by an instructor-led, open-class discussion for around 10 min in which the instructor posed collective questions to the class, and students provided responses. In contrast, the students in the control group engaged in instructor-led lectures using PowerPoint slides including screenshots from the respective simulations. Figure 1 summarizes the experimental method and structure of the recitation sections. To measure the impact of these activities on students’ conceptual learning compared to the traditional instructor-centered approach and the immediate-term retention, post-activity learning assessments consisting of open-ended questions were administered in the two groups during the last 10 min of the class period, and students were offered 2 extra credit points for completing the assessment. Intermediate-term retention was measured by administering analogous 3DL question items on the final exam. The in-class activities on colligative properties and chemical kinetics have been published on the VIPEr and PhET Interactive Simulations platforms, respectively (Ajayan & Eichler, 2024a, 2024b). The phases in-class activity and all the assessment items can be found in the Supplementary Materials.
2.2. Analysis
In order to account for the potential variance in incoming student knowledge across the treatment groups, students completed a multiple-choice instrument, the Chemical Concepts Inventory (CCI), at the beginning of the course during the first week of instruction, and an independent samples t-test was conducted to evaluate any variance in the incoming students’ academic performance. The impact of the collaborative concept development activities on students’ conceptual understanding was evaluated by analyzing the scores from the instructor-made post-activity learning assessment in comparison with the control group. The student retention of the concepts developed was tested during the final exam (it is noted that a repeated measures analysis was not used since the assessment items on the final exam were altered from the post-activity assessments to reduce the potential test–retest effect). For chemical kinetics, analysis of covariance (ANCOVA) was employed to compare post-activity and final exam assessment scores between the experimental and control groups, using Chemical Concept Inventory scores as a covariate to account for prior knowledge. The assumptions for ANCOVA were met for the chemical kinetics assessments. The normal distribution of the dependent variable and covariate was evaluated using the skewness and kurtosis of the distribution plot, equality of variance for the dependent variable was examined using Levene’s test of equality of error variances, and the linearity between the dependent variable and covariate and homogeneity of regression slopes were evaluated using visual inspection of the plots (since the post-activity and final exam evaluations utilized two distinct assessment items, a repeated measures ANCOVA was not employed for the analysis). To address any potential family-wise error that may have arisen due to the use of multiple tests, a Bonferroni correction was applied (p = 0.05/2 = 0.025) (Bender & Lange, 2001). Since the phases and colligative properties assessments violated the assumption of homogeneity of regression slopes, multiple linear regression was used. The tests for assumptions, raw output tables, and model summaries for these analyses are provided in the Supporting Information. The regression coefficients and model summaries for the colligative properties final exam assessment are provided in Table 3, while those for other assessments are available in the Supporting Information (SI). A correlation analysis was carried out to find the relationship between assessment and Concept Inventory scores across the two groups. All the statistical analyses were carried out using the IBM SPSS software program, version 29.0.1.1, and all data were collected and analyzed under an approved IRB protocol (UCR protocol #22127).
3. Results
An independent sample t-test based on the Chemical Concept Inventory (CCI) revealed no significant difference in the incoming academic preparation between the two groups (Table S2, p = 0.606). To evaluate the impact of the collaborative concept development activities and the immediate-term retention, the student performance was evaluated for each of the three activities at the end of the class period. The intermediate-term retention was examined for two of the activities during the final exam. Descriptive statistics shown in Table 1 reveal that the experimental group had higher scores in all the post-activity and final exam assessments compared to the control group. Figure 2 and Table 2 show the ANCOVA results for the chemical kinetics assessments, revealing a statistically significant difference between the two groups for the post-activity assessment (p < 0.001) and the final exam assessment (p < 0.025). The calculated effect sizes for the two assessments calculated from Cohen’s f are 0.570 and 0.160, respectively. The former represents a large effect size, implying that the kinetics concept development activities had a noticeable positive impact on the students’ performance, though its magnitude was reduced over time.
Since the relationship between assessment scores and the Chemical Concept Inventory did not appear to be equivalent across the control and experimental groups for the phases and colligative properties assessments, a multiple linear regression analysis that included a treatment–Concept Inventory score interaction term was carried out to determine if treatment was a predictor of the assessment scores for these two groups. A hierarchical analysis was carried out in which the treatment and Concept Inventory score were first included as independent variables, and then a subsequent model was created that included the treatment, Concept Inventory score, and a treatment*Concept Inventory score interaction as independent variables (see Table 3, Tables S6, S12 and S25). This analysis indicated that the change in R2 was significant upon adding the treatment*Concept Inventory score interaction term and hence this model was retained in the analysis. This suggests that the Concept Inventory score modulates the assessment score. The initial regression analysis revealed a significant effect of the treatment on all the assessments. However, when the treatment*Concept Inventory score interaction term was introduced, the significance of the main effects disappeared, showing no significant difference between the two treatment groups.
It was of interest to determine how the Concept Inventory score impacted the assessment scores across the treatment groups and therefore separate correlation analyses were carried out to explore the relationship between the Concept Inventory and assessment scores. Analysis revealed no significant correlation between the Concept Inventory and assessment scores in the control group for any of the assessments, whereas a significant correlation was found between the Concept Inventory and the assessment scores in the experimental group for all except the chemical kinetics final exam assessment, as shown in Table 4. The Pearson correlation coefficient ranges from 0.20 to 0.35, indicating a weak to moderate relationship between the Concept Inventory and assessment scores in the experimental group.
To determine how the incoming knowledge impacts the assessment score across the treatment groups, the students were divided into three groups based on the Concept Inventory scores (high score group—top third of Concept Inventory scores; medium score group—middle third of Concept Inventory scores; low score group—lower third of Concept Inventory scores). A best-fit linear plot of assessment score vs. treatment was created for each of the assessments across the three levels of Concept Inventory scores, which revealed that students in the high Concept Inventory score group appeared to benefit more from the treatment (see Figures S4, S8, S11, S15 and S18).
4. Discussion
In relation to Research Question 1, the statistically significant differences observed between the experimental and control groups on both the chemical kinetics post-activity and final exam assessments indicated a substantial positive impact of the simulation-based activity on students’ conceptual understanding. Regarding Research Question 2, we compared the effect sizes from the post-activity and final exam assessments to evaluate immediate and intermediate-term retention. While the final exam results remained statistically significant, the effect size appeared to be smaller. A comparison of the post-activity and final exam assessments within the experimental group revealed a 28.4% decrease in the mean score from the post-activity to the final exam assessment. This suggests that although the benefits of the intervention persisted over time, their magnitude diminished, potentially due to the increased complexity of the topic. The reduction in effect size may also be attributed to the confounding effects from the lecture, which took place after the first assessment. In the lecture, students from both the groups participated in flipped and active learning exercises. The lecture activities emphasized conceptual learning through in-class scaffolded worksheets and poll questions where students received multiple opportunities to engage with the material, suggesting that in-class learning may have narrowed the performance gap between the groups over the period from the post-activity assessment to the final exam.
Due to the significant treatment*Concept Inventory score interaction effects, a statistically significant effect of treatment was not observed for the phases and colligative properties assessments. This might be attributed to the fact that the Chemical Concept Inventory had more questions related to solutions and phases and few on kinetics. Therefore, students performing better on the Concept Inventory test might be expected to have higher scores in the related assessments. The Concept Inventory thus may not solely measure or account for students’ incoming academic preparation or prior knowledge. Moreover, since students completed the Concept Inventory independently in an un-proctored environment, there is a possibility that some did not fully engage with the assessment or might have relied on supplementary materials, potentially affecting the reliability of this assessment across the study population. The correlation analysis across the two groups revealed a weak to moderate correlation between the Concept Inventory and assessment scores in the experimental group except for the chemical kinetics final exam and no significant correlation in the control group in any of the assessments. Incoming knowledge thus appeared to moderate the impact of the concept development activities. Given that there was no statistical significance between the two groups in the incoming Concept Inventory scores, this effect may be attributed to the scaffolded design of the activities. The scaffolded or 3DL nature of the activities provides students opportunities to connect new ideas with prior knowledge and might have more benefitted students with a high level of prior knowledge. The impact of this scaffolding appeared to be corroborated by the best-fit linear plots of assessment score vs. treatment across the three levels of Concept Inventory scores. These data indicated that students in the high Concept Inventory score tertile benefitted more from the treatment.
Compared to the control group, which engaged in the instructor-centered lectures where students passively listened to the transmitted information, the concept development activities conducted in the experimental group were more engaging. Students in the experimental group were encouraged to collaborate with their peers to co-construct knowledge actively. They were provided opportunities to engage with the instructor, enabling them to receive immediate feedback and clarify questions in real time. Furthermore, the dynamic and interactive nature of the simulations enabled students to manipulate variables within a controlled environment, allowing them to observe and comprehend the outcomes in real time. This fostered an active, Constructivist learning environment. Additionally, the simulations provided students with a vivid representation of sub-microscopic phenomena, which they were able to connect to the macroscopic properties, supporting learning within Johnstone’s triangle. For example, in the colligative properties simulation, students could observe the self-assembly of water molecules forming the lattice structure of ice, with solute molecules interspersing among the water molecules and hindering this process. This visual representation helped them connect the abstract molecular dynamics to the macroscopic property of freezing-point depression, enhancing their conceptual understanding. Similarly, for the chemical kinetics activity, students often harbor the misconception that no reaction occurs at all if the total average energy is lower than the activation energy. However, through the simulation, students observed that reaction can still take place but at a very slow rate and yield. The guided and scaffolded nature activity along with the interactive simulation helped them understand that the total energy represents an average, and some molecules may possess enough energy to overcome the activation barrier, leading to the formation of products. These activities thus addressed the misconceptions and clarified complex concepts through dynamic, interactive learning, providing a vivid and comprehensive understanding of sub-microscopic events.
The 3DL-inspired design of the activity encouraged students to utilize representation from the simulation as the scientific practices along with core disciplinary ideas such as the molecular-level structure, macroscopic properties, and energetics to provide explanations for the observed phenomena, fostering concept development and a deeper understanding of the topic beyond active learning. For example, in the phases activity, students were asked to draw the atomic or molecular structures and use them to identify the underlying intermolecular forces, explaining how these forces relate to the observed phase transition temperatures shown in the simulation. Moreover, the scaffolded nature of the activities guided students to build on their prior knowledge, helping them to make connections between new ideas and their existing knowledge frameworks. This aligns with Ausubel’s theory of meaningful learning, which emphasizes the importance of anchoring new knowledge into the existing cognitive structures. By connecting new information to what they already knew, students were able to develop a deeper and coherent understanding of the concepts. Some of the lecture activities were also informed by the 3DL framework of learning, and students from both the groups were given opportunities to construct knowledge, which may have led to a smaller knowledge gap on the final assessment.
Simulation-based active learning approaches offer numerous benefits in improving students’ conceptual understanding by providing representations that make the invisible visible. These approaches enhance the inquiry process by allowing multiple trials with rapid feedback, fostering deeper engagement and learning. However, implementing these approaches in classrooms is challenging due to time constraints and large class sizes. Meanwhile, out-of-class activities limit opportunities for peer discussion and collaborative learning, both of which are essential components of the Constructivist approach of learning. The results from our study show the strategic use of discussion or recitation sessions to effectively integrate learning activities into the curriculum. These sessions offer a space for students to collaborate, exchange ideas, and clarify concepts in a more interactive environment. Finally, they allow instructors to address misconceptions in real time, fostering a deeper level of conceptual understanding.
4.1. Limitations
Due to the Concept Inventory–treatment interaction effects, the main effect of the phases and colligative properties activities on the assessments were not significantly revealed in the analysis. Furthermore, prior knowledge appeared to have a moderate impact on the students in the experimental group. Students with a higher level of prior knowledge were found to benefit more from these activities. Further studies are needed to optimize the structure of these activities to better support low-performing students. Moreover, the Chemical Concept Inventory instrument alone may not fully account for or accurately reflect students’ prior knowledge.
Another limitation of this study is that two different graduate teaching assistants (TA) taught the two groups, which could have introduced instructor variability and influenced the assessment performance across the two study groups. Additionally, the impact of in-class learning may have been a confounding factor on the final exam assessment used to measure knowledge retention. The interventions and assessments conducted in this study focus on a subset of the course’s learning objectives. As such, their impact may not be fully representative of all learning objectives, highlighting the need for further research to explore broader effects across the entire curriculum.
Although we have demonstrated the use of recitation sections for pre-class concept development, they cannot serve as the sole source of pre-class learning, as the limited 50 min weekly time allotted for these sessions restricts the scope of the learning objectives that can be covered. While concept development is essential and aligns closely with the Constructivist approach, students are also often assessed on problem-solving skills to apply these concepts to problems. As a result, discussion sections are commonly utilized as problem-solving sessions. The transition of these sections from problem-solving to concept development can hamper the development of analytical proficiency, which is needed for students to excel in such problem-solving assessments. Moreover, this type of integration is hard to implement at smaller institutions without TAs to instruct the recitation sections. Additionally, ensuring device and technology access for all students to use simulations can be challenging in institutions with limited financial resources for device support.
4.2. Implications for Instruction and Conclusions
With respect to Research Question 1, the results were mixed. A statistically significant difference between the two treatment groups on the chemical kinetics assessments indicated a positive impact of the simulation-based activity on students’ conceptual learning. However, no significant difference was observed between the groups on the phases and colligative properties assessments. In relation to Research Question 2, the significant results from the chemical kinetics assessments demonstrated both immediate and intermediate-term retention. However, the decrease in effect sizes suggested that while the impact of the intervention persisted over time, its magnitude diminished.
The study shows how an active learning environment can be structured to emphasize meaningful learning. Incorporating collaborative and constructive learning with a focus on content emphasis can lead to significant learning outcomes. Proper scaffolding of complex concepts around core disciplinary ideas builds on prior knowledge, helping to make connections between new ideas and their existing knowledge frameworks, promoting deeper learning that extends beyond traditional active learning approaches. Additionally, using simulations for concept development benefits diverse learners, particularly those with limited spatial reasoning skills, by helping them connect molecular representations to macroscopic phenomena. This approach aligns with Johnstone’s triangle, enabling students to navigate and integrate multiple levels of chemical representation effectively. Notably, the results of the phases and colligative properties assessments reported here indicate that instructors should pay careful attention to the incoming knowledge of their students, making sure to provide the necessary guidance for students with less incoming knowledge so they can successfully complete the concept development activities.
Pre-class learning in flipped classrooms helps to reduce the cognitive load, ensuring that students are prepared for more effective in-class active learning. Although the use of simulation-based activities in pre-class learning has been shown to effectively help students to develop particle-level mental models of fundamental concepts, their out-of-class nature can limit opportunities for Social Constructivist learning. Implementing these activities within discussion sections fosters collaboration and aligns with Social Constructivist principles. Discussion sections are often used for reviewing the concepts or problem-solving and typically focus on lower-order cognitive skills. Transitioning these sessions into concept development activities encourages higher-order thinking while integrating pre-class learning into a collaborative setting. The smaller size of discussion sections enhances student–student and student–instructor interactions, providing opportunities for immediate, real-time feedback.
In summary, it has been shown that the strategic use of interactive simulations during recitation/discussion sections can promote concept development. A positive impact from these classroom activities was generally observed, with the most significant impact being associated with the concept that is likely least familiar to the students. Key features of these activities included fostering social interaction, providing feedback opportunities, and using a scaffolded design to help students navigate higher-order conceptual learning. This study provides a template for instructors in higher education STEM to structure teaching environments in a way that emphasizes meaningful learning.
Investigation, J.F.E.; data analysis, R.C.; data collection, data analysis, writing—original draft, P.A. All authors have read and agreed to the published version of the manuscript.
The study was conducted in accordance with the Institutional Review Board of the University of California-Riverside (protocol #22127; initial approval 7 September 2022).
Informed consent was obtained from all subjects involved in the study as approved in the UCR IRB protocol #22127.
The statistical analysis output data can be found in the
The authors declare no conflicts of interest.
The following abbreviations are used in this manuscript:
| 3DL | Three-Dimensional Learning |
| ANCOVA | Analysis of Covariance |
| CCI | Chemical Concept Inventory |
| PhET | Physics Education Technology |
| AACT | American Association of Chemistry Teachers |
Footnotes
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Figure 1 Overview of the experimental design and structure of discussion sections.
Figure 2 Bar graph showing the marginal mean score (adjusted for Concept Inventory scores) comparison between the experimental group and the control group on chemical kinetics post-activity and final exam assessments (range of scores possible: 0–10 and 0–4 for the post activity and final exam assessments, respectively). * Designated mean difference is significant.
Summary of descriptive statistics.
| Assessment Type | Treatment Group | Mean | Standard Deviation | |
|---|---|---|---|---|
| Chemical Concept Inventory | Control (n = 103) | 11.0 | 4.46 | |
| Experimental (n = 110) | 10.7 | 4.57 | ||
| Post-Activity Assessments | Phases | Control (n = 104) | 4.34 | 2.55 |
| Experimental (n = 106) | 5.75 | 2.28 | ||
| Colligative Properties | Control (n = 100) | 2.53 | 1.73 | |
| Experimental (n = 112) | 4.34 | 1.83 | ||
| Chemical Kinetics | Control (n = 83) | 5.43 a | 0.190 | |
| Experimental (n = 98) | 7.41 a | 0.175 | ||
| Final Exam Assessments | Colligative Properties | Control (n = 120) | 3.31 | 1.66 |
| Experimental (n = 121) | 4.04 | 1.73 | ||
| Chemical Kinetics | Control (n = 94) | 1.47 a | 0.110 | |
| Experimental (n = 104) | 1.83 a | 0.108 | ||
a Marginal mean (adjusted for Concept Inventory scores).
ANCOVA results from the kinetics post-activity and final exam assessments. Cohen’s f was calculated from η2 using the formula, Cohen’s f = f =
| Chemical Kinetics Assessment | Post-Activity | Final Exam |
|---|---|---|
| F | 58.3 (1, 178) | 5.30 (1, 195) |
| p | <0.001 | 0.0220 |
| Partial η2 | 0.247 | 0.0260 |
| Effect size; Cohen’s f | 0.573 | 0.163 |
Multiple regression model to evaluate the impact of treatment and Concept Inventory score on colligative properties final exam assessment. (a) Summary of model with only treatment and Concept Inventory score as the independent variables; (b) coefficients of model with only treatment and Concept Inventory score as the independent variables; (c) summary of model that includes treatment*Concept Inventory interaction term with treatment and Concept Inventory score as the independent variables; (d) coefficients of model that includes treatment*Concept Inventory interaction term with treatment and Concept Inventory score as the independent variables.
| (a) | ||||||
| Model Summary | ||||||
| Model | R | R 2 | Adjusted R2 | Std. Error of the Estimate | ||
| 1 | 0.304 a | 0.0920 | 0.0830 | 1.67 | ||
| a Predictors: (Constant), Concept Inventory, Treatment | ||||||
| (b) | ||||||
| Coefficients | ||||||
| Model | Unstandardized Coefficients | Standardized Coefficients | t | p | ||
| B | Std. Error | β | ||||
| 1 | (Constant) | 2.46 | 0.341 | 7.22 | <0.001 | |
| Treatment | 0.755 | 0.238 | 0.216 | 3.17 | 0.002 | |
| Concept Inventory | 0.0850 | 0.0270 | 0.214 | 3.14 | 0.002 | |
| Dependent Variable: Colligative properties final exam score | ||||||
| (c) | ||||||
| Model Summary | ||||||
| Model | R | R 2 | Adjusted R2 | Std. Error of the Estimate | ||
| 1 | 0.339 a | 0.115 | 0.101 | 1.66 | ||
| a Predictors: (Constant), Concept Inventory, Treatment*Concept Inventory Interaction | ||||||
| (d) | ||||||
| Coefficients | ||||||
| Model | Unstandardized Coefficients | Standardized Coefficients | t | p | ||
| B | Std. Error | β | ||||
| 1 | (Constant) | 3.18 | 0.467 | 6.81 | <0.001 | |
| Treatment | −0.547 | 0.630 | −0.157 | −0.869 | 0.386 | |
| Treatment*Concept Inventory Interaction | 0.120 | 0.0540 | 0.435 | 2.23 | 0.0270 | |
| Concept Inventory | 0.0180 | 0.0400 | 0.0470 | 0.461 | 0.645 | |
| Dependent Variable: Colligative properties final exam score | ||||||
Pearson’s correlation between assessment scores and Concept Inventory scores.
| Treatment Group | Assessment Score*Concept Inventory Correlation | Pearson Correlation | p |
|---|---|---|---|
| Control | Phases post activity | −0.0880 | 0.429 |
| Colligative properties post activity | −0.0380 | 0.734 | |
| Chemical kinetics post activity | −0.0600 | 0.590 | |
| Colligative properties final exam | 0.0480 | 0.646 | |
| Chemical kinetics final exam | 0.119 | 0.255 | |
| Experimental | Phases post activity | 0.285 | 0.05 |
| Colligative properties | 0.329 | <0.001 | |
| Chemical kinetics post activity | 0.204 | 0.044 | |
| Colligative properties final exam | 0.356 | <0.001 | |
| Chemical kinetics final exam | 0.0870 | 0.380 |
Supplementary Materials
The following supporting information can be downloaded at
Adams, W. K.; Armstrong, Z.; Galovich, C. Can students learn from PhET sims at home, alone. Physics Education Research (PER) Conference on Critical Examination of Laboratory-Centered Instruction and Experimental Research in Physics Education; College Park, MD, USA, July 29–30; 2015; pp. 23-26.
Ajayan, P.; Eichler, J. F. Colligative properties; VIPEr: 2024a; Available online: https://www.ionicviper.org/class-activity/class-activity-colligative-properties (accessed on 29 April 2025).
Ajayan, P.; Eichler, J. F. Using the reaction and rates PhET simulation: Concept development activity for kinetics and rates of reactions; PhET Interactive Simulations, UnIversity of Colorado Boulder: 2024b; Available online: https://phet.colorado.edu/en/activities/7862 (accessed on 29 April 2025).
Auerbach, A. J.; Schussler, E. A vision and change reform of introductory biology shifts faculty perceptions and use of active learning. CBE—Life Sciences Education; 2017; 16,
Banda, H. J.; Nzabahimana, J. Effect of integrating physics education technology simulations on students’ conceptual understanding in physics: A review of literature. Physical Review Physics Education Research; 2021; 17,
Bender, R.; Lange, S. Adjusting for multiple testing—When and how?. Journal of Clinical Epidemiology; 2001; 54,
Casselman, M. D.; Atit, K.; Henbest, G.; Guregyan, C.; Mortezaei, K.; Eichler, J. F. Dissecting the flipped classroom: Using a randomized controlled trial experiment to determine when student learning occurs. Journal of Chemical Education; 2020; 97,
Cooper, M. M.; Caballero, M. D.; Carmel, J. H.; Duffy, E. M.; Ebert-May, D.; Fata-Hartley, C. L.; Herrington, D. G.; Laverty, J. T.; Nelson, P. C.; Posey, L. A. Beyond active learning: Using 3-dimensional learning to create scientifically authentic, student-centered classrooms. PLoS ONE; 2024; 19,
diSessa, A. A. Toward an epistemology of physics. Cognition and Instruction; 1993; 10,
Eichler, J. F. Future of the flipped classroom in chemistry education: Recognizing the value of independent preclass learning and promoting deeper understanding of chemical ways of thinking during in-person instruction. Journal of Chemical Education; 2022; 99,
Herrington, D. G.; Sweeder, R. D. Using simulations and screencasts in online preclass activities to support student building of mental models. Education Sciences; 2024; 14,
Johnstone, A. H. Why is science difficult to learn? Things are seldom what they seem. Journal of Computer Assisted Learning; 1991; 7,
Kuh, G. D. Assessing what really matters to student learning inside the national survey of student engagement. Change: The Magazine of Higher Learning; 2001; 33,
Laverty, J. T.; Underwood, S. M.; Matz, R. L.; Posey, L. A.; Carmel, J. H.; Caballero, M. D.; Fata-Hartley, C. L.; Ebert-May, D.; Jardeleza, S. E.; Cooper, M. M. Characterizing college science assessments: The three-dimensional learning assessment protocol. PLoS ONE; 2016; 11,
Michael, J. Faculty perceptions about barriers to active learning. College Teaching; 2007; 55,
Rahman, T.; Lewis, S. E. Evaluating the evidence base for evidence-based instructional practices in chemistry through meta-analysis. Journal of Research in Science Teaching; 2020; 57,
Rutten, N.; van Joolingen, W. R.; van der Veen, J. T. The learning effects of computer simulations in science education. Computers & Education; 2012; 58,
Schwarz, C. E.; DeGlopper, K. S.; Esselman, B. J.; Stowe, R. L. Tweaking instructional practices was not the answer: How increasing the interactivity of a model-centered organic chemistry course affected student outcomes. Journal of Chemical Education; 2024; 101,
Shadle, S. E.; Marker, A.; Earl, B. Faculty drivers and barriers: Laying the groundwork for undergraduate STEM education reform in academic departments. International Journal of STEM Education; 2017; 4,
Stang, J. B.; Barker, M.; Perez, S.; Ives, J.; Roll, I. Active learning in pre-class assignments: Exploring the use of interactive simulations to enhance reading assignments. arXiv; 2016; arXiv: 1607.04588
Theobald, E. J.; Hill, M. J.; Tran, E.; Agrawal, S.; Arroyo, E. N.; Behling, S.; Chambwe, N.; Cintrón, D. L.; Cooper, J. D.; Dunster, G.; Grummer, J. A.; Hennessey, K.; Hsiao, J.; Iranon, N.; Jones, L.; Jordt, H.; Keller, M.; Lacey, M. E.; Littlefield, C. E. … Freeman, S. Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math. Proceedings of the National Academy of Sciences USA; 2020; 117,
Verawati, N. N. S. P.; Handriani, L. S.; Prahani, B. K. The experimental experience of motion kinematics in biology class using PhET virtual simulation and its impact on learning outcomes. International Journal of Essential Competencies in Education; 2022; 1,
Wieman, C. E.; Adams, W. K.; Perkins, K. K. PhET: Simulations that enhance learning. Science; 2008; 322,
Wu, H.-T.; Mortezaei, K.; Alvelais, T.; Henbest, G.; Murphy, C.; Yezierski, E. J.; Eichler, J. F. Incorporating concept development activities into a flipped classroom structure: Using PhET simulations to put a twist on the flip. Chemistry Education Research and Practice; 2021; 22,
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
