1. Introduction
Recently, flipped learning education has been applied to increase the efficiency of education and learning processes due to rapid changes in the educational environment and to convert students into active learners [1,2].
Flipped learning is a learner-centered teaching method in which learners self-learn online before class, asking questions to the instructor and discussing the content with peers during class [3]. This provides a dynamic mutual learning environment for professors and allows students to focus creatively on their learning needs without time and space constraints [4]. In flipped learning, students manage the learning process through learning content before class, so students’ self-discipline and self-regulation can be improved [5]. Additionally, during this class, students can participate in various active learning activities, such as question-and-answer, discussion, group activities, problem-solving, and receiving immediate feedback [6]. A positive change in academic achievement levels has been observed in medical education with an increasing interest in flipped learning classes and reducing boredom [7].
The first study that applied flip learning in the field of nursing education was published in 2013 [8]. A flipped learning model has been applied in nursing education to reinforce nursing competence according to changes in the clinical environment. It is difficult to acquire sufficient knowledge within limited class hours and clinical practice with the existing teaching methods. This is due to the lack of both direct nursing experience in observation-oriented practice and a connection between the theory and clinical practice. Upon employment in a hospital, a gap is noticed in performing practical tasks in clinical nursing [9]. Thus, nursing educators applied flipped learning to improve the quality of education and prepare for complex nursing practice [1,6]. Flipped learning applied to nursing education has a large amount of learning and professional content, and students are inevitably passive in class. It helps students find lecture materials on their own, perform prior learning, and conduct self-directed learning through active learning in class [10]. It analyzes and integrates learning content and is applied to solve complex clinical situations by improving critical thinking skills [11,12]. Additionally, the learner’s perception of flipped learning also shows a positive response, and because they can watch the videos whenever they want, a high level of satisfaction with video lectures during pre-classes is observed [13]. A change in the teaching method of instructors was urged while emphasizing the importance of insight, situational awareness, critical thinking ability, and clinical reasoning to promote learning in nursing students [14].
Various studies have recently been conducted to identify the effectiveness of flipped learning in nursing education. Prior studies have influenced self-directed learning and learner satisfaction [11]. Critical thinking tendencies have significantly increased compared to conventional teaching methods [10]. Additionally, improved academic achievement [15], learning satisfaction, self-efficacy, knowledge, and communication ability [16] indicated positive effects. However, contradictory research results also exist. In clinical practice education, the experimental group that experienced flipped learning improved in clinical performance [11], as compared to the control group that received lecture-type instruction. While some studies outlined no significant results [17], others reported improved critical thinking tendencies [18]. There are limitations in understanding the effectiveness of flipped learning results, including studies in which thinking tendencies are not significant [19]. When the consistency of these individual research results is insufficient, a meta-analysis can be used to identify factors. Regarding the systematic review and meta-analysis of flipped learning applied to nursing education, two foreign studies have focused on the effects of theoretical knowledge and skill scores [20]. In Korea, no meta-analysis has been reported that systematically and quantitatively evaluated the results of flipped learning education. A meta-analysis was conducted because different educational systems and cultural differences in different countries can affect the effectiveness of flipped learning education [21].
Particularly, clinical competence, critical thinking ability, and self-directedness [12], which are the purpose of flipped learning and are important in nursing education, are contributing to the effect of flipped learning education. The application of the flipped learning technique to clinical practice has a positive effect on learning satisfaction [16,22]. Therefore, the purpose of this study was to confirm the effect size on clinical competence, critical thinking ability, self-directedness, and learning satisfaction. Based on these results, this study provides a scientific basis for evaluating the necessity and validity of flipped learning. It is a basis for educational systems and researchers with diverse cultural backgrounds, evaluating the effectiveness of flipped learning models as a teaching–learning method in nursing.
Research Aims
This study is a meta-analysis of the effects of flipped learning on educational outcome variables applied to domestic and foreign nursing education, such as clinical competence, critical thinking ability, self-directedness, and learning satisfaction. The specific purpose of this study is as follows:
First, we grasped the general characteristics of flipped learning research selected through the search process. Second, the size of the overall effect of flipped learning applied to nursing students was analyzed. Third, the magnitude of the effect of flipped learning for each outcome variable (clinical competence, critical thinking ability, self-directedness, and learning satisfaction) was analyzed.
2. Materials and Methods
2.1. Research Design
This study is a meta-analysis study to confirm the effectiveness of clinical competence, critical thinking ability, self-directedness, and learning satisfaction, variables of flipped learning applied to domestic and foreign nursing education.
2.2. Criteria for Literature Selection
We reviewed the Cochrane Handbook for Systematic Reviews of Interventions 5.1.0 [23]. The literature selection criteria were based on the key questions of Populations, Intervention, Comparison, Outcome, Study design (PICOS) per the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [24]. According to the PICOS, the study subjects (P) were domestic or overseas nursing students, the intervention (I) was flipped learning, comparative mediation (C) conducted for general lecture-style class, the intervention results (O) were related to clinical competence, critical thinking ability, self-directedness, and learning satisfaction and the study types (S) were randomized controlled trials (RCTs) and non-randomized controlled trials (NRCTs). The studies excluded from the literature selection are survey studies, qualitative studies (case studies, descriptive studies), meta-analysis, non-experimental studies, single-group pre- and post-design studies, studies in which the effect size cannot be calculated. In addition, there are studies related to subjects other than nursing students, studies published only as abstracts, and studies in which the language of literature is not English or Korean
2.3. Literature Search Strategy
2.3.1. Literature Search
The literature search in this study was conducted on documents from 1 January 2000 to 30 December 2020, using domestic and international academic search databases (Database, DB).
When selecting a search center for literature search, the COSI (Core, Standard, Ideal) model suggested by the National Library of Medicine (NLM) is used. Among them, Core is a core part of the literature search, including related domestic literature and core databases (e.g., PubMed, EMBASE, Cochrane central, etc.). In this study, the COre search database recommended by the Korea Institute of Health and Medical Sciences was used. The foreign DBs searched included PubMed, CINAHL, EMBASE, MEDLINE, and Web of Science, and the domestic DBs searched included journals and dissertations published through Research Information Service System (RISS), DBpia, Korean Studies Information Service System (KISS), and KMbase. Additional data search was supplemented with a manual search through the references. The keywords searched included “flip learning”, “flipped learning”, “upside down classroom”, “upside down lesson”, “reverse learning”, “reverse progression lesson”, “effect of flip learning”, “flip classes”, and “nursing students”, “flipped classroom”, “flipped class”, “flipped instruction”, “inverted classroom”, “inverted instruction”, “flipping classroom”, “reverse instruction”, “Students, Nursing [MeSH]”, and “nurs * student *”.
2.3.2. Data Collection and Selection
Data collection was independently searched and reviewed by two researchers, and if they did not agree, they would review the data again, until consensus was reached. A list of papers searched through domestic and foreign DBs was prepared, and duplicate papers were removed using End-Note X9, a bibliographic export program. Reviewing the abstract and full text of the manuscript confirmed whether the study satisfied the research selection criteria according to the data selection criteria. The final analysis literature was determined by excluding articles that did not meet the selection criteria. For the final selected paper, author, publication year, title, study design, study subject, control type, study subject age, subject type, pre-learning method, instruction period, outcome variables, and result values were extracted and recorded in the coding table.
2.4. Methodological Quality Assessment
In the methodological quality evaluation of the analysis targets, the RCTs were evaluated for the risk of bias using the Cochrane risk of bias tool (RoB) for randomized trials, and NRCTs were evaluated using the risk of bias tool for non-randomized studies (RoBANs). Two researchers independently conducted these evaluations, and for items that did not match, the study was reviewed together to find consensus. RoB evaluated six areas: generating random assignment order, concealing assignment order, blindfolding research participants and researchers, blindfolding results evaluation, processing insufficient results, and reporting selective results. The RoBAN evaluated eight areas: comparability of target groups, selection of target groups, confounding variables, exposure measurement, blindfolding of outcome assessors, outcome assessment, incomplete outcome data, and selective outcome reporting. Each item is rated as “low”, “uncertain”, or “high” according to the description.
2.5. Data Analysis Method
The data was analyzed using CMA (Comprehensive Meta-Analysis 2.2), a program dedicated for meta-analyses. When calculating the effect size, Cohen’s d tends to overestimate the effect size for small samples. Therefore, Hedges’ g, a corrected standardized mean difference that corrects Cohen’s d, was used [25]. For the interpretation of the effect size, Cohen [26] suggested a small effect size to range between 0.2 and 0.5, a medium effect size to be between 0.5 and 0.8, and a large effect size to be 0.8 or above. The statistical implications of the calculated effect sizes are presented with a 95% confidence interval (CI). The farther away from zero, the stronger the influence. It is interpreted as invalid if the mean effect of zero is included. Recognizing that the research method, group size, intervention method, and comparison group of the studies selected are diverse, a random-effects model that reflected the actual differences between individual studies was applied [26]. In addition, statistical heterogeneity of effect sizes was evaluated using a forest plot, and a funnel plot was used to verify publication bias [27].
2.6. Ethical Considerations
The contents and method of this study were IRB review exemption of Soonchunhyang University (IRB-202102-SB-015).
3. Results
3.1. Selection of Materials
A total of 18 studies were included in the systematic review per the data selection criteria (illustrated in Figure 1). In the first stage, 681 articles were searched using the search strategy for each database. The overlapping 314 episodes were excluded using the bibliographic management program. In the second step, 135 papers were selected by reviewing the titles and abstracts of 232 papers. In the third stage, the original texts of the 135 selected papers were reviewed, and the criteria for exclusion of the research subject and research design were applied. Of these, four non-original papers, one non-English paper, 27 non-conforming research results, six non-nursing students, and 59 studies where the research design did not match the criteria were excluded. Twenty studies were selected from 115 studies. In the fourth stage, the full texts of the 20 studies were reviewed. Two of them were excluded because the result value could not be extracted due to insufficient statistical data, and the final 18 studies were confirmed for meta-analysis.
3.2. Characteristics of Documents Subject to Systematic Literature Review
The research design of the 18 papers selected for the analysis of this study was NRCT 88.9% and RCT 11.1%, and the publication type was journal 88.9% and thesis 11.1%. As for the subject type applied to the intervention, practical subjects accounted for the most at 50%, and video (video content, video learning, lecture video) as the pre-education method was 72.2%. The sample size range of the experimental group was 22 to 287, the control group was 21 to 198, and the number of interventions was 4 to 14. The program application time was 2–3 h per session, but there were studies that did not describe the intervention method in detail. The proportion of outcome variables was 38.9% for clinical performance, 27.8% for critical thinking ability, 27.8% for self-direction, and 33.3% for learning satisfaction (Table 1).
3.3. Evaluation of Literature Quality
Considering the results of the 18 methodological quality assessments in this study, 50% of the studies had a low RoB in randomization order in two RCT papers. For the concealment items in the order of assignment, the low RoB was 50%, and the items for blindfolding of researchers had 100% uncertain RoB. In the case of blindfolding and selective reporting of results, the low RoB was 100%, and the high RoB in insufficient result data was 100%. In the 16 NRCT papers, the low bias was 100% in the subject group comparability, exposure measurement, result evaluation, and selective result reporting. In selecting a target group, the low RoB was 50%, the high RoB was 37.5%, and the uncertainty of the RoB was 12.5%. Regarding the confounding variables, the low RoB was 81.25%, and the high RoB was 18.75%. In the evaluator’s blindfold, the low RoB was 25%, and the uncertainty RoB was 75%. The unsafe result data showed that the low RoB was 25%, and the high RoB was 68.75%; the uncertainty of the RoB was 6.25%. A single group was excluded from the literature selection criteria, and papers with clear pre- and post-evaluations and those planned were included in the literature containing all expected results. The quality of the papers was found to be higher than average because of the overall evaluation (Table 1)
3.4. Effect Size of Flipped Learning
Table 2 and the forest plots present the effect size of flipped learning applied to nursing education on clinical competence, critical thinking ability, learning satisfaction, and self-directedness (Figure 2). The effect size of the entire study was Hedges’ g = 0.68 (95% CI = 0.43~0.92), showing a statistically significant median effect size.
The heterogeneity of the total effect size was I2 = 90.7% (Q = 246.67, p < 0.001), appearing to fall within a large range [18]. The effect size of each effect variable of the flipped learning intervention was statistically significant because the 95% CI excluded 0, and the critical thinking ability was Hedges’ g = 0.87, showing the largest effect size. Next, learning satisfaction (Hedges’ g = 0.79), clinical competence (Hedges’ g = 0.53), and self-directedness (Hedges’ g = 0.37) are shown in Table 2.
By verifying the intervention effect, the subject type and the prior learning method type were used as modulators. In the subject method type, the effect size value is calculated by the practical subjects, the practical and theoretical integrated subjects, and the theoretical subjects. In the practical subjects had Hedges’ g = 0.95, the practical and theoretical integrated subjects had Hedges’ g = 0.36, and the theoretical subjects had Hedges’ g = 0.30. According to the subject type, there was a significant difference in the flipped learning effect (Q = 13.59, p = < 0.001). In the prior learning method type, as the effect size value is calculated by classifying the video method and the method where the video and lecture book were given simultaneously, the Hedges’ g was 0.71 for the video method and 0.45 for the video and lecture book; however, there was no statistically significant difference (Q = 1.22, p = 0.269).
3.5. Publication Bias
The publication bias of this study was analyzed by Egger’s regression test. As a result of statistically checking the significance of the degree of asymmetry, it was found that there was no publication bias (t = 0.82, p = 0.42) (Figure 3).
4. Discussion
This study systematically examined and meta-analyzed the effects of flipped learning on clinical competence, critical thinking ability, self-directedness, and learning satisfaction applied to nursing education to present the effects in an integrated and objective manner. Therefore, the main findings are as follows:
The selected 18 studies considered flipped learning as the intervention group and traditional lecture-style learning as the control group. Of these, the two (11.1%) randomized controlled pre- and post-design studies using an RCT design were conducted overseas. Sixteen pieces (88.9%) were pre-post designs for the non-equivalent control group using the NRCT design, where six were conducted overseas and ten were conducted domestically.
Considering the items with high RoB or uncertain RoB in the methodological quality evaluation of theses, seven papers (38%) in selection bias were not randomly assigned. Part 2 (12.5%) did not present a specific method for generating the randomization order, and in Part 3 (16.6%), the order of allocation was not concealed. Thus, the researcher could predict the intervention group. Additionally, in terms of performance bias, no specific method for blinding research participants and researchers was presented in 14 papers (77.7%). In flipped learning, as instructor intervention is required, there are limitations to blindfolding participants and researchers, influencing the meta-analysis results. Despite the limitations in RCT research methods in flipped learning, various approaches to verify the quality of learning methods must be sought.
Regarding the subject types that applied flipped learning, practical subjects were the highest, with nine studies (50%), followed by seven theoretical subjects (38.9%) and two theory and practical subjects (11.1%). The theoretical subjects were critical thinking, nursing process, pathophysiology, health assessment, and patient safety, and the practical subjects were simulation practice, basic nursing practice, and clinical nursing practice.
Among the analysis papers, when applying flipped learning in practical subjects, nursing techniques cannot be directly performed. Thus, the methods and procedures of nursing techniques are repeatedly learned through audio-visual data [11]. Video content was used [19] or developed and applied by the instructor [31]. As the learning experience provided by the medium is different, the instructors set a clear goal through flipped learning, and the curiosity and motivation of learners were met.
As a result of the meta-analysis of flipped learning studies applied to 18 studies, the overall average effect size was 0.68, corresponding to the median effect size. This is similar to the study showing the median effect size of 0.58 as the overall average effect size of the learning effects of domestic flipped learning [40] and the study showing a median effect size of 0.59 as the effect size of flipped learning on domestic college students [41]. It was confirmed that flipped learning was more effective than traditional lecture-style classes when applied to nursing education.
As a result of comparing the effect sizes of the flipped learning outcome variables in this study, they were found in this order of critical thinking ability, learning satisfaction, clinical competence, and self-direction.
The critical thinking ability had a Hedges’ g of 0.87, showing a large effect size and a high level of heterogeneity. In five studies, the experimental group to which flipped learning was applied showed improved critical thinking ability compared to the control group. It can be predicted that flipped learning, which allows students to ask questions and solve problems on their own through prior learning and to improve critical thinking skills through discussion with other students during class in the classroom [42], is effective in nursing education. Additionally, a previous study [19] showed that although it is possible to acquire knowledge and skills, there is a limit to critical thinking. Instructors must grasp the learner’s disposition, attitude, and learning strategy and find and search for information rather than memorize knowledge; based on this, it is necessary to develop a running program.
As a result of analyzing the effect of flipped learning on learning satisfaction, the intermediate effect size was found to be Hedges’ g = 0.79. When learning flipped learning, information was provided to students, such as videos, animations, and images of clinical cases, inducing interest and resulting in positive responses. Satisfaction was high, as this allowed flexibility to learn at a desired speed and time [43,44]. However, there have also been studies with low learning satisfaction due to tasks assigned from prior learning during flipped learning, online environmental problems for learning, and adaptation to new teaching methods [8]. The instructor should consider the learner’s learning environment when developing flipped learning activities. It is necessary to consider avoiding a learning gap, such as understanding the infrastructure for watching videos and prior learning, and it is necessary to analyze the learning conditions in detail per the learning goal. In other words, it is necessary to promote understanding of learning by providing students with a plan for the learning content and learning method of flipped learning in advance so that they can familiarize themselves with the content to be learned. Additionally, the learner’s learning process was constantly monitored. A strategy for linking the content of prior learning and offline activities is needed by providing feedback on prior learning in the classroom.
As a result of analyzing whether flipped learning is more effective than traditional methods in improving nursing students’ clinical nursing skills, the effect size of clinical competence had a Hedges’ g of 0.53, showing a medium-sized effect. This confirmed that learning the procedure and techniques through videos was effective [45], and it was also confirmed by another study [46] that the clinical performance was improved by flipped learning in clinical practice. Nursing is a practical study. Direct nursing skills and repetitive practice are required to acquire and improve such skills. However, it is difficult to obtain qualitative improvement because of the inability to perform direct skills during flipped learning. When flipped learning is applied in practical classes, nursing skills and nursing theories are learned based on evidence, and in classroom sessions, the contents of prior learning can be directly performed. Additionally, it effectively improves students’ nursing skills through immediate feedback from instructors [47]. Factors for achieving clinical performance include interpersonal relationships and communication, nursing process, critical thinking, nursing intervention, creative thinking, basic nursing, and problem-solving ability [48]. These are improved by applying flipped learning to the subject in the corresponding domain, helping them to effectively perform the clinical nursing skills corresponding to the detailed elements.
In this study, the effect size of self-directedness was small (Hedges’ g = 0.37). The most important aspect of flipped learning is to increase the degree of self-directedness, and in many research results, self-directedness was improved through flipped learning [49,50]. Learners watched video lectures on pre-learning to familiarize themselves with the learning content, and the activities in the classroom were focused on learners. Notably, the self-directedness is improved due to the increased sense of responsibility and activeness in all learning processes [51]. Self-directedness is an important variable affecting learning outcomes, such as learning motivation, academic achievement, and satisfaction [52]. Through flipped learning, the instructors foster the learner’s learning ability and act as facilitators such that learners can have flexibility and develop lifelong learning habits for self-directedness [53]. Accordingly, when the learner’s characteristics and teaching methods are matched, self-directedness is achieved successfully. When designing a flipped learning curriculum, it is necessary to develop educational methods to improve self-directedness ability by focusing on learner-centered education.
Other variables related to the subject type and pre-learning method were the modulating variables that explained the difference in effect size. As a result of meta-ANOVA using the subject type as the moderator variable, the following were practical subjects, theory and practice mixed design subjects, and theory subjects.These results showed a significant difference in the flipped learning effect according to the subject type (Q = 13.59, p ≤ 0.001), and it was confirmed that the effect of a flipped learning class was higher in practical subjects. In a practical subject, flipped learning helps one experience knowledge integrated with observation and experience [12]. The interaction between the instructor and students increases, and the nursing skills gradually improve [8]. Studies have confirmed this effect. Additionally, the effect size value was calculated by classifying the learning methods into video method (15 episodes) and the method where video and lecture books were given (five episodes) simultaneously, indicating that the learning by the video method was higher but not statistically significant (Q = 1.22, p = 0.269). This could be because when the video and lecture book were given simultaneously, the number of assignments to be learned increased, consequently increasing the learning burden, and the lack of time and grade were not reflected. It is necessary for the instructor to explain in advance how the pre-learning videos are related to the lecture contents in order for the students to plan an appropriate amount of pre-study.
It was difficult to measure various variables because there were differences in the specific research methods such as the flip learning education method applied to each study, the number of applications, and the time. In future research, it is necessary to improve the quality of research by describing specific research methods for teaching and learning designs applied to flipped learning classes, such as the characteristics of instructors and learners and the learning process.
Critical thinking, clinical reasoning, and clinical performance are required for nursing students to adapt to the rapidly changing nursing environment [14]. Applying flipped learning to equip self-directedness ability, students learn how to acquire abilities and competencies by applying various information rather than simply accumulating knowledge and facing problems in a medical nursing environment.
The limitations of this study are as follows. An extensive search on flipped learning was performed using flipped learning keywords and a series of other relevant keywords. However, the possibility of publication bias cannot be excluded because the publication languages are limited to English and Korean.
This study selected RCT and NRCT research among the studies related to flipped learning applied to nursing education to increase the reliability of the research results. The academic significance included confirming the magnitude of the effect on clinical Competence, critical thinking ability, self-directedness, and learning satisfaction as outcome variables, an effect that was determined to exist.
5. Conclusions
Flipped learning applied to nursing education is a learning method that effectively improves nursing students’ critical thinking ability, learning satisfaction, clinical competence, and self-directedness. In particular, it helps increase critical thinking ability and clinical competence necessary in nursing education, which occupies a large part of clinical practice education. Several studies have confirmed that flipped learning is being applied to practical education. However, the research results on students’ prior learning methods, the time required, and educator preparation are insufficient. Therefore, detailed methods for future learning types should be developed, and research to prove their effectiveness should be conducted.
Therefore, it is recommended that flipped learning education be actively used as an effective nursing education teaching method. It is expected that continuous research on the effects and learning effects on students will be conducted.
Conceptualization: I.P. and Y.S.; methodology, I.P. and Y.S.; software, I.P. and Y.S.; validation, I.P. and Y.S.; formal analysis, I.P.; investigation, I.P. and Y.S.; resources, I.P. and Y.S.; writing—original draft preparation, I.P.; writing—review and editing, I.P. and Y.S.; visualization, I.P. and Y.S.; supervision Y.S.; project administration, I.P. and Y.S. All authors have read and agreed to the published version of the manuscript.
This research received no external funding.
The study was approved by the Soonchunhyang University Institutional Review Board on Human Subjects Research and Ethics Committees (IRB-202102-SB-015).
Not applicable.
Data from the study are available upon request.
The authors declare no competing interest.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Enlarge this image.Figure 1. Flowchart of study selection.
Enlarge this image.Figure 2. Forest plot of effect size of flip learning for nursing students.
Enlarge this image.Figure 3. Publication bias.
Descriptive summary of included studies.
| No | Author (Year) | Publication | Control Group Program | Participants | ST | Subject | Outcomes | Moderator Variables | Methodological Quality Assessment | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Sample Size | Subject Type | Pre-Class Activities | SD | RCT | - | R | AC | BP | BO | - | IO | SR | ||||||||||
| Exp. | Con. | Theory | Practice | Video | Document | NRCT | CT | SP | CV | MI | BO | ER | IO | SR | ||||||||
| 1 | Cha et al. (2020) [ |
Journal | TAU | 41 | 41 | 6 | Critical Thinking and Nursing Process | Critical thinking skills | ○ | + | + | NRCT | ① | ① | ⓗ | ① | ⓤ | ① | ⓗ | ① | ||
| 2 | Fan et al. (2020) [ |
Journal | TAU | 287 | 198 | - | Theory practice | Self-directedness Learning satisfaction | ○ | ○ | + | + | NRCT | ① | ① | ⓗ | ① | ⓤ | ① | ⓗ | ① | |
| 3 | Gu et al. (2020) [ |
Journal | TAU | 34 | 33 | - | Simulation practice | Clinical competence | ○ | + | NRCT | ① | ⓗ | ① | ① | ① | ① | ① | ① | |||
| 4 | Kim et al. (2020) [ |
Journal | TAU | 135 | 103 | 5 | Critical Thinking and Nursing Process | Self-directedness | ○ | + | NRCT | ① | ⓗ | ① | ① | ① | ① | ① | ① | |||
| 5 | Kim et al. (2020) [ |
Journal | TAU | 38 | 36 | 4 | Basic Nursing Practice | Clinical competence |
○ | + | NRCT | ① | ① | ① | ① | ⓤ | ① | ① | ① | |||
| 6 | Kim OS. (2020) [ |
Journal | TAU | 22 | 26 | 10 | Basic Nursing Practice | Self-directedness | ○ | + | NRCT | ① | ⓗ | ① | ① | ⓤ | ① | ⓗ | ① | |||
| 7 | Jo YS. (2020) [ |
Thesis | TAU | 27 | 26 | 4 | Emergency Nursing | Clinical competence | ○ | + | + | NRCT | ① | ① | ① | ① | ⓤ | ① | ① | ① | ||
| 8 | Cho et al. (2019) [ |
Journal | TAU | 59 | 21 | - | Clinical adult nursing practicum | Learning satisfaction | ○ | + | + | NRCT | ① | ① | ① | ① | ⓤ | ① | ⓗ | ① | ||
| 9 | Jung et al. (2019) [ |
Journal | TAU | 38 | 36 | - | Basic Nursing Practice | Critical thinking skills | ○ | + | NRCT | ① | ① | ① | ① | ⓤ | ① | ① | ① | |||
| 10 | Kim et al. (2019) [ |
Journal | TAU | 32 | 43 | 14 | Patient safety course | Clinical competence | ○ | + | NRCT | ① | ⓗ | ① | ① | ① | ① | ① | ① | |||
| 11 | Im et al. (2019) [ |
Journal | TAU | 34 | 36 | 6 | Psychiatric Nursing Practice | Learning satisfaction | ○ | + | NRCT | ① | ⓤ | ① | ① | ⓤ | ① | ① | ① | |||
| 12 | Lee MH. (2018) [ |
Thesis | TAU | 42 | 39 | 5 | Pathophysiology | Critical thinking skills |
○ | + | NRCT | ① | ① | ① | ① | ⓤ | ① | ① | ① | |||
| 13 | Kim et al. (2017) [ |
Journal | TAU | 106 | 92 | 8 | Simulated learning | Clinical competence |
○ | + | RCT | ⓗ | ⓤ | ⓤ | ① | ⓗ | ① | |||||
| 14 | Kim et al. (2017) [ |
Journal | TAU | 35 | 34 | - | Clinical practice | Learning satisfaction | ○ | + | RCT | ① | ① | ⓤ | ① | ⓗ | ① | |||||
| 15 | Lee et al. (2017) [ |
Journal | TAU | 70 | 76 | 6 | Psychiatric Nursing Practice | Clinical competence |
○ | + | NRCT | ① | ⓗ | ① | ① | ① | ① | ① | ① | |||
| 16 | Park et al. (2017) [ |
Journal | TAU | 48 | 54 | 9 | Basic nursing course | Self-directedness | ○ | + | NRCT | ① | ⓗ | ① | ① | ⓤ | ① | ① | ① | |||
| 17 | Lee et al. (2016) [ |
Journal | TAU | 70 | 70 | 6 | Health assessment | Critical thinking skills | ○ | ○ | + | NRCT | ① | ① | ① | ① | ⓤ | ① | ① | ① | ||
| 18 | Maxwell et al. (2016) [ |
Journal | TAU | 31 | 33 | - | Patient safety | Clinical competence | ○ | + | + | NRCT | ① | ⓤ | ⓗ | ① | ⓤ | ① | ⓤ | ① | ||
TAU = Treatment as Usual; Exp. = Experimental group; Con. = Control group; ST = Session Total; SD = Study design; RCT = Randomized Controlled Trial; NRCT = Randomized Controlled Clinical Trial; Video; R = Random sequence generation; AC = allocation concealment; BP = Blinding of participants and personnel; BO = Blinding of outcome assessment; IO = Incomplete outcome date; SR = Selective reporting; CT = Comparability of target group; SP = Selection of participant; CV = Confounding variables; MI = Measurement of intervention (exposure); ER = Evaluation of results; ① = low risk of bias; ⓗ = high risk of bias; ⓤ = uncertain risk.
Effects of outcome and moderator variables.
| Categories | k | Hedges’ g | 95% CI | Heterogeneity | |||||
|---|---|---|---|---|---|---|---|---|---|
| Lower | Upper | Q | p | I2% | |||||
| Total effect size | 24 | 0.68 | 0.43 | 0.92 | 246.67 | <0.001 | 90.7 | ||
| Outcome |
Clinical competence | 8 | 0.53 | 0.12 | 0.94 | 50.86 | <0.001 | 86 | |
| Critical thinking skills | 5 | 0.87 | 0.37 | 1.37 | 26.04 | <0.001 | 85 | ||
| Learning satisfaction | 6 | 0.79 | 0.36 | 1.22 | 45.53 | <0.001 | 89 | ||
| Self-directedness | 5 | 0.37 | 0.00 | 0.73 | 24.48 | <0.001 | 84 | ||
| Moderator variables | ST | Practice | 12 | 0.95 | 0.71 | 1.20 | 13.59 | <0.001 | |
| Theory | 9 | 0.30 | 0.01 | 0.58 | |||||
| Theory and Practice | 3 | 0.36 | 0.08 | 0.80 | |||||
| PCA | Video | 17 | 0.71 | 0.46 | 0.95 | 1.22 | 0.269 | ||
| Video and document | 7 | 0.45 | 0.08 | 0.83 | |||||
k = number of effect size; 95% CI = 95% confidence interval; Q = total variability; I2 = between-study variability; ST = Subject type; PCA = Pre-class activities.
References
1. Galway, L.P.; Corbett, K.K.; Takaro, T.K.; Tairyan, K.; Frank, E. A novel integration of online and flipped classroom instructional models in public health higher education. BMC Med. Educ.; 2014; 4, 181. [DOI: https://dx.doi.org/10.1186/1472-6920-14-181]
2. Gilboy, M.B.; Heinerichs, S.; Pazzaglia, G. Enhancing student engagement using the flipped classroom. J. Nutr. Educ. Behav.; 2015; 47, pp. 109-114. [DOI: https://dx.doi.org/10.1016/j.jneb.2014.08.008]
3. Bergmann, J.; Sams, A. Flipped Learning: Gateway to Student Engagement; International Society for Technology in Education: Arlington, VA, USA, 2014.
4. Moos, D.C.; Bonde, C. Flipping the classroom: Embedding self-regulated learning prompts in videos. Technol. Knowl. Learn.; 2016; 21, pp. 225-242. [DOI: https://dx.doi.org/10.1007/s10758-015-9269-1]
5. Lai, C.L.; Hwang, G.J. A self-regulated flipped classroom approach to improving students’ learning performance in a mathematics course. Comput. Educ.; 2016; 100, pp. 126-140. [DOI: https://dx.doi.org/10.1016/j.compedu.2016.05.006]
6. Sandi, F.T.; Mombourquette, P. Evaluation of a flipped classroom in an undergraduate business course. Bus. Educ. Accredit.; 2014; 6, pp. 63-71.
7. Chen, F.; Lui, A.M.; Martinelli, S.M. A systematic review of the effectiveness of flipped classrooms in medical education. Med Educ.; 2017; 51, pp. 585-597. [DOI: https://dx.doi.org/10.1111/medu.13272]
8. Missildine, K.; Fountain, R.; Summers, L.; Gosselin, K. Flipping the classroom to improve student performance and satisfaction. J. Nurs. Educ.; 2013; 52, pp. 597-599. [DOI: https://dx.doi.org/10.3928/01484834-20130919-03]
9. Hur, H.; Roh, Y. Effects of a simulation based clinical reasoning practice program on clinical competence in nursing students. J. Korean Acad. Soc. Adult Nurs.; 2013; 25, pp. 574-584. [DOI: https://dx.doi.org/10.7475/kjan.2013.25.5.574]
10. Dehghanzadeh, S.; Jafaraghaee, F. Comparing the effects of traditional lecture and flipped classroom on nursing students’ critical thinking disposition: A quasi-experimental study. Nurse Educ. Today; 2018; 71, pp. 151-156. [DOI: https://dx.doi.org/10.1016/j.nedt.2018.09.027]
11. Cho, M.K.; Kim, M.Y. Outcomes and influential factors applying flipped learning methods in a clinical adult nursing practicum. Int. J. Nurs. Pract.; 2019; 25, e12724. [DOI: https://dx.doi.org/10.1111/ijn.12724]
12. Hamer, L.O. The additive effects of semistructured classroom activities on student learning: An application of classroom-based experiential learning techniques. J. Mark. Educ.; 2000; 22, pp. 25-34. [DOI: https://dx.doi.org/10.1177/0273475300221004]
13. Sait, M.; Siddiqui, Z.; Ashraf, Y. Advances in medical education and practice: Student perceptions of the flipped classroom. Adv. Med Educ. Pract.; 2017; 8, pp. 317-320. [DOI: https://dx.doi.org/10.2147/AMEP.S133328]
14. Handwerker, S.M. Transforming nursing education: A review of current curricular practices in relation to Benner’s latest work. Int. J. Nurs. Educ. Scholarsh.; 2012; 9, pp. 1-19. [DOI: https://dx.doi.org/10.1515/1548-923X.2510]
15. Lee, H.S.; Kang, S.C.; Kim, C.S. A study on the effect of flipped learning on learning motivation and academic achievement. J. Korean Assoc. Comput. Educ.; 2015; 18, pp. 47-57.
16. Kim, H.S.; Kim, M.Y.; Cho, M.K.; Jang, S.J. Effectiveness of applying flipped learning to clinical nursing practicums for nursing students in Korea: A randomized controlled trial. Int. J. Nurs. Pract.; 2017; 23, e12574. [DOI: https://dx.doi.org/10.1111/ijn.12574]
17. Maxwell, K.L.; Wrigh, V.H. Evaluating the effectiveness of two teaching strategies to improve nursing students’ knowledge, skills, and attitudes about quality improvement and patient safety. Nurs. Educ. Perspect.; 2016; 37, pp. 291-292. [DOI: https://dx.doi.org/10.1097/01.NEP.0000000000000043]
18. Cha, J.A.; Kim, J.H. Effects of Flipped learning on the critical thinking disposition, academic achievement and academic self-efficacy of nursing students: A mixed methods study. J. Korean Acad. Soc. Nurs. Educ.; 2020; 26, pp. 25-35. [DOI: https://dx.doi.org/10.5977/jkasne.2020.26.1.25]
19. Kim, A.K.; Yi, S.J. The convergence effect of fundamental nursing practice education using flipped learning on self confidence in performance, academic achievement and critical thinking. J. Digit. Converg.; 2020; 18, pp. 389-399. [DOI: https://dx.doi.org/10.14400/JDC.2020.18.6.389]
20. Hu, R.; Gao, H.; Ye, Y.; Ni, Z.; Jiang, N.; Jiang, X. Effectiveness of flipped classrooms in Chinese baccalaureate nursing education: A meta-analysis of randomized controlled trials. Int. J. Nurs. Stud.; 2018; 79, pp. 94-103. [DOI: https://dx.doi.org/10.1016/j.ijnurstu.2017.11.012]
21. Goh, Y.; Owyong, J.; Seetoh, Y.; Hu, Y.; Chng, M.; Li, Z. Exploring pedagogies used in undergraduate mental health nursing curriculum: An Integrative literature review. Int. J. Ment. Health Nurs.; 2021; 30, pp. 47-61. [DOI: https://dx.doi.org/10.1111/inm.12816]
22. Im, S.; Jang, S. Effects of a clinical practicum using flipped learning among undergraduate nursing students. J. Nurs. Educ.; 2019; 58, pp. 354-356. [DOI: https://dx.doi.org/10.3928/01484834-20190521-06]
23. Higgins, J.P.T.; Altman, D.G.; Gøtzsche, P.C.; Jüni, P.; Moher, D.; Oxman, A.D.; Savovic, J.; Schulz, K.F.; Weeks, L.; Sterne, J.A.C. et al. The cochrane collaboration’s tool for assessing risk of bias in randomised trials. Br. Med J.; 2011; 343, pp. 889-893. [DOI: https://dx.doi.org/10.1136/bmj.d5928]
24. Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G. The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses. The PRISMA statement. PLoS Med.; 2009; 6, e1000097. [DOI: https://dx.doi.org/10.1371/journal.pmed.1000097]
25. Borenstein, M.; Hedges, L.V.; Higgins, J.P.T.; Rothstein, H.R. A basic introduction to fixed effect and random-effects models for meta-analysis. Res. Synth. Methods; 2010; 1, pp. 97-111. [DOI: https://dx.doi.org/10.1002/jrsm.12]
26. Cohen, J. Statistical Power Analysis for the Behavioral Sciences; revised edition, 1997 Academic Press: New York, NY, USA, 1977.
27. Egger, M.; Davey, S.G.; Schneider, M.; Minder, C. Bias in meta-analysis detected by a simple, graphical test. Br. Med J.; 1997; 315, pp. 629-634. [DOI: https://dx.doi.org/10.1136/bmj.315.7109.629]
28. Fan, J.; Tseng, Y.; Chao, L.; Chen, S.; Jane, S. Learning outcomes of a flipped classroom teaching approach in an adult-health nursing course: A quasi-experimental study. BMC Med. Educ.; 2020; 20, 317. [DOI: https://dx.doi.org/10.1186/s12909-020-02240-z]
29. Gu, M.; Sok, S. Effects of simulation practicum using flipped learning for Korean nursing students. Int. J. Environ. Res. Public Health; 2020; 17, 6829. [DOI: https://dx.doi.org/10.3390/ijerph17186829]
30. Kim, S.; Gang, M. The effect of flipped learning based nursing process education on self-directed learning ability and academic achievement of nursing students. J. Korea Entertain. Ind. Assoc.; 2020; 14, pp. 363-372. [DOI: https://dx.doi.org/10.21184/jkeia.2020.10.14.7.363]
31. Kim, O. Effect of flipped learning using media convergence in practice education on academic self-efficacy and self-directed learning of nursing students. J. Converg. Inf. Technol.; 2020; 10, pp. 49-58. [DOI: https://dx.doi.org/10.22156/CS4SMB.2020.10.06.049]
32. Jo, Y. Effects of Neonatal Emergency Nursing Flipped Learning on Neonatal Emergency Nursing Knowledge, Performer’s Confidence and Self-Leadership of Nursing College Students Immediately after Birth. Unpublished. Master’s Thesis; CHA University: Pocheon-si, Korea, 2020.
33. Jung, H.; Yang, Y. Effects of flipped learning-based fundamentals of nursing practice. J. Learn. Cent. Curric. Instr.; 2019; 19, pp. 555-572. [DOI: https://dx.doi.org/10.22251/jlcci.2019.19.23.555]
34. Kim, Y.; Yoon, Y.; Hong, H.; Min, A. Effects of a patient safety course using a flipped classroom approach among undergraduate nursing students: A quasi-experimental study. Nurse Educ. Today; 2019; 79, pp. 180-187. [DOI: https://dx.doi.org/10.1016/j.nedt.2019.05.033]
35. Lee, M. Effect of Flip-Type Clinical Case Based Learning (FCCBL) in Pathophysiology. Unpublished doctoral dissertation; Konkuk University: Seoul, Korea, 2018.
36. Kim, H.; Jang, Y. Flipped learning with simulation in undergraduate nursing education. J. Nurs. Educ.; 2017; 56, pp. 329-336. [DOI: https://dx.doi.org/10.3928/01484834-20170518-03]
37. Lee, M.; Chang, S.; Jang, S. Effects of the flipped classroom approach on the psychiatric nursing practicum course. J. Korean Acad. Psychiatr. Ment. Health Nurs.; 2017; 26, pp. 196-203. [DOI: https://dx.doi.org/10.12934/jkpmhn.2017.26.2.196]
38. Park, J.; Woo, C. The effects of lesson with student-generated questions: Based on flipped utilizing massive open online courses. J. Learn. Cent. Curric. Instr.; 2017; 17, pp. 283-306. [DOI: https://dx.doi.org/10.22251/jlcci.2017.17.11.283]
39. Lee, Y.; Eun, Y. The effect of the flipped learning on self-efficacy, critical thinking disposition, and communication competence of nursing students. J. Korean Acad. Soc. Nurs. Educ.; 2016; 22, pp. 567-576. [DOI: https://dx.doi.org/10.5977/jkasne.2016.22.4.567]
40. Cho, B.; Lee, J. A meta analysis on effects of flipped learning in Korea. J. Digit. Converg.; 2018; 16, pp. 59-73. [DOI: https://dx.doi.org/10.14400/JDC.2018.16.3.059]
41. Lee, H.J.; Ahn, M.S.; Lee, J.G. A meta-analysis of the effects of flipped learning and team-based learning in college classe. J. Learn. Cent. Curric. Instr.; 2018; 18, pp. 1253-1280. [DOI: https://dx.doi.org/10.22251/jlcci.2018.18.21.1253]
42. Jung, M.S.; Kwon, H.J. A structural equation model on core competencies of nursing students. J. Korean Acad. Soc. Nurs. Educ.; 2015; 21, pp. 256-265. [DOI: https://dx.doi.org/10.5977/jkasne.2015.21.2.256]
43. Yui, M.; Azaiza, K.; Salani, D. Flipping the classroom without flipping out the students. Distance Learn. Issue; 2017; 14, pp. 31-42.
44. Critz, C.M.; Knight, D. Using the flipped classroom in graduate nursing education. Nurse Educ.; 2013; 38, pp. 210-213. [DOI: https://dx.doi.org/10.1097/NNE.0b013e3182a0e56a] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23969751]
45. Bloomfield, J.G.; Jones, A. Using e-learning to support clinical skills acquisition: Exploring the experiences and perceptions of graduate first-year pre-registration nursing students—A mixed method study. Nurse Educ. Today; 2013; 33, pp. 1605-1611. [DOI: https://dx.doi.org/10.1016/j.nedt.2013.01.024] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23473860]
46. Pierce, R.; Jeremy, F. Vodcasts and active-learning exercises in a “flipped classroom” model of a renal pharmacotherapy module. Am. J. Pharm. Educ.; 2012; 76, pp. 1-5. [DOI: https://dx.doi.org/10.5688/ajpe7610196]
47. Sinclair, B.; Karen, F. Integrating simulated teaching/learning strategies in undergraduate nursing education. Int. J. Nurs. Educ. Scholarsh.; 2009; 6, pp. 1-11. [DOI: https://dx.doi.org/10.2202/1548-923X.1676]
48. Shim, C.S. Study on the effects of flip learning-based simulation education on the learning flow, learning confidence, communication skills, and clinical competence of nursing students. J. Korea Entertain. Ind. Assoc.; 2019; 13, pp. 541-549. [DOI: https://dx.doi.org/10.21184/jkeia.2019.12.13.8.541]
49. Jung, J.Y. The effect of the flipped learning on self-directed and self-regulation competence of pre-teacher student. J. Learn. Cent. Curric. Instr.; 2017; 17, pp. 215-235. [DOI: https://dx.doi.org/10.22251/jlcci.2017.17.4.215]
50. Choi, Y.S.; Kim, J.Y. Effects of flipped learning in core competencies of nursing students: Based on communication, problem solving, and self-directed learning. Korean Assoc. Learn. Cent. Curric. Instr.; 2018; 18, pp. 1163-1182. [DOI: https://dx.doi.org/10.22251/jlcci.2018.18.19.1163]
51. Lee, D.Y.; Lee, J.Y. The effects of flipped learning classes in nursing students. Med. Leg. Update; 2019; 19, pp. 508-514. [DOI: https://dx.doi.org/10.5958/0974-1283.2019.00094.X]
52. Park, J.Y. The effect of self-directed learning strategies on e-learning pre-learning of nursing students: Focusing on the flow experience. J. Korean Acad. Fundam. Nurs.; 2020; 27, pp. 52-63. [DOI: https://dx.doi.org/10.7739/jkafn.2020.27.1.52]
53. Han, J.; Lee, M.; Jung, B. A study on design education method for development of self-directed learning ability. J. Eng. Educ. Res.; 2009; 12, pp. 115-125.
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
1 Department of Nursing, Shinsung University, 1, Daehak-ro, Jeongmi-myeon, Dangjin-si 31801, Chungcheongnam-do, Korea
2 School of Nursing, Soonchunhyang University, 31 Soonchunhyang 6th Rd, Dongnam-gu, Cheonan-si 31151, Chungcheongnam-do, Korea;