Critical thinking and clinical skills by problem-based learning educational methods: an umbrella systematic review

Abstract

Introduction

Problem-based learning (PBL), based on self-directed learning, is an instructional strategy that shifts the approach from the instructor to the student (student-centered). Notwithstanding its adoption in undergraduate and graduate medical education, its usefulness remains a topic of debate. This study aims to review current worldwide data on the efficacy and utility of critical thinking and clinical skills using PBL educational methods for teaching programs to undergraduate medical students, ursing students, dental students and medical residents.

Methods

The search was conducted in June 2024 in PubMed, Cochrane, Embase, and Web of Science and included all publications regardless of the publication date, study design, or country of origin.

Results

Thirteen meta-analyses and systematic reviews were compiled through database screening. The results showed that PBL effectively enhanced students’ clinical skills and critical thinking abilities; nevertheless, further evidence is needed to confirm this finding.

Conclusion

PBL may be a successful and an effective instructional strategy, not only medical, but also nursing and dental education. Students can learn skills and acquire competencies useful for their medical professionalism through PBL in addition to their existing knowledge.

Introduction

Problem-based learning (PBL), based on self-directed learning, is an instructional strategy that shifts the approach from the instructor to the student (student-centered). Notwithstanding its adoption in undergraduate and graduate medical education, its usefulness remains a topic of debate. This study aims to review current worldwide data on the efficacy and utility of critical thinking and clinical skills using PBL educational methods for teaching programs to undergraduate medical students, ursing students, dental students and medical residents.

Methods

The search was conducted in June 2024 in PubMed, Cochrane, Embase, and Web of Science and included all publications regardless of the publication date, study design, or country of origin.

Results

Thirteen meta-analyses and systematic reviews were compiled through database screening. The results showed that PBL effectively enhanced students’ clinical skills and critical thinking abilities; nevertheless, further evidence is needed to confirm this finding.

Conclusion

PBL may be a successful and an effective instructional strategy, not only medical, but also nursing and dental education. Students can learn skills and acquire competencies useful for their medical professionalism through PBL in addition to their existing knowledge.

Introduction

Real-world situations are typically used as the backdrop for students to learn new information and hone their existing abilities in problem-based learning (PBL) [1]. This methodology was established through educational innovation address concerns about traditional medical education techniques in the late 1960s [2]. Since then, it has been incorporated into several professional training programs and curricula in medical schools worldwide.

With the rapid development of medical education, therefore, cultivating excellent medical professionals is particularly crucial. In clinical education, traditional lecture-based teaching has shortcomings; for example, teachers place too much emphasis on knowledge and passive student learning [3, 4], resulting in low learning efficiency, insufficient clinical thinking ability, and poor clinical practice ability for students [5].In contrast to traditional teaching models, PBL is a student-centered approach. This occurred in “student-centred” environments, characterized by small groups tutorial sessions, with the teachers acting as learning facilitators, to promote, besides meaningful learning, self-learning and effective communication both professionally and in society. Numerous studies have revealed that students using PBL outperform standard curriculum students in problem-solving and self-directed learning [6, 7]. Nonetheless, existing data suggest that the PBL teaching paradigm may not be as effective as the traditional approach in terms of imparting factual information [1, 8]. The PBL model may have limitations in the early stages of teaching, as it requires students to have a certain level of foundational knowledge and concepts. While PBL is suitable for comprehensive courses and can foster students’ ability to solve unstructured problems, it may not provide sufficient instruction in basic knowledge. In fields such as biostatistics, which require a rigorous theoretical foundation, PBL may have limitations as it focuses more on problem-solving and practical application rather than in-depth theoretical instruction [9].Additionally, PBL outcomes may vary depending on the topic.

Numerous studies have shown that PBL significantly enhances critical thinking skills among medical students [10]. Clinical skills refer to the ability to address patient problems. They encompass the fundamental skills required for clinical practice, as well as the specialized skills necessary for specific clinical tasks. These include diagnostic skills, therapeutic skills, nursing skills, procedural skills, emergency skills, monitoring skills, communication skills, and research skills [11]. Ennis defines critical thinking as a reflective practice with the goal of forming beliefs or making wise actions. He emphasizes five key terms in critical thinking: practice, reflection, wisdom, belief, and action [12]. Critical thinking involves purposeful, self-regulatory judgment that results in interpretation, analysis, evaluation, and inference, skills that are invaluable in the clinical setting [13]. PBL, as a teaching strategy, focuses on engaging students in group discussions to address complex and ambiguous clinical problems. This process involves analyzing problems, setting objectives, gathering information, summarizing ideas, and reflecting on problem-solving experiences [14]. Research has demonstrated that this teaching strategy effectively develops critical thinking abilities in medical students [15].

However, the efficacy of PBL as a teaching method for enhancing critical thinking in students remains a topic of debate among researchers. Previous systematic reviews evaluating critical thinking in PBL have produced mixed results, with some studies supporting a positive relationship between PBL and critical thinking and others disputing it [16, 17]. The current umbrella systematic review aimed to assess the overall effectiveness of PBL in improving students’ critical thinking and clinical skills in comparison to standard education methods.

Method

Search strategy and eligibility criteria

Conducting an umbrella review requires setting clear objectives and inclusion criteria, using an open and thorough search strategy, assessing the techniques used, and providing a concise summary of the available data. In this comprehensive review, (1) we included only Meta analysis and Systematic review that studied PBL; (2) we did not limit language; and (3) We will include clinical students and clinicians from various specialties (such as pharmacists, nurses, and dentists) and exclude those who are not clinically relevant.

We searched through several databases to perform a thorough analysis, including the PubMed, Cochrane, Embase, and Web of Science databases for articles published until February 15, 2025. We conducted an umbrella review that examined the association between critical thinking and clinical skills using PBL educational methods with detailed results. This umbrella review was registered in the International Prospective Register of Systematic Reviews (ID: 550,031).

Data extraction

Two authors independently performed the search and screening procedures on their own (ZXY and LJB). They began the screening in accordance with the eligibility requirements and research goals after eliminating duplicate studies. The analyses were evaluated using the random-effects summary, level of comparison, small-study effects, and I2 statistics. They also gathered information on study type, author(s), year, number of participants, outcome-measuring instruments, and I2 statistics. A third senior author (GWL) was involved in this process to address any differences in opinion.

Assessment of methodological quality

Using the A Measurement Tool to Assess Systematic Reviews (AMSTAR-2) standards, which systematically evaluate evidence-based medical studies [18, 19], we further examined the quality of this umbrella review. No overall score was assigned because a high score can ignore some important constraints [20]. When evaluating the quality of the systematic reviews and meta-analyses that included both observational and interventional research, the AMSTAR-2 method is considered to be a dependable and accurate method [21]. AMSTAR-2 evaluates the process of choosing study designs for inclusion, rationale behind eliminating studies, sources of primary study financing, and any conflicts of interest among the reviewers, in contrast to routinely used methods such as the risk of bias in non-randomized intervention studies [22]. Due to this, AMSTAR-2 is frequently used for umbrella evaluations [23].

Assessment of epidemiological credibility

We reanalyzed all the included meta-analyses. High epidemiological credibility indicates the highest level of evidence, with no indication of major heterogeneity or bias. The included studies were further classified into the following categories [24]

1. 1.

‘Persuasive’: statistically significant per the random-effects model with a p-value of < 0.000001, > 1000 cases, low heterogeneity among the selected studies (I2 < 50%), 95% CI (excluding the null value), and no evidence of small-study effects or significant bias.

2. 2.

‘Highly recommended’: statistically significant with a p­-value of < 0.000001, > 1000 cases, with most studies indicating a significant effect.

3. 3.

‘Recommended’: > 1000 cases and significant effects at a p- value of < 0.001.

4. 4.

‘Weak evidence’: nominally significant associations (p< 0.05).

5. 5.

‘Poor evidence’: obtained from samples with < 1000 cases.

Results

Study characteristics

Thirteen of the 50 meta-analyses or systematic reviews that satisfied the qualifying requirements were included in our analysis after duplicates were eliminated (Fig. 1). These thirteen studies were published between 2008 and 2024. The results of this study were characterized by clinical skills and critical thinking. We categorized the included studies into multiple groups according to the criteria and features of each study. We have presented characteristics of the included studies in Tables 1 and 2. The studies conducted by Zhang et al. [25, 26, 17, 27,28,29,30]. were classified as ‘recommended’. However, the studies by Wosinski et al. [32,33,34, 33,34,35, 10] were classified as ‘weak evidence’. The risk of bias in randomized controlled trials (RCTs) was evaluated using traffic light charts, whereas the quality of inclusion in quasi-experimental studies was evaluated using the improved Jadade scale, the Newcastle Ottawa Scale (NOS) for cohort studies, and the 2nd edition of the Cochrane Bias Risk Tool for randomized trials. Wosinski et al. [32,33,34] did not evaluate the quality of their studies. A total of 117 RTC studies from different clinical specialties were included in the literature.

[IMAGE OMITTED: SEE PDF]

[IMAGE OMITTED: SEE PDF]

[IMAGE OMITTED: SEE PDF]

Outcome analysis

Wosinski et al. [32], Kong et al. [26], Sharma et al. [17], Wei et al. [30], Wei et al. [34], and Oliveira et al. [31] found that PBL has significant advantages in critical thinking compared to traditional education or team-based learning (TBL). However, Manuaba et al. [10] suggested that PBL does not have an advantage. Kong [26] proposed that the pooled effect size showed a significant difference in overall critical thinking scores (SMD = 0.33, 95%CI = 0.13–0.52, p = 0.0009) in favor of PBL compared with traditional lectures. The meta-analysis conducted by Ma et al. [27] found that the PBL teaching model significantly increased skill scores by a mean of 1.56 compared with those of the life-based learning (LBL) teaching model (95% CI, 0.87–2.25; P < 0.00001),

Zhang et al. [25], Sayyah et al. [35], Li et al. [28], Liu et al. [29], and Forbes et al [33]. demonstrated the statistical significance of PBL in improving clinical skills compared to traditional education or other educational methods such as TBL for the target audience. Zhang et al. [25] showed that a random-effects model was used to combine the Standard Mean Difference(SMD) values because of significant heterogeneity (I2 = 95%, P < 0.00001) in the 13 studies they examined. Their results showed a significant difference in skill scores (SMD = 2.10, 95%CI [1.38, 2.83]) in favor of PBL compared to traditional teaching. Sharma et al. found that the weighted pooled SMD was 0.44 (confidence interval [CI]: 0.14–0.73, I2 = 65.56%, p < 0. 001). Additionally, Li et al. [28] showed a significant difference in skill scores (WMD = 6.54, 95% CI: 4.55–8.53, p < 0.0001) in favor of the intervention group compared to the LBL group. The pooled effect size in their study showed a significant difference in skill scores Weighted Mean Difference (WMD) = 6.54, 95% CI: 4.55–8.53, p < 0.0001) in favor of the intervention group compared to the LBL group.

Manuaba’s study [10] revealed no significant difference in critical thinking knowledge assessment (p = 0.29), problem-solving (p = 0.47), or self-directed learning (p = 0.34) between the PBL group and non-PBL group. This may be because this meta-analysis did not include randomized control trials (RCT) studies and consisted entirely of case–control, randomized quasi-experimental, and randomized cross-matching studies.

The recommended studies include more rigorous designs such as Randomized Controlled Trials (RCTs) and cohort studies, such as the RCT studies by Ma Y et al. [27] and Li T et al. [28], and the cohort study by Wosinski J et al. [32]. In contrast, studies with weaker evidence are mostly descriptive studies, quasi-experimental studies, and case–control studies, such as the descriptive/quasi-experimental study by Yuan H et al. [34]. and the case–control study by Manuaba [10]. The conclusions of the recommended studies are relatively clear, with Ma Y et al. [27] and Li T et al. [28] both concluding that PBL significantly outperforms LBL in improving skill scores. However, studies with weaker evidence often do not provide clear conclusions or fail to find significant differences, such as Manuaba [10] which found no significant difference between the PBL and non-PBL groups in critical thinking.However, since the study in this literature focused on first-year medical students, it is not unexpected that no differences were found between the PBL group and the non-PBL group in terms of critical thinking, problem-solving, or self-directed learning.

Regarding research subjects from different fields, Both nursing and medical student studies have demonstrated the benefits of PBL. Nursing student studies tend to focus more on critical thinking and problem-solving, as seen in the works of Wosinski J, et al. [32] and Sharma S et al. [17]. In contrast, medical student studies emphasize the improvement of clinical and theoretical skills, as highlighted by Ma Y et al. [27] and Li T et al. [28].

Publication bias

The publication bias assessment also differed among the 13 included studies. Small study effects and excess significance bias. In this umbrella review, we could not evaluate the small-study effects. The results of the excess significance bias analyses are presented in Table 2.. Most of the studies included in this analysis exhibited considerable heterogeneity. To address this, the authors employed a random effects model or conducted subgroup analyses to reduce the impact of heterogeneity. However, several factors may contribute to the observed variability. One significant factor is the lack of supervision and evaluation of instructors in the existing research, which could lead to differences in teaching quality and thus contribute to heterogeneity. Additionally, there are variations in the baseline characteristics and prior learning experiences of the participants. Only a small number of studies addressed factors such as the participants’ prior clinical work experience and their exposure to PBL teaching methods, suggesting that these variables learning ability, clinical experience, and prior PBL trainingncould be potential sources of heterogeneity. Furthermore, the assessment methods used by the training departments have not been standardized across different specialties, resulting in an inconsistent evaluation framework. For assessing competence, long-term evaluations would be more appropriate. The absence of a unified evaluation system could therefore be another contributing factor to the observed heterogeneity.

Residual confounding

We acknowledge that when examining the relationship between PBL and teaching quality, there may be some residual confounding or other biases that may lead to overestimation or underestimation and restrictive causal interferen.

Discussion

Clinical implication

A major issue in medical education is the growing emphasis on competency development, which requires an appropriate teaching approach to achieve it [28]. PBL was first introduced in North America and has been extensively adopted as a single-course or curriculum-wide teaching approach in numerous medical institutions across the world [36]. Our work is crucial in describing the data that are currently available and guiding future research paths on the subject of investigation [37].

PBL has shown certain advantages in research across different fields. In the field of basic biomedical sciences, PBL helps enhance students’ critical thinking and problem-solving abilities. Previous studies also highlighted these advantages in nursing students [17, 32]. In the field of residency training, PBL focuses more on improving clinical skills and theoretical knowledge, with emphasizing these effects in medical students [27, 28]. This cross-disciplinary applicability suggests that PBL has broad educational value and potential.

As an educational method, PBL better achieves educational goals. For nursing students, PBL helps cultivate professionals who can think independently and solve problems effectively by promoting the development of critical thinking and problem-solving skills [17, 32]. For medical students, PBL contributes to the cultivation of doctors with solid medical foundations and clinical practice abilities by improving clinical skills and theoretical knowledge [27, 28]. This alignment with educational goals helps enhance overall educational quality and contributes to the development of more outstanding healthcare professionals for society.

The literature highlights the benefits of PBL for mastering theoretical knowledge, clinical diagnostic thinking, teamwork skills, problem-solving and analysis abilities, document consultation skills, learning interest, and learning efficiency. However, it does not show any benefits for enhancing hands-on, communication, or self-directed learning skills [28]. Additionally, future doctors with PBL training should expect to become more proficient communicators, problem solvers, and self-learners, in addition to possessing vast knowledge [38]. Through a review of the available literature, the authors showed that a PBL curriculum encourages students’ willingness to serve patients [33]. Koh et al. examined how PBL in medical school can impact physician competencies following graduation and discovered that the social and cognitive domains showed the greatest improvement [10]. Therefore, determining which PBL abilities are useful in the long term may not be simple.

First-year medical students’ critical thinking and knowledge, problem-solving skills, and sense of self-direction appeared to be unimpaired by the PBL approach when compared with the traditional teaching methodology [39]. A minimal difference was observed in first-year residents’ initial clinical capability improvement between case/problem-based and lecture-based learning methods [28]. Given this, it is advisable to introduce organized learning early in the curriculum and gradually expand active and self-directed learning afterwards [40]. One study revealed no notable variations in the standards of primary care [41], indicating that a suitable instrument should exist to ascertain the impact of distinct pedagogical approaches on the enduring retention of information.

It is important for policymakers in non-Western nations to understand that PBL is not readily ‘lifted and shifted’ from the Western environment in which it originated and that they need to modify their adoption methods appropriately. This might help educators and medical students ‘hybridize’ with PBL to make it effective in their particular setting. Consequently, it is critical to examine the features of PBL. Numerous medical specialties, including internal medicine, obstetrics and gynecology, anatomy, pathology, and medical cell biology, have demonstrated the benefits of PBL. PBL offers no advantage in terms of operational skills for diagnostic imaging [28]. The current application of PBL in surgical education has also not been widely or thoroughly studied [42]. These studies also provide ideas for further research on teaching methods suitable for different majors and abilities [43]. Therefore, a path for further research should be outlined on the design of suitable teaching methods for different specialties and abilities. The literature has demonstrated that PBL is more enjoyable for students [8]. The teaching method for PBL sees it often combined with other teaching methods, such as LBL and digital teaching. Case-based learning(CBL) takes clinical cases as a starting point, uses relevant knowledge and theory to analyze these cases, and efficiently solves clinical problems. Digital PBL (DPBL) may be more effective than traditional learning or traditional PBL in improving skills [44]. Accordingly, we believe that a combined teaching model can be used for education in limited medical colleges and by teachers [43].

Clinical practice

Primarily, we deduced that educators’ teaching level, learning ability, clinical work experience, and residents’ PBL training experience were potential causes of PBL heterogeneity. Thus, the homogenization of teachers must be on the agenda. Furthermore, the introduction of PBL into the medical school curricula was accompanied by other changes, making it impossible to fully dissociate the findings attributable to problem-based learning from those attributable to other curricula changes [44]. This suggests that consistency in teaching methods should be the premise of subsequent teaching research. When students master certain basic knowledge, PBL education may help them achieve better critical thinking, skill levels, and teaching effects; that is, according to the increase in grade, the teaching frequency of PBL can be gradually increased.

Limitations

This review had some limitations. First, notable variations in study design, samples, locations, and measurements between studies might not have been considered in our review of the key findings of the examined studies. Second, the frequency of PBL sessions was not considered, and the majority were single-center experiences with no significant multicenter research. Third, it is important to mention the limitation of not covering all clinical domains and how this might affect the generalizability of the research findings. If the effectiveness of PBL is overestimated or underestimated in certain specific areas, it could influence its application in other clinical domains. For example, some clinical fields may rely more on the delivery of factual knowledge, while others might place greater emphasis on critical thinking and teamwork skills. Therefore, the results of the study may not be fully applicable to all clinical subjects or educational settings. Fourth, not all of the included studies considered medical students of various years. The workloads of students in teir first to fifth years of study varied significantly in the field of clinical medicine.The frequency of PBL courses may influence the results measured in studies using the same educational intervention. Different frequencies of PBL could lead to variations in students’ learning outcomes, especially in terms of the depth of knowledge acquisition and the development of critical thinking. Frequent PBL sessions may facilitate more frequent interactions and reflections, thereby enhancing student engagement and knowledge internalization.Different PBL sessions might affect students’ cognitive training and problem-solving abilities, thus affecting learning outcomes. Finally, Although current evidence suggests that PBL is effective, the evidence level is relatively low, and there is considerable heterogeneity. Therefore, more high-quality studies are needed in the future to validate its effectiveness. Future research should focus on strengthening supervision and evaluation of instructors, clarifying participants’ learning abilities, clinical experience, and prior PBL training, and using a consistent evaluation system. Only through these measures can the true effectiveness of PBL be scientifically proven.

Conclusion

PBL may be a successful and an effective instructional strategy, not only medical, but also nursing and dental education. Students can learn skills and acquire competencies useful for their medical professionalism through PBL in addition to their existing knowledge.

Data availability

The data that support the findings of this study are available from the corresponding author, [author initials], upon reasonable request.

References

Albanese MA, Mitchell S. Problem-based learning: a review of literature on its outcomes and implementation issues. Acad Med. 1993;68(1):52–81. https://doi.org/10.1097/00001888-199301000-00012. Erratum.In:AcadMed1993;68(8):615. PMID: 8447896.

Neufeld VR, Barrows HS. The “McMaster Philosophy”: an approach to medical education. J Med Educ. 1974;49(11):1040–50 PMID: 4444006.

Cui H, Xie X, Wang B, Zhao Y. Effectiveness of flipped classroom in pharmacy education - a meta-analysis. BMC Med Educ. 2023;23(1):881. https://doi.org/10.1186/s12909-023-04865-2. PMID:37978369;PMCID:PMC10657003.

Zhou J, Zhou S, Huang C, Xu R, Zhang Z, Zeng S, Qian G. Effectiveness of problem-based learning in Chinese pharmacy education: a meta-analysis. BMC Med Educ. 2016;16:23. https://doi.org/10.1186/s12909-016-0546-z. PMID:26787019;PMCID:PMC4719679.

Yan J, Wen Y, Liu X, Deng M, Ye B, Li T, Wang H, Jia C, Liao J, Zhang L. The effectiveness of problem-based learning and case-based learning teaching methods in clinical practical teaching in TACE treatment for hepatocellular carcinoma in China: a bayesian network meta-analysis. BMC Med Educ. 2024;24(1):665. https://doi.org/10.1186/s12909-024-05615-8. PMID:38886707;PMCID:PMC11184776.

Jones R, Higgs R, de Angelis C, Prideaux D. Changing face of medical curricula. Lancet. 2001;357(9257):699–703. https://doi.org/10.1016/S0140-6736(00)04134-9. PMID: 11247568.

Enarson C, Cariaga-Lo L. Influence of curriculum type on student performance in the United States Medical Licensing Examination Step 1 and Step 2 exams: problem-based learning vs. lecture-based curriculum. Med Educ. 2001;35(11):1050–5. https://doi.org/10.1046/j.1365-2923.2001.01058.x. PMID: 11703641.

Vernon DT, Blake RL. Does problem-based learning work? A meta-analysis of evaluative research. Acad Med. 1993;68(7):550–63. https://doi.org/10.1097/00001888-199307000-00015. PMID: 8323649.

Jones RW. Problem-based learning: description, advantages, disadvantages, scenarios and facilitation. Anaesth Intensive Care. 2006;34(4):485–8. https://doi.org/10.1177/0310057X0603400417. PMID: 16913347.

Manuaba IBAP, -No Y, Wu CC. Correction: The effectiveness of problem based learning in improving critical thinking, problem-solving and self-directed learning in first-year medical students: A meta-analysis. PLoS One. 2024;19(5):e0303724. https://doi.org/10.1371/journal.pone.0303724. Erratum for: PLoS One. 2022;17(11):e0277339. https://doi.org/10.1371/journal.pone.0277339. PMID: 38722889; PMCID: PMC11081207.

Delavari S, Barzkar F, MJP Rikers R, Pourahmadi M, Soltani Arabshahi SK, Keshtkar A, Dargahi H, Yaghmaei M, Monajemi A. Teaching and learning clinical reasoning skill in undergraduate medical students: a scoping review. PLoS One. 2024;19(10):e0309606. https://doi.org/10.1371/journal.pone.0309606. PMID: 39413083; PMCID: PMC11482728.

Kurane I, Ennis FA. Induction of interferon alpha from human lymphocytes by autologous, dengue virus-infected monocytes. J Exp Med. 1987;166(4):999–1010. https://doi.org/10.1084/jem.166.4.999. PMID: 3116147; PMCID: PMC2188728.

Holmes NG, Wieman CE, Bonn DA. Teaching critical thinking. Proc Natl Acad Sci U S A. 2015;112(36):11199–204. https://doi.org/10.1073/pnas.1505329112. Epub 2015 Aug 17. PMID: 26283351; PMCID: PMC4568696.

Norman GR. Problem-solving skills, solving problems and problem-based learning. Med Educ. 1988;22(4):279–86. https://doi.org/10.1111/j.1365-2923.1988.tb00754.x. PMID: 3050382.

Imanieh MH, Dehghani SM, Sobhani AR, Haghighat M. Evaluation of problem-based learning in medical students’ education. J Adv Med Educ Prof. 2014;2(1):1–5 PMID: 25512911; PMCID: PMC4235539.

Gao X, Yan D, Zhang Y, Ruan X, Kang T, Wang R, Zheng Q, Chen S, Zhai J. Comparison of the impact of team-based learning and lecture-based learning on nursing students’ core competencies: A systematic review and meta-analysis. Nurse Educ Pract. 2024;76. https://doi.org/10.1016/j.nepr.2024.103945. Epub 2024 Mar 19 PMID: 38554616.

Sharma S, Saragih ID, Tarihoran DETAU, Chou FH. Outcomes of problem-based learning in nurse education: A systematic review and meta-analysis. Nurse Educ Today. 2023;120. https://doi.org/10.1016/j.nedt.2022.105631. Epub 2022 Nov 8. PMID: 36427452.

Poole R, Kennedy OJ, Roderick P, Fallowfield JA, Hayes PC, Parkes J. Coffee consumption and health: umbrella review of meta-analyses of multiple health outcomes. BMJ. 2017;359:j5024. https://doi.org/10.1136/bmj.j5024. Erratum. In: BMJ. 2018;360:k194. https://doi.org/10.1136/bmj.k194. PMID: 29167102; PMCID: PMC5696634.

Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J, Henry DA, Boers M. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol. 2009;62(10):1013–20. https://doi.org/10.1016/j.jclinepi.2008.10.009. Epub 2009 Feb 20 PMID: 19230606.

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, Moher D, Tugwell P, Welch V, Kristjansson E, Henry DA. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008. https://doi.org/10.1136/bmj.j4008. PMID:28935701;PMCID:PMC5833365.

Wu H, Huang CL, Deng JS, Ying CQ, Tung TH, Zhu JS. Positive and negative factors of parents vaccinating their children against COVID-19: an umbrella review. Prev Med Rep. 2024;42:102724. https://doi.org/10.1016/j.pmedr.2024.102724. PMID: 38681061; PMCID: PMC11046294.

Swierz MJ, Storman D, Zajac J, Koperny M, Weglarz P, Staskiewicz W, Gorecka M, Skuza A, Wach A, Kaluzinska K, Bochenek-Cibor J, Johnston BC, Bala MM. Similarities, reliability and gaps in assessing the quality of conduct of systematic reviews using AMSTAR-2 and ROBIS: systematic survey of nutrition reviews. BMC Med Res Methodol. 2021;21(1):261. https://doi.org/10.1186/s12874-021-01457-w. PMID:34837960;PMCID:PMC8627612.

Belbasis L, Bellou V, Evangelou E, Ioannidis JP, Tzoulaki I. Environmental risk factors and multiple sclerosis: an umbrella review of systematic reviews and meta-analyses. Lancet Neurol. 2015;14(3):263–73. https://doi.org/10.1016/S1474-4422(14)70267-4. Epub 2015 Feb 4 PMID: 25662901.

Wu H, Huang CL, Deng JS, Ying CQ, Tung TH, Zhu JS. Positive and negative factors of parents vaccinating their children against COVID-19: An umbrella review. Prev Med Rep. 2024;42. https://doi.org/10.1016/j.pmedr.2024.102724.PMID:38681061;PMCID:PMC11046294.

Zhang S, Xu J, Wang H, Zhang D, Zhang Q, Zou L. Effects of problem-based learning in Chinese radiology education: a systematic review and meta-analysis. Medicine (Baltimore). 2018;97(9):e0069. https://doi.org/10.1097/MD.0000000000010069. PMID:29489669;PMCID:PMC5851720.

Kong LN, Qin B, Zhou YQ, Mou SY, Gao HM. The effectiveness of problem-based learning on development of nursing students’ critical thinking: a systematic review and meta-analysis. Int J Nurs Stud. 2014;51(3):458–69. https://doi.org/10.1016/j.ijnurstu.2013.06.009. Epub 2013 Jul 10 PMID: 23850065.

Ma Y, Lu X. The effectiveness of problem-based learning in pediatric medical education in China: A meta-analysis of randomized controlled trials. Medicine (Baltimore). 2019;98(2):e14052. https://doi.org/10.1097/MD.0000000000014052. PMID:30633204;PMCID:PMC6336616.

Li T, Wang W, Li Z, Wang H, Liu X. Problem-based or lecture-based learning, old topic in the new field: a meta-analysis on the effects of PBL teaching method in Chinese standardized residency training. BMC Med Educ. 2022;22(1):221. https://doi.org/10.1186/s12909-022-03254-5. PMID:35361207;PMCID:PMC8974027.

Liu CX, Ouyang WW, Wang XW, Chen D, Jiang ZL. Comparing hybrid problem-based and lecture learning (PBL + LBL) with LBL pedagogy on clinical curriculum learning for medical students in China: a meta-analysis of randomized controlled trials. Medicine (Baltimore). 2020;99(16):e19687. https://doi.org/10.1097/MD.0000000000019687. PMID:32311943;PMCID:PMC7220526.

Wei B, Wang H, Li F, Long Y, Zhang Q, Liu H, Tang X, Rao M. Effectiveness of Problem-Based Learning on Development of Nursing Students’ Critical Thinking Skills: A Systematic Review and Meta-analysis. Nurse Educ. 2024;49(3):E115–9. https://doi.org/10.1097/NNE.0000000000001548. Epub 2023 Nov 16 PMID: 38016174.

Oliveira LB, Díaz LJ, Carbogim Fda C, Rodrigues AR, Püschel VA. Effectiveness of teaching strategies on the development of critical thinking in undergraduate nursing students: a meta-analysis. Rev Esc Enferm USP. 2016;50(2):355-64. English, Portuguese. https://doi.org/10.1590/S0080-623420160000200023. PMID: 27384217.

Wosinski J, Belcher AE, Dürrenberger Y, Allin AC, Stormacq C, Gerson L. Facilitating problem-based learning among undergraduate nursing students: A qualitative systematic review. Nurse Educ Today. 2018;60:67–74. https://doi.org/10.1016/j.nedt.2017.08.015. Epub 2017 Sep 8 PMID: 29032293.

Forbes HM, Syed MS, Flanagan OL. The Role of Problem-Based Learning in Preparing Medical Students to Work As Community Service-Oriented Primary Care Physicians: A Systematic Literature Review. Cureus. 2023;15(9):e46074. https://doi.org/10.7759/cureus.46074. PMID:37900379;PMCID:PMC10609366.

Yuan H, Williams BA, Fan L. A systematic review of selected evidence on developing nursing students’ critical thinking through problem-based learning. Nurse Educ Today. 2008;28(6):657–63. https://doi.org/10.1016/j.nedt.2007.12.006. Epub 2008 Feb 11 PMID: 18267348.

Sayyah M, Shirbandi K, Saki-Malehi A, Rahim F. Use of a problem-based learning teaching model for undergraduate medical and nursing education: a systematic review and meta-analysis. Adv Med Educ Pract. 2017;8:691–700. https://doi.org/10.2147/AMEP.S143694. PMID:29042827;PMCID:PMC5633274.

Chan SCC, Gondhalekar AR, Choa G, Rashid MA. Adoption of Problem-Based Learning in Medical Schools in Non-Western Countries: A Systematic Review. Teach Learn Med. 2024;36(2):111–22. https://doi.org/10.1080/10401334.2022.2142795. Epub 2022 Nov 29 PMID: 36444767.

Perez A, Green J, Moharrami M, Gianoni-Capenakas S, Kebbe M, Ganatra S, Ball G, Sharmin N. Active learning in undergraduate classroom dental education- a scoping review. PLoS ONE. 2023;18(10):e0293206. https://doi.org/10.1371/journal.pone.0293206. PMID:37883431;PMCID:PMC10602256.

Trullàs JC, Blay C, Sarri E, Pujol R. Effectiveness of problem-based learning methodology in undergraduate medical education: a scoping review. BMC Med Educ. 2022;22(1):104. https://doi.org/10.1186/s12909-022-03154-8. PMID:35177063;PMCID:PMC8851721.

Mennin SP, Friedman M, Skipper B, Kalishman S, Snyder J. Performances on the NBME I, II, and III by medical students in the problem-based learning and conventional tracks at the University of New Mexico. Acad Med. 1993;68(8):616–24. https://doi.org/10.1097/00001888-199308000-00012. PMID: 8352874.

Tamblyn R, Abrahamowicz M, Dauphinee D, Girard N, Bartlett G, Grand’Maison P, Brailovsky C. Effect of a community oriented problem based learning curriculum on quality of primary care delivered by graduates: historical cohort comparison study. BMJ. 2005;331(7523):1002. https://doi.org/10.1136/bmj.38636.582546.7C. Epub 2005 Oct 20. PMID: 16239292; PMCID: PMC1273455.

Chang G, Cook D, Maguire T, Skakun E, Yakimets WW, Warnock GL. Problem-based learning: its role in undergraduate surgical education. Can J Surg. 1995;38(1):13–21 PMID: 7882203.

Zheng QM, Li YY, Yin Q, Zhang N, Wang YP, Li GX, Sun ZG. The effectiveness of problem-based learning compared with lecture-based learning in surgical education: a systematic review and meta-analysis. BMC Med Educ. 2023;23(1):546. https://doi.org/10.1186/s12909-023-04531-7. PMID:37528406;PMCID:PMC10394865.

Hartling L, Spooner C, Tjosvold L, Oswald A. Problem-based learning in pre-clinical medical education: 22 years of outcome research. Med Teach. 2010;32(1):28–35. https://doi.org/10.3109/01421590903200789. PMID: 20095771.

Koh GC, Khoo HE, Wong ML, Koh D. The effects of problem-based learning during medical school on physician competency: a systematic review. CMAJ. 2008;178(1):34–41. https://doi.org/10.1503/cmaj.070565. PMID:18166729;PMCID:PMC2151117.