Effectiveness of Problem-Based Learning versus Traditional Teaching Methods in Improving Acquisition of Radiographic Interpretation Skills among Dental Students—A Systematic Review and Meta-Analysis

Abstract

Problem-based learning is an experiential and student-centred learning method to practice important skills like querying, critical thinking, and collaboration through pair and group work. The study is aimed at comparing the effectiveness of problem-based learning (PBL) and traditional teaching (TT) methods in improving acquisition of radiographic interpretation skills among dental students. Clinical trials (randomized and nonrandomized) were conducted with the help of dental students studying oral radiology using PBL and TT methods and assessing radiographic interpretation skills, knowledge scores, and satisfaction level as outcomes. Articles published from PubMed/MEDLINE, DOAJ, Cochrane Central Register of Controlled Trials, and Web of Science were searched. The quality of the studies was evaluated using the Cochrane Collaboration Tool, the MINORS Checklist, and the Risk of Bias in Nonrandomized Studies of Interventions (ROBIN-I) tool. Meta-analysis was done using Review Manager 5.3. There were twenty-four articles for qualitative synthesis and 13 for meta-analysis. The cumulative mean difference was found to be 0.54 (0.18, 0.90), 4.15 (-0.35, 8.65), and -0.14 (-0.36, 0.08) for radiographic interpretation skills, knowledge scores, and satisfaction level, respectively, showing significant difference favouring PBL as compared to TT except for satisfaction level which favoured the TT group. To understand the long-term effectiveness of PBL over TT methods in oral radiology among dental students, well-designed long-term randomized controlled trials are needed.

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. Introduction

Knowledge transfer is a major concern in improving educational practices. “Knowledge” is categorized into three areas by the cognitive psychologists as “declarative knowledge,” “procedural knowledge,” and an ill-defined grey zone between declarative and procedural knowledge that includes the reasoning skills often described as critical thinking and problem solving [1]. Traditional teaching methods often follow linear or modular learning, which is a highly directed, controlled, and program-centred approach as directed by the tutor, wherein the learners complete the given activities without developing the critical reasoning skill [2].

To develop problem-solving ability, one must progress from convergent thinking to critical thinking by adopting a learner-centric approach, emphasizing on self-directed learning and designing interlinkable yet independent resources that the learner can explain in his or her own words [3]. Problem-based learning (PBL) is an educational approach in which a problem serves as an incentive toward finding solutions leading to active, self-motivated, and dynamic education [4]. PBL was introduced into health sciences education at McMaster University in 1969 and was first introduced to dental education at the Faculty of Odontology in Malmö, Sweden, in 1990 [5].

The core model of PBL (see the overview of Barrows in 1996) is composed of the following six characteristics: learning is student centred, learning occurs in small student groups, tutors are facilitators or guides, problems form the organizing focus and stimulus for learning, problems are a vehicle for the development of problem-solving skills, and new information is attained through self-directed learning [6].

Oral radiology is an indispensable part of undergraduate and postgraduate dental training. Radiographs form an essential diagnostic tool for patient assessment and treatment planning and form the mainstay of all clinical specialties of dentistry [7]. Learning the basic skills of interpretation of intra- and extraoral radiographs requires having (1) mastery in perception, which is the ability to recognize abnormal patterns on a radiograph, and (2) cognition, i.e., the ability to interpret these abnormal patterns to arrive at a diagnosis which are the two distinguishable and inseparable components of visual diagnosis [8].

Traditionally, during the clinical posting of Oral Radiology, the dental students are taught about skills of radiographic interpretation in a batch of 10–12 students by the tutor. This type of learning is passive and teacher-centred, and therefore, the students may develop a minimal capacity for adopting a deep approach to learning, searching for deeper meaning and personal relevance in the topic and are therefore unable to apply learned concepts in new situations competently [9].

The emergence of newer nonlinear teaching and learning methods such as action-based learning, competency-based education, contextual- and inquiry-based learning, lifelong learning, problem-based learning, and self-directed learning showed a tremendous increase of educational literature on conduction and implementation of these new learning appraches [10]. In the same way, oral radiography teaching has been going through a renovation from the traditional didactic system to an approach of problem-based learning whereby students take a more active role in their learning. The flipped classroom (FC) model is an integrated method for learning in which students review content ahead of the classroom session and teachers use class time for active learning [11]. Implementation of syndicate-based learning [9] and the one-minute preceptor (OMP) model provide experiential learning to the students for future practice [12]. Radiology simulator-supported training [13] and digital environments allow visual communication between the educator and the learner, thus promoting appealing and engaging participation and better understanding [14]. The above studies on problem-based learning, as compared to traditional teaching methods, have reported acceptable positive gains in cognitive outcomes.

There are few systematic reviews in dentistry that have compared the effectiveness of PBL with traditional teaching approaches across various specialties of dentistry, but none have been specifically done to assess the effectiveness of these newer approaches in the field of oral medicine and radiology. The aim of this systematic review is to summarize the evidence and compare the effectiveness of PBL with that of traditional teaching approaches in improving acquisition of radiographic interpretation skills among dental students. The null hypothesis is as follows: “There is no difference in PBL and traditional teaching approaches in improving acquisition of radiographic interpretation skills among dental students.”

2. Methods

This systematic review and meta-analysis are written and reported according to the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) statement and registered in PROSPERO under number CRD42020184441. The proposed focused research question in the Patient, Intervention, Comparison, and Outcome (PICO) format is as follows: “Is there a difference in the effectiveness of problem-based learning (PBL) versus traditional teaching methods (TT) in improving acquisition of radiographic interpretation skills among dental students?”

2.1. Search Strategy

A comprehensive electronic search was carried out by two of the authors independently (STK, JG) on databases, such as PubMed/MEDLINE, Cochrane Central Register of Controlled Trials, and Web of Science until June 2020 to retrieve articles in the English language. A specific electronic search of journals presented in Table 1 was conducted. The searches in the clinical trials database, cross-referencing, and grey literature were conducted using Google Scholar, Greylist, and OpenGrey. The complete process of the literature search is mentioned in Screening Process.

Table 1

The search strategy and PICOS tool.

Search strategy
Focused question	Is there a difference in the effectiveness of problem-based learning (PBL) versus traditional teaching (TT) methods in improving acquisition of radiographic interpretation skills among dental students?
Search strategy
Population	(Dental students [MeSH] OR dental undergraduate students [text word] OR undergraduate students [text word] OR dentistry students [text word] OR post graduate students [text word] OR students [text word] OR bachelor of dental surgery [text word])
Intervention	(Problem-based learning [MeSH] OR syndicate learning [text word] OR blended learning [text word] OR schema-based learning [text word] OR smartphone use [text word] OR experiential learning [text word] OR active learning [text word] OR problem based curricula [text word] OR one minute preceptor [text word] OR simulation-based learning [text word] OR conventional training [text word])
Comparisons	(Lecture [MeSH] OR instructional learning [text word] OR instructional method [text word] OR traditional clinical training [text word] OR traditional didactic method [text word])
Outcomes	(X-ray image [text word] OR dental X-ray [text word] OR X-ray diagnosis [text word] OR oral radiography [text word] OR dental radiography [text word] OR radiographic image interpretation [text word] OR interpretation skills [text word] OR diagnostic accuracy [text word] OR dental X-ray diagnostic accuracy [text word] OR dentomaxillofacial radiology [text word] OR radiographic image interpretation [text word])
Study design	(Clinical trials [MeSH] OR randomized controlled studies [text word] OR randomized control trials [MeSH] OR randomized control clinical trial MeSH OR non-randomized control trials [text word] OR quasi experimental studies [text word] OR before and after study design [text word] OR cohort studies [text word] OR in vivo study [text word])
Search combination	#1 AND #2 AND #3 AND #4
Database search
Language	No restriction (articles in English language or other language where English translation is possible.)
Electronic databases	PubMed/MEDLINE, Cochrane Central Register of Controlled Trials, Web of Science
Journals	Dentomaxillofacial Radiology, European Journal of Dental Education, Journal of Contemporary Medical Education, BMC Medical Education, Journal of Dental Education
Period of publication	1-1-2000 to 30-06-2020

Medical Subject Heading (MeSH) terms, keywords, and other free terms combined with Boolean operators (OR, AND) were used for searching articles. The search strategy and population, interventions, comparisons, outcomes, and the study design (PICOS) tool are presented in Table 1.

2.2. Inclusion Criteria Outline according to the PICOS Strategy

(i) Population (P): studies reporting dental undergraduate and postgraduate students who are taught the subject of oral medicine and radiology

(ii) Interventions (I): problem-based learning, case-based learning, syndicate learning, case-based blended learning, one-minute preceptor, small-group teaching, simulation teaching, virtual teaching, schema-based teaching, and active learning

(iii) Comparison (C): dental students who receive other learning schemes that are traditional methods of teaching/non-problem-based learning-lectures, passive learning, and instructional learning

(iv) Outcome (O): primary outcome—improved radiographic interpretation skills, knowledge, and student’s perception; proficiency test of the dental students

(v) Secondary outcome: student perceptions towards PBL methods

(vi) Study design (S): clinical trials, randomized controlled studies, nonrandomized control trials, quasiexperimental studies, before and after study design, and cohort studies comparing the effect of PBL and traditional training

Since RCTs are considered a gold standard of clinical trials, we intended to include those. Also, we included the other studies, as they are the most common study types which included human participants as their subjects. Ultimately, we aimed to summarize the evidence, and we tried to include all the types of study designs mentioned above for the same.

2.3. Exclusion Criteria

Review reports, case series, cross-sectional studies, and survey reports were excluded. The article which did not report the elements of PBL as described by Barrows in 1996 [6] was excluded. In addition, articles reporting about a single intervention were excluded. Articles reporting only abstracts were also excluded.

2.4. Screening Process

The search and screening according to the previously established protocol were conducted by two reviewing authors independently (STK, JG). Firstly, the titles and abstracts were analysed. Secondly, full-text articles were chosen for in-depth reading and analysed as per the inclusion and exclusion criteria for data extraction. All these above processes were done separately by the two reviewers, and then the level of agreement between the two reviewers, calculated by Cohen’s kappa ($k$), was 0.94 for titles and abstracts and 0.96 for full texts. The differences among authors/reviewers were resolved by the third author (AMP) after discussion. For the clarification of doubts and missing data of the included studies, the respective authors were contacted by email if needed (none were contacted in our study).

2.5. Data Extraction

Two independent authors (SM, AML) extracted the following data independently and then correlated the data from the included studies. The data extracted was recorded under following headings: study identification number, authors, study design, follow-up, number of subjects, age, gender, method of education, oral radiology knowledge, radiographic interpretation skills, student’s satisfaction, effect size, and author’s remarks.

2.6. Assessments of the Risk of Bias and Quality

The level of evidence for every included study was assessed using the Joanna Briggs Institute (JBI) Levels of Evidence [15]. Risk of bias for the selected randomized controlled trials (RCTs) was executed by using the Cochrane Collaboration Tool [16] which including random sequence generation, allocation concealment, blinding of participants, incomplete outcome data, selective reporting, and other biases, while quality assessment of the same was done by the Agency for Healthcare Research and Quality (AHRQ) standard [17]. The quality assessment of nonrandomized studies was done using the MINORS checklist [18] with no restriction on the follow-up period being considered appropriate for the included studies, and the risk of bias was done using Risk of Bias in Nonrandomized Studies of Interventions (ROBIN-I) [19].

2.7. Statistical Analysis

Review Manager (RevMan) 5.3 was used for statistical analysis. The combined results were expressed as mean difference (MD) and standard deviation for the continuous data at 95% confidence intervals (CIs) and $P<0.05$ was considered significant. Statistical heterogeneity was assessed by the $I^2$ test at $\alpha =0.10$. For $I^2>50\%$, the random-effects model was applied. A funnel plot (plot of effect size versus standard error) was generated to examine possible publication bias.

3. Results

3.1. Literature Search

The PRISMA statement flowchart summarizing the selection process is shown in Figure 1. Among the 24 full-text articles, 13 were selected after prescreening, applying the eligibility criteria, and addressing the PICOS question. Eleven studies were excluded since 9 did not have appropriate outcome variables and 2 studies had an inappropriate study group; hence, only 13 studies were included in the qualitative and quantitative analysis.

[figure omitted; refer to PDF]

The risk of bias of five nonrandomized studies was executed according to the ROBIN-I tool, where two studies [21, 24] showed low risk and the remaining three studies [8, 20, 22] showed moderate risk of bias (Table 5), whereas quality assessment was assessed using the methodological index for nonrandomized studies (MINORS) [18] for comparative studies wherein three studies [8, 20, 24] showed a score of 20 and the remaining two studies showed the score of 19 [22] and 22 [21] each. The detailed scores of the studies are presented in Table 6.

Table 5

Risk of bias judgement for nonrandomized trials (ROBIN-I tool).

Bias domain	Baghdady MT et al., 2014	Busanello FH et al., 2015	Cruz AD et al., 2014	Ji YA et al., 2018	Kavadella A et al., 2012
Bias due to confounding	N	N	N	N	N
Bias in selection of participants into the study	N	PN	N	N	PN
Bias in classification of interventions	N	N	N	N	N
Bias due to deviations from intended interventions	N	N	N	N	N
Bias due to missing data	PN	N	N	N	N
Bias in measurement of outcomes	N	N	N	N	N
Bias in selection of the reported result	N	N	N	N	N
Overall bias

Green circle=low risk; yellow circle=moderate risk; red circle=high risk; N=number; PN=partial number.

Table 6

Methodological index for nonrandomized studies (MINORS).

	A clearly stated aim	Inclusion of consecutive patients	Prospective collection of data	Endpoints appropriate to the aim of the study	Unbiased assessment of the study endpoint	Follow-up period appropriate to the aim of the study	Loss to follow-up less than 5%	Prospective calculation of the study size	${}^{\ast }$An adequate control group	${}^{\ast }$Contemporary groups	${}^{\ast }$Baseline equivalence of groups	${}^{\ast }$Adequate statistical analyses	Total
Baghdady MT et al. (2014) [8]	2	2	2	2	2	2	0	0	2	2	2	2	20
Busanello FH et al. (2015) [20]	2	1	2	2	2	2	2	0	2	2	1	2	20
Cruz AD et al. (2014) [21]	2	2	2	2	2	2	2	0	2	2	2	2	22
Ji YA et al. (2018) [24]	2	2	2	2	2	2	0	0	2	2	2	2	20
Kavadella A et al. (2012) [22]	2	0	2	2	1	2	2	0	2	2	2	2	19

^†The items are scored 0 (not reported), 1 (reported but inadequate), or 2 (reported and adequate). The global ideal score is 16 for noncomparative studies and 24 for comparative studies. ${}^{\ast }$For study with control group.

3.4. Synthesis of Results

The studies that received any kind of PBL intervention vs. controls concerning the radiographic interpretation skills were analysed first. A study by Baghdady et al. [8] included 2 separate intervention and control groups which were analysed separately as two different studies. Therefore, on deducing the forest plot for twelve studies [8–10, 12, 20, 21, 23, 25–28], the mean difference in the proficiency score showed a positive difference of 0.54 (0.18, 0.90) with the random effect model based on the heterogeneity value of $I^2$ which was statistically significant favouring the intervention group. Out of the twelve included studies in the forest plot for proficiency scores, nine studies [8, 9, 12, 20, 23, 25–28] showed positive mean difference values (Figure 4). A low risk for publication bias for this meta-analysis was indicated by the funnel plot (Figure 5).

[figure omitted; refer to PDF]

The forest plot in Figure 6 demonstrates a significant difference in mean knowledge score between the PBL group and the TT group in all included studies assessed using the random effect model. The implant mean difference in the knowledge score for 116 and 117 students in the PBL and TT groups, respectively, in the included four studies [20, 22, 24, 25] was 4.15 with a minimum and maximum of -0.35 and 8.65, respectively. The funnel plot for this meta-analysis is displayed in Figure 7.

[figure omitted; refer to PDF]

Figure 8 illustrates a forest plot showing a significant difference in satisfaction level favouring the TT group compared to the PBL group in the two included studies [22, 24] assessed using the random effect model. One study by Ji et al. [24] reported a negative change in the mean difference score of satisfaction level. Hence, the overall mean difference in the satisfaction level score showed a negative difference of -0.14 with a minimum and maximum of -0.36 and 0.08, respectively. Figure 9 demonstrates a funnel plot indicating a low risk for publication bias for this meta-analysis.

[figure omitted; refer to PDF][figure omitted; refer to PDF]

4. Discussion

Through this systematic review and meta-analysis, the null hypothesis was rejected, thus indicating that there is a difference in PBL and traditional learning approach in improving acquisition of radiographic interpretation skills among dental students. Traditional lectures are still a common instructional method of delivering information verbally and are mainly a one-way method of communication that does not involve significant students’ participation but relies upon passive learning. Lectures are useful in transmitting core knowledge and concepts especially to a large audience, but due to their nontransactional nature, they do not assess learning, offer varied perspectives, differentiate instruction, or allow students to self-direct [29]. But, the evolving methods of student learning necessitate the evolution of teaching methods [30]. The problem-based learning approach is an active learning method that fosters a variety of skills like teamwork, information finding, discussions, explanation of new information, and decision making among the students [29].

Dental radiology clinical practice mainly consists of radiograph acquisition and image interpretation centred on real clinical cases [24]. Varying approaches adopting the PBL objectives have been implemented in teaching oral radiology and are compared to the passive, teacher-centred traditional teaching approaches.

This systematic review and meta-analysis revealed the different kinds of problem-based learning approaches, which were mainly short-term interventions [8–10, 12, 20–25, 27, 28], except for one study [26] which was found to extend for 8 months. This review selected interventions targeted towards teaching oral radiology wherein all the students already had the basic theoretical knowledge about the principles of radiographic interpretation and radiological features and differential diagnosis [8–10, 12, 20–28]. To run any program of robust nature, funding plays an important role. Of the 13 studies included in the review, only four studies had international funding and collaboration [25–28], while the remaining nine studies were self-funded [25–28]. Ethical approval of a research project also helps to increase the legitimacy of research findings as well as plays a vital role in decision making based on the research results. Ten included studies in this review have mentioned about taking institutional ethical approval before the start of their study [8–10, 12, 20, 23, 25–28].

In the present systematic review, the educational effectiveness of the included studies [8–10, 12, 20–28] was assessed based on (i) the students’ radiographic interpretation skills through mean proficiency and diagnostic accuracy scores, (ii) the students’ performance through the knowledge tests, and

(iii) the level of satisfaction through the PBL methods; except in the study conducted by Ji et al. [24], only surveys on self-awareness of competency were conducted, and the students’ achievement in terms of their true competency were not evaluated. Validity refers to how accurately a method measures what it is intended to measure. To obtain useful results, the methods and instruments used to collect data must be valid which ensures that the discussion of the data and the conclusions drawn are also valid. Twelve out of thirteen studies have tested the validity of the instrument used for assessing the outcomes [ 8–10, 12, 20–23, 25–28]. Without a doubt, all the included studies [8–10, 12, 20–28] have successfully accomplished their study objectives.

For the radiographic interpretation skills, twelve studies [8–10, 12, 20, 21, 23, 25–28] were included in the meta-analysis showing a significant difference favouring experimental groups. The introduction of new problem-based learning methods such as structured algorithm [8], syndicate learning [9], OMP [12], web-based learning [21], simulation [26–28], and virtual-based learning [10, 20, 23, 25] effectively facilitated student’s exploration and self-study along with improved critical thinking as compared to traditional didactic. Also, the absence or passive role of tutors in the groups enabled students to take more responsibility for their own learning which was translated into improved radiographic interpretation skills.

For the knowledge score outcomes, four studies [20, 22, 24, 25] were included in the meta-analysis showing a significant difference favouring experimental groups. This improvement might be because of implementation of active learning methods that organized and systematized knowledge over memorization-based education, thus highlighting the importance of nonlinear education; encouraging students to favour self-directed, engaging learning; and confirming their clinical reasoning and problem-solving through schema-based assignments [24], blended learning [22], digital object learning [20], and virtual learning [25]. Also, a higher level of satisfaction was observed among the learners with improved learning because of easy and greater access to educational content via a virtual learning environment.

For the satisfaction level outcomes, two studies [22, 24] were included in the quantitative synthesis showing a significant difference favouring the traditional teaching group, which was controversial to other studies which were not included in this meta-analysis due to inappropriate outcome measures. The study conducted by Ji et al. [24] reported less satisfaction with respect to their interest in participation in schema-based learning as compared to those who received traditional training; this was because the students were unaccustomed to smartphone-based training, and the lack of immediate feedback by the professor made them lose interest in schema-based learning. In the study conducted by Kavadella et al. [22], it was reported that the students felt that blended learning was more demanding and was much more work than conventional training.

This systematic review included both randomized [9, 10, 12, 23, 25–28] and nonrandomized controlled trials showing heterogeneity in the included studies. Only four studies [9, 21, 24, 27] showed low risk of bias, whereas per AHRQ for randomized controlled trial, only two studies [9, 27] out of eight showed good quality, while five studies [10, 12, 25, 26, 28] were of poor quality due to inadequate random sequence generation, allocation concealment, and blinding of participants and personnel. In all the RCTs included [9, 10, 12, 25–28], except for one by Howerton et al. [23], the outcome assessment was done using objective assessment tools where the investigator had a passive role; hence, blinding of outcome assessment showed low risk of bias. However, in majority of the studies included, the sample size was not estimated leading to small sample sizes and decreased power of study. Also, a loss to follow-up of more than 5% was observed in four studies [8, 23, 24, 26] with 21% being the maximum. In publication bias, all of the funnel plots presented represent smaller studies with no beneficial effects. The methodological quality of majority of the studies is low except for few studies reporting radiographic interpretation skills which indicate absence of bias with replicable methodology (Figures 5, 7, and 9).

The present review has some limitations. It was not possible to fully avoid the clinical heterogeneity among the included studies. The sample size of the studies was small, thus lacking statistical power. Furthermore, the studies were conducted while the students were unused to the new method, which exerted a significant influence on their satisfaction levels. Also, there is a need for more trained staff for the tutoring process and learning the new methods of PBL. Students must relearn how to learn and teachers must relearn how to teach [4, 24]. Individual teaching method-wise analysis was not taken into consideration in the analysis. The subgroup and sensitivity analyses could have been performed to rule out the potential reasons for heterogeneity; however, this was not possible as there were small number of studies included under similar outcomes. There is no long-term evidence on the effectiveness of these interventions in oral radiology in improving radiographic interpretation skills, knowledge scores, and satisfaction levels among dental students. This may be due to the variability of PBL methods deployed in the individual studies.

In order to understand whether or not PBL methods are superior to traditional teaching methods in oral radiology among dental students, well-designed long-term randomized controlled trials are needed. This study was planned with a systematic review and a meta-analysis as these are both highly effective at analysing studies conducted on similar topics [31, 32]. Systematic reviews are efficient in quality evaluation of studies, while a meta-analysis is an objective method to carry out statistical analysis of various studies depending on their quality.

5. Conclusion

In conclusion, there was a difference in the evidence gathered showing that there is a difference between PBL and TL approaches and indicating that problem-based learning methods were effective in improving radiographic interpretation skills and knowledge scores over a short period. PBL was not effective for improving the satisfaction level among the students where the findings were conflicting. However, PBL fostered activation of prior learning, high motivation to learn, and the development of self-directed learning skills among the dental students.

References

[1] ADEA Commission on Change and Innovation in Dental Education, W. D. Hendricson, S. C. Andrieu, D. G. Chadwick, J. E. Chmar, J. R. Cole, M. C. George, G. N. Glickman, J. F. Glover, J. S. Goldberg, N. K. Haden, C. Meyerowitz, L. Neumann, M. Pyle, L. A. Tedesco, R. W. Valachovic, R. G. Weaver, R. L. Winder, S. K. Young, K. L. Kalkwarf, "Educational strategies associated with development of problem-solving, critical thinking, and self-directed learning," Journal of Dental Education, vol. 70 no. 9, pp. 925-936, DOI: 10.1002/j.0022-0337.2006.70.9.tb04163.x, 2006.

[2] A. Alrahlah, "How effective the problem-based learning (PBL) in dental education. A critical review," The Saudi dental journal, vol. 28 no. 4, pp. 155-161, DOI: 10.1016/j.sdentj.2016.08.003, 2016.

[3] W. D. Hendricson, J. H. Kleffner, "Assessing and helping challenging students: Part One, why do some students have difficulty learning?," Journal of Dental Education, vol. 66 no. 1, pp. 43-61, DOI: 10.1002/J.0022-0337.2002.66.1.TB03507.X, 2002.

[4] M. Binshabaib, M. Shalabi, S. Mansour, L. Alageel, N. Alsuwidan, S. S. ALHarthi, "Role of problem-based learning in undergraduate dental education: a questionnaire-based study," Advances in Medical Education and Practice, vol. 10, pp. 79-84, DOI: 10.2147/AMEP.S181658, 2019.

[5] S. H. Bassir, P. Sadr-Eshkevari, S. Amirikhorheh, N. Y. Karimbux, "Problem-based learning in dental education: a systematic review of the literature," Journal of Dental Education, vol. 78 no. 1, pp. 98-109, DOI: 10.1002/j.0022-0337.2014.78.1.tb05661.x, 2014.

[6] H. S. Barrows, "Problem-based learning in medicine and beyond: a brief overview," New Directions for Teaching and Learning, vol. 1996 no. 68,DOI: 10.1002/tl.37219966804, 1996.

[7] V. Kumar, C. C. Gadbury-Amyot, "A case-based and Team-Based learning model in oral and maxillofacial radiology," Journal of Dental Education, vol. 76 no. 3, pp. 330-337, DOI: 10.1002/j.0022-0337.2012.76.3.tb05262.x, 2012.

[8] M. T. Baghdady, H. Carnahan, E. W. Lam, N. N. Woods, "Dental and dental hygiene students’ diagnostic accuracy in oral radiology: effect of diagnostic strategy and instructional method," Journal of Dental Education, vol. 78 no. 9, pp. 1279-1285, DOI: 10.1002/j.0022-0337.2014.78.9.tb05799.x, 2014.

[9] V. K. Lohe, A. Singh, "Evaluation of effectiveness of syndicate learning in oral medicine and radiology: cardinal findings in a batch of final-year students," Astrocyte, vol. 1 no. 4, pp. 301-304, DOI: 10.4103/2349-0977.161625, 2015.

[10] J. Vuchkova, T. Maybury, C. S. Farah, "Digital interactive learning of oral radiographic anatomy," European Journal of Dental Education, vol. 16 no. 1, pp. e79-e87, DOI: 10.1111/j.1600-0579.2011.00679.x, 2012.

[11] H. Morgan, K. McLean, C. Chapman, J. Fitzgerald, A. Yousuf, M. Hammoud, "The flipped classroom for medical students," The Clinical Teacher, vol. 12 no. 3, pp. 155-160, DOI: 10.1111/tct.12328, 2015.

[12] Z. Naik, M. Umarani, "and #8216;One minute Preceptor and #8217; a teaching-learning model for oral radiographic interpretation skill," Journal of Contemporary Medical Education, vol. 3 no. 2, pp. 72-76, DOI: 10.5455/jcme.20150621104618, 2015.

[13] T. Nilsson, J. Ahlqvist, M. Johansson, A. Isberg, "Virtual reality for simulation of radiographic projections: validation of projection geometry," Dentomaxillofacial Radiology, vol. 33 no. 1, pp. 44-50, DOI: 10.1259/dmfr/22722586, 2004.

[14] S. Meckfessel, C. Stühmer, K. H. Bormann, T. Kupka, M. Behrends, H. Matthies, B. Vaske, M. Stiesch, N. C. Gellrich, M. Rücker, "Introduction of e-learning in dental radiology reveals significantly improved results in final examination," Journal of Cranio-Maxillofacial Surgery, vol. 39 no. 1, pp. 40-48, DOI: 10.1016/j.jcms.2010.03.008, 2011.

[15] The Joanna Briggs Institute Levels of Evidence and Grades of Recommendation Working Party, Supporting Document for the Joanna Briggs Institute Levels of Evidence and Grades of Recommendation, 2014.

[16] J. P. Higgins, J. Thomas, J. Chandler, M. Cumpston, T. Li, M. J. Page, V. A. Welch, Cochrane Handbook for Systematic Reviews of Interventions, 2019.

[17] M. Viswanathan, M. T. Ansari, N. D. Berkman, S. Chang, L. Hartling, L. M. McPheeters, P. L. Santaguida, T. Shamliyan, K. Singh, A. Tsertsvadze, J. R. Treadwell, Assessing the risk of bias of individual studies in systematic reviews of health care interventions, 2012.

[18] K. Slim, E. Nini, D. Forestier, F. Kwiatkowski, Y. Panis, J. Chipponi, "Methodological index for non-randomized studies ( MINORS ): development and validation of a new instrument," ANZ journal of surgery, vol. 73 no. 9, pp. 712-716, DOI: 10.1046/j.1445-2197.2003.02748.x, 2003.

[19] J. A. C. Sterne, M. A. Hernán, B. C. Reeves, J. Savović, N. D. Berkman, M. Viswanathan, D. Henry, D. G. Altman, M. T. Ansari, I. Boutron, J. R. Carpenter, A. W. Chan, R. Churchill, J. J. Deeks, A. Hróbjartsson, J. Kirkham, P. Jüni, Y. K. Loke, T. D. Pigott, C. R. Ramsay, D. Regidor, H. R. Rothstein, L. Sandhu, P. L. Santaguida, H. J. Schünemann, B. Shea, I. Shrier, P. Tugwell, L. Turner, J. C. Valentine, H. Waddington, E. Waters, G. A. Wells, P. F. Whiting, J. P. T. Higgins, "ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions," BMJ, vol. 355, article i4919,DOI: 10.1136/bmj.i4919, 2016.

[20] F. H. Busanello, P. F. da Silveira, G. S. Liedke, N. A. Arús, M. B. Vizzotto, H. E. Silveira, H. L. Silveira, "Evaluation of a digital learning object (DLO) to support the learning process in radiographic dental diagnosis," European Journal of Dental Education, vol. 19 no. 4, pp. 222-228, DOI: 10.1111/eje.12125, 2015.

[21] A. D. Cruz, J. J. Costa, S. M. Almeida, "Distance learning in dental radiology: immediate impact of the implementation," Brazilian Dental Science, vol. 17 no. 4, pp. 90-97, DOI: 10.14295/bds.2014.v17i4.930, 2014.

[22] A. Kavadella, K. Tsiklakis, G. Vougiouklakis, A. Lionarakis, "Evaluation of a blended learning course for teaching oral radiology to undergraduate dental students," European Journal of Dental Education, vol. 16 no. 1, pp. e88-e95, DOI: 10.1111/j.1600-0579.2011.00680.x, 2012.

[23] W. B. Howerton, E. Platin, J. Ludlow, D. A. Tyndall, "The influence of computer-assisted instruction on acquiring early skills in intraoral radiography," Journal of Dental Education, vol. 66 no. 10, pp. 1154-1158, DOI: 10.1002/j.0022-0337.2002.66.10.tb03587.x, 2002.

[24] Y. A. Ji, Y. M. Lee, H. D. Lim, W. J. Park, J. H. Jung, J. W. Lee, B. D. Lee, "Smartphone use and schema-based learning in dentomaxillofacial radiology practice: a case report from one College of Dentistry," Dentomaxillofacial Radiology, vol. 47 no. 6,DOI: 10.1259/dmfr.20170463, 2018.

[25] E. Soltanimehr, E. Bahrampour, M. M. Imani, F. Rahimi, B. Almasi, M. Moattari, "Effect of virtual versus traditional education on theoretical knowledge and reporting skills of dental students in radiographic interpretation of bony lesions of the jaw," BMC Medical Education, vol. 19 no. 1,DOI: 10.1186/s12909-019-1649-0, 2019.

[26] T. A. Nilsson, L. R. Hedman, J. B. Ahlqvist, "Dental student skill retention eight months after simulator-supported training in oral radiology," Journal of Dental Education, vol. 75 no. 5, pp. 679-684, DOI: 10.1002/j.0022-0337.2011.75.5.tb05094.x, 2011.

[27] T. A. Nilsson, L. R. Hedman, J. B. Ahlqvist, "A randomized trial of simulation-based versus conventional training of dental student skill at interpreting spatial information in radiographs," Simulation in Healthcare, vol. 2 no. 3, pp. 164-169, DOI: 10.1097/SIH.0b013e31811ec254, 2007.

[28] T. Söderström, L. Häll, T. Nilsson, J. Ahlqvist, "How does collaborative 3D screen-based computer simulation training influence diagnostic skills of radiographic images and peer communication?," Contemporary Educational Technology, vol. 3 no. 4, pp. 293-307, DOI: 10.30935/cedtech/6085, 2012.

[29] P. Cantillon, "Teaching large groups," BMJ, vol. 326 no. 7386, pp. 437-440, DOI: 10.1136/bmj.326.7386.437, 2003.

[30] A. Ramesh, R. Ganguly, "Interactive learning in oral and maxillofacial radiology," Imaging science in dentistry, vol. 46 no. 3, pp. 211-216, DOI: 10.5624/isd.2016.46.3.211, 2016.

[31] E. Ahn, H. Kang, "Introduction to systematic review and meta-analysis," Korean Journal of Anesthesiology, vol. 71 no. 2, pp. 103-112, DOI: 10.4097/kjae.2018.71.2.103, 2018.

[32] K. B. Bhor, V. Vinay, K. Ambildhok, V. Shetty, "Effectiveness of oral health educational interventions on oral health of visually impaired school children: a systematic review and meta-analysis," Special Care in Dentistry, vol. 41 no. 3, pp. 291-308, DOI: 10.1111/scd.12567, 2021.