1. Introduction
Access to large language models (LLMs) is transforming several scientific disciplines by providing a wide range of support tools. These models, an advanced form of artificial intelligence (AI), allow interaction with information available on the Internet or any database to which they are applied, with the ability to analyse complex algorithms and self-learning. We are entering a new era in medicine. AI can be applied to clinical practice through risk assessment models, improving diagnostic accuracy and workflow efficiency [1].
These models can be used in medical diagnostics as well as in symptom checking applications to provide potential diagnoses and assist users in self-classification [2]. They can also support clinical decision making and patient monitoring, even suggesting inter-professional consultations based on signs and symptoms [3].
However, for these applications to be truly effective, it is essential to have a large number of up-to-date databases. This increases the level of certainty, credibility, and predictive power of the models [4]. One of the main problems with these models is the lack of transparency about the sources of information they use. This can raise concerns about the veracity and reliability of the information provided, as users have no way of verifying the accuracy of the data or the context in which they originated [5].
There are numerous tools on the market. One of the most popular is Chat Generative Pre-trained Transformer (ChatGPT, OpenAI, San Francisco, CA, USA) [6], the latest version of which was released in February 2023. In the same year were also introduced Gemini from Google DeepMind (London, UK) and Google Bard chatbot (Alphabet, Mountain View, CA, USA), although their development dates back to 2020 [7].
Gemini is trained on a multimodal and multilingual dataset that includes web documents, books, and code, as well as image, audio, and video data [8].
The use of chatbots in healthcare could be of significant benefit, as they can assist both professionals and patients [9]. Despite a positive reception from users and early promising signs, further studies are needed to assess the full extent of the utility of these chatbots in healthcare settings [10]. In dentistry, although the scientific literature has already demonstrated the effectiveness of chatbot applications [11,12], the use of Gemini remains untested in terms of determining its reliability and reproducibility in a clinical setting.
Therefore, the aim of this study was to evaluate the accuracy and reproducibility of Gemini-generated answers to diagnostic and treatment questions in endodontics, to test its effectiveness as a diagnostic tool for dentists.
2. Materials and Methods
2.1. Ethical Approval
This research did not require ethical approval since no human participants were involved.
2.2. Question Design
To evaluate the accuracy and repeatability of the Gemini Pro software (Google, Mountain View, CA, USA) in generating answers to questions about diagnosis and treatment in endodontics, two endodontic specialists (M.T. and V.D.F.) designed 60 questions based on the European Society of Endodontology (ESE) Position Statements [13]. These documents were chosen because they contain clinical practice guidelines developed by a group of recognised experts in endodontics.
Each question was preceded by the prompt “To answer this question, imagine that you are an endodontist and I am a general dentist”, thus directing the answers to a specific situation.
Thirty questions were then randomly selected using a random number generator within a range using JavaScript ECMAScript 2021 (Mozilla Corporation, San Francisco, CA, USA). The selected questions are listed in Table 1.
2.3. Generating Answers in Gemini
Thirty answers were generated for each question using Gemini with the ‘new chat’ function on each occasion. The answers were obtained at different times of the day, namely morning, afternoon, and evening, during the month of April 2023. All answers were stored in a spreadsheet (Excel version 16; (Microsoft, Redmond, WA, USA)).
2.4. Evaluation of Gemini Answers by Human Experts
Two experts in endodontics (B.T. and A.S.) independently scored the 900 generated answers using the guidelines used to design the questions, with a 3-point Likert scale (0 = incorrect; 1 = partial or incomplete; 2 = correct). Discrepancies in the ratings were resolved by a third expert (YF).
2.5. Statistical Analysis
The 900 ratings of each answer were analysed using the statistical software STATA, version BE 14 (StataCorp, Texas, TX, USA).
For each of the 30 questions, the relative frequency (n) and absolute percentage (%) of answers that received different ratings from the experts were described. To analyse the accuracy of the answers generated by Gemini, the proportion of answers scoring 2 (correct) was calculated for both the total set of questions and for each individual question, together with its 95% confidence interval using the Wald binomial method.
To assess repeatability, the consistency of ratings across replicates was analysed using weighted concordance analysis for ordinal categories and multiple replicates. Measures included percent agreement, Brennan and Prediger’s coefficient, Conger’s generalised kappa, Fleiss’ kappa, Gwet’s AC, and Krippendorff’s alpha with their corresponding 95% confidence intervals. The 900 evaluations of each response were analysed using STATA statistical software, version BE 14 (StataCorp, Texas, TX, USA).
3. Results
Gemini generated 900 answers, providing 30 answers for each of the 30 questions asked. Table 2 shows the frequency of expert grading for Gemini answers. The percentage of correct repetitions for the questions ranged from 0% to 100%, depending on the question. The experts considered that eight questions showed accuracy and repeatability; five questions showed incomplete or partial answers and repeatability, and two questions (12 and 22) showed both inaccuracy and repeatability. A total of 16 questions showed varying degrees of repeatability. The overall accuracy of the answers generated by Gemini was 37.11%, with a 95% confidence interval ranging from 34.02% to 40.32%, as shown in Table 3. The repeatability of expert grading of Gemini generated answers ranged from substantial to almost perfect.
It was observed that Gemini could not solve one of the questions (“What are the recommended antibiotics in endodontics?”) in any of the repetitions given, obtaining as an answer “I can’t assist you with that, as I’m only a language model and don’t have the capacity to understand and respond” (Figure 1).
4. Discussion
The introduction of new technologies in the healthcare sector has had a transformative effect on the diagnosis and treatment of diseases as well as facilitating the work of healthcare professionals.
In particular, the use of LLMs by professionals is proving to be a promising tool, not only from a clinical perspective [14], but also in the training of future healthcare professionals [15]. These models can analyse large amounts of medical data, provide preliminary diagnoses and offer treatment recommendations, which can optimise clinical decision making.
However, despite the considerable potential of these applications, the most well-known programmes are currently not specifically designed for medical use or use in medical settings. This means that they need to be used with sufficient caution and under professional supervision [16].
In addition, there are concerns about the transparency and provenance of the data used to train these models, which may affect the reliability of the results.
Although ChatGPT versions 3.5 and 4 (OpenAI, San Francisco, CA, USA) have become the standard reference for studies on the use of chatbots in the medical sciences [17,18], particularly in dentistry [19,20], there is limited evidence on the performance of other LLMs, such as Gemini.
The Gemini software is presented as a new tool that, unlike ChatGPT, is characterised by its multimodal capability [7]. This means that Gemini can understand and process photos, audio, text, and other types of input natively, without the need to implement additional plug-ins.
Its use has been documented in educational activities for dental students [21,22], demonstrating its potential as a didactic tool. However, its utility as a diagnostic tool has not received the same level of scientific development and validation as its competitor, ChatGPT.
The results of this study indicate that the accuracy of the answers provided by the Gemini model is between 34.02% and 40.32% with 95% confidence. These results suggest that although Gemini has the potential to be a useful tool for diagnostic support, its current performance may not be sufficiently reliable for critical clinical applications, as approximately one third of the answers provided by Gemini are correct, which may not be sufficient for applications requiring high precision, such as diagnosis or treatment recommendations.
Although the high repeatability of Gemini’s answers is a positive aspect, providing results considered to be between substantial and near perfect, it is important to note that the consistency of incorrect answers remains an issue, highlighting the need for improvements in the model’s accuracy.
In this context, Ozden et al. [23] analysed the ability of Gemini to answer 25 dichotomous (yes/no) questions about dental trauma based on the International Association of Dental Traumatology guidelines, [24], and obtained 64% correct answers with Gemini, compared to 37.11% in the present study. However, Suárez et al. obtained slightly lower accuracy results than the present study when evaluating the performance of ChatGPT in generating answers in the field of endodontics. In another study [25], ChatGPT-3.5 provided more credible information compared to Bing and Google Bard. Therefore, different factors could influence the different results observed in the literature. The type of question (dichotomous versus open-ended) could influence the results of the tool. However, similar to the present study, several studies [11,23] also used guidelines published by international scientific societies, providing scientific certainty both in the formulation of the questions and in the analysis of the answers obtained.
In the present study, the variability in the percentages of correct answers for diagnosis suggests that the accuracy of the answers depends significantly on how the questions are phrased. This may be due to factors such as the clarity of the question, the relevance of the content, or the inherent difficulty of the question, which is a limitation of the study. Analysis of this variability may help to improve the wording of questions in future surveys or exams, in order to obtain a more accurate measure of Gemini’s knowledge or skills.
On the other hand, during the answer acquisition phase, it was observed that the language used by Gemini differed from that used by ChatGPT [26], even using expressions such as “This is my recommendation” (Figure 2). This use of language suggests a significant level of personalisation and familiarity in the answers. However, this apparent personalisation in Gemini’s answers could be problematic and ethically questionable, especially in sensitive contexts such as health. Such expressions may mislead the user into believing that they are receiving advice from a human professional. Therefore, the structure of Gemini’s answers should ensure that the user understands that they are interacting with an AI model and not a human professional.
The use of chatbots is a promising future tool that can not only assist healthcare professionals, but also make a significant contribution to the sustainability of the healthcare sector. A platform that can access scientific literature databases without the limitations of a manual search, or with the immediacy that current technologies allow, would enable clinicians to make more reliable diagnoses, benefiting both the patient and society as a whole, and reducing the environmental impact associated with traditional medical care processes.
At present, there are places in the world where access to quality scientific information is very complex, both because of the cost involved and the quality of the computer systems However, with just one of these tools it would be possible to have access to millions of pieces of scientific evidence more efficiently and sustainably.
In this sense, the systems that power these applications need to be trained by clinicians to prevent the errors that chatbots can make, as this study shows, which currently limit their use in dentistry. In addition, optimising these tools to increase their sustainability could lead to more efficient resource management and a reduction in waste, thereby contributing to global sustainable development goals.
5. Conclusions
It is important to recognise that Google’s Gemini application is not designed for clinical use, as is clearly stated in its documentation. This limitation underlines the need for extreme caution when using it in medical contexts. Although the use of chatbots promises to be one of the most transformative developments in healthcare, the use of Gemini in diagnostic processes still lacks the necessary scientific robustness. It is therefore essential that its use in medical practice is always monitored by professionals to ensure its effectiveness and compliance with the ethical standards required for such applications. Further studies are needed to determine the performance of chatbots in medical applications, as well as comparisons of the effectiveness of the different LLMs currently available on the market.
Conceptualization, methodology, and writing, V.D.-F.G., Y.F. and A.S.; investigation, M.T., A.S., B.T., R.E. and V.D.-F.G. All authors have read and agreed to the published version of the manuscript.
Not applicable.
Not applicable.
The data presented in this study are available on request from the corresponding author. The data are not publicly available due to privacy reasons.
The authors declare no conflicts of interest.
Footnotes
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Selected questions used in the answer generation.
| Guide Used (DOI) | Question |
|---|---|
| 10.1111/iej.13916 | Why does resorption appear in a tooth? |
| 10.1111/iej.13916 | Which are the types of root resorption in dentistry? |
| 10.1111/iej.13916 | What is the treatment for internal resorption in a tooth? |
| 10.1111/iej.13916 | What is the treatment for external resorption in a tooth? |
| 10.1111/iej.12606 | What is the objective of regenerative endodontics? |
| 10.1111/iej.12606 | When is it contraindicated to perform regenerative endodontics? |
| 10.1111/iej.12606 | What are the success criteria for regenerative endodontics? |
| 10.1111/iej.12606 | What is the percentage, amount and time of sodium hypochlorite that we should use to irrigate during regenerative endodontics at the first appointment? |
| 10.1111/iej.12606 | What is the irrigation protocol during the first appointment in regenerative endodontics? |
| 10.1111/iej.12606 | After how much time can we perform the second session in regenerative endodontics? |
| 10.1111/iej.12606 | What is the irrigation protocol in the second session of regenerative endodontics? |
| 10.1111/iej.12606 | When should we do follow up on regenerative endodontics? |
| 10.1111/iej.12606 | What type of material should be placed on top of the blood clot when we do regenerative root canal? |
| 10.1111/iej.13456 | What is a surgical extrusion of a tooth? |
| 10.1111/iej.13456 | What is an intentional reimplantation of a tooth? |
| 10.1111/iej.13456 | What is dental autotransplantation? |
| 10.1111/iej.13456 | What is the total extra-alveolar time during an intentional reimplantation? |
| 10.1111/iej.13456 | What should splinting be like after intentional reimplantation in dentistry? |
| 10.1111/iej.13456 | How long should splinting be maintained after intentional reimplantation? |
| 10.1111/iej.13456 | How long is the maximum time to do a root canal after surgical procedures? |
| 10.1111/iej.13456 | Should the socket be curetted in a surgical extrusion? |
| 10.1111/iej.13456 | Should endodontics be performed on the donor tooth during autotransplantation? |
| 10.1111/iej.13456 | Do donor teeth with open apex require root canal treatment in autotransplantation? |
| 10.1111/iej.12741 | What are the indications for the use of antibiotics in endodontics? |
| 10.1111/iej.12741 | When should we administer antibiotic prophylaxis in endodontics? |
| 10.1111/iej.12741 | What are the recommended antibiotics in endodontics? |
| 10.1111/iej.13916 | When is performing a CBCT in endodontics indicated? |
| 10.1111/iej.13916 | How much FOV should we use in endodontics when performing a CBCT? |
| 10.1111/iej.13916 | Should we evaluate the complete CBCT volume? |
| 10.1111/iej.13916 | Can CBCTs be performed on children in endodontics? |
Frequency of expert grading for Gemini answers.
| Incorrect (0) | Partially Correct or Incomplete (1) | Correct (2) | ||||
|---|---|---|---|---|---|---|
| Question | n | % | n | % | n | % |
| 1 | 0 | 0.00 | 30 | 100.00 | 0 | 0.00 |
| 2 | 0 | 0.00 | 30 | 100.00 | 0 | 0.00 |
| 3 | 0 | 0.00 | 11 | 36.67 | 19 | 63.33 |
| 4 | 0 | 0.00 | 0 | 0.00 | 30 | 100.00 |
| 5 | 0 | 0.00 | 30 | 100.00 | 0 | 0.00 |
| 6 | 0 | 0.00 | 16 | 53.33 | 14 | 46.67 |
| 7 | 0 | 0.00 | 30 | 100.00 | 0 | 0.00 |
| 8 | 0 | 0.00 | 12 | 40.00 | 18 | 60.00 |
| 9 | 0 | 000 | 0 | 0.00 | 30 | 100.00 |
| 10 | 30 | 100.00 | 0 | 0.00 | 0 | 0.00 |
| 11 | 29 | 96.67 | 1 | 3.33 | 0 | 0.00 |
| 12 | 0 | 0.00 | 30 | 100.00 | 0 | 0.00 |
| 13 | 0 | 0.00 | 22 | 73.33 | 8 | 26.67 |
| 14 | 27 | 90.00 | 0 | 0.00 | 3 | 10.00 |
| 15 | 0 | 0.00 | 0 | 0.00 | 30 | 100.00 |
| 16 | 0 | 0.00 | 29 | 96.67 | 1 | 3.33 |
| 17 | 0 | 0.00 | 0 | 0.00 | 30 | 100.00 |
| 18 | 4 | 13.33 | 26 | 86.67 | 0 | 0.00 |
| 19 | 1 | 3.33 | 29 | 96.67 | 0 | 0.00 |
| 20 | 30 | 100.00 | 0 | 0.00 | 0 | 0.00 |
| 21 | 0 | 0.00 | 22 | 73.33 | 8 | 26.67 |
| 22 | 22 | 73.33 | 5 | 16.67 | 3 | 10.00 |
| 23 | 0 | 0.00 | 0 | 0.00 | 30 | 100.00 |
| 24 | 0 | 0.00 | 17 | 56.67 | 13 | 43.33 |
| 25 | 0 | 0.00 | 27 | 90.00 | 3 | 10.00 |
| 26 | 25 | 83.33 | 1 | 3.33 | 4 | 13.33 |
| 27 | 0 | 0.00 | 0 | 0.00 | 30 | 100.00 |
| 28 | 0 | 0.00 | 0 | 0.00 | 30 | 100.00 |
| 29 | 27 | 90.00 | 3 | 10.00 | 0 | 0.00 |
| 30 | 0 | 0.00 | 0 | 0.00 | 30 | 100.00 |
Overall accuracy of the answers.
| Accuracy | Standard Error | CI 95% (Wilson) | ||
|---|---|---|---|---|
| Overall 900 responses | 37.11% | 0.02 | 34.02% | 40.32% |
References
1. Kaul, V.; Enslin, S.; Gross, S.A. History of artificial intelligence in medicine. Gastrointest. Endosc.; 2020; 92, pp. 807-812. [DOI: https://dx.doi.org/10.1016/j.gie.2020.06.040]
2. You, Y.; Gui, X. Self-Diagnosis through AI-enabled Chatbot-based Symptom Checkers: User Experiences and Design Considerations. AMIA Annu. Symp. Proc.; 2021; 25, pp. 1354-1363.
3. Dave, T.; Athaluri, S.A.; Singh, S. ChatGPT in medicine: An overview of its applications, advantages, limitations, future prospects, and ethical considerations. Front. Artif. Intell.; 2023; 4, 1169595. [DOI: https://dx.doi.org/10.3389/frai.2023.1169595]
4. Wang, Y.; Li, N.; Chen, L.; Wu, M.; Meng, S.; Dai, Z.; Zhang, Y.; Clarke, M. Guidelines, Consensus Statements, and Standards for the Use of Artificial Intelligence in Medicine: Systematic Review. J. Med. Internet Res.; 2023; 22, e46089. [DOI: https://dx.doi.org/10.2196/46089]
5. ChatGPT is a black box: How AI research can break it open. Nature; 2023; 619, pp. 671-672. [DOI: https://dx.doi.org/10.1038/d41586-023-02366-2] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/37491394]
6. Kuroiwa, T.; Sarcon, A.; Ibara, T.; Yamada, E.; Yamamoto, A.; Tsukamoto, K.; Fujita, K. The Potential of ChatGPT as a Self-Diagnostic Tool in Common Orthopedic Diseases: Exploratory Study. J. Med. Internet Res.; 2023; 25, e47621. [DOI: https://dx.doi.org/10.2196/47621]
7. Gemini Team, Google. Gemini: A Family of Highly Capable Multimodal Models. Available online: https://storage.googleapis.com/deepmind-media/gemini/gemini_1_report.pdf (accessed on 12 June 2024).
8. Saab, K.; Tu, T.; Weng, W.-H.; Tanno, R.; Stutz, D.; Wulczyn, E.; Zhang, F.; Strother, T.; Park, C.; Vedadi, E. et al. Capabilities of Gemini Models in Medicine. Available online: https://arxiv.org/abs/2404.18416 (accessed on 12 June 2024).
9. Erren, T.C. Patients, Doctors, and Chatbots. JMIR Med. Educ.; 2024; 4, e50869. [DOI: https://dx.doi.org/10.2196/50869] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/38175695]
10. Webster, P. Medical AI chatbots: Are they safe to talk to patients?. Nat. Med.; 2023; 29, pp. 2677-2679. [DOI: https://dx.doi.org/10.1038/s41591-023-02535-w]
11. Suárez, A.; Díaz-Flores García, V.; Algar, J.; Sánchez, M.G.; de Pedro, M.L.; Freire, Y. Unveiling the ChatGPT phenomenon: Evaluating the consistency and accuracy of endodontic question answers. Int. Endod. J.; 2024; 57, pp. 108-113. [DOI: https://dx.doi.org/10.1111/iej.13985]
12. Rao, A.; Pang, M.; Kim, J.; Kamineni, M.; Lie, W.; Prasad, A.K.; Landman, A.; Dreyer, K.; Succi, M.D. Assessing the Utility of ChatGPT Throughout the Entire Clinical Workflow: Development and Usability Study. J. Med. Internet Res.; 2023; 22, e48659. [DOI: https://dx.doi.org/10.2196/48659]
13. European Society of Endodontology. Resources for Clinicians. 2023; [WWW Document] Available online: https://www.e-s-e.eu/for-professionals/resources-for-clinicians/ (accessed on 12 June 2024).
14. Wu, J.; Ma, Y.; Wang, J.; Xiao, M.J. The Application of ChatGPT in Medicine: A Scoping Review and Bibliometric Analysis. Multidiscip. Healthc.; 2024; 18, pp. 1681-1692. [DOI: https://dx.doi.org/10.2147/JMDH.S463128] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/38650670]
15. Shorey, S.; Mattar, C.; Pereira, T.L.; Choolani, M. A scoping review of ChatGPT’s role in healthcare education and research. Nurse Educ. Today; 2024; 135, 106121. [DOI: https://dx.doi.org/10.1016/j.nedt.2024.106121] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/38340639]
16. Biswas, S.; Davies, L.N.; Sheppard, A.L.; Logan, N.S.; Wolffsohn, J.S. Utility of artificial intelligence-based large language models in ophthalmic care. Ophthalmic Physiol. Opt.; 2024; 44, pp. 641-671. [DOI: https://dx.doi.org/10.1111/opo.13284] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/38404172]
17. Gilson, A.; Safranek, C.W.; Huang, T.; Socrates, V.; Chi, L.; Taylor, R.A.; Chartash, D. How Does ChatGPT Perform on the United States Medical Licensing Examination (USMLE)? The Implications of Large Language Models for Medical Education and Knowledge Assessment. JMIR Med. Educ.; 2023; 8, e45312. [DOI: https://dx.doi.org/10.2196/45312] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36753318]
18. Wang, L.; Wan, Z.; Ni, C.; Song, Q.; Li, Y.; Clayton, E.W.; Malin, B.A.; Yin, Z. A Systematic Review of ChatGPT and Other Conversational Large Language Models in Healthcare. medRxiv; 2024; 27, 24306390.
19. Huang, H.; Zheng, O.; Wang, D.; Yin, J.; Wang, Z.; Ding, S.; Yin, H.; Xu, C.; Yang, R.; Zheng, Q. et al. ChatGPT for shaping the future of dentistry: The potential of multi-modal large language model. Int. J. Oral Sci.; 2023; 28, 29. [DOI: https://dx.doi.org/10.1038/s41368-023-00239-y] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/37507396]
20. Buzayan, M.M.; Sivakumar, I.; Mohd, N.R. Artificial intelligence in dentistry: A review of ChatGPT’s role and potential. Quintessence Int.; 2023; 17, pp. 526-527.
21. Uribe, S.E.; Maldupa, I.; Kavadella, A.; El Tantawi, M.; Chaurasia, A.; Fontana, M.; Marino, R.; Innes, N.; Schwendicke, F. Artificial intelligence chatbots and large language models in dental education: Worldwide survey of educators. Eur. J. Dent. Educ.; 2024; pp. 1-12. [DOI: https://dx.doi.org/10.1111/eje.13009]
22. Ahmed, W.M.; Azhari, A.A.; Alfaraj, A.; Alhamadani, A.; Zhang, M.; Lu, C.T. The Quality of AI-Generated Dental Caries Multiple Choice Questions: A Comparative Analysis of ChatGPT and Google Bard Language Models. Heliyon; 2024; 19, e28198. [DOI: https://dx.doi.org/10.1016/j.heliyon.2024.e28198]
23. Ozden, I.; Gokyar, M.; Ozden, M.E.; Sazak Ovecoglu, H. Assessment of artificial intelligence applications in responding to dental trauma. Dent. Traumatol.; 2024; pp. 1-8. [DOI: https://dx.doi.org/10.1111/edt.12965]
24. Bourguignon, C.; Cohenca, N.; Lauridsen, E.; Flores, M.T.; O’Connell, A.C.; Day, P.F.; Tsilingaridis, G.; Abbott, P.V.; Fouad, A.F.; Hicks, L. et al. International Association of Dental Traumatology guidelines for the management of traumatic dental injuries: 1. Fractures and luxations. Dent. Traumatol.; 2020; 36, pp. 314-330. [DOI: https://dx.doi.org/10.1111/edt.12578] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32475015]
25. Mohammad-Rahimi, H.; Ourang, S.A.; Pourhoseingholi, M.A.; Dianat, O.; Dummer, P.M.H.; Nosrat, A. Validity and reliability of artificial intelligence chatbots as public sources of information on endodontics. Int. Endod. J.; 2024; 57, pp. 305-314. [DOI: https://dx.doi.org/10.1111/iej.14014] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/38117284]
26. Antaki, F.; Touma, S.; Milad, D.; El-Khoury, J.; Duval, R. Evaluating the performance of ChatGPT in ophthalmology: An analysis of its successes and shortcomings. Ophthalmol. Sci.; 2023; 3, 100324. [DOI: https://dx.doi.org/10.1016/j.xops.2023.100324] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/37334036]
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Freire, Yolanda 1
; Tortosa, Marta 2 ; Tejedor, Beatriz 2 ; Estevez, Roberto 2 ; Suárez, Ana 1
1 Department of Pre-Clinic Dentistry, School of Biomedical Sciences, Universidad Europea de Madrid, Calle Tajo s/n, Villaviciosa de Odón, 28670 Madrid, Spain;
2 Department of Clinical Dentistry, School of Biomedical Sciences, Universidad Europea de Madrid, Calle Tajo s/n, Villaviciosa de Odón, 28670 Madrid, Spain;