Exploring emergency physicians’ knowledge, attitudes, and behaviour towards Choosing Wisely in Taiwan

Abstract

Background

In 2012, the American Board of Internal Medicine Foundation launched the Choosing Wisely campaign to reduce unnecessary care. However, it is unclear how much emergency physicians in Taiwan understand about Choosing Wisely. The purpose of this study was to explore the knowledge, attitude, and behaviour of emergency physicians in Taiwan regarding Choosing Wisely and its related factors; the intention was to identify the baseline knowledge on the basis of which to promote Choosing Wisely in Taiwan.

Methods

This was a cross-sectional study including emergency physicians in Taiwan as research subjects who answered online questionnaires. A 42-item questionnaire was designed according to the Knowledge, Attitude, and Behaviour model (KAB). The questionnaire linkages were delivered to emergency physicians through social media (eg., Line, Facebook) and received assistance from different hospital directors. A total of 162 valid questionnaires were collected. Data analyses include t-test, analysis of variance, chi-square test, Pearson’s correlation, and multivariate linear regression model.

Results

The study determined that although only 38.9% of emergency physicians had heard of Choosing Wisely, the mean correct rate of knowledge score among emergency physicians was 70.1%. Attitude and the behaviour related to Choosing Wisely were positively associated, which means that the more positive the attitude towards Choosing Wisely is, the more positive the behaviour towards Choosing Wisely is. In multiple linear regression analyses, having served as a supervisor, belonging to divisions of health insurance service, and having heard of Choosing Wisely (P < 0.05) positively affect the knowledge of Choosing Wisely, but age presented a negative association.

Conclusion

This study found that physicians’ knowledge does not influence their attitudes and behaviours, which may be related to barriers of practicing Choosing Wisely activities. To effectively promote Choosing Wisely campaign, it is recommended to focus on the significant factors associated with emergency physicians’ perceptions regarding knowledge, attitude, and behavior of Choosing Wisely. Based on these factors, appropriate practice guidelines for Choosing Wisely can be formulated and promoted.

Introduction

Many countries have been paying increasing attention to healthcare quality recently. In a setting with limited medical resources, waste produced by unnecessary care affects the quality of healthcare [1]. Unnecessary care may be defined as ‘care provided without a clear medical basis or where the benefits of treatment outweigh the risks’, and it not only increases healthcare costs but also harms patients and the healthcare system [2,3]. Due to time constraints, patient preferences, fear of patient dissatisfaction, patients’ low knowledge of the harms of low-value care, and the unavailability of tools to support shared decision-making, most physicians perform unnecessary care including ordering too many diagnostic tests in the emergency department [4,5].

In 2012, the American Board of Internal Medicine Foundation launched the Choosing Wisely campaign, based on the following four points to promote conversations between clinicians and patients and to eventually reduce unnecessary care: care decisions are to be made (1) on the basis of evidence-based medicine, (2) ordered tests or procedures are to be not duplicative of other tests or procedures already performed, (3) patients are to be kept free from harm, and (4) all procedures are to be truly necessary. Choosing Wisely has been promoted in more than 20 countries worldwide, including Canada, Australia, and Italy [6–8]. Moreover, Hicks et al. found that resource waste considerably decreased after implementing the Choosing Wisely recommendations [9].

Chen et al. indicated that the literature on Choosing Wisely is flourishing globally, but the uptake rate in Taiwan is slow, which indicates that the research on Choosing Wisely is not yet mature [10]. Promoting Choosing Wisely in Taiwan is an opportunity to reduce unnecessary care, save costs, and improve the quality of healthcare [11].

According to Colla et al., understanding the factors that lead physicians to administer unnecessary care is the key to promoting Choose Wisely decision-making [12]. However, emergency physicians are on the frontline of dealing with critically ill patients and are often faced with urgent and complex situations during medical practice. The American College of Emergency Physicians (ACEP) joined the Choosing Wisely campaign in 2013 and published a list of tests and procedures that doctors and patients should question the necessity of, with the goal of reducing administration of low-value tests and procedures [13]. It remains unknown whether emergency physicians understand Choosing Wisely, with relevant research being limited. The purpose of this study was to investigate Taiwanese emergency physicians’ (a) understanding of ACEP’s Choosing Wisely recommendations, (b) their attitudes towards unnecessary care, and (c) their behavioural motivation for administering unnecessary care. The study hypotheses include: 1) respondents’ characteristics have significant associations with knowledge, attitude, and behaviour of Choosing Wisely; 2) knowledge, attitude, and behaviour have significant correlations with each other. We believe our results can help to facilitate the promotion of the Choosing Wisely campaign in Taiwan in the future.

Materials and methods

This study surveyed emergency physicians in Taiwan. After a literature review and discussion with clinical experts, a 42-item questionnaire was designed according to the Knowledge, Attitude, and Behaviour model (KAB). KAB model is basically based on the theory of planned behaviour states that behavioural modification is manifested in three steps: information acquisition, belief generation and behaviour changes [14–16]. These three steps have been shown to be positively correlated with each other according to previous research [14,16]. The questionnaire is with questions on (1) demographics, (2) knowledge of Choosing Wisely recommendations, (3) attitude towards unnecessary care, and (4) behaviour of delivering unnecessary care. The survey was answered based on a 5-point Likert-type scale. Each construct of KAB has 10 questions. In Knowledge construct, if the respondents provided the right answer, then he/she got one point. The maximum score is 10. In Attitude and Behaviour construct, each of them has 10 questions with 5-point Likert-type scale. Respondents can answer Strong Agree with score of 5, and Strong Disagree with score of 1. The maximum score is 50 for each construct. We invited three emergency physicians to determine expert validity through counting the content validity index, and the pretest questionnaire was distributed from 15 to 28 July 2020. Content validity index (CVI) measures whether a scale is appropriate for the construct of interest. It evaluates whether the domain of content for the construct can be adequately represented by the questionnaire items [17]. We then revised the survey to make the content more readable. A total of 34 pretest questionnaires were received, and the content validity index value was 0.79 which indicates good content validity [17]. We confirmed the applied KAB model’s Cronbach’s α value to be 0.711, indicating an acceptable level of internal consistency. In 2021, there are 2142 emergency physicians specialists approved to give certificates in Taiwan [18]. We set target sample size of 200 which is about 10% of them. We initially planned to apply one month to collect survey but extended the time to get enough responses of survey due to emergency physicians were busy on caring of COVID-19 patients during the study period. The questionnaire linkages were delivered to emergency physicians through social media (eg., Line, Facebook). We also sought assistance from different hospital directors to send questionnaire linkages to emergency physicians. There is no advertisement of the study. Since the responses were all from the online survey, there is no information regarding how many emergency physicians actually clicked the questionnaire linkage in order to estimate response rate. But all of the responses were valid questionnaires. The survey was conducted online between 24 August 2020 and 14 October 2020. We collected a total of 162 valid questionnaires. The study was approved by Kaohsiung Veterans General Hospital’s Institutional Review Board (IRB; IRB number: KSVGH20-CT8-18). The informed consent of the survey respondents was waived through IRB approval. The questionnaire respondents were anonymous.

Data analysis

We conducted descriptive statistical analysis to determine the distributions of respondents’ characteristics and their perspectives on Choosing Wisely. In the dependent samples, the t-test and analysis of variance were used to analyse the knowledge, attitude, and behaviour scores of Choosing Wisely. The chi-square test was used to examine the correlation between demographics and having heard of Choosing Wisely. Pearson’s correlation was performed to explore the relationship among knowledge, attitude, and behaviour. Furthermore, we applied the multivariate linear regression model to estimate the KAB scores. In our study, all analyses were conducted using SPSS 20.0.

Results

Sample characteristics

One hundred sixty-two individuals responded to the questionnaire in this study. Their characteristics were presented in Table 1. Most respondents were emergency specialist physicians (77.2%), and 61.1% of respondents had not heard of Choosing Wisely. Through cross-analysis, we demonstrated that those who had heard of Choosing Wisely were older (P = 0.013), had a high education level (P = 0.034), were seniors with experience in the emergency department (P = 0.005), and had served as a supervisor (P = 0.005).

[Figure omitted. See PDF.]

Score of knowledge, attitude, and behaviour for Choosing Wisely

On the Choosing Wisely knowledge construct, the mean score was 7.01 (total score of 10), and the mean correct rate was 70.1%. Among topics, ‘avoid CT pulmonary angiography for low-risk patients’ scored the highest.(Table 2). The mean score on the Choosing Wisely attitude construct was 31.82 (total score of 50). Among topics, the attitude that ‘past diagnosis and treatment habits will affect your judgement in implementing unnecessary care’ was the most positive, whereas ‘implement unnecessary care for a patient because of high health insurance reimbursements’ was the most negative. The mean score on the Choosing Wisely behaviour construct was 33.64 (total score of 50). Among topic, the most respondents agreed with ‘I want to rule out the most serious diagnosis, and therefore, I order what would be considered unnecessary tests’, and the fewest respondents agreed with ‘I order unnecessary care because of the high patient insurance reimbursements (including National Health Insurance)’.

[Figure omitted. See PDF.]

Table 3 presents the mean scores of respondents on Choosing Wisely’s knowledge, attitude, and behaviour. In the knowledge construct, the mean scores of young respondents, those with less experience in practice, those in the Kaoping divisions of health insurance service, and those who had heard of Choosing Wisely were higher than those of other respondents.

[Figure omitted. See PDF.]

Correlation among knowledge, attitude, and behaviour

Through Pearson’s correlation analysis, only attitude and behaviour in the three dimensions of KAB were determined to have a significant positive correlation (attitude and behaviour: r = 0.647, P < 0.01). However, knowledge-to-attitude and knowledge-to-behaviour had a less significant correlation.

Predictors of preference-based scores

We conducted multiple regression analysis on three dimensions to evaluate the predictive indicators of emergency physicians’ scores on Choosing Wisely’s knowledge, attitudes, and behaviours. The results indicated that age, having served as a supervisor, working in the health insurance division, and having heard of Choosing Wisely were predictors of knowledge score. On the other hand, age had a negative influence, meaning that older age was correlated with lower score for Choosing Wisely knowledge. The R² of this model was 0.261 (Table 4). However, the results of the multiple regression models of attitudes and behaviours were not significant.

[Figure omitted. See PDF.]

Discussion

Statement of principal findings

This study aimed to investigate the knowledge, attitudes, and behaviours of emergency physicians in Taiwan regarding Choosing Wisely. The results showed that only 38.9% of the emergency physicians in Taiwan surveyed had heard of Choosing Wisely, indicating that Choosing Wisely campaign was not well-known in Taiwan. Our results found that attitude and behaviours are positively correlated; however, knowledge does not influence attitudes and behaviours.

Strengths and limitations

In this study, we investigated the perspective of emergency physicians about Choosing Wisely in Taiwan, which was limited reported in previous studies. This study highlights the important factors affecting emergency physicians to implement Choosing Wisely campaign. It will facilitate a better reduction of ordering unnecessary care. However, this study has three limitations. First, this study used questionnaires, and the study results cannot represent the true clinical behaviour of the physicians. In addition, the interviewees voluntarily participated, which means their statements cannot represent the views of all emergency physicians in Taiwan. Additionally, the questionnaire is filled out according to personal interests and the results of the research may be overestimated. Second, only 162 samples were collected in this study, which may have been insufficient. In 2021, there are 2142 emergency physicians specialists approved to give certificates in Taiwan [18]. To verify our sample size is enough for the study, we estimated the sample size by applying a confidence level of 95%, assuming a 5% margin of error, a population frequency of 10%, and population size of 2142. A minimum required sample size of 130 participants was calculated [19]. Therefore, our study sample size is valid for the study. Most of the samples were collected from physicians working in medical centres and working in the southern region. Therefore, the results were possibly affected by respondents being from different levels of hospitals and health insurance districts. Because the survey samples were not through randomly selection, it cannot claim that the study participants are representative of other emergency physicians in Taiwan. Therefore, it must be cautioned to consider external generalizability of the study results. Finally, this study only examined emergency physicians’ knowledge, attitude, and behaviour regarding Choosing Wisely and did not investigate the difficulties of emergency physicians in implementing Choosing Wisely guidelines in clinical practice.

Interpretation within the context of the wider literature

The mean correct rate of physicians in the knowledge construct of Choosing Wisely was 70.1%, which was consistent with the results in the United States and Spain [20,21], indicating that emergency physicians in Taiwan have sufficient knowledge of evidence-based medicine. However, Choosing Wisely is still not prevalently promoted in Taiwan, which indicates that there is a need to strengthen the understanding of the benefits of implementing Choosing Wisely.

Our results suggest that ‘past diagnosis and treatment habits’ may be a major factor in the attitude of emergency physicians towards unnecessary care. The rule of thumb for physicians is to make decisions based on previous clinical experience; this is a cognitive bias. Studies in Brazil have yielded similar results to ours [22,23]. Levinson’s research demonstrated that most people are reluctant to change old habits, and thus, a new approach needs to offer simplified or relatively beneficial results for it to be adopted [8]. Some have recommended using targeted interventions such as audit and feedback and clinical decision support to alert physicians when their decisions deviate from clinical guidelines [24–26] or presenting to physicians the positive effects of implementing Choosing Wisely, which may help improve physicians’ attitudes toward Choosing Wisely. Attitude is expected to affect behaviour. Attitude contains an individual’s opinion (belief/disbelief) or emotional response (liking/disliking) towards an object. Behaviour consists of actions or observable responses that are the result of an attitude [27].

We also found that physicians were influenced by the behaviour of ‘want to rule out (R/O) serious diagnosis’ to administer unnecessary care. This result is consistent with those of a survey of emergency physicians in the United States. Because decision-making rules are imperfect or not completely understood, physicians may be slow to adopt clinical guidelines in clinical practice [28]. Moreover, being concerned about failing to make a diagnosis is considered an aspect of medical culture that is difficult to change [29]. Therefore, the participation of medical providers, management support, cultural reforms, clear data and training, and teamwork may help narrow the gap between guidelines and practice, leading to more beneficial outcomes [30].

In terms of attitude and behaviour, emergency physicians in Taiwan did not agree that ‘insurance benefits’ would affect the implementation of unnecessary care, indicating that emergency physicians in Taiwan are not tempted by the advantage of insurance benefits. This result has also been verified in Charlesworth’s (2014) and Colla’s (2018) studies. Low-value medical care is not related to insurance types or expected insurance benefits [31,32].

The significant effect of age on the knowledge score of Choosing Wisely indicates that younger physicians are more knowledgeable about evidence-based medicine than older physicians. Similar results have been found in past studies, which can be explained by the more up-to-date academic training of younger physicians [21]. For example, Taiwan developed ‘post-graduate year training, PGY’ in 2003, adding evidence-based medicine to medical education; therefore, younger physicians have greater knowledge [33]. By contrast, on the attitude and behaviour score of Choosing Wisely, older age, higher education level, longer practicing experience, longer working experience in the emergency department, and serving as a supervisor were higher than other respondents. We can infer that senior physicians in the emergency department were more aware of the need to reduce unnecessary care. Although younger physicians exhibited greater knowledge, senior physicians performed better in attitude and behaviour, indicating that knowledge is not reflected in attitude and behaviour. The reason may be that younger physicians lack time to follow new decision making methods such as Choosing Wisely due to heavy workload, or they order unnecessary diagnostic tests because of the fear of making wrong decisions and having little experience.

Implications for policy, practice and research

Exploring emergency physicians’ understanding of Choosing Wisely can help evaluate development strategies. Taiwan’s Choosing Wisely is still in the promotion stage. Much unnecessary care is delivered in Taiwan. For example, according to previous studies, the use of antibiotics is very common in Taiwan, and inappropriate prescription of antibiotics can increase the risk of potential adverse effects, incur unnecessary costs, and lead to antibiotic resistance [34,35]. Another example, the elderly with insomnia, anxiety, depression, and other mental disorder were more likely to receive benzodiazepine prescriptions. However, inappropriate use of benzodiazepine may be associated with the risk of fractures [36,37]. It is an opportunity to reduce unnecessary care, save costs, and improve the quality of healthcare through promoting Choosing Wisely in Taiwan [11]. Healthcare policy maker can formulate clearer Choosing Wisely practice guidelines. From 2016, a series of forum discussions of evaluating low-value cares initialized Choosing Wisely Taiwan campaign. Through systematic literature review and discussing with experts, Cochrane Taiwan then suggested five top low-value cares that need to be improved as top priority. In 2020, Joint Commission of Taiwan recommended hospitals to take actions to avoid low-value care and reduce unnecessary resource wastes, and suggested hospitals can follow US Choosing Wisely as references and added it into hospital accreditation item [38,39]. This will lead hospitals to incorporate Choosing Wisely into hospital policies to change physicians’ habits of performing medical treatment and use relevant education and training and related tools, such as the shared decision-making and clinical decision-making system of medical and disease, etc., to improve medical waste and improve the quality of medical treatment.

Conclusion

This study demonstrated that the prevalence of Choosing Wisely adherence is still low in Taiwan. Emergency physicians have sufficient knowledge of practical medicine, but entrenched diagnostic habits and the desire to rule out serious diagnoses continue to lead to delivery of unnecessary care. More effort needs to be made to promote Choosing Wisely in conjunction with the adoption of relevant interventional policies and teamwork.

Acknowledgments

We would like to thank all research team members for their contributions and all emergency physicians for participating in the survey.

Citation: Juang W-C, Chiou SM-J, Yang H-L, Li Y-C (2022) Exploring emergency physicians’ knowledge, attitudes, and behaviour towards Choosing Wisely in Taiwan. PLoS ONE 17(7): e0271346. https://doi.org/10.1371/journal.pone.0271346

About the Authors:

Wang-Chuan Juang

Roles: Conceptualization, Funding acquisition, Methodology, Project administration

Affiliations Department of Quality Management Center, Kaohsiung Veterans General Hospital, Kaohsiung, Taiwan, Department of Emergency Medicine, Kaohsiung Veterans General Hospital, Kaohsiung, Taiwan, Department of Business Management, National Sun Yat‐sen University, Kaohsiung, Taiwan

Sonia Ming-Jiu Chiou

Roles: Writing – original draft, Writing – review & editing

Affiliations Department of Business Management, National Sun Yat‐sen University, Kaohsiung, Taiwan, Institute of Health Care Management, National Sun Yat‐sen University, Kaohsiung, Taiwan

Hui-Ling Yang

Roles: Formal analysis, Writing – original draft

Affiliations Department of Business Management, National Sun Yat‐sen University, Kaohsiung, Taiwan, Institute of Health Care Management, National Sun Yat‐sen University, Kaohsiung, Taiwan, Department of Planning, Kaohsiung Municipal Min-Sheng Hospital, Kaohsiung, Taiwan

Ying-Chun Li

Roles: Conceptualization, Methodology, Supervision, Writing – review & editing

E-mail: [email protected]

Affiliations Department of Business Management, National Sun Yat‐sen University, Kaohsiung, Taiwan, Institute of Health Care Management, National Sun Yat‐sen University, Kaohsiung, Taiwan

https://orcid.org/0000-0001-8516-970X

References

1. Colla , Morden NE, Sequist TD, Schpero WL, Rosenthal MB. Choosing wisely: prevalence and correlates of low-value health care services in the United States. Journal of general internal medicine. 2015;30(2):221–8. pmid:25373832

2. Morgan DJ, Brownlee S, Leppin AL, Kressin N, Dhruva SS, Levin L, et al. Setting a research agenda for medical overuse. Bmj. 2015;351:h4534. pmid:26306661

3. Brownlee S, Chalkidou K, Doust J, Elshaug AG, Glasziou P, Heath I, et al. Evidence for overuse of medical services around the world. The Lancet. 2017;390(10090):156–68. pmid:28077234

4. Buist DS, Chang E, Handley M, Pardee R, Gundersen G, Cheadle A, et al. Primary care clinicians’ perspectives on reducing low-value care in an integrated delivery system. The Permanente Journal. 2016;20(1):41. pmid:26562308

5. Kanzaria HK, Hoffman JR, Probst MA, Caloyeras JP, Berry SH, Brook RH. Emergency physician perceptions of medically unnecessary advanced diagnostic imaging. Academic Emergency Medicine. 2015;22(4):390–8. pmid:25807868

6. Levinson W, Kallewaard M, Bhatia RS, Wolfson D, Shortt S, Kerr EA. ‘Choosing Wisely’: a growing international campaign. BMJ quality & safety. 2015;24(2):167–74. pmid:25552584

7. American Board of Internal Medicine Foundation. Choosing Wisely.2012.Available from: https://www.choosingwisely.org/our-mission/.(accessed 14 July 2021).

8. Levinson W, Born K, Wolfson D. Choosing wisely campaigns: a work in progress. Jama. 2018;319(19):1975–6. pmid:29710232

9. Hicks CW, Liu J, Yang WW, DiBrito SR, Johnson DJ, Brito A, et al. A comprehensive Choosing Wisely quality improvement initiative reduces unnecessary transfusions in an Academic Department of Surgery. The American Journal of Surgery. 2017;214(4):571–6. pmid:28683893

10. Chen C, Tam K-W, Kuo KN. Choosing wisely in health care. Journal of the Formosan Medical Association = Taiwan yi zhi. 2018;117(9):754. pmid:29980349

11. Mafi JN, Russell K, Bortz BA, Dachary M, Hazel WA Jr, Fendrick AM. Low-cost, high-volume health services contribute the most to unnecessary health spending. Health Affairs. 2017;36(10):1701–4. pmid:28971913

12. Colla , Kinsella EA, Morden NE, Meyers DJ, Rosenthal MB, Sequist TD. Physician perceptions of Choosing Wisely and drivers of overuse. Am J Manag Care. 2016;22(5):337–43. pmid:27266435

13. American Board of Internal Medicine Foundation. American College of Emergency Physicians Ten Things Physicians and Patients Should Question.2013. Available from: https://www.choosingwisely.org/societies/american-college-of-emergency-physicians/.(accessed 30 June 2020).

14. Ajzen I. The theory of planned behavior. Organizational Behavior and Human Decision Processes. 1991; 50: 179–211.

15. Bettinghaus EP. Health promotion and the knowledge‐attitude-behavior continuum. Preventive Medicine. 1986; 15: 475–491. pmid:3774779

16. Huang HU, Kuo KM, Lu IC, Wu H, Lin CW, Hsieh MT, et al. The impact of health literacy on knowledge, attitude and decision towards hospice care among community‐dwelling seniors. Health and Social Care in the Community. 2019; 00:1–10. pmid:31215097

17. Polit DF, Beck CT, Owen SV. Is the CVI an Acceptable Indicator of Content Validity? Appraisal and Recommendations. Research in Nursing & Health.2007; 30: 459–467. pmid:17654487

18. Ministry of Health and Welfare. Number of specialists approved to give certificates. 2022. https://dep.mohw.gov.tw/DOS/cp-5301-62356-113.html (in Chinese) Accessed on 2022/04/30.

19. Calculator.net. Sample Size Calculator. https://www.calculator.net/sample-size-calculator.html?type=1&cl=95&ci=5&pp=10&ps=2142&x=69&y=24 (Accessed on 2022/4/29).

20. Tests U. Procedures in the Health Care System. What Physicians Say About The Problem, the Causes, and the Solutions PerryUndem Research/Communication. 2014.

21. Aranaz Andrés JM, Valencia-Martín JL, Vicente-Guijarro J, Díaz-Agero Pérez C, López-Fresneña N, Carrillo I, et al. Low-Value Clinical Practices: Knowledge and Beliefs of Spanish Surgeons and Anesthetists. International Journal of Environmental Research and Public Health. 2020;17(10):3556. pmid:32438657

22. Lam JH, Pickles K, Stanaway FF, Bell KJ. Why clinicians overtest: development of a thematic framework. BMC health services research. 2020;20(1):1–11. pmid:33148242

23. Correia LC, Barcellos GB, Calixto V, Volschan A, Barreto-Filho JA, Lopes RD, et al. ‘Choosing Wisely’culture among Brazilian cardiologists. International Journal for Quality in Health Care. 2018;30(6):437–42. pmid:29506135

24. Ferrari R. Imaging studies in patients with spinal pain: Practice audit evaluation of Choosing Wisely Canada recommendations. Canadian Family Physician. 2016;62(3):e129–e37. pmid:27427564

25. Heekin AM, Kontor J, Sax HC, Keller MS, Wellington A, Weingarten S. Choosing Wisely clinical decision support adherence and associated inpatient outcomes. Am J Manag Care. 2018;24(8):361. pmid:30130028

26. Mehta N, Born K, Fine B. How artificial intelligence can help us ‘Choose Wisely’. Bioelectronic Medicine. 2021;7(1):1–6.

27. Yi Q, Hohashi N. Comparison of perceptions of domestic elder abuse among healthcare workers based on the Knowledge-Attitude-Behavior (KAB) model. PLoS ONE. 2018;13(11): e0206640. pmid:30383824

28. Lin MP, Nguyen T, Probst MA, Richardson LD, Schuur JD. Emergency physician knowledge, attitudes, and behavior regarding ACEP’s choosing wisely recommendations: a survey study. Academic Emergency Medicine. 2017;24(6):668–75. pmid:28164409

29. Morgan DJ, Leppin A, Smith CD, Korenstein D. A practical framework for understanding and reducing medical overuse: conceptualizing overuse through the patient-clinician interaction. Journal of hospital medicine. 2017;12(5):346. pmid:28459906

30. Andersen H, Røvik KA, Ingebrigtsen T. Lean thinking in hospitals: is there a cure for the absence of evidence? A systematic review of reviews. BMJ open. 2014;4(1).

31. Colla CH, Morden NE, Sequist TD, Mainor AJ, Li Z, Rosenthal MB. Payer type and low‐value care: comparing choosing wisely services across commercial and Medicare populations. Health services research. 2018;53(2):730–46. pmid:28217968

32. Charlesworth CJ, Meath TH, Schwartz AL, McConnell KJ. Comparison of low-value care in Medicaid vs commercially insured populations. JAMA internal medicine. 2016;176(7):998–1004. pmid:27244044

33. Lo W-L, Lin Y-G, Pan Y-J, Wu Y-J, Hsieh M-C. Faculty development program for general medicine in Taiwan: Past, present, and future. Tzu Chi Medical Journal. 2014;26(2):64–7. pmid:32288425

34. Chang L-Y, Lai C-C, Chen C-J, Cho C-Y, Luo Y-C, Jeng M-J, et al. Recent trends in prescribing antibiotics for acute tonsillitis in pediatric ambulatory care in Taiwan, 2000–2009: A nationwide population-based study. Journal of Microbiology, Immunology and Infection. 2017;50(4):500–6. pmid:26455488

35. Chang S-C, Chang H-J, Lai M-S. Antibiotic usage in primary care units in Taiwan. International journal of antimicrobial agents. 1999;11(1):23–30. pmid:10075274

36. Cheng JS, Huang WF, Lin KM, Shih YT. Characteristics associated with benzodiazepine usage in elderly outpatients in Taiwan. International Journal of Geriatric Psychiatry: A journal of the psychiatry of late life and allied sciences. 2008;23(6):618–24. pmid:18058834

37. Chan AL, Lin S-J. Trends of benzodiazepine prescribing and the risk of hip fracture in elderly patients in Taiwan: A population-based study. International journal of psychiatry in clinical practice. 2010;14(1):47–52. pmid:24917232

38. Joint Commission of Taiwan. Accreditation. https://www.jct.org.tw/mp-2.html (Accessed on 2022/04/10).

39. Chen KH, Kuo KN, Chen CF. Less is More: An Introduction of Choosing Wisely Concept. Journal of Healthcare Quality. 2021; 15(6): 11–15. (in Chinese).