Importance-based approach to entrustable professional activities for psychiatric residency training

Abstract

Background

Entrustable professional activities (EPAs) are essential tools in implementing CBME and integrating milestones into clinical practice. This study aimed to develop a set of EPAs tailored to the psychiatric residency training context with local institutional needs, and to dissect the differences and factor analysis in perceived EPA importance.

Methods

21 customized EPAs specific to psychiatric residency training at Taipei City Psychiatric Center (TCPC) were developed based on the Taiwanese Society of Psychiatry (TSOP) framework. A two-round Delphi method was used to achieve consensus for the EPAs. The finalized EPAs were evaluated using an online questionnaire, and faculty members and residents rated each activity’s importance on a Likert scale of 0–10. Statistical analyses were performed on perceptions differences between teachers and trainees, and to cluster EPAs into meaningful competency clusters.

Results

The finalized TCPC EPA framework included the TSOP’s original 17 EPAs and four additional EPAs to address advanced skills in psychotherapy, rehabilitation, legal applications, and innovative technologies. The total response rate was 89.6% for questionnaire assessing importance. Faculty consistently rated EPAs as more important than residents did, particularly clinical documentation, teaching and long-term care planning. Factor analysis revealed two loading factors, referring to internal knowledge and an external communication.

Conclusion

The experience of adapting EPAs from global and national standards to fit the characteristics of a local training institution provides a valuable reference for the localization of EPA development. Observed differences in perceptions between trainees and teachers highlight the importance of aligning educational expectations. Enhancing awareness of the underlying dimensions associated with EPAs may help optimize psychiatric training outcomes for both trainees and instructors.

Introduction

Medical education has evolved to embrace competency-based medical education (CBME), which emphasizes students’ practical abilities and performance rather than focusing solely on the length of study or the number of completed courses [1]. The goal of CBME is to ensure that learners acquire the essential competencies needed to address real-world challenges in clinical practice. This approach helps translate the “core competencies” emphasized during training into the actual competence required in the workplace [2]. This shift is closely tied to the six core competencies for physicians established by the Accreditation Council for Graduate Medical Education (ACGME) in the United States. Using this core competency framework, curricula and assessments have been developed to strengthen these competencies [3, 4]. Milestones and entrustable professional activities (EPAs) are currently the most widely used assessment tools for implementing CBME in medical education and training [5]. Milestones focus on tracking the development of physicians’ competencies by defining stages of proficiency that residents must achieve in various clinical skills. EPAs, on the other hand, are units of professional work that trainees are expected to perform independently once sufficient competence is demonstrated. As EPAs are grounded in the concept of trust, assessment focuses on whether a supervisor feels confident to entrust the trainee with the task. Milestones and EPAs are complementary tools designed to integrate continuous assessment into clinical practice [6, 7].

CBME is currently being implemented in psychiatric residency training in Taiwan, drawing key references from the psychiatry milestones developed by the American ACGME workgroup [8] and the EPAs for psychiatry outlined by the Task Force of the American Association of Directors of Psychiatric Residency Training [9]. Since 2017, the Ministry of Health and Welfare has mandated that all specialty medical societies design CBME-based training and certification programs for specialists. In response, the Taiwanese Society of Psychiatry (TSOP) commissioned its education committee, chaired by Professor Cheng-Sheng Chen, to initiate the integration of CBME into the existing psychiatric specialist training and certification program. This initiative led to the collaborative development of the “Framework of EPAs and Milestone Development in Psychiatry Residency Training in Taiwan.” This framework, documented as the first draft by the TSOP, identifies 17 EPAs and has been published in an educational academic journal as a reference [10]. During its development, minor revisions and adjustments were made to the milestones to ensure alignment with Taiwan’s local healthcare context, enhancing both the relevance and practicality of the educational content. EPAs, in particular, are viewed as an opportunity to highlight the distinctive characteristics of each teaching hospital. The framework serves as a foundational blueprint for developing CBME programs tailored to the specific contexts of individual hospitals across Taiwan.

In Taiwan, qualified psychiatry training institutes include medical centers, regional teaching general hospitals, and psychiatric hospitals. Among these, approximately 54% of trainees are based in medical centers, 17% in regional teaching general hospitals, and 29% in psychiatric hospitals. According to a survey conducted as part of the TSOP CBME framework, agreement on several EPAs varied among trainees from these three types of training institutes [10]. As a result, the second step in developing CBME for psychiatry training in Taiwan involves engaging local hospitals in adapting the EPAs to reflect their unique educational characteristics. The Taipei City Psychiatric Center (TCPC), a division of Taipei City Hospital, specializes in psychiatric care, education, and research. Accredited in 1981 as Taiwan’s first psychiatric teaching hospital, TCPC offers comprehensive psychiatric services, including outpatient and inpatient care, community-based programs, and specialized clinics in addiction medicine, geriatric psychiatry, forensic psychiatry, and child and adolescent psychiatry. The initial version of the EPAs developed by TSOP was created by an expert panel based on general hospital settings. In response, TCPC established its own internal CBME development team in 2023 to create a localized version of the EPAs that better reflect the institution’s strengths. This effort also aligns with the second step of the TSOP CBME program, which encourages institutions to tailor EPAs to their specific teaching environments.

This study documents how the CBME team at a psychiatry-specialized teaching hospital identified and defined key EPAs, along with their perceived significance and importance. Analyses were conducted to explore differences in training emphases between psychiatric specialty hospitals and general hospitals for psychiatric residents, as well as discrepancies in the perceived importance of EPAs between trainees and supervisors. Additionally, factor analysis was employed to identify clusters of EPAs based on perceived importance, allowing the resulting dimensions to be introduced to both teachers and trainees. The findings may serve as a valuable reference for EPA development in other institutions. Furthermore, by documenting this process in the scientific literature, we aim to foster international dialogue and contribute to the advancement of CBME in psychiatry within Taiwan.

Methods

Data sources

The CBME development team at TCPC consisted of eight members: the director of the Psychiatry Residency Review Committee (RRC) training program, the director of medical education and research, the head of faculty development and a teaching-oriented attending physician, two core faculty attending psychiatrists, and three fifth-year psychiatry residents. The team undertook three main tasks: First, to review the 17 EPAs from the TSOP framework and evaluate their suitability for implementation at TCPC, including identifying any need for revision. Second, to examine international literature and discuss additional EPAs that could better reflect the unique characteristics of TCPC. Third, to assess the newly selected EPAs for clarity and relevance in their Chinese versions, followed by content validity evaluation. Finally, the perceived importance of all EPAs was evaluated using a questionnaire completed by all attending psychiatrists and psychiatry residents at TCPC.

The Delphi method is also known as the expert opinion method [11], was applied in two rounds for each of the aforementioned tasks. Detailed explanations of the objectives, topics, and requirements were provided to ensure that all experts fully understood the purpose and scope of each task. The TSOP draft EPAs were then subjected to a content validity review using the Content Validity Index (CVI) to evaluate their suitability. Feedback and suggestions were collected, quantified, and analyzed to reach consensus among members of the CBME team. A 5-point Likert scale was used to rate the suitability of each EPA, with higher scores indicating greater suitability. During each Delphi round, the percentage of mode responses and the quartile deviation (QD) were assessed. A suitability score below 4 or a mode percentage below 50% was considered to indicate a lack of consensus. EPAs with a CVI of ≥ 0.8 were considered to have reached acceptable content validity and were included in the finalized TCPC psychiatry EPA framework.

The TCPC psychiatry EPAs were then distributed through an online questionnaire to attending physicians and residents within the hospital. Respondents rated the importance of each EPA using a Likert scale (0–10, where 0 = strongly disagree, 5 = neutral, and 10 = strongly agree). At the time of the survey (January 2024), the hospital had a total of 48 physicians (29 attending psychiatrists, 16 resident physicians, and 3 research fellow psychiatrists). Of these, 43 completed the questionnaires, which corresponded to a response rate of 89.6%. The project was approved by the Research Ethics Committee of Taipei City Hospital (TCHIRB-11304019-E). Written informed consent was obtained from the TCPC CBME development team members. The need for consent to other participates was waived according to that (TCHIRB-11304019-E).

Statistical analysis

A descriptive analysis was used to analyze the importance score between trainees (residents) and teachers (attending psychiatrists) in TCPC. Student’s t-tests were used to assess continuous variables. The normality of the data was assessed using the Kolmogorov–Smirnov test. For non-normally distributed data, we used a nonparametric Wilcoxon rank-sum test. We divided the EPAs as dichotomous items into those with full agreement (Likert scale = 10) and those without full agreement among the respondents regarding the importance and used a chi-squared test to analyze the differences between trainees and teachers. We corrected the variable of working experience (working years) with clinical practice by applying a regression model to examine the effect of being a trainee on the perception of importance among the EPAs. Finally, we used factor analysis to explore the underlying dimensions that explain the importance of the EPAs. All analyses were conducted using IBM SPSS version 24, and the significance level was set at p < 0.05.

Results

The process and outcome of local EPA development at TCPC

The process of developing psychiatry-specific EPAs at TCPC using the Delphi method is illustrated in Fig. 1. The mode percentages and QDs for ratings across the three tasks are presented in e-Table 1. In Task 1, all 17 EPAs originally derived from the TSOP framework were reviewed to assess their contextual suitability for implementation at TCPC. Fourteen of these EPAs achieved a CVI of 0.8 or higher in the first round of Delphi assessment, indicating acceptable content validity. The remaining three EPAs (EPA 7, EPA 14, and EPA 15) required revisions to their contextual descriptions and were re-evaluated in a second round, after which they successfully reached the consensus threshold. In Task 2 focused on the identification of additional EPAs that reflect the specific characteristics and training needs of TCPC. Six new EPAs were proposed, among which four—designated as EPAs 18 through 21—achieved strong content validity in the first Delphi round, with CVIs ranging from 0.875 to 1.0. Two other proposed EPAs—one concerning leadership in inter-professional healthcare teams and another related to the application of quality improvement methodologies in clinical practice—did not meet the required CVI threshold across two Delphi rounds and were therefore excluded from the final framework. In Task 3, the newly added EPAs (18–21) were further evaluated for clarity and relevance in their Chinese-language versions, followed by another content validity assessment. All four EPAs achieved CVIs of 0.8 or higher, confirming their linguistic clarity and contextual appropriateness for implementation. As a result of this multi-step process, the final EPA framework for TCPC consisted of 21 validated EPAs.

[IMAGE OMITTED: SEE PDF]

The 21 EPAs and a corresponding core competency matrix are shown in Table 1. The 21-EPA program included the 17 items of the TSOP CBME framework, which covers general psychiatric practice, consideration of mental capacities, medical conditions and consultation, psychiatric public health, and resident faculty. In addition, newly added EPA items 18–21 concerned the need for more specialized skills and techniques, trainee’s mastery of long-term treatment goals, integration of legal and professional standards, and extension of innovative therapeutic techniques.

[IMAGE OMITTED: SEE PDF]

Discrepancies in perceptions of importance between trainees and teachers across the 21 EPAs

A comparison of the perceived importance of the 21 EPAs between attending psychiatrists (n = 27) and psychiatry residents (n = 16) is presented in e-Table 2. The results revealed statistically significant differences in the overall importance scores between the two groups. On average, attending psychiatrists consistently rated the EPAs as more important than residents did, with a total mean score of 9.04 (standard deviation [SD] = 0.94) compared to 8.23 (SD = 1.40) for residents (p = 0.049). Significant group differences emerged for several specific EPAs, including EPA 1 (p = 0.018), EPA 6 (p = 0.016), EPA 9 (p = 0.032), EPA 10 (p = 0.044), EPA 14 (p = 0.041), EPA 17 (p = 0.009), and EPA 20 (p = 0.018). These findings suggest that particular professional activities were valued differently by residents and their supervising faculty.

Further analysis using dichotomized data is shown in Table 2, where EPAs were classified based on whether respondents indicated full agreement regarding importance (Likert scale score = 10). Significant discrepancies in full agreement rates were observed for EPA 6 (documenting and presenting all clinical data related to a psychiatric patient), EPA 9 (giving or receiving patient handovers to transition care responsibilities), EPA 17 (delivering didactic psychiatry lectures and conducting clinical training sessions for junior physicians, undergraduate medical students, and other medical personnel students), and EPA 20 (assessing and formulating rehabilitation plans and facilitating resource linkage following acute care). In addition, a regression analysis was conducted to control for clinical working experience (measured by years of practice) to isolate the effect of trainee status on perceptions of importance. As shown in Table 3, resident status was associated with significantly lower importance ratings for EPA 1, EPA 6, EPA 10, EPA 17, and EPA 20. Notably, EPA 6, EPA 17, and EPA 20 consistently showed statistically significant group differences across e-Tables 2 and Table 2, and Table 3, suggesting that these EPAs represent key areas of perceptual divergence between trainees and faculty.

[IMAGE OMITTED: SEE PDF]

[IMAGE OMITTED: SEE PDF]

Factor analysis based on the importance of the EPAs

The factor analysis results based on the importance score identified two distinct factors, which explain a significant portion of the variance in the data. In the factor pattern (Fig. 2a), the scatter plot visually demonstrates the distribution of EPAs across the two factors. Factor 1 explains 51.3% of the variance, while factor 2 accounts for an additional 48.7%. The pathway analysis further illustrates the relationships between individual EPAs and their respective factors (Fig. 2b). EPAs in factor 1 (red cluster) exhibit strong loadings and include EPA 7, 8, 9, 14, and 15, which emphasize the role of communication and information management in psychiatric practice. EPAs in factor 2 (blue cluster), such as EPA 1, 2, and 3, reflect the importance of core clinical reasoning and decision-making processes. EPAs with cross-factor pathways indicate some degree of overlap between the two domains, but the factors remain largely distinct. The suitability of the data for factor analysis was supported by a Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy of 0.73 overall, with all individual EPAs yielding KMO values greater than 0.5.

[IMAGE OMITTED: SEE PDF]

Discussion

The meaningful process of tailoring EPAs to institutional contexts

The development of EPAs involves reflective approaches to the daily work of professionals; therefore, the creation of context-specific EPAs at individual hospitals remains valuable [12, 13]. With the establishment of a national EPA framework by the TSOP, this is a timely opportunity for individual hospitals to evaluate the framework’s applicability and to consider developing specialized EPAs that reflect their unique clinical settings and educational strengths. Items 18 to 21 differ from the previous 17 items in terms of depth, focus, and the application of specialized skills. The first 17 items primarily establish foundational clinical skills such as history taking, diagnosis, risk management, and psychoeducation, while items 18 to 21 introduce more specialized and technical abilities. For example, EPA 18 involves evaluating psychological therapy models and administering specific therapies, which require a deeper understanding of therapeutic approaches and the ability to personalize care. EPA 19 emphasizes the integration of legal and ethical considerations into psychiatric practice, highlighting the psychiatrist’s broader societal responsibilities. EPA 20 focuses on formulating rehabilitation plans, linking patients to essential resources, and addressing long-term care needs that go beyond immediate treatment. EPA 21 highlights the administration of repetitive transcranial magnetic stimulation (rTMS) and demonstrates the importance of mastering innovative psychiatric interventions. Together, these advanced EPAs not only broaden the scope of psychiatric training but also prepare residents to become specialized, forward-thinking practitioners equipped to meet evolving clinical and societal needs.

Comparisons between the TSOP framework with international literature are shown in supplementary e-Table 3 (milestones) and e-Table 4 (EPAs). The TCPC EPAs incorporate several distinctive components not explicitly outlined in the AADPRT Psychiatry Task Force framework. Conversely, they also omit certain EPAs that are highlighted in the AADPRT version. For example, TCPC EPA 18 (evaluating psychological therapy models and administering specific types) is partially covered in AADPRT EPAs 9, 11, and 12 to emphasize evaluating and implementing specific psychological therapy models, as well as detailing therapeutic interventions. TCPC EPA 19 (implementing mental health-related laws) is not explicitly covered by the AADPRT and refers to the relevance in local healthcare systems. TCPC EPA 20 (assessing and formulating rehabilitation plans and resource linkage) aligns with the concept of long-term patient care in the concept of AADPRT EPA 1. TCPC EPA 21 is related to administering rTMS, and although not directly covered by AADPRT, it is one of the key development focuses and is one of the hospital’s teaching features. The AADPRT EPAs also include leadership in inter-professional healthcare teams and the application of quality-improvement methodologies in clinical practice. These elements were considered essential and discussed during the development process. However, consensus was not reached on these two EPAs, as the CBME team viewed them as competencies more appropriate for experienced attending physicians and noted the challenges associated with evaluating them during residency training.

Perceptions of the importance of the 21 EPAs between trainees and teachers

These findings indicate that teachers are more aware of the importance of EPAs for trainees and also reflect the gap between teachers’ and trainee’s expectations about advanced experience. EPA 6, EPA 17, and EPA 20 were especially noted through kinds of statistical methods. Perceptions of importance differ between trainees and supervisors regarding EPA 6 (documenting and presenting all clinical data related to a given psychiatric patient), EPA 17 (delivering didactic psychiatry lectures and conducting clinical training for junior physicians, medical students, and other healthcare trainees), and EPA 20 (assessing and formulating rehabilitation plans and establishing resource linkages following acute care), may reflect differing perspectives on future professional roles. Trainees may consider these activities—such as comprehensive documentation, teaching, and rehabilitation planning—as peripheral to their anticipated career paths. For instance, EPA 6 may be undervalued because residents rarely directly encounter legal or medical disputes arising from documentation errors during training. Similarly, EPA 17 might be seen as less relevant by those who do not plan to work in teaching hospitals for their future career, leading them to underestimate the value of teaching medical students. EPA 20 may be underappreciated due to limited opportunities for junior trainees to follow patients longitudinally, making it difficult for them to grasp the importance of long-term care planning. From this point of view, this misalignment may hinder trainee engagement and preparation. It is therefore important to better align expectations and to contextualize the relevance of each EPA within diverse psychiatric career trajectories, in order to address the gaps identified by trainees [14, 15].

Factor analysis of the importance of the EPAs

Based on the importance ratings, EPAs were clustered to two main factors. Factor 2 (blue cluster) emphasizes the ability to diagnose patients, assess risks, and screen for comorbid conditions, all of which rely heavily on a resident’s internal knowledge, judgment, and clinical evaluation skills. Factor 2 is related to knowledge and clinical reasoning competence and focuses on the core clinical skills required for psychiatric residents. This factor represents the foundational cognitive processes that enable psychiatrists to act as effective clinical decision-makers. Factor 1 (red cluster) is related to EPAs for communication and information delivery skills and highlights the outward-facing abilities of psychiatric residents to convey their knowledge and clinical judgments effectively. It encompasses skills such as documenting clinical records, applying knowledge in practice, educating patients, collaborating across professional teams, and promoting mental health awareness. This factor reflects the capacity to externalize professional expertise through communication, documentation, and educational efforts to ensure effective interaction with patients, families, and colleagues.

Sharing the results of the factor analysis with both faculty and trainees may enhance their engagement in competency-based training. For example, the factor loading characteristics of EPAs prompt further consideration of whether they reflect individual learning styles. Individual learner characteristics—such as their tendencies toward interactive versus reflective learning, and their preferred modes of knowledge acquisition—may correspond to their rating categorization of EPAs. Learners who are more inclined toward interactive, communication-based learning may naturally engage more effectively and rating importantly with Factor 1 related EPAs, which emphasize interpersonal communication, education, and collaborative practice. In contrast, those who prefer reflective or solitary learning may align and rate more closely with Factor 2 EPAs, which involve internal clinical reasoning, diagnostic assessment, and analytical decision-making. Similarly, learners who favor text-based study methods may feel more comfortable with the cognitive demands of Factor 2, while those who thrive in experiential, hands-on environments may relate more intuitively to the practice-oriented aspects of Factor 1. By understanding these learner characteristics and learning styles through their rating of importance among EPAs, we may be able to enhance the educational development of trainees, to design teaching strategies that cater to various traits and learning styles, to better suit their individual needs and support diverse developmental goals [16].

Implications and future works

Alongside the first step of the TSOP CBME development, in which CBME-developed international version turn into CBME-developing country’s national guide, this study further enables the framework to be better aligned with the localized context. By adapting global and national standards, the framework becomes more helpful for local teachers and trainees [16,17,18]. Although this study did not use the standard EQual criteria to validate EPAs [19], we explored perceived importance ratings and observed that certain EPAs may be underestimated by trainees. Through factor analysis, we conceptualized the EPAs into two main dimensions: one related to internal clinical knowledge and reasoning, and the other to external communication and collaboration.

Currently, in our institution, residents typically work under indirect supervision and are evaluated based on the six core competencies. After completing four years of required clinical training and participating in core reading sessions, they are eligible to take the written and interview examinations administered by the RRC. Upon passing these examinations, they are considered to have obtained board certification. Therefore, as we gradually incorporate the principles of CBME into our current training and assessment system, it is important for both faculty and trainees to understand the origins and development of local EPAs. We believe this approach can positively influence both teaching practices and learning motivation [20]. We are interested about whether a growth mindset in the context of CBME or the sense of importance of the EPA tasks may impact the outcomes of learning or patient care in clinical education [21, 22]. We plan to follow up on the teaching and learning outcomes in a context where teachers and trainees are already aware of the factors related to EPA importance. The next step for TSOP psychiatry CBME development is to engage in pilot testing across approximately 10 institutions. The experiences from this study will serve as a valuable reference to guide the selection of EPAs for pilot testing and to facilitate feedback collection from learners and supervisors.

When finalizing the three steps for the TSOP CBME implementation in psychiatry residency training, the levels of supervision will provide a structured way to assess the degree of independence a learner can be entrusted with while performing a professional task. The model outlines five levels of supervision based on Ten Cate’s framework (2005) [23], which are critical for making entrustment decisions. The evaluation of trainees’ competencies will be integrated into current four-year training program at qualified training institutes. The formal decisions regarding whether a trainee can independently perform a professional activity are typically made by a Competence Committee (CCC) at the qualified training institutes. These decisions should be based on multiple data points, including direct observations, EPA assessments, and feedback from various supervisors, ensuring a comprehensive and objective evaluation of the learner’s readiness.

Conclusions

In this study, we developed the TCPC EPAs based on the TSOP framework, encompassing both foundational clinical skills and broader competencies in psychiatric practice. The framework aimed to support residents’ progression toward advanced expertise, particularly in psychotherapy, long-term rehabilitation, legal and ethical practice, and emerging psychiatric technologies. This experience of “localizing” EPAs demonstrates how adapting global or national standards to institutional contexts can enhance their relevance and usefulness for local training institutions. Additionally, we identified a misalignment in perceptions of EPA importance between trainees and teachers, which may impede trainee engagement and preparedness. Factor analysis of the importance ratings revealed two core clusters of EPAs: one reflecting internal knowledge and clinical reasoning, and the other emphasizing external communication and information delivery. By increasing awareness and understanding of these underlying dimensions among both trainees and instructors, and by applying the EPAs in a more personalized manner, we anticipate that the effectiveness of psychiatric education and training can be significantly improved.

Data availability

The data that support the findings of this study are available on request from the corresponding author.

References

Whitcomb ME. Transforming medical education: is Competency-Based medical education the right approach?? Acad Medicine: J Association Am Med Colleges. 2016;91(5):618–20.

Frank JR, Snell LS, Cate OT, Holmboe ES, Carraccio C, Swing SR, Harris P, Glasgow NJ, Campbell C, Dath D, et al. Competency-based medical education: theory to practice. Med Teach. 2010;32(8):638–45.

Hawkins RE, Welcher CM, Holmboe ES, Kirk LM, Norcini JJ, Simons KB, Skochelak SE. Implementation of competency-based medical education: are we addressing the concerns and challenges? Med Educ. 2015;49(11):1086–102.

Van Melle E, Frank JR, Holmboe ES, Dagnone D, Stockley D, Sherbino J. A core components framework for evaluating implementation of Competency-Based medical education programs. Acad Medicine: J Association Am Med Colleges. 2019;94(7):1002–9.

Wagner SJ, Reeves S. Milestones and entrustable professional activities: the key to practically translating competencies for interprofessional education? J Interprof Care. 2015;29(5):507–8.

Carraccio C, Englander R, Gilhooly J, Mink R, Hofkosh D, Barone MA, Holmboe ES. Building a framework of entrustable professional activities, supported by competencies and milestones, to Bridge the educational continuum. Acad Medicine: J Association Am Med Colleges. 2017;92(3):324–30.

Swick S, Hall S, Beresin E. Assessing the ACGME competencies in psychiatry training programs. Acad Psychiatry: J Am Association Dir Psychiatric Resid Train Association Acad Psychiatry. 2006;30(4):330–51.

Johnson T, John NJ, Lang M, Shelton PG. Accrediting graduate medical education in psychiatry: past, present, and future. Psychiatr Q. 2017;88(2):235–47.

Young JQ, Hasser C, Hung EK, Kusz M, O’Sullivan PS, Stewart C, Weiss A, Williams N. Developing End-of-Training entrustable professional activities for psychiatry: results and methodological lessons. Acad Medicine: J Association Am Med Colleges. 2018;93(7):1048–54.

Huang M-F, Yeh Y-C, Liu T-L, Chou W-P, Wang P-W, Chen C-S. Perceptions on entrustable professional activities (EPAs) in psychiatry among specialty trainees. J Med Educ. 2022;26(4):33–43.

McPherson S, Reese C, Wendler MC. Methodology update: Delphi studies. Nurs Res. 2018;67(5):404–10.

Hennus MP, Jarrett JB, Taylor DR, Ten Cate O. Twelve tips to develop entrustable professional activities. Med Teach. 2023;45(7):701–7.

Hennus MP, Holzhausen Y, Ladenheim RI, Nel D, Pinilla, S. Approaches to identifying and elaborating entrustable professional activities. In: ten Cate O, Burch VC, Chen HC, Chou FC, Hennus MP. (Eds). Entrustable Professional Activities and Entrustment Decision-Making in Health Professions Education, Chapter 9,101–10.

Andreou V, Peters S, Eggermont J, Schoenmakers B. Co-designing entrustable professional activities in general practitioner’s training: a participatory research study. BMC Med Educ. 2024;24(1):549.

Mann S, Truelove AH, Beesley T, Howden S, Egan R. Resident perceptions of Competency-Based medical education. Can Med Educ J. 2020;11(5):e31–43.

Matsuyama Y, Nomura O, Oikawa S, Kikukawa M, Shimizu I, Gomi H. Competency-based medical education guidelines are context-based: lessons from National guidelines in five countries. Med Teach. 2024;46(sup1):S38–45.

Li SM, Chang YW, Lin MN, Lin CW, Lin CH, Li CC, Lin MH, Chang YF, Sun ZJ, Liaw FY, et al. Importance and fitness of family medicine milestone project from America to Taiwan. BMC Med Educ. 2025;25(1):120.

Frank JR, Snell L, Englander R, Holmboe ES. Implementing competency-based medical education: moving forward. Med Teach. 2017;39(6):568–73.

Taylor DR, Park YS, Egan R, Chan MK, Karpinski J, Touchie C, Snell LS, Tekian A. EQual, a Novel Rubric to Evaluate Entrustable Professional Activities for Quality and Structure. Academic medicine: journal of the Association of American Medical Colleges 2017, 92(11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 56th Annual Research in Medical Education Sessions):S110-s117.

Stephan A, Cheung G, van der Vleuten C. Entrustable professional activities and learning: the postgraduate trainee perspective. Acad Psychiatry: J Am Association Dir Psychiatric Resid Train Association Acad Psychiatry. 2023;47(2):134–42.

Richardson D, Kinnear B, Hauer KE, Turner TL, Warm EJ, Hall AK, Ross S, Thoma B, Van Melle E. Growth mindset in competency-based medical education. Med Teach. 2021;43(7):751–7.

Pinilla S, Lenouvel E, Strik W, Klöppel S, Nissen C, Huwendiek S. Entrustable professional activities in psychiatry: A systematic review. Acad Psychiatry: J Am Association Dir Psychiatric Resid Train Association Acad Psychiatry. 2020;44(1):37–45.

Ten Cate O, Taylor DR. The recommended description of an entrustable professional activity: AMEE guide 140. Med Teach. 2021;43(10):1106–14.