e-Delphi Technique in Postgraduate Registered Nursing Education and Competency Development: A Scoping Review

Abstract

The Delphi technique has become an increasingly popular method to assess and clarify competencies in nursing research as it has been recognized for its ability to capture a profession's collective or implicit knowledge. The Delphi technique is a research method that draws upon the knowledge of a group of experts to review specific content on a subject and reach consensus. One area of research in which inherent knowledge is difficult to assess is the development of clinical nurse instructor competencies (CNIC). Currently, there are no defined competencies for clinical nurse instructors (CNIs) in Canada. This lack of standardized competencies to prepare CNIs poses a threat to the consistency, continuity, and delivery of quality clinical education. Here, Bownes and Giannotti explore the use of the e-Delphi technique to support the future development of CNICs in postgraduate nursing education and competency development.

The Delphi technique has become an increasingly popular method to assess and clarify competencies in nursing research (Schoenly, 2015) as it has been recognized for its ability to capture a profession's collective or implicit knowledge (De Clercq et al., 2011). The Delphi technique is a research method that draws upon the knowledge of a group of experts to review specific content on a subject and reach consensus (Hasson et al., 2000). One area of research in which inherent knowledge is difficult to assess is the development of clinical nurse instructor competencies (CNIC). For this scoping review, a clinical nurse instructor (CNI) is defined as a registered nurse (RN) hired to teach the clinical component of the curriculum in a nursing program. Currently, there are no defined competencies for CNIs in Canada. This lack of standardized competencies to prepare CNIs poses a threat to the consistency, continuity, and delivery of quality clinical education. To help bridge this gap, the use of the e-Delphi technique (a modified version of the Delphi technique) was explored to support the future development of CNICs in postgraduate nursing education and competency development.

The e-Delphi technique methods are the same as in the Delphi technique; however, in eDelphi, all steps in the process are conducted electronically (Keeney et al., 2011). This method was chosen as a suitable methodology to support the future development of CNICs because the eDelphi technique can be used in large geographical areas, expert participants can remain anonymous and can engage in the process on their own schedule, and it is currently used in health care research because of its consensus-finding method. The e-Delphi technique was successfully used by the World Health Organization (WHO) for the similar goal of developing Nurse Educator Core Competencies (NECC) in 2016. Key concepts, types of evidence, and inconsistencies related to the use of e-Delphi in the context of nursing competency development were mapped in this review (Colquhoun et al., 2014).

Background and Significance

Competence, as defined by the International Council of Nurses, is the "ongoing ability of a nurse to integrate and apply the knowledge, skills, judgment, and personal attributes required to practice safely and ethically in a designated role and setting" (2006, p. 2). In a baccalaureate nursing program, essential knowledge, skills, and abilities must be taught in both theory and practice settings (Bownes & Freeman, 2020; Shellenbarger, 2019; WHO, 2016). In Canada, many schools of nursing rely on experienced RNs, hired for their specialized knowledge, to teach students the practical components of the curriculum (Bownes & Freeman, 2020; Canadian Association of Schools of Nursing [CASN], 2016; Hewitt & Lewallen, 2010). CNIs have complex roles as they attempt to help students make theory to practice connections. Although the expectations and responsibilities associated with this role are enormous, CNIs often lack formal education in teaching (Billings & Halstead, 2016). The absence of consistency or regulation of CNI education on a national level has led to significant variations in their preparation across Canada (Bownes & Freeman, 2020).

In recognizing the need for standardization, the WHO (2016) established minimum competencies required for RNs to become nurse educators. To fulfil the role as a nurse educator, nurses must be a graduate of a recognized nursing program, hold a current licence to practice nursing, have completed at least two years of full-time clinical practice within the last five years of their career, and have formal preparation as an educator (WHO, 2016). Adherence to the guidelines established by the WHO is critical to ensure students develop essential nursing knowledge, skills, and abilities. Little research is available to determine whether CNIs undergo formal teaching preparation; in Canada, the minimum requirement is generally to have a higher degree than the students they are instructing (Canadian Nurses Association [CNA], 2022). The Bloomberg School of Nursing, University of Toronto (2022); the Canadian Nurse Educator Institute (CNEI, 2022); and other universities and organizations offer courses to support the CNI role; however, these courses are not a requirement for employment. CNIs may choose not to pursue formal education as courses/certifications can be expensive and time-consuming, and some may not be accessible to all as many are focused on RNs with five years' experience or less (CNEI, 2022). This lack of uniformity in CNI preparation has many implications for the role, including decreased job satisfaction, poor retention rates, inconsistent delivery of the curriculum, and varying knowledge levels for graduates, and ultimately it impacts the quality and safety of patient care (Bownes & Freeman, 2020).

Delphi Use in Nurse Educator Competency Development

In 2016, the WHO used a global Delphi method to develop their NECC. While this was an excellent initiative, the focus was on full-time nurse educators in faculty positions teaching in classroom settings. The National League of Nursing (NLN) in the United States has developed programs to delineate the role of the CNI as a specialty area (Shellenbarger, 2019). However, standardized CNI competencies have not yet been established in the Canadian context. In Canada, the CNA (2022) offers certification for 22 nursing specialties; however, CNIs is not one of them. As a first step towards research that will support CNI competency development and subsequent specialty designation in Canada, implementation of the e-Delphi method can be useful. It entails sending multiple rounds of questionnaires to a panel of anonymous experts. The anonymous responses are combined and returned to the expert panel after each round (Keeney et al., 2001) so that experts can adjust their answers based on how they interpret group responses. This process is designed to generate a true consensus of what the group thinks without risk of bias (Keeney et al., 2011). Given the successful application of the Delphi method in developing nurse educator competencies for in-class instruction (WHO, 2016), the e-Delphi technique was examined to determine its feasibility for establishing CNI competencies. The e-Delphi was specifically chosen over the Delphi for its flexibility, especially given the technological advances that have occurred in the last few years during the pandemic.

The purpose of this scoping review was to understand the extent, range, and nature of the evidence for the use of the e-Delphi technique and to critically appraise its possible use in postgraduate nursing education and competency development to determine whether e-Delphi is a feasible method for developing CNIC in Canada. The aims were to (a) describe and analyze how the e-Delphi technique was used in nursing for competency development, (b) identify gaps in the use of the method, and (c) establish the feasibility of its application to CNIC development in Canada.

Search Strategy

A search of PubMed, Cumulative Index to Nursing (CINAHL), ProQuest, Ovid MEDLINE, and Scopus was undertaken in collaboration with the designated faculty of nursing librarian liaison. The purpose of this collaboration was to help ensure both the rigour and the comprehensiveness of the literature search. Pertinent articles were identified to establish an operational definition of the e-Delphi technique. Articles dated between January 2011 and March 31, 2021, were included in the search. Text/words found in the titles and abstracts of relevant articles and index terms used to describe the articles were used to develop a full search strategy. The search strategy, including all identified keywords and index terms were generated and entered in each database. Key search terms included "Delphi," "nursing," "education," and "competency." Truncation was used in the terms "nursing" (nurs·) and "education" (edu·) to capture all possible permutations.

Inclusion and Exclusion Criteria

Articles included in this scoping review were required to meet the following criteria: they were published in English; published between January 2011 and March 31, 2021; and peer reviewed, and the e-Delphi technique was explicitly used in studies that focused on postgraduate RN competency development or education. Studies that focused on the development of curricula for students in undergraduate programs or pre-licensure education were excluded as they did not fit the aims of the review, as were studies involving patients or members of other health care disciplines outside nursing as experts. Alternative or adapted versions of the e-Delphi were also excluded. Finally, studies that did not meet the basic structure of the e-Delphi technique were excluded. Other exclusions were text and opinion papers, unpublished manuscripts, and guidelines for competencies as they did not fit the aims of the review.

Methods

Study Selection

Following the literature search, all results were uploaded into the citation manager Zotero (Roy Rosenzweig Center for History and New Media, 2016), and then imported into the systematic review manager Covidence (Covidence Systematic Review Software, 2021). Duplicates were removed. The inclusion and exclusion criteria discussed above were formatted into the software a priori and were used for level one screening (titles and abstracts) and level two screening (full text). Two independent reviewers screened titles and abstracts for assessment against the inclusion criteria for the review. Potentially relevant sources were retrieved in full and assessed in detail against the inclusion criteria. The reasons for exclusion of sources in full-text review were included in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) by Tricco et al. (2018) in Table 1. Any discrepancies between reviewers were resolved through deliberation.

Data Extraction

A standardized data extraction form was used and adapted by using recommended headings from the Joanna Briggs Institute (2015), as well as headings that informed the review objective to elicit the key information from the chosen studies. The form was used to chart data from the selected literature sources and adapted into a table (see the Appendix). As e-Delphi is generally used as one part of a mixed methods study, only the key results pertaining to use of e-Delphi in each study were reported.

Results

After duplicates were removed, 265 articles were identified. Two hundred and six articles were removed after level one screening of titles and abstracts, resulting in 59 articles for full-text review. Full-text review excluded 51 articles for wrong population, wrong outcome, wrong study design, no report of consensus, and incorrect use of e-Delphi. Upon completion of level two screening of full text, eight articles met the inclusion criteria (Table 1). One study was from Canada, one from Italy, three from the United States, one from Belgium, one from Australia, and one from Slovenia. The eight remaining sources were analyzed for similarities and key findings in how the researchers used e-Delphi in their competency development. The main components of the e-Delphi method were extracted related to the review aims. This included the study purpose, methods of e-Delphi used, background of expert panel members, reported level of consensus, number of rounds to meet consensus, time between rounds of questions, and number of participants/changing participants.

Study Purposes

Seven of the eight studies reviewed focused on competencies in the clinical setting, while one (De Clercq et al., 2011) focused on competency development in a postgraduate master of nursing program. The specialty units for the studies included burn nursing (Carrougher et al., 2018), cardiac nursing (Bagnasco et al., 2021), critical care nursing (Gill et al., 2017), and medicalsurgical areas (Boyer et al., 2020; Burke et al., 2017). Two were broader as they developed universal nursing competencies (Ličen & Plazar, 2019) and health literacy competencies (Toronto, 2016) by surveying experts.

Methods of e-Delphi Used and Background of Expert Panel Members

All studies reviewed focused on using the e-Delphi technique for competency identification, development, validation, or revision with the objective of supporting continuing postgraduate education for RNs in a specific practice setting. All articles reported their methods for recruiting expert panels through purposive sampling. Panel participants were recruited by contacting hospitals, expert clinicians, advisers, educators, and literature reviews of published experts. Finally, all eight studies were conducted by RNs for postgraduate RN education and competency development or revision.

Bagnasco et al. (2021) and Ličen and Plazar, (2019) both used e-Delphi to identify or develop competencies for a specific group of RNs, where no prior competencies for practice or education existed. In both studies, with competency identification and development as the aim, the classic e-Delphi technique of seeking input from experts in round one was used. This was done as an open-ended survey to generate competencies from the experience of the experts and from the literature. This is the classic use of e-Delphi.

Four of the studies sought to validate or revise an existing set of competencies; Boyer et al. (2020), Burke et al. (2017), Gill et al. (2017), and Toronto (2016). These researchers chose to start the modified e-Delphi with the distribution of pre-determined competencies. The two remaining studies borrowed from both the classical and the modified e-Delphi, which leads to a new hybrid of e-Delphi. This hybrid model falls somewhere between the classical and modified techniques. However, combining the two techniques obscures the method being followed. While the purpose of the Carrougher et al. (2018) study was to establish burn nurse competencies, the researchers chose to replace the idea generation round with a survey round to gather existing burn nurse competencies.

De Clercq et al. (2011) used a set of existing competencies even though their purpose was to identify a new set of competencies. With this research being within the educational realm, they sought existing international competency profiles of master-level nursing education and adapt it to the Flemish context. Many of the sources use pre-existing competencies to inform new competencies. Additionally, all studies in this scoping review used mixed methods designs to strengthen competencies or education plans. Researchers used Likert scales and open text fields to elicit feedback in their surveys and to rate the competencies. All the extracted studies identified the use of electronic survey platforms or e-mail to distribute and collect their data.

Reported Level of Consensus

Each study reported its own level of consensus, with a broad range of 51% to 90% being acceptable. Burke et al. (2017) and Carrougher et al. (2018) chose 51% as their accepted level of consensus, while De Clercq et al. (2011) and Toronto (2016), chose 90% as their definition of consensus. The other four studies defined consensus as 70% (three studies) and 80% (one study).

Number of Rounds to Meet Consensus

The number of rounds to meet consensus in this review ranged from two to four rounds, with the most common number being three. Carrougher et al. (2018) asked participants to submit existing competencies in round one so that the first round had a starting point (working with existing competencies). The number of rounds to reach consensus is specific to the number of participants and the response rate. The classical Delphi uses four rounds (Hsu & Sandford, 2007). The remaining studies were still within an appropriate range, and researchers may modify the rounds to meet individual research aims. Some of the studies counted experts meeting ahead of time to generate a list of competencies as the first round and others did not. The rounds are complete when consensus is met on the competencies, so this is never pre-determined in a study.

Time Range between Survey Rounds

The range of timelines between the e-Delphi rounds in the selected articles varied considerably from two weeks (Boyer et al., 2020) to two years or 104 weeks (Bagnasco et al., 2021; Carrougher et al., 2018). The studies that reported a two-year timeline addressed this as a potential limitation, but they valued the expert opinion and preferred to grant more time to the busy professionals to decrease their study attrition rate (Bagnasco et al., 2021).

Number of Participants and Changing Participants

The literature was ambiguous about the sample size needed for an e-Delphi study. According to Keeney et al. (2011), a consensus should consist of 15 to 30 participants from the same discipline. All studies sustained the recommended number of participants throughout their rounds, except Burke et al. (2017), which had 13 participants in the first round and 10 in the second round.

Discussion

This scoping review identified that multiple versions of the e-Delphi technique are in use today for nursing competency development. Modifications of the e-Delphi technique, however, pose a threat to the credibility, validity, and reliability of the results. Although this inconsistency may occur in many types of surveys or interview-based research, it could be a threat to the uniformity of method in e-Delphi (Keeney et al., 2001). Because of the nature of e-Delphi methodology (qualitative, quantitative, and/or mixed methods), psychometric properties should not be the measures used to interpret data as they are grounded in the positivist lens. Rather, criteria such as transferability, credibility, applicability, or confirmability of findings are much more relevant (Keeney et al., 2001).

Regardless of the version of e-Delphi employed by researchers in their study, this technique was successfully applied to achieve consensus through expert panels by determining, predicting, and exploring group attitudes, needs, and priorities (Keeney et al., 2001). All studies in this review successfully developed competencies for postgraduate RNs. These studies demonstrate the feasibility and utility of e-Delphi methodology in the development of competencies for ongoing nursing education. The accessibility of e-Delphi as a research method allows experts to provide feedback on their own time, without the added risk of bias that can occur when participants are face to face. It can also allow geographically diverse participants to contribute, thereby strengthening the findings by including experts who might not otherwise have been able to participate.

Types of e-Delphi

The classical e-Delphi technique consists of open-ended questions as an idea generation tactic to which an expert panel responds in the initial round of the survey (Keeney et al., 2001, 2011). The feedback from the open-ended responses informs the subsequent rounds. A disadvantage of an open-ended first round of the survey is that it can create an unwieldy number of items for the next round, which can overwhelm the participants, be time-consuming, be costly, and increase attrition (Keeney et al., 2001). Therefore, the modified e-Delphi technique, in which a pre-generated list of items for ranking is employed in the first round, has become widely used. It is acceptable and common to use a structured questionnaire that is based on an extensive review of the literature in round one (Hsu & Sandford, 2007).

This modification of the e-Delphi becomes apparent only after careful comparison between the two kinds of studies and their purposes in this review. Using round one as an idea-generating round or using it to distribute an existing set of competencies are both acceptable uses of e-Delphi according to Hsu and Sandford (2007). The scoping review found that the e-Delphi technique was not used consistently in the eight studies. The subtle nuances of the e-Delphi technique can partially account for researchers' varied application; however, modifications of the e-Delphi technique should be cautioned against. Variations in the application of the e-Delphi may lead to methodological problems. To decrease confusion and increase validity with future e-Delphi studies, it is recommended that researchers delineate the purpose in their title and remain true to their chosen method.

Expert Panel Selection

There are no universally established criteria for selecting the expert participants in an eDelphi study (Keeney et al., 2006). In the literature, it is agreed that they should be familiar with the research topic and willing to provide their expertise throughout the rounds of the survey (Keeney et al., 2001). However, there is debate as to what constitutes an expert in the Delphi technique, which can affect the results of the study (Keeney et al., 2001). A consideration when using the e-Delphi technique is ontological bias in the non-random sampling. This can occur as expert panelists often have a vested interest in participating and staying involved in the study rounds as they progress (Keeney et al., 2001). To increase the validity when implementing the eDelphi technique, it is imperative that researchers conduct background assessments of prospective expert participants including demographics, credentials, and extent of knowledge on the chosen topic. One consideration is the non-random nature of the sampling technique for Delphi. This can be a limitation as participants in the expert panel may be known to one another. To address this issue, some researchers ensured participants were from geographically diverse areas to reduce bias and the likelihood of knowing one another, demonstrating yet another strength of having geographically diverse participants.

Number of Participants

The rates of participant attrition between rounds of questions can threaten validity when using e-Delphi. To reduce this concern, having approximately 15 to 30 respondents from the same discipline is recommended (Keeney et al., 2011); fewer experts are required if the group is considered homogenous (Hansen, 2006, as cited in Bromley, 2015). However, sample size is contingent upon the purpose of the project, design chosen, and time allotted for data collection (Keeney et al., 2001).

All studies in this scoping review sustained an acceptable number of participants through their rounds; however, it is important to note that in some studies, participants changed between rounds (i.e., people who did not participate in round one then participated in round two). This can be as a limitation as it can threaten validity or skew the results of the study. When new members are introduced in rounds two and beyond, the study outcome changes inevitably as participants entering partway through the process are unaware of previous discussions. Accordingly, it is recommended to keep participants consistent in each of the rounds to limit this concern.

Level of Consensus and Number of Rounds

The objective of e-Delphi is to reach consensus by finding the index of central tendency or the most frequently agreed upon response to a survey item (Bromley, 2015). Researchers must establish and define their level of consensus before collecting the data (Keeney et al., 2011), a critical design element that cannot deviate. Reported levels of consensus varied significantly across the eight studies in this scoping review. While recommendations in the literature range from 51 % to 80% agreement for the items on the survey (Keeney et al., 2011), a wide variability may lead to oversights in competency development if the lower end of the scale is used. Therefore, it is recommended to set a high consensus percentage of 80% to 90% to help ensure the credibility of the findings.

Time between Rounds

An extended time between rounds can potentially threaten rigour in the e-Delphi process. Two of the studies in this scoping review continued with the second round two years after completion of round one (Bagnasco et al., 2021; Carrougher et al., 2018). An extended period between rounds makes it questionable whether participants can recall why or what was said in the previous round. Software has been developed that uses real-time Delphi, which relies on rapid completion of the rounds by the expert panels within a specific time after a survey is open (Gordon & Pease, 2006). Use of this software would align with our recommendation of keeping the shortest amount of time possible between rounds.

Feasibility of the e-Delphi Technique

There are many advantages to using the e-Delphi technique. One advantage is that researchers can choose from a variety of statistical analysis to represent the data (Dalkey, 1972). This can ensure each participant is represented in the final version, thereby reducing the risk of social desirability bias (Hsu & Sandford, 2007). Another advantage of e-Delphi is accessibility in the development of consensus. Having access to experts on a national or global scale to confirm or develop competencies in a short time makes this a favourable method, especially in a geographically vast country such as Canada. For example, the use of the Delphi methodology was highly effective in the development of NECC by the WHO in 2016, demonstrating the ability to recruit participants on a global scale. This process helped ensure that the established competencies were truly reflective of what was happening in the field of nursing globally. The WHO was transparent in reporting the details of the Delphi process, outlining the intricate consultative process that was followed at each step. This transparency in reporting lends credibility to the core competencies established using this research method.

Gaps in the Use of e-Delphi

A gap discovered in the use of e-Delphi is the lack of grounding the specific competencies in theory. It was identified by O'Brien et al. (2015) that researchers must review the literature to identify appropriate theories and frameworks. This is essential to ascertain what needs to be measured and helps to avoid the common error of using competencies based simply on what has been done in the past. A reliable theoretical foundation is necessary to identify meaningful competencies that are also current practice (O'Brien et al., 2015). Only one article, Boyer et al. (2020) reported a competency framework that was being adapted and validated. This is the only Canadian study that was included in the scoping review. It was unclear in the literature review portion of the remaining studies whether the competencies were grounded in a particular theory or framework as that detail was not reported. Other questions emerged during the data analysis that speak to the epistemology or ways of knowing within e-Delphi. The use of implicit knowledge of the expert participants raises the question of its legitimacy. In the end, the readers must trust that this knowledge has merit and, therefore, carries authority to inform the answers that are being sought. From an epistemological standpoint, the use of a non-random sampling technique may never result in radical change in approaches as the sample could be inclined to yield a conservative group of established members from one profession.

Conclusion

There is a critical need to develop and standardize competencies for CNIs in Canada. Standardizing CNI competencies can help ensure consistency, continuity, and the delivery of safe, competent clinical education. There are several benefits to using e-Delphi to develop standardized CNI competencies including improving accessibility, reducing geographical barriers to recruit experts, completing surveys when convenient for the participant, and reducing social desirability bias. While there are many variations of the e-Delphi technique in use today, strict adherence to the protocols, along with transparency in the research process, to enhance credibility of the results is recommended. This scoping review demonstrates the feasibility of the e-Delphi technique as a practical methodology to support future CNIC development in Canada.

Sidebar

Recommended Citation

Bownes, Natalie A. and Giannotti, Natalie (2023) "e-Delphi Technique in Postgraduate Registered Nursing Education and Competency Development: A Scoping Review," Quality Advancement in Nursing Education - Avancées en formation infirmière: Vol. 9: Iss. 1, Article 3.

DOI: https://doi.org/10.17483/2368-6669.1348

References

References

Bagnasco, A., Barisone, M., Aleo, G., Watson, R., Catania, G., Zanini, M., Thompson, D. R., & Sasso, L. (2021). An international e-Delphi study to identify core competencies for Italian cardiac nurses. European Journal of Cardiovascular Nursing, 20(J\ 684-691. https://doi.org/10.1093/eurjcn/zvab003

Billings, D. M., & Halstead, J. A. (2016). Teaching in nursing: A guide for faculty (5th ed.). Elsevier.

Bloomberg Faculty of Nursing, University of Toronto. (2022). The foundations and scholarship of clinical teaching, https://bloomberg.nursing.utoronto.ca/pd/courses/nursing-educationcertificate/#content2

Bownes, N. A., & Freeman, M. (2020). Clinical nurse instructor competencies: An exploratory study of role requirements. Quality Advancement in Nursing Education-Avancées En Formation Infirmière, 6(3). https://doi.org/10.17483/2368-6669.1226

Boyer, L., Pepin, J., Dubois, S., Descoteaux, R., Robinette, L., Dery, J., Brunet, F., Bolduc, J., & Deschenes, M. (2020). Adaptation and validation of a nursing competencies framework for clinical practice on a continuum of care from childhood to adulthood: A Delphi study. Nurse Education Today, 93. https://doi.Org/10.1016/j.nedt.2020.104530

Bromley, P. (2015). Using e-Delphi to identify capability requisites for postgraduate certificate in neonatal intensive care nursing. Journal of Neonatal Nursing, 21, 224-236. https://doi.org/10.1016%2Fj ,jnn.2015.09.003

Burke, K. G., Johnson, T., Sites, C., & Barnsteiner, J. (2017). Creating an evidence-based progression for clinical advancement programs. American Journal of Nursing, 777(5), 22-35. https://doi.org/10.1097/01.NAJ.000Q516248.15946.76

Canadian Association of Schools of Nursing. (2016). Registered nurses ' education in Canada statistics: 2014-2015. Registered nurse workforce, Canadian production: potential new supply, http://www.casn.ca/wp-content/uploads/2016/11/2014-2015-SFS-FINALREPORT-suppressed-updated.pdf

Canadian Nurse Educator Institute. (2022). Certification programs, http://cnei-icie.casn.ca/ourprograms/

Canadian Nurses Association. (2022). Certification nursing practice specialties. https://www.cna-aiic.ca/en/certification/get-certified/certification-nursing-practicespecialties

Carrougher, G. J., Hallowed, K. A., Sproul, J. L., Wiggins, B. J., & Mann-Salinas, E. (2018). Burn nurse competencies: Developing consensus using e-Delphi methodology. Journal of Burn Care and Research, 39, 751-759. https://doi.org/10.1093/jbcr/irx036

Colquhoun, H. L., Levac, D., O'Brien, K. K., Straus, S., Tricco, A. C., Perrier, L., Kastner, M., & Moher, D. (2014). Scoping reviews: Time for clarity in definition, methods, and reporting. Journal of Clinical Epidemiology, 67(12), 1291-1294. https://doi.Org/10.1016/j.jclinepi.2014.03.013

Covidence Systematic Review Software. (2021). Veritas Health Innovation, Melbourne, Australia, www.covidence.org

Dalkey, N. C. (1972). The Delphi method: An experimental study of group opinion. In N. C. Dalkey, D. L. Rourke, R. Lewis, & D. Snyder (Eds.), Studies in the quality of life: Delphi and decision-making (pp. 13-54). Lexington Books.

De Clercq, G., Goelen, G., Danschutter, D., Vermeulen, J., & Huyghens, L. (2011). Development of a nursing practice-based competency model for the Flemish Master of Nursing and obstetrics degree. Nurse Education Today, 31, 48-53. http s : //do i. org/10.1016/j.nedt.2010.03.013

Gill, F. J., Kendrick, T., Davies, H., & Greenwood, M. (2017). A two-phase study to revise the Australian practice standards for specialist critical care nurses. Australian Critical Care, 30(3), 173-181. https://doi.Org/10.1016/j.aucc.2016.06.001

Gordon, T., & Pease, A. (2006). RT Delphi: An efficient, "round-less" almost real time Delphi method. Technological Forecasting and Social Change, 73(4), 321-333. https://doi.org/10.1016/j .techfore.2005.09.005

Hasson, F., Keeney, S., & McKenna, H. (2000). Research guidelines for the Delphi survey technique. Journal of Advanced Nursing (Wiley-Blackwell), 32(4), 1008-1015. https://doi.Org/10.1046/i.1365-2648.2000.01567.x

Hewitt, P., & Lewallen, L. (2010). Ready, set, teach! How to transform the clinical nurse expert into the part-time clinical nurse instructor. The Journal of Continuing Education in Nursing, 41(9), 403-407. https://doi.org/10.3928/00220124-201005Q3-10

Hsu, C.-S., & Sandford, B. A. (2007). The Delphi technique: Making sense of consensus. Practical Assessment, Research and Evaluation, 72(10), 1-8. https://doi.org/10.7275/pdz9-th90

International Council of Nurses. (2006). Position statement: Continuing competence as a professional responsibility and public right.<https://www.twna.org.tw/frontend/un07 international/file/B02> Continuing%20Competen ce.pdf

Joanna Briggs Institute. (2015). The Joanna Briggs Institute reviewers ' manual 2015: Methodology for JB1 scoping reviews. University of Adelaide.

Keeney, S., Hasson, F., & McKenna, H. (2001). A critical review of the Delphi technique as a research methodology for nursing. International Journal of Nursing Studies, 38(2), 195200. https://doi.org/10.1016/s0020-7489(00)00044-4

Keeney, S., Hasson, F., & McKenna, H. (2006). Consulting the oracle: ten lessons from using the Delphi technique in nursing research. Journal of Advanced Nursing, 53(2), 205-212. https://doi.Org/10.l 111/j, 1365-2648.2006.03716.x

Keeney, S., Hasson, F., & McKenna, H. (2011). The Delphi technique in nursing and health research. Wiley-Blackwell. https://doi.org/10.1002/9781444392Q29

Ličen, S., & Plazar, N. (2019). Developing a universal nursing competencies framework for registered nurses: A mixed methods approach. Journal of Nursing Scholarship, 51(4), 459-469. https://doi.0rg/lO.l 111/jnu. 12483

O'Brien, J. E., Hagler, D., & Thompson, M. S. (2015). Designing simulation scenarios to support performance assessment validity. The Journal of Continuing Education in Nursing, 46(U\ 492-498. https://d0i.0rg/l0.3928/00220124-20151020-01

Peters, M. D. J., Godfrey, C., McInerney, P., Munn, Z., Tricco, A. C., & Khalil, H. (2020). Scoping reviews. In E. Aromataris & Z. Munn (Eds.), JBI manual for evidence synthesis. JBL https://doi.org/10.46658/JBIMES-20-12

Roy Rosenzweig Center for History and New Media. (2016). Zotero [Computer software], https://www.zotero.org/download

Schoenly, J. (2015). Research priorities in correctional nursing practice: Results of a three-round Delphi study. Journal of Correctional Health Care, 21(4\ 400^107. https://doi.org/10.1177/1078345815600257

Shellenbarger, T. (2019). Clinical nurse educator competencies: Creating an evidence-based practice for academic clinical nurse educators. Wolters Kluwer.

Toronto, C. E. (2016). Health literacy competencies for registered nurses: An e-Delphi study. The Journal of Continuing Education in Nursing, 47(Y2\ 558-565. https://doi.org/10.3928/00220124-2Q161115-09

Tricco, A. C., Lillie, E., Zarin, W., O'Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., Hempel, S., Aki, E. A., Chang, C., McGowan, J., Stewart, L., Härtling, L., Aidcroft, A., Wilson, M. G., Garritty, C., Lewin, S.,... Straus, S. E. (2018). PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Annals of Internal Medicine, 169(7 J 467^173. https://doi.org/10.7326/M180850

World Health Organization. (2016). Nurse educator core competencies.<https://www.who.int/hrh/nursing midwifery/nurse> educator050416.pdf

(ProQuest: Appendix omitted.)