Developing a valid and reliable assessment of knowledge translation (KT) for continuing professional development program of health professionals

Abstract

Introduction

Knowledge Translation (KT) is expected to be a critical learning outcome of a Continuing Professional Development (CPD) program. It continues to serve as an area of interest among educators and healthcare providers due to its importance to evidence-based practice. This study endeavored to develop a valid and reliable KT learning assessment tool in CPD.

Methods

The Inventory of Reflective Vignettes (IRV), an innovative approach of integrating research vignettes, was utilized in crafting the 20-item IRV-KT tool. This instrument includes knowledge creation and action as essential KT constructs. KT competency was assessed in three segments (i.e., before and after CPD event and if in a lecture) using a one-group post-posttest pre-experimental design. Health professionals who successfully completed a CPD program on a knowledge translation topic were asked to complete the IRV-KT during the pilot study (n = 10) and actual implementation (n = 45). Responses were subjected to Cronbach’s reliability and criterion-validity testing.

Results

The initial test of the IRV-KT tool demonstrated a high internal reliability (α = 0.97) and most items yielded acceptable validity scores. During the actual implementation, a higher reliability score of 0.98 was generated with significant correlations between the before-after segments for both KT constructs of creation (r = 0.33, p < 0.05) and action (r = 0.49, p < 0.05). All items have significant positive validity coefficients (r > 0.35, p < 0.05) in all segments of the tool.

Discussion

The study produced a reflective assessment tool to validly and reliably assess KT learning in a CPD. IRV-KT is seen to guide the curriculum process of CPD programs to bridge learning and healthcare outcomes.

Introduction

Knowledge Translation (KT) has been prominent in the health fields (Lang, Wyer & Haynes, 2007; Lencucha, Kothari & Rouse, 2007; Armstrong et al., 2013), and its primary focus has been mostly on practice and research, with some focus on educational and professional development (Urquhart et al., 2013). As a learning approach, KT has no agreed upon conceptual framework and platform (Pablos-Mendez & Shademani, 2006). However, KT seems to have a resemblance to (a) constructivist learning theory (Thomas et al., 2014) on the premise that learners construct new knowledge, as well as (b) outcomes-based education on transforming the declarative knowledge into functioning knowledge in a professional education (Biggs & Tang, 2007; Biggs & Tang, 2011). These principles put forward the importance of integrating knowledge translation in any teaching and learning activity such as in Continuing Professional Development (CPD). There are limited embedded assessments that would capture its natural occurrence in various learning contexts, more so in a CPD environment. Thus, this study worked toward the development of a valid and reliable assessment tool to use in measuring KT competency as a CPD learning outcome. For instance, nurse managers who aspire to conduct KT CPDs as part of their in-service program shall benefit from the tool, as well as academicians who need a valid and reliable instrument to measure trainees’ achievement of KT training outcomes.

Foundation of knowledge translation (KT)

The universal “know-do” gap (i.e., the difference and gap between theory, knowledge, evidence or research, and practice, application, policy or decision) remains a critical challenge even in the contemporary world (Pablos-Mendez & Shademani, 2006; Santesso & Tugwell, 2006; Mustard, 2010; Grimshaw et al., 2012; Atchan, Davis & Foureur, 2014; Health Policy Project, 2014; Zhang et al., 2015). Although knowledge possesses an inherent catalytic potential in advancing the globalized society such as inclusive knowledge societies and knowledge-based economy (CEDEFOP, 2014; Pablos-Mendez & Shademani, 2006; Souter, 2014; UNESCO-IBE, 2013), it requires meaningful and contextual applications to elicit desired outcomes (Santesso & Tugwell, 2006; Bassi et al., 2013). Both CPD and KT offer mutually reinforcing strategies (Graham et al., 2006; Davis & Davis, 2010; Kitto et al., 2013) in placing knowledge to work. In essence, this stresses three-sided (i.e., researchers, educators and practitioners) collaboration (Bjørk et al., 2013) that is similar to the European Commission’s (2015) knowledge triangle—education, research, and innovations.

Several authors (Ho et al., 2004; Wallin, 2009; Metzler & Metz, 2010; Kothari & Armstrong, 2011; Straus, Tetroe & Graham, 2011; Sibley et al., 2011; Collisson et al., 2011; Kastner & Straus, 2012; Muntaner et al., 2012; Wolfe et al., 2012; Bassi et al., 2013; Gholami et al., 2013; Urquhart et al., 2013; Bjørk et al., 2013) have espoused the widely accepted Canadian Institutes of Health Research’s (2014) definition of KT as a continuous knowledge processing (i.e., synthesis, dissemination, exchange and application) within a complex interactive system of researchers and knowledge users. Correspondingly, knowledge-to-action (KTA) process (Graham et al., 2006) generally illustrates KT that offers an alternative framework to bridge the know-do gap in the 21st century. It emphasizes the cyclic coexistence of knowledge creation and application. During the creation phase, knowledge needs to undergo a refinement process such as inquiry, synthesis and development. Whereas, the action part involves activities needed for actual knowledge application such as adaptation, intervention, evaluation, and continuation.

Graham et al. (2006) clarified commonly shared concepts of KT. Knowledge transfer exclusively describes a linear top-down approach (Visram, Goodall & Steven, 2014), and one-way (or impliedly can be two-way) process of getting knowledge with no actual application. This concept reflects a mere sending of information to a user (Majdzadeh et al., 2008), though Froese & Montgomery (2014) have viewed knowledge transfer synonymously with knowledge translation. These views concur with Choi & Johanson’s (2012) comparison of the (a) static, (b) dynamic and (c) knowledge translation perspectives of knowledge transfer, respectively. In the same way, knowledge exchange as a two-way process (Visram, Goodall & Steven, 2014) has been conceived to release some restrictions of transfer. Knowledge exchange expands the transfer concept by emphasizing the collaborative roles between the researchers, leaders, and stakeholders. Though this concept expansion, relevant knowledge is available, although involvement may vary throughout the process. Several studies have compounded these terms as knowledge translation and exchange (KTE) to further strengthen the involved interactive mechanisms (Légaré et al., 2011; Wolfe et al., 2012; Visram, Goodall & Steven, 2014). On the contrary, research utilization (or implementation) directs its effort on the sole use of research evidences. This concept shares almost a similar stand with evidence-based practice (EBP) that uses the best evidence as the ideal knowledge in guiding decision and practice (Wallin, 2009; Metzler & Metz, 2010; Atchan, Davis & Foureur, 2014). Lastly, translational research may seem similar, yet it deals with the transfer of basic scientific knowledge to clinical application. In contrast, Grimshaw et al. (2012) embraced the view of translation research as their frames for knowledge translation. This view refers to the second type of translational research, which deals with improving health outcomes using new clinical knowledge. In spite of their differences, these concepts are subsumed under the broader context of KT.

Strategies of Knowledge Translation (KT)

Despite availability of many theoretical supports and models (Sibley et al., 2011; Brehaut & Eva, 2012; Larrivée, Hamelin-Brabant & Lessard, 2012; Thomas et al., 2014), there has been no single strategy for KT (Metzler & Metz, 2010; Gholami et al., 2011; Grimshaw et al., 2012; Kastner & Straus, 2012; Visram, Goodall & Steven, 2014). At large, the Canadian Institute of Health Research (Canadian Institutes of Health Research, 2014) has specified two (2) general KT approaches. The traditional (Kothari & Wathen, 2013) end-of-grant knowledge translation empowers the researcher to make the stakeholders (e.g., users and clients) and their peers aware of the research evidence as knowledge. This translation may involve typical research dissemination (e.g., conferences, presentations, and publications) and intensive activities such as interactive sessions, educational events and knowledge brokering. As for the integrated knowledge translation (IKT), the knowledge researcher and user engage in a meaningful collaboration toward sustainable outcomes. Its strategies may include action research, knowledge co-creation and participatory methods (Tetroe, 2007). Likewise, Collisson et al. (2011) have pointed out that these KT approaches share a direct link with their knowledge production mode. The independent knowledge production leads to the academic merits of end-of-grant KT, while collaborative generation facilitates integrated KT for a direct contextual application.

Another KT strategy integrates modern technology to yield a Technology-Enabled Knowledge Translation (TEKT) (Bassi et al., 2013; Urquhart et al., 2013). This strategy embodies the convergence of technology and innovation with KT, which supports information collection, production and distribution (Pablos-Mendez & Shademani, 2006; Majdzadeh et al., 2008; Urquhart et al., 2013). Several scholars (Ho et al., 2004; Bassi et al., 2013) have maintained positive views on TEKT’s potentials for online education, web-based intervention, and integrated systems. Similarly, informatics-based knowledge translation optimizes the utilization of information systems for adjunctive mechanisms such as clinical decision support systems and healthcare information systems (Kastner & Straus, 2012). Relatedly, social media open up new opportunities in reinforcing knowledge translation activities and interactions (Hamm et al., 2013). In other situations, the social environment becomes the strategic center of knowledge translation such as the community of practice (Lencucha, Kothari & Rouse, 2007; Urquhart et al., 2013), virtual community (Bassi et al., 2013) and community-based settings (Kothari & Armstrong, 2011).

Interestingly, most KT strategies tend to hold educational activities (Zwarenstein & Reeves, 2006; Majdzadeh et al., 2008; Wallin, 2009; Reitmanova, 2009; Davis & Davis, 2010; Straus et al., 2011; Wahabi & Al-Ansary, 2011; Grimshaw et al., 2012; Larrivée, Hamelin-Brabant & Lessard, 2012; Armstrong et al., 2013; Bassi et al., 2013; Thomas et al., 2014). Regardless of whether a standalone or combined intervention is used, KT activities may include learning activities (e.g., seminar, lectures, workshops, outreach, trainings and meetings) (Graham et al., 2006; Wahabi & Al-Ansary, 2011; Grimshaw et al., 2012; Armstrong et al., 2013; Bassi et al., 2013; Urquhart et al., 2013) and CPD (Grimshaw et al., 2006; Pablos-Mendez & Shademani, 2006; Santesso & Tugwell, 2006; Davis & Davis, 2010; Metzler & Metz, 2010). Under these circumstances, KT and CPD can be overlapping and confusing (Graham et al., 2006). Kitto et al. (2013) expounded the harmonies, distinctions, and associations of these academic domains. Both KT and CPD aspire to improve collaborations, services, and outcomes, though their underlying approach may differ. CPD gives attention to developing competence and performance of health professionals (Enriquez et al., 2015), whereas KT ensures quality information and tools for the stakeholders. Despite their differences, KT and CPD maintain a mutual relationship and share common challenges. KT adapts theoretical foundations from CPD, while CPD exploits KT as a strong rationale for its strategies.

Subjects and Methods

In pursuit of developing a reflective tool for assessing KT learning in a CPD program, this study embarked on utilizing the tool development framework of IL Ong, MJS Diño, MMP Calimag & FA Hidalgo (2017, unpublished data). In their work, they developed the Inventory of Reflective Vignettes (IRV) to guide them in designing a valid and reliable tool for interprofessional learning. Reflective tests utilize vignettes or hypothetical scenarios for reflection (Datta Gupta, Kristensen & Pozzoli, 2010; Hudson & Cairns, 2014; Gesinde, Temitope & David, 2014) that are proven to be highly correlated with the actual outcomes (Colón-Emeric et al., 2018). It can propel concrete thinking and elicit opinions (Santos-Eggimann & Meylan, 2017). The IRV integrated a six-point Likert scale for reflective measurement of desired competencies (a) prior CPD, (b) upon CPD completion, and (c) during a vignette situation. Following the tool development process, the content design involves literature reviews (Shrader et al., 2017) in identifying key constructs of KT and corresponding items. The designed tool went through expert review (n = 5) for face validation. Similarly, a one-group post-posttest pre-experimental design was adopted in this study to establish the validity and reliability measures of the designed KT assessment tool. Post-posttest design refers to the assessment wherein the learners are tasked to reflect on their prior and current competency level (MacDonald et al., 2010). Implementation of this design avoids bias by eliminating pre-assessment survey, but pre-assessment items were included as part of the post-assessment survey (Bottenberg et al., 2013). In addition, the IRV-KT tool primarily intends to provide a self-assessment of KT competency through reflective learning. The study focused on establishing tool validity and reliability for its potential use in any CPD program. As for the assessment of the learning outcomes, the CPD program had its own authentic performance tests with corresponding scoring rubrics.

After the research protocol was approved by an accredited ethics review committee of the Philippine Health Research Ethics Board, the pilot study (n = 10) was conducted on November 2017 in a university-offered CPD program using the exploratory method of teaching for health professionals. In this study, health professionals refer to licensed physicians, nurses and allied health workers like physical therapists and pharmacists. All respondents gave written consent prior the CPD program and their privacy was maintained throughout the study. The pilot sample consisted of nurses pursuing postgraduate degrees with equal distribution in terms of gender. The initial Cronbach’s analysis (Table 1) underscores the high internal reliability (α = 0.97) of the designed tool. IRV-KT has excellent item consistency in estimating KT learning in each segment that is before (α = 0.96), after (α = 0.96), and if (α = 0.99). This is likewise reflected on the identified KT constructs, which are creation (α = 0.96) and action (α = 0.96).

Table 1: Pilot test reliability (n = 10) of the IRV-KT tool.

Constructs (n of items)	Before	After	If	Overall
Creation (n = 10)	0.95	0.95	0.99	0.96
Action (n = 10)	0.94	0.93	0.99	0.95
Overall	0.96	0.96	0.99	0.97

DOI: 10.7717/peerj.5323/table-1

The preliminary analysis also tested the IRV-KT for validity. Its initial face and content validation provided the tool with a criterion. To estimate the validity coefficient, each item was correlated with the summative score of all segments. The statistical validity (Table 2) presented varying coefficients. All “If” (n = 20) and many “Before” (n = 14) items yielded significant correlations (p < 0.05), whereas most “After” items failed to demonstrate criterion validity.

Table 2: Pilot validity testing (n = 10) of the IRV-KT tool.

KT constructs and items	Segments (r)
	Before	After	If
Creation
Share my thoughts/ideas	0.53	0.52	0.70*
Retrieve relevant information	0.90*	0.19	0.96*
Explain the need for information	0.81*	0.61	0.87*
Analyze the usefulness of information	0.69*	0.48	0.86*
Evaluate the information	0.83*	0.69*	0.86*
Appreciate literature reviews	0.73*	0.28	0.91*
Determine the gaps/needs	0.59	0.69*	0.87*
Recommend useful solution	0.93*	0.83*	0.90*
Work well with others	0.52	0.03	0.78*
Develop new products	0.72*	0.32	0.85*
Action
Apply a team approach	0.84*	0.22	0.81*
Use knowledge in many situations	0.59	0.34	0.89*
Fit my knowledge within a context	0.63*	0.32	0.84*
Apply my knowledge	0.51	0.58	0.81*
Build my confidence	0.34	−0.20	0.90*
Make sound decisions	0.69*	0.14	0.84*
Recognize the knowledge in action	0.69*	0.48	0.79*
Reflect on my knowledge use	0.76*	0.76*	0.79*
Enjoy getting feedback	0.65*	0.00	0.75*
Share the results with others	0.73*	0.01	0.81*

DOI: 10.7717/peerj.5323/table-2

Notes:

*Significant at 0.05 alpha level.

IRV-KT went through further improvement using the findings from the pilot testing and focus group discussion. A small sample pilot study was intended to supply preliminary evidence of the tool applicability of use (Kennedy et al., 2018). The statements of each item were reworded for clarity and specificity. In the actual implementation on December 2017, the study participants (n = 45) were tasked to accomplish the IRV-KT at the end of the non-lecture CPD program for health professionals. Majority of the actual respondents are female (60%) nurses (36%) with master’s degrees (40%) or units (22%). Their responses were analyzed to determine its test reliability (Batterham & George, 2003) and criterion-related validity (Peeters & Martin, 2017).

Results

The study evaluated the designed the IRV-KT tool (Tool S1) for validity and reliability of its measures. After the actual implementation (n = 45), the IRV-KT maintained its excellent internal reliability (Table 3) with Cronbach’s coefficient of 0.98 which is higher than the pilot result. Its high internal consistency was cascaded down to all constructs (α = 0.97) of every segments (α = 0.98). This result indicates that IRV-KT can generate a reliable measure of KT learning competencies.

Table 3: IRV-KT test reliability (n = 45).

Constructs (n of items)	Before	After	If	Overall
Creation (n = 10)	0.98	0.97	0.96	0.97
Action (n = 10)	0.96	0.97	0.97	0.97
Overall	0.98	0.98	0.98	0.98

DOI: 10.7717/peerj.5323/table-3

The correlation study regarding segments (Table 4) shows the presence of significant relationship among segments. Surprisingly, only the before and after segments have statistical significance in the creation (r = 0.33, p < 0.05) and action (r = 0.49, p < 0.05) constructs. Although this is expected, the before and after segments have a limited relationship. This finding suggests that IRV-KT can provide reliable and, at the same time, relative estimates of KT learning competency in actual (i.e., before and after CPD) and hypothetical (i.e., vignette) situations.

Table 4: Relation matrix of KT constructs as to segments.

Constructs	Segments (r)
	Before	After	If
Creation
Before	–
After	0.33*	–
If	0.28	0.01	–
Action
Before	–
After	0.49*	–
If	0.16	0.19	–

DOI: 10.7717/peerj.5323/table-4

Notes:

*Significant at 0.05 alpha level.

The estimates of statistical validity (Table 5) show the correlations of the items in every construct. All items have significant positive validity coefficients (r > 0.35, p < 0.05) in all segments. This outcome strongly implies that the IRV-KT is considered to be a beneficial tool in assessing KT learning competencies at different segments. Moreover, it was able to measure baseline performance indicating postdictive validity and vicarious experience showing predictive validity. Thus, the IRV-KT can assess KT as a learning outcome of both actual and circumstantial CPD.

Table 5: Criterion-related validity measures of IRV-KT.

KT constructs and items	Segments (r)
	Before	After	If
Creation
Share my thoughts/ideas with others	0.64*	0.47*	0.52*
Retrieve relevant evidences/information	0.71*	0.52*	0.56*
Explain the need for searching scientific information	0.72*	0.61*	0.57*
Analyze the usefulness of a reliable information	0.76*	0.51*	0.55*
Evaluate the data and information critically	0.73*	0.58*	0.55*
Appreciate the value of reviewing literatures/sources	0.66*	0.55*	0.50*
Determine the gaps/needs of a given situation	0.68*	0.53*	0.63*
Recommend useful solution/information for application	0.68*	0.59*	0.58*
Work well with others in making plans/strategies	0.64*	0.55*	0.51*
Develop new knowledge/products based on the needs	0.71*	0.59*	0.65*
Action
Apply a team approach in the use of knowledge	0.43*	0.61*	0.44*
Use previous and current knowledge in many situations	0.63*	0.66*	0.71*
Fit my knowledge within a specific context/situation	0.60*	0.69*	0.72*
Apply my knowledge to individual situation	0.64*	0.72*	0.68*
Build my confidence in using my knowledge	0.68*	0.65*	0.66*
Make sound decisions using my knowledge	0.64*	0.68*	0.64*
Recognize the value of using knowledge in real action	0.61*	0.63*	0.59*
Reflect on my actual use of knowledge in a situation	0.63*	0.74*	0.68*
Enjoy getting feedback on how I use my knowledge	0.47*	0.62*	0.61*
Share the results of my knowledge use with others	0.49*	0.68*	0.46*

DOI: 10.7717/peerj.5323/table-5

Notes:

*Significant at 0.05 alpha level.

Discussion IRV-KT as a valid and reliable tool

The study findings confirm the validity of the IRV-KT tool. As expected, this instrument would provide valid measures of KT learning, because it was developed using substantial sources. Its strength as a learning tool primarily rests on its capacity to offer reproducible results in both technical (i.e., style and format) and instructional (i.e., content and learning process) aspects of KT as a desirable outcome of a CPD program. The IRV-KT is also reliable in assessing KT constructs of ‘creation’ and ‘action’ that resulted in commendable overall tool reliability.

KT, an area of interest in continuing professional education, is a constructive response to fill in the gray space within the “know-do gap” (Bero et al., 1998; Thompson et al., 2007). For instance, it has been reported that high-quality evidence is not found in actual professional practice (Majumdar, McAlister & Furberg, 2004; Larrivée, Hamelin-Brabant & Lessard, 2012; Negrini et al., 2018). In the field of education, the IRV-KT tool could serve as a potent instrument to ascertain KT competency outcomes among health professionals as lifelong learners in eliciting evidence-based policies, decisions, and practices.

IRV-KT as a reflective and meaningful assessment

The IRV-KT tool is considered useful because it can cultivate a more profound reflection on self-performance and authentic functioning knowledge. Further, it provides a concrete mechanism for metacognitive thinking through awareness of one’s learning (Montagna, Benaglio & Zannini, 2010). Self-reflection has been proven as a valuable exercise in both learning (Semradova & Hubackova, 2015) and instruction (Williamson, 2018). In previous literatures, it is being used to evaluate teaching methods, strategies, and approaches (Klimova, 2014). Reflecting on one’s learning further improves critical thinking and insight that are directly translatable to actions and changes of behavior (Lew & Schmidt, 2011; Asselin & Fain, 2013; Gardner, 2013).

It is essential to assess KT among health professionals for it directly impacts practice quality (Pablos-Mendez & Shademani, 2006). The IRV-KT tool boasts its capacity to identify key processes of knowledge translation through its constructs. It focuses on specific, concrete tasks (and not purely theoretical dimensions) involved in epistemic construction and application. Thus, educators can be significantly benefitted from the direct utilization of this tool in the field of health sciences and healthcare, more so in the CPD learning context.

IRV as a practical tool development framework

Like in the previous study (IL Ong, MJS Diño, MMP Calimag & FA Hidalgo, 2017, unpublished data), using the IRV framework and its procedures can produce valid and reliable reflective assessments. It provides a down to earth framework that is easy to incorporate in real-world scenarios. The demand for instruments focusing on Knowledge Translation is increasing (Ellen et al., 2017), but remains to be a gray area of literature (Chen et al., 2017). Whilst most are focused initially on validity and reliability concepts (Hinkin, Tracey & Enz, 1997), fewer studies have dwelt on the pragmatic educational usage of research instruments. In these cases, the expeditiousness of using the IRV framework is a valuable add-on in developing more assessment and research tools with practical value.

Conclusion

The study designed the IRV-KT and established its validity and reliability in assessing KT in CPD. It identified creation and action as the fundamental constructs of KT. The findings also emphasized the educational value of IRV-KT in providing a reflective assessment of KT competencies and IRV framework in developing sound learning evaluations. This study afforded a solid strategy on KT assessment using post-post test administration. Educational leaders, planners, and implementers in the health sciences are encouraged to utilize the IRV-KT beyond CPD contexts in the evaluation of both teaching and learning. Cognizant of the study limitations and constraints, further studies need to explore IRV-KT by construct validity testing to determine its convergent and discriminant interrelationships. IRV-KT is consequently viewed to be an indispensable instrument for guiding the development, implementation, evaluation, and improvement of CPD programs for health professionals.

Additional Information and Declarations

Competing Interests

The authors declare there are no competing interests.

Author Contributions

Irvin L. Ong conceived and designed the experiments, performed the experiments, analyzed the data, contributed reagents/materials/analysis tools, prepared figures and/or tables, authored or reviewed drafts of the paper, approved the final draft.

Michael Joseph S. Diño conceived and designed the experiments, performed the experiments, contributed reagents/materials/analysis tools, prepared figures and/or tables, authored or reviewed drafts of the paper, approved the final draft.

Maria Minerva P. Calimag and Fe A. Hidalgo conceived and designed the experiments, authored or reviewed drafts of the paper, approved the final draft.

Human Ethics

The following information was supplied relating to ethical approvals (i.e., approving body and any reference numbers):

The Institutional Ethics Review Committee of Our Lady of Fatima University, a level 2 Accredited Research Ethics Committee of the Philippine Health Research Ethics Board granted ethical approval the protocol and to carry out the study.

Data Availability

The following information was supplied regarding data availability:

The raw data are provided in a Supplemental File.

Funding

The authors received no funding for this work.

References

Armstrong R, Waters E, Dobbins M, Anderson L, Moore L, Petticrew M, Clark R, Pettman TL, Burns C, Moodie M, Conning R, Swinburn B+2 more. 2013. Knowledge translation strategies to improve the use of evidence in public health decision making in local government: intervention design and implementation plan. Implementation Science 8:121

Asselin ME, Fain JA. 2013. Effect of reflective practice education on self-reflection, insight, and reflective thinking among experienced nurses: a pilot study. Journal for Nurses in Professional Development 29:111-119

Atchan M, Davis D, Foureur M. 2014. Applying a knowledge translation model to the uptake of the baby friendly health initiative in the Australian health care system. Women and Birth 27:79-85

Bassi J, Lau F, Hagens S, Leaver C, Price M. 2013. Knowledge translation in eHealth: building a virtual community. Studies in Health Technology and Informatics 183:257-262

Batterham AM, George KP. 2003. Reliability in evidence-based clinical practice: a primer for allied health professionals✩. Physical Therapy in Sport 4:122-128

Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA. 1998. Getting research findings into practice: closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. BMJ 317:465-468

Biggs JB, Tang CS. 2007. Teaching for quality learning at University. Maidenhead: McGraw-Hill/Society for Research into Higher Education & Open University Press.

Biggs JB, Tang CS. 2011. Teaching for quality learning at University. Maidenhead: McGraw-Hill/Society for Research into Higher Education/Open University Press.

Bjørk IT, Lomborg K, Nielsen CM, Brynildsen G, Frederiksen A-MS, Larsen K, Reierson IÅ, Sommer I, Stenholt B. 2013. From theoretical model to practical use: an example of knowledge translation. Journal of Advanced Nursing 69:2336-2347

Bottenberg MM, DeWitt JE, Wall GC, Fornoff A, Stelter N, Soltis D, Eastman DK. 2013. Assessment of interprofessional perceptions and attitudes of health professional students in a simulation laboratory setting. Currents in Pharmacy Teaching and Learning 5:167-174

Brehaut JC, Eva KW. 2012. Building theories of knowledge translation interventions: use the entire menu of constructs. Implementation Science 7:114

Canadian Institutes of Health Research. 2014. More about knowledge translation at CIHR—CIHR. (accessed28 April 2015 )

Chen C-Y, Huang T-W, Kuo KN, Tam K-W. 2017. Evidence-based health care: a roadmap for knowledge translation. Journal of the Chinese Medical Association 80:747-749

Choi S-G, Johanson J. 2012. Knowledge translation through expatriates in international knowledge transfer. International Business Review 21:1148-1157

Collisson BA, Benzies K, Mosher AA, Rainey KJ, Tanaka S, Tracey C, Xu C, Olson DM. 2011. Knowledge translation: principles and practicalities for trainees within interdisciplinary health research teams. Clinical And Investigative Medicine. Médecine Clinique Et Experimentale 34:E336–E336

Colón-Emeric CS, Corazzini KN, McConnell ES, Pan W, Toles MP, Hall R, Batchelor-Murphy M, Yap TL, Anderson AL, Burd A, Amarasekara S, Anderson RA+2 more. 2018. Resident vignettes for assessing care quality in nursing homes. Journal of the American Medical Directors Association 19:405-410

Datta Gupta N, Kristensen N, Pozzoli D. 2010. External validation of the use of vignettes in cross-country health studies. Economic Modelling 27:854-865

Davis D, Davis N. 2010. Selecting educational interventions for knowledge translation. Canadian Medical Association Journal 182:E89–E93

Ellen ME, Panisset U, Araujo de Carvalho I, Goodwin J, Beard J. 2017. A knowledge translation framework on ageing and health. Health Policy 121:282-291

Enriquez CM, De Vera O, Salazar R, Ong I, Diño MJ, Ocampo A. 2015. Interblending of eLearning and eHealth towards adult empowerment: its implications to Asian higher education. In: AUPF 2015 proceedings. Guangzhou. Asian University President Forum. 204-209

European Centre for the Development of Vocational Training (CEDEFOP). 2014. Terminology of European education and training policy: a selection of 130 key terms. Luxembourg: Publications Office of the European Union.

European Commission. 2015. Knowledge triangle and innovation. (accessed27 April 2015 )

Froese KA, Montgomery J. 2014. From research to practice: the process of training school psychologists as knowledge transfer professionals. Procedia—Social and Behavioral Sciences 141:375-381

Gardner R. 2013. Introduction to debriefing. Seminars in Perinatology 37:166-174

Gesinde AM, Temitope AA, David OI. 2014. The development and validation of vignette-type instrument for measuring attitude toward poverty. Procedia—Social and Behavioral Sciences 159:442-446

Gholami J, Ahghari S, Motevalian A, Yousefinejad V, Moradi G, Keshtkar A, Alami A, Mazloomzadeh S, Vakili MM, Chaman R, Salehi B, Fazelzadeh O, Majdzadeh R+3 more. 2013. Knowledge translation in Iranian universities: need for serious interventions. Health Research Policy and Systems 11:43

Gholami J, Majdzadeh R, Nedjat S, Nedjat S, Maleki K, Ashoorkhani M, Yazdizadeh B. 2011. How should we assess knowledge translation in research organizations; designing a knowledge translation self-assessment tool for research institutes (SATORI) Health Research Policy and Systems 9:10

Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, Robinson N. 2006. Lost in knowledge translation: time for a map? Journal of Continuing Education in the Health Professions 26:13-24

Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. 2012. Knowledge translation of research findings. Implementation Science 7:50

Grimshaw JM, Santesso N, Cumpston M, Mayhew A, McGowan J. 2006. Knowledge for knowledge translation: the role of the Cochrane Collaboration. Journal of Continuing Education in the Health Professions 26:55-62

Hamm MP, Klassen TP, Scott SD, Moher D, Hartling L. 2013. Education in health research methodology: use of a wiki for knowledge translation. PLOS ONE 8:e64922

Health Policy Project. 2014. Capacity development resource guide: knowledge translation. Washington, D.C.: Future Group, Health Policy Project.

Hinkin TR, Tracey JB, Enz CA. 1997. Scale construction: developing reliable and valid measurement instruments. Journal of Hospitality & Tourism Research 21:100-120

Ho K, Bloch R, Gondocz T, Laprise R, Perrier L, Ryan D, Thivierge R, Wenghofer E. 2004. Technology-enabled knowledge translation: frameworks to promote research and practice. Journal of Continuing Education in the Health Professions 24:90-99

Hudson M, Cairns P. 2014. Interrogating social presence in games with experiential vignettes. Entertainment Computing 5:101-114

Kastner M, Straus SE. 2012. Application of the knowledge-to-action and medical research council frameworks in the development of an osteoporosis clinical decision support tool. Journal of Clinical Epidemiology 65:1163-1170

Kennedy DP, Osilla KC, Hunter SB, Golinelli D, Maksabedian Hernandez E, Tucker JS. 2018. A pilot test of a motivational interviewing social network intervention to reduce substance use among housing first residents. Journal of Substance Abuse Treatment 86:36-44

Kitto S, Bell M, Peller J, Sargeant J, Etchells E, Reeves S, Silver I. 2013. Positioning continuing education: boundaries and intersections between the domains continuing education, knowledge translation, patient safety and quality improvement. Advances in Health Sciences Education 18:141-156

Klimova BF. 2014. Self-reflection in the course evaluation. Procedia—Social and Behavioral Sciences 141:119-123

Kothari A, Armstrong R. 2011. Community-based knowledge translation: unexplored opportunities. Implementation Science 6:59

Kothari A, Wathen CN. 2013. A critical second look at integrated knowledge translation. Health Policy 109:187-191

Lang ES, Wyer PC, Haynes RB. 2007. Knowledge translation: closing the evidence-to-practice gap. Annals of Emergency Medicine 49:355-363

Larrivée M-C, Hamelin-Brabant L, Lessard G. 2012. Knowledge translation in the field of violence against women and children: an assessment of the state of knowledge. Children and Youth Services Review 34:2381-2391

Légaré F, Borduas F, MacLeod T, Sketris I, Campbell B, Jacques A. 2011. Partnerships for knowledge translation and exchange in the context of continuing professional development. Journal of Continuing Education in the Health Professions 31:181-187

Lencucha R, Kothari A, Rouse MJ. 2007. Knowledge translation: a concept for occupational therapy? American Journal of Occupational Therapy 61:593-596

Lew MDN, Schmidt HG. 2011. Self-reflection and academic performance: is there a relationship? Advances in Health Sciences Education 16:529-545

MacDonald CJ, Archibald D, Trumpower D, Casimiro L, Cragg B, Jelley W. 2010. Designing and operationalizing a toolkit of bilingual interprofessional education assessment instruments. Journal of Research in Interprofessional Practice and Education 1:304-316

Majdzadeh R, Sadighi J, Nejat S, Mahani AS, Gholami J. 2008. Knowledge translation for research utilization: design of a knowledge translation model at Tehran University of Medical Sciences. Journal of Continuing Education in the Health Professions 28:270-277

Majumdar SR, McAlister FA, Furberg CD. 2004. From knowledge to practice in chronic cardiovascular disease: a long and winding road. Journal of the American College of Cardiology 43:1738-1742

Metzler MJ, Metz GA. 2010. Analyzing the barriers and supports of knowledge translation using the PEO model. Canadian Journal of Occupational Therapy 77:151-158

Montagna L, Benaglio C, Zannini L. 2010. Reflective writing in nursing education: background, experiences and methods. Assistenza Infermieristica e Ricerca 29:140-152

Muntaner C, Chung H, Murphy K, Ng E. 2012. Barriers to knowledge production, knowledge translation, and urban health policy change: ideological, economic, and political considerations. Journal of Urban Health 89:915-924

Mustard JF. 2010. Preface. In: Bammer G, Michaux A, Sanson A, eds. Bridging the know-do gap knowledge brokering to improve child wellbeing. Acton, A.C.T.: ANU E Press.

Negrini S, Gimigliano F, Arienti C, Kiekens C. 2018. Knowledge translation: the bridging function of cochrane rehabilitation. Archives of Physical Medicine and Rehabilitation 99:1242-1245

Pablos-Mendez A, Shademani R. 2006. Knowledge translation in global health. Journal of Continuing Education in the Health Professions 26:81-86

Peeters MJ, Martin BA. 2017. Validation of learning assessments: a primer. Currents in Pharmacy Teaching and Learning 9:925-933

Reitmanova S. 2009. Knowledge translation in health research: a novel approach to health sciences education. Medical Education Online 14:10

Santesso N, Tugwell P. 2006. Knowledge translation in developing countries. Journal of Continuing Education in the Health Professions 26:87-96

Santos-Eggimann B, Meylan L. 2017. Older citizens’ opinions on long-term care options: a vignette survey. Journal of the American Medical Directors Association 18:326-334

Semradova I, Hubackova S. 2015. Communication self-reflection in language education. Procedia—Social and Behavioral Sciences 182:45-50

Shrader S, Farland MZ, Danielson J, Sicat B, Umland EM. 2017. A systematic review of assessment tools measuring interprofessional education outcomes relevant to pharmacy education. American Journal of Pharmaceutical Education 81:119

Sibley KM, Straus SE, Webster F, Jaglal SB. 2011. Moving balance and mobility evidence in to action: a primer in knowledge translation. Gait & Posture 33:527-531

Souter D. 2014. Building inclusive knowledge societies: a review of UNESCO’s action in implementing the WSIS outcomes. Paris: UNESCO.

Straus SE, Brouwers M, Johnson D, Lavis JN, Légaré F, Majumdar SR, McKibbon KA, Sales AE, Stacey D, Klein G, Grimshaw J+1 more. 2011. Core competencies in the science and practice of knowledge translation: description of a Canadian strategic training initiative. Implementation Science 6:127

Straus SE, Tetroe JM, Graham ID. 2011. Knowledge translation is the use of knowledge in health care decision making. Journal of Clinical Epidemiology 64:6-10

Tetroe J. 2007. Knowledge translation at the Canadian Institutes of Health Research: a primer. A publication of the National Centre for the Dissemination of Disability Research (NCDDR) Focus, Technical Brief No. 18.

Thomas A, Menon A, Boruff J, Rodriguez A, Ahmed S. 2014. Applications of social constructivist learning theories in knowledge translation for healthcare professionals: a scoping review. Implementation Science 9:54

Thompson DS, Estabrooks CA, Scott-Findlay S, Moore K, Wallin L. 2007. Interventions aimed at increasing research use in nursing: a systematic review. Implementation Science 2:15

UNESCO-IBE. 2013. Glossary of curriculum terminology. Geneva: UNESCO International Bureau of Education.

Urquhart R, Cornelissen E, Lal S, Colquhoun H, Klein G, Richmond S, Witteman HO. 2013. A community of practice for knowledge translation trainees: an innovative approach for learning and collaboration: community of practice for knowledge translation. Journal of Continuing Education in the Health Professions 33:274-281

Visram S, Goodall D, Steven A. 2014. Exploring conceptualizations of knowledge translation, transfer and exchange across public health in one UK region: a qualitative mapping study. Public Health 128:497-503

Wahabi HA, Al-Ansary LA. 2011. Innovative teaching methods for capacity building in knowledge translation. BMC Medical Education 11:85

Wallin L. 2009. Knowledge translation and implementation research in nursing. International Journal of Nursing Studies 46:576-587

Williamson JM. 2018. Chapter 9—self-reflection as a way of improving instruction. In: Williamson JM, ed. Teaching to individual differences in science and engineering librarianship. Chandos Publishing. 133-145

Wolfe DM, Sargeant JM, Dobbins M, McEwen SA. 2012. Knowledge translation and exchange in the Canadian microbial food safety system: a quantitative assessment of researcher awareness, attitude, and activities with government policymakers. Food Policy 37:589-599

Zhang Y, Li H, Duan H, Zhao Y. 2015. Mobilizing clinical decision support to facilitate knowledge translation: a case study in China. Computers in Biology and Medicine 60:40-50

Zwarenstein M, Reeves S. 2006. Knowledge translation and interprofessional collaboration: where the rubber of evidence-based care hits the road of teamwork. Journal of Continuing Education in the Health Professions 26:46-54