Introduction
Background
During the last few decades, eHealth solutions have been increasingly introduced in routine health care. Indeed, technological innovations are gradually changing the landscape of health care delivery, monitoring, efficiency, and decision-making [1,2]. eHealth has been defined by the World Health Organization as “the use of information and communication technologies for health,” and it can include a broad spectrum of different types of methods and interventions such as electronic health records, telehealth, web-based health care, and mobile health. However, research has shown that it is difficult to implement and incorporate eHealth in complex and multi-professional settings such as health care organizations [3,4].
Key barriers to implementing eHealth have been highlighted, such as perceptions of an increased workload and workflow disruptions, misalignment with clinical processes, undefined and changed roles, disruptions to face-to-face communication, and staff turnover [4,5]. However, factors that could facilitate implementation have also been identified, such as perceptions that using innovative eHealth could increase quality of care [4]. A review of reviews concluded that activities such as carefully considering the choice of an eHealth solution, engaging key stakeholders, and offering training and education, as well as allowing for adaptations of eHealth solutions, is central to succeeding with implementation efforts [6]. These recommended activities are in line with prevalent implementation theory and research arguing that factors at multiple levels in an organization together influence and contribute to implementation [7]. Furthermore, the aforementioned meta-analysis highlighted perceived benefits and harms to be especially important in eHealth implementation, for instance, perceived consequences on workflow and productivity, as well as expected costs of implementing eHealth [6].
Thus, implementation of eHealth initiatives in health care occurs at multiple organizational levels, is complex, and requires considerable work to succeed. Within implementation science, different theoretical approaches (theories, models, and frameworks) aim to describe, guide, explain, and evaluate implementation efforts [7]. These approaches can be used by implementers to plan, execute, and evaluate implementation efforts. However, in the hierarchical system of health care, clinicians and health service managers, who may have limited knowledge in implementation science, often find themselves responsible for the implementation of new digital solutions. Although, for example, a well-operationalized, multilevel framework derived from implementation theory and empirical data can guide the implementation process, lack of time, resources, and knowledge may lead to a nonsystematic implementation [8]. In addition, failures in implementation may not only lead to loss of money and time, but can also contribute to a decreased willingness among staff to adopt eHealth innovations in the future [9,10].
Furthermore, organizational readiness for change has been recognized and shown to be central to successful implementation [11-13]. It has been conceptualized by Weiner [14] as the shared determination and sense of collective capability to change. The concept of eHealth readiness has been proposed as important for implementing digital innovations in health care successfully [15]. A review on eHealth readiness highlighted multiple dimensions involved in the concept, such as technological aspects (eg, the extent to which technical requirements can be met), motivational aspects (eg, perceived need for technology among users), and availability of resources (eg, financial resources and competency to use technology) [15]. Tools and frameworks for measuring eHealth readiness within health care contexts also exist and are proposed to be used to facilitate implementation [15-17].
Incorporating eHealth solutions in routine health care may encompass a number of different aspects, for example, intervention conception, eHealth readiness assessment, and business and financial plans, as well as a change management plan and details on how implementation can be performed, monitored, evaluated, and sustained. Tools that assess readiness can facilitate the implementation process by guiding and informing stakeholders in a hands-on manner. In addition to improved compliance, targeted implementation efforts are more effective in terms of both costs and use of professionals’ time [18]. However, previous research has highlighted a number of challenges with regard to readiness measures, such as the measures are too theoretical, they do not address implementation issues at employee level, the scope of the measures is too broad, and the measures capture determinants for readiness rather than actual readiness [15,19].
Objective
The aim of this study is to develop and evaluate a tool that could facilitate eHealth implementation in, for example, health care. Thus, this paper describes the development and formative evaluation of E-Ready, a generic self-help tool targeting some domains of eHealth readiness to support implementation of eHealth initiatives. The tool comprises 2 parts: (1) a survey assessing implementation readiness and (2) a hands-on manual with recommended strategies and activities to facilitate implementation.
Methods
Overview
The E-Ready Tool was developed to be used by managers, project leaders, and/or change leaders to facilitate and plan for implementation. E-Ready 1.0 was generated and examined in a formative evaluation process to support its validity.
Development of E-Ready
E-Ready 1.0 was developed based on theory appraisal. This version comprised a survey assessing implementation readiness and a manual that included general information about implementation and strategies that can be used to promote practice change.
Theory Appraisal
Theories on organizational change, implementation, and behavior change were reviewed, for instance, organizational readiness to change [14], determinants of implementation [20-23], and individual behavior change [24,25]. Theoretical constructs from these theories were listed and used to design the content and structure. For the assessment survey, items were generated to capture all the theoretical constructs that had been identified. The theoretical constructs were categorized into 5 domains: (1) capacity to change at organizational and individual levels, (2) culture to change, (3) leadership promoting change, (4) motivation to change among staff, and (5) perceived characteristics of the implementation object. These 5 domains provided an initial structure for both the survey and the manual.
Development of the Readiness Assessment Survey
Survey items were generated for each domain and theoretical construct, that is, providing evidence based on test content [26]. For example, in the domain implementation object, one of the theoretical constructs was “perceived need to conduct the change among the staff.” This construct resulted in 2 proposed items: I understand why we are implementing X at my workplace and I can see a need for X at my workplace.
A total of 38 items were generated initially. Revisions were made in an iterative process and included survey structure, wording of items, wording of response options, and order of items. For example, an effort was made to tailor response options to questions. Furthermore, we revised the structure and divided the survey into main questions and subitems to facilitate the completion of the survey. The response scales were tailored to each item and were formulated to force respondents to choose an answer, that is, the items do not include a middle alternative such as “I have no opinion.” The structure of the survey no longer followed that of 5 theoretical domains; rather, the survey was constructed in a way that facilitated completion. These revisions resulted in 6 sections of questions and additional items, for instance, profession, giving a total of 33 items (Multimedia Appendix 1).
Development of the Manual
The manual was also drafted grounded in the aforementioned 5 theoretical domains: capacity, culture, leadership, motivation, and implementation object. For example, text on why and how the factors relating to the 5 domains are important for successful implementation was included in the manual. The manual was developed with a need to know perspective so that it could also be useful for novice implementers.
Formative Evaluation
Formative evaluation of E-Ready was undertaken in 3 ways: expert panels, cognitive interviews, and statistical tests. In addition, E-Ready was field tested among health care professionals. Revisions of the assessment survey and the manual were conducted continually during the formative evaluation process.
Expert Panels and Cognitive Interviews
In all, 3 workshops with experts (n=15) were conducted with the aim of investigating the content validity and structure of the tool. Participants’ expertise consisted of knowledge as potential end users of the tool (physicians with clinical experience) as well as knowledge and experience in implementation. All participants had explicit responsibility for integrating eHealth in their organization. The first workshop focused on the survey items. Participants were asked to individually complete the assessment survey and then provide written feedback on how they interpreted the items and how they perceived the structure and content of the questions. The individual feedback was later discussed at the workshop. For example, to ensure that individual items captured specific theoretical constructs, the discussions focused on formulations and on how items were understood. During the second and third workshops, the manual was scrutinized. The physicians were asked to give feedback on the content of the manual, for example, the introduction of theories, information on how to use the guide, the layout, if the guide gave enough information, and if the information was clear and understandable.
Cognitive interviews were conducted with health care professionals from 2 health care settings (university hospital and specialist outpatient clinic), including physicians (n=5), registered nurses (n=4), physiotherapists (n=2), behavior therapists (n=2), and dietitians (n=4). Cognitive interviewing is a psychologically oriented method for empirically studying the ways in which individuals mentally process and respond to survey questionnaires. Cognitive interviews provide validity evidence based on response processes by allowing for potential differences in the interpretation of test items.
Data from cognitive interviewing can form the basis for appropriate modifications before further field testing [27]. Cognitive interviews were conducted by letting the participant read the question and then explaining their interpretation and meaning of the question. This technique was used in an applied sense—for the purpose of pretesting questions and determining how the items could be modified—before field testing to make the items more understandable or otherwise easier to answer. Participants were first asked to complete the survey while thinking of an eHealth initiative that they had experienced. They were then further interviewed regarding how they perceived the tool, its structure, content, and if there were survey items that were unclear or should be rephrased. Comments were noted by the researcher throughout the interview.
Iterative revisions were made in parallel with conducting expert panels and cognitive interviews. Revisions of the survey predominantly included eliminating overlap across the items and wording of the items. For example, comments based on the cognitive interviews showed that the content of 4 of the items covered similar areas, resulting in the omission of items. Revisions of the survey during this phase resulted in 29 items. Furthermore, expert data showed that the manual offered a structured way to organize the implementation process and provided insights into how to think about readiness and implementation. Revisions of the manual thus mainly covered language editing.
Statistical Tests
To assess the measurement properties—specifically, evidence based on the internal structure [26] and fit validity [28]—of the E-Ready survey, we conducted a Rasch analysis using Winsteps software (version 4.3.1). In this analysis we included the E-Ready survey data from participants at 3 different workplaces (n=165), all of whom had filled in the E-Ready survey ahead of different eHealth implementation initiatives.
Each subscale was individually evaluated in terms of the following attributes: (1) sample to item targeting, (2) item fit to the model, (3) unidimensionality, and (4) person reliability. Each attribute is described briefly as follows:
Field Testing
As a last step, E-Ready was field tested among health care professionals. The tool was sent through email to 29 physicians and registered nurses at Karolinska University Hospital in Stockholm, Sweden. Participants were asked to complete the survey from personal experiences of implementation of an eHealth initiative. At the end of the survey, they were asked to leave comments. The feedback was positive, and no additional revisions were made. For example, positive feedback was provided regarding using the survey to engage employees in the change process. Furthermore, items investigating leadership and manager support were perceived to be specifically important.
Figure 1 illustrates all activities and their time points.
Enlarge this image.Figure 1. Process of the development and formative evaluation of E-Ready 1.0 to E-Ready 2.0.
Results
Readiness Assessment Survey
The survey (Multimedia Appendix 1) aims to assess implementation readiness. The 6 sections investigate the following: (1) Perceived conditions for change at the workplace, for example, competency and resources for practice change (8 items); (2) Perceived individual conditions for change, for example, prior experience of changing practice (3 items); (3) Perceived support and engagement among management, for example, how urgency for change is communicated by management (5 items); and (4) Perceived readiness among colleagues, for example, how change is collectively valued by colleagues (5 items). There are also items aiming to capture (5) Perceived consequences on status quo, for example, worries regarding how practice change can influence current workflows (5 items) and (6) Perceived workplace attitudes toward change, for example, self-reported attitude toward the proposed change (3 items). In addition, 7 single items investigate compatibility with current work routines: commitment to change and perceived need for change, gender, years worked at current workplace, profession, and years worked in profession. In total, there are 36 items in the final assessment survey, E-Ready 2.0. It takes approximately 10 minutes to complete the survey.
Results from the Rasch analyses showed that item-threshold measures were well covered by the person measures and person measures were fairly well covered by the item thresholds. As illustrated in the person-item threshold histograms (Figure 2A-F), there are gaps among the item thresholds. Furthermore, the subscale Perceived readiness among colleagues (Figure 2D) was negatively skewed (mean –1.91, SD 2.19), and Perceived consequences on status quo (Figure 2E) was positively skewed (mean 1.05, SD 2.88).
Enlarge this image.Figure 2. Person-item threshold histograms for each subscale. 1Conditions for change at the workplace;2Individual conditions for change; 3Perceived support and engagement among management; 4Perceived readiness among colleagues; 5Perceived consequences on status quo; 6Workplace attitudes.
Table 1 provides a summary of the Rasch analysis of the subscales. In short, the subscale Conditions for change at the workplace had all INFIT and OUTFIT MNSQ values within the acceptable range, and the 2 items with INFIT and OUTFIT ZSTD values were slightly outside 2 σ. By a closer inspection of the principal component analysis loadings, we found that a cluster of 3 items addressed the conditions among the employees, whereas the others addressed organizational conditions. The subscale Individual conditions for change also had all INFIT and OUTFIT MNSQ values within the acceptable range, but 2 items with INFIT and OUTFIT ZSTD values were slightly outside 2 σ. The person reliability was lowest for this scale, which, speaking reasonably, is a consequence of having only 3 items. The subscale Perceived support and engagement among management showed some statistical item misfit but supported unidimensionality. By removing the most misfitting item (item E of the E-Ready survey, which addresses management competence and knowledge unlike the others that address management actions), we found that the fit statistics were improved, and the person reliability increased from 0.84 to 0.86.
Table 1. Summary of the Rasch analysis of the E-Ready subscales.
| Conditions for change at the workplace, 8 items | Individual conditions for change, 3 items | Perceived support and engagement among management, 5 items | Readiness among colleagues, 5 items | Perceived consequences on status quo, 5 items | Workplace attitudes, 3 items | ||||||||
| Item fit statistics | |||||||||||||
| MNSQa INFITb, mean (SD) | 1.00 (0.17) | 1.00 (0.26) | 0.99 (1.22) | 0.99 (0.27) | 0.99 (0.42) | 0.98 (0.04) | |||||||
| ZSTDc INFIT, mean (SD) | –0.10 (1.60) | –0.20 (2.30) | –0.20 (1.40) | –0.20 (2.10) | –0.60 (3.60) | –0.20 (0.40) | |||||||
| MNSQ OUTFITd, mean (SD) | 0.99 (0.18) | 1.02 (0.29) | 1.00 (1.25) | 0.98 (0.27) | 1.02 (0.46) | 1.03 (0.09) | |||||||
| ZSTD OUTFIT, mean (SD) | –0.10 (1.70) | 0.00 (2.50) | –0.2 (1.40) | –0.40 (2.10) | –0.40 (3.80) | 0.20 (0.50) | |||||||
| MNSQ INFIT (number of items misfitting) | 0 | 0 | 1 | 0 | 1 | 0 | |||||||
| ZSTD INFIT (number of items misfitting) | 2 | 2 | 2 | 3 | 3 | 0 | |||||||
| MNSQ OUTFIT (number of items misfitting) | 0 | 0 | 1 | 0 | 1 | 0 | |||||||
| ZSTD OUTFIT (number of items misfitting) | 2 | 2 | 2 | 3 | 4 | 0 | |||||||
| Unidimensionality | |||||||||||||
| Eigenvalue unexplained variance in first contrast | 2.43 | 1.78 | 1.65 | 2.37 | 2.22 | 1.48 | |||||||
| Disattenuated correlation cluster, 1-3 | 0.21 | 0.13 | 0.66 | 0.37 | 0.61 | 1.00 | |||||||
| Person reliability | |||||||||||||
| Reliability index | 0.69 | 0.56 | 0.84 | 0.72 | 0.87 | 0.77 | |||||||
| Separation index | 1.50 | 1.13 | 2.30 | 1.60 | 2.55 | 1.85 | |||||||
aMNSQ: mean-square.
bINFIT: inlier-pattern-sensitive fit.
cZSTD: Z-standardized.
dOUTFIT: outlier-pattern-sensitive fit.
The subscale Readiness among colleagues had acceptable INFIT and OUTFIT MNSQ values but INFIT and OUTFIT ZSTD misfit and suffered from unidimensionality. The items comprised both specific tasks (eg, having discussions among colleagues) and overall collaboration, which might explain the weaknesses. The subscale Perceived consequences on status quo showed the highest person reliability, but, at the same time, it suffered from several item misfittings. This could possibly be explained by the nature of the response options and statements, that is, the extent to which the implementation of X would affect several working tasks might not be quantified in a common scale for different implementations at different workplaces. Finally, the subscale Workplace attitudes did not have any misfitting items; it supported unidimensionality and had a person reliability close to the desired 0.8.
As shown in Table 2, correlations among the person measures were low to moderate for all subscales except Perceived consequences on status quo. This subscale showed some negative correlations, especially toward Workplace attitudes (r=–0.30). Statistical differences between the person measures of men and women were present for 1 subscale, Individual conditions for change, where the men had higher measures than the women.
Table 2. Correlations among person measures for the E-Ready subscales.
| Conditions for change at the workplace | Individual conditions for change | Perceived support and engagement among management | Readiness among colleagues | Perceived consequences on status quo | Workplace attitudes | |
| Conditions for change at the workplace | 1.00 | —a | — | — | — | — |
| Individual conditions for change | 0.48 | 1.00 | — | — | — | — |
| Perceived support and engagement among management | 0.50 | 0.28 | 1.00 | — | — | — |
| Readiness among colleagues | 0.29 | 0.21 | 0.53 | 1.00 | — | — |
| Perceived consequences on status quo | –0.16 | –0.02 | 0.01 | –0.02 | 1.00 | — |
| Workplace attitudes | 0.52 | 0.31 | 0.41 | 0.27 | –0.30 | 1.00 |
aNot applicable.
The Manual
The manual is in the form of a 56-slide PowerPoint (Microsoft Corp) document. The manual predominantly includes recommendations for strategies and activities that facilitate implementation. Recommendations are tailored to low-score areas as identified by the survey. The manual includes the following:
Using the E-Ready Tool involves 6 steps where the person responsible for the implementation performs the following actions:
Description of Development and Formative Evaluation of E-Ready
Here, we describe the development and formative evaluation of a novel implementation tool: E-Ready. The tool is a theory-based self-help tool to measure individual and organizational readiness to facilitate eHealth implementation in, for example, health care. The tool consists of a readiness assessment survey and a hands-on manual. The survey measures perceived readiness for change (perceived willingness and capability) in a workplace setting at individual as well as collective levels.
eHealth readiness assessment can be seen as a holistic approach from intervention conception to evaluation and monitoring, as well as sustainability of implementation. The E-Ready Tool considers some key aspects of eHealth readiness, such as perceived conditions for change at group and individual levels, support and engagement level among management, and perceptions among stakeholders about the change per se. In addition, the manual can be used to guide the generation of implementation and evaluation plans. Thus, the tool primarily considers implementation in the local health care setting using a stakeholder perspective. That is, the tool does not explicitly consider other dimensions of eHealth readiness that have been highlighted in previous research, such as governance or societal readiness [33].
The E-Ready Tool was developed from a need to facilitate systematic implementation of eHealth in health care and the urge to meet health care challenges with regard to new digital solutions. Previous research has highlighted difficulties and delays in integrating new technology with existing workflows, tasks, and organizational processes [3,4,34]. Furthermore, studies suggest that measuring and considering the readiness for change within an organization or workplace can facilitate implementation [12,13]. Readiness assessments can thus help to predict and plan for implementation [15]. However, future studies will need to assess the effect of measuring tools such as E-Ready on implementation outcomes, for example, the reach, adoption, and acceptability of eHealth interventions among health care professionals and patients [35]. For example, more knowledge is needed on how different dimensions of organizational readiness (eg, capacity or willingness among staff) influence implementation outcomes in general and at different levels of an organization over time and at specific time points, as well as how the degrees of readiness differ within an organization and how this variance influences implementation outcomes and potentially can be addressed. In addition, an important aspect to be considered for future research is at what time point readiness assessment is the most valuable. As any measurement only captures readiness for the next step of a change process rather than readiness for the implementation as a whole, multiple measurements of readiness may be needed. E-Ready and similar instruments could be used to investigate these knowledge gaps to further our knowledge on the relationship between organizational readiness for change and implementation.
Previous research has highlighted several challenges with existing readiness-measuring tools, for instance, the importance of tools with regard to targeting and capturing issues at employee level, acknowledging that readiness is change- and situation-specific, and the need for tools to capture readiness (eg, perceptions about capability for change) rather than its determinants (eg, resources for enabling change) [15,19]. We have attempted to address these issues in the development of the E-Ready Tool. Our ambition is to provide a brief, theory-based tool that can be used by stakeholders who are novices at implementation. Specifically, E-Ready 2.0 combines readiness assessment with a hands-on manual on how to plan and promote implementation. Combining these 2 components could offer stakeholders accessible and concrete guidance on implementation. The themes that are assessed in the survey and discussed in the manual, although theory-based, were perceived to be relevant among end users, suggesting that E-Ready addresses appropriate employee-level implementation issues. Furthermore, although the themes of the questions in the survey are generic and can be applied to a broad range of eHealth initiatives, the eHealth initiative in question is to be specified at the top of the assessment survey, which will tailor the questions and guide responders toward specific eHealth initiatives. We have striven to enable E-Ready to measure and address readiness for change rather than its determinants mainly by reviewing relevant theory [14] and linking theoretical constructs to the survey items and manual content. Finally, the tool is generic and could potentially also be used as a change management component of implementation in fields other than health care, although the testing so far has been conducted in the health sector. Future studies will have to further investigate the validity and reliability of the E-Ready Tool in larger effectiveness studies.
By considering the fairly well-targeted sample to item and by following the general principle of Rasch fit statistics, we found acceptable measurement properties of E-Ready in terms of internal structure and fit validity. However, both the assessment of unidimensionality and provision of a highly reliable measure suffer from including few items in some of the subscales. Thus, there is room to further extend the survey with additional items to improve the measurement properties of the E-Ready subscales. There is indication of construct underrepresentation [28]; thus, items investigating perceptions of privacy issues associated with eHealth interventions, for example, could be added to improve content coverage and content validity. Moreover, despite the practical significance of measures of different aspects with the subscales, the moderate correlations among person measures indicate that a couple of items could be combined to provide a higher-ordered E-Ready measure. This can reduce the measurement uncertainties and further improve the reliability; however, at the same time, a higher-ordered E-Ready measure might not be of the same clinical significance as measures of subscale attributes.
Methodological Considerations
Among the limitations is the relatively small number of respondents involved in the initial formative evaluation. Considering the qualitative methodology used in the evaluation (ie, expert panels and cognitive interviews), we estimated that the number of participants included would be sufficient to achieve our aim. However, a larger number of participants could have improved generalizability; especially if participants from various health care settings and professions had been included, the rigor would have been enhanced. Thus, continued development work to, for example, better assess the validity and reliability of the survey is needed, as is research on the usability of the manual.
Potential strengths of the study are that end users were included in the formative evaluation, in the expert panels as well as in the cognitive interviews. Involving end users could have increased the usability and acceptability of the tool. However, this needs further investigation in future studies. Using qualitative data from both clinicians and implementers improved and gave support that the content and structure of the tool were relevant and understandable for end users.
Another strength was that we applied the Rasch analysis as part of development. This is not simply a mathematical or statistical approach; rather, it is a specifically metrological approach to human-based measurement [36]. The Rasch analysis provides separate measures of persons and items scaled on the conjoint interval logit scale [37], which in turn provides objective measures that can be used for arithmetic operations.
The Rasch analysis can be viewed as a statistical model used for validating assessment tools [38]. In this work we have addressed some aspects of validity, that is, the expert group provided evidence based on test content, the cognitive interviews provided evidence based on response processes, and the Rasch analyses provided evidence based on internal structure [26,39]. However, we have not evaluated the 2 remaining validity aspects in the Standards for Educational and Psychological Testing [26], namely relations to other variables or consequences of testing. Furthermore, the evidence provided on validity aspects in this study may benefit re-evaluations with added items and in larger samples. Moreover, it should also be noted that validity evaluations of relations to other variables is not a straightforward process because the constructs purported to be measured with E-Ready do not have a gold standard to be compared with. We therefore encourage further work with a focus on construct modeling [40] and qualitative understanding of implementation readiness theory.
Conclusions
E-Ready 2.0 is a new self-help tool to guide implementation targeting health care provider readiness and engagement readiness ahead of eHealth initiatives in, for example, health care settings. E-Ready could be improved further to capture additional aspects of implementation; improvements could also be made by evaluating the tool in a larger sample.
Acknowledgments
The research project was funded by Swelife, a strategic innovation program funded by the Swedish government through the Swedish innovation agency, Vinnova, and by the program’s partners. The funder had no role in the design of the study, data collection and analysis, or the interpretation of data and the write-up of the manuscript.
Authors' Contributions
PD and YTL designed the study, and YTL applied for funding. PD conducted the data collection and analysis of the expert panel workshops, cognitive interviews, and field tests. Statistical tests were conducted by JM and AC. PD and KT generated the E-Ready 1.0 and iterative revisions of the E-Ready Tool. All authors contributed to writing the manuscript.
Conflicts of Interest
None declared.
------------
Abbreviations
| INFIT: inlier-pattern-sensitive fit |
| MNSQ: mean-square |
| OUTFIT: outlier-pattern-sensitive fit |
| ZSTD: Z-standardized |
------------
Edited by G Eysenbach; submitted 20.12.19; peer-reviewed by M Mars, P Merkus, D Hasson; comments to author 26.10.20; revised version received 31.03.21; accepted 08.10.21; published 17.01.22
Currie WL, Seddon JJ. A cross-national analysis of eHealth in the European Union: some policy and research directions. Inf Manag 2014 Sep;51(6):783-797.
World Health Organization. From Innovation to Implementation - EHealth in the WHO European Region. Geneva: World Health Organization; 2016.
Lau R, Stevenson F, Ong BN, Dziedzic K, Treweek S, Eldridge S, et al. Achieving change in primary care - Causes of the evidence to practice gap: systematic reviews of reviews. Implement Sci 2016 Mar 22;11:40
Granja C, Janssen W, Johansen MA. Factors determining the success and failure of eHealth interventions: systematic review of the literature. J Med Internet Res 2018 May 01;20(5):e10235
van den Wijngaart LS, Geense WW, Boehmer AL, Brouwer ML, Hugen CA, van Ewijk BE, et al. Barriers and facilitators when implementing web-based disease monitoring and management as a substitution for regular outpatient care in pediatric asthma: qualitative survey study. J Med Internet Res 2018 Oct 30;20(10):e284
Ross J, Stevenson F, Lau R, Murray E. Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update). Implement Sci 2016 Oct 26;11(1):146
Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci 2015 May 21;10:53
Versluis A, van Luenen S, Meijer E, Honkoop PJ, Pinnock H, Mohr DC, et al. SERIES: eHealth in primary care. Part 4: addressing the challenges of implementation. Eur J Gen Pract 2020 Dec;26(1):140-145
Lennon MR, Bouamrane M, Devlin AM, O'Connor S, O'Donnell C, Chetty U, et al. Readiness for delivering digital health at scale: lessons from a longitudinal qualitative evaluation of a national digital health innovation program in the United Kingdom. J Med Internet Res 2017 Feb 16;19(2):e42
Devlin AM, McGee-Lennon M, O'Donnell CA, Bouamrane M, Agbakoba R, O'Connor S, et al. Delivering digital health and well-being at scale: lessons learned during the implementation of the dallas program in the United Kingdom. J Am Med Inform Assoc 2016 Jan;23(1):48-59
Scaccia JP, Cook BS, Lamont A, Wandersman A, Castellow J, Katz J, et al. A practical implementation science heuristic for organizational readiness: R = MC. J Community Psychol 2015 May;43(4):484-501
Hagedorn HJ, Heideman PW. The relationship between baseline Organizational Readiness to Change Assessment subscale scores and implementation of hepatitis prevention services in substance use disorders treatment clinics: a case study. Implement Sci 2010 Jun 14;5:46
Kelly P, Hegarty J, Barry J, Dyer KR, Horgan A. A systematic review of the relationship between staff perceptions of organizational readiness to change and the process of innovation adoption in substance misuse treatment programs. J Subst Abuse Treat 2017 Sep;80:6-25.
Weiner BJ. A theory of organizational readiness for change. Implement Sci 2009 Oct 19;4:67
Yusif S, Hafeez-Baig A, Soar J. e-Health readiness assessment factors and measuring tools: a systematic review. Int J Med Inform 2017 Nov;107:56-64.
Murray E, May C, Mair F. Development and formative evaluation of the e-Health Implementation Toolkit (e-HIT). BMC Med Inform Decis Mak 2010;10:61
Mauco KL, Scott RE, Mars M. Critical analysis of e-health readiness assessment frameworks: suitability for application in developing countries. J Telemed Telecare 2018 Feb;24(2):110-117.
Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, et al. Tailored interventions to address determinants of practice. Cochrane Database Syst Rev 2015 Apr 29(4):CD005470.
MacFarlane A, Clerkin P, Murray E, Heaney DJ, Wakeling M, Pesola U, et al. The e-Health Implementation Toolkit: qualitative evaluation across four European countries. Implement Sci 2011 Nov 19;6:122
Rogers EM. Diffusion of preventive innovations. Addictive Behav 2002 Nov;27(6):989-993.
Aarons GA, Ehrhart MG, Farahnak LR. The Implementation Leadership Scale (ILS): development of a brief measure of unit level implementation leadership. Implement Sci 2014 Apr 14;9(1):45
Schein EH, Schein PA. Organizational culture and leadership. In: Organizational culture and leadership. Hoboken, New Jersey: John Wiley Sons Inc; 2016.
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009;4:50
Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol 2000 Jan;55(1):68-78.
Michie S, van Stralen M, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci 2011;6:42
The standards for educational and psychological testing. American Psychological Association. URL: https://www.apa.org/science/programs/testing/standards [accessed 2021-11-12]
Willis GB. Analysis of the Cognitive Interview in Questionnaire Design. New York: Oxford University Press; 2015.
Kreiner S. Rasch measurement: transactions of the Rasch Measurement SIG American Educational Research Association. Rasch Measurement Transac 2010;24(1):1246-1251.
Hobart J, Cano S. Improving the evaluation of therapeutic interventions in multiple sclerosis: the role of new psychometric methods. Health Technol Assess 2009 Feb;13(12):iii, ix-x, 1-177
Linacre JM. What do Infit and Outfit, Mean-square and Standardized mean? Rasch Measurement Transactions. URL: https://www.rasch.org/rmt/rmt162f.htm [accessed 2021-11-12]
Multidimensional model - multidimensionality. Winstep. URL: https://www.winsteps.com/winman/multidimensional.htm [accessed 2021-11-12]
Linacre JM. Multidimensional Model - Multidimensionality. https://www.winsteps.com/winman/multidimensional.htm. URL: https://www.winsteps.com/winman/multidimensional.htm [accessed 2021-11-12]
Mauco KL, Scott RE, Mars M. Validation of an e-health readiness assessment framework for developing countries. BMC Health Serv Res 2020 Jun 23;20(1):575
Cresswell K, Sheikh A. Organizational issues in the implementation and adoption of health information technology innovations: an interpretative review. Int J Med Inform 2013 May;82(5):e73-e86.
Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011 Mar;38(2):65-76
Pendrill L. Man as a measurement instrument. NCSLI Measure 2016 May 12;9(4):24-35.
Rasch G. Probabilistic Models for Some Intellige and Attainment Tests. Chicago: MESA Press; 1980.
McAllister S. Introduction to the use of Rasch analysis to assess patient performance. Int J Ther Rehab 2008 Nov;15(11):482-490.
Mui Lim S, Rodger S, Brown T. Using Rasch analysis to establish the construct validity of rehabilitation assessment tools. Int J Ther Rehab 2009 May;16(5):251-260.
Wilson M. Constructing Measures: an Item Response Modeling Approach. Mahwah, New Jersey: Lawrence Erlbaum Associates; 2005.
© 2022. This work is licensed under https://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Thomas, Kristin
; Chakhunashvili, Alexander
; Melin, Jeanette
; Lagerros, Ylva Trolle