Electronic health records (EHR) simulation-based training: a scoping review protocol

Abstract

Introduction

Effective electronic health record (EHR)-based training interventions facilitate improved EHR use for healthcare providers. One such training intervention is simulation-based training that emphasises learning actual tasks through experimentation in a risk-free environment without negative patient outcomes. EHR-specific simulation-based training can be employed to improve EHR use, thereby enhancing healthcare providers’ skills and behaviours. Despite the potential advantages of this type of training, no study has identified and mapped the available evidence. To fill that gap, this scoping review will synthesise the current state of literature on EHR simulation-based training.

Methods and analysis

The Arksey and O’Malley methodological framework will be employed. Three databases (PubMed, Embase and Cumulative Index to Nursing and Allied Health Literature) will be searched for published articles. ProQuest and Google Scholar will be searched to identify unpublished articles. Databases will be searched from inception to 29 January 2020. Only articles written in English, randomised control trials, cohort studies, cross-sectional studies and case-control studies will be considered for inclusion. Two reviewers will independently screen titles and abstracts against inclusion and exclusion criteria. Then, they will review full texts to determine articles for final inclusion. Citation chaining will be conducted to manually screen references of all included studies to identify additional studies not found by the search. A data abstraction form with relevant characteristics will be developed to help address the research question. Descriptive numerical analysis will be used to describe characteristics of included studies. Based on the extracted data, research evidence of EHR simulation-based training will be synthesised.

Ethics and dissemination

Since no primary data will be collected, there will be no formal ethical review. Research findings will be disseminated through publications, presentations and meetings with relevant stakeholders.

Correspondence to Dr Joseph K Nuamah, School of Industrial Engineering and Management, Oklahoma State University, Stillwater, United States; [email protected]

Strengths and limitations of this study

• Identifies and maps available evidence on electronic health records simulation-based training.

• Provides in-depth search strategy, elaborate eligibility criteria and clear data extraction framework to address research question.

• Provides methodologically rigorous template for future scoping review studies that seek to identify and map early evidence for interventions.

• Quality of evidence will not be assessed, and so robustness or generalisability of findings will not be evaluated.

• Results will not answer a clinically meaningful question.

Introduction

Use of electronic health records (EHRs), digital form of the traditional patient and population health information,¹ in the US healthcare system continues to grow.² Despite their benefits, there are unintended consequences, including burnout and workflow disruption, associated with EHR use.^2–4 Inadequate training and education of healthcare providers, among other reasons, may account for these unintended consequences.³ Effective EHR-based training, which closely mimics real-world clinical conditions while replicating the cognitive load that clinicians are subjected to,⁵ can improve healthcare quality and safety.⁶

Simulation is a methodology, not a technology, that substitutes real experiences with replicable guided experiences.⁷ In healthcare, these guided experiences may be physical or computer-based models, live actors or virtual reality platforms.^{7 8} Simulation seeks to replicate clinical scenarios without putting patients at risk.⁸ Simulation-based training (SBT), now a commonplace in healthcare,⁸ emphasises learning actual tasks through experimentation in a risk-free environment without negative patient outcomes. When applied properly, SBT creates a consistent, safe and replicable learning environment.⁹ SBT has been shown to enhance healthcare providers’ skills and behaviours,^{10 11} improve patient safety outcomes¹² and provide valuable feedback.¹³ It has found utility in bridging the gap between medical students’ preclinical knowledge and care of real patients,¹⁴ enhancing surgical skills training,¹⁵ improving performance of emergency medicine residents during central venous catheterisation,¹⁶ and improving perceptual ability of critical care fellows.¹⁷

SBT can take many forms, including part-task simulators, human patient simulators, simulated clinical environments, computer screen-based simulators and virtual reality simulators.^{7 10} Of particular interest in the present study is EHR-specific SBT, a type of computer-screen based simulator, aimed at maximising the use of EHR as a clinical tool. Previous research^{6 18 19} shows that EHR-specific SBT facilitates improved use of EHRs for clinicians. It is worth noting that only one recent study²⁰ has systematically reviewed educational interventions in the use of EHRs. The authors found that all the interventions involved data entry into a simulated EHR with none requiring extraction, aggregation or visualisation of clinical data. They suggested the need to address gaps in training medical students and residents. The study population for this study was only medical students and residents. However, EHR-specific SBT is not beneficial to this population only, but also to other healthcare professionals like physicians and surgeons. Despite the potential advantages of EHR-specific SBT,^{6 10} no study has identified and mapped the available evidence. To fill that gap, this study will synthesise the current state of literature on EHR SBT. We do not wish to use the results of this study to answer a clinically meaningful question. Rather, we are interested in identifying and mapping the available evidence, hence our choice of a scoping review.²¹

Methods and analysis

This protocol conforms to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols (PRISMA-P) checklist.²² We have included a copy of the PRISMA-P checklist as a supplementary file, completed with page numbers indicating where each item can be found in our manuscript. The final publication of this work will adhere to the PRISMA Extension for Scoping Reviews checklist.²³ We will employ the Arksey and O’Malley²⁴ methodological framework: (1) identifying the research question, (2) identifying relevant studies, (3) study selection, (4) charting the data, (5) collating, summarising and reporting the results, and (6) consultation.

Stage 1: identifying the research question

According to Arksey and O’Malley,²⁴ the scoping review research question should be broad enough to summarise the breadth of evidence. In the present study, we seek to synthesise the available evidence by asking the question:

​What is known from the existing literature about EHR SBT?

We did not use a mnemonic to guide the construct of our research question. Rather, we consulted a group of healthcare educators to determine attributes of EHR SBT. Given the exploratory nature of the scoping review, we may refine this research question as we review the literature. We define an EHR as a digitised version of a patient's health information generated as a result of encounters in a healthcare delivery setting and maintained by authorised healthcare providers for planning and for delivering safe and proper treatment.²⁵ Also, we define SBT as computer screen-based simulations intended for learners to acquire knowledge or assess learners’ competency of knowledge attainment and/or provide learners feedback related to clinical knowledge and critical-thinking skills.⁷ These definitions are needed to establish a clear scope to the study and guide the search strategy. We do not have a target population. Consequently, the target population may include students, residents, clerks, technicians, nurses, physicians, managers and regulators.

Stage 2: identifying relevant studies

To be as comprehensive as possible, we will search electronic databases and reference lists of included articles. We will search three databases from inception to 29 January 2020: PubMed, Embase and Cumulative Index to Nursing and Allied Health Literature. Since this is an exploratory study, we will also search ProQuest and Google Scholar to identify unpublished articles. A preliminary search on PubMed, with the help of a health sciences librarian, yielded 235 articles. We present the sample search strategy in table 1. We will use a similar search strategy to retrieve articles from the other databases. Only literature written in English, randomised control trials, cohort studies, cross-sectional studies and case-control studies will be considered for inclusion. We will download citations and remove duplications using the Zotero reference management software.

Table 1

Sample search strategy for PubMed

Search	Query	Items found
#1	((simulat*[tw] OR simulate[tw] OR simulated[tw] OR simulating[tw]) AND (train[tw] OR training[tw] OR trained[tw] OR educat*[tw]))	40 682
#2	Electronic Health Records[mesh] OR Electronic Health Records[tiab] OR Electronic Health Record[tiab] OR Medical Order Entry Systems[mesh] OR Medical Order Entry Systems[tiab] OR Medical Order Entry System[tiab] OR Computerized Provider Order Entry[tiab] OR Computerized Physician Order Entry[tiab] OR Medical Records Systems, Computerized[mesh] OR Electronic Medical Records[tiab] OR Electronic Medical Record[tiab] OR Electronic Health Record[tiab] OR Computerized Medical Record[tiab] OR Computerized Medical Records[tiab] OR electronic documentation[tiab] OR electronic charting[tiab]	59 024
#3	#1 AND #2	235

Stage 3: study selection

We will carry out article selection in two stages. First, two reviewers (JKN and KA) will independently screen titles and abstracts against inclusion and exclusion criteria (see table 2) with the aid of Covidence,²⁶ a web-based screening and data extraction tool.

Table 2

Inclusion and exclusion criteria

Criterion	Inclusion	Exclusion
Technology used for simulation-based training	Computer screen-based	Part-task, human patient, simulated clinical environment, virtual reality
Language	English	Non-English
Text availability	Full text	Abstract only
Study design	Randomised control trial, cohort study, cross-sectional study, case-control study	Animal research study, systematic review, meta-analysis, literature review, scoping review

A third reviewer (LM) will assist in resolving disagreements regarding article eligibility. In the second stage, two reviewers will independently review the selected full-text articles to determine eligibility. Also, we will conduct citation chaining to manually screen references of all included studies to identify additional studies not found by the search. Figure 1 displays the flow of studies from stages 1 to 2.

Generate image description

Figure 1. Flow of studies within stages 1-2. CINAHL, Cumulative Index to Nursing and Allied Health Literature.

Stage 4: charting the data

Through an iterative process, we will develop a data abstraction form with relevant characteristics to help address the research question. Each researcher will independently pilot the data abstraction form about 7–10 studies to determine its consistency with the research question. For each study, variables will be extracted for each category—research, simulation study and EHR (see table 3). Together, these data will form the basis of our analysis.

Table 3

Overview of variables and related classification method

Category	Variable	Classification method
Research	Year of publication	Metadata
	Geolocation
	Number of participants
	Participants’ age group
	Specialty
Simulation study	Aim of simulation	Gaba7
	Unit of participation in simulation
	Experience level of simulation participants
	Knowledge, skill, attitudes addressed
	Site of simulation participation
	Extent of direct participation in simulation
	Feedback accompanying simulation
EHR	Health information and data	Institute of Medicine28
	Results management
	Order entry/management
	Decision support management
	Electronic communication and connectivity
	Patient support
	Administrative processes
	Reporting and population health management

Stage 5: collating, summarising and reporting the results

We will not evaluate the robustness or generalisability of our findings since we do not seek to assess the quality of evidence.²⁴ Rather, we will collate, summarise and report our findings using descriptive numerical analysis. Two reviewers (JKN and KA) will conduct this analysis to present a summary of the nature and distribution of the studies included in the review. We will produce tables and charts mapping the distribution of studies. Based on the data extracted, we will synthesise research evidence of EHR SBT.

Stage 6: consultation

Stakeholder consultation is an essential component of scoping review methodology.²⁷ This stage focuses on the development of a plan to consult with stakeholders to help identify potential studies to include in the review, interpretation of research findings and the dissemination of these findings. We propose to consult with two stakeholders—one with expertise in simulation training and another who has participated in SBT on one or more research projects. The aim is to integrate the experiences of SBT experts and participants to ensure that the design, conduct and knowledge translation of this scoping review is relevant to the population it involves—researchers and healthcare professionals.

Patient and public involvement

There was no patient or public involvement in the design of this protocol.

Anticipated outcome

The scoping review will synthesise the current state of the literature on EHR-specific SBT based on Gaba’s dimensions and the Institute of Medicine’s EHR functionalities. Anticipated outcomes include (1) mapping of the literature on the use of simulation-based EHR training in healthcare, (2) comparison of EHR functionalities across prior simulation-based EHR training studies and (3) outline areas where further research is needed.

Ethics and dissemination

Since no primary data will be collected, there will be no need for a formal ethical review. To our knowledge, this is the first scoping review to identify and map the evidence for EHR-specific SBT for healthcare professionals.

The strength of this scoping review protocol lies in its in-depth search strategy, elaborate eligibility criteria and clear data extraction plan. This protocol provides a methodologically rigorous template for future scoping review studies for identifying and mapping early evidence for interventions. Findings from the review will be submitted to relevant journals such as the British Medical Journal, and BMC Medical Informatics and Decision Making. Further, we aim to share our results with relevant key stakeholders including clinicians, health information managers, EHR vendors, policy-makers and healthcare organisations to provide a direction for future researchers seeking to develop and implement EHR-specific SBT.

Footnote

Contributors JKN contributed to the conceptualisation of the study, wrote and edited the manuscript. KA and LM contributed to the conceptualisation of the study and edited the manuscript.

Funding This project was supported by grant number R18HS025597 from the Agency for Healthcare Research and Quality.

Disclaimer The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.

Patient consent for publication Not required.

Provenance and peer review Not commissioned; externally peer reviewed.

References

¹ Gunter TD, Terry NP. The emergence of national electronic health record architectures in the United States and Australia: models, costs, and questions. J Med Internet Res 2005; 7: e3. doi:10.2196/jmir.7.1.e3 http://www.ncbi.nlm.nih.gov/pubmed/15829475

² Stephenson LS, Gorsuch A, Hersh WR, et al. Participation in EHR based simulation improves recognition of patient safety issues. BMC Med Educ 2014; 14: 224. doi:10.1186/1472-6920-14-224 http://www.ncbi.nlm.nih.gov/pubmed/25336294

³ Ash JS, Sittig DF, Dykstra RH, et al. Categorizing the unintended sociotechnical consequences of computerized provider order entry. Int J Med Inform 2007; 76 Suppl 1: S21–7. doi:10.1016/j.ijmedinf.2006.05.017 http://www.ncbi.nlm.nih.gov/pubmed/16793330

⁴ Colicchio TK, Cimino JJ, Del Fiol G. Unintended consequences of nationwide electronic health record adoption: challenges and opportunities in the Post-Meaningful use era. J Med Internet Res 2019; 21: e13313. doi:10.2196/13313 http://www.ncbi.nlm.nih.gov/pubmed/31162125

⁵ Mohan V, Scholl G, Gold JA. Intelligent simulation model to facilitate EHR training. AMIA Annu Symp Proc 2015; 2015: 925–32. http://www.ncbi.nlm.nih.gov/pubmed/26958229

⁶ March CA, Steiger D, Scholl G, et al. Use of simulation to assess electronic health record safety in the intensive care unit: a pilot study. BMJ Open 2013; 3: e002549. doi:10.1136/bmjopen-2013-002549 http://www.ncbi.nlm.nih.gov/pubmed/23578685

⁷ Gaba DM. The future vision of simulation in health care. Qual Saf Health Care 2004; 13 Suppl 1: i2–10. doi:10.1136/qshc.2004.009878 http://www.ncbi.nlm.nih.gov/pubmed/15465951

⁸ Okuda Y, Bryson EO, DeMaria S, et al. The utility of simulation in medical education: what is the evidence? Mt Sinai J Med 2009; 76: 330–43. doi:10.1002/msj.20127 http://www.ncbi.nlm.nih.gov/pubmed/19642147

⁹ Sunderland A, Nicklin J, Martin A. Simulation and quality in clinical education. Open Med J 2017; 4: 26–34. doi:10.2174/1874220301704010026

¹⁰ Cook DA, Hatala R, Brydges R, et al. Technology-enhanced simulation for health professions education: a systematic review and meta-analysis. JAMA 2011; 306: 978–88. doi:10.1001/jama.2011.1234 http://www.ncbi.nlm.nih.gov/pubmed/21900138

¹¹ Cook DA, Brydges R, Hamstra SJ, et al. Comparative effectiveness of technology-enhanced simulation versus other instructional methods: a systematic review and meta-analysis. Simul Healthc 2012; 7: 308–20. doi:10.1097/SIH.0b013e3182614f95 http://www.ncbi.nlm.nih.gov/pubmed/23032751

¹² Zendejas B, Brydges R, Wang AT, et al. Patient outcomes in simulation-based medical education: a systematic review. J Gen Intern Med 2013; 28: 1078–89. doi:10.1007/s11606-012-2264-5 http://www.ncbi.nlm.nih.gov/pubmed/23595919

¹³ Cook DA, Andersen DK, Combes JR, et al. The value proposition of simulation-based education. Surgery 2018; 163: 944–9. doi:10.1016/j.surg.2017.11.008 http://www.ncbi.nlm.nih.gov/pubmed/29452702

¹⁴ Seidman PA, Maloney LM, Olvet DM, et al. Preclinical simulation training of medical students results in better procedural skills performance in end of the year three objective structured clinical evaluation assessments. Med Sci Educ 2017; 27: 89–96. doi:10.1007/s40670-017-0371-y

¹⁵ Feins RH, Burkhart HM, Conte JV, et al. Simulation-based training in cardiac surgery. Ann Thorac Surg 2017; 103: 312–21. doi:10.1016/j.athoracsur.2016.06.062 http://www.ncbi.nlm.nih.gov/pubmed/27570162

¹⁶ Hoskote SS, Khouli H, Lanoix R, et al. Simulation-based training for emergency medicine residents in sterile technique during central venous catheterization: impact on performance, policy, and outcomes. Acad Emerg Med 2015; 22: 81–7. doi:10.1111/acem.12551 http://www.ncbi.nlm.nih.gov/pubmed/25556399

¹⁷ Lee Chang A, Dym AA, Venegas-Borsellino C, et al. Comparison between simulation-based training and Lecture-based education in teaching situation awareness. A randomized controlled study. Ann Am Thorac Soc 2017; 14: 529–35. doi:10.1513/AnnalsATS.201612-950OC http://www.ncbi.nlm.nih.gov/pubmed/28362531

¹⁸ Lateef F. Simulation-based learning: just like the real thing. J Emerg Trauma Shock 2010; 3: 348–52. doi:10.4103/0974-2700.70743 http://www.ncbi.nlm.nih.gov/pubmed/21063557

¹⁹ Scalese RJ, Obeso VT, Issenberg SB. Simulation technology for skills training and competency assessment in medical education. J Gen Intern Med 2008; 23 Suppl 1: 46–9. doi:10.1007/s11606-007-0283-4 http://www.ncbi.nlm.nih.gov/pubmed/18095044

²⁰ Rajaram A, Hickey Z, Patel N, et al. Training medical students and residents in the use of electronic health records: a systematic review of the literature. J Am Med Inform Assoc 2020; 27: 175–80. doi:10.1093/jamia/ocz178

²¹ Munn Z, Peters MDJ, Stern C, et al. Systematic review or scoping review? guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol 2018; 18: 143. doi:10.1186/s12874-018-0611-x http://www.ncbi.nlm.nih.gov/pubmed/30453902

²² Shamseer L, Moher D, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015; 349: g7647. doi:10.1136/bmj.g7647

²³ Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 2018; 169: 467–73. doi:10.7326/M18-0850 http://www.ncbi.nlm.nih.gov/pubmed/30178033

²⁴ Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol 2005; 8: 19–32. doi:10.1080/1364557032000119616

²⁵ Häyrinen K, Saranto K, Nykänen P. Definition, structure, content, use and impacts of electronic health records: a review of the research literature. Int J Med Inform 2008; 77: 291–304. doi:10.1016/j.ijmedinf.2007.09.001 http://www.ncbi.nlm.nih.gov/pubmed/17951106

²⁶ Covidence. Better systematic review management. Available: https://www.covidence.org/home [Accessed 13 Dec 2019 ].

²⁷ Colquhoun HL, Levac D, O'Brien KK, et al. Scoping reviews: time for clarity in definition, methods, and reporting. J Clin Epidemiol 2014; 67: 1291–4. doi:10.1016/j.jclinepi.2014.03.013 http://www.ncbi.nlm.nih.gov/pubmed/25034198

²⁸ Institute of Medicine (US) Committee on Data Standards for Patient Safety. Key capabilities of an electronic health record system: letter report. Washington, DC: National Academies Press, 2003.