Impact of electronic health records on predefined safety outcomes in patients admitted to hospital: a scoping review

Abstract

Objectives

Review available evidence for impact of electronic health records (EHRs) on predefined patient safety outcomes in interventional studies to identify gaps in current knowledge and design interventions for future research.

Design

Scoping review to map existing evidence and identify gaps for future research.

Data sources

PubMed, the Cochrane Library, EMBASE, Trial registers.

Study selection

Eligibility criteria: We conducted a scoping review of bibliographic databases and the grey literature of randomised and non-randomised trials describing interventions targeting a list of fourteen predefined areas of safety. The search was limited to manuscripts published between January 2008 and December 2018 of studies in adult inpatient settings and complemented by a targeted search for studies using a sample of EHR vendors. Studies were categorised according to methodology, intervention characteristics and safety outcome.

Results from identified studies were grouped around common themes of safety measures.

Results

The search yielded 583 articles of which 24 articles were included. The identified studies were largely from US academic medical centres, heterogeneous in study conduct, definitions, treatment protocols and study outcome reporting. Of the 24 included studies effective safety themes included medication reconciliation, decision support for prescribing medications, communication between teams, infection prevention and measures of EHR-specific harm. Heterogeneity of the interventions and study characteristics precluded a systematic meta-analysis. Most studies reported process measures and not patient-level safety outcomes: We found no or limited evidence in 13 of 14 predefined safety areas, with good evidence limited to medication safety.

Conclusions

Published evidence for EHR impact on safety outcomes from interventional studies is limited and does not permit firm conclusions regarding the full safety impact of EHRs or support recommendations about ideal design features. The review highlights the need for greater transparency in quality assurance of existing EHRs and further research into suitable metrics and study designs.

Correspondence to Dr Christian Peter Subbe; [email protected]

Strengths and limitations of this study

• Scoping review to identify the gaps in research on assessing the impact of electronic health records) on patient safety.

• Only interventional clinical studies were included.

• Limitation of search to terms from a previously validated authoritative search strategy.

• Exclusion of observational and feasibility studies.

Introduction

Caring for patients with complex conditions safely and competently mandates having access to the right information at the right time.¹ Ineffective sharing of information between providers and patients seriously impedes the quality and safety of patient care and is a leading cause of adverse events in hospital.² Harm from medical care is common, has a significant associated morbidity and mortality and affects the mental health of staff as well as the financial performance of institutions.³ A small number of categories of patient harm account for the bulk of adverse events.⁴ Most interventions aimed at reducing harm have included introducing a digital health record while restructuring the patient documentation and communication.⁵

It is widely accepted wisdom that the introduction of comprehensive systems for documentation and communication such as electronic health records (EHRs) should improve the safer delivery of care. Mortality improves after implementation of EHRs in smaller non-teaching hospitals.⁶ The number of reported adverse events changes after implementation of EHRs with ‘meaningful usage’ functionality⁷ but it is unclear whether changes are due to improved practice or changed event reporting. There are technical standards for EHR implementation and metrics for meaningful usage have focused on technical and efficiency aspects but not safety outcomes.⁸ There is hence the need to review the existing evidence for this specific aspect of care at a time of increasing spread of EHRs.

The objective of this scoping review is to map key concepts as a basis for a deeper understanding on the effects of electronic record systems on commonly used clinical safety metrics while identifying gaps in our current knowledge to inform design of future research and the design of more effective EHRs.

Methods

Scoping reviews are a traceable method of ‘mapping’ areas of research and highlighting gaps in the literature for future research.⁹ Scoping reviews are a useful tool in the ever-increasing arsenal of evidence synthesis approaches and require rigorous and transparent methods to ensure that the results are trustworthy.¹⁰ We used OMalley’s and Arksey¹¹ framework for undertaking a scoping review. This methodology summarises the evidence available on a topic in order to convey the breadth and depth of that topic by mapping the existing literature in a field of interest in terms of the volume, nature and characteristics of the primary research and identify gaps in the existing literature. In line with the methodology of scoping reviews, a formal evaluation of the quality of the studies was not undertaken.

The review included the following five key phases¹: identifying the research question,² identifying relevant studies,³ study selection,⁴ charting the data and⁵ collating, summarising, and reporting the results. A detailed review protocol can be obtained from the primary author on request.

A checklist for the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews can be found in online supplemental appendix 1.

Research question

This review was guided by the question: ‘How do patients admitted to hospital (P) benefit from implementation of an EHR (I) compared with patients not exposed to this or exposed to a different technology (C) in relation to commonly used outcome measures of safe care(O).’ Our PICO¹² search strategy for identifying and selection of studies is outlined below. The studies were divided into categories based on similarities in their main objectives/findings and the themes discussed.

Data sources and search strategy

The initial search was undertaken in March 2019 on studies published between January 2008 and December 2018, in the following databases: PubMed (including MEDLINE) and Embase, the European Trials Register, the Australian New Zealand Clinical Trials Register, the International Standard Randomised Controlled Trial Register and the Cochrane Library with supplementary searches on Google. The databases were selected to be comprehensive and to cover a broad range of disciplines. No limits on language, subject or type were placed on the database search. The initial search was conducted in March 2019 with the supplementary searches run in December 2019.

We used a validated algorithm from a literature review on search terms for studies on patient safety¹³ that was subsequently used by an authoritative systematic review of interventions to reduce adverse events in hospital.¹⁴ Online supplemental appendix 2 provides a sample listing of the search query terms tailored to the specific requirements of each database.

Fourteen topics of patient safety were identified in the review,¹⁴ including adverse drug events, infection, delirium, adverse event after hospital discharge or clinical handover, fall, adverse event in surgery, cardiopulmonary arrest, venous thromboembolism, staffing, pressure ulcer, mechanical complication and underfeeding, clinical pathway, safety culture, external inspection. EHRs were defined according to the National Centre for Biotechnological Information as Media that facilitate transportability of pertinent information concerning patients illness across varied providers and geographical locations.¹⁵

Study selection process

The study initial selection for inclusion was based on the title and abstract of the studies that were reviewed to preclude waste of resources in procuring articles that did not meet the minimum inclusion criteria. Two of the authors (CS and GT) reviewed titles, references and abstracts generated by the original search against the agreed inclusion and exclusion criteria. When the title and abstract provided insufficient information to determine the relevance, a full-text copy of the article was retrieved and reviewed. For the final selection, a full-text copy of each study was examined to determine if it fulfilled the inclusion criteria. The references of eligible studies were manually checked to identify additional relevant studies that were missed in the database searches (snowballing). The studies were reviewed for their research design and internal validity and the references of the selected studies were manually checked to identify additional relevant studies that were missed in the database search.

Eligibility criteria

Inclusion criteria: Record systems can be applied to inpatient or outpatient settings as well as to systems in community, primary or secondary care. This review focuses on medical record systems that are being used to support care of adult patients admitted to hospital wards. The review included publications identified in any language that reported experimental interventions in clinical trials that tested how records influenced patient safety. Only studies comparing two interventions or an intervention against usual or standard care were included. Studies excluded at this phase if they were found to not meet the eligibility criteria

Exclusion criteria: Study protocols, case series, technical descriptions, conference abstracts and studies limited to primary care records, outpatient care and highly specialised environments such as cardiac catheterisation laboratories, operating rooms or day-case units were excluded. Systematic reviews have been undertaken to document the safety impact of electronic prescribing systems. Studies examining the effects of interventions after hospital discharge were outside of the scope.

Supplemental searches

In order to validate the search strategy, additional searches were undertaken against the name of commonly used EHR vendors from the USA and UK identified from a Google search of EHRs companies. In order to assure the capture of important themes additional searches against the names of a sample of 12 major providers of electronic records was undertaken (online supplemental appendix 2). A total of 451 studies were screened. Four clinical trials that fulfilled inclusion criteria were identified. One of these¹⁶ reviewed safety alerts about gastrointestinal prophylaxis in a population that included inpatient and outpatient. The study did not allow to differentiate between the two groups and the study was thus excluded. Supplementary searches identified one further trial.¹⁷

Data extraction

Each article that met the study eligibility criteria was abstracted by using a standardised form based on a template by the Cochrane Collaboration.¹⁸ The data were extracted from the studies using an extraction tool that included the following items: article identifiers (authors, year of publication, objective); study identifiers (sample size, design, country, length of follow-up, inclusion and exclusion criteria); setting and population; outcome measures.

We organised the study characteristics in a tabular form. The identified studies were summarised according to key themes based on similarities of their main intervention and metrics and mapped against the 14 safety topics.

Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of the research. The study was not formally registered.

Patients or the public involvement

Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this literature review.

Role of the funding source

The Health Foundation provided funding for the study through an improvement science fellowship (CS). The funding agency did not participate in study conception, data collection, analyses, manuscript preparation, the decision to submit the manuscript for publication or any other part of the study.

Results

Search results

The initial searches identified 60 articles for full-text review in the scoping review and further analyses. Twenty-four papers met the eligibility and inclusion criteria and underwent a full-text abstraction (table 1). Because of heterogeneity of the study designs, participants and outcome measures, a meta-analysis was not feasible. The flow of articles through identification to final inclusion is represented in figure 1.

Table 1

Synopsis of 24 identified studies

Author	Country	RCT	Intervention	Type of safety metric	Unit of measurement	Impact
Abramson	USA	No	Transition between EHRs	Medication safety	Clinicians	n.s.
Adelman	USA	Yes	Change in version of EHR	System safety: wrong patient orders	Clinicians	Identification-re-entry function resulted in lower error rate (p<0.001).
Awdishu	USA	Yes	Notification: AKI	Medication safety: AKI	Clinicians	Adjusted prescriptions increased (p<0.001).
Barnett	USA	No	Transition between EHRs	Adverse event reporting: PSI-90, death and readmissions	Patients	n.s.
Boockvar	USA	Yes	Link to community EHR	Medication safety: Reconciliation	Patients	n.s.
Cardozo	USA	No	Notification: Trauma	Clinical pathway: cervical-spine clearance protocol	Patients	Improved compliance rate with pathway.
Cho	USA	No	EHR generated lists	Alerts	Clinical unit	Reduction in catheter related infections (p<0.05).
Cho	Korea	No	Notification: Falls risk assessment	Falls	Patients	Unchanged rate of falls.
Colpaert	Belgium	No	Transition to electronic system	Medication safety	Patients	Reduction in prescription errors (p<0.001).
Cook	USA	No	Transition to electronic system	Medication safety: antibiotic prescribing	Patients	Reduction in nosocomial infections (p<0.07).
Dowding	USA	No	Transition to electronic system	Hospital acquired pressure ulcers and falls	Patients	Increased documentation rates for hospital acquired pressure ulcers.
Fahey	USA	No	Change in version of EHR	Medication safety: wrong dosage of chemotherapy	Clinicians	Decrease in dosage error (n=0) compared with manual rounding (n=4).
Hess	USA	No	Transition from paper to electronic system	Medication safety: wrong dosage in chemotherapy	Clinicians	n.s.
Mishra	USA	No	Notification: Medication dosage	Medication safety: monitoring of Vancomycin dosage	Patients	Increase in frequency of trough levels (p<0.01).
Mohsen	USA	No	Change in version of EHR	Venous thrombembolism Reduction in inappropriate alerts	Patients	Alert reduction (p<0.001), increase in alert effectiveness (p<0.001), but decrease in alert efficiency (p=0.007).
Muhlenkamp	USA	Yes	Notification: Dosage alerts	Medication safety: removal of inappropriate or unnecessary alerts	Patients	Decrease in dosage alerts by 3.6%.
Nanchal	USA	Yes	Change in version of EHR	ICU handover: occurrence of non-routine events	Clinicians	Structured sign-out process reduced the occurrence of non-routine events reported by residents (p=0.005).
Nendaz	Switzerland	Yes	Notification: VTE risk assessment	Medication safety: decision support for VTE prophylaxis	Patients	Less overprescribing with e-alerts (p<0.01).
Schnipper	USA	Yes	Medication Reconciliation	Medication safety: adverse drug events	Patients	Changes significant at discharge but not admission.
Silbernagel	Switzerland	Yes	Notification: Complications of Atrial fibrillation	Medication safety: anticoagulation	Patients	Adequate prescription increased from 16% to 22% (p=0.021).
Spirk	Switzerland	Yes	Notification: VTE prophylaxis	Medication safety: VTE prophylaxis	Patients	n.s.
Weiss	USA	Yes	Checklist in EHR	Medication Safety: Antibiotic prescribing	Patients	Increase in number of days with empirical antibiotics (p<0.002).
Westbrook	Australia	No	Implementation of two EHRs	Medication Safety	Patients	44% reduction in serious errors, increase in system errors.
Wilson	USA	Yes	Notification: AKI	Medication Safety: AKI	Patients	Increase in creatinine checks (p<0.05) and reduction in deaths and dialysis (p<0.01) only in surgical stratum.

AKI, acute kidney injury; EHR, electronic health record; ICU, intensive care unit; n.s., not significant; PSI, Patient Safety Indicator (PSI-90); PSI-90, Patient Safety and Adverse Events Composite for the International Classification of Diseases; RCT, randomised controlled trial; VTE, venous thromboembolism.

Figure 1. Flow diagram of literature search of impact of electronic health records.

General characteristics of included studies

The studies originated from a number of countries: 18 from the USA, three from Switzerland and one each from Australia, Belgium and Korea. The studies involved general hospital wards areas, critical care^{19 20} and laboratory settings.²¹ Studies almost exclusively originated from academic medical institutions.

Eleven studies were randomised controlled trials; 13 studies were observational before-and-after studies or parallel group studies comparing electronic records with paper records^20–25 and other electronic records.^{25 26} The methodological quality of the studies was not formally assessed in line with the framework of scoping reviews.

The majority of studies involved only a single institution, some involved a group of hospitals and in one study, the authors reported from one geographical region.²⁷ The small number of multicentre studies involving between 2^{23 28} and 29²⁹ hospitals. The study duration ranged from a single month to 5 years with most studies lasting 6–18 months.

The studies examined interventions created by installing new electronic systems, changes delivered within an existing system and changes between different electronic systems.

The unit of examination were patients, hospitals units, pathology specimens and categories of healthcare professionals: nurses, physicians, prescribers.

Processes by which EHRs aimed to effect changes in safety outcomes

The majority of studies used interventions that created information aimed to influence the behaviour of physicians or prescribers, one study was aimed at nurses and no study was aimed at patients. The interventions included randomisation that was delivered at hospital, clinical units, clinician or patient levels. The comparative studies reviewed changes in adverse event reporting in hospitals implementing EHRs to those that did not implement EHR or in clinical departments preimplementation and postimplementation. Alerts were created for a random sample of patients or for a random sample of clinicians. Most studies reported on compliance with processes associated with safe care. Only a limited number of studies reported on actual adverse events or harm.^{24 25 27 29–32}

Metrics of impact

Results were mapped against the 14 predefined topics of patient safety (table 2): Significant evidence was identified for the topic of adverse drug events and limited evidence for the topics of clinical handover, venous thromboembolism, clinical pathways, pressure ulcers and falls. No evidence was identified for seven of the predefined topics.

Table 2

Synthesis of evidence for impact of implementation of EHR on predefined patient safety areas¹⁴

Patient safety area	Evidence for impact	Limitations
Adverse drug events	Evidence identified	Evidence for effects on documentation of allergies, drug interactions (process measures) and rate and reporting of adverse events (outcomes measures). Additional evidence from literature on specialist systems.
Infection	Limited evidence identified	Changes to antibiotic prescribing (process measure) and catheter related infections (outcome measure).
Delirium	None identified
Adverse event after hospital discharge or clinical handover	Limited evidence identified	The review was limited to effects in hospital. There was limited evidence for impact on clinical handover with reduction of ‘non-routine-events’ (outcome measure).
Falls	Limited evidence identified	No change in falls rates (outcome measure).
Adverse event in surgery	None identified
Cardiopulmonary arrests	Limited evidence identified	Evidence for reduced rate of cardiopulmonary arrests (outcome measure) from literature on specialist systems only.
Venous thromboembolism	Limited evidence identified	Changes in prescribing of prophylactic interventions (process measure).
Staffing	None identified
Pressure ulcer	Limited evidence identified	Improved documentation (process measure).
Mechanical complication and underfeeding	None identified
Clinical pathway	Limited evidence identified	Improved readability (process measure).
Safety culture	None identified
External inspection	None identified

EHR, electronic health record.

Identified studies were linked to safety themes. The patient safety themes identified included (1) the use of electronic notifications as reminders or alerts,^{22 25 31 33–39} (2) electronic notifications specifically in relation to medication safety,^{19 20 22 23 26–28 31–36 40–43} (3) communication between teams,^{27 28 44} (4) prevention and treatment of infections,^{19 22 24} and (5) harm caused by the architecture of the EHRs.^{29 45 46}

Theme (1): Electronic reminders: Automated notifications were used to alert prescribers to good practices in prescribing of antibiotics,^{19 22 24 43} prevention of falls and hospital acquired pressure ulcers,²⁵ oral anticoagulants,³⁵ thrombosis prophylaxis^{31 38} and nephrotoxic medications.^{33 36}

Best practice alerts for prescribing of antibiotics on general wards²² elicited only a response in 19% of prescribers in one study, with most of the responders following the advice that resulted in a reduction in the number of broad-spectrum antibiotics prescribed.

A study in a medical intensive care unit used checklists for antibiotics in the EHRs.¹⁹ These checklists were more effective on their own when compared with additional face-to-face prompting by a dedicated resident in changing the antibiotics to empirical antibiotics. Adverse events were not reported. The length of stay in the intensive care unit and standardised mortality rates were not different between the intervention and control groups.

The electronic reminders for clinicians to prescribe oral anticoagulants in patients with stroke and atrial fibrillation³⁵ resulted in a relative improvement in the rates of appropriate prescribing from 16% to 22%, however, the adverse effects were not reported.

The computer-generated alerts about rising creatinine levels that indicated acute kidney injuries resulted in a significantly higher rate of repeat creatinine laboratory checks.³⁶ There was a small increase in the subgroup of surgical ward patients in the number of renal consults ordered and in subsequent dialysis sessions. The primary combined outcomes of maximum creatinine rise, dialysis or death at 7 days, however, did not change.

Implementation of risk assessments for falls and hospital acquired pressure ulcers led to improved documentation rates²⁵: Falls rates did not change and the rate of hospital acquired pressure ulcers dropped continuously over the period of the investigation but no step-change after implementation of the EHR.

An electronic protocol for the clearance of the cervical spine after mechanical trauma resulted in improved documentation.³⁷ A falls-prediction algorithm⁴⁷ created a notification tool for falls prevention—this was tested against a non-matched control group.

Theme (2): Medication safety: The studies included reconciliation of medications,^{27 28} anticoagulants,^{31 35} antibiotic prescribing,^{19 22 24} acute kidney injury,^{33 36} calculating and monitoring of correct dosage^{32 42 43} and error-reporting.^{20 23 48} The effects on patient outcomes were either not reported or small and limited to subgroups of patients.

Theme (3): Communication between teams: Medication reconciliation on admission to the hospital was the focus of two studies.^{27 28} The reconciliation on hospital admission led to no measurable impact on safety outcomes. The electronic handover was related to a reduction in clinician reported ‘non-routine events’.⁴⁴

Theme (4): Infection: The prescribing practice of antibiotics^{19 22 24} was examined. Significant impact on patient outcomes was reported in one study with a fall in only one of several examined nosocomial infections.²⁴ A list of indwelling devices generated by the EHR was used to inform multi-disciplinary rounds with some evidence of lower exposure to risk.³⁹ The evidence was lacking on surrogate metrics describing the clinical course of infections such as the patients’ white cell count, level of C reactive protein or vital signs.

Theme (5): Harm caused by the EHR: The potential harm caused by introduction of the EHR was measured through a novel ‘retract-and-reorder’ tool^{45 46} that captured when clinicians prescribed corrected prescriptions and were reordered again for other patients. The majority of these events were likely near-misses. A reduction of harm from ‘wrong patient’ orders were attempted through the repeat of identity checks/verification⁴⁵ and a reduction in the number of maximum opened patient records.⁴⁶ A summary nationally reported measure of patient harm was used in another study to quantify the impact of transitions between medical records.²⁹

Additional gaps in understanding of impact of EHRs on safety outcomes

Studies reported limited explanatory context required to fully understand the likelihood of an impactful implementation such as staff workload, patient satisfaction, staff satisfaction or health economic outcomes. Staff satisfaction was measured in a single study⁴⁴ and only one study reported a patient-reported outcome measure: Adverse events collected through telephone interviews in the study on electronic discharge notifications were not specified and not affected by the intervention.³⁰

We found limitations in measurement of attributable harm at the patient level: A study examining the effect of a Health Information Exchange on adverse drug events found only 37 adverse events in 381 patients²⁷: All reported adverse events were characterised by temporary symptoms (eg, pain) or temporary organ dysfunction (eg, a rise in creatinine), and none caused serious or permanent harm. A study using electronic alerts for acute kidney injury³⁶ examined events such as the administration of contrast in patients-at-risk without clinical validation of the preventability of these events.

There was some degree of innovative functionality specific to electronic systems in relation to safety outcomes: An EHR specific ‘retract-and-reorder’ measure^{45 46} and a ‘patient safety composite measure’ for a selected validated summary indicator, the 'Patient Safety and Adverse Events Composite for the International Classification of Diseases' (PSI-90) ²⁹ were described. We were unable to identify a single trial using personal health records (PHRs) or patient portals in a hospital that reported on safety outcomes.

Discussion

This is the first scoping review, to the authors’ knowledge, to systematically evaluate the impact of EHR interventions on patient safety metrics in hospital. We found little published evidence for positive effects of EHRs on safety metrics that commonly feature in the literature such as hospital acquired infection, medication safety, allergies, falls. The review identified some evidence for a meaningful impact of EHRs in hospitals on surrogate outcomes that was largely restricted to changes in hospital prescribing practices. Limited follow-up periods might have been too short to capture the lasting effects beyond the immediate implementation period.

The review did not examine studies in primary care or paediatrics. Mortality was not included as a primary safety outcome as it depends on a large number of variables including the patient casemix but there are indications that patient mortality improves in a subgroup of hospitals that have implemented EHRs.⁶

Direct comparative clinical studies of EHRs by different vendors were missing. We were only able to identify two studies that directly compared EHRs. The first, a non-clinical study tested the safety processes in a simulated environment,⁴⁹ and demonstrated large differences in the number of computer keyboard clicks and the time required to perform basic work tasks, and the second, an observational audit study that compared the prescription errors between two EHRs.⁵⁰

We found no evidence for EHR related patient engagement at any level. Patients have been called the first line of defence or the ‘smoke alarm’ to raise alerts about potential patient harm and are able if invited to do so, to play a key role in monitoring their safety across the health continuum.^{51 52} PHRs held by patients might provide an obvious tool for enhanced patient safety but the evidence for a safety impact in primary care is limited to medication safety.⁵³ The American Veterans Administration Healthcare system has undertaken a robust evaluation of their PHR that indicates a better adherence to treatment plans but little data on whether this adherence leads to safer or cost-effective care⁵⁴ and patients’ active contribution to documentation systems in hospital is likely to enhance care.^{55 56}

Our scooping review has several limitations. First the search strategy was limited to safety outcomes predefined by a group of experts⁴ and we focused exclusively on EHRs. It is not clear whether other safety relevant outcomes could have been found in other studies of EHRs. Second, we focused on interventional studies to obtain a higher graded evidence and it is possible that safety outcomes are described in observational studies. Third, there is an understanding that monitoring systems for specific diseases that can be displayed through an EHR might be of benefit for safety outcomes such as measuring blood sugar levels in patients with diabetes⁵⁷ or the electrocardiogram in patients with a coronary event.⁵⁸ For unselected patient groups, there is evidence for the value of systems’ monitoring of vital signs that might be linked to an EHR or have their own recording systems; these authors have illustrated an impact on relevant clinical and safety outcomes^59–61 although with some methodological challenges.⁶² Fourth, the studies identified in this review used exemplar conditions and applications of electronic records. Frameworks to classify safety incidents in a broader, real world context^{63 64} are missing. Fifth, the number of studies identified was small and despite using a robust, systematic search strategy we were unable to generate a hierarchy of effective or ineffective EHR interventions. The comparison between EHR systems is difficult given the lack of operational and interoperative standards,⁶⁵ the lack of transparent data by the vendors and even in a simulated environment straight comparisons are exceedingly rare.⁴⁹ Sixth, the overwhelming number of studies originated in the USA which is highly influenced by the US healthcare regulatory and reimbursement schemes that are rather different from other healthcare systems. Finally, scoping reviews are not intended to assess the quality of the literature analysed. Nevertheless, this scoping review provides a comprehensive overview of the existing research and has clearly identified key themes and challenges for broader research which is needed.

EHRs can be used in many different ways in different hospitals. Linking the EHR intervention to a specific outcome might therefore be challenging even where process changes are the endpoints. Randomised trials might not be the most appropriate methodology for EHR evaluation and other generic service interventions because the effects at system level might be too diffuse. Carefully designed observational and adaptive interventional studies are needed to allow appropriate evaluation of service and policy interventions in this area.⁶⁶

The authoritative peer-reviewed search strategy deployed to identify publications reporting on patient safety topics uses a mix of process and outcome measures. Definition of these is subject to interpretation—that is, organisational culture could be used as an outcome measure as part of the quadruple aim or as a process that facilitates better quality of care for patients. Conceptually it would, however, be difficult to identify changes in outcomes without a model of change that does not involve some measure of change in process. Outcomes will of course depend on fidelity of implementation of processes but the absence of changes in safety critical processes is therefore likely to signify an absence in changes in safety outcomes.

The implementation of EHRs has got safety implications well beyond the scope of this review which range from the reliability of software and hardware, design or systems and user interfaces and risk of abuse and fraud.⁶⁷ We have also not examined the broader context of implementations: evidence suggests that nurses working in hospitals with no EHRs report poorer quality of care and patient safety⁶⁸ and cultural context and trust might modify impact.^69–71

Clinicians at the coal-face of care complain bitterly about poorly designed and supported EHR systems, which have unsuitable interfaces,⁷² add to workload, and fail to respond to change requests in a timely manner.⁷³ EHR’s are reported be the number one reason for clinician burnout and dissatisfaction.⁷⁴ Given the enormous investment costs in the development and deployment of the technology and the emerging evidence of the negative effects of EHR on clinician burnout,^{75 76} the lack of reported benefits in sustainable and measurable safety outcomes is surprising. We share the concerns of others that there is largely ‘anecdotal evidence of the fundamental expected benefits and risks relating to the organisational efficiency resulting from the storage and management facilities within the EHR and thus the potential for secondary uses’.⁷⁷ Health information systems designed for and by a clinical teams using a technology that enables real-time adaptation might provide greater efficiency for the staff in decreasing the time to complete standard tasks.⁷⁸

Unstructured and fragmented information is at the core of countless serious adverse events and the link between fragmented information and patient harm is well established in the literature.⁷⁹ Human factors and ergonomics design is part of the safety assurance of medical devices⁸⁰ but not the commonly used EHRs.

The EHRs are among the most expensive capital investments that health system leaders undertake with cost for an installation by a single organisation up to a billion dollars⁸¹ despite the absence of evidence for cost-effectiveness,⁸² and routine complaints about the deleterious effects of implementation on clinicians and their workflow.⁸³ EHRs have been introduced with an expectation of workflow and safety improvements that have failed to materialise.⁸⁴ An Australian study demonstrated that systematic errors in the usage of EHRs are common, and the audited files of 629 patients admitted to hospital were found to contain 493 errors related to the EHR and accounted for 42% of prescription errors,⁸⁵

Our review outlines a rich area for several key research questions, including the need to develop a clearer description of EHR interventions, using uniform and validated outcomes measures, and attending to care provider’s needs, attitudes and training.⁸⁶ Given the erosion of trust in the data safety of large projects, smaller pilots in multiple locations might be needed to develop EHR systems that aid patients, professionals and policy-makers.⁸⁷ Enormous amounts of data relevant to patient safety are collated within EHRs. It is likely that hospitals and vendors are undertaking internal reviews of safety outcomes for purposes of audit, quality improvement, internal quality assurance or research. Given the size of the investment in EHRs and the adverse public health impact of patient safety it would seem that these type of datasets should be made public for research and quality assurance.

Conclusions

The clinical consequences of EHR use for patients might be considerable but the available studies suggest a limited understanding about the safety or potentially harmful outcomes following the implementation of EHRs. The literature contains inadequate evidence to guide policy or a digital strategy for healthcare jurisdictions in how best to select and implement EHRs.

Our review highlights an urgent need for greater transparency in quality assurance of existing EHRs and further research into suitable outcome metrics and appropriate study designs.

Data availability statement

All data relevant to the study are included in the article or uploaded as online supplemental information.

Ethics statements

Patient consent for publication

Not required.

Ethics approval

This study did not involve human material or human data, so an ethics approval was not needed.

Footnote

Twitter @csubbe

Correction notice This article has been corrected since it first published. The provenance and peer review statement has been included.

Contributors CS and PB designed the study; CS and GT screened the articles, CS, GT and PB synthesised and interpreted the data; CS, GT and PB drafted and revised the manuscript; all authors approved of the submitted version to be published; all authors agreed to be accountable for all aspects of the work.

Funding This work was supported by for an Improvement Science Fellowship from The Health Foundation, London, UK (Unique Award Reference Number: AIMS 109820).

Competing interests CPS is partially funded through an Improvement Science Fellowship to investigate the use of Personal Health Records on patient safety.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

References

¹ Nobili A, Garattini S, Mannucci PM. Multiple diseases and polypharmacy in the elderly: challenges for the internist of the third millennium. J Comorb 2011; 1: 28–44. doi:10.15256/joc.2011.1.4 http://www.ncbi.nlm.nih.gov/pubmed/29090134

² Lawton R, McEachan RRC, Giles SJ, et al. Development of an evidence-based framework of factors contributing to patient safety incidents in hospital settings: a systematic review. BMJ Qual Saf 2012; 21: 369–80. doi:10.1136/bmjqs-2011-000443 http://www.ncbi.nlm.nih.gov/pubmed/22421911

³ Seys D, Wu AW, Van Gerven E, et al. Health care professionals as second victims after adverse events: a systematic review. Eval Health Prof 2013; 36: 135–62. doi:10.1177/0163278712458918 http://www.ncbi.nlm.nih.gov/pubmed/22976126

⁴ Coleman AL, Staff A, Emptage NP. Secretary for quality of care. BMJ Open 2014; 7.

⁵ Singh H, Naik AD, Rao R, et al. Reducing diagnostic errors through effective communication: harnessing the power of information technology. J Gen Intern Med 2008; 23: 489–94. doi:10.1007/s11606-007-0393-z http://www.ncbi.nlm.nih.gov/pubmed/18373151

⁶ Lin SC, Jha AK, Adler-Milstein J. Electronic health records associated with lower hospital mortality after systems have time to mature. Health Aff 2018; 37: 1128–35. doi:10.1377/hlthaff.2017.1658

⁷ Furukawa MF, Spector WD, Rhona Limcangco M, et al. Meaningful use of health information technology and declines in in-hospital adverse drug events. J Am Med Inform Assoc 2017; 24: 729–36. doi:10.1093/jamia/ocw183 http://www.ncbi.nlm.nih.gov/pubmed/28339642

⁸ Virginio LA, Ricarte ILM. Identification of patient safety risks associated with electronic health records: a software quality perspective. Stud Health Technol Inform 2015; 216: 55–9. http://www.ncbi.nlm.nih.gov/pubmed/26262009

⁹ Colquhoun HL, Levac D, O'Brien KK, et al. Scoping reviews: time for clarity in definition, methods, and reporting. J Clin Epidemiol 2014; 67: 1291–4. doi:10.1016/j.jclinepi.2014.03.013 http://www.ncbi.nlm.nih.gov/pubmed/25034198

¹⁰ Munn Z, Peters MDJ, Stern C, et al. Systematic review or scoping review? guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol 2018; 18: 143. doi:10.1186/s12874-018-0611-x http://www.ncbi.nlm.nih.gov/pubmed/30453902

¹¹ O’Malley H, Arksey L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol 2005; 8: 19–32.

¹² Richardson WS, Wilson MC, Nishikawa J, et al. The well-built clinical question: a key to evidence-based decisions. ACP J Club 1995; 123: A12–13. http://www.ncbi.nlm.nih.gov/pubmed/7582737

¹³ Tanon AA, Champagne F, Contandriopoulos A-P, et al. Patient safety and systematic reviews: finding papers indexed in MEDLINE, EMBASE and CINAHL. Qual Saf Health Care 2010; 19: 452–61. doi:10.1136/qshc.2008.031401 http://www.ncbi.nlm.nih.gov/pubmed/20457733

¹⁴ Zegers M, Hesselink G, Geense W, et al. Evidence-Based interventions to reduce adverse events in hospitals: a systematic review of systematic reviews. BMJ Open 2016; 6: e012555. doi:10.1136/bmjopen-2016-012555 http://www.ncbi.nlm.nih.gov/pubmed/27687901

¹⁵ U.S. national library for medicine [Internet]. Available: https://meshb.nlm.nih.gov/record/ui?ui=D057286 [Accessed 5 September 2019 ].

¹⁶ Lilih S, Pereboom M, van der Hoeven RTM, et al. Improving the effectiveness of drug safety alerts to increase adherence to the Guideline for gastrointestinal prophylaxis. Int J Med Inform 2017; 97: 139–44. doi:10.1016/j.ijmedinf.2016.10.002 http://www.ncbi.nlm.nih.gov/pubmed/27919373

¹⁷ Schnipper JL, Hamann C, Ndumele CD, et al. Effect of an electronic medication reconciliation application and process redesign on potential adverse drug events: a cluster-randomized trial. Arch Intern Med 2009; 169: 771. doi:10.1001/archinternmed.2009.51 http://www.ncbi.nlm.nih.gov/pubmed/19398689

¹⁸ Cochrane effective practice and organisation of care review group data collection checklist Cochrane effective practice and organisation of care review group (EPOC) data collection checklist.

¹⁹ Weiss CH, Dibardino D, Rho J, et al. A clinical trial comparing physician prompting with an unprompted automated electronic checklist to reduce empirical antibiotic utilization. Crit Care Med 2013; 41: 2563–9. doi:10.1097/CCM.0b013e318298291a http://www.ncbi.nlm.nih.gov/pubmed/23939354

²⁰ Colpaert K, Claus B, Somers A, et al. Impact of computerized physician order entry on medication prescription errors in the intensive care unit: a controlled cross-sectional trial. Crit Care 2006; 10: R21. doi:10.1186/cc3983 http://www.ncbi.nlm.nih.gov/pubmed/16469126

²¹ Kinonen CL, Watkin WG, Gleason BC, et al. Effects of computer reminders on complications of peripheral venous catheters and nurses’ adherence to a guideline in paediatric care-a cluster randomised study. Puebla I, editor. J Gen Intern Med 2017; 39: 868–74.

²² Schulz L, Osterby K, Fox B. The use of best practice alerts with the development of an antimicrobial stewardship navigator to promote antibiotic de-escalation in the electronic medical record. Infect Control Hosp Epidemiol 2013; 34: 1259–65. doi:10.1086/673977 http://www.ncbi.nlm.nih.gov/pubmed/24225610

²³ Westbrook JI, Reckmann M, Li L, et al. Effects of two commercial electronic prescribing systems on prescribing error rates in hospital in-patients: a before and after study. PLoS Med 2012; 9: e1001164. doi:10.1371/journal.pmed.1001164 http://www.ncbi.nlm.nih.gov/pubmed/22303286

²⁴ Cook PP, Rizzo S, Gooch M, et al. Sustained reduction in antimicrobial use and decrease in methicillin-resistant Staphylococcus aureus and Clostridium difficile infections following implementation of an electronic medical record at a tertiary-care teaching hospital. J Antimicrob Chemother 2011; 66: 205–9. doi:10.1093/jac/dkq404 http://www.ncbi.nlm.nih.gov/pubmed/21059617

²⁵ Dowding DW, Turley M, Garrido T. The impact of an electronic health record on nurse sensitive patient outcomes: an interrupted time series analysis. J Am Med Inform Assoc 2012; 19: 615–20. doi:10.1136/amiajnl-2011-000504 http://www.ncbi.nlm.nih.gov/pubmed/22174327

²⁶ Abramson EL, Malhotra S, Fischer K, et al. Transitioning between electronic health records: effects on ambulatory prescribing safety. J Gen Intern Med 2011; 26: 868–74. doi:10.1007/s11606-011-1703-z http://www.ncbi.nlm.nih.gov/pubmed/21499828

²⁷ Boockvar KS, Ho W, Pruskowski J, et al. Effect of health information exchange on recognition of medication discrepancies is interrupted when data charges are introduced: results of a cluster-randomized controlled trial. J Am Med Inform Assoc 2017; 24: 1095–101. doi:10.1093/jamia/ocx044 http://www.ncbi.nlm.nih.gov/pubmed/28505367

²⁸ Schnipper JL, Hamann C, Ndumele CD, et al. Effect of an electronic medication reconciliation application and process redesign on potential adverse drug events: a cluster-randomized trial. Arch Intern Med 2009; 169: 771–80. doi:10.1001/archinternmed.2009.51 http://www.ncbi.nlm.nih.gov/pubmed/19398689

²⁹ Barnett ML, Mehrotra A, Jena AB. Adverse inpatient outcomes during the transition to a new electronic health record system: observational study. BMJ 2016; 354: i3835. doi:10.1136/bmj.i3835 http://www.ncbi.nlm.nih.gov/pubmed/27471242

³⁰ Santana MJ, Holroyd-Leduc J, Southern DA, et al. A randomised controlled trial assessing the efficacy of an electronic discharge communication tool for preventing death or hospital readmission. BMJ Qual Saf 2017; 26: 993–1003. doi:10.1136/bmjqs-2017-006635 http://www.ncbi.nlm.nih.gov/pubmed/28821597

³¹ Spirk D, Stuck AK, Hager A, et al. Electronic alert system for improving appropriate thromboprophylaxis in hospitalized medical patients: a randomized controlled trial. J Thromb Haemost 2017; 15: 2138–46. doi:10.1111/jth.13812 http://www.ncbi.nlm.nih.gov/pubmed/28836340

³² Hess E, Palmer SE, Stivers A, et al. Impact of an electronic health record transition on chemotherapy error reporting. J Oncol Pharm Pract 2020; 26: 787–93. doi:10.1177/1078155219870590 http://www.ncbi.nlm.nih.gov/pubmed/31483749

³³ Awdishu L, Coates CR, Lyddane A, et al. The impact of real-time alerting on appropriate prescribing in kidney disease: a cluster randomized controlled trial. J Am Med Inform Assoc 2016; 23: 609–16. doi:10.1093/jamia/ocv159 http://www.ncbi.nlm.nih.gov/pubmed/26615182

³⁴ Nendaz MR, Chopard P, Lovis C, et al. Adequacy of venous thromboprophylaxis in acutely ill medical patients (impart): multisite comparison of different clinical decision support systems. J Thromb Haemost 2010; 8: 1230–4. doi:10.1111/j.1538-7836.2010.03817.x http://www.ncbi.nlm.nih.gov/pubmed/20175871

³⁵ Silbernagel G, Spirk D, Hager A, et al. Electronic alert system for improving stroke prevention among hospitalized Oral-Anticoagulation-Naïve patients with atrial fibrillation: a randomized trial. J Am Heart Assoc 2016; 5. doi: doi:10.1161/JAHA.116.003776. [Epub ahead of print: 22 07 2016 ]. http://www.ncbi.nlm.nih.gov/pubmed/27451467

³⁶ Wilson FP, Shashaty M, Testani J, et al. Automated, electronic alerts for acute kidney injury: a single-blind, parallel-group, randomised controlled trial. Lancet 2015; 385: 1966–74. doi:10.1016/S0140-6736(15)60266-5 http://www.ncbi.nlm.nih.gov/pubmed/25726515

³⁷ Cardozo S, Angus LDG. Use of an electronic C-spine clearance strategy to ensure compliance with confrontational examinations. Journal of Trauma Nursing 2015; 22: 255–60. doi:10.1097/JTN.0000000000000150

³⁸ Mohsen A, Kuperman E, McDanel J, et al. Using electronic clinical quality measures (eCQMs) to perform a venous thromboembolism prophylaxis rapid cycle quality improvement initiative. Jt Comm J Qual Patient Saf 2019; 45: 750–6. doi:10.1016/j.jcjq.2019.07.011 http://www.ncbi.nlm.nih.gov/pubmed/31474516

³⁹ Cho HJ, Khalil S, Poeran J, et al. "Lose the Tube": A Choosing Wisely initiative to reduce catheter-associated urinary tract infections in hospitalist-led inpatient units. Am J Infect Control 2017; 45: 333–5. doi:10.1016/j.ajic.2016.10.023 http://www.ncbi.nlm.nih.gov/pubmed/27919427

⁴⁰ Whalen K, Lynch E, Moawad I, et al. Transition to a new electronic health record and pediatric medication safety: lessons learned in pediatrics within a large academic health system. J Am Med Inform Assoc 2018; 25: 848–54. doi:10.1093/jamia/ocy034 http://www.ncbi.nlm.nih.gov/pubmed/29688461

⁴¹ Muhlenkamp R, Ash N, Ziegenbusch K, et al. Effect of modifying dose alerts in an electronic health record on frequency of alerts. Am J Heal Pharm 2019; 76: S1–8. doi:10.1093/ajhp/zxy016

⁴² Fahey OG, Koth SM, Bergsbaken JJ, et al. Automated parenteral chemotherapy dose-banding to improve patient safety and decrease drug costs. J Oncol Pharm Pract 2020; 26: 345–50. doi:10.1177/1078155219846958 http://www.ncbi.nlm.nih.gov/pubmed/31046608

⁴³ Mishra V, Chouinard M, Keiser J, et al. Automating vancomycin monitoring to improve patient safety. Jt Comm J Qual Patient Saf 2019; 45: 757–62. doi:10.1016/j.jcjq.2019.07.001 http://www.ncbi.nlm.nih.gov/pubmed/31526711

⁴⁴ Nanchal R, Aebly B, Graves G, et al. Controlled trial to improve resident sign-out in a medical intensive care unit. BMJ Qual Saf 2017; 26: 987–92. doi:10.1136/bmjqs-2017-006657 http://www.ncbi.nlm.nih.gov/pubmed/28784841

⁴⁵ Adelman JS, Kalkut GE, Schechter CB, et al. Understanding and preventing wrong-patient electronic orders: a randomized controlled trial. J Am Med Inform Assoc 2013; 20: 305–10. doi:10.1136/amiajnl-2012-001055 http://www.ncbi.nlm.nih.gov/pubmed/22753810

⁴⁶ Adelman JS, Applebaum JR, Schechter CB, et al. Effect of restriction of the number of concurrently open records in an electronic health record on Wrong-Patient order errors: a randomized clinical trial. JAMA 2019; 321: 1780. doi:10.1001/jama.2019.3698 http://www.ncbi.nlm.nih.gov/pubmed/31087021

⁴⁷ Cho I, Chung E. Effect of automatic inpatient fall prediction using routinely captured EMR data: preliminary results. Stud Health Technol Inform 2016; 225: 828–9. http://www.ncbi.nlm.nih.gov/pubmed/27332363

⁴⁸ Liao TV, Rabinovich M, Abraham P, et al. Evaluation of medication errors with implementation of electronic health record technology in the medical intensive care unit. Open Access J Clin Trials 2017; 9: 31–40. doi:10.2147/OAJCT.S131211

⁴⁹ Ratwani RM, Savage E, Will A, et al. A usability and safety analysis of electronic health records: a multi-center study. J Am Med Inform Assoc 2018; 25: 1197–201. doi:10.1093/jamia/ocy088 http://www.ncbi.nlm.nih.gov/pubmed/29982549

⁵⁰ Westbrook JI, Baysari MT, Li L, et al. The safety of electronic prescribing: manifestations, mechanisms, and rates of system-related errors associated with two commercial systems in hospitals. J Am Med Inform Assoc 2013; 20: 1159–67. doi:10.1136/amiajnl-2013-001745

⁵¹ O’hara J, Isden R. Identifying risks and monitoring safety: the role of patients and citizens. London, 2013.

⁵² Subbe CP, Øvretveit J, Quinn N, et al. Digital technology: opportunities and barriers for usage of personal health records in hospital - report from a -workshop of the health informatics unit at the Royal -College of physicians. Future Healthc J 2019; 6: 52–6. doi:10.7861/futurehosp.6-1-52 http://www.ncbi.nlm.nih.gov/pubmed/31098587

⁵³ De LS, Mold F, Sheikh A, et al. Patients’ online access to their electronic health records and linked online services : a systematic interpretative review, 2014: 1–12.

⁵⁴ Nazi KM, Hogan TP, Mcinnes DK. Evaluating patient access to electronic health records results from a survey of Veterans 2013; 51: 52–6.

⁵⁵ Renggli JF, Eken C, Siegrist V, et al. Usability of a web-based software tool for history taking in the emergency department. Acute Med 2020; 19: 131–7. http://www.ncbi.nlm.nih.gov/pubmed/33020756

⁵⁶ Subbe C. Who is allowed to read and write? Acute Med J 2020; 19: 116–7.

⁵⁷ Levitt DL, Silver KD, Spanakis EK. Inpatient continuous glucose monitoring and glycemic outcomes. J Diabetes Sci Technol 2017; 11: 1028–35. doi:10.1177/1932296817698499 http://www.ncbi.nlm.nih.gov/pubmed/28290224

⁵⁸ Ryan TJ, Anderson JL, Antman EM, et al. ACC/AHA Guidelines for the Management of Patients With Acute Myocardial Infarction:Executive Summary. Circulation 1996; 94: 2341–50. doi:10.1161/01.CIR.94.9.2341

⁵⁹ Jones S, Mullally M, Ingleby S, et al. Bedside electronic capture of clinical observations and automated clinical alerts to improve compliance with an early warning score protocol. Crit Care Resusc 2011; 13: 83–8. http://www.ncbi.nlm.nih.gov/pubmed/21627575

⁶⁰ Brown H, Terrence J, Vasquez P, et al. Continuous monitoring in an inpatient medical-surgical unit: a controlled clinical trial. Am J Med 2014; 127: 226–32. doi:10.1016/j.amjmed.2013.12.004 http://www.ncbi.nlm.nih.gov/pubmed/24342543

⁶¹ Subbe CP, Duller B, Bellomo R. Effect of an automated notification system for deteriorating ward patients on clinical outcomes. Crit Care 2017; 21: 52. doi:10.1186/s13054-017-1635-z http://www.ncbi.nlm.nih.gov/pubmed/28288655

⁶² Smith GB, Prytherch DR, Schmidt PE, et al. Bedside electronic capture - can it influence length of stay? Crit Care Resusc 2011; 13: 281–2. http://www.ncbi.nlm.nih.gov/pubmed/22129292

⁶³ Scanlon MC, Karsh B-T, Saran KA. Risk-Based patient safety metrics. advances in patient safety: new directions and alternative approaches (vol. 1: assessment, 2008.

⁶⁴ Sittig DF, Singh H. A red-flag-based approach to risk management of EHR-related safety concerns. J Healthc Risk Manag 2013; 33: 21–6. doi:10.1002/jhrm.21123 http://www.ncbi.nlm.nih.gov/pubmed/24078205

⁶⁵ Ratwani RM, Hettinger AZ, Fairbanks RJ. Barriers to comparing the usability of electronic health records. J Am Med Inform Assoc 2017; 24: ocw117: e191–3. doi:10.1093/jamia/ocw117 http://www.ncbi.nlm.nih.gov/pubmed/27572813

⁶⁶ Lilford RJ, Chilton PJ, Hemming K, et al. Evaluating policy and service interventions: framework to guide selection and interpretation of study end points. BMJ 2010; 341: c4412–c4413. doi:10.1136/bmj.c4413 http://www.ncbi.nlm.nih.gov/pubmed/20802000

⁶⁷ Bowman S. Impact of electronic health record systems on information integrity: quality and safety implications. vol. 10, perspectives in health information management / AHIMA, American health information management association. American Health Information Management Association 2013.

⁶⁸ Kutney-Lee A, Kelly D. The effect of hospital electronic health record adoption on nurse-assessed quality of care and patient safety. J Nurs Adm 2011; 41: 466–72. doi:10.1097/NNA.0b013e3182346e4b http://www.ncbi.nlm.nih.gov/pubmed/22033316

⁶⁹ Randell R, Abdulwahid M, Greenhalgh J. How and in what contexts does networked health it improve patient safety? elicitation of theories from the literature. in: studies in health technology and informatics. IOS Press 2019: 753–7.

⁷⁰ Dixon-Woods M, Bosk CL, Aveling EL, et al. Explaining Michigan: developing an ex post theory of a quality improvement program. Milbank Q 2011; 89: 167–205. doi:10.1111/j.1468-0009.2011.00625.x http://www.ncbi.nlm.nih.gov/pubmed/21676020

⁷¹ Dixon-Woods M, Leslie M, Tarrant C, et al. Explaining matching Michigan: an ethnographic study of a patient safety program. Implement Sci 2013; 8: 70. doi:10.1186/1748-5908-8-70 http://www.ncbi.nlm.nih.gov/pubmed/23786847

⁷² Patrick JD, Ieraci S. Good hit and bad hit. Med J Aust 2013; 198: 205. doi:10.5694/mja12.11350 http://www.ncbi.nlm.nih.gov/pubmed/23451964

⁷³ Patrick JD. The validity of personal experiences in evaluating hit. Appl Clin Inform 2010; 1: 462–5. doi:10.4338/ACI-2010-10-IE-0058 http://www.ncbi.nlm.nih.gov/pubmed/23616854

⁷⁴ Gardner RL, Cooper E, Haskell J, et al. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc 2019; 26: 106–14. doi:10.1093/jamia/ocy145 http://www.ncbi.nlm.nih.gov/pubmed/30517663

⁷⁵ Assis-Hassid S, Grosz BJ, Zimlichman E, et al. Assessing EHR use during Hospital morning rounds: a multi-faceted study. PLoS One 2019; 14: e0212816. doi:10.1371/journal.pone.0212816 http://www.ncbi.nlm.nih.gov/pubmed/30802267

⁷⁶ Downing NL, Bates DW, Longhurst CA. Physician burnout in the electronic health record era. Ann Intern Med 2019; 170: 216. doi:10.7326/L18-0604 http://www.ncbi.nlm.nih.gov/pubmed/30716744

⁷⁷ Black AD, Car J, Pagliari C, et al. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med 2011; 8: e1000387. doi:10.1371/journal.pmed.1000387 http://www.ncbi.nlm.nih.gov/pubmed/21267058

⁷⁸ Patrick J, Barach P, Besiso A. Management, and Implementation: The Rise of the Emergent Clinical Information System and the Chief Medical Information Officer. In: Sanchez J, Barach P, Johnson H, eds. Perioperative patient safety and quality: principles and practice. Springer US, 2017.

⁷⁹ Coiera E. Technology, cognition and error. BMJ Qual Saf 2015; 24: 417–22. doi:10.1136/bmjqs-2014-003484 http://www.ncbi.nlm.nih.gov/pubmed/26092566

⁸⁰ Carayon P, Xie A, Kianfar S. Human factors and ergonomics as a patient safety practice. BMJ Qual Saf 2014; 23: 196–205. doi:10.1136/bmjqs-2013-001812 http://www.ncbi.nlm.nih.gov/pubmed/23813211

⁸¹ Monica K. Top 5 Most Expensive EHR Implementations of 2017 [Internet]. Ehrinelligence. [cited 31 May 2019], 2017. Available: https://ehrintelligence.com/news/top-5-most-expensive-ehr-implementations-of-2017

⁸² Reis ZSN, Maia TA, Marcolino MS, et al. Is there evidence of cost benefits of electronic medical records, Standards, or Interoperability in hospital information systems? overview of systematic reviews. JMIR Med Inform 2017; 5: e26. doi:10.2196/medinform.7400 http://www.ncbi.nlm.nih.gov/pubmed/28851681

⁸³ Fry E, Schulte F. Death by a Thousand Clicks: Where Electronic Health Records Went Wrong | Fortune [Internet]. Fortune, 2019. Available: http://fortune.com/longform/medical-records/ [Accessed cited 2019 May 31 ].

⁸⁴ Poissant L, Pereira J, Tamblyn R, et al. The impact of electronic health records on time efficiency of physicians and nurses: a systematic review. J Am Med Inform Assoc 2005; 12: 505–16. doi:10.1197/jamia.M1700 http://www.ncbi.nlm.nih.gov/pubmed/15905487

⁸⁵ Westbrook JI, Baysari MT, Li L, et al. The safety of electronic prescribing: manifestations, mechanisms, and rates of system-related errors associated with two commercial systems in hospitals. J Am Med Inform Assoc 2013; 20: 1159–67. doi:10.1136/amiajnl-2013-001745 http://www.ncbi.nlm.nih.gov/pubmed/23721982

⁸⁶ Sittig DF, Wright A, Ash J, et al. New unintended adverse consequences of electronic health records. Yearb Med Inform 2016; 1: 7–12. doi:10.15265/IY-2016-023 http://www.ncbi.nlm.nih.gov/pubmed/27830226

⁸⁷ Frankel RM, Tilden VP, Suchman A. Physicians' trust in one another. JAMA 2019; 321: 1345. doi:10.1001/jama.2018.20569 http://www.ncbi.nlm.nih.gov/pubmed/30901023