Impact of e-alert for detection of acute kidney injury on processes of care and outcomes: protocol for a systematic review and meta-analysis

Abstract

Introduction

Acute kidney injury (AKI) is a common complication in hospitalised patients. It imposes significant risk for major morbidity and mortality. Moreover, patients suffering an episode of AKI consume considerable health resources. Recently, a number of studies have evaluated the implementation of automated electronic alerts (e-alerts) configured from electronic medical records (EMR) and clinical information systems (CIS) to warn healthcare providers of early or impending AKI in hospitalised patients. The impact of e-alerts on care processes, patient outcomes and health resource use, however, remains uncertain.

Methods and analysis

We will perform a systematic review to describe and appraise e-alerts for AKI, and evaluate their impact on processes of care, clinical outcomes and health services use. In consultation with a research librarian, a search strategy will be developed and electronic databases (ie, MEDLINE, EMBASE, CINAHL, Cochrane Library and Inspec via Engineering Village) searched. Selected grey literature sources will also be searched. Search themes will focus on e-alerts and AKI. Citation screening, selection, quality assessment and data abstraction will be performed in duplicate. The primary analysis will be narrative; however, where feasible, pooled analysis will be performed. Each e-alert will be described according to trigger, type of alert, target recipient and degree of intrusiveness. Pooled effect estimates will be described, where applicable.

Ethics and dissemination

Our systematic review will synthesise the literature on the value of e-alerts to detect AKI, and their impact on processes, patient-centred outcomes and resource use, and also identify key knowledge gaps and barriers to implementation. This is a fundamental step in a broader research programme aimed to understand the ideal structure of e-alerts, target population and methods for implementation, to derive benefit. Research ethics approval is not required for this review.

Systematic review registration number

CRD42016033033.

Correspondence to Dr Sean M Bagshaw; [email protected]

Background

Acute kidney injury (AKI) is an increasingly encountered complication, affecting 13–18% of hospitalised patients¹ and up to 60% of those admitted to an intensive care unit (ICU).^{2 3} Importantly, AKI has a significant modifying impact on patient outcome, imposing an increased risk for major morbidity, including chronic kidney disease (CKD), accelerated progression to end-stage kidney disease (ESKD) and mortality. Prior observational data have shown even relatively small increases in serum creatinine of 27 µmol/L (0.3 mg/dL) have been associated with several fold increased risk of mortality.¹ Moreover, patients suffering an episode of AKI consume greater resources and incur higher costs, largely from intensified monitoring, investigations and support necessitating longer hospital stays.

Consensus statements by expert panels currently recommend early tailored investigations and management measures for AKI such as urinalysis, ultrasound, drug dose adjustment and avoidance of nephrotoxins.^{4 5} The impact of these recommendations, which are mostly focused on harm avoidance, remains to be clearly determined. One of the challenges on evaluating the impact of these and other process of care measures (ie, monitoring, investigations, interventions) is the early recognition of AKI by clinicians. For example, Wilson et al⁶ showed that >25% of patients whose creatinine doubled had no documentation of AKI in their medical record.

In 1994, Rind et al proposed a software algorithm that automatically tracked creatinine changes and, once a threshold was reached, sent an alert through the hospital mailbox to the responsible team. However, this alert process integrated neither clinical decision support nor specific recommendations related to further monitoring, investigations or treatments.⁷ Since this publication, a number of studies using various designs of ‘alerts’, some automated and some relying on human interactions, have been described.⁸ In AKI, electronic alerts (e-alerts) are generally triggered by changes in serum creatinine and/or urinary output. Studies have evaluated the impact of these alerts on both the care process (ie, enhanced monitoring, added testing, modification or discontinuation of potential nephrotoxic drugs, etc) and on patient-related clinical outcomes (ie, worsening AKI, receipt of renal replacement therapy (RRT), mortality, etc). However, studies to date have shown inconsistent findings, with some showing improved outcomes^9–11 and others describing no differences.^{12 13} Moreover, wide variation in the methodological processes for e-alerting have been described in the literature, such as criteria and thresholds for triggering activation, the format of the alert, the target recipient of the alert and the degree of intrusiveness.

These observations would imply that the relative benefits for developing and implementing an e-alert system, along with its idealised structure for the detection of AKI and its impact on patient care processes, outcomes and health resource use, remain uncertain. Indeed, the Acute Dialysis Quality Initiative (ADQI) recently convened a consensus meeting focused on big data applications for AKI,¹⁴ including the need for continued development, refinement and rigorous evaluation of e-alerting in AKI.^{15 16} Accordingly, we propose to conduct an evidence synthesis and meta-analysis to describe the various e-alerts systems for AKI detection, and to assess their impact for patient care, outcomes and resource use.

Objectives

The aims of our systematic review are to:

• Describe the definitions and methods utilised for designing and implementing an e-alert (ie, automated, partially automated, target audience, intrusiveness) for AKI.

• Determine the impact of electronic alerting for AKI compared with no alerting on quality of care indicators and processes of care (ie, changes in frequency of monitoring, investigations (including urinalysis and ultrasound) and management (including medication review, chart documentation of AKI, decrease in the use of nephrotoxins, drugs dosage adjustment, fluid prescription, vasopressors or diuretics use, time to action and ICU or nephrology consult)).

• Determine the impact of electronic alerting for AKI compared with no alerting on patient-centred clinical outcomes (ie, peak creatinine, progression of AKI, proportion of patients fulfilling criteria for KDIGO stage 3 or RIFLE stage F, receipt of RRT, kidney recovery and mortality).

• Determine the impact of electronic alerting for AKI compared with no alerting on health services use (ie, ICU admission, ICU readmission, ICU length of stay, hospital length of stay).

Methods

Study design

A systematic review will be performed to characterise e-alerts for AKI and assess their impact on processes of care, clinical outcomes and health services use, using the guidelines from the Cochrane Collaboration and Centre for Reviews and Dissemination, and described according to the PRISMA-P guideline (available at: http://www.systematicreviewsjournal.com/content/4/1/1) (see online supplementary appendix 1).

Study registration

The systematic review is registered at PROSPERO (http://www.crd.york.ac.uk/prospero). Registration number CRD42016033033.

Criteria for considering studies for this review

Inclusion criteria

• Design: original data from randomised or quasi-randomised trials, observational cohort studies or before and after studies.

• Population: all hospitalised patients (ie, paediatric or adult) admitted to an ICU or a ward (ie, exclude emergency department and outpatient settings).

• Intervention: studies that implement an e-alert (ie, automated or partially automated) for the detection and diagnosis of AKI, using a clearly defined operational definition (ie, RIFLE (Risk, Injury, Failure, Loss, End-stage kidney disease), AKIN (Acute Kidney Injury Network), KDIGO (Kidney Disease: Improving Global Outcomes), other, etc).

• Outcomes: studies that report the impact of AKI e-alerts on at least one process of care indicator, patient-centred outcome or measure of health resource utilisation.

Exclusion criteria

Studies that do not fulfil all of the above criteria; those that are published in a language other than English or French; or those using non-electronic alerts, will be excluded.

Search methods for identification of studies

PROSPERO (http://www.crd.york.ac.uk/prospero) was searched for any registered systematic reviews on this topic (9 October 2015).

The search strategy will be developed in consultation with a research librarian at the Alberta Research Centre for Health Evidence (ARCHE) at the University of Alberta. The search strategy will undergo further peer-review by a second research librarian using the Peer Review of Electronic Search Strategies checklist.¹⁷ A comprehensive search for AKI and e-alerts concepts will be conducted in bibliographic databases: Ovid MEDLINE, Ovid EMBASE, CINAHL, Cochrane Library and Inspec. We will also search grey literature sources for health technology assessments and technical reports. Websites for e-alert technology producers (eg, Epic, Philips, iMDsoft, Cerner) will be searched in addition to the conference proceedings from the American Society of Nephrology, Society of Critical Care Medicine, International Symposium on Intensive Care and Emergency Medicine, European Society of Intensive Care Medicine and American Medical Informatics Association (AMIA). Concept searches for AKI will use a modified version of a published search filter,¹⁸ and database searches will be limited to publications from 1990 to current in English and French. Appropriate truncation and wildcards will be used in the search to account for plurals and/or variations in the spelling of search terms. Bibliographic records will be exported to an EndNote X7 (Thomson Reuters, Philadelphia, Pennsylvania, USA) database for screening. The cited and citing references of selected key studies will also be searched for relevant articles. See online supplementary appendix 2 for the proposed Ovid MEDLINE strategy.

Study selection

Potentially eligible articles will be initially identified by having two authors independently review the titles and abstracts of all articles identified by the search. The full text of all articles deemed potentially relevant will be retrieved and two authors will independently review the full text for inclusion, using predefined eligibility criteria. Any disagreements that arise will be resolved through discussion or referral to a third party.

Data extraction

Data will be abstracted from relevant studies, using a standardised electronic data collection form (see online supplementary appendix 3). This form will undergo pilot testing. This abstraction will be performed in duplicate by the same two authors. Any disagreements that arise will be resolved through discussion or referral to a third party. The authors of the retrieved studies and/or documents will be contacted for further information as necessary. Study methodological quality will be rated using the Modified Downs and Black¹⁹ checklist (see online supplementary appendix 4).

Outcomes

• Primary patient-centred outcome will be all-cause mortality as primarily determined by each study. Secondary outcomes will be ICU mortality, 28-day/30-day mortality, hospital mortality, peak creatinine, progression of AKI, proportion of patients fulfilling criteria for KDIGO stage 3 or RIFLE stage F, receipt of RRT and kidney recovery.

• Primary outcome for processes of care indicators will be dose adjustment and/or discontinuation of nephrotoxins. Secondary outcomes will be changes in frequency of monitoring, investigations (including urinalysis and ultrasound) and management (including medication review, chart documentation of AKI, fluid prescription, vasopressors or diuretics use, time to action and ICU or nephrology consult).

• Primary outcome for health service use will be hospital length of stay. Secondary outcomes will be ICU admission, ICU length of stay and ICU readmission.

Analysis

The primary analysis will be mixed narrative and meta-analytic where feasible. Each alert will be described according to trigger, type of alert, recipient and degree of intrusiveness (either passive, active, disruptive or very disruptive). We adapted our intrusiveness scale from a tool developed by Partners and Health Care for drug–drug interactions and published by Paterno et al.²⁰ All authors reached consensus on use of the modified version (see online supplementary appendix 5 for details). When feasible, data will be summarised and pooled to generate effect estimates of the impact of e-alerts on selected patient-centred outcomes, health resource use and processes of care. We will assess and quantify statistical heterogeneity for each pooled summary estimate, using Cochran's Q statistic and the I² statistic, respectively.²¹ Pooled analysis will be performed using random effects models and reported as ORs with 95% CIs for categorical variables and weighted mean differences 95% CIs for continuous variables, respectively. Subgroup analysis for categorical variables or meta-regression for continuous variables will be performed to assess for possible sources of heterogeneity according to the following predefined variables: criteria use for AKI definition (KDIGO, RIFLE, AKIN), type of unit (mixed ward/ICU vs ward alone), study design (observational vs randomised controlled trial vs before and after), study quality (good vs moderate vs poor (see online supplementary appendix 4 for definitions)) and degree of intrusiveness of the alert (passive vs active vs disruptive/very disruptive). Publication bias will be assessed using Egger's regression model and visualised with a funnel plot.²² Finally, the strength of the body of evidence will be assessed using the GRADE evidence system (http://clinicalevidence.bmj.com/x/set/static/ebm/learn/665072.html). All analyses will be performed using STATA statistical software, V.14 (Stata Corp, College Station, Texas, USA).

STRENGTHS AND LIMITATIONS OF THIS STUDY

Our review will be strengthening by a rigorous and systematic literature search strategy, including having undergoing peer review; clearly defined study eligibility criteria; clearly defined primary and secondary endpoints for evidence synthesis; and an a priori plan for interrogating the strength of evidence generated from our review. Some limitations are to be expected. First, based on the previous preliminary search of the literature, we expect most of the studies to be included in our systematic review will have a before and after design, have moderate quality and show heterogeneity in treatment effect contingent on the process or outcome measured. Moreover, as the context in which an e-alert is introduced may affect its effectiveness and the context may vary across studies, we expect this parameter to increase heterogeneity between the selected studies. Accordingly, we may not be able to draw firm conclusions with a high level of confidence on the efficacy and effectiveness of e-alerts in AKI. Importantly, we anticipate the majority of studies to have short-term follow-up for patient-centred outcomes.

Conclusion

In conclusion, this systematic review aims to critically appraise the scope of e-alerts developed for the detection and classification of AKI, along with defining the ideal type and format of e-alert for AKI. We recognise there may be context-specific e-alert methods more suitable for selected circumstances. Importantly, we aim to define the most robust and important care processes, patient outcomes and resource use indicators that should be integrated, and measured, when developing and implementing an e-alert system and/or performing future clinical investigations or quality assurance audits. Finally, we aim to synthesise the available evidence on the impact of AKI e-alerts on these same processes of care, and patient-centred and health resources utilisation outcomes.

Ethics and dissemination

Our systematic review will synthesise the literature on the value of e-alerts to detect AKI, to impact care processes, patient-centred outcomes and resource use, and also identify key knowledge gaps and barriers to implementation. This is a fundamental step in a broader research programme aimed to understand the ideal structure of e-alerts, target populations and methods for implementation to derive benefit. Research ethics approval is not required for this review. The results will be presented in national as well as international conferences in poster or oral presentations. The final manuscript will be published in a peer-reviewed journal.

The authors acknowledge Tara Landry for peer reviewing the strategy.

Footnote

Contributors SMB conceived the study. PL and SMB drafted the manuscript. RF created the research strategy. P-MV, NMS, FPW and OR reviewed the manuscript and provided their comment. SMB is the guarantor.

Competing interests SMB holds a Canada Research Chair in Critical Care Nephrology.

Provenance and peer review Not commissioned; externally peer reviewed.

References

¹ Chertow GM, Burdick E, Honour M, et al. Acute kidney injury, mortality, length of stay, and costs in hospitalized patients. J Am Soc Nephrol 2005; 16: 3365–70. doi:10.1681/ASN.2004090740

² Bouchard J, Acharya A, Cerda J, et al. A Prospective International Multicenter Study of AKI in the Intensive Care Unit. Clin J Am Soc Nephrol 2015; 10: 1324–31. doi:10.2215/CJN.04360514

³ Hoste EA, Bagshaw SM, Bellomo R, et al. Epidemiology of acute kidney injury in critically ill patients: the multinational AKI-EPI study. Intensive Care Med 2015; 41: 1411–23. doi:10.1007/s00134-015-3934-7

⁴ Mehta RL, Kellum JA, Shah SV, et al. Acute Kidney Injury Network: report of an initiative to improve outcomes in acute kidney injury. Crit Care 2007; 11: R31. doi:10.1186/cc5713

⁵ Kidney Disease: Improving Global Outcomes (KDIGO) Acute Kidney Injury Work Group. KDIGO clinical practice guideline for acute kidney injury. Kidney Int 2012; 2: 1–138. doi:10.1038/kisup.2012.1

⁶ Wilson FP, Bansal AD, Jasti SK, et al. The impact of documentation of severe acute kidney injury on mortality. Clin Nephrol 2013; 80: 417–25. doi:10.5414/CN108072

⁷ Rind DM, Safran C, Phillips RS, et al. Effect of computer-based alerts on the treatment and outcomes of hospitalized patients. Arch Intern Med 1994; 154: 1511–17. doi:10.1001/archinte.1994.00420130107014

⁸ Selby NM, Crowley L, Fluck RJ, et al. Use of electronic results reporting to diagnose and monitor AKI in hospitalized patients. Clin J Am Soc Nephrol 2012; 7: 533–40. doi:10.2215/CJN.08970911

⁹ Kolhe NV, Staples D, Reilly T, et al. Impact of compliance with a care bundle on acute kidney injury outcomes: a prospective observational study. PLoS ONE 2015; 10: e0132279. doi:10.1371/journal.pone.0132279

¹⁰ Terrell KM, Perkins AJ, Hui SL, et al. Computerized decision support for medication dosing in renal insufficiency: a randomized, controlled trial. Ann Emerg Med 2010; 56: 623–9. doi:10.1016/j.annemergmed.2010.03.025

¹¹ Colpaert K, Hoste EA, Steurbaut K, et al. Impact of real-time electronic alerting of acute kidney injury on therapeutic intervention and progression of RIFLE class. Crit Care Med 2012; 40: 1164–70. doi:10.1097/CCM.0b013e3182387a6b

¹² Wilson FP, Shashaty M, Testani J, et al. Automated, electronic alerts for acute kidney injury: a single-blind, parallel-group, randomised controlled trial. Lancet 2015; 385: 1966–74. doi:10.1016/S0140-6736(15)60266-5

¹³ Thomas ME, Sitch A, Baharani J, et al. Earlier intervention for acute kidney injury: evaluation of an outreach service and a long-term follow-up. Nephrol Dial Transplant 2015; 30: 239–44. doi:10.1093/ndt/gfu316

¹⁴ Bagshaw SM, Goldstein SL, Ronco C, et al. ADQI 15 Consensus Group. Acute kidney injury in the era of big data: the 15th Consensus Conference of the Acute Dialysis Quality Initiative (ADQI). Can J Kid Health Dis 2016; 3: 5. doi:10.1186/s40697-016-0103-z

¹⁵ Hoste EA, Kashani K, Gibney N, et al. 15 ADQI Consensus Group. Impact of electronic-alerting of acute kidney injury: workgroup statements from the 15(th) ADQI consensus Conference. Can J Kid Health Dis 2016; 3: 10. doi:10.1186/s40697-016-0101-1

¹⁶ James MT, Hobson CE, Darmon M, et al. Acute Dialysis Quality Initiative (ADQI) Consensus Group. Applications for detection of acute kidney injury using electronic medical records and clinical information systems: workgroup statements from the 15(th) ADQI consensus Conference. Can J Kid Health Dis 2016; 3: 9. doi:10.1186/s40697-016-0100-2

¹⁷ Sampson M, McGowan J, Cogo E, et al. An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol 2009; 62: 944–52. doi:10.1016/j.jclinepi.2008.10.012

¹⁸ Hildebrand AM, Iansavichus AV, Haynes RB, et al. High-performance information search filters for acute kidney injury content in PubMed, Ovid Medline and Embase. Nephrol Dial Transplant 2014; 29: 823–32. doi:10.1093/ndt/gft531

¹⁹ Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health 1998; 52: 377–84. doi:10.1136/jech.52.6.377

²⁰ Paterno MD, Maviglia SM, Gorman PN, et al. Tiering drug-drug interaction alerts by severity increases compliance rates. J Am Med Inform Assoc 2009; 16: 40–6. doi:10.1197/jamia.M2808

²¹ Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med 2002; 21: 1539–58. doi:10.1002/sim.1186

²² Egger M, Davey Smith G, Schneider M, et al. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997; 315: 629–34. doi:10.1136/bmj.315.7109.629