Applying natural language processing to identify emergency department and observation encounters for worsening heart failure

Abstract

Aims

Worsening heart failure (WHF) events occurring in non‐inpatient settings are becoming increasingly recognized, with implications for prognostication. We evaluate the performance of a natural language processing (NLP)‐based approach compared with traditional diagnostic coding for non‐inpatient clinical encounters and left ventricular ejection fraction (LVEF).

Methods and results

We compared characteristics for encounters that did vs. did not meet WHF criteria, stratified by care setting [i.e. emergency department (ED) and observation stay]. Overall, 8407 (22%) encounters met NLP‐based criteria for WHF (3909 ED visits and 4498 observation stays). The use of an NLP‐derived definition adjudicated 3983 (12%) of non‐primary HF diagnoses as meeting consensus definitions for WHF. The most common diagnosis indicated in these encounters was dyspnoea. Results were primarily driven by observation stays, in which 2205 (23%) encounters with a secondary HF diagnosis met the WHF definition by NLP.

Conclusions

The use of standard claims‐based adjudication for primary diagnosis in the non‐inpatient setting may lead to misclassification of WHF events in the ED and overestimate observation stays. Primary diagnoses alone may underestimate the burden of WHF in non‐hospitalized settings.

Introduction

Historically, there has been a focus on hospitalizations for heart failure (HF) as a marker of disease progression and severity. There is a growing recognition that worsening HF (WHF) events may occur in non-inpatient settings, including in the emergency department (ED) and observation units. These WHF events carry adverse prognostic importance and are increasingly captured in clinical trials using structured follow-up protocols. Concurrently, the healthy policy emphasis on 30 day rehospitalization rates may have unintentionally led to a shift in WHF events to non-hospitalized settings.¹ We previously reported that ED visits/observation stays and outpatient encounters account for half of WHF events using rule-based natural language processing (NLP) applied to electronic health record (EHR) data.^2,3 In this report, we evaluate the performance of an NLP-based approach compared with traditional diagnostic coding for non-inpatient clinical encounters by left ventricular ejection fraction (LVEF).

Methods

We included all ED and hospital-based observation stays between 1 January 2010 and 31 December 2019 with a diagnostic code for HF in any position across Kaiser Permanente Northern California (KPNC), a sociodemographically diverse integrated healthcare delivery system. As previously described, we used validated rule-based NLP algorithms applied to structured and unstructured EHR data to identify episodes of WHF.² WHF events were defined as ≥1 symptom, ≥2 objective findings, including ≥1 sign, and new administration of intravenous (IV) loop diuretics and/or new haemodialysis. These diagnostic criteria are based on a standardized consensus definition of WHF previously developed and validated in collaboration with the U.S. FDA.^4,5 NLP performance characteristics were outlined previously.² They stated that the inclusion of admissions with a primary and secondary discharge diagnosis of HF that satisfied systematic, prespecified diagnostic criteria resulted in a more than two-fold increase in the perceived population burden of hospitalizations for WHF. Additionally, 90% of hospitalizations with a primary discharge diagnosis of HF and approximately 30% of hospitalizations with a secondary discharge diagnosis of HF met the criteria for WHF.²

We compared characteristics for encounters that did vs. did not meet WHF criteria, stratified by care setting (i.e. ED and observation stay). To distinguish discrete WHF episodes during follow-up within each year, we applied standardized rules to collapse immediately consecutive encounters (i.e. ED visit followed by hospitalization for WHF). ED visits/observation stays for WHF occurring within 3 days prior to hospitalization or after discharge were removed and indicated as hospitalization for WHF. Outpatient WHF encounters within 3 days before or after the remaining ED visits/observation stays were indicated as ED/observation stays; consecutive outpatient WHF encounters occurring within 3 days of each other were collapsed.² We further compared the proportion of clinical encounters meeting the NLP-based WHF criteria by care setting (i.e. ED vs. observation stay) to HF identification based on diagnostic coding (i.e. in both primary and secondary coding positions). Separate analyses were performed by LVEF category at time of encounter [HF with reduced ejection fraction (HFrEF): <40%, HF with mildly reduced ejection fraction (HFmrEF): 40–49%, HF with preserved ejection fraction (HFpEF): ≥50%, and unknown ejection fraction].

Results

We identified 38 652 qualifying encounters with at least one diagnostic code for HF between 2010 and 2019: 26 599 (69%) were ED visits, and 12 053 (31%) were observation stays. The mean age was 75 ± 14 years, 50% were women, and 45% identified as ethnic minorities. Overall, 8407 (22%) encounters met NLP-based criteria for WHF (3909 ED visits and 4498 observation stays). Among 3344 ED encounters and 2517 observation stay encounters in which HF was the primary diagnosis, 2131 (64%) and 2293 (91%) met consensus NLP-derived definitions of WHF, respectively. The use of an NLP-derived definition adjudicated 3983 (12%) of non-primary HF diagnoses as meeting consensus definitions for WHF. The most common diagnosis indicated in these encounters was dyspnoea. Results were primarily driven by observation stays, in which 2205 (23%) encounters with a secondary HF diagnosis met the WHF definition by NLP (Figure 1A). Similar patterns were observed in patients with HFrEF, HFmrEF, HFpEF, and those with unknown LVEF across both encounter types (Figure 1B).

[IMAGE OMITTED. SEE PDF]

Discussion

In assessing >35 000 qualifying encounters, we generally found a varying concordance between primary diagnoses for HF and an NLP-based identification among non-inpatient presentations. Findings were most concordant for observation stays, while more than one in three ED primary diagnoses of HF did not meet consensus WHF definitions when using NLP. Using an NLP-based approach led to the classification of approximately one in four observation encounters as meeting WHF criteria, in which HF was not listed as the primary diagnosis.

Increasing data has suggested that WHF events outside traditional inpatient settings may carry important prognostic information. Recent HF clinical trials have incorporated WHF episodes treated in non-inpatient settings into primary endpoint analyses.⁵ The use of primary diagnoses for HF endpoint ascertainment has been proposed as an alternative to clinical adjudication to enhance pragmatism and reduce costs in clinical trials. Our analysis suggests that this approach may approximate consensus definitions for WHF for observation stays but may substantially misclassify ED presentations inappropriately as WHF events; NLP or traditional clinical endpoint committee (CEC)-based approaches may be needed to ensure consensus WHF definitions are met for these encounters.

Importantly, the true burden of WHF may be substantially higher than defined using primary diagnoses of HF alone, as many encounters without a primary diagnosis of HF met NLP-based consensus definitions. Given the accelerated adoption of data science to structured and unstructured EHR data, an NLP-based approach for endpoint ascertainment holds promise, particularly for trial designs in which adjudication committees may be infeasible or costly. However, future directions should include an assessment of the accuracy and comprehensiveness of this approach to CEC adjudication.

The differential performance of primary diagnosis vs. NLP-based identification of WHF among ED and observation stays is hypothesis-generating. The focus on ED throughput and lack of incentives around coding may lead providers to default to readily available diagnoses (e.g. HF), particularly in those with a known history, leading to misclassification. A secondary reason may be that the need for IV diuretics as part of the NLP likely excludes lower risk patients who might have mild symptoms, and the ED provider decides that adjustment of oral outpatient diuretics is sufficient. Observation visits may more closely resemble inpatient encounters (e.g. IV diuretic use—which is an inclusion criterion in some observation units), with greater time for further refinement of coding practices, explaining the greater concordance. However, our observations indicate that primary diagnoses may substantially underreport WHF event rates. These findings may have increased importance given the rise of HFmrEF and HFpEF, a condition with greater comorbid burden and an expected greater number of secondary coding diagnoses.

Overall, we offer a novel and potentially more complete approach to identifying WHF in non-inpatient settings. These data may inform quality improvement activities, HF resource planning, clinical trial recruitment, and outcome ascertainment. Furthermore, real-world evidence studies may benefit from these methods to identify the burden of WHF more accurately in non-inpatient settings.

This study has several limitations. We included encounters with primary or secondary diagnoses of HF; additional encounters without an HF diagnosis meeting NLP-based WHF definitions may have been missed. Given the accelerated adoption of data science to structured and unstructured EHR data, an NLP-based approach for endpoint ascertainment holds promise, particularly for trial designs in which adjudication committees may be infeasible or costly. However, future directions should include an assessment of the accuracy and comprehensiveness of this approach and CEC adjudication. These are data from one large integrated healthcare system; coding practices may not generalize to other care settings. LVEF was not systematically assessed at the time of presentation and may have a limited classification. The definition of urgent outpatient visits for HF in most trials includes office visits, which were not included in our study; NLP was applied to office visits and therefore does not fully replace traditional CEC outpatient WHF events.

Claims-based adjudication using the primary diagnosis in non-inpatient settings may lead to more misclassification of WHF events in the ED and underestimate observation stay events across the spectrum of LVEF.

Conflict of interest

Dr A.P.A. has received relevant research support through grants to his institution from the National Heart, Lung, and Blood Institute (K23HL150159), the American Heart Association (2nd Century Early Faculty Independence Award), The Permanente Medical Group, the Northern California Community Health Program, the Garfield Memorial Fund, Abbott Laboratories, Amarin Pharma, Inc., Edwards Lifesciences LLC, Esperion Therapeutics, Inc., and Novartis. Dr A.S.B. has received research grant support for his institution from the National Institutes of Health/National Heart, Lung, and Blood Institute, the National Institutes of Health/National Institute on Aging, the American College of Cardiology Foundation, and the Centers for Disease Control and Prevention. All other authors have no relevant financial disclosures.

Funding

The study was supported by research grants from Novartis AG (East Hanover, NJ, USA) and the Kaiser Permanente Northern California Community Benefit Program.

References

Wadhera RK, Joynt Maddox KE, Kazi DS, Shen C, Yeh RW. Hospital revisits within 30 days after discharge for medical conditions targeted by the Hospital Readmissions Reduction Program in the United States: National retrospective analysis. BMJ 2019;366: [eLocator: l4563]. doi:10.1136/bmj.l4563

Ambrosy AP, Parikh RV, Sung SH, Narayanan A, Masson R, Lam PQ, et al. A natural language processing–based approach for identifying hospitalizations for worsening heart failure within an integrated health care delivery system. JAMA Netw Open 2021;4: [eLocator: e2135152]. doi:10.1001/jamanetworkopen.2021.35152

Ambrosy AP, Parikh RV, Sung SH, Tan TC, Narayanan A, Masson R, et al. Analysis of worsening heart failure events in an integrated health care system. J Am Coll Cardiol 2022;80:111‐122. doi:10.1016/j.jacc.2022.04.045

Hicks K, Mahaffey KW, Mehran R, Nissen SE, Wiviott SD, Dunn B, et al. 2017 cardiovascular and stroke endpoint definitions for clinical trials. Circulation 2018;137:961‐972. doi:10.1161/CIRCULATIONAHA.117.033502

Vaduganathan M, Cunningham JW, Claggett B, Causland FM, Barkoudah E, Finn P, et al. Worsening heart failure episodes outside a hospital setting in heart failure with preserved ejection fraction. J Am Coll Cardiol‐Heart Fail 2021;9:374‐382. doi:10.1016/j.jchf.2021.01.014