Validation of the age, neutrophil to lymphocyte ratio, C reactive protein score on 28 day mortality in the National COVID cohort collaborative

Abstract

Identifying patients at high mortality risk can improve outcomes in SARS-CoV-2 pneumonia (COVID-19). We validate a prognostic model for mortality in patients hospitalized with COVID-19 receiving dexamethasone using a retrospective multi-centered study. This is a retrospective cohort study using the National COVID Cohort Collaborative (NC3) including 9,708 adult patients admitted for COVID-19 who received dexamethasone within 24 h of admission and remained hospitalized for 72 h. Previous work from a single-center cohort informed selection of prognostic variables including Age, day 3 neutrophil-lymphocyte Ratio, and day 3 C-reactive protein level (ARC Score). Variables from the development cohort were analyzed in a training cohort, and the resulting model was tested in a validation cohort. Age and day 3 measures of the neutrophil-lymphocyte ratio and C-reactive protein level were included in a logistic regression model to predict 28-day mortality. The 28-day mortality in this patient population was 15.4%. The area under the curve for the ARC model was 0.77 (95% confidence interval, 0.74–0.79). The Age, neutrophil-lymphocyte Ratio, and C-reactive protein (ARC) score identifies COVID-19 patients with a high risk of mortality within 28 days of hospitalization using clinical information on day 3 of hospitalization. ARC scores perform well across all variants of concern.

Introduction

SARS-CoV-2 continues to evolve and impact the healthcare community with high infection rates, hospitalizations, and deaths¹. Although effective vaccines and therapies have been employed to reduce morbidity and mortality, estimates of mortality for hospitalized patients with hypoxemia ranged from 7.1 to 17.1% during the first year of the pandemic². Early cohort studies in patients with COVID-19 demonstrated statistically significantly higher C-reactive protein (CRP) levels in patients requiring intensive care compared to those who did not require intensive care^3,4. Higher CRP levels, lymphopenia, and neutrophilia are demonstrated in non-survivors through day 30 post-infection⁵. These studies were completed before the widespread adoption of corticosteroids for patients with severe illness. Subsequent studies show that reduction in CRP levels in the 72 h after administration of corticosteroids by 50% is associated with a significant reduction in risk of death (OR 0.37, 95% CI 0.21–0.65)⁶. Investigations of pre-clinical inflammatory markers demonstrate higher levels of IL-6, IL-8, and TNF-alpha in survivors⁷. Phenotyping using TNF-alpha and clinically available labs (neutrophil and lymphocyte levels and d-dimer) can discriminate patients with favorable outcomes (hyperinflammatory phenotype) after corticosteroid administration and those with poor outcomes (non-hyperinflammatory phenotype)⁸.

These studies suggest that immune dysregulation is a central feature of severe COVID-19. Rapid ascertainment of this dysregulation early during hospitalization may help predict individual patient trajectory, response to therapy, or indication for additional treatments. These models depend on the clinical availability of laboratory markers or surrogate measures of immune dysregulation. While previous studies have identified important factors informing the mechanism and risk of mortality, the models are difficult to incorporate into bedside practice given unclear predictive thresholds or lack of widespread clinical availability of assays. Moreover, these reports are temporally limited to specific coronavirus variants, and a predictive tool generalizable to the larger COVID pandemic is needed. Dexamethasone is now standard of care for hospitalized COVID patients requiring supplemental oxygen, and analyses of clinical factors incorporating dexamethasone in the treatment pathway are necessary⁹.

Previously existing prognostic models (CURB-65, NEWS2, and qSOFA) applied to COVID-19 lack discriminatory accuracy, largely underestimating risk¹⁰. Newly developed scores can be overly cumbersome to calculate, have not been validated in geographically and demographically diverse datasets, or have ambiguous (single vital sign measurement at poorly defined time points) or clinically unavailable measures (such as cytokine profiles)^{11, 12–13}. A readily available bedside tool to predict mortality in those with COVID-19 pneumonia receiving dexamethasone in the hospital can help to triage patients to the appropriate level of care and predict those who may have worse outcomes¹⁴.

The National COVID Cohort Collaborative data enclave (N3C) was established as an open science collaborative between Clinical and Translational Science Award Programs in the United States and the National Center for Advancing Translational Science to facilitate efficient, transparent, and reproducible collaborative analytics during the COVID-19 pandemic¹⁵. The database collects electronic health record data from 76 centers in the United States for patients diagnosed with COVID-19. At the time of this cohort construction, the database contained comprehensive electronic health record data for > 14 million patients, including 5.6 million COVID-19 positive patients. N3C is an ideal platform for developing and validating models for COVID-19 related illnesses.

In this analysis, we developed and validated a clinical prediction score that could be applied early in the hospital stay to predict 28-day mortality in patients admitted to the hospital with COVID-19 pneumonia who received dexamethasone. We set out to identify a prognostic model utilizing demographic and readily available clinical data to maximize widespread utility.

Methods

Study design and base population

The retrospective cohort was derived from a large, multicenter data enclave (the National COVID Cohort Collaborative or N3C) utilizing level 3 deidentified data¹⁵. The study was performed in accordance with the principles of the Declaration of Helsinki, institutional review board approval was obtained, and requirement for informed consent was waived by the University of North Carolina Office of Human Research Ethics before the start of the study (IRB study number 22–0220, “External cohort validation of the ARC score for COVID-19 28-day mortality and escalation in O2 therapy in hospitalized patients,” date of approval: March 3, 2022). The N3C Data Enclave uses Palantir Foundry to integrate and harmonize electronic health record data from COVID positive patients and matched controls from 76 institutions across the United States. From the cohort of 5.6 million individuals, the base population was defined as any adult patient (age ≥18 years) with a COVID-19 diagnosis (by PCR or antigen test) hospitalized for 3 or more days with an admission date in the range of 3 days prior to until 10 days after diagnosis and who received dexamethasone (by oral or intravenous administration) within 24 h of COVID-19 diagnosis. Date of diagnosis was defined as the earliest positive COVID-19 test and limited to January 1, 2020, to June 20, 2022, for this analysis. Age was defined at the time of admission. Patients arriving to the participating centers on or placed on invasive mechanical ventilation or extracorporeal membrane oxygenation prior to inpatient admission were excluded from analysis as treatments at transferring facilities are not available in the data enclave.

Variable selection

Previous work from a single-center cohort informed selection of prognostic variables in the multicenter model¹⁴. Adults with COVID-19 admitted to the University of North Carolina Medical Center (Chapel Hill, NC) were included in a model development cohort. Those receiving dexamethasone for hypoxemia on day 1 of hospitalization between September 19, 2020, and January 15, 2021, and hospitalized through day 3 with day 3 laboratory variables and day 28 vital status were included in analysis (n = 80).¹⁴ Patients on invasive mechanical ventilation or extracorporeal membrane oxygenation prior to admission were excluded from analysis. Clinical characteristics associated with increased day 28 mortality were identified in bivariable analysis and incorporated into a multivariable regression model with sequential simplification by removing non-significant independent variables. The most parsimonious model included age, day 3 CRP levels (mg/L, normal range ≤10 mg/L), and day 3 neutrophil to lymphocyte ratio¹⁴.

In the multicenter cohort from the N3C data enclave, key demographic variables, medical comorbidities, and clinical variables during hospitalization were examined within the base population. Day 3 laboratory variables were available for those patients who had complete blood counts with differential (10⁹ cells/L or cells/µL) and CRP (excluding high-sensitivity assays) levels (mg/L, normal range ≤10 mg/L) ordered on day 3. This subset of patients comprised the final analytic cohort (Fig. 1). Descriptive demographic variables included age, race, ethnicity, sex, and body mass index. Comorbidities included prevalent diabetes, chronic obstructive pulmonary disease, coronary artery disease, chronic kidney disease, and metastatic cancer.

Outcomes

Day 28 vital status is captured in N3C primarily through the Electronic Health Record data which reliably captures deaths during hospitalization. Additionally, N3C has incorporated external mortality data through privacy preserving record linkage (PPRL) which captures deaths occurring after discharge from the hospital from government records and private obituary resources.

Modeling and statistical analysis

Descriptive statistics are presented with the mean and standard deviation for normally distributed continuous variables and median and interquartile range for nonnormally distributed continuous variables, and number and proportions for categorical variables. The analytic cohort was randomly partitioned into training and testing cohorts in a 3:1 ratio using a random sequence generated in R statistical software. We re-calculated our prior single center predictive regression model, including age at hospital admission, day 3 neutrophil to lymphocyte ratio, and day 3 CRP level in a logistic regression model with day 28 vital status as the outcome variable with 7,281 patients from the N3C training cohort. This logistic regression model was then validated using the reserved testing cohort (n = 2,427). Assessments of model discrimination and calibration for the validation cohort were performed with the area under the receiver operator characteristics curve and the Hosmer-Lemeshow goodness-of-fit statistic.

Several sensitivity analyses were performed to evaluate the model. First, the model’s performance was evaluated across the pandemic’s Alpha, Delta, and Omicron variant waves. The Centers for Disease Control and Prevention’s genetic variant tracker database was utilized to estimate the likely variant of infection based on test positivity date¹⁶. Samples testing positive on a date with a greater than 90% share of a single variant were included in the variant-based analysis. Second, the performance of the model was evaluated in a cohort restricted to patients receiving tocilizumab or baricitinib. Additionally, the clinical characteristics and outcomes of participants without complete day 3 laboratory data were compared to those included in the model (those with day 3 laboratory data) to evaluate the similarity between this subset and the participants with complete data. Finally, an equity subgroup analysis was performed across sex, race, and ethnicity according to previously published guidance from the RE²-AIM framework^17,18.

This study conforms to the TRIPOD guidelines for development of predictive algorithms. Analysis was performed using R (R Core Team, 2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/.

Results

The N3C data enclave included 5.6 million COVID-19 positive individuals across 75 centers at the time of analysis. Of those, 493,940 were hospitalized in a range of 3 days prior to and 10 days after test positivity (Fig. 1).

Fig. 1 [Images not available. See PDF.]

Cohort construction within the N3C data enclave, inclusion and exclusion criteria, and resultant sample size.

A total of 442,351 of these individuals were aged 18 or older at the time of diagnosis. Patients on invasive mechanical ventilation or extracorporeal membrane oxygenation prior to admission were excluded (7,545 and 549, respectively) as data from transferring hospitals is not available in the data enclave. After selecting for patients who received dexamethasone within 24 h of admission, 61,013 patients remained. A total of 48,595 patients remained hospitalized for 3 or more days. Within the cohort of hospitalized patients, 9,708 (20%) had complete data for risk variables and were included in analyses. Randomization of the 9,708 participants in the analytic cohort in a 3:1 ratio yielded a training cohort (n = 7,281) and testing cohort (n = 2,427) for the model. The demographic features, risk variables, and hospital characteristics were similar in the randomly partitioned training and testing cohorts (Table 1).

Table 1. Key demographic, clinical, treatment, and outcome variables in training, test, and full cohorts.

		Training Cohort	Test Cohort	Full Cohort
n		7,281	2,427	9,708
Female, no. (%)		3,102 (42.6)	1,051 (43.3)	4,153 (42.8)
Age, mean (SD)		62.5 (15.8)	61.8 (15.8)	62.3 (15.8)
Race, no. (%)
Asian		198 (2.7)	49 (2.0)	247 (2.5)
Black or African American		1,056 (14.5)	351 (14.5)	1,407 (14.5)
Hawaiian or Pacific Islander		13 (0.2)	1 (0.0)	14 (0.1)
Missing		1,016 (14.0)	348 (14.3)	1,364 (14.1)
Other Race		44 (0.6)	13 (0.5)	57 (0.6)
White		4,954 (68.0)	1,665 (68.6)	6,619 (68.2)
Ethnicity, no. (%)
Hispanic		1,064 (14.6)	359 (14.8)	1,423 (14.7)
Non-Hispanic		5,541 (76.1)	1,845 (76.0)	7,386 (76.1)
Missing		676 (9.3)	223 (9.2)	899 (9.3)
Diabetes mellitus, no. (%)		3,290 (45.2)	1,123 (46.3)	4,413 (45.5)
COPD, no. (%)		1,422 (19.5)	462 (19.0)	1,884 (19.4)
Coronary artery disease, no. (%)		675 (9.3)	234 (9.6)	909 (9.4)
Metastatic Cancer, no. (%)		227 (3.1)	58 (2.4)	285 (2.9)
Chronic kidney disease, no. (%)		1,839 (25.3)	614 (25.3)	2,453 (25.3)
BMI (kg/m2), mean (SD)		32.95 (8.6)	33.51 (9.1)	33.09 (8.7)
Day 3 neutrophil count (cells/µL), median (IQR)		6.81 (4.77, 9.30)	6.87 (4.77, 9.30)	6.83 (4.77, 9.30)
Day 3 lymphocyte count (cells/µL), median (IQR)		0.90 (0.60, 1.30)	0.90 (0.60, 1.30)	0.90 (0.60, 1.30)
Day 3 C-reactive protein level (mg/L), median (IQR)		33.0 (15.9, 63.2)	33.8 (15.8, 63.0)	33.2 (15.9, 63.2)
Day 3 neutrophil to lymphocyte ratio, median (IQR)		7.58 (4.43, 13.20)	7.45 (4.43, 12.76)	7.56 (4.43, 13.10)
Dexamethasone duration of therapy (days), median (IQR)		6.5 (3.0, 10.5)	6.5 (3.0, 10.5)	6.5 (3.0, 10.5)
Tocilizumab exposure, no. (%)		458 (6.3)	167 (6.9)	625 (6.4)
Baricitinib exposure, no. (%)		740 (10.2)	284 (11.7)	1024 (10.5)
Hospital length of stay, median (IQR)		8.0 (5.0, 14.0)	8.0 (5.0, 14.0)	8.0 (5.0, 14.0)
28-day mortality, no. (%)		1,123 (15.4)	369 (15.2)	1,492 (15.4)

The mean age of patients was 62.3 (standard deviation 15.8), and 43% were female. Patients represented diverse racial and ethnic backgrounds but were predominantly white and non-Hispanic (68% and 76%, respectively). Mortality at day 28 in this cohort of patients with COVID-19 hospitalized with hypoxemia was 15.4%. Comorbidities associated with COVID-19 morbidity were well represented in the cohort with diabetes in 46% of patients, chronic obstructive pulmonary disease in 19%, coronary artery disease in 9%, chronic kidney disease in 25%, and metastatic cancer in 3% of patients.

The 28-day mortality in the training cohort was 15.4% (1,123 deaths). Bivariable analysis with rank-sum testing of age, day 3 CRP, day 3 neutrophils, day 3 lymphocytes, and day 3 neutrophil to lymphocyte ratio demonstrated statistically significant differences in all variables between participants alive and deceased at day 28 with p-values of < 0.001 for all comparisons (Table 2).

Table 2. Laboratory values, clinical characteristics, and treatment exposures in those alive and deceased at day 28 in the training cohort. Bivariate analysis by day-28 vital status.

	Alive	Deceased	p-value1
n	6,158	1,123
Female, no. (%)	2,654 (56.9)	448 (39.9)	0.049
Age, mean (SD)	61.0 (15.7)	70.8 (13.2)	< 0.001
Diabetes mellitus, no. (%)	2,695 (43.8)	595 (53.0)	< 0.001
COPD, no. (%)	1,138 (18.5)	284 (25.3)	< 0.001
Coronary artery disease, no. (%)	528 (8.6)	147 (13.1)	< 0.001
Cancer, no. (%)	184 (3.0)	43 (3.8)	0.162
Chronic kidney disease, no. (%)	1,413 (22.9)	427 (37.9)	< 0.001
BMI (kg/m2), mean (SD)	33.2 (8.7)	31.6 (7.8)	< 0.001
Day 3 neutrophil count (cells/µL), median (IQR)	6.52 (4.60, 8.90)	8.40 (6.00, 11.52)	< 0.001
Day 3 lymphocyte count (cells/µL), median (IQR)	0.93 (0.60, 1.39)	0.60 (0.40, 0.91)	< 0.001
Day 3 C-reactive protein level (mg/L), median (IQR)	30.4 (14.6, 56.8)	57.3 (27.0, 103.9)	< 0.001
Day 3 neutrophil to lymphocyte ratio, median (IQR)	6.8 (4.10, 11.58)	13.57 (8.46, 21.92)	< 0.001
Dexamethasone duration of therapy (days), median (IQR)	6.0 (3.0, 10.5)	8.0 (4.0, 11.0)	< 0.001
Tocilizumab exposure, no. (%)	346 (5.6)	112 (10.0)	< 0.001
Baricitinib exposure, no. (%)	575 (9.3)	165 (14.7)	< 0.001
Remdesivir exposure, no. (%)	2,234 (37.9)	431 (38.4)	0.339
Hospital length of stay, median (IQR)	7.0 (5.0, 12.0)	13.0 (8.0, 18.0)	< 0.001
1. T−test for normally distributed continuous variables, rank sum testing for non−normally distributed continuous variables, chi−squared testing for categorical variables

This analysis indicates greater age (71 vs. 61 years), higher day 3 CRP (57.3 vs. 30.4 mg/L), and higher day 3 neutrophil to lymphocyte ratio (13.6 vs. 6.8) in those deceased at day 28. Logistic regression on the continuous variables of age, day 3 CRP, and day 3 neutrophil to lymphocyte ratio for prediction of day 28 mortality showed statistically significant odds ratios for all components of the regression model (S1 in the online data supplement) with a p-values < 0.001 for all odds ratios. The odds ratio for mortality for every 10-year increase in age was 1.51 (1.44–1.59), 10 unit increase in day 3 neutrophil to lymphocyte ratio was 1.32 (1.25–1.40), and 25 unit increase in day 3 CRP 1.22 (1.19–1.25).

The regression model from the training cohort was then applied to the testing cohort for validation. The area under the receiver-operator characteristic curve was 0.77 (95% CI 0.74–0.79). This model is a statistically significant improvement over individual variables that contribute to the model (age, day 3 neutrophil to lymphocyte ratio, and day 3 CRP), with p-values < 0.01 for all comparisons (Fig. 2).

Fig. 2 [Images not available. See PDF.]

ARC model performance in reserved validation (test) cohort.

The Hosmer-Lemeshow goodness-of-fit statistic showed a non-statistically significant p-value of 0.14 for 4 groups, indicating no evidence of poor model fit. A proposed dichotomization of variables for age of ≥ 70, CRP ≥ 70 mg/L, and neutrophil to lymphocyte ratio ≥ 10 allows for a more rapid application of the prognostic score at the bedside. The inflection points in the relationship between the age and laboratory values and mortality at day 28 in the training data set determined these cut-points. This dichotomization of risk variables assigns 1 point for each value above the cut point and determines the ARC score totaling the number of points an individual accumulates. This dichotomization does not significantly change the AUC of the model in the test set compared to continuous variables (p = 0.10) and simplifies score for bedside deployment (S2). Mortality by ARC score for the training and test cohorts is displayed in Table 3. In the testing cohort, increasing ARC score was associated with increasing 28-day mortality, as follows: 0, 3.9% mortality; 1, 14.7% mortality; 2, 29.2% mortality; 3, 48.9% mortality.

Table 3. ARC scores and associated day-28 mortality for training (calibration) and testing (validation) cohorts.

	Training Cohort		Testing Cohort
ARC Score1	n (%)	28-d mortality (95% CI)	n (%)	28-d mortality (95% CI)
0	2829 (38.9%)	4.5% (3.8, 5.4)	970 (39.9%)	3.9% (2.9, 5.3)
1	2573 (35.3%)	13.7% (12.4, 15.1)	857 (35.2%)	14.7% (12.4, 17.3)
2	1463 (20.1%)	29.6% (27.3, 32.0)	469 (19.3%)	29.2% (25.1, 33.6)
3	416 (5.7%)	50.5% (45.6, 55.4)	137 (5.6%)	48.9% (40.3, 57.6)

1. ARC score calculated by adding 1 point for each criterion met, a value derived from the beta coefficients of the logistic regression model using the dichotomous variables: Age of ≥70, CRP ≥70 mg/L, and Neutrophil to Lymphocyte ratio ≥10.

A sensitivity analysis with the participants excluded for incomplete day 3 laboratory data compared demographic, comorbid, and hospitalization specific variables to participants included in the cohorts. Although several variables were statistically different, clinically meaningful differences (e.g. 28-day mortality) were not observed between the analytic cohort and those excluded for missing variables (S3 in the online data supplement). Time-based analysis for broad classifications of variants of concern (Alpha, Delta, and Omicron) was determined using data from the Center for Disease Control and Prevention’s genomic surveillance dataset established through the National SARS-CoV-2 Strain Surveillance program and sequencing by commercial and academic laboratories. Strain identification was assigned if a single circulating strain accounted for greater than 90% of the samples sequenced in the CDC genomic surveillance set at the time of the positive test. The established timeframes for the Alpha, Delta, and Omicron variants were January 1, 2020, to May 29, 2021; July 23, 2021, to November 22, 2021; and January 8, 2022, onward, respectively. SARS-CoV-2 variant-specific analysis resulted in a statistically significantly decreased performance of the model for the Delta strain but not for the widely circulating Omicron strain, using Alpha as the reference (Fig. 3).

Fig. 3 [Images not available. See PDF.]

ARC model performance in Alpha, Delta, and Omicron variants.

The area under the ROC remains robust (0.78 for Alpha, 0.73 for Delta, and 0.77 for Omicron), suggesting good predictive value across the evolution of viral variants, vaccination, and inpatient treatment paradigms. The model performed less well in a sub-group analysis of patients receiving dexamethasone plus an additional immune modulator therapeutic (tocilizumab or baricitinib) with an area under the ROC of 0.72 but continued to outperform any of the individual elements in the score in this group of 1,637 individuals (S4 in the online data supplement). Introducing regression splines with a single knot to the neutrophil-to-lymphocyte ratio and CRP variables to account for inflection points in the relationship between these variables and mortality did not improve the performance of the model (data not shown).

Equity subgroup analysis was performed on the training dataset, evaluating the performance of the ARC model across sex, race, and ethnicity (S5). There were no statistically significant differences in model performance across sex with an area under the ROC of 0.77 (0.73–0.80) amongst males and 0.77 (0.73–0.81) amonst females (p = 0.94). Additionally, there were no statistically significant differences in model performance between those identified as white (area under the ROC of 0.76 [0.73–0.79]) and those identified as black or African American (0.72 [0.64–0.79]), (p = 0.37). Pairwise comparisons with other racial identities were not performed because of low sample sizes. Finally, there were no statistically significant differences in model performance across ethnicity with an area under the ROC of 0.76 (0.73–0.79) amongst non-Hispanic individuals and 0.82 (0.76–0.88) amonst Hispanic individuals (p = 0.07). Smoothed regression of continuous variables included in the model against mortality was performed in order to ensure a monotonic relationship between the variables and outcomes (S6).

Discussion

While previous existing clinical prediction scores have been reported and are useful, we set out to determine if a few key, readily available data commonly collected could predict outcomes in severe COVID-19. Using a large, broadly generalizable, representative United States based cohort, we have shown that the novel and simple ARC score (Age, Day 3 Neutrophil to Lymphocyte Ratio, Day 3 CRP) can accurately predict 28-day mortality among patients hospitalized for COVID-19 receiving dexamethasone. This score, informed by analyses from a small single-center cohort, was developed and validated in a large, multi-center database. The model performs well in the single-center derivation cohort and has similar discrimination in the multi-center testing cohort. The model has immediate bedside applicability and predicts a relevant, near-term outcome (28-day mortality). The cohorts from which this score was developed reflect the current treatment landscape for severe COVID-19 (receipt of dexamethasone in the context of hypoxemia), strengthening the clinical relevance of these findings. The score requires 3 elements, age, blood count with differential, and CRP, and is easily adapted to the bedside, including in lower resource settings. Data collection on day three of dexamethasone treatment reflects the physiologic changes after glucocorticoid administration and the current treatment paradigm for severe COVID-19. This score can help mobilize resources for patients more likely to deteriorate and communicate the expected prognosis and severity of disease with patients and family members.

This novel clinical prediction score contributes to the available models for predicting trajectory of patients hospitalized with severe COVID-19 focusing on similar mortality measures. Existing models developed before the onset of the pandemic significantly underestimate risk in patients hospitalized with COVID-19¹⁰. Models developed at the beginning of the pandemic have not been validated in the context of subsequent COVID-19 variants and treatment paradigms and are more complex to implement at the bedside without superior predictive capabilities (AUC for the 4 C score 0.77)¹¹. External validation of 32 models in a cohort of patients in the Greater Paris University Hospitals hospitalized between January 2020 and April 2021 did not demonstrate any with superior performance to the ARC score, with the best performance in the 4 C and ABCS models (AUC 0.79 for both)¹³.

Notably, the ARC score performs well across broadly classified variants, and specifically in the currently circulating Omicron variant. SARS-CoV-2 variants continue to evolve and spread, and mortality remains significant despite immunity from vaccination and prior infection and the availability of therapeutic agents. Further investigation as new variants emerge will be essential to demonstrate this model’s future validity.

The biological mechanisms explaining the neutrophil to lymphocyte ratio and C-reactive protein relationship with mortality in patients hospitalized with severe COVID-19 and receiving dexamethasone warrant further investigation. Lymphopenia occurs in COVID-19 without corticosteroids and likely involves several mechanisms¹⁹. Lymphopenia can be a significant component of the mechanism of action for corticosteroids²⁰. In some COVID-19 trials, “decreased lymphocyte count’ or “lymphopenia” are listed as adverse events when comparing corticosteroids to other immune modulators²¹. Attempting to uncouple the contributions to lymphopenia to either the biology of SARS-CoV2 infection or the mechanism of action for corticosteroids needs further investigation, particularly in clinical trials comparing other immune modulators to dexamethasone for efficacy and safety.

Given the pleiotropic effects of corticosteroids, the ability to narrow down specific signaling pathways could provide an enhanced understanding of the inflammatory mechanisms of COVID-19. Previous and ongoing work by our group demonstrates significant increases in the soluble mediators APRIL, Osteopontin, Pentraxin-3, and sTNFR-1 at day 3 in patients deceased at day 28²². Further analysis, including the impact of vaccine-derived immunity, is ongoing and may further explain the immune pathways important in determining outcomes or guiding additional therapies in severe COVID-19.

This data source and model have limitations. The use of receipt of dexamethasone as a surrogate measure of moderate to severe COVID-19 requiring new or increased supplemental oxygen may include patients who received this drug but did not require supplemental oxygen and does exclude patients who had moderate to severe disease but did not receive dexamethasone. Additionally, the dataset does not have high resolution for the exact level of oxygen support required at the time of enrollment in this study. Furthermore, the outcomes chosen for clinical prediction models for COVID-19 are not entirely consistent, from in-hospital mortality to time specific mortality ranging from 14 to 60 days. Many clinical prediction models include worst vital signs within a specific time window. We were unable to evaluate the additive validity of these measures in our model due to the limited availability and heterogeneity of vital sign data in the data source used in this study. The additive predictive value of vital signs should be evaluated in the future. The derivation and validation cohorts comprise patients treated at tertiary care centers, which may limit generalizability to other inpatient settings. This score is limited to patients admitted with COVID-19 and should not be applied to patients with other viral or non-viral respiratory infections. The US population may not reflect international care of COVID-19 patients. The number of predictive variables was limited such that the resulting model was simple and could be externally validated. Additional variables including pre-existing comorbid medical conditions, vaccination status, and co-morbid bacterial superinfection may have improved the model’s discrimination and warrant future investigation. Despite this, the validation of this model leverages one of the most extensive datasets available for COVID-19 research. Furthermore, this study uses robust statistical methods for derivation and validation, providing a simple model accounting for the standard of care for severe COVID-19.

Conclusions

The ARC score represents a simple method to predict 28-day mortality in patients hospitalized with COVID-19 receiving dexamethasone. The availability of clinical information required to calculate the ARC score facilitates implementation in various resource settings. This work identifies factors that warrant further research to understand how immune dysregulation impacts mortality and response to dexamethasone therapy for severe COVID-19.

Acknowledgements

The analyses described in this publication were conducted with data or tools accessed through the NCATS N3C Data Enclave covid.cd2h.org/enclave and supported by CD2H - The National COVID Cohort Collaborative (N3C) IDeA CTR Collaboration 3U24TR002306-04S2 NCATS U24 TR002306. This research was possible because of the patients whose information is included within the data from participating organizations (covid.cd2h.org/dtas) and the organizations and scientists (covid.cd2h.org/duas) who have contributed to the on-going development of this community resource (cite this https://doi.org/10.1093/jamia/ocaa196).

Author contributions

M.B.D., J.R.M., W.A.F., S.M.W. and G.S.D. designed the initial concept. B.J.S. and M.B.D. performed the statistical analysis. K.K.J., C.L., L.A. C. R., and B.J.S. collected the data. B.J.S., J.R.M., and M.B.D. wrote the manuscript. All authors provided critical revisions and approved the final manuscript.

Funding

B.J.S was supported by 5T32HL007106-45 during this study. No funder or funding agency had any specific role in the conceptualization, design, data collection, analysis, decision to publish, or preparation of the manuscript.

Data availability

To access patient-level data from the N3C consortium, institutions must have a signed Data Use Agreement executed with NCATS and principal investigators must complete mandatory training along with submitting a Data Use Request (DUR) to N3C. All code used for analysis can be found on GitHub. To request N3C data access follow instructions at http://covid.cd2h.org/onboarding.

Declarations

Ethics approval and consent to participate

The study was performed in accordance with the principles of the Declaration of Helsinki, institutional review board approval was obtained from the University of North Carolina Office of Human Research Ethics before the start of the study (IRB study number 22–0220, “External cohort validation of the ARC score for COVID-19 28-day mortality and escalation in O2 therapy in hospitalized patients,” date of approval: March 3, 2022).

Competing interests

The authors declare no competing interests.

Abbreviations

N3C

National COVID Cohort Collaborative

CRP

C–reactive protein

ARC score

Age, Neutrophil to Lymphocyte Ratio, C–reactive protein

ROC

Receiver–operater characteristic curve

sTNFR

1–soluble tumor necrosis factor receptor 1

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

1. United States of America: WHO Coronavirus Disease (COVID-19). Dashboard https://covid19.who.int/region/amro/country/us. Accessed 24 August 2022.

2. Garg, S et al. Clinical trends among U.S. Adults hospitalized with COVID-19, March to December 2020: A Cross-Sectional study. Ann. Intern. Med.; 2021; 174, 10 pp. 1409-1419.

3. Broman, N et al. IL-6 and other biomarkers as predictors of severity in COVID-19. Ann. Med.; 2021; 53, 1 pp. 410-412.

4. Mueller, A. A. et al. Inflammatory biomarker trends predict respiratory decline in COVID-19 patients. Cell Rep. Med.1(17). (2020).

5. Tan, Y. et al. Integrating longitudinal clinical laboratory tests with targeted proteomic and transcriptomic analyses reveal the landscape of host responses in COVID-19. Cell Discov.7(42). (2021).

6. Cui, Z et al. Early and significant reductions in C-reactive protein levels after corticosteroid therapy is associated with reduced mortality in patients with COVID-19. J. Hosp. Med.; 2021; 16, 3 pp. 142-148.

7. Valle, DMD et al. An inflammatory cytokine signature predicts COVID-19 severity and survival. Nat. Med.; 2020; 26, pp. 1636-1643.

8. Chen, H et al. Corticosteroid therapy is associated with improved outcome in critically ill patients with COVID-19 with hyper inflammatory phenotype. Chest Infections; 2021; 159, 5 pp. 1793-1802.

9. RECOVERY Collaborative Groupet al. Dexamethasone in hospitalized patients with Covid-19. N Engl. J. Med.; 2021; 384, 8 pp. 693-704.

10. Bradley, P. et al. Utility of established prognostic scores in COVID-19 hospital admissions: Multicentre prospective evaluation of CURB-65, NEWS2 and qSOFA. BMJ Open. Resp. Res 7e000729. (2020).

11. Jones, A et al. External validation of the 4 C mortality score among COVID-19 patients admitted to hospital in ontario, canada: a retrospective study. Sci. Rep.; 2021; 11, 18638.2021NatSR.1118638J

12. Garibaldi, BT et al. Patient trajectories among persons hospitalized for COVID-19: A cohort study. Ann. Intern. Med.; 2020; 174, 1 pp. 33-41.

13. Lombardi, Y et al. External validation of prognostic scores for COVID-19: a multicenter cohort study of patients hospitalized in greater Paris university hospitals. Intensive Care Med.; 2021; 47, pp. 1426-1439.

14. Lu, C. et al. The age-neutrophil to lymphocyte Ratio-C reactive protein (ARC) score: Predicting mortality in patients with COVID-19 infection receiving dexamethasone therapy. Presented at: ATS 2022; San Francisco, CA.

15. Haendel, MA et al. The National COVID cohort collaborative (N3C): rationale, design, infrastructure, and deployment. J. Am. Med. Inform. Assoc.; 2021; 28, 3 pp. 427-443.

16. Centers for Disease Control and Prevention. COVID Data Tracker. Atlanta, GA: US Department of Health and Human Services, CDC; (2022). https://covid.cdc.gov/covid-data-tracker. Accessed December 12.

17. Ingraham, N. E., Jones, E. K., King, S. & Dries, J., et al. Re-aiming equity evaluation in clinical decision support: A scoping review of equity assessments in surgical decision support systems. AnnSurg. 277, 359–364. (2023).

18. Lupei, M. I. et al., et al. A 12-hospital prospective evaluation of a clinical decision support prognostic algorithm based on logistic regression as a form of machine learning to facilitate decision making for patients with suspected COVID-19. PLoS ONE. 17(1). (2022).

19. Tavakolpour, S; Rakhshandehroo, T; Wei, EX; Rashidian, M. Lymphopenia during the COVID-19 infection: what it shows and what can be learned. Immunol. Lett.; 2020; 225, pp. 31-32.

20. Fauci, AS. Mechanisms of corticosteroid action on lymphocyte subpopulations. II. Differential effects of in vivo hydrocortisone, prednisone and dexamethasone on in vitro expression of lymphocyte function. Clin. Exp. Immunol.; 1976; 24, 1 pp. 54-62.

21. Wolfe, C. R. et al. Baricitinib versus dexamethasone for adults hospitalised with COVID-19 (ACTT-4): A randomised, double-blind, double placebo-controlled trial. Lancet Respir. Med. (2022).

22. Appleton, L. et al. The Kinetics of Blood Soluble Mediators in Patients Requiring Hospitalization for COVID-19 Infection before and during Dexamethasone Therapy (ATS, 2022).