Predicting in-hospital mortality from Coronavirus Disease 2019: A simple validated app for clinical use

About the Authors:

Bianca Magro

Contributed equally to this work with: Bianca Magro, Valentina Zuccaro

Roles Conceptualization, Data curation, Investigation, Writing – original draft

Affiliation: Gastroenterology Hepatology and Transplantation, ASST Papa Giovanni XXIII–Bergamo, Bergamo, Italy

Valentina Zuccaro

Contributed equally to this work with: Bianca Magro, Valentina Zuccaro

Roles Conceptualization, Data curation, Investigation, Writing – original draft

Affiliation: Department of Infectious Diseases, IRCCS Fondazione Policlinico San Matteo, Pavia, Italy

Luca Novelli

Roles Data curation, Investigation

Affiliation: Pneumology Unit, ASST Papa Giovanni XXIII–Bergamo, Bergamo, Italy

ORCID logo https://orcid.org/0000-0002-2705-248X

Lorenzo Zileri

Roles Conceptualization, Data curation, Investigation, Validation, Writing – original draft, Writing – review & editing

Affiliation: UOC di Medicina Interna e Gastroenterologia, Dipartimento di Scienze Gastroenterologiche, Endocrino-Metaboliche e Nefro-Urologiche, Fondazione Policlinico Universitario A. Gemelli IRCCS, Rome, Italy

Ciro Celsa

Roles Conceptualization, Formal analysis, Investigation, Methodology, Writing – original draft, Writing – review & editing

Affiliations Section of Gastroenterology and Hepatology, Department of Health Promotion, Mother and Child Care, Internal Medicine and Medical Specialties, University of Palermo, Palermo, Italy, Department of Surgical, Oncological and Oral Sciences (Di.Chir.On.S.), University of Palermo, Palermo, Italy

ORCID logo https://orcid.org/0000-0002-5662-2162

Federico Raimondi

Roles Data curation, Investigation

Affiliation: Pneumology Unit, ASST Papa Giovanni XXIII–Bergamo, Bergamo, Italy

Mauro Gori

Roles Investigation

Affiliation: Cardiovascular Department and Cardiology 1 Unit, ASST Papa Giovanni XXIII–Bergamo, Bergamo, Italy

Giulia Cammà

Roles Conceptualization, Data curation, Investigation, Visualization, Writing – original draft, Writing – review & editing

Affiliation: UOC di Medicina Interna e Gastroenterologia, Dipartimento di Scienze Gastroenterologiche, Endocrino-Metaboliche e Nefro-Urologiche, Fondazione Policlinico Universitario A. Gemelli IRCCS, Rome, Italy

Salvatore Battaglia

Roles Conceptualization, Formal analysis, Methodology, Software, Writing – original draft

Affiliation: Department of Economics, Business and Statistics (SEAS), University of Palermo, Palermo, Italy

Vincenzo Giuseppe Genova

Roles Formal analysis, Software, Writing – original draft

Affiliation: Department of Economics, Business and Statistics (SEAS), University of Palermo, Palermo, Italy

Laura Paris

Roles Data curation, Investigation

Affiliation: Hematology and Bone Marrow Transplant Unit, ASST Papa Giovanni XXIII–Bergamo, Bergamo, Italy

Matteo Tacelli

Roles Data curation, Investigation, Writing – original draft

Affiliation: Gastroenterology Hepatology and Transplantation, ASST Papa Giovanni XXIII–Bergamo, Bergamo, Italy

Francesco Antonio Mancarella

Roles Data curation, Investigation

Affiliation: UOC di Medicina Interna e Gastroenterologia, Dipartimento di Scienze Gastroenterologiche, Endocrino-Metaboliche e Nefro-Urologiche, Fondazione Policlinico Universitario A. Gemelli IRCCS, Rome, Italy

Marco Enea

Roles Formal analysis, Methodology, Software, Writing – review & editing

Affiliation: Department of Health Promotion, Mother and Child Care, Internal Medicine and Medical Specialties, University of Palermo, Palermo, Italy

Massimo Attanasio

Roles Formal analysis, Methodology, Software, Validation

Affiliation: Department of Economics, Business and Statistics (SEAS), University of Palermo, Palermo, Italy

Michele Senni

Roles Data curation, Investigation

Affiliation: Cardiovascular Department and Cardiology 1 Unit, ASST Papa Giovanni XXIII–Bergamo, Bergamo, Italy

Fabiano Di Marco

Roles Data curation, Investigation

Affiliation: Pneumology Unit, ASST Papa Giovanni XXIII–Bergamo, Bergamo, Italy

Luca Ferdinando Lorini

Roles Data curation, Investigation

Affiliation: Emergency and Intensive care Department, ASST Papa Giovanni XXIII–Bergamo, Bergamo, Italy

Stefano Fagiuoli

Roles Conceptualization, Project administration, Supervision, Validation, Writing – review & editing

Affiliation: Gastroenterology Hepatology and Transplantation, ASST Papa Giovanni XXIII–Bergamo, Bergamo, Italy

Raffaele Bruno

Roles Conceptualization, Data curation, Project administration, Supervision, Validation, Writing – review & editing

Affiliations Department of Infectious Diseases, IRCCS Fondazione Policlinico San Matteo, Pavia, Italy, Department of Clinical, Surgical, Diagnostic and Pediatric Sciences, University of Pavia, Pavia, Italy

Calogero Cammà

Roles Conceptualization, Data curation, Funding acquisition, Methodology, Supervision, Validation, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliation: Section of Gastroenterology and Hepatology, Department of Health Promotion, Mother and Child Care, Internal Medicine and Medical Specialties, University of Palermo, Palermo, Italy

Antonio Gasbarrini

Roles Conceptualization, Supervision, Validation, Writing – review & editing

Affiliation: UOC di Medicina Interna e Gastroenterologia, Dipartimento di Scienze Gastroenterologiche, Endocrino-Metaboliche e Nefro-Urologiche, Fondazione Policlinico Universitario A. Gemelli IRCCS, Rome, Italy

Introduction

Severe acute respiratory syndrome coronavirus 12 (SARS-CoV-2) was first identified in China in December 2019 and has since spread rapidly all over the world [1]. Outside China, Italy was the first western country to be involved and the first case was diagnosed on February, 21. During the initial weeks of the pandemic, the rapid increase in cases overwhelmed the capacity of the National Health System to receive and manage patients and to respond in terms of availability of health resources [2]. In the context of triaging patients in emergency departments or in special clinics set up during an acute outbreak, the lack of clinical criteria to identify the most severe cases and to define the evolution of the disease has made the management of the pandemic even more difficult.

A risk stratification of COVID-19 patients is crucial in order to improve the healthcare organization and to best manage a new potential second wave of the epidemic in the coming winter. In this complex epidemiological and clinical scenario, a competing risks model is a robust statistical method to predict patients risk profile when more than one competing event, such as in-hospital mortality and discharge, is present [3]. The aims of this retrospective multicenter study are 1) to derive a simple clinical prediction rule capable of promptly identifying risk factors for in-hospital mortality and discharge in hospitalized patients with COVID-19 by competing risks analysis; 2) to validate this prediction rule in an external validation cohort; 3) to design a free web-app for calculating the risk of in-hospital mortality (COVID-CALC).

Materials and methods

Sources of data and definition of variables

We analyzed an integrated database that contained clinical, laboratory and treatment data from all hospitalized patients with a diagnosis of COVID-19 at three Italian referral tertiary centers, two in Lombardy (the “eye of the SARS-COV-2 storm” in Italy) (Bergamo and Pavia) and one in Lazio (Rome): 1) Hospital Papa Giovanni XXIII, Bergamo, Lombardy; 2) Fondazione IRCCS Policlinico San Matteo, Pavia, Lombardy; 3) Fondazione Policlinico Univerisitario A. Gemelli IRCCS, Rome, Lazio. All consecutive patients admitted between February 22nd and April 7th, 2020 were enrolled and were followed up until April 30th, 2020.

Information on the history and physical examination of hospitalized patients with COVID-19 were abstracted from chart reviews by medical officers at each hospital. Variables collected through standardized recording forms included age, sex, comorbidities, smoking status, time of symptoms onset and time of hospital admission. Additional variables were the presence of fever (defined as axillary temperature of at least 37.5°C), dyspnoea, cough and diarrhea. Investigations consisted of chest radiography or computed tomography and hematologic and biochemical blood tests, including complete blood count, coagulation profile, glutamic pyruvic transaminase (GPT), lactate dehydrogenase (LDH), C-reactive protein (CRP), and creatine kinase. Arterial-blood gas analysis (ABG) was performed when clinical signs of oxygen impairment were detected (e.g. tachypnoea and hypoxia). P/F ratio was calculated as the ratio between PaO2 and FiO2.

Laboratory confirmation of the SARS COV-2 infection was defined as positive real-time reverse transcriptase polymerase chain reaction (RT-PCR) from nasal and pharyngeal swab; samples were prospectively collected and analyzed at the Molecular Virology Units of each center according to the WHO guidelines and Corman et al. protocols [4, 5]. More details are provided in Supplementary Materials.

Treatments included the use of antiviral therapy (Lopinavir/ritonavir or darunavir/ritonavir), hydroxychloroquine, enoxaparin, and immunomodulatory/immunosuppressive therapy such as corticosteroids, Tocilizumab and Sarilumab. Lopinavir/ritonavir 400/100 mg was administered orally twice daily for 14 days, while Darunavir/ritonavir 800/100 mg was administered once daily for 14 days. Hydroxychloroquine 600 mg was administered twice on day 1, followed by a dose of 400 mg daily for 7 days. Enoxaparin 1 mg/kg was given once or twice daily. Corticosteroid treatment consisted in dexamethasone 20 mg daily for 5 days or methylprednisolone 1 mg/kg intravenously daily for 5 days. Tocilizumab 8 mg/kg was administered intravenously in 1 or 2 doses. A second dose was administered after 8–12 hours from the first dose in patients with inadequate response. Sarilumab 400 mg was administered intravenously once. Oxygen support consisted of low (cannula and simple masks) and high flow (Venturi and reservoir masks, Nasal High Flow), helmet continuous positive airway pressure (CPAP) or non-invasive ventilation (NIV). The choice for the oxygen support was determined by rapid deterioration of P/F ratio and upgraded if a further worsening after two hours of treatment was detected.

The Institutional Review Boards of the three centres (ASST Papa Giovanni XXIII–Bergamo, Italy; IRCCS Fondazione Policlinico San Matteo, Pavia, Italy; Fondazione Policlinico Universitario A. Gemelli IRCCS, Rome, Italy) which comply with the Declaration of Helsinki and its revisions, approved this study. All accessed data were fully anonymized.

Outcomes

The primary outcome was in-hospital mortality. Discharge was analyzed as a competing event in the competing risks survival analysis. The competing risks model proposed by Fine and Gray was applied [6].

The criteria for discharge were absence of fever, resolution of respiratory symptoms, oxygen saturation higher than 94% and two consecutive nasal swab negative for SARS-CoV-2 obtained at least 24 hours apart.

Statistical analysis

Data for continuous variables are expressed as mean and standard deviation or median and interquartile ranges (IQR), and data for categorical variables are expressed as frequency and percentage. Differences between continuous data were assessed by Student t test or by Mann-Whitney U test. Differences between categorical variables were assessed by χ2 test.

In-hospital mortality and discharge were evaluated by competing risks survival analysis, represented by cumulative incidence function (CIF) [3]. The Fine and Gray proportional sub-distribution hazard model was fitted in order to estimate the effect of covariates on CIFs in-hospital death and discharge [6]. Covariates used for multivariate analyses were chosen based on their significance in the univariate analysis (p<0.10). Covariates in the final model with a p-value <0.05 were considered statistically significant. The results are presented as adjusted hazard ratios (HR) and their 95% confidence intervals (CI). Competing risks analyses were performed in SAS version 9.4. Hotdeck missing imputation data and the assessment of discrimination and calibration and were performed in R Core Team (2019). We used hot.deck function from hot.deck library in R 3.8.0.

Derivation and validation process

Risk factors for in-hospital mortality and discharge identified by competing risks multivariate analysis in the derivation set were used to generate a prediction rule. The probability of dying or of being discharged within 40 days after hospital admission was computed for a hypothetical patient identified by a combination of prognostic factors. The prediction accuracy of the fitted models was assessed by discrimination and calibration both in the derivation (internal validation) and validation cohorts (external validation) [7]. Discrimination of the models was assessed by the area under the receiver operating characteristic curve (AUC or C-index) [8]. Calibration was evaluated by comparing the predicted probability with the observed probability at a certain time point by a calibration plot. Finally, the Brier score, which takes into account both the discrimination and the calibration at the same time, was also calculated. It is defined as the expected squared distance between the observed status at that time and the predicted probability [9]. Thus, a smaller value of the Brier score indicates a better model.

To assess the internal validity of the prediction rule, the derivation set was randomly split into a training set (70%) and a test set (30%) [10]. The external validation of the prediction rule was carried out with data from an external validation cohort, represented by the Rome unit, in terms of discrimination, calibration and the Brier score.

The prediction rule has been translated into a web-app that is freely available to the public (COVID-CALC: https://sites.google.com/community.unipa.it/covid-19riskpredictions/c19-rp). (S1 Fig).

Results

From February 22nd to April 7th, 2020, a total of 2191 consecutive confirmed cases of COVID-19 were observed. Baseline characteristics of patients stratified according to derivation (n = 1810) and validation cohort (n = 381) are shown in Table 1. Median age was 67 years (IQR, 56–77 years) and 45% of patients were 70 years or older. Sixty-nine percent of patients were male. In 27.5% of patients, at least one comorbidity was present, with hypertension and diabetes being the most common (43% and 17% of patients, respectively). Median time from symptoms onset to hospital admission was 8 days (IQR 5–11 days). At hospital admission, fever was present in 85%, dyspnoea in 56% and cough in 44% of patients. Lymphocyte count lower than 1000/mmc was observed in 77% of the patients, and platelet count was lower than 150000/mmc in 37.5% of patients. CRP was increased in 83% of patients, and LDH resulted elevated in 88% of patients. In the derivation cohort, male sex, hypertension and obesity were significantly more frequent and the prevalence of chronic kidney disease, chronic obstructive lung disease and malignancies was significantly lower in comparison with the validation set. Patients in the derivation set had higher median GPT, CRP, LDH, and D-dimer levels, higher lymphocyte count and lower P/F ratio, in comparison with patients in the validation set.

[Figure omitted. See PDF.]

Table 1. Demographic, clinical and laboratory characteristics of patients with Coronavirus Disease-19 on hospital admission in the derivation and the validations cohorts.

https://doi.org/10.1371/journal.pone.0245281.t001

Data on treatments and outcomes according to derivation and validation cohorts are reported in Table 2. Corticosteroid treatment was administered to 129 patients (11%. Data available in 1164 patients). Enoxaparin was given to 254 patients (21%. Data available in 1218 patients). Hydroxychloroquine was administered to 931 patients (80%. Data available in 1163 patients). Seven-hundred seventy-nine patients (49%) received antiviral treatment with Lopinavir/ritonavir and 242 with Darunavir/ritonavir (15%) (Data available in 1593 patients). Tocilizumab was administered in 112 patients (9%) and Sarilumab in 51 patients (13%) (Data available in 1233 patients). One-hundred sixty-four patients (9%) received non-invasive ventilation (Data available in 1765 patients).

[Figure omitted. See PDF.]

Table 2. Treatments and outcomes of patients with Coronavirus Disease-19 in the derivation and the validation cohorts.

https://doi.org/10.1371/journal.pone.0245281.t002

Clinical outcomes

At the end of follow-up, 540 patients had died (24.6%), 302 (13.7%) had been transferred to ICU, 1358 patients (62.0%) had been discharged and 258 were still hospitalized. Median time from symptoms onset to death and from hospital admission to death were 13 days (IQR 9–19) and 5 days (IQR 3–10), respectively. Median time from hospital admission to ICU admission was 3.5 days (IQR 1–6). Median time from hospital admission to discharge was 10 days (IQR 6–16).

The CIFs for in-hospital mortality and discharge in the derivation and validation cohorts are shown in Fig 1. In-hospital mortality at 7 and 21 days was 16% and 26% in the derivation cohort and 5% and 10% in the validation cohort, respectively. Discharge rates at 7 and 21 days were 22% and 52% in the derivation cohort and 6% and 62% in the validation cohort, respectively.

[Figure omitted. See PDF.]

Fig 1.

Cumulative incidence functions (CIFs) for in-hospital mortality and discharge of patients with Coronavirus Disease-19 in the derivation (1A) and validation cohorts (1B).

https://doi.org/10.1371/journal.pone.0245281.g001

Risk factors for in-hospital mortality

Univariate analysis for in-hospital mortality in the derivation cohort is reported in S1 Table in S1 File. In the multivariate model, seven variables were independently associated with in-hospital mortality: age (HR 1.08, 95% CI 1.07–1.09, p<0.0001), male sex (HR 1.62, 95% CI 1.30–2.00, p<0.0001), duration of symptoms before hospital admission shorter than 10 days (HR 1.72, 95% CI 1.39–2.12, p<0.0001), type 2 diabetes (HR 1.21, 95% CI 1.02–1.45, p = 0.044), coronary heart disease (HR 1.40, 95% CI 1.09–1.80, p = 0.009), chronic liver disease (HR 1.78, 95% CI 1.16–2.72, p = 0.008), and LDH levels (HR 1.0003, 95% CI 1.0002–1.0005, p<0.0001) (Table 3). Similar results were obtained for discharge model (see S1 Table in S1 File and Table 3). When covariates with p-value <0.20 at univariate analysis were included in the multivariate model, similar results were obtained (S2 Table in S1 File).

[Figure omitted. See PDF.]

Table 3. Risk factors for in-hospital mortality and discharge of patients with Coronavirus Disease-19 in the derivation cohort.

https://doi.org/10.1371/journal.pone.0245281.t003

These risk factors were used to construct a model encompassing patients grouped into “best”, “intermediate” and “worst” profiles. CIFs for the best (60 years old, female, duration of symptoms before hospital admission longer than 10 days, no comorbidities, and LDH levels of 250 U/L), the intermediate (70 years old, male, duration of symptoms before hospital admission shorter than 10 days, chronic liver disease, LDH levels of 300 U/L) and the worst profiles (80 years old, male, duration of symptoms before hospital admission shorter than 10 days, coronary heart disease, chronic liver disease, diabetes, LDH levels of 400 U/L) are shown in Fig 2. In the best profile, 7- and 21-day in-hospital mortality was 5% and 8%, respectively; in the intermediate profile, 7- and 21-day in-hospital mortality was 18% and 28%, respectively; in the worst profile, 7- and 21-day in-hospital mortality was 52% and 70%, respectively.

[Figure omitted. See PDF.]

Fig 2. Cumulative Incidence Functions (CIFs) for in-hospital mortality of patients with Coronavirus Disease-19 according to three different patient profiles.

A: best profile (60 years old, female sex, duration of symptoms before hospital admission longer than 10 days, no comorbidities, and LDH levels of 250 U/L). B: intermediate profile (70 years old, male sex, duration of symptoms before hospital admission shorter than 10 days, chronic liver disease, LDH levels of 300 U/L.) C: worst profile (80 years old, male sex, duration of symptoms before hospital admission shorter than 10 days, coronary heart disease, chronic liver disease, diabetes, LDH levels of 400 U/L).

https://doi.org/10.1371/journal.pone.0245281.g002

Validation of the prediction rule

By internal validation, the AUC based on the data from the derivation cohort was good (AUC = 0.822, 95% CI 0.722–0.922). The accuracy in the validation cohort was similar to that of the derivation cohort (AUC = 0.820, 95% CI 0.724–0.920). S2 Fig shows the calibration plot of the model for in-hospital mortality. The Brier score was 14.3 in the derivation cohort and 16.9 in the validation cohort. Similar results were obtained for discharge model (see S3 Fig)

The prediction for in-hospital mortality has been translated into a web-based app (COVID-CALC) to obtain both the CIF for in-hospital mortality (predicted curve) and confidence intervals for the CIF at 7, 14 and 21 days (https://sites.google.com/community.unipa.it/covid-19riskpredictions/c19-rp).

Discussion

In this study, we developed and validated a simple clinical prediction rule able to predict in-hospital mortality of hospitalized patients with COVID-19, considering discharge as a competing risk. In our analysis, seven variables (older age, male sex, shorter duration of symptoms before hospital admission, diabetes, coronary heart disease, chronic liver disease, and LDH levels) were independent risk factors for in-hospital death, as shown by a competing risks multivariate analysis. External validation of this prediction rule showed good discrimination and calibration. To support clinicians in the risk stratification, a web-based app was developed.

From a practical point of view, our prediction rule could help physicians to improve the allocation of medical resources, potentially reducing the overcrowding that we have witnessed in healthcare systems which significantly impacted mortality worldwide during the COVID-19 pandemic. Several prediction models have been previously published aiming to stratify the risk of in-hospital mortality in patients with COVID-19, in both Western and Eastern countries [11–16]. Particularly, the 4C Mortality score [11], including age, sex, number of comorbidities, respiratory rate, oxygen saturation, level of consciousness, urea and CRP levels, was developed in a cohort of more than 35,000 European patients, showing a good discrimination for mortality (AUC = 0.79). Moreover, a 10-item risk score predicting the occurrence of critical illness, defined as a composite of ICU admission, invasive ventilation, or death, was recently validated in a Chinese cohort, showing an AUC of 0.88 [12]. However, our methodological approach was quite different to those used in the above quoted studies. It should be noted that the use of a composite endpoint considers ICU and death to be equal, which may not be true. Moreover, the traditional logistic regression model neglects to model discharge as a competing endpoint. Our competing risks analysis may provide further insights into the effect of clinical covariates on the separate endpoint components [17, 18].

Results of our analyses confirmed those of previous reports from China and the USA [19–21], showing older age as the most important risk factor for in-hospital death in COVID-19. However, we found higher in-hospital mortality in comparison to other studies [19, 20]. The demographic structure of the Italian population could be a reason for this finding. In 2019, Italy resulted as being the European country with the highest proportion of elderly people, with about a quarter of the population aged older than 65 years [22]. Not surprisingly, the median age in our cohort was 67 years, that is higher if compared with that observed in other studies.

Interestingly, in our analysis comorbidities were associated with in-hospital death independently from age and other covariates. In our study, the prevalence of comorbidities was similar to that reported in other Western countries [21], but it was higher when compared to Chinese patients [19, 20], with cardiovascular comorbidities, including coronary heart disease, resulting as the most common. Regarding chronic liver disease, our findings are also in line with the results of two international reporting registries of 152 patients (103 of them with cirrhosis), showing a mortality of about 40% [23].

A shorter duration of the symptoms before hospital admission was independently associated with higher in-hospital mortality. This is a novel finding, and it could be argued that patients with the most severe disease were hospitalized shortly after symptoms onset, while those who were hospitalized after a longer duration of symptoms were those with milder disease.

LDH levels resulted as being independently associated with a higher risk of in-hospital death. LDH is released from cells upon damage of cytoplasmic membrane and its levels might reflect tissue necrosis related to immune hyperactivity, which thus relates to poor outcome [24]. The prognostic role of LDH has also been reported in other Chinese reports [25, 26] and in studies conducted on other coronaviruses [27].

Regarding treatments, it should be underlined that the aim of our analysis was not to assess their efficacy on clinical outcomes of hospitalized patients with COVID-19. Therefore, we described solely the employed treatments and their differences among centers, as the design of our analysis does not allow us to draw firm conclusions regarding the efficacy and safety of the treatments available for COVID-19.

Our study suffers from several limitations. 1) Retrospective studies have many problems that reduce their internal and external validity and selection bias can lead to incorrect results and spurious associations. However, we believe that selection bias could not be relevant as only consecutive patients with COVID-19 were included. 2) A limitation of any prediction rule is the generalizability of results to different populations and settings. However, we performed an external validation that showed good calibration and discrimination. 3) Our derivation and validation cohorts showed significant baseline clinical differences, probably because data were collected in two different settings (Northern vs Central Italy) with different degrees of overcrowding for healthcare systems. However, it should be underlined that hospitalization criteria were similar among participating centres. 4) Patients in our cohort were collected during the early phase of the spread of the infection locally, therefore it may not fit during different epidemic periods. Whether this prediction rule will also apply to patients observed at a later phase of the pandemic remains to be tested. 5) Mortality was limited to in-hospital death, and we assumed discharged patients to still be alive during the study period. 6) The sample size of our validation cohort was relatively small, probably reflecting the differences in disease burden between Northern and Central Italy. 7) The high number of missing data on treatments, particularly regarding corticosteroids use, hampered their inclusion in the prediction rule. However, it should be underlined that the effects of most drug interventions are currently highly uncertain, particularly for the timing of steroids use and the optimal dosage of hydroxychloroquine [28], and no definitive evidence exists that therapies could result in important benefits and harms for any outcomes, as recently reported in a network meta-analysis [29]. 8) LDH was included in our prediction rule, although LDH levels may be not always available. In order to accommodate for possible LDH missingness in the app, we implemented two different prediction rules: the first one based on the final model including LDH values when available, the second one based on a model estimated including all the risk factors of the final model but LDH. Moreover, in the latter case the app will warn the prediction rule is not accurate as the first one.

In conclusion, we developed and validated a simple prediction rule capable of accurately predicting the risk for in-hospital mortality and discharge of patients with COVID-19. Our prediction rule could improve the triage and management of patients with COVID-19 in different epidemiological and healthcare organization settings.

Supporting information

[Figure omitted. See PDF.]

S1 Checklist.

https://doi.org/10.1371/journal.pone.0245281.s001

(DOCX)

S1 Fig. The online web-based calculator for predicting in-hospital mortality among patients with Coronavirus Disease-19.

https://doi.org/10.1371/journal.pone.0245281.s002

(TIF)

S2 Fig. Calibration curves for predicting in-hospital mortality of patients with Coronavirus Disease-19 in the derivation and validation cohorts.

https://doi.org/10.1371/journal.pone.0245281.s003

(TIF)

S3 Fig. Calibration curves for predicting discharge of patients with Coronavirus Disease-19 in the derivation and validation cohorts.

https://doi.org/10.1371/journal.pone.0245281.s004

(TIF)

S1 File.

https://doi.org/10.1371/journal.pone.0245281.s005

(DOCX)

S1 Dataset.

https://doi.org/10.1371/journal.pone.0245281.s006

(XLSX)

Citation: Magro B, Zuccaro V, Novelli L, Zileri L, Celsa C, Raimondi F, et al. (2021) Predicting in-hospital mortality from Coronavirus Disease 2019: A simple validated app for clinical use. PLoS ONE 16(1): e0245281. https://doi.org/10.1371/journal.pone.0245281

References

1. https://www.who.int/dg/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19—11-march-2020

2. Remuzzi A, Remuzzi G. COVID-19 and Italy: what next?. Lancet. 2020;395(10231):1225‐1228. pmid:32178769

3. Putter H, Fiocco M, Geskus RB. Tutorial in biostatistics: competing risks and multi-state models. Stat Med. 2007;26(11):2389‐2430. pmid:17031868

4. Coronavirus disease (COVID-19) technical guidance: Laboratory testing for 2019-nCoV in humans. https://www.who.int/emergencies/diseases/novel-coronavirus-2019/technical-guidance/laboratory-guidance

5. Corman VM, Landt O, Kaiser M, et al. Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR. Euro Surveill. 2020;25(3):2000045.

6. Fine JP et al. A proportional hazards model for the sub-distribution of a competing risk. J Am Stat Assoc 1999; 94:496–509.

7. Zhang Z, Cortese G, Combescure C, et al. Overview of model validation for survival regression model with competing risks using melanoma study data. Ann Transl Med. 2018;6(16):325. pmid:30364028

8. Schoop R, Beyersmann J, Schumacher M, et al. Quantifying the predictive accuracy of time-to-event models in the presence of competing risks. Biom J 2011;53:88–112. pmid:21259311

9. Gerds TA, Andersen PK, Kattan MW. Calibration plots for risk prediction models in the presence of competing risks. Stat Med 2014;33:3191–203. pmid:24668611

10. Dobbin KK, Simon RM. Optimally splitting cases for training and testing high dimensional classifiers. BMC Med Genomics. 2011;4:31. pmid:21477282

11. Knight SR, Ho A, Pius R, Buchan I, et al. Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score. BMJ. 2020 Sep 9;370:m3339. Erratum in: BMJ. 2020 Nov 13;371:m4334. pmid:32907855

12. Liang W, Liang H, Ou L, et al. Development and Validation of a Clinical Risk Score to Predict the Occurrence of Critical Illness in Hospitalized Patients With COVID-19 [published online ahead of print, 2020 May 12]. JAMA Intern Med. 2020;e202033. pmid:32396163

13. Yadaw AS, Li YC, Bose S, Iyengar R, Bunyavanich S, Pandey G. Clinical features of COVID-19 mortality: development and validation of a clinical prediction model. Lancet Digit Health. 2020 Oct;2(10):e516–e525. pmid:32984797

14. Fumagalli C, Rozzini R, Vannini M, et al. Clinical risk score to predict in-hospital mortality in COVID-19 patients: a retrospective cohort study. BMJ Open. 2020 Sep 25;10(9):e040729. pmid:32978207

15. Wynants L, Van Calster B, Collins GS, et al. Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal. BMJ. 2020 Apr 7;369:m1328. Erratum in: BMJ. 2020 Jun 3;369:m2204. pmid:32265220

16. Mesas AE, Cavero-Redondo I, Álvarez-Bueno C, et al. Predictors of in-hospital COVID-19 mortality: A comprehensive systematic review and meta-analysis exploring differences by age, sex and health conditions. PLoS One. 2020 Nov 3;15(11):e0241742. pmid:33141836

17. Wolbers M, Koller MT, Stel VS, et al. Competing risks analyses: objectives and approaches. Eur Heart J. 2014;35(42):2936‐2941. pmid:24711436

18. Putter H, Schumacher M, van Houwelingen HC. On the relation between the cause-specific hazard and the subdistribution rate for competing risks data: The Fine-Gray model revisited. Biom J. 2020;62(3):790‐807. pmid:32128860

19. Zhou F, Yu T, Du R, et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet. 2020 Mar 28;395(10229):1054–1062. pmid:32171076

20. Wang D, Hu B, Hu C, et al. Clinical Characteristics of 138 Hospitalized Patients With 2019 Novel Coronavirus-Infected Pneumonia in Wuhan, China [published online ahead of print, 2020 Feb 7]. JAMA. 2020;323(11):1061‐1069. pmid:32031570

21. Richardson S, Hirsch JS, Narasimhan M, et al. Presenting Characteristics, Comorbidities, and Outcomes Among 5700 Patients Hospitalized With COVID-19 in the New York City Area [published online ahead of print, 2020 Apr 22]. JAMA. 2020;e206775. pmid:32320003

22. Statista. Share of population that are aged 65 years and older in European countries in 2019. March, 2020. https://www.statista.com/statistics/1105835/share-of-elderly-population-in-europe-by-country/

23. Moon AM, Webb GJ, Aloman C, et al. High mortality rates for SARS-CoV-2 infection in patients with pre-existing chronic liver disease and cirrhosis: Preliminary results from an international registry [published online ahead of print, 2020 May 21]. J Hepatol. 2020;S0168-8278(20)30305-6. pmid:32446714

24. Tsui PT, Kwok ML, Yuen H, Lai ST. Severe acute respiratory syndrome: clinical outcome and prognostic correlates. Emerg Infect Dis. 2003;9(9):1064‐1069. pmid:14519241

25. Chen N, Zhou M, Dong X, et al. Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study. Lancet. 2020;395(10223):507‐513. pmid:32007143

26. Huang C, Wang Y, Li X, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China [published correction appears in Lancet. 2020 Jan 30;:]. Lancet. 2020;395(10223):497‐506. pmid:31986264

27. Al Ghamdi M, Alghamdi KM, Ghandoora Y, et al. Treatment outcomes for patients with Middle Eastern Respiratory Syndrome Coronavirus (MERS CoV) infection at a coronavirus referral center in the Kingdom of Saudi Arabia. BMC Infect Dis. 2016;16:174. pmid:27097824

28. Di Castelnuovo A, Costanzo S, Cassone A, et al. Low dose hydroxychloroquine is associated with lower mortality in COVID-19: a meta-analysis of 26 studies and 44,521 patients. medRxiv 2020.11.01.20223958; https://doi.org/10.1101/2020.11.01.20223958

29. Siemieniuk RA, Bartoszko JJ, Ge L, et al. Drug treatments for covid-19: living systematic review and network meta-analysis. BMJ. 2020 Jul 30;370:m2980. Update in: BMJ. 2020 Sep 11;370:m3536 pmid:32732190