Prediction Model for Critically Ill Patients with Acute Respiratory Distress Syndrome

Abstract

Background and objectives

Acute respiratory distress syndrome (ARDS) is a major cause respiratory failure in intensive care unit (ICU). Early recognition of patients at high risk of death is of vital importance in managing them. The aim of the study was to establish a prediction model by using variables that were readily available in routine clinical practice.

Methods

The study was a secondary analysis of data obtained from the NHLBI Biologic Specimen and Data Repository Information Coordinating Center. Patients were enrolled between August 2007 and July 2008 from 33 hospitals. Demographics and laboratory findings were extracted from dataset. Univariate analyses were performed to screen variables with p<0.3. Then these variables were subject to automatic stepwise forward selection with significance level of 0.1. Interaction terms and fractional polynomials were examined for variables in the main effect model. Multiple imputations and bootstraps procedures were used to obtain estimations of coefficients with better external validation. Overall model fit and logistic regression diagnostics were explored.

Main result

A total of 282 ARDS patients were included for model development. The final model included eight variables without interaction terms and non-linear functions. Because the variable coefficients changed substantially after exclusion of most poorly fitted and influential subjects, we estimated the coefficient after exclusion of these outliers. The equation for the fitted model was: g(Χ)=0.06×age(in years)+2.23(if on vasopressor)+1.37×potassium (mmol/l)-0.007×platelet count (×109)+0.03×heart rate (/min)-0.29×Hb(g/dl)-0.67×T(°C)+0.01×PaO_2+13, and the probability of death π(Χ)=eg(Χ)/(1+eg(Χ)).

Conclusion

The study established a prediction model for ARDS patients requiring mechanical ventilation. The model was examined with rigorous methodology and can be used for risk stratification in ARDS patients.

About the Authors:

Zhongheng Zhang

* E-mail: [email protected]

Affiliation: Department of Critical Care Medicine, Jinhua Municipal Central Hospital, Jinhua Hospital of Zhejiang University, Zhejiang, P.R. China

Hongying Ni

Affiliation: Department of Critical Care Medicine, Jinhua Municipal Central Hospital, Jinhua Hospital of Zhejiang University, Zhejiang, P.R. China

Introduction

Acute respiratory distress syndrome (ARDS) is a severe form of acute lung injury most commonly seen in intensive care unit (ICU) and it is associated with significant morbidity and mortality. The incidence of ARDS is estimated to be approximately 40–80 per 100,000 patient-years [1–4]. and the figure can vary with different definitions of ARDS. The mortality rate in patients with established ARDS is around 50–60% [4–6]. More recently, due to advances in the management of ARDS such as use of low tidal volume ventilation and extracorporeal membrane oxygenation, the mortality has shown a reduction to around 30% [7, 8]. However, There is limited supportive evidence that specific interventions can decrease mortality in ARDS, and the mortality of ARDS does not show significant reduction over time.[9–12] Therefore, ARDS remains to be a great challenge to clinicians.

The initial step in mortality reduction is to identify risk factors for poor clinical outcomes. This is an area being extensively studied. For instance, sepsis-induced ARDS has been found to be associated with increased risk of death as compared with other caused.[13] Patients with lower BAL levels of procollagen peptide showed lower mortality than those with higher levels.[14] However, most of these studies investigated risk factors in isolation. Because there are multiple factors working together to determine the final outcomes of ARDS patients, it is more clinically useful to develop a prediction model for risk stratification. Gajic O and colleagues[15] developed a well-calibrated model for mortality prediction for ARDS patients, which however required information on organ functions three days after intubation. In another study, risk tertiles model was developed for predicting mortality in ARDS. However, the study categorized continuous variables into tertiles, which is thought to be associated with information loss [16]. In the present study, we aimed to develop a prediction model for ARDS patients requiring mechanical ventilation. The principal in developing the model is a balance between parsimony and model fitting. Furthermore, the variables included in the model should be readily available in routine clinical practices.

Methods

The performance of the secondary data analysis was approved by the ethics committee of Jinhua municipal central hospital and informed consent was waived. Patient records or information was anonymized and de-identified prior to analysis. The study was a secondary analysis of data obtained from the NHLBI Biologic Specimen and Data Repository Information Coordinating Center. The original randomized controlled trial was entitled “Randomized, Placebo-controlled Clinical Trial of an Aerosolized b2-Agonist for Treatment of Acute Lung Injury” (NCT 00434993) and has been published elsewhere.[17]

Patients were enrolled between August 2007 and July 2008 from 33 hospitals of the National heart, lung, and blood institute ARDS clinical trials network. Inclusion criteria were 1) patient had to be intubated and receiving mechanical ventilation; 2) bilateral infiltrates consistent with edema on chest X-ray, 3) had an PaO2/FiO2<300, 4) no clinical evidence of left atrial hypertension. The definition of ARDS in the study was made according to the American-European Consensus Conference (AECC) criteria [18]. In Berlin definition, the use of PEEP was considered and ARDS was categorized into mild, moderate and severe forms [19]. Patients were excluded if 1) they had coexisting chronic lung disease; 2) unable to obtain consent; 3) acute myocardial infarction; 4) chronic liver disease; 5) neuromuscular disease.

Data extraction

Data were extracted from the dataset obtained from the NHLBI Biologic Specimen and Data Repository Information Coordinating Center after approval. The variable name was annotated in a file named “data dictionary”. The following variables were extracted: age, gender, body mass index (BMI), past history of cigarette smoking and alcohol consumption, types of ICU (medical ICU, surgical ICU and mixed ICU), admission type (unscheduled surgery, scheduled surgery and medical admission), causes of ARDS (sepsis, transfusion, aspiration, pneumonia, other lung conditions), admission source (operating room, emergency department, floor ward and others), comorbidities (chronic dialysis, leukemia, immunodeficiency, cirrhosis, diabetes, hypertension, myocardial infarction, heart failure, vascular disease, dementia, chronic pulmonary disease, arthritis, peptic ulcer), vasopressor use, hemoglobin, white blood cell, platelet count, creatinine, bilirubin, sodium, potassium, glucose, bicarbonate, phosphate, magnesium, total protein, albumin, minimal glucose, FiO2, PaO2, PaCO2, pH, lowest and highest temperature, lowest and highest systolic blood pressure, lowest and highest mean blood pressure, lowest and highest respiratory rate, urine output (24 hours), transfusion of RBC, FFP transfusion. All measurements, including laboratory and physiological findings were performed within the first 24 hours.

Statistical analysis

Univariate logistic regression model was performed to screen factors associated with mortality in ARDS patients requiring mechanical ventilation. The dependent variable was a binary outcome with “1” indicated death and “0” indicated survival. Independent variables were categorized into continuous variables and indicator variables. Continuous variables such as age, laboratory measurements and urine output were reported their ORs for each one unit increase in the parameter. Indicator variables were reported their ORs for each category as referenced to the base status. For instance, patients exposed to cigarette smoking were compared to those without cigarette smoking, and OR was reported for the variable. Alcohol intake was reported as the frequency and we dichotomized them into patients with and without history of alcohol intake. Smokers were reported as non-smoker, former smoker and current smoker. We combined the latter two categories as smokers. There were eight categories for the variable of admission sources. Because too many categories might compromise the statistical power, we grouped patients from operating room and recovery room together.

Variables with p< 0.3 in univariate analysis are included for automatic stepwise selection of covariates. Furthermore, variables with prevalence<10% or with >15% missing observations were excluded from further analysis. In the study, phosphate and bilirubin both showed more than 30% missing observations and was excluded from multivariable analysis despite their significance in univariate analysis. Stepwise forward selection of variable with p<0.1 was performed to screen variables independently associated with mortality. The automatic selection would finally generate a main effect model. To make full use of data information and improve statistical power, we use multiple imputation technique other than the conventional complete case analysis as a sensitivity test to examine the robustness of our result.[20] Overfitting with optimism in coefficient estimate was another concern in building a prediction model.[21] Thus, we use the bootstrapping technique to adjust for coefficients estimated from conventional multivariable analysis.[22] Bootstrapping was a technique of resampling with replacement and here we repeated the sampling for 500 times. The bootstrapping sampling technique would shrink the estimated coefficient but provide better prediction to future samples.

Potential interactions among included covariates in the main effect model were tested by including them one by one. To simplify this process, we created a local macro called “covariate” to store all covariates in the main effect model and thereafter the process can be automated by using foreach syntax. Interaction terms with p<0.1 would be included in the model. Linearity of covariates on logit scale is a fundamental assumption in model fitting. Therefore, we employed multivariable fractional polynomials (MFP) to test whether other power terms were superior to the linear term.[23, 24] Firstly, the best fitting one-term and two-term models were modeled by choosing power transformations from the set <-2, -1, -0.5, 0, 0.5, 1, 2, 3>, where 0 denoted the log transformation. Next, the closed test procedure was performed in which the best fitting two-term model was compared with the linear model. If the two-term model was significantly better than the linear one (p<0.05), two-term model was then compared to the best fitting one-term model. Otherwise, linear model was adopted. The procedure continued until there was no statistical significance and the best fitting model was chosen.

Model fit would be assessed from two aspects: a summary measure and regression diagnostics. The Hosmer-Lemeshow tests would be employed in which grouping of covariate pattern was based on the estimated probability.[25] The Hosmer-Lemeshow goodness-of-fit statistic was obtained by calculating Pearson Chi-square statistic from the g×2 table of observed and estimated frequencies. The variable g refers to the number of groups. A statistical significance level p<0.05 indicates that the model is significantly different from the observed outcome. Furthermore, the discrimination of the fitted model was assessed graphically. Logistic regression diagnostics were calculated to see if the model fit over the entire set of covariate patterns.[26] Statistics including leverage, change in Pearson Chi-square (Δχ2), change in deviance (ΔD) and Cook’s distance (Δ) would be plotted against the estimated probability of death ().[27] The aim was to examine cases lied far away from the others. New model fitting would be performed by excluding these outliers. However, diagnostics statistics were used to identify influential subjects and the decision on exclusion should incorporate subject matter considerations.

All statistical analyses were performed by using the STATA 13.1 (StataCorp, College Station, Texas 77845 USA).

Results

A total of 2688 patients were initially screened. Then 2406 patients were excluded, remaining 282 ARDS patients who required invasive mechanical ventilation. The most common reasons for exclusion were chronic lung disease (19.2%), unable to obtain consent (15.2%) and time window exceeded (14.7%). There were 61 non-survivors and 221 survivors, with an overall mortality rate of 21.63%. Univariate logistic regression analysis (Table 1) showed that age (OR: 1.04, 95% CI: 1.02–1.06), admission from floor ward (OR: 3.70, 95% CI: 1.02–13.46), leukemia (OR: 7.68, 95% CI: 1.37–43.01), vascular disease (OR: 11.38, 95% CI: 1.16–111.42), vasopressor use (OR: 5.08, 95% CI: 2.60–9.92), platelet count (OR: 0.996, 95% CI: 0.993–0.999), potassium (OR: 1.98, 95% CI: 1.27–3.08), bicarbonate (OR: 0.93, 95% CI: 0.88–0.98), phosphate (OR: 1.40, 95% CI: 1.09–1.81), pH (OR: 0.028, 95% CI: 0.002–0.41), lowest temperature (OR: 0.70, 95% CI: 0.53–0.93), highest temperature (OR: 0.63, 95% CI: 0.45–0.88), lowest systolic pressure (OR: 0.98, 95% CI: 0.96–0.99), urine output (OR: 0.9997, 95% CI: 0.9995–0.9999). Five variables contained missing values: bilirubin (49), phosphate (129), PaO2 (9), pH (9), mean blood pressure (2), urine output (2). Because the variables bilirubin and phosphate had too many missing values (>15%), they were excluded from analysis. Furthermore, the comorbidity variables including leukemia, immunodeficiency and vascular disease were excluded because the prevalence was less than 10%. There was no significant difference on PEEP (9.1±3.7 vs.9.3±3.3 cmH2O; p = 0.71) and FiO2 (0.58±0.19 vs.0.57±0.17; p = 0.69) between survivors and non-survivors (Fig. 1).

[Figure omitted. See PDF.]

Fig 1. Graphical presentation of PEEP and FiO2 in survivors and non-survivors at the onset of ARDS.

There was no significant difference on PEEP (9.1±3.7 vs.9.3±3.3 cmH2O; p = 0.71) and FiO2 (0.58±0.19 vs.0.57±0.17; p = 0.69) between survivors and non-survivors.

https://doi.org/10.1371/journal.pone.0120641.g001

[Figure omitted. See PDF.]

Table 1. Univariate analysis by mortality.

https://doi.org/10.1371/journal.pone.0120641.t001

As a result, a total of 21 covariates were entered into the full model. After stepwise forward selection with p = 0.1, eight covariates remained in the model (Table 2), including age (coefficient: 0.05, 95% CI: 0.027–0.077), vasopressor (coefficient: 1.47, 95% CI: 0.70–2.23), potassium (coefficient: 0.78, 95% CI: 0.22–1.34), platelet (coefficient: -0.005, 95% CI: -0.008, -0.001), hemoglobin (coefficient: -0.20, 95% CI: -0.41–0.004), highest heart rate (coefficient: 0.028, 95% CI: 0.01–0.046), highest temperature (coefficient: -0.45, 95% CI: -0.87, -0.02) and PaO2 (coefficient: 0.007, 95% CI: -0.00021–0.013). These coefficients were obtained by using complete case analysis. After multiple imputations procedure, the estimated coefficients shrunk towards zero for all covariates. The coefficients remained unchanged as compared to the original analysis with bootstrap estimation. At this stage, we would like to use the original complete case analysis as the main effect model because of the parsimony principal in estimation analysis.

[Figure omitted. See PDF.]

Table 2. Main effect model after stepwise selection of covariates†.

https://doi.org/10.1371/journal.pone.0120641.t002

Interaction terms were evaluated for all possible interactions, which showed no statistically significant interactions among variables. Linearity assumption for continuous variables in the main effect model was assessed by using multiple fractional polynomials, which showed that other non-linear functions were no better than the linear one. As a result we adopted the original main effect model as the final model. Overall model fit was assessed by using Hosmer-Lemeshow goodness-of-fit test, which showed a χ2 (df = 8) of 6.54 (p = 0.59). Graphical assessment of the overall fit was shown in Fig. 2. The discrimination of the model was good with an area under operating characteristics curve of 0.85. In survivors, the predicted probability of death () gathered below 0.2, indicating a good negative predictive power of the model. However, the scattered evenly in non-survivors, indicating a limited power of positive predictive value. Plot of Logistic regression diagnostics versus estimated probability of death () were shown in Fig. 3. From the figures we identified five covariate patterns with large values of ΔD or Δχ2 (poorest fit) and two with outlying values of Δ (largest influence). These covariate patterns (#72, 126, 137, 147, 171, 207) were shown in table 3. Because there were many continuous covariates, one covariate pattern corresponded to one subject. For instance, the subject #137 was characterized by old age, vasopressor use, hyperkalemia and thrombocytopenia, which was a covariate pattern of high probability of death. However, the subject was observed to survive which violated the fitted model and thus it was considered as an outlier. We examined how the exclusion of these outliers could influence the estimation of coefficients (Table 4). The result showed that all coefficients changed significantly after exclusion of outliers. We would like to use this model as the prediction model for probability of death in future ARDS patients requiring mechanical ventilation:where

[Figure omitted. See PDF.]

Fig 2. Four diagnostic plots to describe discrimination in a model fit with an area under operating characteristics curve of 0.85.

In survivors, the predicted probability of death () gathered below 0.2, indicating a good negative predictive power of the model. However, scattered more evenly in non-survivors, indicating a limited power of positive predictive value.

https://doi.org/10.1371/journal.pone.0120641.g002

[Figure omitted. See PDF.]

Fig 3. Logistic regression diagnostics plotted against estimated probability of death ().

The upper left shows the leverage versus . The influence diagnostic Δ is plotted versus in upper right panel. We noted two points lie somewhat away from the rest of the data. The lower right panel shows the change in Pearson Chi-square as a function of the estimated probability of death, which is helpful in identifying poorest fit points. In this figure, five points are poorly fitted in the top left and top right corner of the plot (Δχ2>10). The size of the symbol is proportional to Δ, allowing us to more clearly ascertain the relative contribution of residual and leverage to Δ. The largest circle in the right corner correspond to a moderate leverage and a large Δχ2, indicating that high leverage might not be a contributing factor. The same five points are shown in lower right panel, but note that the range of Δχ2 is much greater than the change in deviance (ΔD). As a result we identified five covariate patterns with large values of ΔD or Δχ2 (poorest fit) and two with outlying values of Δ (largest influence).

https://doi.org/10.1371/journal.pone.0120641.g003

[Figure omitted. See PDF.]

Table 3. Covariate patterns that do not fit or have considerable influence on estimated coefficient.

https://doi.org/10.1371/journal.pone.0120641.t003

[Figure omitted. See PDF.]

Table 4. Estimated coefficients from all data, the percent change when covariate patterns with poorest fit and largest influence were deleted.

https://doi.org/10.1371/journal.pone.0120641.t004

The prediction model was compared with APACHE III score for its discrimination in predicting mortality (Fig. 4). The result showed that the prediction model had better discrimination than APACHE III (AUC: 0.85, 95% CI: 0.79–0.90 vs. AUC: 0.77 95% CI: 0.70–0.84; p = 0.037).

[Figure omitted. See PDF.]

Fig 4. Comparison of area under ROC for APACHE III and prediction model.

The result showed that the prediction model had better discrimination than APACHE III (AUC: 0.85, 95% CI: 0.79–0.90 vs AUC: 0.77 95% CI: 0.70–0.84; p = 0.037).

https://doi.org/10.1371/journal.pone.0120641.g004

Discussion

The study, by using a prospectively collected dataset, established a prediction model for ARDS patients requiring mechanical ventilation. The model included eight covariates without interaction and non-linear functions. The parsimony of the model may improve the prediction accuracy for future cohort with similar characteristics.[28] The model discrimination was good as reflected by an area under ROC of 0.85. Parameters incorporated into the model were all collected within 24 hours of ICU entry, allowing for early recognition of patients at high risk of death.

For a prediction model to be clinically useful, it should be easy to use. In the study we incorporated variables that were readily available in routine clinical practice. In the model, we found that old age was a significant independent risk factor for death. This is consistent with other studies and prediction models.[29–31] Vasopressor use was also found to be an independent risk factor for mortality. Vasopressor use indicates circulatory failure which is established to be associated with multiple organ failures (e.g. acute kidney injury) in critically ill patients.[32, 33] Organ failure such as acute kidney injury is a well-known mortality risk factor that is also supported by our previous study.[34] Platelet count was also found to be associated with mortality risk in this cohort. However, our previous study showed that it was platelet distribution width and mean platelet volume, rather than platelet count that were independently associated with mortality risk.[35] In that study, we included unselected critically ill patients, and the mortality was slightly higher. The difference in study population and severity of illness may partly explain the disparity between these two cohorts.

Because the prediction model was established with single cohort without external validation, overfitting is a major concern. We employed bootstraps procedure to shrink coefficient and chose model with the principal of parsimony. However, the result showed that the bootstrap procedure did not change the coefficient, indicating that the estimated coefficient is less likely to be biased. There is no evidence of substantial problem with model fit as reflected by the non-significance of Hosmer-Lemeshow goodness-of-fit test. However, such overall model fit cannot exclude some outlying observations. We therefore further examined model fit over the entire set of covariate patterns. As a result, six covariate patterns showed large values in diagnostic statistics, indicating they are either poorly fitted or influential. After exclusion of these six subjects, the coefficients were substantially changed and we choose to retain the model with the outliers excluded. The strength of this technique is to exclude the influence of minority of outlying covariate patterns. However, the shortcoming is certainly that our model cannot be used for subjects with those covariate patterns.

Several limitations need to be acknowledged in the study. First, the major methodological flaws of this secondary analysis is the use of subjects enrolled in a RCT, instead of using data from an observational cohort of consecutive patients. Patients included in a RCT are a selected population that differs from the common patient with the diagnosis under study. For example, the overall mortality rate of this selected group of patients (21%) is below other figures reported in recent epidemiological studies[4–6]. Second, the present analysis is the lack of a validation cohort to test the model. The trial was stopped early due to futility of the intervention. Thus the sample size was small and the dataset cannot be split to training subset and validation subset. However, we examined the overall model fit, as well as the influence of outliers. Furthermore, the problem of overfitting was addressed by using bootstraps procedure to shrink coefficient and the final model was chosen with the principal of parsimony. Third, the reported model is not specific for ARDS patients. In fact, all predictive variables are not specific for ARDS. Thus it would be interesting to test our prediction model in patients without ARDS. Forth, The study suffers slightly from using a single, local cohort, such that international generalisability is questionable. This can be addressed by validating our prediction model in ARDS cohorts from other institutions.

In aggregate, the present study established a prediction model for ARDS patients requiring mechanical ventilation. The model contained eight covariates that are readily available in routine clinical practice and can be applied to all critical care settings. Interaction terms or non-linear functions are not included in the model for parsimony.

Acknowledgments

This Manuscript was prepared using ALTA Research Materials obtained from the NHLBI Biologic Specimen and Data Repository Information Coordinating Center and does not necessarily reflect the opinions or views of the ALTA or the NHLBI.

Author Contributions

Conceived and designed the experiments: ZZ. Performed the experiments: ZZ HN. Analyzed the data: ZZ. Contributed reagents/materials/analysis tools: ZZ. Wrote the paper: HN ZZ.

Citation: Zhang Z, Ni H (2015) Prediction Model for Critically Ill Patients with Acute Respiratory Distress Syndrome. PLoS ONE 10(3): e0120641. https://doi.org/10.1371/journal.pone.0120641

References

1. Hudson LD, Steinberg KP. Epidemiology of acute lung injury and ARDS. Chest. 1999;116(1 Suppl):74S–82S. pmid:10424602

2. Li G, Malinchoc M, Cartin-Ceba R, Venkata CV, Kor DJ, Peters SG, et al. Eight-year trend of acute respiratory distress syndrome: a population-based study in Olmsted County, Minnesota. American journal of respiratory and critical care medicine. 2011;183(1):59–66. pmid:20693377

3. Rubenfeld GD, Caldwell E, Peabody E, Weaver J, Martin DP, Neff M, et al. Incidence and outcomes of acute lung injury. The New England journal of medicine. 2005;353(16):1685–93. pmid:16236739

4. Wong JJ, Loh TF, Testoni D, Yeo JG, Mok YH, Lee JH. Epidemiology of pediatric acute respiratory distress syndrome in singapore: risk factors and predictive respiratory indices for mortality. Frontiers in pediatrics. 2014;2:78. pmid:25121078

5. Khandelwal N, Hough CL, Bansal A, Veenstra DL, Treggiari MM. Long-term survival in patients with severe acute respiratory distress syndrome and rescue therapies for refractory hypoxemia*. Critical care medicine. 2014;42(7):1610–8. pmid:24732240

6. Villar J, Fernandez RL, Ambros A, Parra L, Blanco J, Dominguez-Berrot AM, et al. A Clinical Classification of the Acute Respiratory Distress Syndrome for Predicting Outcome and Guiding Medical Therapy. Critical care medicine. 2014.

7. Tang SS, Redmond K, Griffiths M, Ladas G, Goldstraw P, Dusmet M. The mortality from acute respiratory distress syndrome after pulmonary resection is reducing: a 10-year single institutional experience. European journal of cardio-thoracic surgery: official journal of the European Association for Cardio-thoracic Surgery. 2008;34(4):898–902. pmid:18768325

8. Abel SJ, Finney SJ, Brett SJ, Keogh BF, Morgan CJ, Evans TW. Reduced mortality in association with the acute respiratory distress syndrome (ARDS). Thorax. 1998;53(4):292–4. pmid:9741374

9. Krafft P, Fridrich P, Pernerstorfer T, Fitzgerald RD, Koc D, Schneider B, et al. The acute respiratory distress syndrome: definitions, severity and clinical outcome. An analysis of 101 clinical investigations. Intensive care medicine. 1996;22(6):519–29. pmid:8814466

10. Shekar K. ARDS: rest the lungs or the ventilator? Intensive care medicine. 2014;40(8):1184. pmid:24993464

11. Tonelli AR, Zein J, Adams J, Ioannidis JP. Effects of interventions on survival in acute respiratory distress syndrome: an umbrella review of 159 published randomized trials and 29 meta-analyses. Intensive care medicine. 2014;40(6):769–87. pmid:24667919

12. Villar J, Kacmarek RM, Guerin C. Clinical trials in patients with the acute respiratory distress syndrome: burn after reading. Intensive care medicine. 2014;40(6):900–2. pmid:24718644

13. Hudson LD, Milberg JA, Anardi D, Maunder RJ. Clinical risks for development of the acute respiratory distress syndrome. American journal of respiratory and critical care medicine. 1995;151(2 Pt 1):293–301. pmid:7842182

14. . Clark JG, Milberg JA, Steinberg KP, Hudson LD. Type III procollagen peptide in the adult respiratory distress syndrome. Association of increased peptide levels in bronchoalveolar lavage fluid with increased risk for death. Ann Intern Med. 1995;122: 17–23. pmid:7985891

15. Gajic O, Afessa B, Thompson BT, Frutos-Vivar F, Malinchoc M, Rubenfeld GD, et al. Prediction of death and prolonged mechanical ventilation in acute lung injury. Critical care. 2007;11(3):R53. pmid:17493273

16. Villar J, Perez-Mendez L, Basaldua S, Blanco J, Aguilar G, Toral D, et al. A risk tertiles model for predicting mortality in patients with acute respiratory distress syndrome: age, plateau pressure, and P(aO(2))/F(IO(2)) at ARDS onset can predict mortality. Respiratory care. 2011;56(4):420–8. pmid:21255500

17. National Heart L, Blood Institute Acute Respiratory Distress Syndrome Clinical Trials N, Matthay MA, Brower RG, Carson S, Douglas IS, et al. Randomized, placebo-controlled clinical trial of an aerosolized beta(2)-agonist for treatment of acute lung injury. American journal of respiratory and critical care medicine. 2011;184(5):561–8. pmid:21562125

18. Bernard GR, Artigas A, Brigham KL, Carlet J, Falke K, Hudson L, et al. The American-European Consensus Conference on ARDS. Definitions, mechanisms, relevant outcomes, and clinical trial coordination. American journal of respiratory and critical care medicine. 1994;149(3 Pt 1):818–24. pmid:7509706

19. Force ADT, Ranieri VM, Rubenfeld GD, Thompson BT, Ferguson ND, Caldwell E, et al. Acute respiratory distress syndrome: the Berlin Definition. Jama. 2012;307(23):2526–33. pmid:22797452

20. Schafer JL. Multiple imputation: a primer. Statistical methods in medical research. 1999;8(1):3–15. pmid:10347857

21. Babyak MA. What you see may not be what you get: a brief, nontechnical introduction to overfitting in regression-type models. Psychosomatic medicine. 2004;66(3):411–21. pmid:15184705

22. Carpenter J, Bithell J. Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticians. Statistics in medicine. 2000;19(9):1141–64. pmid:10797513

23. Royston P, Sauerbrei W. Building multivariable regression models with continuous covariates in clinical epidemiology—with an emphasis on fractional polynomials. Methods of information in medicine. 2005;44(4):561–71. pmid:16342923

24. Sauerbrei W, Royston P, Binder H. Selection of important variables and determination of functional form for continuous predictors in multivariable model building. Statistics in medicine. 2007;26(30):5512–28. pmid:18058845

25. Lemeshow S, Hosmer DW Jr. A review of goodness of fit statistics for use in the development of logistic regression models. American journal of epidemiology. 1982;115(1):92–106. pmid:7055134

26. Hosmer DW, Taber S, Lemeshow S. The importance of assessing the fit of logistic regression models: a case study. American Journal of Public Health. 1991;81(12):1630–5. pmid:1746660

27. Zhu H, Ibrahim JG, Cho H. Perturbation and Scaled Cook's Distance. Annals of statistics. 2012;40(2):785–811. pmid:23152646

28. Greenland S. Bayesian perspectives for epidemiological research. II. Regression analysis. International Journal of Epidemiology. 2007;36(1):195–202. pmid:17329317

29. Baldwin MR, Narain WR, Wunsch H, Schluger NW, Cooke JT, Maurer MS, et al. A prognostic model for 6-month mortality in elderly survivors of critical illness. Chest. 2013;143(4):910–9. pmid:23632902

30. Hsieh YZ, Su MC, Wang CH, Wang PC. Prediction of survival of ICU patients using computational intelligence. Computers in biology and medicine. 2014;47:13–9. pmid:24508564

31. Loekito E, Bailey J, Bellomo R, Hart GK, Hegarty C, Davey P, et al. Common laboratory tests predict imminent death in ward patients. Resuscitation. 2013;84(3):280–5. pmid:22863543

32. Panwar R, Lanyon N, Davies AR, Bailey M, Pilcher D, Bellomo R. Mean perfusion pressure deficit during the initial management of shock—an observational cohort study. Journal of Critical Care. 2013;28(5):816–24. pmid:23849541

33. Sulzer JK, Whitaker AM, Molina PE. Hypertonic saline resuscitation enhances blood pressure recovery and decreases organ injury following hemorrhage in acute alcohol intoxicated rodents. Journal of Trauma and Acute Care Surgery. 2013;74(1):196–202. pmid:23147176

34. Zhang Z, Xu X, Ni H, Deng H. Urine output on ICU entry is associated with hospital mortality in unselected critically ill patients. Journal of Nephrology. 2014;27(1):65–71. pmid:24424722

35. Zhang Z, Xu X, Ni H, Deng H. Platelet indices are novel predictors of hospital mortality in intensive care unit patients. Journal of Critical Care. 2014.