Evaluation of the SpO2/FiO2 ratio as a predictor of intensive care unit transfers in respiratory ward patients for whom the rapid response system has been activated

Abstract

Efforts to detect patient deterioration early have led to the development of early warning score (EWS) models. However, these models are disease-nonspecific and have shown variable accuracy in predicting unexpected critical events. Here, we propose a simpler and more accurate method for predicting risk in respiratory ward patients. This retrospective study analyzed adult patients who were admitted to the respiratory ward and detected using the rapid response system (RRS). Study outcomes included transfer to the intensive care unit (ICU) within 24 hours after RRS activation and in-hospital mortality. Prediction power of existing EWS models including Modified EWS (MEWS), National EWS (NEWS), and VitalPAC EWS (ViEWS) and SpO2/FiO2 (SF) ratio were compared to each other using the area under the receiver operating characteristic curve (AUROC). Overall, 456 patients were included; median age was 75 years (interquartile range: 65–80) and 344 (75.4%) were male. Seventy-three (16.0%) and 79 (17.3%) patients were transferred to the ICU and died. The SF ratio displayed better or comparable predictive accuracy for unexpected ICU transfer (AUROC: 0.744) compared to MEWS (0.744 vs. 0.653, P = 0.03), NEWS (0.744 vs. 0.667, P = 0.04), and ViEWS (0.744 vs. 0.675, P = 0.06). For in-hospital mortality, although there was no statistical difference, the AUROC of the SF ratio (0.660) was higher than that of each of the preexisting EWS models. In comparison with the preexisting EWS models, the SF ratio showed better or comparable predictive accuracy for unexpected ICU transfers in the respiratory wards.

About the Authors:

Won Gun Kwack

Roles Conceptualization, Data curation, Formal analysis, Writing – original draft, Writing – review & editing

Affiliations Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Republic of Korea, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea

Dong Seon Lee

Roles Data curation

Affiliation: Interdepartment of Critical Care Medicine, Seoul National University Bundang Hospital, Seongnam, Republic of Korea

Hyunju Min

Roles Data curation

Affiliation: Interdepartment of Critical Care Medicine, Seoul National University Bundang Hospital, Seongnam, Republic of Korea

Yun Young Choi

Roles Data curation

Affiliation: Interdepartment of Critical Care Medicine, Seoul National University Bundang Hospital, Seongnam, Republic of Korea

Miae Yun

Roles Data curation

Affiliation: Interdepartment of Critical Care Medicine, Seoul National University Bundang Hospital, Seongnam, Republic of Korea

Youlim Kim

Roles Formal analysis

Affiliations Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Republic of Korea, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea

Sang Hoon Lee

Roles Formal analysis

Affiliations Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Republic of Korea, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea

Inae Song

Roles Supervision

Affiliation: Department of Anesthesiology, Seoul National University Bundang Hospital, Seongnam, Republic of Korea

ORCID logo http://orcid.org/0000-0002-7814-4253

Jong Sun Park

Roles Supervision

Affiliations Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Republic of Korea, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea

Young-Jae Cho

Roles Supervision

Affiliations Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Republic of Korea, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea

You Hwan Jo

Roles Supervision

Affiliation: Department of Emergency Medicine, Seoul National University Bundang Hospital, Seongnam, Republic of Korea

Ho Il Yoon

Roles Writing – review & editing

Affiliations Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Republic of Korea, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea

Jae Ho Lee

Roles Writing – review & editing

Affiliations Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Republic of Korea, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea

Choon-Taek Lee

Roles Writing – review & editing

Affiliations Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Republic of Korea, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea

Yeon Joo Lee

Roles Conceptualization, Writing – review & editing

* E-mail: [email protected]

Affiliations Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul National University Bundang Hospital, Seongnam, Republic of Korea, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea

ORCID logo http://orcid.org/0000-0001-7697-4272

Introduction

Respiratory diseases are among the leading causes of the disability- adjusted life-years lost and mortality worldwide, accounting for one-quarter of the total global mortality [1]. In Korea, pneumonia has been the most common cause of hospital admission over the of last six years [2]. Patients admitted respiratory ward are more likely to need the assistance of a rapid response system (RRS) due to their vulnerable health status, including frequent desaturation, which can become critical. For these reasons, it is very important to predict acute deterioration of inpatients and initiate appropriate management in the respiratory ward.

Since the conception of the original early warning score (EWS) in the 1990s, various revised EWS such as Modified EWS (MEWS), National EWS (NEWS), and VitalPAC EWS (ViEWS) have been developed and used to discriminate the risk of catastrophic deterioration in general ward [3–6]. Although a number of studies reported that EWS systems can predict clinical outcomes, including unexpected intensive care unit (ICU) admission and cardiopulmonary arrest [7], there are remaining concerns that most EWS systems are disease-nonspecific and have relatively low predictive accuracy. Accordingly, new models with additional factors such as lactic acid or d-dimer levels, which have not been included in traditional EWS models, have been developed and validated to improve accuracy according to targeted patient groups [8, 9]. There is also a need for tailored EWS models to ensure practical usefulness in the respiratory wards [10].

The noninvasive and continuous pulse oximetry saturation (SpO2) is frequently checked in general wards and is included in the standard monitoring required in most ICUs [11]. In a previous study, the SpO2/fraction of inspired oxygen (FiO2) ratio (or SF ratio) was correlated with partial pressure of arterial oxygen (PaO2)/ FiO2 ratio (or PF ratio), which is a severity index in acute lung injury and acute respiratory distress syndrome (ARDS) [12]. Additionally, the SF ratio based ARDS patients demonstrated similar clinical characteristics and outcomes, duration of ventilator support and ICU stay, when compared with the PF ratio based ARDS patients [13]. In light of these findings, the SF ratio may be considered an easily applicable tool for the early recognition of patients with the potential for acute deterioration. However, to our knowledge, there is no report that the discrimination power of the SF ratio alone, and in comparison with other models, is a predictor for patients admitted in the respiratory ward.

The present study aimed to evaluate the ability of the SF ratio to predict acute deterioration in patients in the respiratory ward and to compare its predictive power with conventional EWS models (MEWS, NEWS, and ViEWS).

Materials and methods

Study design and patients

We conducted a retrospective cohort study using the RRS registry of the Seoul National University Bundang Hospital (SNUBH), a 1,360-bed tertiary care academic hospital, from January 2015 to August 2017. We extracted the data of patients admitted in the respiratory ward from the RRS registry. Detailed demographics, clinical data, and physical findings at RRS activation were ascertained through an electronic medical record (EMR) chart review. Collected variables included age, sex, Charlson comorbidity index score [14], cause of admission, systolic and diastolic blood pressure, heart rate, respiratory rate, body temperature, conscious level (AVPU: alert, responds to voice, responds to pain, and unresponsive), SpO2, the presence of oxygen therapy, FiO2, patients disposition after the RRS intervention, length of hospital stay, and survival status at discharge. Subsequently, the SF ratio, MEWS, NEWS, and ViEWS were obtained. The MEWS, NEWS, and ViEWS were calculated by summing the total score according to each of the predefined categories (Table 1 and S1 and S2 Tables). The primary outcomes were unplanned ICU transfer within 24 hours after RRS activation and in-hospital mortality. We compared the various scores with the SF ratio using a receiver operating characteristic (ROC) curve analysis.

[Figure omitted. See PDF.]

Table 1. Modified early warning score (MEWS).

https://doi.org/10.1371/journal.pone.0201632.t001

Rapid response system design

Our hospital has operated a part-timed RRS since October 2012. The RRS team has a role to timely detect and manage deteriorating patients in general ward [15]. RRS activation is possible in two ways except when a cardiopulmonary resuscitation request (CPR) is announced. The main way in which RRS is activated is through an automatic EMR screening system based on 10 screening criteria, each with a predefined threshold, including physiological and laboratory parameters (S3 Table). When a patient meets any of the screening criteria s/he is presented to a monitoring board and dedicated RRS nurses conduct a primary check up on the patient. Patients are also identified via direct requests from healthcare providers including residents or nurses. In the present study, if one patient had multiple RRS activations over the one admission period, only the first event was included. In case of multiple admissions during the study period, the same inclusion/exclusion criteria were applied individually to each admission. Patient events were excluded if they were documented as a do not resuscitate (DNR) prior to RRS activation or if they had RRS activation for CPR.

Ethical approval

The present study was performed in accordance with the Declaration of Helsinki. The Institutional Review Board (IRB) of the Seoul National University Bundang Hospital (SNUBH) approved the study (approved IRB No.: B-1801-447-105) and informed consent was waived due to the retrospective study design.

Statistical analysis

Continuous data and categorical data were presented as medians and interquartile ranges (IQR) or mean ± standard deviation. All continuous and categorical variables were compared using a Student’s t-test or Mann–Whitney U test, and Chi-square test, respectively. The accuracy of each parameter or models for predicting ICU transfers and in-hospital mortality were quantified using the area under the ROC curve (AUROC) and comparisons for each AUROC were determined using the De Long’s test [16]. The Youden index point was calculated as the optimal threshold for the ROC analysis [17]. To assess reasonable cutoff values, sensitivity, specificity, positive predictive values (PPV), negative predictive values (NPV), positive likelihood ratio (PLR), and negative likelihood ratio (NLR) were also investigated. All data were statistically analyzed using SPSS 23.0 for Windows (SPSS, Chicago, IL, USA) and MedCalc 17.9.7 for Windows (MedCalc, Ostend, Belgium) and P-values of less than 0.05 were considered as significant.

Results

During study period, 1018 patients required practical RRS, however 562 were ineligible based on the exclusion criteria (duplicated cases: 336; prior DNR: 153; CPR: 5; monitoring in transfer and simple procedure assistant: 68). Consequently, 456 patients were included in this study. The median age of enrolled participants was 75 (IQR: 65–80) years and 344 (75.4%) were male. The median SF ratio was 294 (IQR: 209–388). ICU transfer within the 24 hours following RRS activation and in-hospital mortality occurred in 90 (19.7%) and 79 (17.3%) patients, respectively (Table 2).

[Figure omitted. See PDF.]

Table 2. Baseline characteristics and clinical outcomes of RRS-activated patients in respiratory wards.

https://doi.org/10.1371/journal.pone.0201632.t002

The differences in characteristics and calculated predicting parameters at the time of RRS activation based on the patient’ disposition after RRS intervention and in-hospital mortality were identified (S4 and S5 Tables). Compared to the general ward group, the ICU transfer group had a lower SF ratio (median 165 vs. 320, P < 0.01) and a higher score for all of the conventional EWS models including MEWS, NEWS, and ViEWS (P < 0.01). The mortality group also had a lower SF ratio (median 217 vs. 307, P < 0.01) and a higher score for all of conventional EWS models than those who survived (P < 0.01). Additionally, the Charlson comorbidity index, chronic respiratory disease, presence of a neoplasm, and interstitial lung disease (cause of admission) were also significantly higher in the mortality group (P < 0.01). Multivariate analysis, adjusted for age, sex, Charlson comorbidity index, chronic respiratory disease, presence of a neoplasm, and interstitial lung disease (cause of admission) for each predictive indicator, showed that the SF ratio, MEWS, NEWS and ViEWS had significant association with ICU transfers and in-hospital mortality (Table 3).

[Figure omitted. See PDF.]

Table 3. Predictive values for intensive care unit transfers and mortality.

https://doi.org/10.1371/journal.pone.0201632.t003

In the ROC curve analysis (Fig 1) for ICU transfers, the AUROC of the SF ratio (0.744; 95% confidence interval [CI] 0.684 to 0.803) was significantly higher than that of MEWS (0.653; 0.59 to 0.716), and NEWS (0.667; 0.605 to 0.73). The SF ratio also had a higher AUROC compared to ViEWS, however this difference was not significant (0.744 vs. 0.675, P = 0.06). For in-hospital mortality (Fig 2), although there were no significant differences (P > 0.05), the AUROC of the SF ratio (0.660; 0.614 to 0.703) was higher than that any of each EWS models.

[Figure omitted. See PDF.]

Fig 1. Comparison of the area of under the receiver operating characteristics curve for intensive care unit transfers within 24 hours of rapid response system activation.

MEWS, Modified Early Warning Score; NEWS, National Early Warning Score; ViEWS, VitalPAC Early Warning Score; AUROC, area under the receiver operating characteristic curve; ICU, intensive care unit. *SF ratio: SpO2/FiO2 ratio.

https://doi.org/10.1371/journal.pone.0201632.g001

[Figure omitted. See PDF.]

Fig 2. Comparison of the area of under the receiver operating characteristics curve for in-hospital mortality.

MEWS, Modified Early Warning Score; NEWS, National Early Warning Score; ViEWS, VitalPAC Early Warning Score; AUROC, area under the receiver operating characteristic curve. *SF ratio: SpO2/FiO2 ratio.

https://doi.org/10.1371/journal.pone.0201632.g002

For ICU transfers, diagnostic performance of the SF ratio including sensitivity, specificity, PPV, NPV, PLR, and NLR were assessed (Table 4). The Youden index cutoff of 170 had a sensitivity, specificity, PPV, and NPV of 52.2%, 89.6%, 55.3%, and 88.4%, respectively. On the contrary, a cutoff of 300 showed the highest sensitivity, with a specificity of more than 50%. Using 300 as the cutoff resulted in a sensitivity, specificity, PPV, and NPV of 78.8%, 53.8%, 29.8%, and 91.3%, respectively.

[Figure omitted. See PDF.]

Table 4. Predictive values for intensive care unit transfers.

https://doi.org/10.1371/journal.pone.0201632.t004

Discussion

The present study indicated that, as compared with several preexisting EWS systems (MEWS, NEWS and ViEWS), the SF ratio offered superior or equal prediction accuracy for ICU transfers from the respiratory ward. Additionally, the SF ratio had greater prediction accuracy for in-hospital mortality than each of the conventional EWS models, however this difference was not statistically significant. To the best of our knowledge, this is the first study to evaluate the SF ratio as a predictor of unplanned ICU transfers and in-hospital mortality in patients admitted in a respiratory ward.

Early recognition of a critical deterioration can be a matter of life or death in the practical clinical field. The decision regarding whether a patient needs intensive care is important and always difficult. Furthermore, if the available ICU resources are limited, the decision becomes even more difficult. Therefore, useful tools as predictors of acute deterioration have been required to assist healthcare providers with precise decision making. Previous studies have found that most deteriorating patients already showed changes in their vital signs before the 6- to 24-hour period that preceded the catastrophic developments in their cases, such as unexpected ICU transfer or cardiopulmonary arrest [18–20]. For this reason, various EWS models based on physiologic parameters have been suggested and applied. However, in order for these instruments to be actually used in clinical practice, prediction accuracy and simplicity should be taken into account. For improvement in predicting accuracy, diverse modifications such as changes in weighting scores and additional factors have been piloted based on specific diseases or the locations that patients presented. Yoo et al. reported the combination of MEWS and blood lactate was more accurate than MEWS alone in prediction of ICU transfers in patients with severe sepsis/septic shock [21]. Jo et al. calculated score that was the sum of ViEWS and blood lactate, which are referred to as ViEW-L score. In patients admitted to the medical ICU via the emergency department, ViEW-L score offered the same or better predictive than ViEWS alone, simplified acute physiology scores (SAPS) II, and SAPS III for hospital mortality [22]. The combination of NEWS and D-dimer also appears to identify patients at risk of mortality within 30 days of admission [23]. Furthermore, using the sum of simplified regression weights, gastrointestinal early warning score (EWS-GI) composed of physiologic and laboratory parameters was suggested as a predictor for ICU transfers in gastroenterology wards [24].

Most of the previous modifications of EWS were more focused on the improvement of predictive accuracy than simplicity for easy application. In the present study, we found that the ability of the SF ratio for the prediction of ICU transfers and in-hospital mortality was better or comparable to conventional EWS models. Our findings highlight the importance of achieving a balance between improvements in accuracy and simplicity. The inclusion of more associated factors improved the accuracy in most, but not all cases. In the present study, the combination MEWS and the SF ratio using predicted probability exhibited more predictive accuracy than not only conventional EWS models but also the SF ratio for ICU transfers and in-hospital mortality (see online supplementary file S1 and S2 Figs). However, in actual clinical practice, score calculation according to complex scoring categories or formula is very difficult, even if an automatic calculation system can be employed to assist with the calculation. This might therefore pose as a barrier to wider application. The decision regarding the need for the introduction of further intensive care should takes into account the patient’s condition and clinical markers in a comprehensive manner. Risk factors and EWS models cannot be absolute criterion, but rather auxiliary tools. We do not anticipate that the SF ratio should be an absolute criterion, replacing the already existing EWS. Additionally, the SF ratio does not provide PaCO2 or acid-base status, two other potentially important clinical findings that reflect the patient’s state. Therefore, we suggest that the SF ratio could be a comparable or additional tool to be implemented for decision making in the respiratory ward. In the future, studies on new independent predictive factors or modified EWS should consider not only the improvement of accuracy but also user-friendly models as auxiliary tools. Furthermore, validation of the SF ratio for various situations including mixed or other patient populations, such as medical and surgical, is needed. Additionally, there is a need for well controlled studies regarding treatment options, such as the frequency of ICU treatment and definitive DNR state during hospitalization, in order to investigate more accurate predictive power for prognosis including in-hospital mortality. Such findings are expected to be useful and interesting.

We investigated the performance of SF ratio cutoff values of 170 (which was selected based on Youden’s index) and 300 for predicting unexpected ICU transfers. Using SF ratio categories of ≤170 vs. >170, we observed a higher specificity (89.6%) than sensitivity (52.2%), and the PLR and NLR values were 5.0 and 0.53, respectively. In contrast, using SF ratio categories of ≤300 vs. >300, we observed a higher sensitivity (78.8%) than specificity (53.8%), and the PLR and NLR values were 1.71 and 0.39, respectively. Based on these findings, a cutoff value of 170 could be used as a rule-in point and a cutoff value of 300 could be used as a rule-out point [25]. Furthermore, the previous study which evaluated the correlation between the SF ratio and the PF ratio, found that SF ratios of 235 and 315 corresponded to PF ratios of 200 and 300, respectively. These are the oxygenation criteria for defining moderate and mild ARDS according to the Berlin definition [26]. It is interesting that the suggested cutoff values in the present study were similar to the PF ratio criteria as the severity index of ARDS. By validating proper cutoff values, a reasonable tool for decision making could be devised. Still, it is necessary to determine whether it is more important to focus on rule-in or rule-out points as cutoff values for ICU transfers, depending on the facilities of individual ICUs.

Our study was subject to several limitations. First, there might be the difference between the real SF ratio and the estimated SF ratio. In present study, FiO2 was estimated based on the amount of O2 supply according to the type of tools and this could result in the difference between the real supplied oxygen FiO2 and estimated FiO2 with the exception of high flow nasal cannula. Furthermore, for an SpO2 value of over 97%, the oxyhemoglobin dissociation curve was nearly flat [27]. We cannot be sure that the oxygen supply was titrated appropriately in the patients with SpO2 values over 97%. Second, the RRS registry of our institution was constructed prospectively, and we analyzed data extracted from the registry. Therefore, the data were relatively accurate. However, there may be factors associated with ICU transfer and mortality that were not investigated due to the study’s retrospective nature. The presence subsequent RRS activations following the first activation and a patient’s history of receiving CPR are two such factors that may need to be evaluated in the future. Finally, we only compared the SF ratio with conventional EWS systems, which only consisted of physiologic variables. To evaluate the usefulness of the SF ratio, studies should also compare it with newly derived EWS models, such as recent combinations of preexisting EWS models with additional factors.

In conclusion, in the respiratory ward, compared to the preexisting EWS models including MEWS, NEWS and ViEWS, the SF ratio is simple formula which showed better or comparable predictive accuracy for unexpected ICU transfers. Therefore, the SF ratio with cutoff values of 170 and 300, can play a role in predicting acute deterioration and in communication along RRS chain. Future well controlled studies of the prognosis based on SF ratios are needed to investigate the actual clinical benefits.

Supporting information

[Figure omitted. See PDF.]

S1 Table. National early warning score (NEWS).

SBP, systolic blood pressure; A alert, V responds to verbal stimuli; P, responds to pain only; U, unresponsive to stimuli.

https://doi.org/10.1371/journal.pone.0201632.s001

(DOCX)

S2 Table. VitalPAC early warning score (ViEWS).

SBP, systolic blood pressure; A, alert, V responds to verbal stimuli; P, responds to pain only; U, unresponsive to stimuli.

https://doi.org/10.1371/journal.pone.0201632.s002

(DOCX)

S3 Table. Rapid response system activation criteria.

SBP, systolic blood pressure; HR, heart rate; RR, respiratory rate; BT, body temperature; SpO2, pulse oximetry saturation; pH, potential of hydrogen; PaCO2, partial pressure of carbon dioxide; PaO2, partial pressure of oxygen.

https://doi.org/10.1371/journal.pone.0201632.s003

(DOCX)

S4 Table. Comparisons based on patients disposition.

Data are presented as medians (interquartile range) or numbers (percentage) of patients ICU, intensive care unit; CAD, coronary artery disease; CVA, cerebrovascular accident definition; COPD, chronic obstructive pulmonary disease; MEWS, Modified Early Warning Score; NEWS, National Early Warning Score; ViEWS, VitalPAC Early Warning Score; SD, standard deviation; CRP, C-reactive protein. *SF ratio: SpO2/FiO2 ratio.

https://doi.org/10.1371/journal.pone.0201632.s004

(DOCX)

S5 Table. Comparison based on survival.

Data are presented as medians (interquartile range) or numbers (percentage) of patients. CAD, coronary artery disease; CVA, cerebrovascular accident definition; COPD, chronic obstructive pulmonary disease; MEWS, Modified Early Warning Score; NEWS, National Early Warning Score; ViEWS, VitalPAC Early Warning Score; SD, standard deviation; CRP, C-reactive protein. *SF ratio: SpO2/FiO2 ratio.

https://doi.org/10.1371/journal.pone.0201632.s005

(DOCX)

S1 Fig. Comparison of the area of under the receiver operating characteristics curve for intensive care unit transfers within 24 hours of rapid response system activation.

MEWS, Modified Early Warning Score; NEWS, National Early Warning Score; ViEWS, VitalPAC Early Warning Score; AUROC, area under the receiver operating characteristic curve; ICU, intensive care unit. *MEWS-SF ratio: the combination MEWS and SF ratio was calculated using predicted probability; †SF ratio: SpO2/FiO2 ratio.

https://doi.org/10.1371/journal.pone.0201632.s006

(DOCX)

S2 Fig. Comparison of the area of under the receiver operating characteristics curve for in-hospital mortality.

MEWS, Modified Early Warning Score; NEWS, National Early Warning Score; ViEWS, VitalPAC Early Warning Score; AUROC, area under the receiver operating characteristic curve. *MEWS-SF ratio: the combination MEWS and SF ratio was calculated using predicted probability; †SF ratio: SpO2/FiO2 ratio.

https://doi.org/10.1371/journal.pone.0201632.s007

(DOCX)

Acknowledgments

We thank the members of the Seoul National University Bundang Hospital Medical Alert First Responder (SAFER) team. Furthermore, we thank the members of the Division of Statistics of Medical Research Collaborating Center at SNUBH for performing the statistical analyses.

Citation: Kwack WG, Lee DS, Min H, Choi YY, Yun M, Kim Y, et al. (2018) Evaluation of the SpO2/FiO2 ratio as a predictor of intensive care unit transfers in respiratory ward patients for whom the rapid response system has been activated. PLoS ONE 13(7): e0201632. https://doi.org/10.1371/journal.pone.0201632

References

1. Ferkol T, Schraufnagel D. The global burden of respiratory disease. Annals of the American Thoracic Society. 2014;11(3):404–6. pmid:24673696

2. Health Insurance Reveiw and Assessment Service. Medical Statistics information. 2017; 11: 17. Available from: http://opendata.hira.or.kr/op/opc/olapHifrqSickInfo.do.

3. Morgan R, Williams F, Wright M. An early warning scoring system for detecting developing critical illness. Clin Intensive Care. 1997;8(2):100.

4. Subbe C, Kruger M, Rutherford P, Gemmel L. Validation of a modified Early Warning Score in medical admissions. Qjm. 2001;94(10):521–6. pmid:11588210

5. Prytherch DR, Smith GB, Schmidt PE, Featherstone PI. ViEWS—Towards a national early warning score for detecting adult inpatient deterioration. Resuscitation. 2010;81(8):932–7. pmid:20637974

6. Williams B, Alberti G, Ball C, Bell D, Binks R, Durham L. National Early Warning Score (NEWS): standardising the assessment of acute-illness severity in the NHS. London: The Royal College of Physicians. 2012.

7. Mapp ID, Davis LL, Krowchuk H. Prevention of unplanned intensive care unit admissions and hospital mortality by early warning systems. Dimensions of Critical Care Nursing. 2013;32(6):300–9. pmid:24100432

8. Stenhouse C, Coates S, Tivey M, Allsop P, Parker T. Prospective evaluation of a modified Early Warning Score to aid earlier detection of patients developing critical illness on a general surgical ward. Br J Anaesth. 2000;84(5):663.

9. Jo S, Jeong T, Lee JB, Jin Y, Yoon J, Park B. Validation of modified early warning score using serum lactate level in community-acquired pneumonia patients. The National Early Warning Score-Lactate score. The American journal of emergency medicine. 2016;34(3):536–41.

10. Lee YJ, Lee DS. Differences in the Clinical Characteristics of Rapid Response System Activation in Patients Admitted to Medical or Surgical Services. 2017;32(4):688–94.

11. Jubran A. Pulse oximetry. Critical care (London, England). 2015;19:272.

12. Rice TW, Wheeler AP, Bernard GR, Hayden DL, Schoenfeld DA, Ware LB. Comparison of the SpO2/FIO2 ratio and the PaO2/FIO2 ratio in patients with acute lung injury or ARDS. Chest. 2007;132(2):410–7. pmid:17573487

13. Chen W, Janz DR, Shaver CM, Bernard GR, Bastarache JA, Ware LB. Clinical Characteristics and Outcomes Are Similar in ARDS Diagnosed by Oxygen Saturation/Fio2 Ratio Compared With Pao2/Fio2 Ratio. Chest. 2015;148(6):1477–83. pmid:26271028

14. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. Journal of chronic diseases. 1987;40(5):373–83. pmid:3558716

15. Kim Y, Lee DS, Min H, Choi YY, Lee EY, Song I, et al. Effectiveness Analysis of a Part-Time Rapid Response System During Operation Versus Nonoperation. Critical care medicine. 2017;45(6):e592–e9. pmid:28346260

16. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–45. pmid:3203132

17. Perkins NJ, Schisterman EF. The inconsistency of "optimal" cutpoints obtained using two criteria based on the receiver operating characteristic curve. American journal of epidemiology. 2006;163(7):670–5. pmid:16410346

18. Hillman KM, Bristow PJ, Chey T, Daffurn K, Jacques T, Norman SL, et al. Duration of life-threatening antecedents prior to intensive care admission. Intensive care medicine. 2002;28(11):1629–34. pmid:12415452

19. Schein RM, Hazday N, Pena M, Ruben BH, Sprung CL. Clinical antecedents to in-hospital cardiopulmonary arrest. Chest. 1990;98(6):1388–92. pmid:2245680

20. Franklin C, Mathew J. Developing strategies to prevent inhospital cardiac arrest: analyzing responses of physicians and nurses in the hours before the event. Critical care medicine. 1994;22(2):244–7. pmid:8306682

21. Yoo JW, Lee JR, Jung YK, Choi SH, Son JS, Kang BJ, et al. A combination of early warning score and lactate to predict intensive care unit transfer of inpatients with severe sepsis/septic shock. The Korean journal of internal medicine. 2015;30(4):471–7. pmid:26161013

22. Jo S, Lee JB, Jin YH, Jeong TO, Yoon JC, Jun YK, et al. Modified early warning score with rapid lactate level in critically ill medical patients: the ViEWS-L score. Emergency medicine journal: EMJ. 2013;30(2):123–9. pmid:22427404

23. Nickel CH, Kellett J, Cooksley T, Bingisser R, Henriksen DP, Brabrand M. Combined use of the National Early Warning Score and D-dimer levels to predict 30-day and 365-day mortality in medical patients. Resuscitation. 2016;106:49–52. pmid:27339095

24. Kim WY, Lee J, Lee JR, Jung YK, Kim HJ, Huh JW, et al. A risk scoring model based on vital signs and laboratory data predicting transfer to the intensive care unit of patients admitted to gastroenterology wards. Journal of critical care. 2017;40:213–7. pmid:28445859

25. Boyko EJ. Ruling out or ruling in disease with the most sensitive or specific diagnostic test: short cut or wrong turn? Medical decision making: an international journal of the Society for Medical Decision Making. 1994;14(2):175–9.

26. Ranieri VM, Rubenfeld GD, Thompson BT, Ferguson ND, Caldwell E, Fan E, et al. Acute respiratory distress syndrome: the Berlin Definition. Jama. 2012;307(23):2526–33. pmid:22797452

27. Morgan TJ. The oxyhaemoglobin dissociation curve in critical illness. Critical care and resuscitation: journal of the Australasian Academy of Critical Care Medicine. 1999;1(1):93–100.