Designing a more efficient, effective and safe Medical Emergency Team (MET) service using data analysis

Abstract

Introduction

Hospitals have seen a rise in Medical Emergency Team (MET) reviews. We hypothesised that the commonest MET calls result in similar treatments. Our aim was to design a pre-emptive management algorithm that allowed direct institution of treatment to patients without having to wait for attendance of the MET team and to model its potential impact on MET call incidence and patient outcomes.

Methods

Data was extracted for all MET calls from the hospital database. Association rule data mining techniques were used to identify the most common combinations of MET call causes, outcomes and therapies.

Results

There were 13,656 MET calls during the 34-month study period in 7936 patients. The most common MET call was for hypotension [31%, (2459/7936)]. These MET calls were strongly associated with the immediate administration of intra-venous fluid (70% [1714/2459] v 13% [739/5477] p<0.001), unless the patient was located on a respiratory ward (adjusted OR 0.41 [95%CI 0.25–0.67] p<0.001), had a cardiac cause for admission (adjusted OR 0.61 [95%CI 0.50–0.75] p<0.001) or was under the care of the heart failure team (adjusted OR 0.29 [95%CI 0.19–0.42] p<0.001).

Modelling the effect of a pre-emptive management algorithm for immediate fluid administration without MET activation on data from a test period of 24 months following the study period, suggested it would lead to a 68.7% (2541/3697) reduction in MET calls for hypotension and a 19.6% (2541/12938) reduction in total METs without adverse effects on patients.

Conclusion

Routinely collected data and analytic techniques can be used to develop a pre-emptive management algorithm to administer intravenous fluid therapy to a specific group of hypotensive patients without the need to initiate a MET call. This could both lead to earlier treatment for the patient and less total MET calls.

About the Authors:

Christoph Bergmeir

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Software, Supervision, Validation, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliation: Faculty of Information Technology, Monash University, Clayton, Australia

ORCID http://orcid.org/0000-0002-3665-9021

Irma Bilgrami

Roles Conceptualization, Investigation, Methodology, Project administration, Validation, Writing – original draft, Writing – review & editing

Affiliation: Intensive Care Specialist, Departments of Anaesthesia, Intensive Care and Pain Management, Western Health, Gordon Street, Footscray, Vic, Australia

Christopher Bain

Roles Conceptualization, Data curation, Methodology, Project administration, Resources, Software, Writing – review & editing

Affiliation: Faculty of Information Technology, Monash University, Clayton, Australia

Geoffrey I. Webb

Roles Conceptualization, Methodology, Project administration, Software, Supervision, Writing – review & editing

Affiliation: Faculty of Information Technology, Monash University, Clayton, Australia

Judit Orosz

Roles Investigation, Resources, Supervision, Validation, Writing – review & editing

Affiliations Department of Intensive Care Medicine, Commercial Road, The Alfred Hospital, Prahran, Vic, Australia, The Australian and New Zealand Intensive Care (ANZIC)–Research Centre, School of Public Health and Preventive Medicine, Monash University, Prahran, Vic, Australia

David Pilcher

Roles Conceptualization, Data curation, Methodology, Project administration, Supervision, Validation, Writing – original draft, Writing – review & editing

Affiliations Department of Intensive Care Medicine, Commercial Road, The Alfred Hospital, Prahran, Vic, Australia, The Australian and New Zealand Intensive Care (ANZIC)–Research Centre, School of Public Health and Preventive Medicine, Monash University, Prahran, Vic, Australia

Introduction

Over the last two decades, Rapid Response Systems and Medical Emergency Teams (METs) have become widely adopted in hospitals across Australia and New Zealand [1,2]. It has become an expectation and a minimum standard of care that such systems detect and respond to physiological deterioration in a timely fashion [3,4]. The Australian Commission on Safety and Quality in Health Care (ACSQHC) has supported this endeavour and released a National Consensus Statement on management of the deteriorating patient [5]. Recent data suggests an increasing number of MET reviews and this number varies from 1.35/ 1000 admissions to 71.3/1000 admissions [6]. The reasons behind the increase and the variability are unclear. Local hospital culture and resource availability may play a role [7,8]. Only a limited number of hospitals receive funding dedicated to the MET [8]. This may have implications not only for resource allocation but also individual patient care.

The traditional design of a Rapid Response System has four components. An afferent arm or the event detection arm that triggers the response; an efferent arm or the responder arm (the MET); an administrative structure; and a quality improvement arm [9]. Given the rising number of rapid response calls, this ‘blanket’ efferent and afferent response to every trigger event may not be a sustainable model for the future.

We hypothesised that the commonest MET triggers resulted in similar treatments and that it might be possible to identify a cohort of patients in whom treatment could be directly initiated without having to await attendance of the MET. This might increase the efficiency of the MET system, through a total reduction in MET numbers and earlier administration of appropriate treatment to patients. Our aim was to develop a ‘pre-emptive management algorithm’ using available institutional data, for immediate administration of therapy at the most common MET calls and to model its potential impact on MET call incidence and patient outcomes, prior to implementation into real world practice.

Because large datasets of information are commonly now available in healthcare [10], advanced exploratory data analysis techniques that can identify potentially useful patterns in data are of ever greater value. Statistically sound association rule techniques systematically explore complex interaction in data while using multiple testing corrections to ensure statistical validity [11]. In consequence they are ideally suited to identifying the most common combinations of MET call causes, outcomes and therapies, allowing identification of combinations of variables that are useful in practice to incorporate into an algorithm.

Methods

Ethics

The Alfred Health Human Ethics Committee approved the study and provided a waiver for informed consent (Project 580/14).

Setting

The Alfred hospital is a tertiary referral centre in Melbourne, Australia. It provides a state service for trauma, burns, cystic fibrosis, heart, lung and bone marrow transplant, extracorporeal cardiac and respiratory support, hyperbaric medicine and infectious diseases including HIV. The hospital admits more than 35,000 adult patients a year and has more than 3000 admissions to the Intensive Care Unit (ICU) per year.

The hospital has a mature MET service, established in early 2000s. The MET service is a 24-hour service, led by the ICU senior registrar and an Intensive Care trained nurse. The on call medical registrar and a member of the home team also attend. The MET responds to more than 6000 reviews every year. MET calls are mandatory if a patient meets pre-specified criteria (see supplementary material–S1 Table).

At the end of every MET call, the reason for the call, its location, therapy given and patient outcome are entered into a hospital wide MET database. The patient’s vital signs at the time of MET call are recorded in the patient’s records but not entered into the database.

Study design

We conducted a retrospective observational study and extracted data for all MET calls between March 2012 (when the complete electronic MET database was implemented) and December 2016. Data from March 2012 to December 2014 was used to develop an algorithm which was subsequently validated on data from January 2015 to December 2016.

Data were extracted from the “REASON Discovery Platform” [12], the hospital’s corporate data platform which routinely aggregated, on a nightly basis, information from the MET database, demographics, pathology, admitting unit, admission/discharge dates from administrative databases and diagnostic coding data. The following data was collected for every MET call: timing of activation, the clinical trigger for the call and any therapies administered. We also measured patient outcomes post MET call including transfer to ICU or another high acuity area, repeat MET calls, length of stay in hospital and in-hospital mortality.

Statistical analysis and association rule mining techniques

Association rule data mining techniques, implemented in the software Magnum Opus®, were used to search for patterns in the dataset and to identify the most common combinations of MET call causes, outcomes and therapies for inclusion in the ‘pre-emptive management algorithm’. Classical univariate pair-wise testing and multivariable logistic regression tests were then employed to check the validity of the findings from the association rule data mining and to test the potential impact of the algorithm.

Magnum Opus® performs statistically sound association rule mining, by combining k-optimal association discovery techniques [13], to find the k most interesting associations according to a defined criterion, such as lift, leverage, strength, support, or coverage. It achieves efficiency in the search by using the OPUS search method [14], and incorporates facilities for filtering the discovered associations to discard those that are likely to be spurious [11]. Magnum Opus® is available for use at [15], an open source implementation of OPUS is available at [16]. This advanced exploratory data analysis technique considers all possible correlations between combinations of up to three variables with a target. It uses the Bonferroni correction to ensure strict control over familywise error despite the combinatorial explosion in the number of tests performed. Interactions of interest are identified by measuring the frequency with which combinations of values occurred jointly, relative to the frequency that would be expected were the relevant combination of values statistically independent of the target.

Following identification of ‘candidate combinations’ of MET calls, outcomes and therapies using the above techniques, univariate statistical pair-wise testing without adjustment for multiple testing was employed using Student T-, Chi-square, Fisher’s exact or Wilcoxon rank sum tests as appropriate depending on the distribution of the data. Multivariable logistic regression in the R programming language [17], function glm, from the R stats package was used with associations reported as odds ratios with 95% confidence interval.

Only the first MET call per patient was considered for initial analysis and development of the algorithm. Modelling the effect of the algorithm was performed on this dataset. The primary outcome considered was the modelled number of MET calls, with the algorithm in place. Other outcomes considered included: in-hospital mortality, mortality at the MET call, proportion of patients admitted to ICU and length of stay in hospital.

Results

The initial study period (March 2012 to December 2014), in the following also called the training period, was used for exploratory data analysis and to develop the algorithm. The results in the following are results from this period. During the training period, there were a total of 13,656 MET calls in 7936 patients. This equated to a MET dose of 144 METs per 1000 hospital admissions. MET call activity increased during the study period, with yearly mean daily counts of 11.0, 13.8, and 14.5 MET calls respectively, for the years 2012, 2013, and 2014. Frequencies of calling criteria are shown in Table 1. Of the 7936 patients, 7027 (88.5%) had a single calling criterion for MET activation.

[Figure omitted. See PDF.]

Table 1. MET activations in the training period. Data includes only the first MET call per patient.

MET = Medical Emergency Team, CPR = Cardiopulmonary resuscitation.

https://doi.org/10.1371/journal.pone.0188688.t001

Identification of target group for algorithm development: The commonest MET call & therapy combination

Hypotension, defined as systolic blood pressure <90mmHg, was the most common sole MET call criterion (30.9%, 2459/7936). Hypotension was also present in 267 out of 909 cases with a combination of calling criteria, which led to activation of the MET (29.3%, 267/909) (Table 1)

Association rule data mining identified a MET call for hypotension and administration of intravenous fluid as the single most common combination of calling criterion and therapy in the training period. Compared to MET calls for other reasons, patients who had MET calls for hypotension were more commonly given intra-venous fluid therapy (70% [1714/2459] v 13% [739/5477] p<0.001), more commonly managed on the ward (94% [2316/2459] v 85% [4647/5477] p<0.003) and had fewer multiple METs (35% [867/2459] vs. 38% [2103/ 5477] p<0.008), were less likely to go to ICU (2% [52/ 2459] vs. 6% [311/5477] p<0.001) and had a lower mortality (5% [133/2459] vs. 11% [620/5477] p<0.001) than those who had MET calls for other reasons. Comparison of patients who received a MET review for hypotension to patients who received a MET review for other criteria is shown in S2 Table of the supplementary material.

Development of the pre-emptive management algorithm for patients with MET calls for hypotension

Having identified the most common therapy given to hypotensive patients (intravenous fluid), the characteristics and outcomes of patients who received this therapy at a MET call were compared to those who were not given fluid (Table 2), in the training period. Patients given fluid were older, more commonly under the care of general surgical, trauma or orthopaedic units and wards, less commonly had primary cardiac diagnoses, and more commonly had more interventions performed at the MET call but remained on the ward with equal frequency compared to those where fluid was not administered. The only difference in outcomes between the two groups, was that patients who were given fluid had a higher number of multiple METs (38% [649/1714] vs. 29% [218/745] p<0.001).

[Figure omitted. See PDF.]

Table 2. Comparison of patients who were given intravenous fluid therapy to those not given fluid therapy, at medical emergency team (MET) calls for hypotension, in the training period.

Due to space constraints, for clinical units, MET call locations, and interventions, only results with (p < 0.05) are reported. mmol/L = Millimole per litre, micromol/L = micromoles per litre, g/dL = Grams per decilitre.

https://doi.org/10.1371/journal.pone.0188688.t002

Association rule data mining, together with a subsequent multivariate logistic regression confirmed that the most important correlations were seen if the patient was located on a respiratory ward (adjusted OR 0.41 [95%CI 0.25–0.67] p<0.001), had a cardiac cause for admission (adjusted OR 0.61 [95%CI 0.50–0.75] p<0.001) or was under the care of the heart failure team (adjusted OR 0.29 [95%CI 0.19–0.42] p<0.001), in which case the patient was less likely to receive intravenous fluid.

Based on the above findings from the training period, an algorithm for direct nurse initiated administration of intravenous fluid was developed and its potential impact on MET call numbers and patients’ outcomes was modelled. The algorithm is shown in Fig 1. Unless on the respiratory ward, admitted with a cardiac diagnosis or under the care of the heart failure team, any patient who had systolic hypotension < 90mmHg was to receive intravenous fluid without need for a MET call. In all other cases, a MET was to be called.

[Figure omitted. See PDF.]

Fig 1. Flow chart of the proposed algorithm.

https://doi.org/10.1371/journal.pone.0188688.g001

Modelling the impact of the pre-emptive management algorithm.

We subsequently assessed the impact of the algorithm using a separate test dataset, comprised of 12,938 MET calls in 7106 patients from the 24 months period of January 2015 to December 2016. All the results in the following are results for this test period.

Assessment of primary outcome.

If the algorithm had been applied, there would have been a 68.7% (2541/3697) reduction in MET calls for hypotension and a 19.6% (2541/12938) reduction in total METs (Table 3) in the test period. This would also have resulted in 50.0% (1045/2091) patients who had a MET call for hypotension, receiving the intravenous fluids without actually having to activate a MET review (‘true positives’). All other patients would have continued to receive a MET review.

[Figure omitted. See PDF.]

Table 3. Modelled estimated effect of algorithm for immediate administration of intravenous fluid to patients with systolic blood pressure (SBP) < 90mmHg, on medical emergency team (MET) calls in the test period.

https://doi.org/10.1371/journal.pone.0188688.t003

Assessment of outcomes in patients in whom the algorithm predicted ‘inadvertent fluid therapy’

The algorithm would have resulted in intravenous fluid administration to 19.3% (403/2091) of patients who did not actually receive it (‘false positives’ or ‘inadvertent fluid therapy’)(Table 3), in the test period. These patients (the ‘false positives’) appeared to be a ‘lower risk’ group, when compared to those who both received intravenous fluids and the algorithm predicted fluid administration. They had lower mortality and in 57.8% of these patients, the MET provided advice only and did not order any investigations or prescribe a therapy (Table 4).

[Figure omitted. See PDF.]

Table 4. Comparison of interventions and outcomes of hypotensive patients who did receive intravenous fluid and were predicted to do so by the algorithm (‘true positives’), vs. those who did not receive intravenous fluid but would have been predicted to do so (‘false positives’), in the test period.

‘True positives’ represent those with MET calls for SBP <90 mmHg, predicted to get intravenous fluid who did receive it. ‘False positives’ represent those with MET calls for SBP <90 mmHg, predicted to get intravenous fluid who did not receive it. The p values are for comparison between True and False positive cases. MET = Medical Emergency Team, ECG = Electrocardiogram, ABG = Arterial Blood Gas.

https://doi.org/10.1371/journal.pone.0188688.t004

Discussion

Summary of findings

In this study of 7936 patients at a tertiary centre over a 34-month period (training period), the most common combination of MET call and therapy was observed in patients who received intravenous fluid at a MET call for a systolic blood pressure < 90mmHg. A combination of data association mining and classical statistical techniques allowed the development and assessment of a pre-emptive management algorithm for immediate management of these patients without having to initiate a MET call. Modelling the effect of this algorithm for immediate administration of intravenous fluid to hypotensive patients on an independent test period suggested that it would lead to a reduction in overall numbers of MET calls and if applied, it would have led to inadvertent intravenous therapy to a small proportion of low risk patients and was thus unlikely to result in adverse effects to patients.

Comparison to previous studies

Literature describing common MET triggers is limited [6]. These have been described in various frequencies in single centre studies [18,19]. In our study, the most common trigger for a MET response was a systolic blood pressure <90mmHg. These patients were more likely to be managed on the ward and less likely to have multiple METs, an important finding as patients with repeat METs are known to be a higher risk group [20]. In hospital mortality of hypotensive patients in our study (5%) was lower than the range reported in literature (25%) [6]. The ICU admission rate (2%) was also lower than the reported rates in literature (10–25%). These outcomes suggest that this is an ideal group for application of a pre-emptive management algorithm.

Data describing therapies administered during MET calls is limited [19,21]. One study found oxygen therapy or ventilatory support as the most common intervention provided by MET [19]. We found MET reviews for hypotension were strongly associated with fluid administration unless the patient was on the respiratory ward, had a cardiac diagnosis or was admitted under the heart failure team. We also found that patients receiving fluid therapy at a MET call for hypotension had a higher number of multiple METs than those who did not receive fluid therapy. The relevance of this is unclear as all other patient outcomes were similar in both groups.

Implications

We have shown that routinely collected hospital data can be used to develop a pre-emptive management algorithm, which might facilitate administration of therapy to a specific group of hypotensive patients. Modelling implementation of this algorithm suggested its introduction would be associated with a reduction in MET call numbers without compromising patient outcomes. In the face of rapid adoption of MET systems and an increasing workload [2] this demonstrates the potential systems redesign and safe resource reallocation.

Patients who receive a MET response are a high-risk group [6,7] and a delay in activating MET calls leads to higher mortality [22]. The traditional model of the MET system has a disadvantage [2,9] in that treatment can only be given once the effector response arrives at the patient’s bedside. Given the high MET dose in our hospital, there is a danger of delayed response times with the potential to cause harm. One of the potential advantages of implementing this pre-emptive management algorithm is to avoid delay.

Strengths

We used a large, well-established database run by trained staff. The study period of 34 months allowed us to study established trends. We used association rule mining techniques to examine the data and results were consistent when the standard statistical techniques were utilised. The data mining techniques involved can be generalised to any large dataset. We have demonstrated that evolution and development of clinical systems may be achieved through analysis of data rather than relying solely on opinion, supporting a move towards evidence based practice.

The algorithm designed was simple and is directly applicable to our hospital setting which makes likelihood of successful implementation greater. Internal validity of our study was supported by biologically plausible findings such as the fact that cardiac and respiratory patients who had hypotension were less likely than others to receive intravenous fluid.

Limitations

This was an observational study and the impact of unmeasured confounders (e.g. staffing levels, education, handover processes) is unknown. This work describes methodology for development of a management algorithm and cannot be considered definitive assessment of its applicability. This work is the first phase prior to assessment of such an algorithm in practice. The proposed algorithm has not been implemented or tested in a controlled trial or in clinical practice. Moreover, although the algorithm has been prospectively validated, and the principles of its development may be broadly relevant, since this is a single centre study, the applicability of the findings and the relevance of the algorithm to other hospitals and healthcare systems is unknown.

Furthermore, although our analysis suggested that patients who might be managed by our proposed algorithm (and would not have a MET call) were a 'low risk' group, a retrospective study such as this cannot fully address all potential safety concerns with its implementation. For instance, there might need to be a systolic blood pressure level albeit a lower value, at which a MET call would still have to be called.

Accuracy of data elements is unknown. Absolute values of systolic blood pressure were not recorded. Hence it is unknown if there are subgroups of patients with very low blood pressures which might be disadvantaged by our algorithm and failure to immediately activate a MET call. However, the relatively good outcomes in these patients suggested they were not a high-risk group overall. The incidence, quality and impact of clinical assessment of patients is unknown. However it is recognised that fluid assessment in the critically ill is challenging and the accuracy of physical assessment in determining fluid state has been questioned [23].

Future work

Future studies should assess whether the same analytic techniques can be applied in other hospital settings to develop similar algorithms to improve MET systems. Locally, routine recording of absolute numerical values for physiological observations at MET calls is needed, and careful assessment of the algorithm in a clinical arena is required.

Conclusion

Using routinely collected data and specific analytic techniques, it is possible to develop a pre-emptive management algorithm to administer intravenous fluid therapy to a specific group of hypotensive patients without the need to initiate a MET call. Application of such an algorithm is likely to be safe, result in a reduction in total MET calls, improved efficiency of the MET system and may lead to improved patient outcomes.

Supporting information

[Figure omitted. See PDF.]

S1 Table. MET calling criteria.

https://doi.org/10.1371/journal.pone.0188688.s001

(PDF)

S2 Table. Comparison of medical emergency team (MET) calls for hypotension to MET calls for other reasons in the training period.

https://doi.org/10.1371/journal.pone.0188688.s002

(PDF)

S3 Table. Patient characteristics associated with no fluid administration, in the training period.

https://doi.org/10.1371/journal.pone.0188688.s003

(PDF)

Acknowledgments

This research has been supported by the Australian Research Council under grant DP140100087.

Citation: Bergmeir C, Bilgrami I, Bain C, Webb GI, Orosz J, Pilcher D (2017) Designing a more efficient, effective and safe Medical Emergency Team (MET) service using data analysis. PLoS ONE 12(12): e0188688. https://doi.org/10.1371/journal.pone.0188688

References

1. Lee A, Bishop G, Hillman KM, Daffurn K (1995) The Medical Emergency Team. Anaesth Intensive Care 23: 183–186. pmid:7793590

2. Jones D, George C, Hart GK, Bellomo R, Martin J (2008) Introduction of medical emergency teams in Australia and New Zealand: a multi-centre study. Crit Care 12: R46. pmid:18394192

3. Jones D, Bates S, Warrillow S, Goldsmith D, Kattula A, Way M et al. (2006) Effect of an education programme on the utilization of a medical emergency team in a teaching hospital. Intern Med J 36: 231–236. pmid:16640740

4. Jones DA, Mitra B, Barbetti J, Choate K, Leong T, Bellomo R (2006) Increasing the use of an existing medical emergency team in a teaching hospital. Anaesth Intensive Care 34: 731–735. pmid:17183890

5. Australian Commission on Safety and Quality In Health Care http://www.safetyandquality.gov.au/our-work/recognising-and-responding-to-clinical-deterioration/.

6. Jones D (2014) The epidemiology of adult Rapid Response Team patients in Australia. Anaesth Intensive Care 42: 213–219. pmid:24580387

7. Anzics-Core Met dose investigators (2013) Mortality of rapid response team patients in Australia: a multicentre study. Crit Care Resusc 15: 273–278. pmid:24289508

8. Jones D, Drennan K, Hart GK, Bellomo R, Web SA (2012) Rapid Response Team composition, resourcing and calling criteria in Australia. Resuscitation 83: 563–567. pmid:22067975

9. Devita MA, Bellomo R, Hillman K, Kellum J, Rotondi A, Teres D et al. (2006) Findings of the first consensus conference on medical emergency teams. Crit Care Med 34: 2463–2478. pmid:16878033

10. Hermon R, Williams PAH. Big data in healthcare: What is it used for?; 2014; Cowan University, Joondalup Campus, Perth, Western Australia.

11. Webb GI (2007) Discovering Significant Patterns. Machine Learning 68: 1–33.

12. Bain C, Mac Manus C,. Advancing data management and usage in a major Australian health service: The REASON discovery platform™; 2014. pp. 38–43.

13. Webb G, I., Zhang S (2005) k-Optimal-Rule-Discovery. Data Mining and Knowledge Discovery 10: 39–79.

14. Webb G, I. (1995) OPUS: An Efficient Admissible Algorithm For Unordered Search. Journal of Artificial Intelligence Research 3: 431–465.

15. BigML: Machine Learning Online Platform. www.bigml.com.

16. Webb GI, Bergmeir C, Dempster A (2017) opusminer: OPUS Miner Algorithm for Filtered Top-k Association Discovery. R package version 0.1–0. https://CRAN.R-project.org/package=opusminer.

17. R Core Team (2014) R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org/.

18. Jones D, Duke G, Green J, Briedis J, Bellomo R, Casamento A et al. (2006) Medical emergency team syndromes and an approach to their management. Crit Care 10: R30. pmid:16507153

19. Kenward G, Castle N, Hodgetts T, Shaikh L (2004) Evaluation of a medical emergency team one year after implementation. Resuscitation 61: 257–263. pmid:15172703

20. Calzavacca P, Licari E, Tee A, Mercer I, Haase M, Haase-Fielitz A et al. (2010) Features and outcome of patients receiving multiple Medical Emergency Team reviews. Resuscitation 81: 1509–1515. pmid:20673606

21. Quach JL, Downey AW, Haase M, Haase-Fielitz A, Jones D, Bellomo R (2008) Characteristics and outcomes of patients receiving a medical emergency team review for respiratory distress or hypotension. J Crit Care 23: 325–331. pmid:18725036

22. Trinkle RM, Flabouris A (2011) Documenting Rapid Response System afferent limb failure and associated patient outcomes. Resuscitation 82: 810–814. pmid:21497982

23. Marik PE (2009) Techniques for assessment of intravascular volume in critically ill patients. J Intensive Care Med 24: 329–337. pmid:19648183