Effect of Performance Improvement Programs on Compliance with Sepsis Bundles and Mortality: A Systematic Review and Meta-Analysis of Observational Studies

Abstract

Background

Several reports suggest that implementation of the Surviving Sepsis Campaign (SSC) guidelines is associated with mortality reduction in sepsis. However, adherence to the guideline-based resuscitation and management sepsis bundles is still poor.

Objective

To perform a systematic review of studies evaluating the impact of performance improvement programs on compliance with Surviving Sepsis Campaign (SSC) guideline-based bundles and/or mortality.

Data Sources

Medline (PubMed), Scopus and Intercollegiate Studies Institute Web of Knowledge databases from 2004 (first publication of the SSC guidelines) to October 2014.

Study Selection

Studies on adult patients with sepsis, severe sepsis or septic shock that evaluated changes in compliance to individual/combined bundle targets and/or mortality following the implementation of performance improvement programs. Interventions may consist of educational programs, process changes or both.

Data Extraction

Data from the included studies were extracted independently by two authors. Unadjusted binary data were collected in order to calculate odds ratios (OR) for compliance to individual/combined bundle targets. Adjusted (if available) or unadjusted data of mortality were collected. Random-effects models were used for the data synthesis.

Results

Fifty observational studies were selected. Despite high inconsistency across studies, performance improvement programs were associated with increased compliance with the complete 6-hour bundle (OR = 4.12 [95% confidence interval 2.95-5.76], I2 = 87.72%, k = 25, N = 50,081) and the complete 24-hour bundle (OR = 2.57 [1.74-3.77], I2 = 85.22%, k = 11, N = 45,846) and with a reduction in mortality (OR = 0.66 [0.61-0.72], I2 = 87.93%, k = 48, N = 434,447). Funnel plots showed asymmetry.

Conclusions

Performance improvement programs are associated with increased adherence to resuscitation and management sepsis bundles and with reduced mortality in patients with sepsis, severe sepsis or septic shock.

About the Authors:

Elisa Damiani

* E-mail: [email protected]

Affiliation: Anesthesia and Intensive Care Unit, Department of Biomedical Sciences and Public Health, Università Politecnica delle Marche, Via Tronto 10, 60126 Torrette di Ancona, Italy

Abele Donati

Affiliation: Anesthesia and Intensive Care Unit, Department of Biomedical Sciences and Public Health, Università Politecnica delle Marche, Via Tronto 10, 60126 Torrette di Ancona, Italy

Giulia Serafini

Affiliation: Department of Anesthesiology and Intensive Care, Modena University Hospital, L.go del Pozzo 71, 41100 Modena, Italy

Laura Rinaldi

Affiliation: Department of Anesthesiology and Intensive Care, Modena University Hospital, L.go del Pozzo 71, 41100 Modena, Italy

Erica Adrario

Affiliation: Anesthesia and Intensive Care Unit, Department of Biomedical Sciences and Public Health, Università Politecnica delle Marche, Via Tronto 10, 60126 Torrette di Ancona, Italy

Paolo Pelaia

Affiliation: Anesthesia and Intensive Care Unit, Department of Biomedical Sciences and Public Health, Università Politecnica delle Marche, Via Tronto 10, 60126 Torrette di Ancona, Italy

Stefano Busani

Affiliation: Department of Anesthesiology and Intensive Care, Modena University Hospital, L.go del Pozzo 71, 41100 Modena, Italy

Massimo Girardis

Affiliation: Department of Anesthesiology and Intensive Care, Modena University Hospital, L.go del Pozzo 71, 41100 Modena, Italy

Introduction

Sepsis is a major healthcare problem, with increasing incidence and persistently poor outcome [1]. The Surviving Sepsis Campaign (SSC) was first launched in 2002 with the goals of increasing clinician and public awareness of the sepsis problem, improving the standard of care and reducing mortality [2]. The SSC Guidelines for management of severe sepsis and septic shock were first published in 2004 and lastly updated in 2012 [3], providing recommendations that are intended to guide clinical practice. In order to facilitate the implementation of the guidelines, the key elements were organized into two bundles of care, the “resuscitation” and “management” bundle, including interventions to be accomplished within 6 and 24 hours, respectively [4]. A bundle is a set of diagnostic or therapeutic processes that when implemented as a group may act synergistically, providing a greater survival benefit than each individual component. Several reports showed that compliance with 6-hour and 24-hour sepsis bundles was associated with lower risk of death in patients with severe sepsis and septic shock [5, 6].

Nonetheless, transferring evidence into clinical practice has proven difficult. Adherence to SSC guidelines is still poor, especially among non-intensive care specialists [7]. Performance improvement initiatives (varying from educational programs [8], introduction of clinical decision support tools [9] or dedicated medical staff [10]) were instituted worldwide in the last years in order to address the piecemeal application of sepsis bundles. We performed a systematic review of studies that evaluated the impact of performance improvement programs on compliance to SSC guidelines and/or mortality in septic patients.

Methods

This report adheres to the Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) standards for reporting systematic review and meta-analysis studies (S1 Table). No review protocol exists for the present systematic review.

Eligibility criteria

Studies were eligible for inclusion if they were observational (prospective or retrospective cohort or case-control studies) or randomized controlled trials (RCTs) investigating the effect of a performance improvement program on the implementation of sepsis care and/or patient outcome. Participants were required to be adult patients with sepsis, severe sepsis or septic shock. Sepsis had to be defined according to the criteria established by the American College of Chest Physicians and Society of Critical Care Medicine Consensus Conference [11]. The performance improvement program could be any intervention aimed at improving compliance to one or more components of the 6-hour or 24-hour sepsis bundles as based on the 2004 SSC guidelines [12] (Table 1). The comparator group included adult patients with sepsis, severe sepsis or septic shock who were treated before or without the influence of the implementation program. The primary outcome was changes in compliance to individual and/or combined bundle targets after the performance improvement program. The secondary outcome was mortality.

[Figure omitted. See PDF.]

Table 1. Sepsis bundles.

https://doi.org/10.1371/journal.pone.0125827.t001

Search strategy

Medline (PubMed), Scopus and Intercollegiate Studies Institute (ISI) Web of Knowledge databases were searched from 2004 (first publication of the SSC guidelines). The main search was run on 29th May 2014 and updated weekly until October 2014. The keywords “sepsis”, “septic shock”, “bundle”, “bundled care”, “guidelines”, “surviving sepsis campaign”, “implementation”, “compliance”, “performance improvement program”, “quality improvement program” were typed in various combinations using boolean operators (see the detailed search strategy in S1 Text). Hand searches of reference lists of articles and relevant literature reviews were used to complement the computer search. Content pages of the main journals on critical care medicine were hand-searched in order to find any relevant in-press article. The search was limited to English language studies published in peer-reviewed journals.

Study selection and data extraction

Two independent reviewers screened all identified records (title and abstract) and assessed the selected full-text articles for eligibility. Disagreements were resolved through discussion. Descriptive, methodological and outcome data were extracted from all the eligible studies by two reviewers who worked independently using a predefined data extraction form. The following data were collected: design, study period, country, number of centers included, setting (emergency department [ED], intensive care unit [ICU], hospital), type of center (teaching/non-teaching hospital), type of admission (medical, surgical, mixed), mean age, gender distribution, severity of sepsis (sepsis, severe sepsis and/or septic shock), severity scores (Acute Physiology and Chronic Health Evaluation [APACHE] II score if available; alternatively the Sequential Organ Failure Assessment [SOFA] or the Simplified Acute Physiology Score [SAPS] II), characteristics of the performance improvement program implemented, data for primary and secondary outcome of interest. Severity of illness was recoded as a dichotomous variable based on an expected mortality <50% (APACHE II≤23 or SAPS II≤51 or SOFA≤12, “low severity”) or ≥50% (APACHE II≥24 or SAPS II≥52 or SOFA≥13, “high severity”). Performance improvement programs were categorized as educational only, process change only or both educational and process change based on the type of interventions implemented. It was defined as educational any program aimed at increasing the awareness of sepsis disease by educating medical/nurse staff in sepsis pathogenesis, diagnosis and treatment and promoting the widespread divulgation of this information. An educational program may consist of lectures and training sessions, posters, pocket/bedside reminder cards. A program was categorized as process change if it was intended to induce variations in the standard sepsis management by means of clinical decision support tools (screening tools, predefined order sets, treatment algorithms as flow-charts or checklists, computer applications for bedside monitoring and alert systems, laboratory routine exam sets for sepsis) or the introduction of dedicated staff (sepsis teams, sepsis dedicated units).

Assessment of risk of bias and study quality

Risk of bias was assessed by determining whether each study controlled for two pre-defined confounding factors: illness severity (as defined by APACHE II, SOFA or SAPS) and age. These factors were selected as expected to exert the greatest impact on the outcome, besides being the ones most widely reported by the authors. A study was considered to adequately control for each of these factors if it: a—restricted participant selection so that both groups had the same value for the confounder; b—demonstrated balance between groups for the confounder (p>0.05 in the between-group comparison and/or absolute difference ≤10%); c—matched on the confounder; or d—adjusted for the confounder in statistical analyses to quantify the effect size (ES) for mortality and reported adjusted results. Study quality was assessed by means of the Newcastle-Ottawa Scale (NOS) for cohort/case-control studies [13]. Groups (cohorts/cases and controls) were considered to be comparable if the study controlled for illness severity and/or age. The item was considered as non-fulfilled if the authors adjusted for illness severity and/or age but results of the multivariable models were not reported. Adequacy of follow-up was considered as fulfilled if at least hospital or 28-day mortality were reported. Completeness of follow-up was fulfilled if the authors reported a number/percentage of patients excluded from the analysis because of missing mortality data ≤10% of total. If the number of exclusions was not stated clearly, the item was considered as non-fulfilled.

Statistical analysis

Odds ratio (OR) was chosen as the effect estimate for the data synthesis. As the majority of studies did not report adjusted ORs for the primary outcome of interest (compliance to individual SSC guidelines targets or bundles), unadjusted binary data (number if available, otherwise percentage of patients in which the item was fulfilled in intervention and control groups) were extracted for the calculation of ORs. For the secondary outcome (mortality), adjusted ORs were extracted whenever available; if the authors reported the results of more than one multivariable regression models, the one including severity of illness (as represented by APACHE II, SOFA or SAPS) and/or age was selected; if adjusted ORs were not reported or if the authors reported different effect estimates (Hazard Ratio [HR], Relative Risk [RR]), unadjusted binary data were extracted and used for the reconstruction of OR and data synthesis. Adjusted HR or RR values were not used in the statistical analysis. The influence of this method of analysis (use of adjusted or unadjusted data) on the calculated ES was tested through sensitivity analysis. Hospital mortality was considered whenever available, otherwise the longest-term mortality was used. Data were synthesized using meta-analytic methods [14, 15], and statistically pooled by the standard meta-analysis approach, i.e. studies were weighted by the inverse of the sampling variance. The DerSimonian and Laird random effects model was used as a conservative approach to account for different sources of variation among studies. Forest plots were constructed to graphically represent the results. Q statistics were used to assess heterogeneity among studies. A significant Q value indicates a lack of homogeneity of findings among studies [14, 15]. I2 statistics were then used to quantify the proportion of observed inconsistency across study results not explained by chance [16]. I2 values of <25%, 50% and >75% represent low, moderate and high inconsistency, respectively [16]. Several variables were identified and their effects on outcome examined. Sensitivity analyses were performed by excluding each single study at a time in order to assess the influence of each study on the pooled ES. Categorical variables were treated as moderators and the ES was assessed and compared across subgroups formed by these moderators. Continuous variables were examined as covariates using random effects meta-regression. Meta-regression and sub-group analyses were performed to assess the effect of study quality (NOS score, risk of bias) on the calculated estimates. The presence of publication bias was investigated through funnel plots both visually and formally by trim and fill analysis and Eggers’s linear regression method [17]. A p value less than 0.05 was used to indicate statistical significance. All analyses were conducted using a computer software package (ProMeta Version 2, Internovi, Cesena FC, Italy).

Results

The study selection process is described in Fig 1. Among the initial 1,560 records, 50 studies published between 2006 and 2014 met our inclusion criteria [6, 8–10, 18–63]. Study characteristics are shown in S2 Table. Three studies evaluated two interventions in subsequent periods [53, 57, 60]; each part of these studies was included separately in order to consider the effect of each different intervention. Of the 53 resulting studies, the majority was prospective (23 before-after, 5 time-series analyses, 2 cohort, 1 case-control), 11 were retrospective (8 before-after, 2 cohort, 1 case-control), 11 were historically controlled investigations. No eligible RCT was found. Thirty-eight studies were single-center (72%), 15 were multi-center (28%). Thirty-six studies (68%) out of 53 were on patients with severe sepsis or septic shock, 11 (21%) also included patients with sepsis, 6 (11%) focused only on patients with septic shock. The performance improvement programs implemented were only educational in 17 studies out of 53 (32%), only process changes in 13 (25%), both educational and process changes in 23 (43%). A more detailed description of each program is provided in S2 Table. These programs were implemented in ICU (22 studies; 41%), ED (18; 34%), or the whole hospital (13; 25%). The participating centers were teaching hospitals in most cases (31 studies; 58%). In eight studies the post-intervention phase had been divided in different time periods in order to observe the effect of the performance improvement programs over time (8, 20, 27, 38, 43, 46, 48, 54); in these cases, the final post-intervention period was considered for our analysis. Outcomes measured in each individual study are shown in S3 Table. Risk of bias and study quality assessment is reported in S4 Table.

[Figure omitted. See PDF.]

Fig 1. Flow-chart showing the study selection process.

https://doi.org/10.1371/journal.pone.0125827.g001

Compliance with 6-hour sepsis bundle

Twenty-five studies reported changes in compliance with the complete 6-hour bundle after the performance improvement program, for a total of 50,081 episodes of sepsis, severe sepsis or septic shock evaluated [6, 8, 10, 21, 23, 25, 27, 32, 34, 38, 39–41, 45, 46, 48, 51, 53, 54, 56, 60, 62, 63]. Despite high inconsistency between studies (Q (24) = 195.48, p <0.001, I2 = 87.72%), the majority showed a significant increase in compliance (Fig 2). The overall ES indicated a positive association between the quality improvement interventions and compliance with the resuscitation bundle (OR = 4.12 [95% confidence interval 2.95–5.76], p <0.001, Fig 2). This result did not change significantly when any of the included studies was removed from the analysis in the sensitivity analysis. Funnel plot showed asymmetry (S1 Fig), however with a minor effect on the ES as indicated by the trim and fill analysis (estimated ES: OR = 3.45 [2.49–4.78], p <0.001; number of trimmed studies: 5). The Egger’s linear regression test confirmed the possible presence of a publication bias (p = 0.005). We searched for possible sources of heterogeneity related to different characteristics of studies, patients or programs implemented (Table 2). The ES significantly changed based on the study design (lower ES shown by time-series analyses) and region (highest ESs for studies conducted in North America and Asia). Seven multi-center studies showed lower ES as compared to single-center investigations (k = 18). Studies on patients with higher illness severity (k = 5) consistently showed a larger increase in compliance as compared to those on patients with a lower risk of death. Ten studies that implemented both education and process change programs yielded a higher increase in compliance as compared to those implementing educational (k = 10) or process change (k = 5) interventions alone. A consistent improvement was observed among studies with a lower initial compliance. A greater increase in compliance was observed among studies where only teaching hospitals had been involved (k = 13) as compared to those where both teaching and non-teaching centers had participated (k = 5).

[Figure omitted. See PDF.]

Fig 2. Forest plot showing individual and overall ES of studies that evaluated changes in compliance with the complete 6-hour bundle following the implementation of the performance improvement program (k = 25).

The size of the boxes is inversely proportional to the size of the result study variance, so that more precise studies have larger boxes. The ES is expressed as odds ratio (OR) and the correspondent 95% confidence interval (CI). An OR above 1.00 (right side of the plot) indicates an association between the intervention and increased compliance. ES = effect size; CI = confidence interval; Sig. = p value.

https://doi.org/10.1371/journal.pone.0125827.g002

[Figure omitted. See PDF.]

Table 2. Moderator analysis: compliance with the complete 6-hour sepsis bundle (k = 25).

https://doi.org/10.1371/journal.pone.0125827.t002

A variable number of studies (ranging from 8 to 35) evaluated changes in compliance with individual 6-hour bundle targets. Individual ES and overall estimates are graphically shown in Figs 3 and 4. Although inconsistency between studies was generally high, an increased adherence was visible for all bundle components. Four studies [42, 53, 58] demonstrated a significant decrease in the use of vasopressors after the interventions, which was probably a consequence of the increased adherence to fluid resuscitation. The majority of the funnel plots showed asymmetry, however with a minor effect on the final ES in most cases (S2 Fig).

[Figure omitted. See PDF.]

Fig 3. Forest plots showing individual and overall ES of studies that evaluated changes in compliance with individual 6-hour bundle targets following the implementation of the performance improvement program. (A) Measure lactate (k = 31); (B) Blood cultures (k = 28); (C) Antibiotics (k = 35); (D) Fluid resuscitation (k = 24).

The size of the boxes is inversely proportional to the size of the result study variance, so that more precise studies have larger boxes. The ES is expressed as odds ratio (OR) and the correspondent 95% confidence interval (CI). An OR above 1.00 (right side of the plot) indicates an association between the intervention and increased compliance. ES = effect size; CI = confidence interval; Sig. = p value.

https://doi.org/10.1371/journal.pone.0125827.g003

[Figure omitted. See PDF.]

Fig 4. Forest plots showing individual and overall ES of studies that evaluated changes in compliance with individual 6-hour bundle targets following the implementation of the performance improvement program. (A) Measure central venous pressure (k = 8); (B) Central venous pressure above 8 mmHg (k = 16); (C) Measure SvO2 (k = 8); (D) SvO2 above 70% (k = 15); (E) Mean arterial pressure above 65 mmHg (k = 9); (F) Use of vasopressors (k = 9).

The size of the boxes is inversely proportional to the size of the result study variance, so that more precise studies have larger boxes. The ES is expressed as odds ratio (OR) and the correspondent 95% confidence interval (CI). An OR above 1.00 (right side of the plot) indicates an association between the intervention and increased compliance. ES = effect size; CI = confidence interval; Sig. = p value.

https://doi.org/10.1371/journal.pone.0125827.g004

Compliance with 24-hour sepsis bundle

Eleven studies evaluated changes in compliance with the complete 24-hour bundle following the performance improvement program [6, 8, 10, 21, 25, 27, 38, 54, 56, 62, 63] in a total of 45,846 patients. Most of the studies reported a significant increase in compliance following the implementation of the program, although inconsistency between study results was high (Q (10) = 67.97, p<0.001, I2 = 85.22%). The combined OR was 2.57 [1.74–3.77] (p<0.001) (Fig 5). Sensitivity analysis showed that the removal of any of the included studies did not change the overall OR significantly. The funnel plot showed asymmetry (S3 Fig). The trim and fill analysis revealed a possible significant effect of the publication bias on the overall ES (estimated ES: OR = 1.43 [0.94–2.17] p = 0.094, number of trimmed studies: 5). The Egger’s linear regression test confirmed the possibility of a publication bias (p = 0.048). In the subgroup analyses (Table 3), single-center studies (k = 6) showed higher increase in the adherence to 24-hour bundle in comparison to multi-center studies (k = 5). Four studies that implemented both educational and process change programs showed higher increase in compliance with 24-hour bundle as compared to studies in which only educational or process change interventions were implemented; pure process change programs did not produce a significant improvement in compliance (k = 3). A larger and consistent effect was observed among studies that reported the lowest compliance at baseline (k = 2). A greater increase in compliance tended to be shown by studies where only teaching hospitals had been involved (k = 3) as compared to those where both teaching and non-teaching centers had participated (k = 4), while studies involving only non-teaching hospital (k = 4) did not show a significant improvement in the adherence to the 24-hour bundle.

[Figure omitted. See PDF.]

Fig 5. Forest plot showing individual and overall ES of studies that evaluated changes in compliance with the complete 24-hour bundle following the implementation of the performance improvement program (k = 11).

The size of the boxes is inversely proportional to the size of the result study variance, so that more precise studies have larger boxes. The ES is expressed as odds ratio (OR) and the correspondent 95% confidence interval (CI). An OR above 1.00 (right side of the plot) indicates an association between the intervention and increased compliance. ES = effect size; CI = confidence interval; Sig. = p value.

https://doi.org/10.1371/journal.pone.0125827.g005

[Figure omitted. See PDF.]

Table 3. Moderator analysis: compliance with 24-hour sepsis bundle (k = 11).

https://doi.org/10.1371/journal.pone.0125827.t003

A variable number of studies evaluated changes in compliance with individual 24-hour bundle targets (lung protective ventilation: 9; low-dose steroids: 20; Drotrecogin alfa activated: 16; glucose control: 13). High inconsistency across study findings was observed in general. Individual ES, overall estimates and heterogeneity analysis are shown in Fig 6. Funnel plots for the outcomes lung-protective ventilation and low-dose steroids showed asymmetry (S4 Fig).

[Figure omitted. See PDF.]

Fig 6. Forest plots showing individual and overall ES of studies that evaluated changes in compliance with individual 24-hour bundle targets following the implementation of the performance improvement program. (A) Lung protective ventilation (k = 9); (B) Steroids (k = 20); (C) Drotrecogin alfa (activated) (k = 16); (D) Glucose control (k = 13).

The size of the boxes is inversely proportional to the size of the result study variance, so that more precise studies have larger boxes. The ES is expressed as odds ratio (OR) and the correspondent 95% confidence interval (CI). An OR above 1.00 (right side of the plot) indicates an association between the intervention and increased compliance. ES = effect size; CI = confidence interval; Sig. = p value.

https://doi.org/10.1371/journal.pone.0125827.g006

Mortality

Forty-eight studies evaluated changes in mortality following the implementation of the performance improvement program [6, 8–10, 18–25, 27–33, 35–45, 47, 48, 50, 52–63] in a total of 434,447 patients. Despite high inconsistency between study findings (Q (47) = 389.39, p<0.001, I2 = 87.93%), the majority of studies showed a significant decrease in mortality; the overall OR was 0.66 [0.61–0.72], p<0.001 (Fig 7). The removal of any of the included studies in the sensitivity analysis did not change the results significantly. Funnel plot analysis indicated a possible publication bias (S5 Fig), however with a minor effect on the final ES (estimated ES: OR = 0.77 [0.71–0.83], p<0.001; number of trimmed studies: 15). The Egger’s linear regression test confirmed the possible presence of a publication bias (p<0.001). Subgroup analyses were performed in order to investigate potential sources of heterogeneity (Table 4). The association between performance improvement program and mortality was influenced by study design (with time-series analyses showing higher ES as compared to historically controlled or before-after investigations), number of centers included (stronger association with reduced mortality among single-center studies), and region. The ES was influenced by the type of intervention. The strongest association with reduced mortality was shown by the studies that implemented both educational and process change programs (k = 22).

[Figure omitted. See PDF.]

Fig 7. Forest plot showing individual and overall ES of studies that evaluated changes in mortality following the implementation of the performance improvement program (k = 48).

The size of the boxes is inversely proportional to the size of the result study variance, so that more precise studies have larger boxes. The ES is expressed as odds ratio (OR) and the correspondent 95% confidence interval (CI). An OR below 1.00 (left side of the plot) indicates an association between the intervention and decreased mortality. ES = effect size; CI = confidence interval; Sig. = p value.

https://doi.org/10.1371/journal.pone.0125827.g007

[Figure omitted. See PDF.]

Table 4. Moderator analysis of studies that evaluated changes in mortality following the performance improvement program (k = 48).

https://doi.org/10.1371/journal.pone.0125827.t004

Sensitivity analysis

In meta-regression analyses, study quality (as indicated by the NOS score) did not show any significant impact on the observed ESs (p = 0.809 for compliance with 6-hour bundle; p = 0.397 for compliance with 24-hour bundle; p = 0.705 for mortality). Sensitivity analyses for primary and secondary outcomes were performed by excluding the studies that did not control for confounders (severity of illness and age) and had an NOS score <6 (1st quartile). The results of these analyses did not show a significant impact on the final ESs: OR = 3.71 [1.90–7.28] (I2 = 86.71%, p<0.001) for compliance with 6-hour bundle (k = 8); OR = 1.83 [1.01–3.30] (I2 = 66.14%, p = 0.019) for compliance to 24-hour bundle (k = 5); OR = 0.65 [0.53–0.80] (I2 = 79.27%, p<0.001) for mortality (k = 16). The overall ES for mortality did not change when the analysis was restricted to studies for which the adjusted outcome data were available (OR = 0.67 [0.56–0.80], I2 = 90.77%, p<0.001, k = 12).

Discussion

The present systematic review and meta-analysis of 50 observational studies showed that the implementation of performance improvement programs increases compliance with the SSC guideline-based sepsis bundles and seems to be associated with decreased mortality in patients with sepsis, severe sepsis or septic shock.

Performance improvement initiatives varied from pure educational programs (including lectures, meetings, bedside training and educational materials) to interventions specifically aimed at inducing a variation in standard sepsis care (predefined order sets, clinical decision support and/or screening tools, activation of sepsis teams). Our analysis outlined the importance of medical/nurse staff education on sepsis pathogenesis, diagnosis and treatment. Education alone was able to improve compliance with the complete resuscitation and management bundles and was associated with a reduction in mortality. Pure process change programs were only able to improve compliance with the resuscitation bundle, but were still associated with a significant and consistent reduction in mortality. However, the largest increase in adherence to 6-hour and 24-hour bundles was induced by interventions including both an educational program and process changes, which were also associated with the greatest survival benefit. Therapies delivered early in the first 6 hours may have been the main determinant of survival in studies implementing process change interventions. A general increase in the attention to sepsis care may also have played a key role, rather than the implementation of the bundles per se.

In a prospective study by Levy et al [64], compliance to the complete sepsis resuscitation bundle was higher in the USA than in Europe; the unadjusted mortality was lower in the USA, but this difference disappeared after severity adjustment. In our analysis, the performance improvement programs induced a more than 8-fold increase in compliance with the complete 6-hour bundle in North-America. In Europe, compliance with 6-hour bundle increased by about 2 times. A higher percentage of North-American studies were performed in the ED (39% versus 23% in Europe), while in Europe the majority of studies were conducted in the ICU (61% versus 32% in North-America). It is reasonable that sepsis management in the ED mainly focus on earlier interventions. The interventions were associated with decreased mortality in North-America and non-significant survival benefit in Europe. Levy et al [64] showed that in the USA the majority of patients were admitted to the ICU from the ED, while in Europe 51.5% was admitted from the wards. Taken together, these data would suggest that an early more aggressive approach plays a key role in improving survival. Delay in sepsis diagnosis and/or admission to the ICU may be the major responsible for worse outcome, beyond the adherence to specific guideline-based targets.

In a retrospective study by Kang et al [65], care from experienced nurses (≥3 years of clinical experience), senior residents or board-certified emergency physicians was associated with higher compliance with the resuscitation bundle. We could not specifically evaluate the influence of healthcare staff experience on the effect of the quality improvement interventions. Interestingly however, the most consistent improvement in the adherence to the complete bundles was found among studies that reported the lowest initial compliance. In these cases, the performance improvement programs tended to be more strongly associated with a reduction in mortality. Efforts should be focused on the clinical settings where the adherence to SSC guidelines recommendations is poor, where quality improvement initiatives may be particularly beneficial in improving sepsis standard of care and patient outcome.

The implementation of performance improvement programs was more strongly associated with increased compliance with the resuscitation bundle when more severe patients were considered (severe sepsis and/or septic shock, higher severity score). Studies including only patients with septic shock showed the greatest survival benefit. It is reasonable that baseline compliance to the complete resuscitation bundle was lower for more severe patients, where physicians used to give priority to certain interventions over others. Our results would suggest that implementing a protocolized sepsis care may especially favor the prompt delivery of all recommended interventions in patients with higher risk of death.

In all the included studies sepsis treatment was based on 2004 or 2008 SSC-guidelines. In the updated 2012 SSC-guidelines [3], the bundles were revised: the resuscitation bundle was broken into two parts (3-hour and 6-hour), the management bundle was dropped. The guideline recommendations have undergone numerous changes as new more robust data emerged regarding type of fluids of choice, glucose control, corticosteroid administration [3]. Recombinant activated protein C was withdrawn from the market as it showed no consistent survival benefit [66]. Although this may limit the direct applicability of our results to the current clinical practice, it does not question the efficacy of performance improvement initiatives in promoting the best quality of care and increasing the clinicians’ awareness towards sepsis disease. In the present study, we showed that performance improvement initiatives increased the number of patients in whom the CVP and ScvO2 targets were achieved. Two recent multi-center RCTs [67, 68] failed to demonstrate a survival benefit of an early-goal directed therapy protocol based on CVP and ScvO2 measurement; protocolized care did not show any superiority over usual care, thus questioning the real advantages of SSC sepsis bundles. We did not evaluate the association between compliance with combined/individual bundle targets and mortality. Our data suggest a decrease in mortality following the implementation of performance improvement programs but cannot support any causality with the increased adherence to SSC guidelines. Observational studies are less methodologically rigorous than RCTs and prone to numerous biases. Secular trends may have been responsible for the observed decrease in mortality irrespectively of the interventions implemented. Of note, time-series analyses that are at lower risk for confounding by secular trends showed less extreme ES as compared to historically controlled or before-after investigations. Despite the failure of numerous RCTs in the past decade, recent reports suggest that the outcome of septic patients is improving over time [69, 70]. Judicious use of the SSC recommendations in the real clinical practice, in respect of the individual patient characteristics, may partly explain the discrepancy between these reports and data from RCTs. The worldwide implementation of performance improvement programs may have contributed substantially in reducing mortality from sepsis in the past decade, by increasing the attention of clinicians to sepsis care. These programs may be regarded as a tool for the divulgation of new robust evidences and their prompt implementation in the clinical practice. Levy et al. [38] reported a 7% decline in the risk of mortality for every additional quarter a site participates in the SSC program. We did not evaluate the relationship between time of participation in a performance improvement program and the compliance with SSC bundles achieved, nor did we consider the impact of time elapsed after the completion of the program, since only the final post-intervention phase was considered when compliance had been reported for different time periods. Nonetheless, it is worth noting that time-series analyses generally showed a persistent monotonic increase in compliance over time after the implementation of the interventions [8, 27, 38, 43, 46, 48, 54].

Our analysis has several limitations. Firstly, all the included studies were observational investigations and cannot thus support any causality between the quality improvement programs, increase in bundle compliance and reduction in mortality. Secondly, changes in adherence to SSC-guidelines or outcome over time may have occurred independently of the program implemented and could have influenced the results. Similarly, differences in disease severity or other confounders between the intervention and control groups could have biased the analysis. Nonetheless, whenever possible, we extracted adjusted data for the calculation of the ES for mortality (available for 12 studies out of 48); risk of bias and study quality were assessed and their influence on the results was explored by sensitivity analyses. Thirdly, we found high inconsistency between study findings, which may limit the validity of our data synthesis. We performed subgroup analyses in order to evaluate possible sources of heterogeneity, but cannot exclude the influence of unmeasured confounding factors. Of note, we could not evaluate the potential impact of any rewards or punishments delivered to healthcare providers based on the achieved compliance with the guidelines, as these were not generally reported in the studies. Fourth, although we performed an extensive review of the main electronic databases, we cannot be sure to have included all relevant studies. We did not consider studies that evaluated quality improvement initiatives implemented before the first publication of the SSC guidelines. In addition, it was not possible to take into account changes of the bundles occurring over time. Lastly, the possible publication bias may have influenced our results: the asymmetry of most of the funnel plots indicates that smaller studies tended to overestimate the association analyzed. We did not include unpublished studies, dissertations, or abstracts from conference proceedings. This may have contributed to the publication bias. However, we decided to consider only published materials in order to ensure that only higher quality, peer-reviewed studies were included in the meta-analysis. Similarly, the study selection was limited to English language studies because of cost reasons. This may have introduced an additional bias.

Clinical implications

A widespread activation of educational programs that could increase the clinician’s awareness of the sepsis burden is desirable. These initiatives should not be restricted to the ICU setting but could be extended to the ED and the wards. These may include conference lectures, bedside teaching and simulation training focusing on the definition of sepsis syndrome and providing indications for an early diagnosis and prompt treatment. These programs should also be regarded as a tool for the divulgation of new evidences. Regular auditing on all patients treated in the ICU for severe sepsis or septic shock and feedback on the implementation of sepsis care are feasible initiatives that could prove useful in increasing the attention to the management of these patients. Additional simple tools, such as posters, bedside or pocket reminder cards, predefined order sets, flow charts and checklists may facilitate the early recognition of sepsis and implementation of the recommended therapies. A further step could be the introduction of electronic clinical decision support tools (e. g. sepsis alert systems) that may be developed in the future. Finally, it could be useful to have a “Sepsis Team” available in the hospital, to be activated as soon as possible in cases of suspected severe sepsis or septic shock, which may help to define the best diagnostic/therapeutic strategy.

Conclusions

The present systematic review and meta-analysis on 50 observational studies showed that performance improvement programs were associated with increased compliance with sepsis resuscitation and management bundles and a reduction in mortality in patients with sepsis, severe sepsis or septic shock. Interventions including both an educational program and process changes were associated with the largest increase in compliance with SSC bundles and the greatest survival benefit. Studies including more severe patients showed a greater improvement in compliance with 6-hour bundle. Quality improvement initiatives represent a valuable tool to promote the best quality of care for septic patients.

Key Messages

* Performance improvement programs were associated with an increase in compliance with the SSC-guideline based sepsis resuscitation and management bundles and in the adherence to individual bundle targets

* The implementation of quality improvement initiatives was associated with decreased mortality in patients with sepsis, severe sepsis or septic shock

Supporting Information

[Figure omitted. See PDF.]

S1 Fig. Funnel plot and trim-and-fill analysis of studies that evaluated compliance with the complete 6-hour sepsis bundle.

https://doi.org/10.1371/journal.pone.0125827.s001

(PDF)

S2 Fig. Funnel plot and trim-and-fill analysis of studies that evaluated compliance to individual 6-hour bundle targets.

(A) Measure lactate; (B) Blood cultures; (C) Antibiotics; (D) Fluid resuscitation; (E) Measure central venous pressure; (F) Central venous pressure above 8 mmHg; (G) Measure SvO2; (H) SvO2 above 70%; (I) Mean arterial pressure above 65 mmHg; (L) Use of vasopressors.

https://doi.org/10.1371/journal.pone.0125827.s002

(PDF)

S3 Fig. Funnel plot and trim-and-fill analysis of studies that evaluated compliance to the complete 24-hour sepsis bundle.

https://doi.org/10.1371/journal.pone.0125827.s003

(PDF)

S4 Fig. Funnel plot and trim-and-fill analysis of studies evaluating compliance with individual 24-hour bundle targets.

(A) Lung protective ventilation; (B) Low-dose steroids; (C) Drotrecogin alfa (activated); (D) Glucose control.

https://doi.org/10.1371/journal.pone.0125827.s004

(PDF)

S5 Fig. Funnel plot and trim-and-fill analysis of studies that evaluated mortality.

https://doi.org/10.1371/journal.pone.0125827.s005

(PDF)

S1 Table. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2009 Check-list.

https://doi.org/10.1371/journal.pone.0125827.s006

(PDF)

S2 Table. Study characteristics.

https://doi.org/10.1371/journal.pone.0125827.s007

(PDF)

S3 Table. Outcomes measured in the analyzed studies.

https://doi.org/10.1371/journal.pone.0125827.s008

(PDF)

S4 Table. Risk of bias and study quality assessment.

https://doi.org/10.1371/journal.pone.0125827.s009

(PDF)

S1 Text. Search strategy used for PubMed (Medline) and adapted for the other electronic databases.

https://doi.org/10.1371/journal.pone.0125827.s010

(PDF)

Acknowledgments

We thank the authors who kindly answered to our queries.

Author Contributions

Conceived and designed the experiments: ED AD PP SB MG. Performed the experiments: ED GS LR EA. Analyzed the data: ED GS LR EA MG. Contributed reagents/materials/analysis tools: AD PP. Wrote the paper: ED AD GS LR EA PP SB MG.

Citation: Damiani E, Donati A, Serafini G, Rinaldi L, Adrario E, Pelaia P, et al. (2015) Effect of Performance Improvement Programs on Compliance with Sepsis Bundles and Mortality: A Systematic Review and Meta-Analysis of Observational Studies. PLoS ONE 10(5): e0125827. https://doi.org/10.1371/journal.pone.0125827

References

1. Mayr FB, Yende S, Angus DC. Epidemiology of severe sepsis. Virulence. 2014; 5: 4–11. pmid:24335434

2. Townsend SR, Schorr C, Levy MM, Dellinger RP.Reducing mortality in severe sepsis: The Surviving Sepsis Campaign. Clin Chest Med. 2008; 29: 721–733. pmid:18954706

3. Dellinger RP, Levy MM, Rhodes A, Annane D, Gerlach H, Opal SM, et al. Surviving Sepsis Campaign: International Guidelines for the Management of Severe Sepsis and Septic Shock: 2012. Crit Care Med. 2013; 41: 580–637. pmid:23353941

4. Levy MM, Pronovost PJ, Dellinger RP, Townsend S, Resar RK, Clemmer TP, et al. Sepsis change bundles: Converting guidelines into meaningful change in behavior and clinical outcome. Crit Care Med. 2004; 32: S595–S597. pmid:15542969

5. Gao F, Melody T, Daniels DF, Giles S, Fox S. The impact of compliance with 6-hour and 24-hour sepsis bundles on hospital-mortality in patients with severe sepsis: a prospective observational study. Crit Care. 2005; 9: R764–R770. pmid:16356225

6. Van Zanten ARH, Brinkman S, Arbous MS, Abu-Hanna A, Levy MM, de Keizer NF; Netherlands Patient Safety Agency Sepsis Expert Group. Guideline bundles adherence and mortality in severe sepsis and septic shock. Crit Care Med. 2014; 42: 1890–1898. pmid:24670937

7. Djurkovic S, Baracaldo JC, Guerra JA, Sartorius J, Haupt MT. A survey of clinicians addressing the approach to the management of severe sepsis and septic shock in the United States. J Crit Care. 2010; 25: 658.e1–658.e6.

8. Levy MM, Dellinger RP, Townsend SR, Linde-Zwirble WT, Marshall JC, Bion J, et al. The Surviving Sepsis Campaign: results of an international guideline-based performance improvement program targeting severe sepsis. Intensive Care Med. 2010; 36: 222–231. pmid:20069275

9. Giuliano KK, Lecardo M, Staul LA. Impact of Protocol Watch on compliance with the Surviving Sepsis Campaign. Am J Crit Care. 2011; 20: 330–331.

10. Berg GM, Vasquez DG, Hale LDS, Nyberg SM, Moran DA. Evaluation of process variations in noncompliance in implementation of evidence-based sepsis care. J Healthc Qual. 2013; 35: 60–69. pmid:22093135

11. Levy MM, Fink MP, Marshall JC, Abraham E, Angus D, Cook D, et al. 2001 SCCM/ESICM/ACCP/ATS/SIS sepsis definitions conference. Crit Care Med. 2003; 31: 1250–1256. pmid:12682500

12. Dellinger RP, Carlet JM, Masur H, Gerlach H, Calandra T, Cohen J, et al. Surviving Sepsis Campaign guidelines for management of severe sepsis and septic shock. Intensive Care Med. 2004; 30: 536–555. pmid:14997291

13. Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health. 1998; 52:377–84. pmid:9764259

14. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to Meta-Analysis. Chichester, UK: John Wiley and Sons, Ltd; 2009.

15. Lipsey MW, Wilson DB, eds. Practical Meta-Analysis. Vol 49. Thousand Oaks, CA: Sage; 2001.

16. Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analysis. BMJ. 2003; 327: 557–560. pmid:12958120

17. Sterne JAC, Egger M. Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis. J Clin Epidemiol. 2001; 54: 1046–1055. pmid:11576817

18. Bond CM, Djogovic D, Villa-Roel C, Bullard MJ, Meurer DP, Rowe BH. Pilot study comparing sepsis management with and without electronic clinical practice guidelines in an academic emergency department. J Emerg Med. 2013; 44: 698–708. pmid:23137959

19. Cannon CM, Holthaus CV, Zubrow MT, Posa P, Gunaga S, Kella V, et al. The GENESIS Project (GENeralized Early Sepsis Intervention Strategies): a multicenter quality improvement collaborative. J Intensive Care Med. 2012;

20. Capuzzo M, Rambaldi M, Pinelli G, Campesato M, Pigna A, Zanello M, et al. Hospital staff education on severe sepsis/septic shock and hospital mortality: an original hypothesis. BMC Anesthesiology. 2012; 12: 28. pmid:23167611

21. Castellanos-Ortega A, Suberviola B, Garcia-Astudillo LA, Holanda MS, Ortiz F, Llorca J, et al. Impact of the Surviving Sepsis Campaign protocols on hospital length of stay and mortality in septic shock patients: Results of a three-year follow-up quasi-experimental study. Crit Care Med. 2010; 38: 1036–1043. pmid:20154597

22. Chen YC, Chang SC, Pu C, Tang GJ. The impact of nationwide education program on clinical practice in sepsis care and mortality of severe sepsis: a population-based study in Taiwan. PLoS ONE. 2013; 8: e77414. pmid:24124616

23. De Miguel-Yanes JM, Munoz-Gonzalez J, Andueza-Lillo JA, Moyano-Villaseca B, Gonzalez-Ramallo VJ, Bustamante-Fermosel A. Implementation of a bundle of actions to improve adherence to the Surviving Sepsis Campaign guidelines at the ED. Am J Emerg Med. 2009; 27: 668–674. pmid:19751623

24. El Solh AA, Akinnusi ME, Alsawalha LN, Pineda LA. Outcome in septic shock in older adults after implementation of the sepsis “bundle”. JAGS. 2008; 56: 272–278.

25. Ferrer R, Artigas A, Levy MM, Blanco J, Gonzalez-Dias G, Garnacho-Montero J, et al. Improvement in process of care and outcome after a multicenter severe sepsis educational program in Spain. JAMA. 2008; 299: 2294–2303. pmid:18492971

26. Francis M, Rich T, Williamson T, Peterson D. Effect of an emergency department sepsis protocol on time to antibiotics in severe sepsis. CJEM. 2010; 12: 303–310. pmid:20650022

27. Girardis M, Rinaldi L, Donno L, Marietta M, Codeluppi M, Marchegiano P, et al. Effects on management and outcome of severe sepsis and septic shock patients admitted to the intensive care unit after implementation of a sepsis program: a pilot study. Crit Care. 2009; 13: R143. pmid:19728879

28. Gurnani PK, Patel GP, Crank CW, Vais D, Lateef O, Akimov S, et al. Impact of the implementation of a sepsis protocol for the management of fluid-refractory septic shock: a single-center, before-and-after study. Clinical Therapeutics. 2010; 32: 1285–1293. pmid:20678676

29. Heppner HJ, Singler K, Kwetkat A, Popp S, Esslinger AS, Bahrmann P, et al. Do clinical guidelines improve management of sepsis in critically ill elderly patients? A before-and-after study of the implementation of a sepsis protocol. Wien Klin Wochenschr. 2012; 124: 692–698. pmid:22948390

30. Hoo WES, Muehlberg K, Ferraro RG, Jumaoas MC. Successes and lessons learned implementing the sepsis bundle. J Healthc Qual. 2009; 31: 9–15. pmid:19753803

31. Jacob ST, Banura P, Baeten JM, Moore CC, Meya D, Nakiyingi L, et al. The impact of early monitored management on survival in hospitalized adult Ugandan patients with severe sepsis: a prospective intervention study. Crit Care Med. 2012; 40: 2050–2058. pmid:22564958

32. Jeon K, Shin TG, Sim MS, Suh GY, Lim SY, Song HG, et al. Improvements in compliance with resuscitation bundles and achievement of end points after an educational program on the management of severe sepsis and septic shock. SHOCK. 2012; 37: 463–467. pmid:22301605

33. Jones AE, Troyer JL, Kline JA. Cost-effectiveness of an emergency department based early sepsis resuscitation protocol. Crit Care Med. 2011; 39: 1306–1312. pmid:21336115

34. Kuan WS, Mahadevan M, Tan JH, Guo J, Ibrahim I. Feasibility of introduction and implementation of the Surviving Sepsis Campaign bundle in a Singapore Emergency Department. Eur J Emerg Med. 2013; 20: 344–349. pmid:23211272

35. Laguna-Pérez A, Chilet-Rosell E, Delgado Lacosta M, Alvarez-Dardet C, Uris Selles J, Munoz-Mendoza CL. Clinical pathway intervention compliance and effectiveness when used in the treatment of patients with severe sepsis and septic shock at an Intensive Care Unit in Spain. Rev Lat Am Enfermagem. 2012; 20: 635–643. pmid:22990147

36. LaRosa JA, Ahmad N, Feinberg M, Shah M, DiBrienza R, Studer S. The use of an early alert system to improve compliance with sepsis bundles and to assess impact on mortality. Crit Care Res Pract. 2012; 2012: 980369. pmid:22461981

37. Lefrant JY, Muller L, Raillard A, Jung B, Beaudroit L, Favier L, et al. Reduction of the severe sepsis or septic shock associated mortality by reinforcement of the recommendations bundle: a multicenter study. Ann Fr Anesth Reanim. 2010; 29: 621–628. pmid:20634026

38. Levy MM, Rhodes A, Phillips GS, Townsend SR, Schorr CA, Beale R, et al. Surviving Sepsis Campaign: Association between performance metrics and outcomes in a 7.5-year study. Crit Care Med. 2014; 43: 3–12.

39. MacRedmond R, Hollohan K, Stenstrom R, Nebre R, Jaswal D, Dodek P. Introduction of a comprehensive management protocol for severe sepsis is associated with sustained improvements in timeliness of care and survival. Qual Saf Health Care. 2010; 19: e46. pmid:20671074

40. McKinley BA, Moore LJ, Sucher JF, Todd SR, Turner KL, Valdivia A, et al. Computer protocol facilitates evidence-based Care of sepsis in the surgical intensive care unit. J Trauma. 2011; 70: 1153–1167. pmid:21610430

41. Memon JI, Rehmani RS, Alaithan AM, El Gammal A, Lone TM, Ghorab K, et al. Impact of 6-hour sepsis resuscitation bundle compliance on hospital mortality in a Saudi hospital. Crit Care Res Pract. 2012; 2012: 273268. pmid:23082248

42. Micek ST, Roubinian N, Heuring T, Bode M, Williams J, Harrison C, et al. Before—after study of a standardized hospital order set for the management of septic shock. Crit Care Med. 2006; 34: 2707–2713. pmid:16943733

43. Miller RR 3rd, Dong L, Nelson NC, Brown SM, Kuttler KG, Probst DR, et al. Multicenter implementation of a severe sepsis and septic shock treatment bundle. Am J Respir Crit Care Med. 2013; 188: 77–82. pmid:23631750

44. Moore LJ, Jones SL, Kreiner LA, McKinley B, Sucher JF, Todd SR, et al. Validation of a screening tool for the early identification of sepsis. J Trauma. 2009; 66: 1539–1547. pmid:19509612

45. Na S, Kuan WS, Mahadevan M, Li CH, Shrikhande P, Ray S, et al. Implementation of early goal-directed therapy and the surviving sepsis campaign resuscitation bundle in Asia. Int J Qual Health Care. 2012; 24: 452–462. pmid:22899698

46. Nguyen HB, Corbett SW, Steele R, Banta J, Clark RT, Hayes SR, et al. Implementation of a bundle of quality indicators for the early management of severe sepsis and septic shock is associated with decreased mortality. Crit Care Med. 2007; 35: 1105–1112. pmid:17334251

47. Nguyen HM, Schiavoni A, Scott KD, Tanios MA. Implementation of sepsis management guideline in a community-based teaching hospital—can education be potentially beneficial for septic patients? Int J Clin Pract. 2012; 66: 705–710. pmid:22698423

48. Noritomi DT, Ranzani OT, Monteiro MB, Ferreira EM, Santos SR, Leibel F, et al. Implementation of a multifaceted sepsis education program in an emerging country setting: clinical outcomes and cost-effectiveness in a long-term follow-up study. Intensive Care Med. 2014; 40: 182–191. pmid:24146003

49. Palleschi MT, Sirianni S, O’Connor N, Dunn D, Hasenau SM. An interprofessional process to improve early identification and treatment for sepsis. J Healthc Qualt. 2014; 36: 23–31. pmid:23534854

50. Patocka C, Turner J, Xue X, Segal E. Evaluation of an emergency department triage screening tool for suspected severe sepsis and septic shock. J Healthc Qual. 2014; 36: 52–61. pmid:24372995

51. Plambech MZ, Lurie AI, Ipsen HL. Initial, successful implementation of sepsis guidelines in an emergency department. Dan Med J. 2012; 59: A4545. pmid:23290285

52. Sawyer AM, Deal EN, Labelle AJ, Witt C, Thiel SW, Heard K, et al. Implementation of a real-time computerized sepsis alert in nonintensive care unit patients. Crit Care Med. 2011; 39: 469–473. pmid:21169824

53. Schramm GE, Kashyap R, Mullon JJ, Gajic O, Afessa B. Septic shock: a multidisciplinary response team and weekly feedback to clinicians improve the process of care and mortality. Crit Care Med. 2011; 39: 252–258. pmid:21057312

54. Seoane L, Winterbottom F, Nash T, Behrhorst J, Chacko E, Shum L, et al. Using quality improvement principles to improve the care of patients with severe sepsis and septic shock. Ochsner J. 2013; 13: 359–366. pmid:24052765

55. Shapiro NI, Howell MD, Talmor D, Lahey D, Ngo L, Buras J, et al. Implementation and outcomes of the Multiple Urgent Sepsis Therapy (MUST) protocol. Crit Care Med. 2006; 34: 1025–1032. pmid:16484890

56. Shiramizo SC, Marra AR, Durao MS, Paes AT, Edmond MB, Pavao dos Santos OF. Decreasing mortality in severe sepsis and septic shock patients by implementing a sepsis bundle in a hospital setting. PLoS ONE. 2011; 6: e26790. pmid:22073193

57. Silverman LZ, Hoesel M, Desai A, Posa P, Purtill MA, Brandt MM. It takes an intensivist. Am J Surg. 2011; 201: 320–323. pmid:21367371

58. Sweet DD, Jaswal D, Fu W, Bouchard M, Sivapalan P, Rachel J, et al. Effect of an emergency department sepsis protocol on the care of septic patients admitted to the intensive care unit. CJEM. 2010; 12: 414–20. pmid:20880435

59. Thiel SW, Asghar MF, Micek ST, Reichlay RM, Doherty JA, Kollef MH. Hospital-wide impact of a standardized order set for the management of bacteremic severe sepsis. Crit Care Med. 2009; 37: 819–824. pmid:19237883

60. Tromp M, Hulscher M, Bleeker-Rovers CP, Peters L, van den Berg DTNA, Borm GF, et al. The role of nurses in the recognition and treatment of patients with sepsis in the emergency department: a prospective before-and-after intervention study. Int J Nurs Stud. 2010; 47: 1464–1473. pmid:20494356

61. Vallée F, Fourcade O, Marty P, Sanchez P, Samii K, Genestal M. The hemodynamic “target”: a visual tool of goal-directed therapy for septic patients. Clinics. 2007; 62: 447–54. pmid:17823708

62. Wang Z, Xiong Y, Schorr C, Dellinger RP. Impact of sepsis bundle strategy on outcomes of patients suffering from severe sepsis and septic shock in China. J Emerg Med. 2013; 44: 735–741. pmid:23332802

63. Westphal GA, Koening A, Caldeira Filho M, Feijò J, de Oliveira LT, Nunes F, et al. Reduced mortality after the implementation of a protocol for the early detection of severe sepsis. J Crit Care. 2011; 26: 76–81. pmid:21036531

64. Levy MM, Artigas A, Phillips GS, Rhodes A, Beale R, Osborn T, et al. Outcomes of the Surviving Sepsis Campaign in intensive care units in the USA and Europe: a prospective cohort study. Lancet Infect Dis. 2012; 12: 919–924. pmid:23103175

65. Kang MJ, Shin TG, Jo IJ, Jeon K, Suh GY, Sim MS, et al. Factors influencing compliance with early resuscitation bundle in the management of severe sepsis and septic shock. Shock. 2012; 38: 474–9. pmid:23042195

66. Ranieri VM, Thompson BT, Barie PS, Dhainaut JF, Douglas IS, Finfer S, et al. Drotrecogin alfa (activated) in adults with septic shock. N Engl J Med. 2012; 366: 2055–2064. pmid:22616830

67. ProCESS Investigators, Yealy DM, Kellum JA, Huang DT, Barnato AE, Weissfeld LA, et al. A randomized trial of protocol-based care for early septic shock. N Engl J Med. 2014; 370: 1683–1693. pmid:24635773

68. The ARISE Investigators and the ANZICS Clinical Trials Group. Goal-directed resuscitation for patients with early septic shock. N Engl J Med. 2014; 371: 1496–506. pmid:25272316

69. Stevenson EK, Rubenstein AR, Radin GT, Wiener RS, Walkey AJ. Two decades of mortality trends among patients with severe sepsis: a comparative meta-analysis. Crit Care Med. 2014; 42: 2–7.

70. Kaukonen KM, Bailey M, Suzuki S, Pilcher D, Bellomo R. Mortality related to severe sepsis and septic shock among critically ill patients in Australia and New Zealand, 2000–2012. JAMA. 2014; 311: 1308–1316. pmid:24638143