Critical Appraisal of the Quality of Clinical Practice Guidelines for Stress Ulcer Prophylaxis

Abstract

Background and Objective

Inappropriate use of stress ulcer prophylaxis (SUP) is common in many hospitals. High-quality clinical practice guidelines (CPGs) produce better patient outcomes and promote cost-effective clinical care. Thus, the objective of this study was to evaluate the quality of CPGs for SUP.

Methods

A search was conducted for SUP CPGs using PubMed, Embase, China National Knowledge Infrastructure (CNKI), Chinese Biomedical Literature database (CBM), guideline websites and Google (until March 1, 2015). The quality of CPGs was independently assessed by two assessors using the Appraisal of Guidelines for Research & Evaluation II (AGREE II) instrument, and the specific recommendations in the CPGs were summarized and evaluated.

Results

A total of 7 CPGs for SUP were included. The highest median scores were in the clarity of presentation domain (89%), and the lowest median scores were in the editorial independence domain (0%). The rigor of development, stakeholder involvement, and applicability domains all scored below 40%. The specific recommendations for SUP varied, and the recommendations were inconsistent with the supporting evidence.

Conclusions

The overall quality of CPGs for SUP was relatively low, and no specific SUP CPG can be recommended. Not only should the AGREE II instrument be used to determine the quality of CPGs, but also the recommendations should be appraised based on supporting evidence, which would contribute to the development of high-quality CPGs.

About the Authors:

Zhi-Kang Ye

Affiliation: Department of Pharmacy, Beijing Chao-Yang Hospital, Capital Medical University, Beijing, China

Ying Liu

Affiliation: Department of Pharmacy, Beijing Chao-Yang Hospital, Capital Medical University, Beijing, China

Xiang-Li Cui

Affiliation: Department of Pharmacy, Beijing Chao-Yang Hospital, Capital Medical University, Beijing, China

Li-Hong Liu

* E-mail: [email protected]

Affiliation: Department of Pharmacy, Beijing Chao-Yang Hospital, Capital Medical University, Beijing, China

Introduction

Stress ulcer prophylaxis (SUP) is commonly used to decrease gastrointestinal bleeding in both critically and non-critically ill patients [1–3]. To help clinicians appropriately use SUP, several organizations have developed clinical practice guidelines (CPGs) for SUP.

Many studies suggested that the inappropriate use of SUP, based on these guidelines, is common in hospital [4–12]. CPGs are “statements that include recommendations intended to optimize patient care. They are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care option” [13]. High-quality CPGs provide effective recommendations, produce better patient outcomes and promote cost-effective clinical care [14]. Both high AGREE II domain scores and high-quality recommendations are needed to produce excellent CPGs.

To the best of our knowledge, there has been no evaluation of SUP guidelines. Thus, we conducted this study to evaluate the quality of a group of international CPGs for SUP and to help develop, update or improve SUP guidelines, and to help clinicians reduce the inappropriate use of SUP.

Methods

Literature Search

We searched PubMed, Embase, China National Knowledge Infrastructure (CNKI) and Chinese Biomedical Literature Database (CBM) for SUP guidelines (until March 1, 2015). The text words and Medical Subject Headings (MeSH) terms were as follows: (guideline or guidelines or consensus) and “stress ulcer prophylaxis”. The CPGs search was conducted on major guideline websites, including the National Guideline Clearinghouse, National Institute for Health and Clinical Excellence, Scottish Intercollegiate Guidelines Network, Guidelines International Network and China Guideline Clearinghouse. The search term was “stress ulcer prophylaxis”. Google was also searched using the terms “guideline” and “stress ulcer prophylaxis”, and we reviewed the first 100 results. No restriction on language was applied.

Selection of Guidelines

CPGs for SUP included those that provided clinical recommendations and strategies to assist health care practitioners in making decisions and those that were endorsed by medical specialty associations or relevant professional societies. We excluded guidelines that were not original, were duplications, or were explanations of CPGs.

Quality Evaluation

Two assessors, one with experience in developing and evaluating guidelines (Z.K.Y) and another assessor (Y.L), used the online training tools recommended by the AGREE collaboration before conducting appraisals. They independently evaluated the included guidelines using the AGREE II instrument [15], which consists of 23 items organized into six domains. Each item was rated on a seven-point scale from 1 (strongly disagree) to 7 (strongly agree). To resolve discrepancies between the two assessors, we referred to the method used in a previous study: if the scores assigned by the assessors differed by one point, the lower score was used; if they differed by two points, they were averaged; and if they varied by three points or more, a consensus was reached after a discussion [16]. A scaled domain score was calculated as follow: (obtained score—minimum possible score)/(maximum possible score—minimum possible score). The overall recommendation about the included CPGs was based on the scores for each item and the quality of the CPGs’ recommendations.

After determining the quality of SUP guidelines using the AGREE II instrument, the specific recommendations made in the included guidelines were summarized and evaluated, including indications for SUP, agents for SUP and duration of prophylaxis.

Results

Study Selection

A total of 600 records were identified, of which, 185 were duplications, 385 were either not relevant to SUP or not guidelines after screening titles and abstracts, and the remaining 30 records were retrieved for full text. Finally, 7 CPGs (ASHP [17], EAST [18], ORMC [19], VUMC [20], DASAIM [21], NMJC [22], KAUH [23]) were included (see Fig 1). The demographic characteristics for included guidelines are presented in Table 1. All CPGs were published between 1999 and 2013. The ASHP and DASAIM CPGs were found in medical databases [17,21], the EAST CPG was found on a guideline website [18], and the remaining CPGs were found through Google searches. The DASAIM CPG rated the quality of evidence and graded the strength of recommendations using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach, and the DASAIM CPG was developed specifically for intensive care unit patients [21].

[Figure omitted. See PDF.]

Fig 1. Flowchart for systematically searching and selecting the guidelines.

https://doi.org/10.1371/journal.pone.0155020.g001

[Figure omitted. See PDF.]

Table 1. Characteristics of clinical practice guidelines.

ASHP: American Society of Health-System Pharmacists; EAST: Eastern Association for the Surgery of Trauma; ORMC: Orlando Regional Medical Center; VUMC: Vanderbilt University Medical Center; DASAIM: Danish Society of Anesthesiology and Intensive Care Medicine; DSIT: Danish Society of Intensive Care Medicine; EB: Editorial Board; NMJC: National Medical Journal of China; DIC: Drug Information Center; KAUH: King Abdullah University Hospital.

https://doi.org/10.1371/journal.pone.0155020.t001

Scope and Purpose

The domain-standardized scores for SUP CPGs and overall recommendation were presented in Table 2. The median score for the scope and purpose domain was 67% (range 39–100%). Only the ORMC and NMJC CPGs scored below 60% in this domain [19,22]. Most guidelines clearly described their overall objectives, health questions and target populations.

[Figure omitted. See PDF.]

Table 2. AGREE II domain-standardized scores for CPGs on SUP.

https://doi.org/10.1371/journal.pone.0155020.t002

Stakeholder Involvement

The median score for the stakeholder involvement domain was 39% (range 17–61%). Only the AHSP CPG scored above 60% for this domain [17]. No guideline considered the views and preferences of the target population, and no guideline clearly described their members’ roles in the guideline development process. No methodology experts or health economists were included in the guideline development group.

Rigor of Development

The median score for the rigor of development domain was 27% (range 6–77%). The ASHP and DASAIM CPGs scored above 60% [17,21]. Only the AHSP and DASAIM CPGs clearly described systematic methods of searching for evidence [17,21]. Only the DASAIM CPG clearly described methods for formulating recommendations and for conducting external reviews [21]. Only the DASAIM CPG used the GRADE approach to developing guidelines and reporting the recommendations based on an underlying systematic review [21]. No guideline described their procedures for updating guidelines.

Clarity of Presentation

The median score for the clarity of presentation domain was 89% (range 67–94%). All guidelines scored above 60%. Most guidelines provided specific, unambiguous and easily identifiable recommendations.

Applicability

The median score for the applicability domain was 38% (21–54%). No guideline scored above 60%. No guideline systematically described the facilitators and barriers of its applications very well. Most guidelines did not sufficiently consider the costs of applying their recommendations, and no guideline involved a health economist in finding and analyzing cost information.

Editorial Independence

The median score for the editorial independence domain was 0% (0–33%). No guideline clearly provided funding information. Only the DASAIM CPG described the competing interests of the guideline development group members [21]. The AHSP CPG described the method by which potential competing interests were sought, but competing interests results were not provided [17]. Other guidelines did not provide information about competing interests.

Specific Recommendations

Indications for SUP.

In Table 3, indications for SUP was described. Six CPGs (ASHP, EAST, ORMC, VUMC, NMJC and KAUH) recommended mechanical ventilation, coagulopathy and traumatic brain injury as the indications for SUP. Five CPGs (ASHP, ORMC, VUMC, NMJC and KAUH) recommended that a history of gastrointestinal ulceration or bleeding within one year was an indication for SUP. Five CPGs (ASHP, EAST, ORMC, NMJC and KAUH) recommended that major burn injury was an indication for SUP. Four CPGs (EAST, ORMC, VUMC and NMJC) recommended that sepsis was an indication for SUP, while two CPGs (ASHP and KAUH) recommended that sepsis was a minor risk factor for SUP. Three CPGs (ASHP, EAST and NMJC) recommended that multi-trauma was an indication for SUP. Four CPGs (ASHP, EAST, VUMC and KAUH) recommended that receiving high-dose corticosteroids was a minor risk for SUP. The DASAIM CPG did not make any recommendation about indications for SUP because of insufficient evidence. When we evaluated the original studies supporting these recommendations, we found no high-quality study on indications for SUP. Thus, recommendations on indications for SUP were not very consistent across guidelines.

[Figure omitted. See PDF.]

Table 3. Specific recommendations from CPGs.

NR: not reported; GI: gastrointestinal.

https://doi.org/10.1371/journal.pone.0155020.t003

Agents for SUP.

Recommendations on agents for SUP were not consistent across these guidelines. The DASAIM CPG recommended using proton pump inhibitors (PPIs) rather than histamine 2 receptor antagonists (H₂RAs) for SUP, five CPGs (EAST, ORMC, VUMC, NMJC and KAUH) recommended using both PPIs and H₂RAs, and the ASHP CPG did not recommended using PPIs. Three CPGs (ASHP, NMJC and KAUH) recommended using antacids for SUP, while two CPGs (ASHP, KAUH) recommended using sucralfate for SUP. The EAST CPG recommended cytoprotective agents for SUP, and the NMJC CPG recommended mucosal protective agents.

The DASAIM CPG recommended using PPIs rather than H₂RAs for SUP based on one published systematic review, which suggesting that PPIs were more effective than H₂RAs at reducing clinically important upper gastrointestinal bleeding (RR = 0.36, 95%CI 0.19–0.68 P = 0.002). However, after excluding trials at high risk or unclear risk of bias, the results suggested that there was no significant difference between PPIs and H₂RAs (RR = 0.60, 95%CI 0.27–1.35 P = 0.21) [24]. Therefore, the recommendations of included CPGs were not consistent with the supporting evidence.

Duration of Prophylaxis.

Recommendations on duration of prophylaxis are moderately consistent. Five CPGs (ASHP, EAST, ORMC, VUMC and KAUH) recommended that prophylaxis should be discontinued when there was no risk factor for SUP, three CPGs (EAST, ORMC, VUMC) recommended that prophylaxis be discontinued when the patient can tolerate enteral feeding, and two CPGs (DASAIM and NMJC) did not provide recommendations on duration of prophylaxis. When we evaluated the original studies supporting the recommendations, we found no-high quality studies on duration of prophylaxis.

Overall Assessment

The DASAIM, ASHP and EAST CPGs were recommended with modifications [17,18,21], while the remaining CPGs were not recommended [19,20,22,23].

Discussion

This is the first study to systematically evaluate the quality of SUP guidelines. The overall quality of these CPGs was relatively low; the clarity of presentation domain showed the highest scores, while the editorial independence domain showed the lowest scores. Not only should the AGREE II instrument be used to determine the quality of CPGs, but also the recommendations should be appraised based on supporting evidence.

These guidelines had high clarity of presentation scores, indicating that this domain was more easily achieved than other domains. The editorial independence domain was poorly described. No guideline provided funding information, and only two guidelines described competing interests. Perhaps guideline developers do not like disclosing their funding information, or perhaps they do not realize the importance of conflict of interest disclosures and management. Studies have shown that financial conflicts of interest are prevalent among CPGs in a variety of clinical areas [25,26], and some evidence suggests that such financial conflicts of interest may affect guideline recommendations [27]. Therefore, guideline developers should strongly emphasize the editorial independence domain.

The rigor of development, stakeholder involvement and applicability domains all scored below 40%, and there were serious methodological flaws in these three domains. The median score for the rigor of development domain was 27%. Most guidelines described neither systematic methods for evidence searching nor methods for formulating recommendations very well. Only the DASAIM CPG indicated that the guideline was reviewed by external experts, and no guideline provided procedures for updating guidelines. The median score for the applicability domain was 38%. Most guidelines did not consider the potential resource implications of applying recommendations, nor did they pay proper attention to potential barriers to guideline implementation.

The median score for the stakeholder involvement domain was 39%. No guideline considered the views and preferences of target populations; however, the involvement of patients in decision making might promote patient guideline adherence and improve clinical outcomes [28]. Most guidelines did not provided clear information on guideline development group members, especially on the members’ roles in the guideline development group. No guideline involved methodological experts or health economists, which explains why all of these guidelines scored low in the rigor of development domain. The guideline manuals published by World Health Organization and the National Institute for Health and Clinical Excellence both recommended that methodologists be involved in guideline development [29,30]. Guideline developers should pay more attention to the composition of guideline development groups and incorporate patient preferences. The median score for the scope and purpose domain was 67%, and most guidelines described their overall objectives, specific health questions and target populations well.

The specific recommendations made in the included CPGs varied, probably due to the lack of high-quality studies on the indications for SUP and the duration of prophylaxis. Guideline developers failed to critically appraise the validity of the evidence, which led to the inconsistency between the recommendations and the supporting evidence. The DASAIM CPG rated the quality of evidence and graded the strength of recommendations using the GRADE approach, and recommendations were formulated based on a systematic review, thus scoring the highest in the rigor of development. However, high AGREE II domain scores do not imply that a guideline should be recommended. When we appraised the DASAIM CPG, we found that some recommendations were not consistent with the supporting evidence. The outcomes of trials with low risk of bias did not suggest that PPIs were better than H₂RAs. Guideline developers should critically appraise the validity of systematic reviews and other sources of evidence formulating recommendations.

All potentially relevant studies were retrieved by searching medical databases, five guideline websites and Google. This study also has limitation. The AGREE II instrument established an evaluation system for guideline development and reporting, but the appraisal of guideline recommendations is not stated.

In conclusion, the overall quality of CPGs for SUP was relatively low, no specific SUP CPG can be recommended to guide clinical practice. Not only should the AGREE II instrument be used to determine the quality of CPGs, but also the recommendations should be appraised based on supporting evidence, which would contribute to the development of high-quality SUP CPGs.

Supporting Information

[Figure omitted. See PDF.]

S1 Fig. PRISMA Flow Diagram.

https://doi.org/10.1371/journal.pone.0155020.s001

(DOC)

S1 Table. PRISMA Checklist.

https://doi.org/10.1371/journal.pone.0155020.s002

(DOC)

Acknowledgments

The authors thank Suo-Di Zhai for his expertise regarding guideline evaluation and Meng-Meng Zhang for her suggestions.

Author Contributions

Conceived and designed the experiments: ZKY LHL. Performed the experiments: ZKY YL. Analyzed the data: ZKY YL XLC. Contributed reagents/materials/analysis tools: ZKY YL XLC. Wrote the paper: ZKY. Revised the manuscript: ZKY YL XLC LHL. Approved the final version of the manuscript: ZKY YL XLC LHL.

Citation: Ye Z-K, Liu Y, Cui X-L, Liu L-H (2016) Critical Appraisal of the Quality of Clinical Practice Guidelines for Stress Ulcer Prophylaxis. PLoS ONE 11(5): e0155020. https://doi.org/10.1371/journal.pone.0155020

References

1. Daley RJ, Rebuck JA, Welage LS, Rogers FB (2004) Prevention of stress ulceration: current trends in critical care. Crit Care Med 32: 2008–2013. pmid:15483408

2. Cook DJ, Reeve BK, Guyatt GH, Heyland DK, Griffith LE, et al. (1996) Stress ulcer prophylaxis in critically ill patients. Resolving discordant meta-analyses. JAMA 275: 308–314. pmid:8544272

3. Janicki T, Stewart S (2007) Stress-ulcer prophylaxis for general medical patients: a review of the evidence. J Hosp Med 2: 86–92. pmid:17427249

4. Zeitoun A, Zeineddine M, Dimassi H (2011) Stress ulcer prophylaxis guidelines: Are they being implemented in Lebanese health care centers? World J Gastrointest Pharmacol Ther 2: 27–35. pmid:21860840

5. Sheikh-Taha M, Alaeddine S, Nassif J (2012) Use of acid suppressive therapy in hospitalized non-critically ill patients. World J Gastrointest Pharmacol Ther 3: 93–96. pmid:23494814

6. Alsultan MS, Mayet AY, Malhani AA, Alshaikh MK (2010) Pattern of intravenous proton pump inhibitors use in ICU and Non-ICU setting: a prospective observational study. Saudi J Gastroenterol 16: 275–279. pmid:20871193

7. Bez C, Perrottet N, Zingg T, Leung Ki EL, Demartines N, et al. (2013) Stress ulcer prophylaxis in non-critically ill patients: a prospective evaluation of current practice in a general surgery department. J Eval Clin Pract 19: 374–378. pmid:22420909

8. Quenot JP, Mentec H, Feihl F, Annane D, Melot C, et al. (2008) Bedside adherence to clinical practice guidelines in the intensive care unit: the TECLA study. Intensive Care Med 34: 1393–1400. pmid:18365174

9. Tasaka CL, Burg C, VanOsdol SJ, Bekeart L, Anglemyer A, et al. (2014) An interprofessional approach to reducing the overutilization of stress ulcer prophylaxis in adult medical and surgical intensive care units. Ann Pharmacother 48: 462–469. pmid:24473490

10. Khalili H, Dashti-Khavidaki S, Hossein Talasaz AH, Tabeefar H, Hendoiee N (2010) Descriptive analysis of a clinical pharmacy intervention to improve the appropriate use of stress ulcer prophylaxis in a hospital infectious disease ward. Journal of managed care pharmacy: JMCP 16: 114–121. pmid:20178396

11. Chen DG (2014) Drug Analysis of the Appropriate Use of Stress Ulcer Prophylaxis in 200 Post-operation Patients. Journal of Yangtze Univeristy (Nat Sci Edit) 11: 21–22 (in Chinese).

12. Wan H (2014) Drug Analysis of the Appropriate Use PPIs for Stress Ulcer Prophylaxis in 276 Surgery Patients. Strait Pharmaceutical Journal 26: 97–99 (in Chinese).

13. Institute of Medicine (2011) Clinical Practice Guidelines We Can Trust. The National Academies Press.

14. Grimshaw J, Freemantle N, Wallace S, Russell I, Hurwitz B, et al. (1995) Developing and implementing clinical practice guidelines. Qual Health Care 4: 55–64. pmid:10142039

15. The AGREE Next Steps Consortium (2009) Appraisal of Guidelines for Research and Evaluation. Available: http://www.agreetrust.org/. Accessed 2 June 2013.

16. Holmer HK, Ogden LA, Burda BU, Norris SL (2013) Quality of clinical practice guidelines for glycemic control in type 2 diabetes mellitus. PLoS One 8: e58625. pmid:23577058

17. Armstrong TA, Coursin DB, Devlin J, Duke JS, Fish D, et al. (1999) ASHP therapeutic guidelines on stress ulcer prophylaxis. American Journal of Health-System Pharmacy 56: 347–379. pmid:10690219

18. Eastern Association for the Surgery of Trauma (2008) Practice Management Guidelines for Stress Ulcer Prophylaxis. http://www.g-i-n.net/library/international-guidelines-library/@@guideline_search_results?type=basic&basic-searchable-text=stress+ulcer+prophylaxis. Accessed 15 March 2015.

19. Orlando Regional Medical Center (2011) Stress Ulcer Prophylaxis. http://www.surgicalcriticalcare.net/Guidelines/stress%20ulcer%20prophylaxis%202011.pdf. Accessed 15 March 2015.

20. Vanderbilt University Medical Center (2005) Gastrointestinal Stress Ulcer Prophylaxis Guideline. http://traumaburn.com/Protocols/StressUlcerProphylaxisVisio.pdf. Accessed 15 March 2015.

21. Madsen KR, Lorentzen K, Clausen N, Oberg E, Kirkegaard PR, et al. (2014) Guideline for stress ulcer prophylaxis in the intensive care unit. Dan Med J 61: C4811. pmid:24814922

22. Xiao SD, Xu GM, Li ZS, Huang YT, Sun YH, et al. (2002) Consensus guidelines for stress ulcer prophylaxis. National Medical Journal of China 82: 1000–1001 (in Chinese).

23. King Abdullah University Hospital Stress Ulcer Prophylaxis (SUP) Guideline. http://www.just.edu.jo/DIC/ClinicGuidlines/Stress%20Ulcer%20Prophylaxis.pdf. Accessed 15 March 2015.

24. Alhazzani W, Alenezi F, Jaeschke RZ, Moayyedi P, Cook DJ (2013) Proton pump inhibitors versus histamine 2 receptor antagonists for stress ulcer prophylaxis in critically ill patients: a systematic review and meta-analysis. Crit Care Med 41: 693–705. pmid:23318494

25. Neuman J, Korenstein D, Ross JS, Keyhani S (2011) Prevalence of financial conflicts of interest among panel members producing clinical practice guidelines in Canada and United States: cross sectional study. BMJ 343: d5621. pmid:21990257

26. Norris SL, Holmer HK, Ogden LA, Burda BU (2011) Conflict of interest in clinical practice guideline development: a systematic review. PLoS One 6: e25153. pmid:22039406

27. Norris SL, Burda BU, Holmer HK, Ogden LA, Fu R, et al. (2012) Author's specialty and conflicts of interest contribute to conflicting guidelines for screening mammography. J Clin Epidemiol 65: 725–733. pmid:22498428

28. Hahn DL (2009) Importance of evidence grading for guideline implementation: the example of asthma. Ann Fam Med 7: 364–369. pmid:19597175

29. World Health Organization (2014) WHO Handbook for Guideline Development 2nd edition. http://www.who.int/kms/handbook_2nd_ed.pdf?ua=1. Accessed 24 March 2015.

30. National Institute for Health and Clinical Excellence (2012) The guidelines manual. http://publications.nice.org.uk/pmg6. Accessed 24 March 2015.