Application of Agree II Instrument for Appraisal of Postpartum Hemorrhage Clinical Practice Guidelines in Bosnia and Herzegovina

Abstract

Introduction: Federal Minister of Health and Minister of Health and Social Welfare of the Republika Srpska as a Governmental health authorities in Federation of Bosnia and Herzegovina (FBiH) and the Republika Srpska nominated/appointed health professionals as their representatives to a multidisciplinary Guidelines Adaptation Group (GAG). This group started with its work in September 2015. The main purpose of the guidelines development exercise is to develop guidelines with worldwide recognized methodology for clinical guidelines development and adaptation. At the end of this consultancy, GAG would have develop a clinical practice guideline on Postpartum hemorrhage (PPH) through the adaptation method, starting from published international clinical guidelines and adapting it according to the country specific requirements. Methodology: During the process of identifying the best guideline for adaptation, the GAG had to pass several steps. One of the crucial steps was to identify the questions related to clinical practice and health policy for which answers are needed to be addressed by the guideline. These questions included relevant issues regarding the topic area such as diagnosis, prognosis, intervention, service delivery, and training. After that, six guidelines have been researched by the six members of the GAG to see if they answered the identified questions. Evaluating the methodological quality of the selected clinical guidelines was a second essential step before deciding which ones could best fit the needs and interests. AGREE II instrument has been chosen as methods for evaluating clinical guideline quality and appropriateness. Four appraisers conducted the assessment of each of the selected guidelines for PPH. All appraisers passed the training for the AGREE II instrument before conducting appraisals, as recommended by the AGREE collaboration. Each of the four guidelines was rated independently with the AGREE II tool by each appraiser. Results: The highest score was obtained by the WHO recommendations for postpartum hemorrhage guidelines and each of four assessors recommended it to be adapted and further implemented. Discussion: In spite of several serous attempts and two Agencies for quality improvement in BiH the whole process of guideline adaptation is still in the beginning and therefore the establishment of a strong connection with similar institutions and organizations from our region and the rest of the world is very important. Cooperation and collaboration across institutions and countries and key stakeholders have potential to significantly improve the validity and quality of the adapted guidelines and to positively influence implementation.

PROFESSIONAL PAPER

Application of Agree II Instrument for Appraisal of Postpartum Hemorrhage Clinical Practice Guidelines in Bosnia and Herzegovina

Ahmed Novo1, Andreja Subotic-popovic2, Savka Strbac3, Alma Kandic4, and Mihai Horga5

1 Agency for Quality and Accreditation in Healthcare of Federation of Bosnia and Herzegovina (AKAZ), Bosnia and Herzegovina

2 Ministry of Health and Social Welfare of the Republika Srpska, Bosnia and Herzegovina

3 The Republika Srpska Agency for Certication, Accreditation and Health Care Improvement (ASKVA), Bosnia and Herzegovina,

4 Clinical Centre University of Sarajevo, Clinic for Gynecology and Obstetrics, Bosnia and Herzegovina

5 East European Institute for Reproductive Health, Romania.

Corresponding author: Ahmed Novo, MD,

PhD, Ferhadija 17, 71000 Sarajevo, Bosnia and

Herzegovina, e-mail: [email protected]

ABSTRACTIntroduction: Federal Minister of Health and Minister of Health and Social Welfare of the Republika Srpska as a Governmental health authorities in Federation of Bosnia and Herzegovina (FBiH) and the Republika Srpska nominated/appointed health professionals as their representatives to a multidisciplinary Guidelines Adaptation Group (GAG). This group started with its work in September 2015. The main purpose of the guidelines development exercise is to develop guidelines with worldwide recognized methodology for clinical guidelines development and adaptation. At the end of this consultancy, GAG would have develop a clinical practice guideline on Postpartum hemorrhage (PPH) through the adaptation method, starting from published international clinical guidelines and adapting it according to the country specic requirements. Methodology: During the process of identifying the best guideline for adaptation, the GAG had to pass several steps. One of the crucial steps was to identify the questions related to clinical practice and health policy for which answers are needed to be addressed by the guideline. These questions included relevant issues regarding the topic area such as diagnosis, prognosis, intervention, service delivery, and training. After that, six guidelines have been researched by the six members of the GAG to see if they answered the identied questions. Evaluating the methodological quality of the selected clinical guidelines was a second essential step before deciding which ones could best t the needs and interests. AGREE II instrument has been chosen as methods for evaluating clinical guideline quality and appropriateness. Four appraisers conducted the assessment of each of the selected guidelines for PPH. All appraisers passed the training for the AGREE II instrument before conducting appraisals, as recommended by the AGREE collaboration. Each of the four guidelines was rated independently with the AGREE II tool by each appraiser. Results: The highest score was obtained by the WHO recommendations for postpartum hemorrhage guidelines and each of four assessors recommended it to be adapted and further implemented. Discussion: In spite of several serous attempts and two Agencies for quality improvement in BiH the whole process of guideline adaptation is still in the beginning and therefore the establishment of a strong connection with similar institutions and organizations from our region and the rest of the world is very important. Cooperation and collaboration across institutions and countries and key stakeholders have potential to signicantly improve the validity and quality of the adapted guidelines and to positively inuence implementation.

Key words: AGREE instrument, appraisal of clinical practice guidelines, postpartum hemorrhage.

1. INTRODUCTION

Evidence based clinical practice guidelines (CPGs) are not routinely used in Bosnia and Herzegovina (1). Although evidence-based medicine is widely accepted in the developed world as a contemporary approach to solving medical problems (although not always without resistance), its signicance and implementation in developing countries still needs to be evaluated, and in the majority of these countries is still in its initial phase (2).

doi: 10.5455/aim.2016.24.211-214

ACTA INFORM MED. 2016 JUN; 24(3): 211-214

Received: MAR 13, 2016 Accepted: APR 29,

2016

2016 Ahmed Novo, Andreja Subotic

popovic, Savka Strbac, Alma Kandic, and

Mihai HorgaThis is an Open Access article distributed

under the terms of the Creative Commons

Attribution Non-Commercial License (http://

creativecommons.org/licenses/by-nc/4.0/)

which permits unrestricted non-commercial

use, distribution, and reproduction in any

medium, provided the original work is

properly cited.

Clinical practice guidelines (CPGs are based on the best possible scientic evidence (3), expert opinion and professional consensus and they take into account patients and all key stake-holders (4). In Bosnia and Herzegovina CGPs have advisory character and as such have to be exible so as to reconcile the most recent scientic and clinical achievements with what can realistically be achieved in the health sector of BiH. Their development must be sustainable process, taking into account

PROFESSIONAL PAPER / ACTA INFORM MED. 2016 JUN; 24(3): 211-214 211

Application of Agree II Instrument for Appraisal of Postpartum Hemorrhage Clinical Practice Guidelines

the limited resources. CPGs are developed by multidisciplinary teams of health professionals and other stakeholders fostering partnerships, cooperation, active participation and fair representation of all stakeholders.

In 2013, UNFPA Regional Office for Eastern Europe and Central Asia in partnership with the East European Institute of Reproductive Health (EEIRH) and the Royal College of Obstetricians and Gynecologists (RCOG) started a regional initiative on building capacity of local institutions to develop/adapt and implement evidence-based clinical guidelines as an integral part of improving quality of care in sexual and reproductive health (SRH). The purpose of the initiative is to contribute to improving quality of care of SRH services through harmonization and institutionalization of development/adaptation and implementation of guidelines. BiH participated in 2013 with ve representatives from various health institutions at the regional training for adaptation/development and implementation, including monitoring and evaluation, of evidence-based clinical guidelines.

UNFPA Country Office for BiH supports the Federal Ministry of Health and Ministry of Health and Social Welfare of the Republika Srpska in developing/adapting guidelines on SRH. Both ministries mentioned above have requested UNFPA and other development partners to provide technical support in developing/adapting new guidelines/protocols. To respond to this request, UNFPA Country Office for BiH has conducted training in July 2014 for developing the capacity of specic institutions to develop, adapt, review and monitor implementation of guidelines and protocols on SRH, taking into account dierent administrative structures in BiH (two entities: Federation of Bosni and Herzegovina and Republika Srpska and one district: Brcko district). The UNFPA and its implementing partner in BiH Partnership in Public Health plan to support Guidelines Adaptation Group (GAG) to develop clinical guidelines for Postpartum Hemorrhage with the ultimate purpose/aim of reducing the maternal mortality in the country.

Postpartum hemorrhage ( PPH) is the most common form of obstetric hemorrhage and is a leading cause of maternal morbidity and mortality. Approximately 30% ( in some countries over 50%) of direct maternal deaths worldwide are due to hemorrhage, mostly in the postpartum period. Primary PPH is dened as excessive bleeding in the rst 24 hours post birth, a blood loss in excess of 500 ml after vaginal birth and a blood loss in excess of 1000 ml after caesarean section (CS). PPH can be minor (500-1000 ml) or major (more than 1000 ml). Major can be divided to moderate (1000-2000 ml) or severe (more than 2000 ml). Secondary PPH is abnormal or excessive bleeding from the birth canal between 24 hours and 12 weeks postnatal.

There is no single denition for PPH. Diagnosing PPH is an emergent situation most commonly through estimation of volume of blood loss and changes in the hemodynamic state. The World Health Organization`s International Classication of Diseases (ICD-10) denes postpartum hemorrhage as hemorrhage after delivery of fetus or infant. Most cases of PPH have no identiable risk factors therefore vigilance and early recognition of hemorrhage is required. Risk factors include suspected or proven placental abruption, known placenta previa, multiple pregnancy, preeclampsia/gestational

hypertension; previous PPH, obesity, anemia; retained placenta, operative vaginal delivery, mediolateral episiotomy, prolonged labor, big baby (>4kg), pyrexia in labor, age (>40 years, not multiparous).

Federal Minister of Health and Minister of Health and Social Welfare of the Republika Srpska as a Governmental health authorities in Federation of Bosnia and Herzegovina (FBiH) and the Republika Srpska nominated/appointed health professionals as their representatives to the members of multidisciplinary working group named Guidelines Adaptation Group (GAG). GAG started with its work in September 2015. The main purpose of the guidelines development exercise is to develop guidelines with worldwide recognized methodology for clinical guidelines development and adaptation. At the end of this consultancy, GAG would have develop a clinical guideline on Postpartum hemorrhage through the adaptation method, starting from one or more published international clinical guidelines and adapting it according to the country specic requirements. The Project was planned to be executed through the following phases: Establishment of Steering/Oversight Committee for the guidelines programme, Topic selection, Establishment of the Guideline Adaptation Group (GAG), and Guideline adaptation. The value and usefulness of the guidelines will be evaluated using the AGREE II instrument.

Agency for Quality and Accreditation in Healthcare in Federation of Bosnia and Herzegovina (AKAZ) and Agency for Certication, Accreditation and Health Care Improvement of the Republic of Srpska (ASKVA), are institutions in FBiH and the Republika Srpska for dening, measuring, monitoring and improving standards of clinical practice and management of services in Bosnia and Herzegovina (BiH). AKAZ is also a federal center for the denition, evaluation and dissemination of evidence based clinical guidelines for effective and efficient clinical practice (5).

2. METHODOLOGY

During the process of identifying the best guideline for adaptation, the Guideline Adaptation Group (GAG) has to pass a several steps. One of the crucial steps was to identify the questions related to clinical practice and health policy for which answers are needed to be addressed by the guideline. These questions included relevant issues regarding the topic area such as diagnosis, prognosis, intervention, service delivery, and training. Thus, the GAG members identied ve key group of questions related to clinical practice (prevention and treatment) in postpartum hemorrhage consists of 37 questions: Denition of the postpartum hemorrhage (minor PPH and major PPH);

Risk factors for postpartum hemorrhage (PPH) (a) Etiologic category and process and 2. Clinical risk factors) identications and how they can be minimized;

PPH prevention (use of uterotonics for the prevention of PPH);

PPH treatment (Before admission in the hospital and transportation and during the admission in the hospital);

Organization of the care of postpartum hemorrhage. After that six guidelines (RCOG GTG 52 Prevention and

212 PROFESSIONAL PAPER / ACTA INFORM MED. 2016 JUN; 24(3): 211-214

Application of Agree II Instrument for Appraisal of Postpartum Hemorrhage Clinical Practice Guidelines

Management of Postpartum Hemorrhage 2011, FIGO post-partum hemorrhage low settings, 2012, QUEENSLAND Clinical Guidelines-Primary postpartum hemorrhage 2013, WHO recommendations for postpartum hemorrhage, 2012, RANZCOG Management of Postpartum hemorrhage, 2015, Royal College of Physicians of Ireland Postpartum Hemorrhage, 2012) have been researched by the six members of the GAG to see if these guidelines answered the identied questions. The results showed that three guidelines should be evaluated using the AGREE II methodology: RCOG GTG 52 Prevention and Management of Postpartum Hemorrhage (2011), WHO recommendations for postpartum hemorrhage (2012), and Royal College of Physicians of Ireland Post-partum hemorrhage (2012).

Evaluating the methodological quality of clinical practice guidelines is essential step before deciding which ones could t bests our needs and interests. AGREE II instrument has been chosen as methods for evaluating clinical guideline quality and appropriateness.

The AGREE II tool encompasses 23 items in six domains:1. Scope and purpose (3 items),2. Stakeholder involvement (3 items),3. Rigor of development (8 items),4. Clarity of presentation (3 items),5. Applicability (4 items), and6. Editorial independence (2 items).

The domain scope and purpose assesses whether the guideline describes its overall objective and target population clearly. A guideline should entail a clear denition of the target users, as well as demonstrate that the views and preferences of the target population (e.g. patients, public) have been sought and that the guideline development group includes all relevant professional groups which are assessed in stakeholder improvement.

The next domain assesses the rigor of guideline development, encompassing systematic literature searching methodology, transparency of evidence-gathering process and whether evidence is explicitly linked to the recommendations. It also asks if the guideline has been externally reviewed prior to publication and if an updating strategy has been documented.

The fourth domain examines clarity of guideline presentation. Recommendations must be specic, unambiguous and easily identiable. Dierent options for management of the condition or health issue should be clearly presented.

Applicability examines whether guidelines describe facilitators and barriers to application and if they provide advice and/or tools on how the recommendations can be put into practice. It also assesses resource implications for guideline application and monitoring and/or auditing criteria. Transparency of guideline funding bodies and conicts of interest were examined in editorial independence (6).

Each item is rated from 1 (strongly disagree) to 7 (strongly agree). Detailed criteria for each item are available within the AGREE II tool to assist the appraiser (1)]. The appraisers were asked to provide comments to justify their rating. They also gave an overall assessment of the guideline from 1 (lowest) to 7 (highest) and were asked to state if they would recommend the guideline, recommend it with modications or not recommend it.

Average appraisal scores were calculated for each appraiser by taking the average rating (1-7) for all items of a single guideline. Percentages for each domain were then calculated, this was done by adding all four appraiser ratings (1-7) of items within a single domain (obtained score) and scaling by maximum and minimum possible domain scores and converting to a percentage.

Four appraisers conducted the assessment of each of the selected guidelines for PPH. All appraisers passed the training for the AGREE II instrument before conducting appraisals, as recommended by the AGREE collaboration. Each of the four guidelines was rated independently with the AGREE II tool by each appraiser. Appraisers did not communicate or confer with each other during the appraisal process. The results were shared anonymously among the authors.

3. RESULTS

Domain 1. Scope and Purpose is concerned with the overall aim of the guideline, the specic health questions, and the target population (items 1-3). Those three items are: The overall objective(s) of the guideline is (are) specically described, 2. The health question(s) covered by the guideline is (are) specically described and 3. The population (patients, public, etc.) to whom the guideline is meant to apply is specifically described. Each of four assessors rated each item from 1 to 7 as shown in the Table 1 for all three chosen guidelines.

Assessors Item 1 Item 2 Item 3 Total Assessor 1 7 7 7 21 Assessor 2 6 7 1 14 Assessor 3 7 7 2 16 Assessor 4 7 7 7 21 Total 27 28 17 72

Table 1 Quality score for domain 1, WHO recommendations for

postpartum hemorrhage

Quality score is calculated for each of the six AGREE II domains. The six domain scores are independent and have not been aggregated into a single quality score. Domain scores are calculated by summing up all the scores of the individual items in a domain and by scaling the total as a percentage of the maximum possible score for that domain according to the following formula: Obtained score - minimum possible score / maximum possible score - minimum possible score x 100.

Results for the domain 1, WHO recommendations for postpartum hemorrhage are the following:

Quality score was calculated for all six domains and all tree guidelines and results of the appraisal are presented in Table 2.

No. AGREE II domains WHO CPG RCOP IRE

LAND CPG

RCOG UK

CPG

1. Scope and Purpose 83% 86% 54%2. Stakeholder Involvement 86% 51% 58%3. Rigor of Development 98% 55% 74%4. Clarity of Presentation 100% 90% 85%5. Applicability 97% 46% 58%6. Editorial Independence 100% 0 54%7. Overall assessment 7 5 6 Table 2. Results of the appraisal

PROFESSIONAL PAPER / ACTA INFORM MED. 2016 JUN; 24(3): 211-214 213

Application of Agree II Instrument for Appraisal of Postpartum Hemorrhage Clinical Practice Guidelines

The highest score obtained WHO recommendations for postpartum hemorrhage guidelines and each of four assessors recommended it to be adopted and further implemented.

4. DISCUSSION

Since its original release in 2003, the Appraisal of Guidelines for Research and Evaluation (AGREE) Instrument has advanced the science of clinical practice guidelines (CPG) appraisal and quickly became the international gold standard for PG evaluation and development. The AGREE Instrument has been translated into many languages, endorsed by several organizations e.g., National Institute for Health and Clinical Excellence, and used by many development groups e.g., WHO Advisory Committee on Health Research (7).

Health professionals should choose among many various dierent CPG which sometimes could be even conicting. They may oer concise instructions on which diagnostic or screening tests to order, how to provide medical or surgical services, how long patients should stay in hospital, or other details of clinical practice. Flawed clinical guidelines harm practitioners by providing inaccurate scientic information and clinical advice, thereby compromising the quality of care. They may encourage ineective, harmful, or wasteful interventions. Even when guidelines are correct, clinicians often nd them inconvenient and time consuming to use (8). Conicting guidelines from dierent professional bodies can also confuse and frustrate practitioners (9).

In Bosnia and Herzegovina health professionals at First BH Cochrane Symposium dened additional challenges in connection to the development and implementation of CPGs: Poor knowledge and importance of evidence-based medicine and Cochrane Database of Systematic Reviews;

Poor knowledge of literature search in order to seek out or look for the evidence;

Poor knowledge of methods of adaptation; Weak organizational culture and the weak professional structures;

The lack of a consistent structure for the presentation, adaptation/adoption and implementation of guidelines;

The lack of serious nancial support for the development of CPGs;

Lack of incentives to implement CPGs into practice and quality improvement (10);

Lack of understanding and resistance to accept whole process of development, adaptation and implementation of CPGs;

Credible system of clinical practice guidelines (CPG) development/adaptation supported by key stakeholders in health-care system including patient organizations and professionals associations.

Efficient process of development/adaptation includes: Well

dened development/adaptation of CPGs; evaluation of CPGs; dissemination and implementation of CPGs; time for updating CPGs and ratication of CPGs from professional societies. Development of adapted evidence-based clinical guidelines in developing countries such as Bosnia and Herzegovina represents a very important way to apply international guidelines in a culturally competent way in diverse, transitional settings as BiH is, in order to promote and improve the quality of health services. In spite of several serous attempts and two Agencies for quality improvement in BiH, AKAZ and ASKVA the whole process of guideline adaptation is still in the beginning and therefore the establishment of a strong connection with similar institutions and organizations from our region and the rest of the world are essential. Cooperation and collaboration across institutions and countries and key stakeholders have potential to signicantly improve the validity and quality of the adapted guidelines and to positively inuence implementation.

Conict of interest: None declared.

REFERENCES

1. Serdarevic F, Vranic S, Dizdarevic J, Masic I, Rianovic Z. Towards Evidence-Based Clinical Practice Guidelines: The Case of Bosnia and Herzegovina. Acta Inform Med. 2008; 16(2): 88-92.

2. Chinnock P, Siegfried N, Clarke M. Is evidence-based medicine relevant to the developing world. eCAM, 2005; 2: 321-4.

3. Field MJ, Lohr KN. Guidelines for clinical practice: From development to use. IOM (Institute of Medicine). Washington DC: National Academy Press, 1992.

4. Field MJ, Lohr KN. Institute of Medicine (IOM). Health Professions Education: A Bridge to Quality. Washington, DC: National Academy Press, 2003.

5. Mahmic-Kaknjo M, Novo A, Krleza-Jeric K. First BH Cochrane Symposium held. Mater Sociomed. 2016 Feb; 28(1): 74-6.

6. The AGREE Collaboration. Appraisal of Guidelines for Research and Evaluation. Accessed from http://www.agreetrust. org on 10 November 2015.

7. The original AGREE Instrument. Accessed on 15 February 2016 from http://www.agreetrust.org/resource-centre/ the-original-agree-instrument/

8. Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J. Potential benets, limitations, and harms of clinical guidelines. BMJ. 1999 February 20; 318(7182): 527-30.

9. Feder G. Management of mild hypertension: which guidelines to follow? BMJ. 1994; 308: 470-1.

10. Novo A. Implementing science into practice - role of AKAZ. Accessed from https://www.researchgate.net/prole/Ahmed_ Novo/publications on 22 Nov 2015.

11. UNFPA Regional Office for Eastern Europe and Central Asia. Development and Implementation of Clinical Guidelines for Sexual and Reproductive Health. Programme Guidance for Countries in Eastern Europe and Central Asia. 2012.

214 PROFESSIONAL PAPER / ACTA INFORM MED. 2016 JUN; 24(3): 211-214