Variation in antibiotic prescription rates in febrile children presenting to emergency departments across Europe (MOFICHE): A multicentre observational study

Abstract

[...]the participating hospitals were either university hospitals (n = 9) or large teaching hospitals (n = 3), and 11 EDs had paediatric intensive care facilities. Collected data included age, sex, season, referral, comorbidity (chronic condition expected to last at least 1 year) [22], triage urgency, fever duration, fever measured at ED, presence of “red traffic light” symptoms for identifying risk of serious illness (alarming signs) (from the National Institute for Health and Care Excellence [NICE] guideline on fever [23]: decreased consciousness, ill appearance, work of breathing, meningeal signs, focal neurology, non-blanching rash, dehydration, status epilepticus), previous antibiotic use, vital signs (heart rate, respiratory rate, oxygen saturation, temperature, capillary refill time), laboratory results (white blood cell count, C-reactive protein [CRP], urinalysis), imaging (chest X-ray and other imaging), microbiological investigations (cultures and respiratory viral tests), and disposition (intensive care unit admission, general ward admission or discharge). The focus of infection was categorised as upper respiratory tract (otitis media, tonsillitis/pharyngitis, other), lower respiratory tract, gastrointestinal tract, urinary tract, skin, musculoskeletal, sepsis, central nervous system, flu-like illness, childhood exanthem, inflammatory syndrome, undifferentiated fever, or other. CRP, C-reactive protein; LRTI, lower respiratory tract infection; URTI, upper respiratory tract infection. *Patients could have identified viral co-infection. https://doi.org/10.1371/journal.pmed.1003208.g001 We aimed to improve data quality and standardised data collection by using a training module for the local clinical and research teams to optimise clinical assessment and data collection for febrile children.

Introduction

Fever is one of the most common reasons for children to visit the emergency department (ED), and most visits are accounted for by self-limiting infections [1,2]. The proportion of children with a serious bacterial infection that needs treatment with antibiotics ranges from 7% to 13%, while antibiotic prescription rates in febrile children at EDs are between 19% and 64% [3–5]. Inappropriate antibiotic use, including the unnecessary use of broad-spectrum antibiotics, remains high in children, promoting the emergence of antimicrobial resistance [6–9]. Inappropriate antibiotic prescriptions were described in around 30% of outpatient prescriptions. However, these outpatient settings mainly involve primary care, and limited studies are available on specific emergency care [6,10].

Large variability exists between countries in antibiotic prescriptions in inpatient and outpatient settings, according to several large studies [6,8,11–14]. In general, these large studies did not adjust for differences in populations. In children, previous studies have demonstrated substantial variation of antibiotic use in general outpatient settings in the United States and Europe, indicating possible overuse of antibiotics [5,6,10,15].

A literature review on antibiotic prescription rates and their determinants in febrile children in emergency care found large heterogeneity of studied populations, which limited the ability to draw conclusions [16]. One recent European study, focusing solely on EDs, showed significant differences in antibiotic prescription rates in otherwise healthy children with respiratory tract infections (RTIs) [5]. Large studies at EDs across Europe are lacking that cover antibiotic and broad-spectrum prescriptions in all febrile children, including patients with comorbidity, patients with detailed clinical information, and patients in different diagnostic groups. Additionally, previous studies have addressed appropriate prescribing based on diagnosis coded with the International Classification of Diseases [6,10,17]. This classification, however, may not accurately take into account bacterial versus viral aetiology. Antibiotic prescription rates for viral and bacterial disease using a structured classification have not yet been investigated at EDs.

A better understanding of variability in antibiotic prescriptions in EDs and its relation with bacterial or viral disease, taking into account differences in case mix, is essential for the development and implementation of interventions to optimise antibiotic use. In addition, knowledge regarding variation of prescribing in infections where antibiotic prescription is inappropriate, such as prescriptions in viral disease, prescriptions of inappropriate duration, or prescriptions that are not concordant with guidelines, could target and improve implementation of antimicrobial stewardship guidelines at the ED level.

In this study, we aim to investigate the variation and appropriateness of rates and types of antibiotic prescription in febrile children attending 12 different EDs in Europe.

This is a main analysis of the MOFICHE (Management and Outcome of Fever in Children in Europe) study, which is embedded in the PERFORM (Personalised Risk assessment in Febrile illness to Optimise Real-life Management across the European Union) project (https://www.perform2020.org) [18]. MOFICHE is an observational multicentre study that studies the management and outcome of febrile children in Europe using routine data. The overall aim of PERFORM is to improve management of febrile children and to improve diagnosis through development of new diagnostic tests to discriminate viral and bacterial infections in children.

Methods

Study design

MOFICHE is a prospective observational study using data that are collected as part of routine care. This study is reported as per the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guideline (S1 Text), and data were analysed using an a priori statistical analysis plan (S2 Text). The study was approved by the ethics committees in the participating hospitals, and the need for informed consent was waived (S3 Text).

Study population and setting

Children aged 0–18 years presenting with fever (temperature ≥ 38.0°C) or a history of fever (fever within 72 hours before ED visit) were included. Twelve EDs from 8 European countries participated in this study: Austria, Germany, Greece, Latvia, the Netherlands (n = 3), Spain, Slovenia, and the United Kingdom (n = 3). The EDs were included because they all participated in the PERFORM project. Characteristics of these EDs are described in S4 Text and in a previous publication [19]. In short, the participating hospitals were either university hospitals (n = 9) or large teaching hospitals (n = 3), and 11 EDs had paediatric intensive care facilities. Nine EDs were paediatric focused, and 3 EDs served both children and adults. Care for febrile children was supervised by general paediatricians (7 EDs), by paediatric emergency physicians (2 EDs), or by a general paediatrician or a (paediatric) emergency physician (3 EDs). All data were available in electronic healthcare records in 5 EDs, 1 ED used paper records, and 6 EDs used a combination of paper and electronic healthcare records.

Data were collected from January 2017 until April 2018, and for at least 1 year at each site to include all seasons. The period of data collection per month ranged from 1 week per month to the whole month in the participating hospitals (S4 Text).

Sample size

We expected to include 40,000 children with at most 5% missing data. Pilot data showed an overall antibiotic prescription rate of 30%. Applying 10 events per variable, this study is large enough to analyse over 1,000 determinants for the outcome antibiotic prescription [20]. We performed a post hoc sample size estimation for a desired width of the 95% confidence interval (CI) of standardised antibiotic prescription rate per ED. The expected width of the CI of the standardised prescription rate was below 0.5 for the smallest ED.

Data collection

Data were collected as part of routine ED care. The local research team entered data from patient records in an electronic case record form (eCRF) [21]. Collected data included age, sex, season, referral, comorbidity (chronic condition expected to last at least 1 year) [22], triage urgency, fever duration, fever measured at ED, presence of “red traffic light” symptoms for identifying risk of serious illness (alarming signs) (from the National Institute for Health and Care Excellence [NICE] guideline on fever [23]: decreased consciousness, ill appearance, work of breathing, meningeal signs, focal neurology, non-blanching rash, dehydration, status epilepticus), previous antibiotic use, vital signs (heart rate, respiratory rate, oxygen saturation, temperature, capillary refill time), laboratory results (white blood cell count, C-reactive protein [CRP], urinalysis), imaging (chest X-ray and other imaging), microbiological investigations (cultures and respiratory viral tests), and disposition (intensive care unit admission, general ward admission or discharge). We collected data on antibiotics prescribed in the ED or started on the first day of hospital admission (type, route of administration, and duration). The focus of infection was categorised as upper respiratory tract (otitis media, tonsillitis/pharyngitis, other), lower respiratory tract, gastrointestinal tract, urinary tract, skin, musculoskeletal, sepsis, central nervous system, flu-like illness, childhood exanthem, inflammatory syndrome, undifferentiated fever, or other.

To date, no reference standard exists to classify the cause of infection in routine ED practice [24]. The PERFORM consortium adapted the consensus-based flowchart from Herberg and colleagues [25,26], combining all available clinical data, investigation results such as CRP, cultures, and imaging. This flowchart was used to define the presumed cause of infection for each patient visit: definite bacterial, probable bacterial, bacterial syndrome, unknown bacterial/viral, viral syndrome, probable viral, definite viral, trivial, inflammatory syndrome, and other (Fig 1). The diagnosis definite bacterial infection was assigned only when a sterile site culture identified pathogenic bacteria. The diagnosis ‘probable bacterial infection’ was assigned when a bacterial syndrome was suspected but no bacteria were identified and CRP was above 60 mg/l. Patients with clinical bacterial symptoms and CRP ≤ 60 mg/l or no CRP were classified as ‘bacterial syndrome’. Children with suspected viral infections were classified as ‘viral syndrome’ (no CRP or CRP > 60 mg/l) or ‘definite viral’ (CRP ≤ 60 mg/l) when a virus was identified that matched the clinical symptoms. Children with a viral syndrome and CRP ≤ 60 mg/l, but no identified virus, were classified as ‘probable viral’. Children who did not fit these definitions were classified as unknown bacterial/viral. Children with mixed infections (bacterial and viral co-infection) were classified as bacterial. Children with trivial infections, inflammatory syndrome, or other infections were classified as ‘other’.

[Figure omitted. See PDF.]

Fig 1. Categorisation of presumed cause of infection.

CRP, C-reactive protein; LRTI, lower respiratory tract infection; URTI, upper respiratory tract infection. *Patients could have identified viral co-infection.

https://doi.org/10.1371/journal.pmed.1003208.g001

We aimed to improve data quality and standardised data collection by using a training module for the local clinical and research teams to optimise clinical assessment and data collection for febrile children. This training module included clarification of the individual alarming signs and classification examples of common diagnoses. Furthermore, entry guidelines for the eCRF were available, monthly teleconferences and biannual meetings were organised, and quarterly reports of data quality for each ED were discussed. These consortium teleconferences also included discussion of difficult cases.

Antibiotic classification

Antibiotics were categorised using the Anatomical Therapeutic Chemical classification including beta-lactamase sensitive penicillins (J01CE); beta-lactamase resistant penicillins (J01CF); penicillins with extended spectrum (J01CA); combinations of penicillins including beta-lactamase inhibitors (J01CR); macrolides (J01FA); first-generation, second-generation, and third-generation cephalosporins (J01DB, J01DC, J01DD); trimethoprim and sulphonamides (J01EA01, J01EE01); aminoglycosides (J01GB); quinolones (J01MA); glycopeptides (J01XA); and other antibiotics.

In addition, we compared the prescription of narrow-spectrum and broad-spectrum antibiotics. We explored the definitions reported in previous studies on antibiotic classification and used an expert opinion panel including paediatric infectious disease specialists and general paediatricians (PERFORM partners), to establish the final classification into broad-spectrum and narrow-spectrum for all systemic antibiotics [5,6,11,15,27,28]. Narrow-spectrum antibiotics comprised penicillins (e.g., amoxicillin) and first-generation cephalosporins. Broad-spectrum antibiotics included penicillins with beta-lactamase inhibitor combinations (e.g., amoxicillin/clavulanic acid), macrolides, aminoglycosides, glycopeptides, and second-generation and third-generation cephalosporins. Prescriptions of both broad-spectrum and narrow-spectrum antibiotics in the same patient were considered broad-spectrum. Topical antibiotics were not included. Details of this classification are presented in S5 Text.

Outcomes

We assessed various aspects of antibiotic prescription: (1) antibiotic prescription rate; (2) the proportion of antibiotics that were broad-spectrum versus narrow-spectrum; (3) the proportion of antibiotics of ‘likely appropriate’ indication (presumed bacterial), ‘likely inappropriate’ indication (presumed viral), or ‘inconclusive’ indication (unknown bacterial/viral); (4) the proportion of oral antibiotics of inappropriate duration; and (5) the proportion of oral antibiotics that matched the antibiotic type in the local guideline (‘guideline-concordant’) in uncomplicated urinary and upper and lower RTIs. Antibiotic prescriptions were classified as likely appropriate in presumed bacterial infections (definite bacterial, probable bacterial, bacterial syndrome), likely inappropriate in presumed viral infections (definite viral, probable viral, viral syndrome), and inconclusive in unknown bacterial/viral infections or other infections. Inappropriate duration was defined as >10 days for treatment of tonsillitis with beta-lactamase sensitive penicillins (J01CE) and >7 days for all other prescriptions according to recommendations by international guidelines [29–31]. In addition, guideline-concordant prescription in patients with uncomplicated RTIs and uncomplicated urinary tract infections was defined according to the local guideline (S6 Text). Uncomplicated infections were defined as infections in previously healthy children who did not receive therapeutic antibiotic treatment before the ED visit.

Data analysis

Missing values were assumed to be missing at random, and therefore we used multiple imputation by chained equations with the MICE package in R for the regression analysis. We excluded patients with missing data on antibiotic prescription, presumed cause of infection, and focus of infection [32]. Only the first visit was included for patients who visited the ED again within 5 days.

First, we performed a descriptive analysis of the frequency of antibiotic prescription and broad-spectrum and narrow-spectrum prescription, including ranges across EDs. For all outcomes, we calculated the overall proportion and proportion per ED. Second, we used multilevel logistic regression with a random intercept for each ED to study variation between EDs in antibiotic prescription, broad-spectrum prescription versus narrow-spectrum prescription, and intravenous/intramuscular versus oral prescriptions [33]. In an adjusted model we corrected for patient-level factors and for hospital-level factors influencing antibiotic prescribing. Patient-level factors were selected a priori according to the literature [3,4,23,34,35] and included general characteristics (age, sex, season, comorbidity, referral [referred versus self-referred]), markers for disease severity such as triage urgency (high urgency [immediate, very urgent, urgent] versus low urgency [standard, non-urgent]), fever duration in days, fever measured at ED visit (≥38°C), and presence of NICE guideline “red traffic light” alarming signs (0, 1, ≥2). We investigated diagnostics, including CRP (not performed or <20, 20–60, or >60 mg/l) [25,36], chest X-ray (not performed, normal, abnormal), and urinalysis (not performed, normal, abnormal [positive for leukocyte esterase and/or nitrite]). Furthermore, we included focus of infection (upper respiratory tract, lower respiratory tract, gastrointestinal, urinary tract, undifferentiated fever, skin/musculoskeletal, sepsis/central nervous system, flu-like illness/childhood exanthem, inflammatory/other) and diagnostic groups according to cause as classified by the flowchart in Fig 1: presumed bacterial (definite bacterial, probable bacterial, bacterial syndrome), unknown bacterial/viral, presumed viral (definite viral, probable viral, viral syndrome), and other.

For the hospital-level factors, we explored variables that varied between hospitals and were related to antibiotic prescribing [19,37–40]: total number of ED visits, supervision, availability of point-of-care tests (streptococcal antigen test and CRP), and primary care during out-of-office hours. We included hospital-level factors if they improved the model using univariate analysis. Linearity of continuous variables was tested using restricted cubic splines. Specifications of the adjusted model are presented in Table 1 and in S7 Text.

[Figure omitted. See PDF.]

Table 1. Variables in the adjusted model.

https://doi.org/10.1371/journal.pmed.1003208.t001

Variation in antibiotic prescription rates between EDs was determined by 2 measures: standardised prescription rates and median odds ratios (MORs). We calculated standardised antibiotic prescription rates using indirect standardisation, where the expected number of antibiotic prescriptions was standardised to the average ED. Standardised antibiotic prescription ratios are the ratio between observed antibiotic prescriptions in an ED and the expected antibiotic prescriptions in an ED. The expected number of antibiotic prescriptions was estimated through the adjusted model, by summing the predicted probabilities from the adjusted model of antibiotic prescription for each of the patients. Standardised rates > 1 indicate higher prescription rates than expected, and standardized rates < 1 indicate lower prescription rates than expected. We visualised standardised rates in a heat map.

The MOR is a measure of variation between high- and low-prescribing clusters of EDs. The MOR reflects the difference in probability of receiving antibiotics comparing similar patients attending an ED with high antibiotic prescribing and an ED with low antibiotic prescribing. If the MOR is equal to 1.00, there is no variation between clusters, and if the MOR is high, this indicates important between-cluster variation [41,42].

Stratified analyses were performed in patients with and without comorbidities. Also, since antimicrobial resistance patterns vary greatly between European countries, standardised rates of broad-spectrum versus narrow-spectrum antibiotic prescription were compared with antimicrobial resistance data of invasive isolates on a national level and at the hospital level [11,43] (S8 Text). Correlations were calculated using the 2-tailed Spearman’s rank coefficient (ρ). A p-value below 0.05 was considered significant.

Appropriateness of antibiotic prescriptions

We calculated standardised rates for antibiotic prescription and broad-spectrum prescription in groups of presumed viral infections, presumed bacterial infections, and unknown bacterial/viral infections. Next, we assessed the proportion of all antibiotic prescriptions that were likely appropriate, likely inappropriate, and inconclusive. For all oral prescriptions, we calculated the proportion of prescriptions that were both inappropriate in indication (likely inappropriate) and of inappropriate duration, and the proportion of prescriptions that were either inappropriate in indication or inappropriate in duration. In uncomplicated RTIs and urinary tract infections, we calculated the proportion of all oral prescriptions that were inappropriate for all the 3 measures (indication, duration, and guideline concordance), and the proportion of prescriptions that were inappropriate in any of the 3 measures. R version 3.4 was used for the analysis and visualisation of the data.

Results

Study population

Of the total population of 38,480 patients, we excluded 738 patients based on missing data of antibiotics or diagnosis, and the repeated visit of 2,092 patients to the same ED. Compared to patients with complete outcome data, patients with missing data were similar in age, sex, comorbidity, and admission rate (S9 Text). In addition, there were no differences in completeness of outcomes and diagnosis between discharged and admitted patients.

For the analysis, we included 35,650 febrile children (median age 2.8 years [IQR 1.3–5.6], 54.6% male). The different EDs varied substantially in patients who were referred (range: 4.9%–99.2%), were ill appearing (range: 0.8%–47.4%), or had any comorbidity (range: 5.1%–65.3%) (Table 2). The most common infections were upper respiratory tract (n = 18,783, 52.7%), lower respiratory tract (n = 5,167, 14.5%), gastrointestinal tract (n = 3,694, 10.4%), and undifferentiated fever (n = 2,784, 7.8%). The incidence of sepsis and central nervous system infections was low (n = 270, 0.8%). The majority of the children had a presumed viral infection (n = 20,383, 57.2%); presumed bacterial infections occurred in 22.1% of the patients (definite bacterial/bacterial syndrome, 4.1%; probable bacterial, 18.1%), and unknown bacterial/viral infections in 14.6% (n = 5,200) (Table 3).

[Figure omitted. See PDF.]

Table 2. Patient characteristics of the study population (n = 35,650).

https://doi.org/10.1371/journal.pmed.1003208.t002

[Figure omitted. See PDF.]

Table 3. Patient characteristics of the study population: Outcomes, n = 35,650.

https://doi.org/10.1371/journal.pmed.1003208.t003

Overall antibiotic prescriptions

The overall antibiotic prescription rate was 31.9% (n = 11,371), of which 67.2% (7,636/11,731) were oral administrations and 31.3% (3,564/11,371) were administered intravenously or intramuscularly (153 children received a single dose at the ED). One-third of patients were treated with antibiotics for over 7 days (3,534/9,391, 37.6%) (S1 Fig). The types of antibiotics most often prescribed were penicillins with extended spectrum (3,220/11,371, 28.3%), combinations of penicillins with beta-lactamase inhibitors (2,309/11,371, 20.3%), and beta-lactamase sensitive penicillins (2,001/11,371, 17.6%). Half of the prescribed antibiotics were broad-spectrum agents (5,887/11,371, 51.7%). The most prescribed broad-spectrum antibiotics were combinations of penicillins with beta-lactamase inhibitors (2,309/11,371, 20.3%), second-generation cephalosporins (1,154/11,371, 10.1%), and third-generation cephalosporins (1,097/11,371, 9.6%) (Table 4; Fig 2).

[Figure omitted. See PDF.]

Fig 2. Antibiotic classes of prescribed antibiotics across EDs, n = 35,650.

Red shades indicate broad-spectrum classes, blue shades indicate narrow-spectrum classes, and grey shades indicate unclassified classes and prescriptions of unknown class. EDs are sorted by antibiotic prescription rate. ED, emergency department; NL, the Netherlands; UK, United Kingdom.

https://doi.org/10.1371/journal.pmed.1003208.g002

[Figure omitted. See PDF.]

Table 4. Frequencies of antibiotic classes and ranges across EDs (n = 11,371).

https://doi.org/10.1371/journal.pmed.1003208.t004

Variation of overall antibiotic prescription and broad-spectrum prescription

The proportion of febrile children receiving an antibiotic prescription ranged from 22.4% to 41.6% across EDs, and the proportion of those prescriptions that were for broad-spectrum agents ranged from 33.0% to 90.3%. Of the broad-spectrum agents, penicillins with beta-lactamase inhibitors had the largest variation (range 1.4%–59.0%), but other broad-spectrum agents varied as well (range 17.3%–37.0%). Fig 3 presents the standardised prescription rates from the adjusted model. None of the hospital-level factors was related with antibiotic prescription (p-value range: 0.14–0.77). After correction for general patient characteristics (age, sex, season, comorbidity, referral), disease severity (triage urgency, fever duration, fever measured at ED, alarming signs), diagnostics, focus of infection, and cause of infection, variability of antibiotic prescriptions remained between EDs in the adjusted model (range of standardised prescription rates: 0.77–1.35; MOR 2.41). Variation was also observed for intravenous versus oral administration (range of standardised rates: 0.29–1.31; MOR 2.60) and prescription of broad-spectrum antibiotics versus narrow-spectrum antibiotics (range of standardised rates: 0.65–1.75; MOR 3.20). Stratified for comorbidity, standardised antibiotic prescription rates and broad-spectrum rates were comparable in children with and without comorbidity. Higher standardised rates for broad-spectrum antibiotics were not related to higher antimicrobial resistance percentages on a national level or on a hospital level (S8 Text). Results of variation of antibiotic and broad-spectrum prescriptions for RTIs are provided in S10 Text.

[Figure omitted. See PDF.]

Fig 3. Heat map of standardised prescription rates by ED (95% CI).

All adjusted for age, sex, season, comorbidity, referral, triage urgency, fever measured at ED, fever duration, alarming signs, CRP, chest X-ray, urinalysis, focus of infection, and cause of infection. EDs are ordered according to standardised antibiotic prescribing rate, from low to high on the left vertical axis. The coloured boxes represent rank of standardised rate for each ED: Red indicates rates > 1, blue indicates rates < 1, and rates equal to 1 are white. *Also adjusted for previous antibiotic use. ED, emergency department; NL, the Netherlands; UK, United Kingdom.

https://doi.org/10.1371/journal.pmed.1003208.g003

Variation of antibiotic and broad-spectrum prescriptions in viral infections, bacterial infections, and unknown bacterial/viral infections

The antibiotic prescription rate was 6.9% (1,418/20,383) for presumed viral infections (range across EDs: 0.4%–18.9%), 88.8% (1,289/1,451) for definite bacterial infections (range across EDs: 83.5%–96.2%), 94.7% (6,097/6,438) for probable bacterial/bacterial syndrome infections (range across EDs: 81.2%–99.3%), and 45.2% (2,348/5,200) for unknown bacterial/viral infections (range across EDs: 1.7%–79.3%) (S2 Fig).

Adjusted for general characteristics, disease severity, diagnostics, and focus of infection, we observed variation for antibiotic prescriptions in presumed viral infections (range of standardised rates: 0.05–3.29; MOR 4.91) and unknown bacterial/viral infections (range of standardised rates: 0.05–1.78; MOR 4.78) (Fig 4). Antibiotic prescriptions varied less for patients with presumed bacterial infections (range of standardised rates: 0.91–1.06; MOR 2.32). The proportion of broad-spectrum prescriptions was 74.1% (1,037/1,399) for presumed viral infections (range across EDs: 38.9%–91.4%), 68.5% (880/1,284) for definite bacterial infections (range across EDs: 39.2%–96.0%), 43.2% for probable bacterial/bacterial syndrome infections (2,628/6,081, range across EDs: 28.5%–86.3%), and 51.6% (1,191/2,306) for unknown bacterial/viral infections (range across EDs: 20.0%–95.7%) (S2 Fig). After adjustment, differences for broad-spectrum versus narrow-spectrum antibiotics remained for presumed viral infections (range of standardised rates: 0.57–1.54; MOR 2.59), presumed bacterial infections (range of standardised rates: 0.66–1.86; MOR 3.09), and unknown bacterial/viral infections (range of standardised rates: 0.44–1.64; MOR 3.70) (S3 Fig).

[Figure omitted. See PDF.]

Fig 4. Heat map of standardised antibiotic prescription rates by ED for presumed viral, presumed bacterial, and unknown bacterial/viral infections (95% CI).

All adjusted for age, sex, season, comorbidity, referral, triage urgency, fever duration, alarming signs, CRP, chest X-ray, urinalysis, and focus of infection. EDs are ordered according to standardised antibiotic prescribing rate, from low to high on the left vertical axis. The coloured boxes represent rank of standardised rate for each ED: Red indicates rates > 1, blue indicates rates < 1, and rates equal to 1 are white. ED, emergency department; NL, the Netherlands; UK, United Kingdom.

https://doi.org/10.1371/journal.pmed.1003208.g004

Variation in prescriptions of appropriate indication and appropriate duration

Of all antibiotic prescriptions, 65.0% (7,386/11,371) were determined to be likely appropriate (range across EDs: 23.7%–98.9%), 12.5% (1,418/11,371) were likely inappropriate (range across EDs: 0.6%–29.3%), and 22.6% (2,567/11,371) were inconclusive (range across EDs: 0.5%–61.7%).

Oral antibiotic prescriptions with inappropriate duration were found in 20.0% (1,525/7,636) of prescriptions, and this ranged from 4.4% to 59.0% across EDs (Fig 5). Of all oral antibiotic prescriptions, 2.1% (134/7,636) were of both inappropriate indication and inappropriate duration (range across EDs: 0.0%–8.4%), whereas 30.0% (2,294/7,636) were either of inappropriate indication or of inappropriate duration (range across EDs: 11.3%–69.9%).

[Figure omitted. See PDF.]

Fig 5. Heat map of inappropriateness of antibiotic prescriptions across EDs.

EDs are ordered according to proportion of inappropriately indicated prescriptions, from low to high on the left vertical axis. The coloured boxes represent rank of proportion for each ED: Red indicates the highest proportion, and white indicates the lowest proportion. ED, emergency department; NL, the Netherlands; UK, United Kingdom.

https://doi.org/10.1371/journal.pmed.1003208.g005

Variation of appropriate prescriptions in uncomplicated RTIs and urinary tract infections

In uncomplicated RTIs, oral prescriptions were not guideline-concordant in 22.3% (973/4,373) of prescriptions (range across EDs: 11.8%–47.3%) (Fig 5). In this group, the proportion of prescriptions that were inappropriate in all 3 measures (indication, duration, and guideline concordance) was 0.7% (31/4,373), whilst 42.3% (1,850/4,373) were inappropriate in any of the 3 measures (range across EDs: 15.7%–80.9%). In uncomplicated urinary tract infections, oral prescriptions were not concordant with the local guideline in 45.1% of prescriptions (152/337) (range across EDs: 11.1%–100%), and 65.9% (222/337) were inappropriate in any of the 3 measures (range across EDs: 11.1%–100%).

Discussion

In this large prospective multicentre study, we found diversity in antibiotic prescriptions, and in particular broad-spectrum antibiotic prescriptions, for febrile children attending different EDs in Europe. After adjustment for general characteristics, disease severity, diagnostics, and focus of infection, we observed minor variation in antibiotic prescriptions for bacterial infections, and larger variability in antibiotic prescriptions for viral infections and unknown bacterial/viral infections. Moreover, one-third of all antibiotic prescriptions were of inappropriate or inconclusive indication, and 20% of oral prescriptions were of inappropriate duration, with large variation across EDs. Between EDs, the proportion of oral prescriptions that were not concordant with the local guideline varied from 12% to 47% in RTIs and from 11% to 100% in urinary tract infections.

Our study supports previous studies that reported variable antibiotic prescribing for all febrile children, but found less variation than a previous study in children with RTIs across 28 European EDs (range of standardised rates: 0.5–2.0) [5]. In contrast to this study, our study corrected for aetiology of infection—bacterial, viral, or unknown—based on a standardised flowchart. Studies in the US on diversity in outpatient antibiotic prescribing found regional differences in both antibiotic and broad-spectrum prescribing [6,10,17]. These studies, however, were not focused on ED visits alone since all ambulatory visits were included.

The Access, Watch, and Reserve (AWaRE) classification has recently been used to classify global antibiotic prescriptions in 2 studies assessing oral formulations and use of inpatient antibiotics in children [14,44,45]. These studies confirmed variable patterns of antibiotic prescribing between countries, but did not adjust for differences in population and did not report data of emergency care visits. Further, the AWaRE classification led to a substantial proportion of unclassified antibiotics in our study population (12.2%; range across EDs: 1.9%–26.9%) and absence of the reserve category.

Previous studies in the US have evaluated appropriateness of antibiotic prescribing in children defined by ICD codes. Poole et al. [46] found that prescriptions were in general not indicated in 32% of emergency care visits in children. Additionally, overall prescription of first-line antibiotics (amoxicillin, amoxicillin-clavulanate) ranged from 50% to 78% for RTIs in children [46–48]. We found a similar rate of guideline-concordant prescriptions in RTIs (78%), whilst guideline concordance was defined differently for most EDs: amoxicillin and narrow-spectrum penicillins according to the local guideline. One ED (UK, 3) used amoxicillin-clavulanate as first-line for RTIs. Our study is the first to our knowledge to evaluate appropriateness of antibiotic prescribing in febrile children visiting different EDs in Europe, using a structured flowchart categorising viral, bacterial, and unclassified infections, and taking local guidelines into account.

Strengths of this European multicentre study include the large sample size, detailed patient information, recruitment in a diverse range of ED settings in 8 EU countries, and recruitment over a full year to reflect seasonal variation. Furthermore, a rigorous, standardised structured assessment of all cases was carried out to establish the presumed cause of infection, using a consensus-based flowchart taking into account clinical syndrome, CRP, and culture results. Previous studies have addressed appropriate prescribing for diagnoses based on ICD codes. This classification, however, may not accurately take into account bacterial or viral aetiology [6,10,17,46]. Our large sample size enabled adjustment for hospital- and patient-level factors influencing antibiotic use in the EDs [19].

This study has some limitations. First, the included EDs are not representative of all febrile children attending the ED in that country. The EDs participating in this study are university hospitals or large teaching centres with intensive care unit facilities involved in paediatric infectious disease research collaborations. Fever and sepsis guidelines were available in all EDs [19]. Therefore, these EDs represent a high standard of care, and generalisation of our findings to smaller hospitals or to a regional or national level should be undertaken with caution. However, we corrected for the most important confounders including comorbidities, multiple markers of disease severity, and focus and presumed cause of infection. Second, although the experience of the physician (resident or consultant) and clinician specialty are related with antibiotic prescription [49,50], we could not adjust for physician background at the patient level. However, we evaluated the contribution of supervision to antibiotic prescription at the hospital level. In our study, supervision was not related to antibiotic prescription. Our efforts to improve data quality by training clinical and research staff might have influenced common clinical practice. Since this training focused on awareness of alarming signs in the clinical assessment of the febrile child, it is unlikely that it influenced antibiotic prescription. Furthermore, we only included the first visit of patients who repeatedly visited the ED, since data collection did not include secondary visits in all EDs.

Differences in antibiotic prescribing could be influenced by differences in immunisation coverage. In our study, countries with lower coverage for pneumococcal vaccinations (<90%) (Germany, Slovenia) did not have higher antibiotic prescriptions at the ED [19,51].

We found large variation in broad-spectrum prescriptions across the different EDs. Increased antimicrobial resistance rates could possibly explain higher broad-spectrum prescribing. We compared broad-spectrum rates with national data for antimicrobial resistance and hospital methicillin resistance rates. Interestingly, EDs based in countries with higher antimicrobial resistance on a population level (e.g., Greece, Spain) prescribed less broad-spectrum agents than expected in the ED. These hospitals with higher burden of national antimicrobial resistance may perceive more problems with antimicrobial resistance and might feel a greater pressure to reduce antibiotic prescriptions in the ED. It should be noted that antibiotic prescribing in the ED will not be representative of antibiotic prescription patterns of primary care in the community.

The diversity in antibiotic prescribing across different EDs appears not to be associated with antimicrobial resistance or immunisation coverage. Although the ideal antibiotic prescription rate is unknown, the diversity in antibiotic prescribing suggests overprescribing. Prescription rates were above the average incidence of serious bacterial infections. We found variation in antibiotic prescription rates, even when adjusting for general characteristics, disease severity, diagnostics, and focus and cause of infection.

This suggests room for improvement in reduction of antibiotic prescriptions and especially broad-spectrum prescriptions at the ED. EDs with higher antibiotic prescription rates did not necessarily prescribe more broad-spectrum antibiotics. The ED with the highest standardised broad-spectrum rate (UK, 3) did not have a high proportion of inappropriate prescriptions for RTIs. In only this ED, amoxicillin-clavulanate (broad-spectrum) was the first-choice agent for uncomplicated RTIs, which could explain the higher broad-spectrum rate in this ED. Studies demonstrated that use of narrow-spectrum antibiotics compared to broad-spectrum antibiotics leads to similar clinical outcomes and to fewer adverse events [28,52]. Unnecessary use of broad-spectrum antibiotics potentially increases resistance rates even further.

In addition, diversity of antibiotic prescription increased with diagnostic uncertainty. After adjustment for general characteristics, disease severity, diagnostics, and focus of infection, we observed minor variation in antibiotic prescriptions for bacterial infections, and larger variability in antibiotic prescriptions for viral infections and unknown bacterial/viral infections. In general, EDs with higher antibiotic prescription rates in viral infections also had higher antibiotic prescription rates in unknown bacterial/viral infections. This indicates that overprescribing in viral infections is linked to higher prescriptions in unknown bacterial/viral infections. Diagnostic uncertainty in patients with an unclear cause of infection could be reduced by improved targeted antibiotic prescription from new diagnostic signatures of bacterial and viral infection.

We evaluated appropriateness in indication, duration, and guideline concordance. Ideally, EDs should target 100% appropriateness in these 3 aspects of antibiotic prescribing. In our study, we did not observe a clear association between inappropriately indicated prescriptions and prescriptions of inappropriate duration. This indicates that guideline implementations should focus on these different aspects of appropriate antibiotic prescribing to ensure prescriptions of appropriate indication, duration, and antibiotic selection. Furthermore, quality improvement initiatives should be emphasised in EDs with higher proportions of inappropriate prescriptions. In addition, future antimicrobial stewardship interventions across Europe should focus on reducing broad-spectrum treatment and antibiotic use in viral infections.

To conclude, we found substantial variation in antibiotic prescriptions and especially broad-spectrum antibiotic prescriptions in European EDs after adjustment for patient characteristics, disease severity, diagnostics, and focus and cause of infection. The proportion of antibiotic prescriptions in bacterial infections was comparable between EDs, but diversity was especially large in antibiotic prescriptions for viral infections and unknown viral/bacterial infections. This variation indicates overprescription of antibiotics in these groups of patients. Furthermore, indications of prescriptions were inappropriate or inconclusive in one-third of prescriptions, and this proportion varied between EDs. In respiratory and urinary infections, guideline concordance of prescriptions varied widely across EDs. Until better diagnostics are available to accurately differentiate between bacterial and viral aetiologies, we strongly urge the implementation of antimicrobial stewardship guidelines to reduce antibiotic prescription in febrile children across Europe.

Supporting information

S1 Fig. Duration of prescribed antibiotics.

https://doi.org/10.1371/journal.pmed.1003208.s001

(PDF)

S2 Fig. Range of antibiotic prescriptions and broad-spectrum prescriptions by emergency department (ED) for viral, bacterial, and unknown bacterial/viral infections.

(A) antibiotic prescriptions; (B) broad-spectrum prescriptions.

https://doi.org/10.1371/journal.pmed.1003208.s002

(PDF)

S3 Fig. Heat map of standardised broad-spectrum versus narrow-spectrum rates for viral, bacterial, or unknown bacterial/viral infections.

https://doi.org/10.1371/journal.pmed.1003208.s003

(PDF)

S1 Text. STROBE checklist.

https://doi.org/10.1371/journal.pmed.1003208.s004

(PDF)

S2 Text. Statistical analysis plan.

https://doi.org/10.1371/journal.pmed.1003208.s005

(PDF)

S3 Text. Ethics committees of participating hospitals.

https://doi.org/10.1371/journal.pmed.1003208.s006

(PDF)

S4 Text. Hospital characteristics.

https://doi.org/10.1371/journal.pmed.1003208.s007

(PDF)

S5 Text. Broad-spectrum and narrow-spectrum antibiotic definitions.

https://doi.org/10.1371/journal.pmed.1003208.s008

(PDF)

S6 Text. Local guidelines of antibiotic treatment.

https://doi.org/10.1371/journal.pmed.1003208.s009

(PDF)

S7 Text. Details of the adjusted model.

https://doi.org/10.1371/journal.pmed.1003208.s010

(PDF)

S8 Text. National and hospital antimicrobial resistance data.

https://doi.org/10.1371/journal.pmed.1003208.s011

(PDF)

S9 Text. Descriptive characteristics of cases with complete outcomes and cases with missing outcomes.

https://doi.org/10.1371/journal.pmed.1003208.s012

(PDF)

S10 Text. Variation of antibiotic and broad-spectrum prescription in lower RTIs and otitis media, tonsillitis/pharyngitis, and other upper RTIs.

https://doi.org/10.1371/journal.pmed.1003208.s013

(PDF)

S11 Text. PERFORM consortium.

https://doi.org/10.1371/journal.pmed.1003208.s014

(PDF)

Acknowledgments

We acknowledge all research nurses for their help in collecting data, and Anda Nagle (Riga) and the Institute of Microbiology at University Medical Centre Ljubljana for their help in collecting data on antimicrobial resistance. Members of the PERFORM consortium are listed in S11 Text.

Citation: Hagedoorn NN, Borensztajn DM, Nijman R, Balode A, von Both U, Carrol ED, et al. (2020) Variation in antibiotic prescription rates in febrile children presenting to emergency departments across Europe (MOFICHE): A multicentre observational study. PLoS Med 17(8): e1003208. https://doi.org/10.1371/journal.pmed.1003208

References

1. Poropat F, Heinz P, Barbi E, Ventura A. Comparison of two European paediatric emergency departments: does primary care organisation influence emergency attendance? Ital J Pediatr. 2017;43(1):29. pmid:28274237

2. Sands R, Shanmugavadivel D, Stephenson T, Wood D. Medical problems presenting to paediatric emergency departments: 10 years on. Emerg Med J. 2012;29(5):379–82. pmid:21609944

3. Craig JC, Williams GJ, Jones M, Codarini M, Macaskill P, Hayen A, et al. The accuracy of clinical symptoms and signs for the diagnosis of serious bacterial infection in young febrile children: prospective cohort study of 15 781 febrile illnesses. BMJ. 2010;340:c1594. pmid:20406860

4. Nijman RG, Vergouwe Y, Thompson M, van Veen M, van Meurs AH, van der Lei J, et al. Clinical prediction model to aid emergency doctors managing febrile children at risk of serious bacterial infections: diagnostic study. BMJ. 2013;346:f1706. pmid:23550046

5. van de Maat J, van de Voort E, Mintegi S, Gervaix A, Nieboer D, Moll H, et al. Antibiotic prescription for febrile children in European emergency departments: a cross-sectional, observational study. Lancet Infect Dis. 2019;19:382–91. pmid:30827808

6. Hersh AL, Shapiro DJ, Pavia AT, Shah SS. Antibiotic prescribing in ambulatory pediatrics in the United States. Pediatrics. 2011;128(6):1053–61. pmid:22065263

7. Vaz LE, Kleinman KP, Raebel MA, Nordin JD, Lakoma MD, Dutta-Linn MM, et al. Recent trends in outpatient antibiotic use in children. Pediatrics. 2014;133(3):375–85. pmid:24488744

8. Goossens H, Ferech M, Vander Stichele R, Elseviers M, ESAC Project Group. Outpatient antibiotic use in Europe and association with resistance: a cross-national database study. Lancet. 2005;365(9459):579–87. pmid:15708101

9. Costelloe C, Metcalfe C, Lovering A, Mant D, Hay AD. Effect of antibiotic prescribing in primary care on antimicrobial resistance in individual patients: systematic review and meta-analysis. BMJ. 2010;340:c2096. pmid:20483949

10. Fleming-Dutra KE, Hersh AL, Shapiro DJ, Bartoces M, Enns EA, File TM Jr, et al. Prevalence of inappropriate antibiotic prescriptions among US ambulatory care visits, 2010–2011. JAMA. 2016;315(17):1864–73. pmid:27139059

11. European Centre for Disease Prevention and Control. Surveillance of antimicrobial resistance in Europe 2017—annual report of the European Antimicrobial Resistance Surveillance Network (EARS-Net). Solna (Sweden): European Centre for Disease Prevention and Control; 2018 [cited 2019 Mar 15]. Available from: https://ecdc.europa.eu/sites/portal/files/documents/EARS-Net-report-2017-update-jan-2019.pdf.

12. Van Boeckel TP, Gandra S, Ashok A, Caudron Q, Grenfell BT, Levin SA, et al. Global antibiotic consumption 2000 to 2010: an analysis of national pharmaceutical sales data. Lancet Infect Dis. 2014;14(8):742–50. pmid:25022435

13. Zanichelli V, Monnier AA, Gyssens IC, Adriaenssens N, Versporten A, Pulcini C, et al. Variation in antibiotic use among and within different settings: a systematic review. J Antimicrob Chemother. 2018;73(Suppl 6):vi17–29.

14. Hsia Y, Sharland M, Jackson C, Wong ICK, Magrini N, Bielicki JA. Consumption of oral antibiotic formulations for young children according to the WHO Access, Watch, Reserve (AWaRe) antibiotic groups: an analysis of sales data from 70 middle-income and high-income countries. Lancet Infect Dis. 2019;19(1):67–75. pmid:30522834

15. Sarpong EM, Miller GE. Narrow- and broad-spectrum antibiotic use among U.S. children. Health Serv Res. 2015;50(3):830–46. pmid:25424240

16. van de Voort EMF, Mintegi S, Gervaix A, Moll HA, Oostenbrink R. Antibiotic use in febrile children presenting to the emergency department: a systematic review. Front Pediatr. 2018;6:260. pmid:30349814

17. Hicks LA, Bartoces MG, Roberts RM, Suda KJ, Hunkler RJ, Taylor TH Jr, et al. US outpatient antibiotic prescribing variation according to geography, patient population, and provider specialty in 2011. Clin Infect Dis. 2015;60(9):1308–16. pmid:25747410

18. PERFORM. About PERFORM: Personalised Risk assessment in Febrile illness to Optimise Real-life Management London: PERFORM; 2019 [cited 2019 Sep 10]. Available from: https://www.perform2020.org/.

19. Borensztajn D, Yeung S, Hagedoorn NN, Balode A, von Both U, Carrol ED, et al. Diversity in the emergency care for febrile children in Europe: a questionnaire study. BMJ Paediatr Open. 2019;3(1):e000456. pmid:31338429

20. Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996;49(12):1373–9. pmid:8970487

21. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–81. pmid:18929686

22. Simon TD, Cawthon ML, Stanford S, Popalisky J, Lyons D, Woodcox P, et al. Pediatric medical complexity algorithm: a new method to stratify children by medical complexity. Pediatrics. 2014;133(6):e1647–54. pmid:24819580

23. National Institute for Health and Care Excellence. Fever in under 5s: assessment and initial management. Clinical guideline CG160. London: National Institute for Health and Care Excellence; 2013 [cited 2020 Jul 30]. Available from: https://www.nice.org.uk/guidance/cg160.

24. van Houten CB, Oved K, Eden E, Cohen A, Engelhard D, Boers S, et al. Observational multi-centre, prospective study to characterize novel pathogen-and host-related factors in hospitalized patients with lower respiratory tract infections and/or sepsis—the “TAILORED-Treatment” study. BMC Infect Dis. 2018;18(1):377. pmid:30086729

25. Herberg JA, Kaforou M, Wright VJ, Shailes H, Eleftherohorinou H, Hoggart CJ, et al. Diagnostic test accuracy of a 2-transcript host RNA signature for discriminating bacterial vs viral infection in febrile children. JAMA. 2016;316(8):835–45. pmid:27552617

26. Martinon-Torres F, Salas A, Rivero-Calle I, Cebey-Lopez M, Pardo-Seco J, Herberg JA, et al. Life-threatening infections in children in Europe (the EUCLIDS Project): a prospective cohort study. Lancet Child Adolesc Health. 2018;2(6):404–14. pmid:30169282

27. Aabenhus R, Siersma V, Hansen MP, Bjerrum L. Antibiotic prescribing in Danish general practice 2004–13. J Antimicrob Chemother. 2016;71(8):2286–94. pmid:27107098

28. Gerber JS, Ross RK, Bryan M, Localio AR, Szymczak JE, Wasserman R, et al. Association of broad- vs narrow-spectrum antibiotics with treatment failure, adverse events, and quality of life in children with acute respiratory tract infections. JAMA. 2017;318(23):2325–36. pmid:29260224

29. National Institute for Health and Care Excellence. Antimicrobial prescribing guidelines. London: National Institute for Health and Care Excellence; 2020 [cited 2020 Feb 17]. Available from: https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/antimicrobial-prescribing-guidelines.

30. Public Health England, National Institute for Health and Care Excellence. Managing common infections: guidance for primary care. London: Public Health England; 2019 [cited 2020 Feb 17]. Available from: https://www.gov.uk/government/publications/managing-common-infections-guidance-for-primary-care.

31. Pouwels KB, Hopkins S, Llewelyn MJ, Walker AS, McNulty CA, Robotham JV. Duration of antibiotic treatment for common infections in English primary care: cross sectional analysis and comparison with guidelines. BMJ. 2019;364:l440. pmid:30814052

32. von Hippel PT. 4. Regression with missing Ys: an improved strategy for analyzing multiply imputed data. Sociol Methodol. 2007;37(1):83–117.

33. Bates D, Maechler M, Bolker B, Walker S. Fitting linear mixed-effects models using lme4. J Stat Softw. 2015;67:1–48.

34. Van den Bruel A, Haj-Hassan T, Thompson M, Buntinx F, Mant D, European Research Network on Recognising Serious Infection investigators. Diagnostic value of clinical features at presentation to identify serious infection in children in developed countries: a systematic review. Lancet. 2010;375(9717):834–45. pmid:20132979

35. Galetto-Lacour A, Zamora SA, Andreola B, Bressan S, Lacroix L, Da Dalt L, et al. Validation of a laboratory risk index score for the identification of severe bacterial infection in children with fever without source. Arch Dis Child. 2010;95(12):968–73. pmid:20515973

36. Van den Bruel A, Thompson MJ, Haj-Hassan T, Stevens R, Moll H, Lakhanpaul M, et al. Diagnostic value of laboratory tests in identifying serious infections in febrile children: systematic review. BMJ. 2011;342:d3082. pmid:21653621

37. Gidengil CA, Linder JA, Hunter G, Setodji C, Mehrotra A. The volume-quality relationship in antibiotic prescribing: when more isn’t better. Inquiry. 2015;52:0046958015571130. pmid:25672338

38. Khine H, Goldman DL, Avner JR. Management of fever in postpneumococcal vaccine era: comparison of management practices by pediatric emergency medicine and general emergency medicine physicians. Emerg Med Int. 2014;2014:702053. pmid:24982807

39. Bird C, Winzor G, Lemon K, Moffat A, Newton T, Gray J. A pragmatic study to evaluate the use of a rapid diagnostic test to detect group A streptococcal pharyngitis in children with the aim of reducing antibiotic use in a UK emergency department. Pediatr Emerg Care. 2018 Jul 24. pmid:30045356

40. Lemiengre MB, Verbakel JY, Colman R, De Burghgraeve T, Buntinx F, Aertgeerts B, et al. Reducing inappropriate antibiotic prescribing for children in primary care: a cluster randomised controlled trial of two interventions. Br J Gen Pract. 2018;68(668):e204–10. pmid:29440016

41. Sanagou M, Wolfe R, Forbes A, Reid CM. Hospital-level associations with 30-day patient mortality after cardiac surgery: a tutorial on the application and interpretation of marginal and multilevel logistic regression. BMC Med Res Methodol. 2012;12:28. pmid:22409732

42. Di Martino M, Lallo A, Kirchmayer U, Davoli M, Fusco D. Prevalence of antibiotic prescription in pediatric outpatients in Italy: the role of local health districts and primary care physicians in determining variation. A multilevel design for healthcare decision support. BMC Public Health. 2017;17(1):886. pmid:29149875

43. World Health Organization. Prioritization of pathogens to guide discovery, research and development of new antibiotics for drug-resistant bacterial infections, including tuberculosis. Geneva: World Health Organization; 2017 [cited 2019 Mar 25]. Available from: https://www.who.int/medicines/areas/rational_use/PPLreport_2017_09_19.pdf?ua=1.

44. World Health Organization. WHO model list of essential medicines for children. Geneva: World Health Organization; 2017.

45. Hsia Y, Lee BR, Versporten A, Yang Y, Bielicki J, Jackson C, et al. Use of the WHO Access, Watch, and Reserve classification to define patterns of hospital antibiotic use (AWaRe): an analysis of paediatric survey data from 56 countries. Lancet Glob Health. 2019;7(7):e861–71. pmid:31200888

46. Poole NM, Shapiro DJ, Fleming-Dutra KE, Hicks LA, Hersh AL, Kronman MP. Antibiotic prescribing for children in United States emergency departments: 2009–2014. Pediatrics. 2019;143(2):e20181056. pmid:30622156

47. Hersh AL, Fleming-Dutra KE, Shapiro DJ, Hyun DY, Hicks LA, Outpatient Antibiotic Use Target-Setting Workgroup. Frequency of first-line antibiotic selection among US ambulatory care visits for otitis media, sinusitis, and pharyngitis. JAMA Intern Med. 2016;176(12):1870–2. pmid:27775770

48. Palms DL, Hicks LA, Bartoces M, Hersh AL, Zetts R, Hyun DY, et al. First-line antibiotic selection in outpatient settings. Antimicrob Agents Chemother. 2019;63(11):e01060–19. pmid:31548186

49. Frost HM, McLean HQ, Chow BDW. Variability in antibiotic prescribing for upper respiratory illnesses by provider specialty. J Pediatr. 2018;203:76–85.e8. pmid:30195553

50. Agiro A, Gautam S, Wall E, Hackell J, Helm M, Barron J, et al. Variation in outpatient antibiotic dispensing for respiratory infections in children by clinician specialty and treatment setting. Pediatr Infect Dis J. 2018;37(12):1248–54. pmid:30408006

51. World Health Organization. WHO UNICEF review of national immunization coverage, 1980–2017. Geneva: World Health Organization; 2019 [cited 2019 Mar 22]. Available from: http://apps.who.int/immunization_monitoring/globalsummary/wucoveragecountrylist.html.

52. Williams DJ, Hall M, Shah SS, Parikh K, Tyler A, Neuman MI, et al. Narrow vs broad-spectrum antimicrobial therapy for children hospitalized with pneumonia. Pediatrics. 2013;132(5):e1141–8. pmid:24167170