Oxygen systems and quality of care for children with pneumonia, malaria and diarrhoea: Analysis of a stepped-wedge trial in Nigeria

Abstract

Objectives

To evaluate the effect of improved hospital oxygen systems on quality of care (QOC) for children with severe pneumonia, severe malaria, and diarrhoea with severe dehydration.

Design

Stepped-wedge cluster randomised trial (unblinded), randomised at hospital-level.

Setting

12 hospitals in south-west Nigeria.

Participants

7,141 children (aged 28 days to 14 years) admitted with severe pneumonia, severe malaria or diarrhoea with severe dehydration between January 2014 and October 2017.

Interventions

Phase 1 (pulse oximetry) introduced pulse oximetry for all admitted children. Phase 2 (full oxygen system) (i) standardised oxygen equipment package, (ii) clinical education and support, (iii) technical training and support, and (iv) infrastructure and systems support.

Outcome measures

We used quantitative QOC scores evaluating assessment, diagnosis, treatment, and monitoring practices against World Health Organization and Nigerian standards. We evaluated mean differences in QOC scores between study periods (baseline, oximetry, full oxygen system), using mixed-effects linear regression.

Results

7,141 eligible participants; 6,893 (96.5%) had adequate data for analysis. Mean paediatric QOC score (maximum 6) increased from 1.64 to 3.00 (adjusted mean difference 1.39; 95% CI 1.08–1.69, p<0.001) for severe pneumonia and 2.81 to 4.04 (aMD 1.53; 95% CI 1.23–1.83, p<0.001) for severe malaria, comparing the full intervention to baseline, but did not change for diarrhoea with severe dehydration (aMD -0.12; 95% CI -0.46–0.23, p = 0.501). After excluding practices directly related to pulse oximetry and oxygen, we found aMD 0.23 for severe pneumonia (95% CI -0.02–0.48, p = 0.072) and 0.65 for severe malaria (95% CI 0.41–0.89, p<0.001) comparing full intervention to baseline. Sub-analysis showed some improvements (and no deterioration) in care processes not directly related to oxygen or pulse oximetry.

Conclusion

Improvements in hospital oxygen systems were associated with higher QOC scores, attributable to better use of pulse oximetry and oxygen as well as broader improvements in clinical care, with no negative distortions in care practices.

Trial registration

ACTRN12617000341325

About the Authors:

Hamish R. Graham

Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Supervision, Visualization, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliations Centre for International Child Health, The Royal Children’s Hospital, MCRI, University of Melbourne, Parkville, Australia, Department of Paediatrics, University College Hospital, Ibadan, Nigeria

Jaclyn Maher

Roles Formal analysis, Investigation, Methodology, Visualization, Writing – original draft, Writing – review & editing

Affiliation: Department of Paediatrics, Royal Children’s Hospital, University of Melbourne, Parkville, Australia

ORCID logo https://orcid.org/0000-0002-1209-5081

Ayobami A. Bakare

Roles Data curation, Methodology, Project administration, Writing – review & editing

Affiliation: Department of Paediatrics, University College Hospital, Ibadan, Nigeria

Cattram D. Nguyen

Roles Formal analysis, Methodology, Writing – review & editing

Affiliations Department of Paediatrics, Royal Children’s Hospital, University of Melbourne, Parkville, Australia, Clinical Epidemiology and Biostatistics Unit, MCRI, Royal Children’s Hospital, Parkville, Australia

Adejumoke I. Ayede

Roles Project administration, Writing – review & editing

Affiliations Department of Paediatrics, University College Hospital, Ibadan, Nigeria, Department of Paediatrics, University of Ibadan, Ibadan, Nigeria

Oladapo B. Oyewole

Roles Data curation, Writing – review & editing

Affiliation: Department of Paediatrics, University College Hospital, Ibadan, Nigeria

Amy Gray

Roles Methodology, Writing – review & editing

Affiliation: Centre for International Child Health, The Royal Children’s Hospital, MCRI, University of Melbourne, Parkville, Australia

Rasa Izadnegahdar

Roles Funding acquisition, Writing – review & editing

Affiliations Bill and Melinda Gates Foundation, Seattle, Washington, United States of America, Department of Pediatrics, University of Washington, Seattle, Washington, United States of America

Trevor Duke

Roles Conceptualization, Funding acquisition, Methodology, Supervision, Writing – review & editing

Affiliation: Centre for International Child Health, The Royal Children’s Hospital, MCRI, University of Melbourne, Parkville, Australia

Adegoke G. Falade

Roles Conceptualization, Data curation, Funding acquisition, Methodology, Project administration, Resources, Supervision, Writing – review & editing

Affiliations Department of Paediatrics, University College Hospital, Ibadan, Nigeria, Department of Paediatrics, University of Ibadan, Ibadan, Nigeria

Introduction

Effective oxygen systems are an important element of quality hospital care and are critical for achieving good clinical outcomes for patients with severe pneumonia [1, 2]. Improvements in hospital oxygen systems have been shown to reduce inpatient child mortality from pneumonia by 35–58%, yet oxygen access for patients remains poor in many low- and middle-income countries (LMICs) [3–8]. The COVID-19 pandemic has further highlighted the importance of improving hospital oxygen systems, and the fatal consequences of deficient systems, pushing oxygen access higher onto the global health agenda [9, 10].

Improvement in hospital oxygen systems requires attention to multiple domains which are relevant to the overall quality of care in health facilities, including structures (e.g. equipment, policy, financing), care processes, (e.g. detections of hypoxaemia and appropriate use of oxygen) and therefore outcomes (e.g. case fatality) [4, 11, 12]. This includes activities to increase availability of oxygen (e.g. oxygen equipment procurement, maintenance and repair), improve clinical use of oxygen (e.g. routine pulse oximetry and clinical guidelines) and connect relevant staff cadres (e.g. doctors, nurses, technicians, managers).

Findings from previous oxygen improvement programs suggest that clinical impacts are more significant if oxygen is approached as part of a more comprehensive quality improvement program [3, 6, 8]. Authors postulated that improvements in oxygen systems may have contributed to broader improvements in structures and processes of care, thereby enabling the oxygen improvement program to achieve more substantial reductions in mortality, than would have been possible with a narrower approach to oxygen systems [3, 6]. However, it is also possible that broad-reaching interventions to improve oxygen systems could have unintended negative effects on quality of care.

Nigeria is a populous lower middle-income country in sub-Saharan Africa with a mortality rate in children under five years of age (U-5) of 120 deaths per 1000 live births—the second highest rate globally in 2018 [13, 14]. Nigeria contributes one-third of U-5 malaria deaths and one-sixth of U-5 pneumonia deaths globally [15]. Hospital oxygen systems in Nigeria are generally weak, with recent studies showing that only one in five children with evidence of low blood oxygen (hypoxaemia) received oxygen therapy, despite hypoxaemia being common (10% of hospitalised children had a blood oxygen saturation <90%) and fatal [16, 17].

The Nigeria Oxygen Implementation program implemented and evaluated improved oxygen systems in 12 secondary health facilities in south-west Nigeria. Results are published elsewhere [5, 16–22]. We found that the introduction of pulse oximetry and improved oxygen systems contributed to marked improvements in pulse oximetry and oxygen practices and was associated with reduced risk of death for children admitted with pneumonia, but not other illnesses [5]. Importantly, we observed the greatest impact on pneumonia mortality with the introduction of pulse oximetry (prior to implementing the full oxygen system) and received qualitative feedback that pulse oximetry may have had broader positive effects on other care practices [5, 19]. However, we do not know the extent to which our multi-faceted oxygen improvement program may have impacted (positively or negatively) on broader quality of care.

This paper seeks to evaluate the effect of a multi-faceted oxygen system intervention on the quality of care provided for children (aged 1 month to 14 years) admitted to hospital with severe pneumonia, severe malaria and diarrhoea with severe dehydration, and explore potential positive and negative effects beyond oxygen-related aspects of care.

Methods

Ethics statement

We obtained ethics approval from the University of Melbourne (1543797.1) and University of Ibadan/University College Hospital Ethics Committee, Ibadan, Nigeria (UI/EC/16/0413). Patients whose records were accessed during this study sought treatment between November 2013 and October 2017. Both retrospective and prospective data collection occurred between September 2015 and December 2017. Data was fully anonymised upon access by the authors. The ethics committees did not require us to have individual patient consent.

Study design

This study was a stepped-wedge cluster randomised trial conducted in 12 Nigerian hospitals. Following a pre-specified timetable, we introduced pulse oximetry to all 12 hospitals in November 2015, followed by stepped introduction of the full oxygen system—clusters of three hospitals, every four months between March 2016 and March 2017, with gradual withdrawal of support to encourage full site self-sustainability by the end of 2019 (Fig 1, S1 Fig). Risk of contamination was minimised by the geographic separation of hospitals. Full study details are available elsewhere [5, 18].

[Figure omitted. See PDF.]

Fig 1. Participant inclusion in analysis, by diagnosis (all steps).

https://doi.org/10.1371/journal.pone.0254229.g001

Sample

We selected hospitals in collaboration with state health authorities, identifying secondary-level health facilities that admitted children (see study protocol for detail: DOI 10.1186/s13063-017-2241-8) [18]. The 12 hospitals included a mix of government and mission (private non-profit) hospitals across four states in south-west Nigeria, including three hospitals that focussed on paediatric care (details in S1 File). Hospital representatives provided consent for participation prior to randomisation and we did not require individual patient/carer consent. We collected data on all children (aged below 15 years), admitted to participating hospitals during the study period.

For this analysis, we included participants aged 28 days to 14 years who met World Health Organization (WHO) definitions for severe pneumonia, severe malaria or diarrhoea with severe dehydration (S1 Table). We accepted multiple diagnoses (Fig 1). For outcome analysis we excluded participants admitted in the two-weeks at the beginning of each crossover period (“wash-out period”), to implement the intervention and avoid contamination.

Randomisation

We randomly allocated hospitals to receive the full oxygen system at pre-specified dates using a computer-based random sequence generator. Concealment of the intervention or its start date was not possible as implementation required active participation from hospital staff.

Intervention

The intervention involved two phases. Phase 1 (pulse oximetry) introduced pulse oximetry for all admitted children and included provision of pulse oximeters and basic training. Phase 2 (full oxygen system) involved (i) a standardised oxygen equipment package, (ii) clinical education and support, (iii) technical training and support, and (iv) infrastructure and systems support. These all focussed on pulse oximetry and oxygen therapy, without substantially addressing broader clinical or technical topics. We designed the intervention based on experience from previous oxygen projects [18, 23, 24], WHO guidelines [25–27], and were informed by educational, behaviour change and implementation theories and strategies [28–32]. Further information on local adaptation of the intervention is published elsewhere [5, 18].

Public & patient involvement

We did not involve patients directly in the planning or conduct of this study. We involved members of the hospital teams, government, and the WHO throughout the planning, implementation, and evaluation of the program, and supported public events at facilities to share results.

Procedures

Trained nurse data collectors at each facility extracted data from case notes (made by clerking doctors as per usual procedure) using standardised Case Report Forms (CRFs) after identifying eligible patients from the ward admission book. The CRFs included data on i) demographics, ii) initial assessment, iii) oxygen therapy, iv) findings & case definitions and v) processes of care for each of pneumonia, malaria and diarrhoea. We collected data retrospectively for two years pre-intervention (January 2014 to October 2015), and prospectively between November 2015 and October 2017.

During the pre-intervention period, hospitals provided care as usual, including their existing supplies of oxygen. During the pulse oximetry period, hospitals continued usual care with the additional availability of project-supplied pulse oximeters. Following the full oxygen system intervention (starting March 2016, July 2016, November 2016, or March 2017), hospitals provided clinical care using the new oxygen systems, clinical guidelines and maintenance protocols.

Program theory

We hypothesized that the intervention would improve (i) reliability of oxygen availability, and (ii) healthcare worker use of oxygen, thereby increasing the proportion of hypoxaemic patients receiving oxygen, contributing towards reduced inpatient mortality. We expected the benefit to be greatest for children with pneumonia, for whom hypoxaemia is very common. We acknowledged the possibility that a multi-faceted oxygen intervention may affect quality of care more broadly, both positively (e.g. improved monitoring of sick patients) and negatively (e.g. over-reliance on oxygen to the exclusion of other therapies).

Outcomes

This study focused on pre-specified process outcomes. The primary outcome for this study was the mean change in quality of care (QOC) composite scores for children admitted with severe pneumonia, severe malaria and diarrhoea with severe dehydration.

We used data extracted from the case notes to classify diagnoses according to the presence of syndrome defining clinical signs, using standard WHO case definitions, (S1 Table) and generate disease specific QOC scores (S2 Table). The QOC scores evaluated assessment, diagnosis, treatment, and monitoring against WHO and Nigerian standard guidelines for each illness [27, 33]. The content and weighting of each domain reflected the key steps in management: assessment of primary and secondary signs/symptoms; diagnostic classification; treatment with primary and supportive therapies; and monitoring of vital signs. We designed this score based on previous work in Laos [34], and it is conceptually similar to the recently validated Paediatric Admission Quality of Care (PAQC) score from Kenya [35, 36]. Where clinical signs, clinical investigations or treatment were not documented, we assumed they were not present or done. We conducted quality checks on the data extraction after initial training, and periodically throughout the program, to satisfy ourselves that the data accurately represented what was documented in patient files.

We assigned disease-specific scores at the patient-level. Where children had more than one of the three diagnoses, they were assigned a score for each illness. We report both the disease specific QOC score totals and individual component scores (assessment, diagnosis, management, monitoring) to show how each domain contributed to the overall score. To evaluate the intervention’s effect on non-oxygen-related aspects of care, we excluded score components related directly to pulse oximetry and oxygen therapy (“modified score”).

Statistical analysis

We performed all analyses using Stata 15.1 (Statacorp, Texas, USA). We described patient characteristics by study period and summarised data using frequencies and percentages for categorical variables, and medians and interquartile ranges for continuous variables. We represented the crude means both in tabular form and visually in heat-map tables, by hospital and time period.

We reported intervention effects as mean differences in total QOC scores, adjusted for clustering and time using mixed-effects linear regression models. Our analysis model included fixed effects for intervention and time (four-month periods) and random effects for cluster (hospital), and was fitted to individual-level continuous outcome data [37, 38]. We found evidence of hospital-time interaction (by comparing the simple model with a model that included a variance component for random by-time effects, using a likelihood ratio test), and adjusted models accordingly. Primary analysis included the pre-intervention data, pulse oximetry period, and the full oxygen system (steps 1–12) to provide estimates of effect of pulse oximetry and full oxygen system periods against baseline. We also performed a restricted analysis (steps 7–12) comparing the stepped introduction of the full oxygen system to the pulse oximetry period alone (S3 Table).

Results

During the study period, 16,432 children were admitted to the participating hospitals, including 6,893 who met criteria for inclusion and for whom adequate data was available (Fig 1). Individual hospitals contributed between 113 and 1310 participants each, of whom almost two-thirds had severe malaria, approximately one-third had severe pneumonia, and <10% had diarrhoea with severe dehydration (Table 1). Further information, by illness, can be found in S4 Table.

[Figure omitted. See PDF.]

Table 1. Population characteristics of participants, by study period.

https://doi.org/10.1371/journal.pone.0254229.t001

The distribution of patient diagnoses varied slightly over the study periods. Patients were slightly older and tended to present with more severe illness (e.g. higher proportion with altered conscious state or hypoxaemia), during the pulse oximetry and full oxygen system periods.

Intervention effects

Tables 2 and 3 show the crude and adjusted mean QOC scores for children with severe pneumonia, severe malaria and diarrhoea with severe dehydration. Fig 2 demonstrates the trend in QOC score means over time, by hospital for each illness. Additional detail on score components contained in S5 Table.

[Figure omitted. See PDF.]

Fig 2. Crude QOC score total means (maximum six points) for severe pneumonia, severe malaria and diarrhoea with severe dehydration, by hospital and step.

Each cell contains the crude mean QOC score for a hospital during that step. For each of the three panels in this figure, the 12 hospitals are represented on the y-axis, whereas the steps of the trial are represented on the x-axis. The colour gradient extends from red (lowest score) through yellow to green (highest scores), providing a visual representation of change in scores over time. Blue spark-lines show the hospital- specific trend for easy reference. When the denominator to compute the cell rate is 0, cells are coloured in white. Data from November/December 2013 not available in step 1.

https://doi.org/10.1371/journal.pone.0254229.g002

[Figure omitted. See PDF.]

Table 2. Mean QOC scores for children with severe pneumonia, malaria, and diarrhoea, showing crude changes over time and by score component (assessment, diagnosis, treatment, monitoring).

https://doi.org/10.1371/journal.pone.0254229.t002

[Figure omitted. See PDF.]

Table 3. Effect of the intervention on non-modified and modified mean QOC score totals and components, comparing the pulse oximetry and full oxygen system periods to baseline (see S3 Table for restricted analysis comparing full oxygen system period to pulse oximetry period).

https://doi.org/10.1371/journal.pone.0254229.t003

Severe pneumonia.

Crude mean QOC scores for children with severe pneumonia increased almost two-fold, from 1.64 to 3.00 (out of a maximum 6 points), between the pre-intervention and full oxygen system periods (Table 2). Heat map tables showed that this change was largely achieved during the pulse oximetry period, with considerable variability between hospitals (Fig 2). Adjusted analysis (Table 3) showed that mean QOC score totals for children with severe pneumonia increased by 0.66 points (95% CI 0.22–1.09, p = 0.003) in the pulse oximetry period and 1.39 points (95% CI 1.08–1.69, p<0.001) in the full oxygen period, compared to baseline.

Analysis of the score components showed that assessment, treatment, and monitoring scores for children with severe pneumonia increased across the intervention periods, compared to baseline, with no change in diagnosis scores (Table 3). Assessment scores increased by 0.41 points (out of a maximum 2 points) (95% CI 0.13–0.69, p = 0.004) in the pulse oximetry period and 0.74 points (95% CI 0.54–0.93, p<0.001) in the full oxygen period, compared to baseline. Diagnosis scores (max 1) showed no evidence of change in either the pulse oximetry (aMD = -0.04; -0.12–0.04, p = 0.342) or the full oxygen system periods (aMD = -0.02; -0.08–0.04, p = 0.559), when compared to baseline. Treatment scores (max 2) changed minimally in the pulse oximetry period (aMD = 0.15; 95% CI -0.01–0.31, p = 0.072), but were higher in the full oxygen period (aMD = 0.32, 95% CI 0.20–0.44, p<0.001), compared to baseline. Monitoring scores (max 1) also changed minimally in the pulse oximetry period (aMD = 0.19; 0.001–0.39, p = 0.049) but were higher in the full oxygen period (aMD = 0.37, 95% CI 0.24–0.51, p<0.000), compared to baseline.

Modified QOC score totals (omitting score items directly related to pulse oximetry and oxygen therapy) for children with severe pneumonia showed lesser change over the study period (Tables 2 and 3), suggesting that most of the improvements in QOC scores were directly related to pulse oximetry and oxygen therapy practices. Crude mean modified QOC scores for children with severe pneumonia were 3.16 (SD 0.90) in the pre-intervention period, 3.09 (SD 0.85) in the pulse oximetry period and 3.21 (SD 0.82) in the full oxygen system period (max 6) (Table 2). Adjusted analysis showed that the mean difference in modified QOC score was -0.17 points in the pulse oximetry period (-0.53–0.19, p = 0.351), and 0.23 points in the full oxygen system period (-0.02–0.48, p = 0.072), compared to baseline. Component analysis suggested minimal effect on modified assessment (aMD -0.05; -0.22,0.13, p = 0.601), treatment (aMD 0.17; 0.05–0.29, p = 0.006) and monitoring (aMD = 0.13; 0.02–0.24, p = 0.016) scores in the full oxygen system period, compared to baseline.

Detailed sub-analysis of score components for children with severe pneumonia revealed substantial improvements in pulse oximetry coverage (increasing from 6% to 95% between pre-intervention and full oxygen system periods) and oxygen for children with hypoxaemia on admission (50% to 83%) (S5 Table). However, there was relatively poor documentation of feeding status (~50%), diagnostic classification (one-third correctly classified as severe pneumonia) and antibiotic prescribing (<6% prescribed appropriate first-line antibiotics), with minimal change over the study period.

Severe malaria.

Crude mean QOC scores for children with severe malaria increased from 2.81 to 4.04 (out of a maximum 6 points), between the pre-intervention and full oxygen system periods (Table 2). Heat map tables (Fig 2) show a gradual trend towards improved quality of care for children admitted with severe malaria across each of the study periods, with some variability over time and between hospitals. Adjusted analysis (Table 3) showed that mean QOC non-modified score totals for children with severe malaria increased in the pulse oximetry period by 1.26 points (95% CI 0.85–1.68, p<0.001) and the full oxygen system period by 1.53 points (95% CI 1.23–1.83, p<0.001), compared to baseline.

Analysis of the score components showed that assessment, diagnosis, treatment and monitoring scores for severe malaria all increased in the intervention periods, compared to baseline. Assessment scores (out of a maximum 2 points) were increased in the pulse oximetry period by 0.43 points (95% CI 0.20–0.67, p = <0.001) and in the full oxygen period by 0.67 points (95% CI 0.51–0.83, p<0.000) compared to baseline. Diagnosis scores (max 1) were increased in the pulse oximetry period by 0.15 points (95% CI 0.0.06–0.25, p = 0.002) and full oxygen period by 0.12 (95% CI 0.04–0.10, p = 0.002) compared to baseline. Treatment scores (max 2) were increased in the pulse oximetry period by 0.55 points (95% CI 0.33–0.78, p<0.001) and full oxygen period by 0.56 points (95% CI 0.39–0.73, p<0.001), compared to baseline. Monitoring scores (max 1) showed no change in the pulse oximetry period (aMD = 0.12, 95% CI -0.04–0.27, p = 0.139) but were higher in the full oxygen period (aMD = 0.18, 95% CI 0.07, 0.29, p = 0.002), compared to baseline.

Modified QOC score totals (omitting score items directly related to pulse oximetry and oxygen therapy) for children with severe malaria showed positive change over the study period (Table 3). Crude mean modified QOC scores for children with severe malaria were 3.95 (SD 1.11) in the pre-intervention period, 4.14 (SD 0.96) in the pulse oximetry period, and 4.28 (SD 0.90) in the full oxygen system period (max 6). Adjusted analysis of modified QOC scores showed that scores were higher in the pulse oximetry period (aMD = 0.68, 95% CI 0.35–0.89, p<0.001) and full oxygen period (aMD = 0.65, 95% CI 0.41–0.89, p<0.001), compared to baseline, with all the improvement realised during the pulse oximetry period. Component analysis suggested no effect on modified assessment scores (aMD = -0.06, 95% CI = -0.18–0.06, p = 0.303), but higher modified treatment (aMD = 0.47, 95% CI 0.27–0.68, p<0.001) and monitoring scores (aMD = 0.13; 0.02–0.24, p = 0.016) in the full oxygen system period, compared to baseline.

Detailed sub-analysis of score components for children with severe malaria revealed substantial improvements in pulse oximetry coverage (increasing from 8% to 95% between pre-intervention and full oxygen system periods) and oxygen for children with hypoxaemia on admission (15% to 46%) (S5 Table). However, there was relatively poor documentation of feeding status or blood sugar level on admission (~50% coverage).

Diarrhoea with severe dehydration.

Crude mean QOC scores for children changed minimally, between the pre-intervention and full oxygen system periods (Table 2), with no obvious trends evident in heat map tables (Fig 2). Adjusted analysis (Table 3) showed that mean QOC scores did not change in either the pulse oximetry period (MD = -0.29, 95% CI -0.78, 0.21, p = 0.259) or the full oxygen system period (MD = -0.12, 95% CI -0.46, 0.23, p = 0.501), compared to baseline. Component analysis showed no change in any of assessment (aMD = -0.20, 95% CI -0.44–0.04, p = 0.097), diagnosis (aMD = -0.05, 95% CI -0.15–0.05, p = 0.292), treatment (aMD = 0.10, 95% CI -0.08–0.28, p = 0.264) or monitoring scores (aMD = -0.002, 95% CI -0.16–0.15, p = 0.976), compared to baseline for diarrhoea with severe dehydration.

Detailed sub-analysis of score components for children with severe diarrhoeal disease revealed relatively poor documentation of feeding status on admission (~50% coverage), fluid prescribing (<15% prescribed appropriate rehydration fluids), and antibiotic prescribing (<20% appropriate), with little change over the study periods (S5 Table).

Discussion

This study reports results from the Nigeria Oxygen Implementation program regarding the impact of a multifaceted oxygen system intervention on paediatric care practices for children with severe pneumonia, severe malaria and diarrhoea with severe dehydration. We found an improvement in QOC score totals following the introduction of pulse oximetry and improved oxygen systems, for both severe pneumonia and severe malaria, compared to usual care at baseline. These changes were smaller, but still evident, after excluding QOC score features that were directly related to pulse oximetry and oxygen therapy, suggesting that the oxygen intervention had broader beneficial effects on care processes. We found no change in care processes for children with diarrhoea and severe dehydration, a condition for which pulse oximetry and oxygen therapy have minimal role.

How do changes in care practices for conditions correlate with observed changes (and lack of change) in clinical outcomes?

In previous analysis we reported the improved oxygen system intervention was associated with most significant clinical benefit for children with pneumonia (without detectable change in mortality for children with malaria or diarrhoea), and that most of this improvement was attributable to introduction of pulse oximetry [5].

In this analysis, we find similar improvements in QOC scores for children with severe pneumonia and children with severe malaria, and no change in QOC scores for children with severe diarrhoeal disease. However, the improvement in QOC for children with severe pneumonia appear to be primarily driven by oxygen-related care practices (i.e. pulse oximetry and oxygen therapy), whereas the improvement in QOC for children with malaria appears to be split evenly between oxygen-related and broader components of care. Furthermore, the prevalence of hypoxaemia among children with severe pneumonia was four-fold higher than among children with severe malaria or diarrhoea (45% versus 13 and 10% respectively) (S4 Table).

These results may suggest that the main driver of mortality benefit in the Nigeria Oxygen Implementation program was improved identification, monitoring and treatment of hypoxaemia (rather than broader improvements in quality of care), realising greatest impact in the population with highest hypoxaemia burden (children with pneumonia), despite the broader improvements in QOC detected for children with malaria.

Our results from Nigeria share similarities and differences with results from oxygen improvement programs in other low- and middle-income countries (LMICs) [3, 6]. In Papua New Guinea (PNG), Duke et al suggested that the most probable explanation for the observed 35% mortality reduction for children admitted to hospital with pneumonia is “the improved system and better quality of care that accompanies such a system” [3, 6]. Following this rationale, subsequent work in Laos and remote clinics in the PNG highlands introduced improved oxygen systems alongside other clinical education and quality improvement activities—not only addressing pneumonia and oxygen therapy, but also many other important aspects of hospital care for children [6, 8]. These programs reported substantial mortality benefits for children with pneumonia, and the PNG program also documented substantial benefits for children overall [6, 8].

How does a multi-faceted oxygen systems intervention affect broader care practices?

There are various ways that an oxygen improvement program could influence broader care practices, particularly relating to assessment and monitoring.

Qualitative data from our program in Nigeria suggested that nurses came to appreciate oximetry as a valuable tool for identifying and monitoring severely ill patients, saying it made their work easier and reduced stress [19]. This could conceivably lead to improvements in vital signs assessment and monitoring more broadly. However, while we observed clear improvements in the way oxygen was used as pulse oximetry practices improved, we did not observe broader effects on the assessment of other clinical signs or patient monitoring more broadly. This may be explained by relatively high baseline rates vital signs documentation, both on admission (>95% had vital signs documented) and over subsequent days (>85% had vital signs documented at least 3 times per day) (S5 Table). We did observe some improvement in vital signs monitoring during admission for children with severe pneumonia and malaria, but not for those with diarrhoea.

Pulse oximetry is the best way to detect hypoxaemia, with far greater sensitivity and specificity than any combination of clinical signs [16, 39, 40]. This could therefore assist healthcare workers in diagnosis, particularly for severe pneumonia [41]. However, we observed improved diagnosis scores for severe malaria, with no change in diagnosis scores for severe pneumonia or diarrhoea. This is despite relatively good diagnostic classification for children with severe malaria at baseline, but clear room for improvement for diagnostic classification of children with pneumonia (only one-third were identified as having severe pneumonia) (S5 Table). This finding of poor diagnostic classification of pneumonia is consistent with other reports from LMICs [42, 43] and is particularly concerning because accurate diagnosis correlates with correct treatment and survival [21].

In Laos, authors observed higher treatment completion rates for children with pneumonia, suggesting that oxygen therapy facilitated other aspects of treatment completion and prevented discharge against medical advice [6]. This may fit with anecdotal reports from healthcare workers that having pulse oximetry and an easy-to-use oxygen system makes work easier, but we had not necessarily expected our oxygen systems improvements to strongly influence broader treatment practices.

In fact, we observed improved treatment scores for children with severe pneumonia and malaria—over and above the improvements due to oxygen therapy. The changes in treatment for children with pneumonia were small, driven by improvements in fluid therapy. Notably, the proportion of children with severe pneumonia who received appropriate antibiotics remained very low even in the full oxygen system period (5.8%), largely due to the prescribing of broad-spectrum cephalosporins as first-line therapy (S5 Table).

The improvement in treatment for children with severe malaria was more substantial, driven by improvements in antimalarial prescribing (from 64% to 87%), with relatively good practices relating to anaemia (>98% received appropriate therapy) and fluids (~70% appropriate) across all time periods. With malaria recognition being relatively good in this malaria-endemic region [21], pulse oximetry use may have formed part of a more confident or thorough assessment process, thereby contributing to general improvements in diagnosis and treatment [21]. But we are aware of malaria-related training that occurred in some participating facilities during the study period, and these may also have contributed to the observed change.

Quality of care, context and emergence

We have presented data on care processes from a particular sub-Saharan African context and explored this in relation to a complex intervention. Quantitative quality of care scores have been shown to correlate with clinical outcomes [35]. However, quality of care scores can only measure some elements of quality of care, omitting important factors such as the relationships between, and attitudes of, healthcare workers, other staff, and patients [36, 44, 45]. Clinical outcomes emerge from the interaction between different care practices, particular interventions, and broader care structures and social contexts. Our findings do help explain how the Nigeria Oxygen Implementation program worked, particularly how improvements to hospital oxygen systems may influence broader aspects of care. However, they also raise many more questions for us, and for consideration in other contexts: how do oxygen-related care practices interact with other care practices: what degree of change in practice is required to translate into clinical benefits; how would this differ in different contexts or with interventions that addressed clinical care more holistically?

Limitations

Our use of a stepped-wedge design was useful for evaluating quantitative outcome changes and was flexible enough to embed implementation learnings. However, it does not remove all confounding—particularly relating to time. First, are the major economic and social changes in Nigeria that occurred during the intervention, including change in government, economic recession and disruptions due to industrial action in state hospitals leading to hospital closures [46]. Delayed salary payment and economic hardship may have led to the demotivation of health facility staff, potentially altering or reducing the interventions impact on broader quality improvement [47]. Second, malaria awareness and training activities occurred in many hospitals during the study period as part of a national push for better diagnosis and treatment for severe malaria [48]. While we used a pre-specified analysis method, both positive and negative findings should be interpreted with caution due to the risk of chance findings.

Similar to others conducting facility-based implementation research, we extracted data from clinical documents [49]. We minimised the amount of missing data with the use of dedicated research nurses, with our audit suggesting that documentation practices overall were very good [22]. With the exception of standardised case definitions, we did not impose additional criteria on healthcare workers regarding their documentation. We assumed signs were not present if they were not documented, which could result in diagnostic misclassification or missed signs of severity—generally influencing results towards the null. We simplified the original scoring plan for antibiotics and antimalarials to omit requirements for correct dosing, due to inconsistencies with units recorded—generally increasing treatment scores.

Practical implications

This study was part of a pragmatic field trial, which aimed to better understand how to improve oxygen access to children in Nigeria and other LMICs. Our findings provide practical lessons for those who seek to improve hospital oxygen systems and improve paediatric quality of care more broadly:

1. Improvement in hospital oxygen systems must be approached systematically. Efforts to build oxygen systems integrated with broader efforts to improve healthcare systems (e.g. equipment maintenance, emergency triage, case management, clinical documentation) may deliver greater improvements in the quality of care provided to patients—and better clinical outcomes. Oxygen is potentially a concrete focus on which broader improvement aims can be realised, if leveraged strategically.

2. Pulse oximetry is the key driver towards improved oxygen access. Together with improved oxygen delivery systems, pulse oximetry may drive broader improvements in the assessment and treatment of severely ill children and should be prioritised as the “fifth vital sign” for all children admitted to hospital.

These findings also highlight key areas for future practice and learning in the care of severely unwell children, including opportunities to: improve pneumonia assessment, diagnosis, and antibiotic prescribing; and further evaluate care processes (e.g. timeliness of treatment delivery) not measured in this study that may underpin a mortality benefit.

Conclusion

We found that improvements in hospital oxygen systems were associated with higher quality of care scores for children admitted with severe pneumonia and severe malaria, in 12 Nigerian hospitals. These improvements were attributable to better use of pulse oximetry and oxygen as well as broader improvements in clinical care. We found no evidence of negative distortions in care practices resulting from oxygen systems interventions. Oxygen improvement programs may realise greater effect if oxygen improvement efforts are integrated alongside more comprehensive efforts to improve quality of care.

Supporting information

S1 Fig. Characteristics of the Nigeria Oxygen Implementation project.

https://doi.org/10.1371/journal.pone.0254229.s001

(DOCX)

S1 File. Characteristics of 12 secondary-level hospitals in southwest Nigeria: Paediatric and neonatal wards (adapted from baseline needs assessment [26].

https://doi.org/10.1371/journal.pone.0254229.s002

(DOCX)

S1 Table. Case definitions of severe pneumonia, severe malaria or diarrhoea with severe dehydration.

https://doi.org/10.1371/journal.pone.0254229.s003

(DOCX)

S2 Table. Quality of care scores for severe pneumonia, severe malaria and diarrhoea with severe dehydration.

https://doi.org/10.1371/journal.pone.0254229.s004

(DOCX)

S3 Table. Quality of care score restricted and extended analysis.

https://doi.org/10.1371/journal.pone.0254229.s005

(DOCX)

S4 Table. Population characteristics of participants by study period and illness.

https://doi.org/10.1371/journal.pone.0254229.s006

(DOCX)

S5 Table. Quality of care score feature and component sub-analysis.

https://doi.org/10.1371/journal.pone.0254229.s007

(DOCX)

Acknowledgments

We thank the participating hospitals and their clinical, technical, and managerial staff. We thank all members of the Nigeria Oxygen Implementation team; representatives from the Federal Ministry of Health; representatives from the State Ministry of Health and hospital management boards in Oyo, Ondo, Ogun, and Osun states; and support staff at the Centre for International Child Health in Melbourne, Australia. We thank the study nurses Abiala Temitope, Adejubu Oyinlade, Adeleke Taiwo, Adesola Timilehin, Ajani Oluwatobi, Akamo Temitope, Akinrende Eniola, Alawode Juilet, Arubuolawe Elizabeth, Bakare Oluwabunmi Roseline, Eleyinmi Ayorinde Joseph, Kehinde Ayanfe, Mojoyinola Maria Oghenetekime, Ogbor Susan, Ogunkinle Alaba Segun, Ogunola Mayowa, Olaniyan Ibukun Abrouse and Sijuade Oluwaseun.

Citation: Graham HR, Maher J, Bakare AA, Nguyen CD, Ayede AI, Oyewole OB, et al. (2021) Oxygen systems and quality of care for children with pneumonia, malaria and diarrhoea: Analysis of a stepped-wedge trial in Nigeria. PLoS ONE 16(7): e0254229. https://doi.org/10.1371/journal.pone.0254229

References

1. Duke T, Graham S, Cherian M, Ginsburg A, English M, Howie S, et al. Oxygen is an essential medicine: a call for international action [Unresolved issues]. The International journal of tuberculosis and lung disease. 2010;14(11):1362–8. pmid:20937173

2. The World Health Organization, UNICEF. Executive summary: ending preventable child deaths from pneumonia and diarrhoea by 2025: the integrated global action plan for pneumonia and diarrhoea (GAPPD). World Health Organization; 2013.

3. Duke T, Wandi F, Jonathan M, Matai S, Kaupa M, Saavu M, et al. Improved oxygen systems for childhood pneumonia: a multihospital effectiveness study in Papua New Guinea. Lancet. 2008;372(9646):1328–33. pmid:18708248

4. Graham H, Tosif S, Gray A, Qazi S, Campbell H, Peel D, et al. Providing oxygen to children in hospitals: a realist review. Bull World Health Organ. 2017;95(4):288–302. pmid:28479624

5. Graham HR, Bakare AA, Ayede AI, Gray AZ, McPake B, Peel D, et al. Oxygen systems to improve clinical care and outcomes for children and neonates: A stepped-wedge cluster-randomised trial in Nigeria. PLoS medicine. 2019;16(11):e1002951–e. pmid:31710601

6. Gray AZ, Morpeth M, Duke T, Peel D, Winter C, Satvady M, et al. Improved oxygen systems in district hospitals in Lao PDR: A prospective field trial of the impact on outcomes. BMJ Paediatrics Open. 2017;1(1). pmid:29637121

7. Pulsan F, Duke T. Response to oxygen therapy using oxygen concentrators run off solar power in children with respiratory distress in remote primary health facilities in Papua New Guinea. Tropical Doctor.0(0):0049475520947886. pmid:32807026

8. Duke T, Pulsan F, Panauwe D, Hwaihwanje I, Sa’avu M, Kaupa M, et al. Solar-powered oxygen, quality improvement and child pneumonia deaths: a large-scale effectiveness study. Archives of Disease in Childhood. 2020. pmid:33067311

9. Bassat Q, Watkins K, Peterson S, Bijleveld P, Detjen A, Winn J, et al. The first Global Pneumonia Forum: recommendations in the time of coronavirus. Lancet Glob Health. 2020;8(6):e762–e3. pmid:32446342

10. Baker T, Schell CO, Petersen DB, Sawe H, Khalid K, Mndolo S, et al. Essential care of critical illness must not be forgotten in the COVID-19 pandemic. The Lancet. 2020;395(10232):1253–4.

11. Lohr KN. Medicare: a strategy for quality assurance: National Academies Press; 1990.

12. Donabedian A. The quality of care: how can it be assessed? Jama. 1988;260(12):1743–8. pmid:3045356

13. Mortality rate, under-5 (per 1,000 live births)—Nigeria [Internet]. 2019. https://data-worldbank-org.ezp.lib.unimelb.edu.au/indicator/SH.DYN.MORT?locations=NG&most_recent_value_desc=true.

14. Dibia N, Onwuchekwa J. Taxation and Economic Growth in Nigeria. 2019.

15. WHO and Maternal and Child Epidemiology Estimation Group (MCEE). UNICEF Data: Monitoring the Situation of Children and Women: Cause of death: UNICEF; 2015 [http://data.unicef.org.

16. Graham H, Bakare AA, Ayede AI, Oyewole OB, Gray A, Peel D, et al. Hypoxaemia in hospitalised children and neonates: A prospective cohort study in Nigerian secondary-level hospitals. EClinicalMedicine. 2019;16:51–63. pmid:31832620

17. Bakare AA, Graham H, Ayede AI, Peel D, Olatinwo O, Oyewole OB, et al. Providing oxygen to children and newborns: a multi-faceted technical and clinical assessment of oxygen access and oxygen use in secondary-level hospitals in southwest Nigeria. International Health. 2019;12(1):60–8.

18. Graham HR, Ayede AI, Bakare AA, Oyewole OB, Peel D, Gray A, et al. Improving oxygen therapy for children and neonates in secondary hospitals in Nigeria: study protocol for a stepped-wedge cluster randomised trial. Trials. 2017;18(1):502. pmid:29078810

19. Graham HR, Bakare AA, Gray A, Ayede AI, Qazi S, McPake B, et al. Adoption of paediatric and neonatal pulse oximetry by 12 hospitals in Nigeria: a mixed-methods realist evaluation. BMJ global health. 2018;3:e000812. pmid:29989086

20. Walker PJB, Bakare AA, Ayede AI, Oluwafemi RO, Olubosede OA, Olafimihan IV, et al. Using intermittent pulse oximetry to guide neonatal oxygen therapy in a low-resource context. Archives of Disease in Childhood—Fetal and Neonatal Edition. 2019:fetalneonatal-2019-317630.

21. Graham H, Bakare AA, Ayede AI, Oyewole OB, Gray A, Neal E, et al. Diagnosis of pneumonia and malaria in Nigerian hospitals: A prospective cohort study. Pediatric Pulmonology. 2020. pmid:32074408

22. Graham HR, Ayede AI, Bakare AA, Oyewole OB, Peel D, Falade AG, et al. Oxygen for children and newborns in non-tertiary hospitals in South-west Nigeria: A needs assessment. African journal of medicine and medical sciences. 2016;45(1):31–49. pmid:28686826

23. Matai S, Peel D, Wandi F, Jonathan M, Subhi R, Duke T. Implementing an oxygen programme in hospitals in Papua New Guinea. Ann Trop Paediatr. 2008;28:71–8. pmid:18318953

24. Subhi R. Oxygen systems in health facilities with limited resources: A Trial in Papua New Guinea. Melbourne: University of Melbourne; 2008.

25. WHO. Oxygen therapy for children. Geneva: World Health Organization; 2016.

26. WHO. Technical Specifications for Oxygen Concentrators. In: Organization WH, editor. WHO Medical Device Technical Series. Geneva: World Health Organization; 2015.

27. WHO. Pocket Book of Hospital care for children: guidelines for the management of common childhood illnesses. 2nd ed. Geneva: World Health Organization; 2013.

28. Merrill MD. First Principles of Instruction. In: Reigeluth CM, Carr A, editors. Instructional Design Theories and Models: Building a Common Knowledge Base. III. New York: Routledge Publishers; 2009.

29. Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6:42. pmid:21513547

30. Cane J, O’Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;7(37). pmid:22530986

31. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21. pmid:25889199

32. Bandura A. Social foundations of thought and action: A social cognitive theory: Prentice-Hall, Inc; 1986.

33. Olowu A, Elusiyan J, Esangbedo D, Ekure E, Esezobor C, Falade A, et al. Management of community acquired pneumonia (CAP) in children: Clinical practice guidelines by the Paediatrics Association of Nigeria (PAN). Nigerian Journal of Paediatrics. 2015;42(4):283–92.

34. Gray AZ, Soukaloun D, Soumphonphakdy B, Duke T. Implementing WHO hospital guidelines improves quality of paediatric care in central hospitals in Lao PDR. Tropical Medicine & International Health. 2015;20(4):484–92. pmid:25529576

35. Opondo C, Allen E, Todd J, English M. Association of the Paediatric Admission Quality of Care score with mortality in Kenyan hospitals: a validation study. Lancet Glob Health. 2018;6(2):e203–e10. pmid:29389541

36. Opondo C, Allen E, Todd J, English M. The Paediatric Admission Quality of Care (PAQC) score: designing a tool to measure the quality of early inpatient paediatric care in a low-income setting. Tropical Medicine & International Health. 2016;21(10):1334–45.

37. Hemming K, Haines TP, Chilton PJ, Girling AJ, Lilford RJ. The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. BMJ Case Reports. 2015;350:h391–h.

38. Davey C, Hargreaves J, Thompson JA, Copas AJ, Beard E, Lewis JJ, et al. Analysis and reporting of stepped wedge randomised controlled trials: synthesis and critical appraisal of published studies, 2010 to 2014. Trials. 2015;16:358. pmid:26278667

39. Wandi F, Peel D, Duke T. Hypoxaemia among children in rural hospitals in Papua New Guinea: epidemiology and resource availability—a study to support a national oxygen programme. Ann Trop Paediatr. 2006;26:277–84. pmid:17132292

40. Duke T, Mgone J, Frank D. Hypoxaemia in children with severe pneumonia in Papua New Guinea. The international journal of tuberculosis and lung disease: the official journal of the International Union against Tuberculosis and Lung Disease. 2001;5:511–9. pmid:11409576

41. Enoch AJ, English M, Shepperd S. Does pulse oximeter use impact health outcomes? A systematic review. Arch Dis Child. 2016;101(8):694–700. pmid:26699537

42. Izadnegahdar R, Cohen AL, Klugman KP, Qazi SA. Review: Childhood pneumonia in developing countries. The Lancet Respiratory Medicine. 2013;1:574–84. pmid:24461618

43. Obidike E, Oguonu T, Eze J, Ndu I, Ayuk A, Ukoha M, et al. Misdiagnosis of pneumonia, bronchiolitis and reactive airway disease in children: A retrospective case review series in South East, Nigeria. Current Pediatric Research. 2018;22(2).

44. Donabedian A. The Quality of Care: How Can It Be Assessed? Jama. 1988(260):1743–8. pmid:3045356

45. Hanefeld J, Powell-Jackson T, Balabanova D. Understanding and measuring quality of care: dealing with complexity. Bull World Health Organ. 2017;95(5):368–74. pmid:28479638

46. Oleribe OO, Udofia D, Oladipo O, Ishola TA, Taylor-Robinson SD. Healthcare workers’ industrial action in Nigeria: a cross-sectional survey of Nigerian physicians. Hum Resour Health. 2018;16(1):54. pmid:30333035

47. Eko JE. Implication of economic recession on the health care delivery system in Nigeria. Soc Sci. 2017;6(1):14–8.

48. Centers for Disease Control and Prevention. Confronting Malaria on the Frontline in Nigeria 2018 [https://www.cdc.gov/globalhealth/stories/confronting_malaria_on_the_frontline.html.

49. English M, Ayieko P, Nyamai R, Were F, Githanga D, Irimu G. What do we think we are doing? How might a clinical information network be promoting implementation of recommended paediatric care practices in Kenyan hospitals? Health Res Policy Syst. 2017;15(1):4. pmid:28153020