Parental and healthcare professional concern in the diagnosis of paediatric sepsis: a protocol for a prospective multicentre observational study

Abstract

Introduction

Paediatric sepsis is a major contributor to morbidity and mortality worldwide. Assessing concern from parents and healthcare professionals to determine disease severity in a child evaluated for sepsis remains a field requiring further investigation. This study aims to determine the diagnostic accuracy of parental and healthcare professional concern in the diagnosis of children evaluated for sepsis.

Methods and analysis

This prospective multicentre observational study will be conducted over a 24-month period in the paediatric emergency department (ED) at two tertiary Australian hospitals. A cross-sectional survey design will be used to assess the level of concern in parents, nurses and doctors for children presenting to ED and undergoing assessment for sepsis. The primary outcome is a diagnosis of sepsis, defined as suspected infection plus organ dysfunction at time of survey completion. Secondary outcomes include suspected or proven infection and development of organ dysfunction, defined as a Paediatric Sequential Organ Failure Assessment Score >0, within 48 hours of presentation, paediatric intensive care unit admission, confirmed or probable bacterial infection independent of organ dysfunction, and hospital length of stay.

Ethics and dissemination

Ethics approval was obtained from Children’s Health Queensland’s Human Research Ethics Committee (HREC/17/QRCH/85). Findings will be shared with relevant stakeholders and disseminated via conferences and peer-reviewed journals

Trial registration number

WHO Universal Trial Number, U1111-1256-4537; ANZCTR number, ACTRN1262000134092.

Correspondence to Amanda Harley; [email protected]

Strengths and limitations of this study

• This is expected to be the largest study to date, which assesses the diagnostic accuracy of parental and healthcare professional concern in recognising sepsis within the Emergency Department.

• Prospective assessment of parental, nursing and medical concern will be undertaken both quantitatively and qualitatively.

• The study will be emedded in an institutional paediatric sepsis pathway, reducing barriers for staff engagement.

• Consideration for other sources of diagnostic bias as a result of referral, previous history and concomitant interventions is required.

Introduction

Sepsis is a major contributor to morbidity and mortality in children worldwide.¹ The WHO recently identified sepsis as a key health priority, outlining the high global burden of this time critical and often preventable disease.² While the latest definition of paediatric sepsis dates back to 2005,³ the definition of sepsis in adults was redefined in 2016 as ‘life threatening organ dysfunction caused by a dysregulated host response to infection’.⁴ Globally, close to 50 million patients suffer from sepsis each year, with over 10 million sepsis-related deaths,⁵ the highest incidence affecting infants and children. While the highest burden related to sepsis affects low-income and middle-income settings, sepsis remains among the leading causes of (potentially preventable) morbidity and mortality in high-income countries too, accounting for a total cost of US$7.31 billion in the USA alone.⁶ These high economic costs, along with the persistently high prevalence and morbidity of paediatric sepsis, highlight the urgent need for further research into earlier sepsis recognition.

Prompt identification is well recognised as fundamental for early intervention and treatment of sepsis. In a large retrospective study of children with sepsis, the delay in the administration of a sepsis treatment bundle consisting of intravenous antibiotics, fluids and blood cultures was associated with a significant increase in mortality.⁷ The majority of paediatric sepsis deaths occur within the first 48 hours of initial admission to the intensive care unit,⁸ emphasising the need for prompt recognition and resuscitation. The new Surviving Sepsis Guidelines⁹ further emphasise the need for early detection as it is a critical survival factor for paediatric sepsis, with timely and appropriate initiation of interventions being linked to improved patient outcomes.

Paediatric sepsis often starts as an insidious condition, which poses many challenges for healthcare professionals to accurately and timely diagnose it. This is due to the vague and non-specific nature of the disease, coupled with a relatively low incidence rate compared with the number of children presenting to the emergency department (ED) with febrile illness.¹⁰ Consequently, the risk for a missed diagnosis is high and subsequent repercussions are potentially lifelong or fatal.¹¹ In its early stages, sepsis often resembles many other common febrile illnesses with the clinical signs of fever, tachycardia and tachypnoea.¹² Parents, as experts of their child, may be valuable in the identification of sepsis and discrimination of the condition from other milder illnesses.

Observational studies suggest that parents may recognise illness severity before nurses and doctors, independent of key clinical signs.¹³ Root cause analyses and anecdotal data after fatal paediatric sepsis outcomes established that children often re-presented several times and parents commonly indicated concerns that the ‘illness was different’.¹³ A more holistic and family-centred-care approach incorporating collaboration between the child’s family and treating team has the potential to enhance the timely recognition of sepsis.¹⁴

The current diagnostic approach for sepsis rely predominately on clinician-guided assessment or physiology-based tools, which pose numerous challenges due to the complex nature of paediatric physiology.¹⁵ While the search for more precise biomarkers for sepsis continues, little is known in relation to using concern as a diagnostic tool to aid in earlier recognition.¹⁶ In addition to parental concern, the gut feeling or intuition of healthcare professionals may contribute to the recognition of sepsis.¹⁷ In the primary care setting, a gut feeling that ‘something was wrong’ reported by clinicians was linked with a high specificity and positive likelihood ratio for serious bacterial infections.¹⁸ The inclusion of parental and healthcare professional concern in the diagnostic approach has the potential ability to improve specificity, thereby increasing sepsis recognition.

We hypothesise that the inclusion of parental and healthcare professional concern in the ED will improve diagnostic accuracy and early recognition of paediatric sepsis. The main objective of this study is to determine the diagnostic accuracy of concern levels in parents, doctors and nurses to recognise paediatric sepsis in a prospective multicentre observational study.

Methods and analysis

Study design

This prospective multicentre observational cohort study will use a cross-sectional survey tool designed to independently assess the level of concern in parents, nurses and doctors for children who present to the ED and are evaluated for sepsis. The planned duration for the project will be at least 24 months for recruitment with 6 months for data cleaning, analyses and write up. The study started recruitment in December 2018 . This study has been designed to fulfil criteria for Standards for Reporting Diagnostic accuracy studies (STARD).¹⁹

Study setting

This study will be conducted across the dedicated paediatric EDs at two tertiary Australian hospitals: Queensland Children’s Hospital, which receives approximately 6600 presentations each month, and Gold Coast University Hospital, which receives approximately 2300 paediatric presentations each month.

Participants

Eligible participants will be children aged between 30 days and 18 years presenting to the ED and evaluated for sepsis via the institutional sepsis pathway and/or undergoing blood culture sampling for suspected infection (table 1).

Study criteria

Table 1

Study inclusion and exclusion criteria

Inclusion criteria	Exclusion criteria
Child aged 30 days–18 years old Presented to ED Evaluated for sepsis on the sepsis pathway and/or having blood culture sampling Survey completed during ED stay, aiming to be completed at time closest to triage presentation Parent/caregiver attending with child, treating doctor and/or nurse available for survey	Parents who do not speak English Children with high suspicion of SARS-CoV-2 infection* Patients in clinical areas outside the ED such as the paediatric intensive care unit

*Research governance did not permit researchers to risk exposure to SARS-CoV-2 infection.

Test methods

The study surveys have been designed for parents, nurses and doctors, incorporating both quantitative and qualitative measures (figures 1–3). To ensure consistent comparison, all surveys have the same basic design and content, with minor adaptions on the parent/carer survey to reflect the participant role (parent vs nurse/doctor). Participants are asked to rate the degree to which they agree or disagree with a statement or question using a 5-point Likert scale,²⁰ followed by two free text questions. The surveys were piloted 20 times to ensure questionnaire feasibility. Job title and years of experience for participating doctors and nurses will be collected.¹⁷

Figure 1. Parental survey.

Figure 2. Nursing survey.

Figure 3. Medical survey.

The surveys will be distributed to one of the child’s parents/caregivers, nurse and doctor on presentation and will be completed during the ED stay. The distribution of these surveys will occur 7 days a week 24 hours a day through the ED staff with support from the dedicated research team. We aim for surveys to be completed at time closest to triage and within 4 hours from initial presentation. This window for survey administration was determined based on the current Australian National Emergency Assess Target guidelines, which stipulate that patients must be admitted, discharged or transferred from ED within 4 hours of initial presentation.²¹ These surveys will be embedded within the Queensland paediatric sepsis pathway, which was developed and implemented across Queensland paediatric EDs from 2018.

Sample size

A minimum of 450 patients will be recruited over the two sites. This minimum sample size was selected based on a sample size calculation, which revealed that with an expected prevalence of 10% and an expected improvement in sensitivity from 0.6 to 0.8, a sample size of 450 is needed.²²

Data collection

The patient demographics, information regarding the presentation and illness severity at baseline will be collected from the medical record. In addition, the worst measure of physiological parameters and maximum level of support during the first 48 hours will be captured. Illness severity will be determined using the Paediatric Sequential Organ Failure Assessment (pSOFA) Score.²³ Data will be recorded into a secure RedCap case report form.

Analysis plan

Suspected or proven infection in the presence of organ dysfunction, defined as a pSOFA Score >0 at time of assessment, is defined as the primary outcome. Given the ongoing controversy around paediatric sepsis definitions,¹² sensitivity analyses with organ dysfunction defined as per the 2005 International Paediatric Definitions Consensus Conference will be performed.^{3 24} Secondary outcomes are defined as: (a) suspected or proven infection and development of organ dysfunction, defined as a pSOFA Score >0, within 48 hours of presentation; (b) admission to the Paediatric Intensive Care Unit; (c) confirmed or probable bacterial infection independent of organ dysfunction and (d) hospital length of stay. The likelihood of bacterial versus viral infection will be determined by an independent assessor using all available laboratory, microbiological and clinical information with adjudication of patients.²⁵ Bacterial infection will be categorised as confirmed bacterial infection (positive microbiological cultures compatible with the clinical syndrome, and decision by the treating physician to treat for at least 5 days or until death with antibiotics) or probable bacterial infection (negative microbiological cultures in the presence of a clinical syndrome of bacterial infection and increased C reactive protein, and decision by the treating physician to treat for at least 5 days or until death with antibiotics). Viral infection will be categorised as probable viral infection (negative microbiological tests in the presence of a clinical syndrome of viral infection such as bronchiolitis) or proven viral infection (positive microbiological testing in the presence of a clinical syndrome of viral infection). If the presentation is determined to be of non-infectious or unknown origin, it will be classed as infection of uncertain origin, or as non-infectious conditions.

Descriptive analyses will report on the demographics and baseline patient features. Description on the level of completeness of the surveys (parental, nursing and medical) will be provided and any differences in demographics will be investigated between children who have completed surveys from all three participant groups and those who have missing surveys.

To assess the relationship between the concern ratings and outcome, first, an exploratory factor analysis will be performed on the four concern questions assessed in the surveys to determine whether the questions are measuring the same latent construct (‘concern’) or if more than one construct is present. In addition, the internal consistency and inter-rater reliability of the concern questions will be assessed using Cronbach’s alpha and the intraclass correlation using a one-way random effects model, respectively. Based on the results of the factor analysis, a factor score will be created and used as a measure of concern in the regression models. In addition, the relationship between the four individual concern questions with the primary outcome will be assessed through bivariate logistic regression models. The question that provides the best prediction of sepsis will be identified as the one that has the highest unadjusted OR and area under the receiver operating characteristic curve (AUROC), with 95% CIs being reported alongside all effects.

Next, regression models will be derived for the primary and secondary outcomes to assess the associations between the concern factor score and the ‘best’ concern question with the outcome. Other demographic characteristics and physiological variables, which are associated with the outcomes of interest, will be included in the model as control variables. The AUROC, sensitivity, specificity, negative and positive likelihood ratios (along with associated 95% CIs) will be calculated to assess model fit and predictive performance.

All regression modelling will be performed on each of the three responder cohorts separately (ie, children with a parental survey, children with a nurse survey and children with a medical survey completed) to identify whether the effect of concern on the outcomes is dependent on the responder. P values below 0.05 will be considered as statistically significant. All analyses will be performed by an expert statistician using R.²⁶

A preplanned secondary analysis will assess qualitative data from the survey free text questions using the framework method.²⁷ These free texts will then be examined and sorted into multiple categories to determine commonalities and differences. These categories will then be sorted into themes for the three groups: parents, doctors and nurses. ²⁷

Strengths

A strength of this study is its prospective observational study design with a large multicentre cohort of children evaluated for sepsis. In contrast to previous studies, which more broadly captured serious bacterial infections¹⁸ or fever,²⁸ the present study captures sepsis defined as suspected/proven infection with organ dysfunction as the main outcome. The study design enables assessment of the role of parental and healthcare professional concern in diagnosing paediatric sepsis and compares the respective diagnostic accuracies with the diagnostic performance of the routine diagnostic process.

This study aims to address an established gap regarding the significance of parental and healthcare professional concern in predicting disease severity in children with infection. Outcomes can inform the design of improved sepsis recognition tools. While the study will be conducted within the ED, findings relating to the use of concern as a red flag and a prompt for further investigation and assessment could be translated into other clinical settings.

Limitations

This study presents several limitations. It is expected that some patients will only have partial sets of surveys completed due to circumstances such as a parent being deemed unfit to complete a research survey for various reasons, or the attending nurse or doctor not completing a survey. The incomplete sets of surveys for patients are anticipated given the pragmatic nature of the study and will be a consideration when conducting analysis and reporting. Bias could occur through children who are more clinically well having a greater number of concern surveys completed, as opposed to more clinically unwell children, whereby parents may be too distressed and healthcare professionals otherwise occupied treating the child.²⁹ Standardised dissemination of study education will aim to reduce potential bias related to variances on how the surveys are administered to parents, doctors and nurses. Implementation of an educational script will eliminate the use of words such as sepsis, organ dysfunction or death, which may potentially heighten concern levels or result in changes to concern. Finally, the study will be performed in two sites working within the same healthcare system in a high-income country, and, hence, similar studies in low-income and middle-income settings will be required to assess generalisability.

Ethics and dissemination

Informed verbal consent will be gained from the parent/caregiver, nurse and doctor at the time of survey administration. It will be reiterated to all parties that they have the right to refuse participation at initial time of consent or withdraw at any stage without affecting patient care or their employment, as applicable. The survey and study design have been approved by the Children’s Health Queensland’s Human Research Ethics Committee (HREC/17/QRCH/85).

Findings will be shared with relevant stakeholders and disseminated via conferences and peer-reviewed journals.

Ethics statements

Patient consent for publication

Not required.

Footnote

Twitter @SeverZoe, @shanegeorge77, @theamandaharley

Contributors LJS conceived the project and provided the technical and intellectual inputs supporting ZS and AH in writing this manuscript and approved it for submission. ZS produced the first draft and subsequent drafts of the paper, supported by AH. AH and ZS lead central coordination and contributed to the design of the analysis plan. ZS, LJS, MJ, SG and AH were involved in the review and provided inputs on the manuscript. All authors reviewed and approved various drafts and the final paper.

Funding This work is supported by the Emergency Medicine Foundation, Queensland, Australia EMJS-327R31-2019-HARLEY. LJS was supported by a Practitioner Fellowship from NHMRC and by the Children"s Hospital Foundation, Brisbane, Australia.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

References

¹ Fleischmann-Struzek C, Goldfarb DM, Schlattmann P, et al. The global burden of paediatric and neonatal sepsis: a systematic review. Lancet Respir Med 2018; 6: 223–30. doi:10.1016/S2213-2600(18)30063-8 http://www.ncbi.nlm.nih.gov/pubmed/29508706

² Kissoon N, Reinhart K, Daniels R, et al. Sepsis in children: global implications of the world health assembly resolution on sepsis. Pediatr Crit Care Med 2017; 18: e625–7. doi:10.1097/PCC.0000000000001340 http://www.ncbi.nlm.nih.gov/pubmed/28914721

³ Goldstein B, Giroir B, Randolph A, et al. International pediatric sepsis consensus conference: definitions for sepsis and organ dysfunction in pediatrics. Pediatr Crit Care Med 2005; 6: 2–8. doi:10.1097/01.PCC.0000149131.72248.E6 http://www.ncbi.nlm.nih.gov/pubmed/15636651

⁴ Singer M, Deutschman CS, Seymour CW, et al. The third International consensus definitions for sepsis and septic shock (Sepsis-3). JAMA 2016; 315: 801–10. doi:10.1001/jama.2016.0287 http://www.ncbi.nlm.nih.gov/pubmed/26903338

⁵ Rudd KE, Johnson SC, Agesa KM, et al. Global, regional, and national sepsis incidence and mortality, 1990-2017: analysis for the global burden of disease study. Lancet 2020; 395: 200–11. doi:10.1016/S0140-6736(19)32989-7 http://www.ncbi.nlm.nih.gov/pubmed/31954465

⁶ Carlton E. Cost of Pediatric Severe Sepsis Hospitalizations | Critical Care Medicine | JAMA Pediatrics | JAMA Network [Internet], 2019. Available: https://jamanetwork.com/journals/jamapediatrics/article-abstract/2748380 [Accessed 07 Apr 2020 ].

⁷ Evans IVR, Phillips GS, Alpern ER, et al. Association between the new York sepsis care mandate and in-hospital mortality for pediatric sepsis. JAMA 2018; 320: 358–67. doi:10.1001/jama.2018.9071 http://www.ncbi.nlm.nih.gov/pubmed/30043064

⁸ Schlapbach LJ, MacLaren G, Festa M, et al. Prediction of pediatric sepsis mortality within 1 h of intensive care admission. Intensive Care Med 2017; 43: 1085–96. doi:10.1007/s00134-017-4701-8 http://www.ncbi.nlm.nih.gov/pubmed/28220227

⁹ Weiss SL, Peters MJ, Alhazzani W, et al. Surviving sepsis campaign international guidelines for the management of septic shock and sepsis-associated organ dysfunction in children. Pediatr Crit Care Med 2020; 21: e52–106. doi:10.1097/PCC.0000000000002198 http://www.ncbi.nlm.nih.gov/pubmed/32032273

¹⁰ Schlapbach LJ, Weiss SL, Wolf J. Reducing collateral damage from mandates for time to antibiotics in pediatric Sepsis-Primum non nocere. JAMA Pediatr 2019; 173: 409–10. doi:10.1001/jamapediatrics.2019.0174 http://www.ncbi.nlm.nih.gov/pubmed/30882879

¹¹ Schlapbach LJ, Straney L, Alexander J, et al. Mortality related to invasive infections, sepsis, and septic shock in critically ill children in Australia and New Zealand, 2002-13: a multicentre retrospective cohort study. Lancet Infect Dis 2015; 15: 46–54. doi:10.1016/S1473-3099(14)71003-5 http://www.ncbi.nlm.nih.gov/pubmed/25471555

¹² Schlapbach LJ, Kissoon N. Defining pediatric sepsis. JAMA Pediatr 2018; 172: 313–4. doi:10.1001/jamapediatrics.2017.5208 http://www.ncbi.nlm.nih.gov/pubmed/29459982

¹³ Launay E, Gras-Le Guen C, Martinot A, et al. Why children with severe bacterial infection die: a population-based study of determinants and consequences of suboptimal care with a special emphasis on methodological issues. PLoS One 2014; 9: e107286. doi:10.1371/journal.pone.0107286 http://www.ncbi.nlm.nih.gov/pubmed/25247401

¹⁴ Dennis C, Baxter P, Ploeg J, et al. Models of partnership within family-centred care in the acute paediatric setting: a discussion paper. J Adv Nurs 2017; 73: 361–74. doi:10.1111/jan.13178 http://www.ncbi.nlm.nih.gov/pubmed/27706840

¹⁵ Roland D, Snelson E. "So why didn"t you think this baby was ill?" Decision-making in acute paediatrics. Arch Dis Child Educ Pract Ed 2019; 104: 43–8. doi:10.1136/archdischild-2017-313199 http://www.ncbi.nlm.nih.gov/pubmed/29496733

¹⁶ Thompson M, Van den Bruel A, Verbakel J, et al. Systematic review and validation of prediction rules for identifying children with serious infections in emergency departments and urgent-access primary care. Health Technol Assess 2012; 16: 1–100. doi:10.3310/hta16150 http://www.ncbi.nlm.nih.gov/pubmed/22452986

¹⁷ Roland D. Have we forgotten to teach how to think? Emerg Med J 2017; 34: 68–9. doi:10.1136/emermed-2016-206211 http://www.ncbi.nlm.nih.gov/pubmed/28111368

¹⁸ Van den Bruel A, Thompson M, Buntinx F, et al. Clinicians" gut feeling about serious infections in children: observational study. BMJ 2012; 345: e6144. doi:10.1136/bmj.e6144 http://www.ncbi.nlm.nih.gov/pubmed/23015034

¹⁹ Cohen JF, Korevaar DA, Altman DG, et al. STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration. BMJ Open 2016; 6: e012799. doi:10.1136/bmjopen-2016-012799 http://www.ncbi.nlm.nih.gov/pubmed/28137831

²⁰ Sullivan GM, Artino AR. Analyzing and interpreting data from likert-type scales. J Grad Med Educ 2013; 5: 541–2. doi:10.4300/JGME-5-4-18 http://www.ncbi.nlm.nih.gov/pubmed/24454995

²¹ Sullivan C, Staib A, Khanna S, et al. The National emergency access target (NEAT) and the 4-hour rule: time to review the target. Med J Aust 2016; 204: 354. doi:10.5694/mja15.01177 http://www.ncbi.nlm.nih.gov/pubmed/27169971

²² Bujang MA, Adnan TH. Requirements for minimum sample size for sensitivity and specificity analysis. J Clin Diagn Res 2016; 10: YE01–6. doi:10.7860/JCDR/2016/18129.8744 http://www.ncbi.nlm.nih.gov/pubmed/27891446

²³ Matics TJ, Sanchez-Pinto LN. Adaptation and validation of a pediatric sequential organ failure assessment score and evaluation of the Sepsis-3 definitions in critically ill children. JAMA Pediatr 2017; 171: e172352. doi:10.1001/jamapediatrics.2017.2352 http://www.ncbi.nlm.nih.gov/pubmed/28783810

²⁴ Gebara BM. Values for systolic blood pressure. Pediatr Crit Care Med 2005; 6: 500. doi:10.1097/01.PCC.0000164344.07588.83 http://www.ncbi.nlm.nih.gov/pubmed/16003219

²⁵ Herberg JA, Kaforou M, Wright VJ, et al. Diagnostic test accuracy of a 2-Transcript host RNA signature for discriminating bacterial vs viral infection in febrile children. JAMA 2016; 316: 835–45. doi:10.1001/jama.2016.11236 http://www.ncbi.nlm.nih.gov/pubmed/27552617

²⁶ R Core Team. A language and environment for statistical computing. R Foundation for Statistical Computing [Internet], 2019. Available: https://www.R-project.org/

²⁷ Gale NK, Heath G, Cameron E, et al. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol 2013; 13: 117. doi:10.1186/1471-2288-13-117 http://www.ncbi.nlm.nih.gov/pubmed/24047204

²⁸ Urbane UN, Gaidule-Logina D, Gardovska D, et al. Value of parental concern and clinician"s gut feeling in recognition of serious bacterial infections: a prospective observational study. BMC Pediatr 2019; 19: 219. doi:10.1186/s12887-019-1591-7 http://www.ncbi.nlm.nih.gov/pubmed/31269915

²⁹ Hammer GP, du Prel J-B, Blettner M. Avoiding bias in observational studies. Dtsch Arztebl Int 2009; 106: 664–8.