Factors associated with children’s seasonal influenza diagnosis by their guardians: a prospective observational study in Japan

Abstract

Introduction: Children’s influenza diagnosis by their guardians has been reported to be highly accurate, but clinical factors that improve the reliability of a guardian’s diagnosis are unclear.

Aim: To determine the accuracy of guardians’ influenza diagnosis of their children, investigate clinical factors that improve the diagnostic accuracy, and determine the additional clinical value of the guardian’s diagnosis.

Methods: A prospective observational study was conducted at a primary care clinic in Japan from December 2017 to April 2019. Pre-examination checklists completed by guardians accompanying children aged <12 years with clinically suspected influenza were analysed. Receiver-operating characteristic curve analysis was performed to evaluate the diagnostic accuracy at multiple cut-off points and to compare the area under the curve (AUC), using a rapid influenza diagnostic test as the reference standard. Multivariable logistic regression was performed to validate additional contribution of guardians’ diagnosis.

Results: A total of 112 patient pairs of child (median age, 6 years) and guardian (mother, 81.2%; father, 16.1%; grandmother, 1.8%; other, 0.9%) were included in the analysis. The AUC for guardians’ influenza diagnosis was higher in mothers (0.72), as well as pairs with children with a history of influenza (0.72), guardians who were aware of the influenza epidemic (0.71), and unvaccinated children (0.76), than in other guardians. After multivariate analysis, the AUC increased significantly from 0.79 to 0.85.

Discussion: Guardians’ influenza diagnosis for their children was highly accurate. We identified factors that improve the accuracy of the guardians’ diagnosis and demonstrated that the guardians’ diagnosis can support physicians’ diagnostic accuracy.

Introduction

Seasonal influenza is an acute respiratory tract infection caused by the influenza virus and a common disease among children visiting primary care clinics in winter. In the 2017–18 winter season, approximately 0.86 million influenza outpatients aged <5 years were reported in Japan¹ and 2.88 million were reported in the United States.² The influenza virus is transmitted from person to person via respiratory secretions, and children are important vectors for the spread of the disease.^3,4 The viral-shedding period can be prolonged, especially in younger and immunocompromised children.^5,6 However, influenza symptoms in children are not always classic or characteristic,⁷ and younger children are often unable to accurately describe their symptoms to their guardians. Self-diagnosis and self-prescription using the internet are expected to increase with advancements in information and communication technology.⁸ If the diagnostic ability and treatment provided by guardians, as the children’s closest observers, are accurate, this could reduce need for visits to medical institutions and consequent risk of spreading the infection.

WHAT GAP THIS FILLS

What is already known: The accuracy of influenza diagnosis by children’s guardians has been reported to be high, helping physicians to diagnose the disease.

What this study adds: The accuracy of guardians’ influenza diagnosis was higher in mothers, children with a history of influenza, guardians who were aware of the influenza epidemic, and unvaccinated children.

A retrospective observational study showed that a quantitative diagnosis using a continuous indicator (in contrast to a qualitative diagnosis, which is a binary evaluation of yes/no) of seasonal influenza in children by their guardians could help establish a confirmatory diagnosis when the probability was higher than the optimal cut-off point (80%), and an exclusionary diagnosis when the probability was low (50%).⁹ However, there have been no prospective observational studies conducted to confirm the quantitative diagnostic accuracy by guardians, and no studies of factors affecting the accuracy of a quantitative influenza diagnosis by guardians. Therefore, we conducted a prospective observational study to determine the accuracy of quantitative influenza diagnosis of children by their guardians, describe factors affecting guardians’ diagnosis, and analyse the additional clinical value of guardians’ influenza diagnosis.

Methods

This prospective observational study was conducted at Rokkasho Centre for Community and Family Medicine, a community clinic in Rokkasyo-mura, Aomori, Japan. Data were collected on factors potentially affecting the accuracy of influenza diagnosis by guardians. We then verified the guardians’ diagnostic data for accuracy and assessed the additional diagnostic value of the guardians’ diagnosis.

Ethical approval and consent to participate

This study received ethical approval from the Medical Ethics Committee of Hirosaki University (approval number: 2017-1100). Written informed consent was obtained from the parents or guardians of all patients included in the study. All data were anonymised at the time of data collection, and confidentiality was ensured throughout the study.

Patients

Trained nurses who conducted preliminary medical interviews invited all accompanying persons (guardians) of the children with clinically suspected seasonal influenza (eg upper respiratory tract symptoms and fever) who visited the study clinic to complete a pre-examination checklist during two consecutive influenza seasons from December 2017 to April 2019. Patients who met all the following criteria were included: aged <12 years; the guardian had completed a pre-examination checklist before the patient was assessed by a doctor; and the patient underwent a rapid influenza diagnostic test. Patients were excluded from the study if guardians did not complete all items on the pre-examination checklist and did not provide written consent.

Data collection

Data were collected using pre-examination checklists (Supplementary File S1, Supplementary File S2) and electronic medical records. Items included in both data sources reflected clinical symptoms and factors important for guardians’ diagnosis according to past studies,^9–15 or symptoms and factors that were considered clinically important. The pre-examination checklist included the following: the guardian’s relationship to the patient, the child’s influenza history, the child’s influenza vaccination status, the child’s medication intake before medical visits, and the guardian’s awareness of the child’s contact with persons with influenza. Clinical symptoms, the axillary temperature at home, time of symptom onset, the severity of the current symptoms, and the guardians’ diagnosis are presented as a percentage (%). The following data were collected from the patients’ medical records: age, sex, clinical signs (axillary temperature and pulse rate at the clinic), time of the rapid influenza diagnostic test, results of the rapid influenza diagnostic test, and final diagnosis by the physician.

Quantitative diagnosis was defined as a continuous variable and qualitative diagnosis as a categorical variable index such as yes/no. For quantitative diagnosis, guardians were asked to provide an estimate of the possibility that their children were suffering from influenza, reported as a percentage, on the pre-examination checklist. We considered that by converting the guardians’ estimation into a numerical value such as percentage, we could deal with it as if it were laboratory data.

Diagnosis of influenza

As a reference standard for influenza diagnosis, a rapid diagnostic kit (Prime Check Flu, Alfresa Co., Ltd., Japan) was used. This kit was reported to have a sensitivity of 98.4% and a specificity of 97.6%, using virus isolation and culture as the reference standard. The decision to conduct the test was made by the physician who examined the patient. The test result was determined by a clinical laboratory technician, independent of the physician who examined the patient and the nurses who conducted the pre-examination.

Classification of factors

Factors that affect guardians’ influenza diagnostic accuracy were classified as reliable or unreliable information. Reliable information included: objective findings, such as body temperature (fever) and duration of symptoms; child’s influenza vaccination history; the guardian’s relationship to the child; and awareness of the child having contact with anyone with influenza. Unreliable information included the child’s subjective symptoms, such as disease onset (acute or gradual) and pain, because guardians could not accurately recognise their children’s symptoms. In particular, younger children cannot always communicate their symptoms to their guardians. In this study, only reliable factors were included in the analysis.

Statistical analysis

The accuracy of the guardians’ quantitative influenza diagnosis (%) was analysed by receiver-operating characteristic (ROC) curve analysis to determine the sensitivity, specificity, and likelihood ratio of their diagnosis at various cut-off points and to determine the optimal cut-off point. The area under the curve (AUC) was used to evaluate the discriminatory power of guardians’ diagnosis. The AUC results were interpreted according to the following criteria:¹⁶ AUC = 0.5, non-informative; 0.5 < AUC ≤ 0.7, less accurate; 0.7 < AUC ≤ 0.9, moderately accurate; 0.9 < AUC ≤ 1, highly accurate; AUC = 1, perfect test. A subgroup analysis was performed to evaluate the differences between groups with and without a factor using the AUC as an index. The sensitivity and specificity of each cut-off point were calculated. Multivariable logistic regression was performed to validate the additional contribution of the guardians’ influenza diagnosis.

Sample size calculations were performed according to the following assumptions: AUC, 0.7 based on a previous study;⁹ influenza positive rate, 50%; α, 0.05, and power, 0.9. Using these assumptions, the required sample size for AUC analysis was estimated to be 66. The target sample size was set at ≥100 to enable subgroup analysis. All statistical analyses were performed using EZR version 1.40 (Saitama Medical Centre, Jichi Medical University, Saitama, Japan).¹⁷

Results

In total, 162 patients met the inclusion criteria, and 112 pairs of patients (median age [interquartile range] 6 years [4, 8]) and guardians (mother, 81.2%; father, 16.1%; grandmother, 1.8%; other, 0.9%) were included in the analysis (Fig. 1, Table 1). In the ROC analysis, the AUC for the guardians’ influenza diagnosis was 0.69 (95% confidence interval [CI]: 0.59–0.79). The optimal cut-off point was 70%, and the sensitivity, specificity, positive likelihood ratio, and negative likelihood ratio were 59.6% (95% CI: 45.8–72.4%), 69.1% (95% CI: 55.2–80.9%), 1.93 (95% CI: 1.23–3.02), and 0.58 (95% CI: 0.41–0.84), respectively (Fig. 2). The negative likelihood ratio at a 30% cut-off point was 0.12 (95% CI: 0.02–0.93), and the positive likelihood ratio at a 90% cut-off point was 2.77 (95% CI: 1.36–5.67). Table 2 shows the sensitivity and specificity of guardians’ influenza diagnosis at various cut-off points.

[Image omitted: See PDF]

[Image omitted: See PDF]

[Image omitted: See PDF]

[Image omitted: See PDF]

In subgroup analyses, the AUCs were higher for: mothers (0.72), guardians of children with an influenza history (0.72), guardians who were aware of the epidemic (0.71), and guardians of unvaccinated children (0.76). AUCs were lower for guardians whose children were aged <4 years (0.52) or who were unaware of the epidemic (0.57); and were lowest for fathers (0.48) (Table 3). The ROC curve and sensitivity and specificity at major cut-off points for each subgroup are shown in Supplementary File S3.

[Image omitted: See PDF]

The AUC increased significantly from 0.79 to 0.85 (P = 0.04) when adding information provided by the guardian (the guardians' influenza diagnosis likelihood estimate (%), guardian type, patient’s history of contact with persons with influenza, patient’s history of influenza, and patient’s influenza vaccination status) and basic clinical information (age, sex, headache, nasal discharge, cough, joint and muscle pain, fatigue, and axillary temperature taken at home) (Fig. 3).

[Image omitted: See PDF]

Discussion

This prospective observational study confirmed that guardians’ quantitative influenza diagnosis was useful for both definitive and exclusionary diagnoses, especially when the guardians’ reports revealed an extremely high (85.5% specificity at 90% cut-off point) or low (98.2% sensitivity at 30% cut-off point) probability, and identified factors that improve guardians’ diagnostic accuracy. Multivariable logistic regression analysis showed that information on guardians’ diagnosis and factors that improve guardians’ diagnostic accuracy further increased overall clinical diagnostic accuracy when added to basic clinical information for an influenza diagnosis.

A previous report showed that guardians’ qualitative influenza diagnosis had a low diagnostic accuracy (sensitivity 42.4%, specificity 52.6%).¹⁸ In addition, an earlier retrospective observational study found that the guardians’ quantitative influenza diagnosis could change the cut-off point in the same manner as ordinary laboratory data. Therefore, a quantitative diagnosis was more useful than a qualitative diagnosis for diagnosing influenza (90.9% sensitivity at a 50% cut-off point, 92.3% specificity at an 80% cut-off point, AUC 0.82).¹⁰ This prospective observational study confirmed the accuracy of guardians’ quantitative influenza diagnosis in their children (98.2% sensitivity at 30% cut-off point, 85.5% specificity at 90% cut-off point, AUC 0.69), and the results are consistent with those of a previous retrospective observational study.⁹ The AUC of this study was lower than that reported previously, which may be due to random error. The accuracy of physicians’ qualitative clinical diagnosis of influenza in children (sensitivity 38%, specificity 91%) has been reported, which corresponds to a cut-off value of 90% for guardian diagnosis in our study.¹⁹

Implications for practice

Influenza is a common disease, and many patients recover after symptomatic treatment. However, some patients develop severe disease; the influenza virus causes pandemics. Utilisation of guardians’ diagnoses could have some social benefits. First, it may reduce the need to visit medical institutions for an influenza diagnosis, thus reducing virus transmission. Aside from the examination performed at a medical institution, the guardians’ diagnosis can be by a remote medical interview.

Second, even when a patient visits a medical institution, a guardian’s diagnosis can improve a physician’s influenza diagnostic accuracy. Generally, influenza tends to be overlooked, especially in the early stage, which sometimes delays patients’ recovery.²⁰ This study showed that a guardian’s diagnosis is useful even in the early stage of influenza when the sensitivity of the rapid influenza diagnostic test is low.²¹ A more accurate diagnosis by physicians in the early stage of influenza can improve disease control in the community.

Guardians’ diagnoses of illnesses in their children are a useful social resource in emergencies, such as pandemics. During emergencies, it is reasonable and efficient to use the guardians’ diagnosis for infection control. For example, when the value of a guardian’s diagnosis is extremely high because of factors that can improve the diagnosis, treatment can commence without the child visiting a medical institution. In the future, the medical information obtained outside medical institutions will increase because of the innovations in information and communication technology.²² This may affect guardians’ diagnosis and the use of over-the-counter drugs.⁸ In the future, physicians may be increasingly required to resolve guardians’ concerns caused by excessive medical information and to utilise the guardians’ diagnoses.²³

Strengths and limitations

This prospective study had less selection bias than the previous retrospective study⁹ and showed the usefulness of guardians’ quantitative influenza diagnosis of their children. The diagnostic accuracy tended to be lower in guardians of children aged <4 years and higher in guardians whose children had not been vaccinated or who had a history of influenza. This result indicates that diagnosis by guardians is difficult with children aged <4 years, as young children cannot express their subjective symptoms clearly. In contrast, symptoms in unvaccinated children may be typical of influenza and therefore, it may be relatively easy for guardians to diagnose influenza in their children. Prior experience of influenza in their children may improve the diagnostic accuracy because guardians can compare their children’s current and past influenza symptoms.

Second, we examined factors related to guardians’ characteristics. Diagnostic accuracy tended to be higher in mothers and lower in fathers. Among the developed countries, fathers in Japan spend the least time performing housework and childcare²⁴ and only 14.5–23.5% of fathers take time off from work to take care of their children when they are sick.²⁵ This may be related to Japan’s working culture rather than gender differences in diagnostic accuracy.

Finally, we reviewed factors related to the environment in which guardians made their diagnosis. The accuracy of diagnosis tended to be higher in guardians who were aware of the influenza epidemic situation. We considered this an environment-related factor rather than a factor related to the guardian’s characteristics. An environment where information on epidemics can be accurately shared, such as schools and nurseries, is likely to effectively improve a guardian’s diagnosis by comparing symptoms with other patients and considering the possibility of infectious exposure such as contact with other known patients.

Our study has some limitations. We did not include a sufficient number of younger children, such as infants. Younger children may have fewer symptoms than older children and are less able to communicate their symptoms to their guardian, which may negatively affect the guardians’ diagnostic accuracy. Larger studies with a sufficient number of patients in each age group for subgroup analysis are needed.

Second, to perform reliable research, more accurate reference standards should be used. In Japan, it is usually difficult to perform polymerase chain reaction tests or virus cultures that are more reliable for influenza diagnosis than rapid influenza diagnostic tests, especially in primary care clinics. Furthermore, the strains observed in the 2018–19 influenza season and the 2017–18 influenza season were different.²⁶ More accurate virus testing methods are needed to clarify the difference in the guardians’ diagnostic accuracy between different strains of the influenza virus.

As well, the study could not adequately analyse the guardians’ background, such as smartphone and internet use, educational background, work, family and friend structure, and medical history, because data were collected from the medical chart and pre-examination checklist of children who were suspected to have influenza. A more detailed assessment of the guardians’ diagnostic accuracy requires a different approach to data collection beyond general practice for influenza. In addition, the accuracy of the guardians’ diagnosis needs to be verified in other countries. Influenza is a common disease because it occurs repeatedly worldwide every year and the symptoms are relatively easy to understand. However, considering the differences in national healthcare systems and the people’s behaviour toward influenza, differences in guardians’ diagnostic accuracy should be assessed separately in each country.

Conclusions

We conducted a prospective observational study and confirmed that guardians’ influenza diagnosis of their children was useful for both definitive and exclusionary diagnoses, especially when their reports revealed an extremely high or low probability, and we identified some factors that could be used to improve the accuracy of guardians’ influenza diagnosis. Moreover, the results revealed that adding information about guardians’ diagnosis and factors that improve guardians’ diagnosis to basic clinical information can improve physicians’ clinical diagnostic accuracy. Guardians’ influenza diagnosis of their children could prevent the spread of the infection and reduce its social burden.

Data availability

The data generated and analysed during this study are mostly included in this published article. Additional data are available from the corresponding author on reasonable request.

Competing interests

The authors have no competing interests to declare.

Declaration of funding

This study was conducted without external funding.

Prior presentation

Some of the results contained in this manuscript were presented at the 11th Annual Conference of the Japan Primary Care Association, which was held online in Hiroshima from 23 July to 31 August 2020 (https://www.c-linkage.co.jp/jpca2020/en/).

Supplementary material

Supplementary material is available online.

References

[1] National Institute of Infectious Diseases. Influenza in this winter (2019/20 season). Tokyo: National Institute of Infectious Diseases; 2020. Available from https://www.niid.go.jp/niid/images/idsc/disease/influ/fludoco1920.pdf [Accessed 15 April 2021].

[2] Centers for Disease Control and Prevention. Estimated Influenza Illnesses, Medical visits, Hospitalizations, and Deaths in the United States — 2019–2020 Influenza Season. Atlanta: Centers for Disease Control and Prevention; 2021. Available at https://www.cdc.gov/flu/about/burden/2019-2020.html [Accessed 15 April 2021].

[3] Cauchemez S, Ferguson NM, Fox A, et al. Determinants of influenza transmission in South East Asia: insights from a household cohort study in Vietnam. PLoS Pathog 2014; 10 e1004310

[4] Pannaraj PS, Wang HL, Rivas H, et al. School-located influenza vaccination decreases laboratory-confirmed influenza and improves school attendance. Clin Infect Dis 2014; 59 325–32.

[5] Ng S, Lopez R, Kuan G, et al. The timeline of influenza virus shedding in children and adults in a household transmission study of influenza in Managua, Nicaragua. Pediatr Infect Dis J 2016; 35 583–6.

[6] Memoli MJ, Athota R, Reed S, et al. The natural history of influenza infection in the severely immunocompromised vs nonimmunocompromised hosts. Clin Infect Dis 2014; 58 214–24.

[7] Molinari NAM, Ortega-Sanchez IR, Messonnier ML, et al. The annual impact of seasonal influenza in the US: measuring disease burden and costs. Vaccine 2007; 25 5086–96.

[8] Robertson N, Polonsky M, McQuilken L. Are my symptoms serious Dr Google? A resource-based typology of value co-destruction in online self-diagnosis. Australas Market J 2014; 22 246–56.

[9] Maita H, Kobayashi T, Osawa H, Kato H. Seasonal influenza among children diagnosed by their guardians: a small pilot study in Japan. Prim Health Care Res Dev 2018; 20 55

[10] Maita H, Kobayashi T, Osawa H, Kato H. Self-diagnosis of seasonal influenza in a rural primary care setting in Japan: a cross sectional observational study. PLoS One 2018; 13 e0197163

[11] Gupta K, Hooton TM, Roberts PL, Stamm WE. Patient-initiated treatment of uncomplicated recurrent urinary tract infections in young women. Ann Intern Med 2001; 135 9–16.

[12] Bent S, Nallamothu BK, Simel DL, et al. Does this woman have an acute uncomplicated urinary tract infection? JAMA 2002; 287 2701–10.

[13] Pilger D, Khakban A, Heukelbach J, Feldmeier H. Self-diagnosis of active head lice infestation by individuals from an impoverished community: high sensitivity and specificity. Rev Inst Med Trop Sao Paulo 2008; 50 121–2.

[14] Abbott J. Clinical and microscopic diagnosis of vaginal yeast infection: a prospective analysis. Ann Emerg Med 1995; 25 587–91.

[15] Weakley J, Webber MP, Ye F, et al. Agreement between obstructive airways disease diagnoses from self-report questionnaires and medical records. Prev Med 2013; 57 38–42.

[16] Greiner M, Pfeiffer D, Smith RD. Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests. Prev Vet Med 2000; 45 23–41.

[17] Kanda Y. Investigation of the freely available easy-to-use software ‘EZR’ for medical statistics. Bone Marrow Transplant 2013; 48 452–8.

[18] Jutel A, Baker MG, Stanley J, et al. Self-diagnosis of influenza during a pandemic: a cross-sectional survey. BMJ Open 2011; 1 e000234

[19] Peltola V, Reunanen T, Ziegler T, et al. Accuracy of clinical diagnosis of influenza in outpatient children. Clin Infect Dis 2005; 41 1198–200.

[20] Aoki FY, Macleod MD, Paggiaro P, et al. Early administration of oral oseltamivir increases the benefits of influenza treatment. J Antimicrob Chemother 2003; 51 123–9.

[21] Mitamura K, Sugaya N. [Diagnosis and Treatment of influenza-clinical investigation on viral shedding in children with influenza]. Uirusu 2006; 56 109–16. [in Japanese]

[22] Tan SSL, Goonawardene N. Internet health information seeking and the patient-physician relationship: a systematic review. J Med Internet Res 2017; 19 e9

[23] Kobayashi T, Maita H, Kato H. Challenges on the medical front due to individualisation of medical equipment in ageing developed countries, and proposed solutions. BMJ Innovations 2017; 3 127–9.

[24] Government of Japan Cabinet Office. Husband’s cooperation. Tokyo: Government of Japan Cabinet Office; 2019. Available at https://www8.cao.go.jp/shoushi/shoushika/data/ottonokyouryoku.html [Accessed 13 April 2021].

[25] Kato H. Relational factors in having children. Bulletin of the Continuing Education Research Institute, Hokkaido Asai Gakuen University 2001; 1 81–95.

[26] Ministry of Health, Labour, and Welfare. Weekly reports of influenza isolation/detection in Japan, 2014/15 – 2018/2019 season. Infectious Agents Surveillance Report. Tokyo: Ministry of Health, Labour, and Welfare; 2019. Available at https://nesid4g.mhlw.go.jp/Byogentai/Pdf/data2j.pdf [Accessed 13 April 2021].