Developing a video expert panel as a reference standard to evaluate respiratory rate counting in paediatric pneumonia diagnosis: protocol for a cross-sectional study

Abstract

Introduction

Manual counting of respiratory rate (RR) in children is challenging for health workers and can result in misdiagnosis of pneumonia. Some novel RR counting devices automate the counting of RR and classification of fast breathing. The absence of an appropriate reference standard to evaluate the performance of these devices is a challenge. If good quality videos could be captured, with RR interpretation from these videos systematically conducted by an expert panel, it could act as a reference standard. This study is designed to develop a video expert panel (VEP) as a reference standard to evaluate RR counting for identifying pneumonia in children.

Methods and analysis

Using a cross-sectional design, we will enrol children aged 0–59 months presenting with suspected pneumonia at different levels of health facilities in Dhaka and Sylhet, Bangladesh. We will videorecord a physician/health worker counting RR manually and also using an automated RR counter (Children’s Automated Respiration Monitor) from each child. We will establish a standard operating procedure for capturing quality videos, make a set of reference videos, and train and standardise the VEP members using the reference videos. After that, we will assess the performance of the VEP as a reference standard to evaluate RR counting. We will calculate the mean difference and proportions of agreement within±2 breaths per minute and create Bland-Altman plots with limits of agreement between VEP members.

Ethics and dissemination

The study protocol was approved by the National Research Ethics Committee of Bangladesh Medical Research Council, Bangladesh (registration number: 39315022021) and Edinburgh Medical School Research Ethics Committee (EMREC), Edinburgh, UK (REC Reference: 21-EMREC-040). Dissemination of the study findings will be through conference presentations and publications in peer-reviewed scientific journals.

Correspondence to Dr Ahad Mahmud Khan; [email protected]

STRENGTHS AND LIMITATIONS OF THIS STUDY

• Expert paediatricians will provide feedback to develop a standard operating procedure for videography of child chest movements.

• The video expert panel will be trained and standardised using the expert paediatrician-interpreted reference videos.

• Video expert panel members will be masked to respiratory rate counted by each other and to respiratory rate manual counts, and with the automated counter.

• Children with varying severity of illness will be enrolled from different levels of health facilities in Bangladesh.

• Despite the availability of multiple respiratory rate counters (eg, Children’s Automated Respiration Monitor, ChARM, Rad-G, uPM60), only the ChARM device will be used, which is a limitation in this study.

Introduction

Pneumonia is one of the leading causes of mortality in children aged below 5 years worldwide.¹ Approximately 68 million pneumonia episodes and 0.65 million deaths occur annually in under-5 children due to pneumonia.² The highest number of deaths due to pneumonia among children below 5 years were in sub-Saharan Africa, South Asia and Southeast Asia. There were an estimated 2.7 million episodes of pneumonia in Bangladesh and 21 275 deaths due to childhood pneumonia in 2015.³

According to the WHO guidelines, pneumonia diagnosis in children for frontline health workers is primarily based on increased respiratory rate (RR) and/or chest indrawing.^{4 5} Fast breathing is the most common sign of pneumonia. It is most commonly identified by counting the RR manually.⁵ However, manual counting can be challenging for the health workers,⁶ often leading to incorrect diagnosis and, consequently, inappropriate treatment.^7–9

Better performing RR diagnostics to support frontline health workers might improve the diagnosis of pneumonia. Three automated RR counting devices are considered suitable for use by health workers, that is, Children’s Automated Respiration Monitor (ChARM),^{10 11} Rad-G^{12 13} and uPM60.¹⁴ The Philips ChARM converts chest movements detected by accelerometers into a precise breathing count using specially designed algorithms. The device is placed around the child’s belly, and it automatically counts the number of respirations for a particular period, calculates RR and classifies fast breathing according to WHO guidelines.^{10 11} The usability and acceptability of this device have been tested in a study in Ethiopia¹⁵ and Nepal.¹⁶

A reference standard is essential to evaluate the performance of RR counters in field settings. However, the absence of an appropriate reference standard is a challenge. Different reference standards have been used in various studies.^{17 18} Most of the existing studies have used manual RR count by an expert person (eg, physician, nurse) as the reference standard.¹⁹ Few studies have been found using automated monitors.^{20 21} The possible biases using a human expert’s count as the reference standard include difficulty in measuring the RR over the same simultaneous period and inconsistencies in human expert RR counting.⁶ The automated monitors do not measure chest movements directly, but other variables such as carbon dioxide, pulse oximeter signal or photoplethysmogram, sound etc, are used to extract RR.^{19 22–24}

Videography of a child’s chest movements and interpretation by the experts could be used as a reference standard.^{17 18 25} If quality videos could be recorded and the interpretation of RR from these videos could be systematically conducted by a video expert panel (VEP), it could be an ideal and non-biased reference standard. This study aims to develop a VEP as a reference standard for evaluating RR counting manually and using ChARM in paediatric pneumonia diagnosis. We will establish a standard operating procedure (SOP) for capturing quality videos, make a set of reference videos, and train and standardise the VEP members using the reference videos. The SOP, recorded videos and evaluation methods could be used for evaluating new RR counting devices.

Study objectives

Primary objectives

• To develop an SOP for video recording a child’s chest movements for RR interpretation.

• To create a set of reference videos for the training and standardisation of the VEP members.

• To train and standardise the VEP members to interpret RR using the reference videos.

• To assess the performance of the VEP to evaluate RR counting manually and with ChARM based on videos captured in real-world settings.

Secondary objectives

• To assess the accuracy of the ChARM device in counting RR compared with the VEP as the reference.

• To determine the duration of counting RR using the ChARM device.

• To explore the potential influence of the ChARM device on RR count compared with standard observation techniques.

• To assess the agreement of manual RR count by healthcare staff with RR counted by VEP.

Methods and analysis

Study design and settings

This will be a cross-sectional study. First, we will capture videos of child chest movements, develop an SOP of videography, make a set of reference videos and use these reference videos to train and standardise the VEP. We will record these videos from the inpatient department (IPD) of the Institute of Child and Mother Health (ICMH), Dhaka, Bangladesh where children with different severity of illness are admitted. After that, we will use the VEP as a reference standard to evaluate RR counting. The videos will be recorded from different levels of health facilities in Bangladesh. Three community clinics (CCs) and a subdistrict hospital (Zakiganj Upazila Health Complex) in Sylhet and ICMH in Dhaka will be selected. CCs are the lower-level health facility in Bangladesh staffed by community healthcare providers (CHCP).²⁶ Patients from the CCs are referred to subdistrict hospitals. Patients often go to subdistrict hospitals directly.²⁷ The study started on 6 December 2021 and is planned to be completed by December 2022.

Study population

Inclusion criteria

• Age <2 months presenting with any illness.

• Age 2–59 months presenting with cough and/or difficulty breathing.

Exclusion criteria

• Presence of any danger sign (unable to drink or feed, vomit everything, convulsions, lethargic or unconscious).

• The parents are unwilling to provide consent.

Study procedures

Enrolment and consenting

In the hospital, a physician will screen all children who will visit the outpatient clinic (subdistrict hospital) or are already admitted to IPD (ICMH) and enrol eligible children. In the CC, the CHCP will screen every child who will visit his/her CC. If a child becomes eligible, the CHCP will enrol the child and contact research staff who will visit the CC immediately and will complete data collection. Informed written consent will be obtained from the child’s mother, father or other available caregiver before data collection.

Developing SOP of videography

Research staff will record videos of child chest movements from 30 children. The Canon EOS M50 camera²⁸ will be used for recording videos. If a good source of natural light is not available, we will use a light-emitting diode lamp for lighting. The child’s clothes will be removed from the lower end of the neck through the umbilicus so that the chest and belly are exposed. The child’s face or any other identifiable part will not be recorded. The videography procedure (eg, camera positions, angles and levels of light, use of tripod) will be varied between children depending on the child and environmental factors (eg, child condition, position, adequate exposure, place of assessment). Videorecording will be done both when counting RR manually twice and using the ChARM device twice.

Once the child is calm, the research staff will start videorecording, and the physician will tap the microphone with his/her finger and begin counting RR. The physician will count RR for 1 min and tap the microphone again when counting is finished, and the research staff will stop the recording. After that, the physician will attach the ChARM device, and the research staff will start videorecording. Once the child is calm, the physician will start the ChARM device and also begin counting RR manually, and the videorecording will be done in parallel to the physician’s use of ChARM like before. The physician will count the total number of breaths until ChARM completes its counting. The ChARM device will show a green or red signal when it finishes its count. The videorecording will continue until the device is done. The time taken by the ChARM device to get the RR will be recorded. A maximum of 3 min will be allowed for the ChARM to register a reading or display an error message. Otherwise, it will be documented as a failed attempt. The physician will make a maximum of three attempts for each manual count and ChARM count.

Each video file will be saved in the camera with a file name. The research staff will note the file name in the data collection sheet. Then the videos will be downloaded from the camera and saved on a password-protected computer and the videos will be deleted the camera. After that he will edit the video by removing additional portions of the recording, removing sound or identifiable features, and reducing file size. Video Editing Software (Adobe Premiere Pro) will be used for video editing.

There will be two expert paediatricians (ie, medical practitioners having a postgraduate degree in paediatrics and specialised in treating sick children). These videos will be sent to them for their feedback on video quality (decision of using tripod, camera positions, angles, light or other issues) and to count RR. Their feedback will be incorporated, and the videorecording procedures will be modified accordingly. An SOP for videography will be developed to maintain consistency across all study hospitals/clinics.

Figure 1 shows the schematic diagram of developing the SOP for videography.

Figure 1. Developing SOP of videography. ChARM, Children’s Automated Respiration Monitor; SOP, standard operating procedure .

Making a set of reference videos

Following the development of the SOP, videos will be recorded from 50 children using this standardised procedure. A physician will count the RR manually once and with the ChARM device once, and videorecording will be done simultaneously. The same procedure described in figure 1 will be followed. If the physician fails in their first attempt to have a satisfactory recording, they will make two more attempts for each manual and ChARM counts. The reason for the attempted failure will be documented. The condition of the child during RR measurement will also be noted.

There will be three expert paediatricians. Each video will randomly be allocated to two of the paediatricians. If both paediatricians disagree, that is, disagreement in readability or difference of RR>±2 breaths per minute (bpm), then the video will be sent to the third paediatrician. If any two paediatricians agree, that is, agreement in interpretability and difference of RR≤±2 bpm, the videos will be considered reference videos. The mean RR count of paediatricians in agreement will be considered as the final RR count.

Figure 2 presents the schematic diagram of making a set of reference videos.

Figure 2. Making a set of reference videos. ChARM, Children’s Automated Respiration Monitor, RR, respiratory rate.

Training and standardisation of VEP members

Six local physicians with a Bachelor of Medicine and Bachelor of Surgery(MBBS) degree will be invited for training and standardisation. The training session will be conducted using an online platform. The physicians will be trained on counting RR using videos of known RR counts with different level of breathing from reference videos. Training will be provided following the WHO Integrated Management of Childhood Illness guidelines.^{4 5} The principal investigator (PI) will conduct the training sessions. After completion of training, they will be evaluated individually in a blinded manner. Twenty videos with different level of breathing will be assigned to each VEP member from the reference videos. The RR interpreted by each VEP member will be considered accurate if this is within±2 bpm of the RR interpreted by expert paediatricians. Those physicians who achieve a pass mark (at least 80%) in the evaluation will be considered to work as VEP members in the study.

Evaluating the performance of VEP

The performance of VEP will be evaluated using the videos recorded in different levels of health facilities (CCs, subdistrict hospital and ICMH). RR of each child will be assessed twice, that is, manually and using ChARM by the CHCP in the CC and by a physician in the hospital (as described in figure 1). The videos will be recorded following the SOP. Each video will be randomly assigned to two VEP members to interpret the RR. If there is an agreement of being uninterpretable, the videos will be excluded. If the RR is≤±2 bpm between both members, their average RR will be considered final. However, if there is a disagreement in interpretability or the difference of the RR count is >2 bpm between the first two members, the video will be sent to a third-panel member. If there are inconsistencies among all three members (if the RR is not ≤2 bpm between any two members), the videos will be sent to a fourth-panel member. If there is agreement between the fourth-panel member and any other member who read the video earlier, their average RR will be considered final. However, if there is no agreement between any two panel members, the videos will be excluded from the analysis (figure 3).

Figure 3. Respiratory rate interpretation from videorecording by video expert panel. ChARM, Children’s Automated Respiration Monitor, RR, respiratory rate; VEP, video expert panel.

Quality control measures

The videos will be checked periodically if they adhere to the SOP to ensure quality by the PI. Any quality issues will be reported, and corrective feedback will be provided. The expert paediatricians will view a 10% random sample of videos in which they are blinded to the panel’s final classification. The panel’s overall performance will be evaluated using the expert’s interpretation as the reference. Lastly, to monitor individual member interpretation reproducibility, all members will randomly be reassigned 20% of images they interpreted previously (at least 1 week earlier) to check intrareader agreement.

Development of a web-based automated system

A web-based automated system will be developed. This system will use the Hypertext Preprocessor platform for the user interface and the Structured Query Language (SQL) platform for the database. The purpose of developing this system is an automated distribution of the videorecordings among the expert paediatricians and VEP members, the input of readings by the members and the generation of automated reports. The system will be password protected and connected to a secured server. The expert paediatricians and VEP members will use this programme. Each will need to log in with their user ID and password. Access to each expert paediatrician and VEP member will be limited to what they are assigned. When logged in, they will see the list of recordings assigned to them. They will open the video on the computers and interpret the RR. The VEP members will be blinded to other members’ readings.

Operational definitions

Interpretability

The video will be considered ‘interpretable’ when the expert paediatrician or VEP member will be able to view the child’s chest movements and measure RR for the whole duration. On the other hand, a video will be counted as ‘uninterpretable’ if the physician will not be able to view and count RR for the whole duration of the recording. The videos can be unreadable for various reasons, for example, wrong position and angle of the camera, inappropriate focus, poor lighting, inadequate exposure or if the child is agitated, moving, crying, etc.

Agreement and disagreement

The agreement will be defined when two expert paediatricians or VEP members interpret the video as

• uninterpretable.

• Interpretable and the difference between RR count≤±2 bpm.

Disagreement is when

• one expert paediatrician/VEP member interprets the video as interpretable, and another interprets it as unreadable or

• the difference between their RR count is>2 bpm.

Data collection and storage

Field data (age, sex, RR, child condition during assessment) will be collected in paper forms. After entry, data will be transferred to a password-protected server (SQL Server 2008 R2) located in Dhaka, Bangladesh, in real-time using internet connectivity. Recorded videos will be transferred to encrypted OneDrive/Dropbox cloud storage. The VEP members will interpret RR from recorded videos using the web-based automated software, and the data will also be stored on the server in real time. Deidentified data will be transferred and stored on the DataShare server at the University of Edinburgh, UK.

Sample size calculation

The videos from the first 30 children will be used to establish the best procedure for capturing videos. This number was purposively chosen as a qualitative assessment of the videos will be done. We assume that 30 children would be enough to develop the SOP. However, we will consider recruiting more if after 30 children a consistent SOP has not been finalised. After finalisation of the SOP, the videos from 50 children will be used for making a set of reference videos to train and standardise VEP. We did not use any statistical formula to calculate this sample size. These children will be recruited from the IPD of a hospital. Based on our experience in this setting, we assume that we will get a significant number of children with fast breathing. However, we will consider recruiting further children if our target has not been achieved. To evaluate the performance of VEP, Bland-Altman’s statistical methods for assessing agreement between two methods of clinical measurement are used.^{29 30} A total of 226 interpretable videos of each method (manual and ChARM) are required, which will provide 90% power to assess agreement in RR counts between two VEP members, considering type-I error α=0.05, expected mean difference±0.5, expected SD 1.5 and maximum allowed difference 4. Assuming 80% interpretability of the videos and 20% failure to record videos, the required number of children is about 350.

Statistical analysis

To standardise the VEP members, we will estimate the mean difference of RR counts and the percent agreement within±2 bpm between each VEP member and expert paediatricians. We will consider expert paediatricians’ counts as the gold standard. To assess the performance of VEP members, we will produce Bland-Altman plots²⁹ to assess the agreement in RR counts and calculate the percent agreement between primary panel members. We will calculate mean difference of RR counts and the per cent agreement within±2 bpm to assess the intrareader agreement of each VEP member. We will use Cohen’s kappa statistic to estimate the inter-reader agreement of identifying fast breathing between panel members. We will also measure the mean difference of RR counts and the percent agreement within±2 bpm between each VEP member and expert paediatricians from QC data. To evaluate the accuracy of the ChARM device and manual counts, we will produce Bland-Altman plots to assess the agreement in RR counts of ChARM counts, and manual counts with VEP counts. We will measure the mean difference of RR counts and the percent agreement within±2 bpm. We will also use Cohen’s kappa statistic to assess the agreement in identifying fast breathing between these methods. The mean time to measure the RR by the ChARM device will be assessed. To assess the potential influence of the ChARM device on the RR of the children, we will calculate the mean difference of RR between the first and second measurements interpreted by the VEP.

Ethics and dissemination

Ethical approval was obtained from the National Research Ethics Committee of Bangladesh Medical Research Council, Bangladesh (Registration Number: 39315022021), and Edinburgh Medical School Research Ethics Committee (EMREC), Edinburgh, UK (REC Reference: 21-EMREC-040). Informed written consent will be taken from the parent or guardian of each child. Dissemination will be through conference presentations and publications in peer-reviewed journals. Anonymised data files will also be stored securely in the DataStore repository at the University of Edinburgh, UK and will be shared after the publication of the main paper.

The authors are grateful for the support and contributions of Dr Arunangshu Dutta Roy, Asim Nehal, Rizouan Ur Rashid, Rakib Bhuiyan, Dr Nusrat Sharmin Asma, Dr Farjana Hossain, Dr Muhammad Shariful Islam, Dr Md. Jahurul Islam, and the Ministry of Health and Family Welfare, Government of Bangladesh. The authors also thank RESPIRE collaboration for their contribution in bringing the manuscript to its final shape. The RESPIRE collaboration comprises the UK Grant holders, Partners and research teams as listed on the RESPIRE website (www.ed.ac.uk/usher/respire) including Linda Bauld.

Ethics statements

Patient consent for publication

Not applicable.

Footnote

Twitter @tinylungsglobal, @CarinaTKing

Contributors AMK and HC conceptualised and designed this study. HC, SC, AB, EDM and TS provided mentorship to AMK. HC, SC, EDM and CK critically reviewed the study design. AMK, SA, MMR will be involved in project management. KN, RS and AA will review the videos and provide feedback to video quality. NHC and MSI will be responsible for data management. AMK drafted the manuscript, and all authors critically reviewed and approved the final manuscript before submission.

Funding This research was funded by the UK National Institute for Health Research (NIHR) (Global Health Research Unit on Respiratory Health (RESPIRE); 16/136/109) using UK aid from the UK Government to support global health research.

Disclaimer The views expressed in this publication are those of the author(s) and not necessarily those of the NIHR or the UK Government.

Competing interests None declared.

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review Not commissioned; externally peer reviewed.

References

¹ Liu L, Oza S, Hogan D, et al. Global, regional, and national causes of under-5 mortality in 2000-15: an updated systematic analysis with implications for the sustainable development goals. Lancet 2016; 388: 3027–35. doi:10.1016/S0140-6736(16)31593-8 http://www.ncbi.nlm.nih.gov/pubmed/27839855

² GBD 2016 Lower Respiratory Infections Collaborators. Estimates of the global, regional, and national morbidity, mortality, and aetiologies of lower respiratory infections in 195 countries, 1990-2016: a systematic analysis for the global burden of disease study 2016. Lancet Infect Dis 2018; 18: 1191–210. doi:10.1016/S1473-3099(18)30310-4 http://www.ncbi.nlm.nih.gov/pubmed/30243584

³ GBD 2015 LRI Collaborators. Estimates of the global, regional, and national morbidity, mortality, and aetiologies of lower respiratory tract infections in 195 countries: a systematic analysis for the global burden of disease study 2015. Lancet Infect Dis 2017; 17: 1133–61. doi:10.1016/S1473-3099(17)30396-1 http://www.ncbi.nlm.nih.gov/pubmed/28843578

⁴ World Health Organization. IMCI information package. Geneva: Integrated Management of Childhood Illness (IMCI, 1999.

⁵ World Health Organization. Integrated management of childhood illness: chart booklet. Geneva: World Health Organization, 2014.

⁶ Khan AM, O’Donald A, Shi T, et al. Accuracy of non-physician health workers in respiratory rate measurement to identify paediatric pneumonia in low- and middle-income countries: a systematic review and meta-analysis. J Glob Health 2022; 12: 04037. doi:10.7189/jogh.12.04037

⁷ Källander K, Tomson G, Nsabagasani X, et al. Can community health workers and caretakers recognise pneumonia in children? experiences from Western Uganda. Trans R Soc Trop Med Hyg 2006; 100: 956–63. doi:10.1016/j.trstmh.2005.11.004 http://www.ncbi.nlm.nih.gov/pubmed/16455119

⁸ Mukanga D, Babirye R, Peterson S, et al. Can lay community health workers be trained to use diagnostics to distinguish and treat malaria and pneumonia in children? lessons from rural Uganda. Trop Med Int Health 2011; 16: 1234–42. doi:10.1111/j.1365-3156.2011.02831.x http://www.ncbi.nlm.nih.gov/pubmed/21752163

⁹ Miller NP, Amouzou A, Tafesse M, et al. Integrated community case management of childhood illness in Ethiopia: implementation strength and quality of care. Am J Trop Med Hyg 2014; 91: 424–34. doi:10.4269/ajtmh.13-0751 http://www.ncbi.nlm.nih.gov/pubmed/24799369

¹⁰ Shah R, Jenda G, Lwesha V, et al. An integrated diagnostic device for neonatal sepsis and childhood pneumonia. J Public Health Afr 2018; 9: 780. doi:10.4081/jphia.2018.780 http://www.ncbi.nlm.nih.gov/pubmed/30079170

¹¹ Philips. Philips introduces new diagnostic device to help prevent childhood pneumonia in low-resource countries, 2019. Available: file:///C:/Users/Ahad/Downloads/Backgrounder%20-%20Philips%20Children%E2%80%99s%20Automated%20Respiration%20Monitor%20(1).pdf [Accessed 18 Nov 2019 ].

¹² Masimo. Masimo introduces Rad-G™ pulse oximeter, 2017. Available: http://masimocorp.gcs-web.com/news-releases/news-release-details/masimo-introduces-rad-gtm-pulse-oximeter [Accessed 07 Nov 2018 ].

¹³ Trademarkia. RAD-G Trademark information, 2017. Available: https://www.trademarkia.com/radg-87345900.html [Accessed 07 Nov 2018 ].

¹⁴ Utech. uPM60 handheld vital signs monitor, 2020. Available: http://www.chinautech.com/cn/col.jsp?id=161 [Accessed 10 May 2020 ].

¹⁵ Ward C, Baker K, Smith H, et al. Usability and acceptability of an automated respiratory rate counter to assess children for symptoms of pneumonia: A cross-sectional study in Ethiopia. Acta Paediatr 2020; 109: 1196–206. doi:10.1111/apa.15074 http://www.ncbi.nlm.nih.gov/pubmed/31638714

¹⁶ Källander K, Ward C, Smith H, et al. Usability and acceptability of an automated respiratory rate counter to assess childhood pneumonia in Nepal. Acta Paediatr 2020; 109: 1207–20. doi:10.1111/apa.15108 http://www.ncbi.nlm.nih.gov/pubmed/31762072

¹⁷ Källander K, Marks S, Baker K, et al. Selecting a reference standard for evaluating respiratory rate devices to diagnose symptoms of pneumonia in children under 5: lessons learned from resource-poor settings in sub-Saharan Africa and Asia. American Journal of Tropical Medicine and Hygiene Conference 2017, 2017.

¹⁸ Ansermino JM, Dumont G, Ginsburg AS. How uncertain is our reference standard for respiratory rate measurement? Am J Respir Crit Care Med 2019; 199: 1036–7. doi:10.1164/rccm.201812-2266LE http://www.ncbi.nlm.nih.gov/pubmed/30673505

¹⁹ Ginsburg AS, Lenahan JL, Izadnegahdar R, et al. A systematic review of tools to measure respiratory rate in order to identify childhood pneumonia. Am J Respir Crit Care Med 2018; 197: 1116–27. doi:10.1164/rccm.201711-2233CI http://www.ncbi.nlm.nih.gov/pubmed/29474107

²⁰ Baker K, Alfvén T, Mucunguzi A, et al. Performance of four respiratory rate counters to support community health workers to detect the symptoms of pneumonia in children in low resource settings: a prospective, multicentre, hospital-based, single-blinded, comparative trial. EClinicalMedicine 2019; 12: 20–30. doi:10.1016/j.eclinm.2019.05.013 http://www.ncbi.nlm.nih.gov/pubmed/31388660

²¹ Becker C, Achermann S, Rocque M, et al. Camera-based measurement of respiratory rates is reliable. Eur J Emerg Med 2018; 25: 416–22. doi:10.1097/MEJ.0000000000000476 http://www.ncbi.nlm.nih.gov/pubmed/28574856

²² Charlton PH, Bonnici T, Tarassenko L, et al. An assessment of algorithms to estimate respiratory rate from the electrocardiogram and photoplethysmogram. Physiol Meas 2016; 37: 610–26. doi:10.1088/0967-3334/37/4/610 http://www.ncbi.nlm.nih.gov/pubmed/27027672

²³ Li W, Liu J, Yu L. [Study on Respiratory Rate Monitoring Method Based on Breath Sounds]. Zhongguo Yi Liao Qi Xie Za Zhi 2018; 42: 391–4. doi:10.3969/j.issn.1671-7104.2018.06.001 http://www.ncbi.nlm.nih.gov/pubmed/30560613

²⁴ Daw W, Daw W, Kingshott RN, et al. Medical devices for measuring respiratory rate in children: a review. Journal of Advances in Biomedical Engineering and Technology 2016; 3: 21–7. doi:10.15379/2409-3394.2016.03.01.04

²⁵ Stratil A-S, Ward C, Habte T, et al. Evaluating the interrater agreement and acceptability of a new reference tool for assessing respiratory rate in children under five with cough and/or difficulty breathing. J Trop Pediatr 2021; 67: fmab046. doi:10.1093/tropej/fmab046 http://www.ncbi.nlm.nih.gov/pubmed/34124753

²⁶ Karim RM, Abdullah MS, Rahman AM, et al. Identifying role of perceived quality and satisfaction on the utilization status of the community clinic services; Bangladesh context. BMC Health Serv Res 2016; 16: 204. doi:10.1186/s12913-016-1461-z http://www.ncbi.nlm.nih.gov/pubmed/27341806

²⁷ Ahmed S, Siddique AK, Iqbal A, et al. Causes for hospitalizations at upazila health complexes in Bangladesh. J Health Popul Nutr 2010; 28: 399–404. doi:10.3329/jhpn.v28i4.6047 http://www.ncbi.nlm.nih.gov/pubmed/20824984

²⁸ Canon. Canon EOS M50, 2020. Available: https://www.canon-europe.com/cameras/eos-m50/ [Accessed 21 Jun 2021 ].

²⁹ Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1: 307–10. http://www.ncbi.nlm.nih.gov/pubmed/2868172

³⁰ Lu M-J, Zhong W-H, Liu Y-X, et al. Sample size for assessing agreement between two methods of measurement by Bland-Altman method. Int J Biostat 2016; 12: /j/ijb.2016.12.issue-2/ijb-2015-0039/ijb-2015-0039.xml. doi:10.1515/ijb-2015-0039 http://www.ncbi.nlm.nih.gov/pubmed/27838682