The Value of the First Clinical Impression as Assessed by 18 Observations in Patients Presenting to the Emergency Department

Abstract

The first clinical impression of emergency patients conveys a myriad of information that has been incompletely elucidated. In this prospective, observational study, the value of the first clinical impression, assessed by 18 observations, to predict the need for timely medical attention, the need for hospital admission, and in-hospital mortality in 1506 adult patients presenting to the triage desk of an emergency department was determined. Machine learning models were used for statistical analysis. The first clinical impression could predict the need for timely medical attention [area under the receiver operating characteristic curve (AUC ROC), 0.73; p = 0.01] and hospital admission (AUC ROC, 0.8; p = 0.004), but not in-hospital mortality (AUC ROC, 0.72; p = 0.13). The five most important features informing the prediction models were age, ability to walk, admission by emergency medical services, lying on a stretcher, breathing pattern, and bringing a suitcase. The inability to walk at triage presentation was highly predictive of both the need for timely medical attention (p < 0.001) and the need for hospital admission (p < 0.001). In conclusion, the first clinical impression of emergency patients presenting to the triage desk can predict the need for timely medical attention and hospital admission. Important components of the first clinical impression were identified.

1. Introduction

The first clinical impression of emergency patients conveys a myriad of information. Experienced clinicians recognize and correctly interpret the patient’s general appearance (e.g., skin color of the face, body position, ability to walk), subtle clinical signs (e.g., breathing pattern, the position of the patient’s hand), as well as acoustic (e.g., changes of the voice) and olfactory (e.g., urine or alcohol odor) clues within a few seconds of meeting an emergency patient. Despite its practical importance, the value of the first clinical impression in the assessment of emergency patients has not yet been scientifically elucidated. So far, only a handful of studies have shown that by using clinical intuition, emergency department staff can predict the need for hospital admission [1,2,3], presence of sepsis [4,5], disease severity, and the risk of short-term mortality [6,7,8]. However, in some of these studies, clinical intuition was also based on vital parameter readings [4], and none of these analyses established which clinical signs and indicators informed the clinicians’ intuition. Knowing what signs to look for and how to correctly interpret them would not only allow for better evaluation of emergency cases, but also for the structured education of young clinicians in good patient assessment.

In this clinical study, we determined the value of the first clinical impression, assessed by 18 observations, to predict the need for timely medical attention, the need for hospital admission, and in-hospital mortality in adult patients presenting to the triage desk of an emergency department.

2. Materials and Methods

The analysis was designed as an explorative, prospective, observational, single-center cohort study. During ten randomly chosen days from November 2019 until January 2020, it was conducted at the emergency department of the Kepler University Hospital, a tertiary university teaching hospital with 1830 beds. The emergency department serves all adult patients with emergency or acute conditions except for those following trauma and patients with obstetrical, psychiatric, or ophthalmologic emergencies. The study protocol was reviewed and approved by the ethics committee of Johannes Kepler University (1175/2019). In view of the fact that written informed consent before study enrolment would have highly likely influenced the study results by changing the patients’ behavior when entering the emergency department, no study-related interventions were made, and only anonymized data were documented, and written informed consent was waived.

2.1. Study Patients

All patients presenting to the triage area of the emergency department were eligible for study enrollment. Patients < 18 years were excluded. In addition, patients who were referred to the emergency department by emergency medical services and were in a critical or life-threatening condition were not evaluated in the triage area but were directly admitted to the resuscitation bay. Similarly, patients who were referred by emergency medical services and had a high clinical suspicion of a stroke or ST-elevation myocardial infarction were directly admitted to one of the emergency department’s acute care areas.

2.2. Data Collection

Data collection took place in the emergency department’s triage area, which is a large room with two triage desks, each with a chair in front of the desk. The triage area is separated by a wall that ensures patient confidentiality but also allows to oversee both triage desks at its end. Study-related data were only gathered from the moment the patient entered the room until medical history taking started. Triage was performed by specially trained triage nurses with the use of a computer-based Manchester Triage System application. On workdays between 7 a.m. and 7 p.m., study data were collected in real time using an electronic case report form on a hand-held device. To eliminate inter-rater variability, all first clinical impression-related study data were collected by one final-year medical student. Before data collection started, the student had worked in the emergency department as a physician assistant for 6 weeks. During this time, specific emphasis was put on training the student to recognize clinical signs and indicators as screened for in the case report form.

The first clinical impression was assessed by collecting 18 clinical observations. In view of the lack of objective criteria to describe components of the first clinical impression, these 18 observations were defined by a group of experienced physicians and nurses of the study hospital. The case report form was anonymized and included the admission number of the patient, time and mode of admission, ability to walk at triage presentation (assessed by the observation of how the patient entered the room and approached the triage desk), body posture when seated or lying on a stretcher, order of sheets on the stretcher, facial expression, skin color of the face, breathing pattern, mental impression, presence of a vomitus bag, type of clothes and shoes, hygienic state, presence of a suitcase or chaperone, and conversation details (before medical history taking started). Using the admission number on the electronic case report forms, another researcher extracted the following data from the electronic data system of the emergency department: age, sex, vital parameters measured in the triage area, management priority as defined by the Manchester Triage System, need for timely medical attention, need for hospital admission, need for intensive care unit admission, and in-hospital mortality. All data were merged into one electronic database, which was locked following the completion of patient recruitment.

2.3. Definitions

Study patients were categorized as being unable to walk when they were admitted to the emergency department triage area, either in an EMS chair or on an EMS stretcher. The need for timely medical attention was defined in accordance with the Manchester Triage System categories of red and orange. These categories are referred to as “immediate” and “very urgent” with a maximum waiting time until medical attention of 0 and 10 min, respectively.

2.4. Study Endpoints

The primary endpoint of this study was to determine the value of the first clinical impression to predict the need for timely medical attention, the need for hospital admission, and in-hospital mortality. As secondary study endpoints, we sought to identify the five most important features contributing to the prediction of each of the primary study endpoints (if significant).

2.5. Statistical Analysis

In view of the scarce published literature on this topic and the explorative character of our study, no sample size calculation could be performed. We assumed that enrolment of a convenience sample of 1500 patients would be sufficient to produce clinically meaningful answers to the study questions.

Following the locking of the database and plausibility control of all entered values, statistical analyses were performed using the R software package (R version 4.1.2; https://www.R-project.org/ (accessed on 1 October 2022). Descriptive statistical methods were applied to report demographic, clinical, and outcome data. To determine the value of the first clinical impression (including all 18 observations and the time of admission) to predict the need for timely medical attention, the need for hospital admission, and in-hospital mortality, we used these endpoints as binary variables and applied automated machine learning methods [9]. The package (AutoML 3.36.1.5; H₂O.ai (accessed on 1 October 2022), Mountain View, CA, USA) used includes the process of automating tasks in the machine learning pipeline, such as data preprocessing, hyperparameter tuning, model selection, and evaluation. H₂O AutoML trains and cross-validates the following models: three pre-specified XGBoost gradient boosting machine models, a fixed grid of generalized linear models, a default random forest, five pre-specified H₂O gradient boosting machine models, a near-default deep neural net, an extremely randomized forest, a random grid of XGBoost gradient boosting machine models, a random grid of H₂O gradient boosting machine models, a random grid of deep neural nets, a stacked ensemble of all the models trained above, as well as a “best of family” stacked ensemble that contains the best-performing model for each algorithm class. After the models had been trained, the model performance was compared using log loss and root mean squared error. The best model was then chosen for the prediction. Training of the model was conducted after a random 80%/20% (training dataset/validation dataset) split of the data on the training data set, whereas validation was performed on the validation data set. In order to determine the feature importance of each single feature, we used the Boruta package for R (version 7.0.0). For validation, cross-tabulation analyses were applied to define the sensitivity, specificity, and positive and negative predictive values of selected single features.

Data are presented as median values with interquartile ranges or absolute values with percentages. p-values < 0.05 were considered to indicate statistical significance.

3. Results

During the observation period, 1950 patients presented to the emergency department, of whom 1534 were screened for eligibility and 1506 were included in the analysis (Figure 1). The characteristics of the study population are summarized in Table 1.

The first clinical impression could predict the need for timely medical attention with an area under the receiver operating characteristic curve (AUC-ROC) of 0.73, a sensitivity of 53.1%, a specificity of 77.3%, a positive predictive value of 38.6%, a negative predictive value of 86%, and an accuracy of 72.2% (p = 0.01). Similarly, the first clinical impression could predict the need for hospital admission with an AUC ROC of 0.8, a sensitivity of 77.1%, a specificity of 70.7%, a positive predictive value of 62.8%, a negative predictive value of 82.8%, and an accuracy of 73.2% (CI95%, 67.8–78.1%) (p = 0.004). The first clinical impression could not predict in-hospital mortality (AUC ROC, 0.72; sensitivity, 0%; specificity, 100%; positive predictive value, 0%; negative predictive value, 98.7%; accuracy, 98.7%) (p = 0.13).

The five most important features informing the models to predict the need for timely medical attention and the need for hospital admission are presented in Figure 2. Following age, the ability to walk at triage presentation was the most important clinical feature in predicting whether a patient required timely medical attention or hospital admission. Frequencies of the need for timely medical attention and hospital admission depending on the ability to walk as well as the predictive value of the inability to walk at triage presentation are displayed in Figure 3.

4. Discussion

In this prospective, observational study, we found that the first clinical impression of patients presenting to the triage desk in an emergency department could reliably predict the need for timely medical attention and the need for hospital admission, but not in-hospital mortality. Important features informing the prediction models were age, ability to walk, admission by emergency medical services, lying on a stretcher, breathing pattern, and bringing a suitcase. The inability to walk at triage presentation was highly predictive of both the need for timely medical attention and the need for hospital admission.

Due to center-specific admission pathways for critically ill patients and those with time-critical conditions such as ST-elevation myocardial infarction and stroke, our population largely included non-critically ill patients. This is important to remember when interpreting the study results and comparing our results to those of other authors. We deliberately chose one final-year medical student to collect data on the first clinical impression. Thus, we avoided any inter-rater bias and made sure that the clinical experience of the researcher did not bias the recognition and interpretation of components of the first clinical impression. Another methodological aspect of this study requiring discussion is the statistical analysis. We applied machine learning methods in an attempt to simulate the human brain’s ability to simultaneously gather and, most importantly, process a multitude of non-linear information [10,11,12].

The first clinical impression could reliably predict the need for timely medical attention in this population. Our study is the first to highlight this specific value of the first clinical impression of emergency patients. We used the two highest management priorities of the Manchester Triage System to define the need for timely medical attention. Although validation testing of the Manchester Triage System in a general emergency department has shown good results [13], it has also been criticized for potential under-triage at higher urgency levels [14]. The finding that the first clinical impression predicts the need for hospital admission with high accuracy confirms the results of previous research. Two prospective, observational studies from Switzerland using a similar set-up found that clinical intuition ratings of emergency physicians achieved comparable values to predict the need for hospital admission as observed in our analysis [1,2]. Similarly, studies conducted in emergency departments in the Netherlands and Canada reported that the intuition of clinicians, informed by the first clinical impression before triage, could predict the need for hospitalization [3,8]. The result that the first clinical impression was not predictive of in-hospital mortality in our cohort was surprising but is likely to be due to the fact that the number of patients dying during hospital admission was very low (n = 10, 0.7%), rendering the analysis underpowered to detect a significant association using machine learning methods. This is in contrast to other studies, which reported that the first clinical impression could also predict the risk of death [3,8].

An important feature of our study is that we did not use intuition to interpret the first clinical impression but evaluated 18 specific components of the first clinical impression. This allowed us to determine which observations of the first clinical impression might be especially important to screen for when evaluating an emergency patient at triage. Our results suggested six components of particular relevance. Apart from age, the ability to walk at triage presentation was the most important clinical feature informing both prediction models. Being unable to walk at triage presentation was highly predictive of the need for timely medical attention and hospital admission. This appears physiologically sound and might not only reflect the degree of frailty [15] but also the extent to which the patient’s physiologic reserve has been compromised by the acute condition. Similarly, the inability to walk at hospital admission proved to be an accurate predictor of the risk of in-hospital death in three African studies [16,17,18].

Certain limitations need to be taken into account when interpreting the results of our study. First, 18 components of the first clinical impression were selected based on clinical experience. Nonetheless, it is highly likely that the first clinical impression contains many more signs and indicators. Therefore, the true value of the clinical impression has likely been underestimated by our analysis. Second, as this was a single-center study, its results may not be extrapolated to other centers, particularly not to other cultural settings [19]. Third, given that all 18 observations of the first clinical impression were collected by one researcher, we cannot exclude that recognition and interpretation of the first clinical impression are subject to clinically relevant inter-rater variability [20].

5. Conclusions

The first clinical impression of emergency patients presenting to the triage desk can predict the need for timely medical attention and hospital admission. Important features informing the prediction models were age, ability to walk, admission by emergency medical services, lying on a stretcher, breathing pattern, and bringing a suitcase. The inability to walk at triage presentation was highly predictive of both the need for timely medical attention and the need for hospital admission.

Author Contributions

T.T. designed the study protocol, collected data, interpreted the study results and revised the manuscript for important intellectual content; S.K. collected data, interpreted the study results and revised the manuscript for important intellectual content; M.W.D. designed the study protocol, interpreted the study results and drafted the first version of the manuscript; R.S. designed the study protocol, interpreted the study results and revised the manuscript for important intellectual content; J.M. designed the study protocol, analyzed the data, interpreted the study results and revised the manuscript for important intellectual content. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the ethics committee of the Johannes Kepler University (1175/2019).

Informed Consent Statement

In view of the fact that written informed consent before study enrolment would have highly likely influenced the study results by changing the patients’ behavior when entering the emergency department, no study-related interventions were made, and only anonymized data were documented; written informed consent was waived.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to GDPR and local regulatory/data privacy provisions.

Conflicts of Interest

The authors declare no conflict of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Table

Figure 1. CONSORT flow diagram. ED, emergency department.

Figure 2. Feature importance of automated machine learning models to predict the need for timely medical attention and the need for hospital admission. EMS, emergency medical services.

Figure 3. Frequencies of the need for timely medical attention and hospital admission depending on the ability to walk as well as the predictive value of the inability to walk at triage presentation. NPV, negative predictive value; PPV, positive predictive value; Sens., sensitivity; Spec., specificity.

Table 1

Characteristics of study patients.

N		1506
Age	years	60 (39–77)
Male sex	n (%)	731 (48.5)
Mode of admission
self-admission	n (%)	820 (54.4)
admission through EMS	n (%)	668 (44.4)
intra-hospital transfer	n (%)	18 (1.2)
Ability to walk at triage presentation
able to walk unassisted	n (%)	639 (42.4)
walks with a limb	n (%)	142 (9.4)
can only walk with assistance (e.g., by another person or a walking aid)	n (%)	26 (1.7)
admitted in an EMS seat	n (%)	529 (35.1)
admitted on an EMS stretcher	n (%)	170 (11.3)
Body posture when seated for triage evaluation	n (%)	1336 (88.7)
no apparent abnormality	n (%)	899 (59.7)
stiff	n (%)	101 (6.7)
bent over	n (%)	94 (6.2)
hand on specific body part	n (%)	73 (4.8)
reduced muscle tone (in patients unable to walk)	n (%)	55 (3.7)
restless	n (%)	16 (1.1)
Body posture when lying on EMS stretcher	n (%)	170 (11.3)
supine	n (%)	74 (4.9)
semi-recumbent	n (%)	68 (4.5)
recovery position	n (%)	14 (0.9)
head of stretcher elevated > 60°	n (%)	7 (0.5)
supine with knee roll	n (%)	7 (0.5)
Sheets on stretcher unordered	n (%)	21 (1.4)
Facial expression
no apparent abnormality	n (%)	1061 (70.5)
frowning	n (%)	314 (20.8)
anxious	n (%)	64 (4.2)
distorted in pain	n (%)	60 (4)
Skin color of the face
normal	n (%)	1153 (76.6)
pale	n (%)	223 (14.8)
plethoric	n (%)	90 (6)
icteric	n (%)	17 (1.1)
cyanotic	n (%)	13 (0.9)
greyish	n (%)	7 (0.5)
Breathing pattern
no apparent abnormality	n (%)	1308 (86.9)
increased work of breathing	n (%)	107 (7.1)
noisy, grunting breathing	n (%)	57 (3.8)
rapid breathing	n (%)	29 (1.9)
Mental impression
normal	n (%)	1137 (75.5)
anxious	n (%)	154 (10.2)
disoriented	n (%)	86 (5.7)
fatigued	n (%)	77 (5.1)
depressed	n (%)	31 (2.1)
aggressive	n (%)	10 (0.7)
psychotic	n (%)	4 (0.3)
Vomitus bag present	n (%)	31 (2.1)
Clothes
outdoor clothes	n (%)	955 (63.4)
indoor clothes	n (%)	486 (32.3)
night gown	n (%)	52 (3.4)
work clothes	n (%)	14 (0.9)
Hygienic state
neat and groomed	n (%)	1378 (91.5)
unkempt	n (%)	109 (7.2)
urine odor	n (%)	7 (0.5)
alcohol odor	n (%)	1 (0.1)
Shoes
street shoes	n (%)	1238 (82.2)
slippers	n (%)	183 (12.2)
no shoes	n (%)	85 (5.6)
Patient brought suitcase with him/her	n (%)	418 (27.8)
Chaperone present	n (%)	373 (24.8)
Conversation
patient speaks with ED staff	n (%)	852 (56.6)
EMS staff speaks with ED staff	n (%)	508 (33.7)
chaperone speaks with ED staff	n (%)	144 (9.6)
Vital Parameters
heart rate	bpm	83 (73–98)
systolic arterial blood pressure	mmHg	145 (130–165)
temperature	°C	36.5 (36.3–36.8)
plethysmographic oxygen saturation	%	96 (95–98)
Need for timely medical attention	n (%)	275 (18.3)
Need for hospital admission	n (%)	571 (37.9)
Need for intensive care unit admission	n (%)	13 (0.9)
In-hospital mortality	n (%)	10 (0.7)

ED, emergency department; EMS, emergency medical services. Data are given as median values with interquartile ranges, if not otherwise indicated.

References

1. Bingisser, R.; Baerlocher, S.M.; Kuster, T.; Ortega, R.N.; Nickel, C.H. Physicians’ Disease Severity Ratings are Non-Inferior to the Emergency Severity Index. J. Clin. Med.; 2020; 9, 762. [DOI: https://dx.doi.org/10.3390/jcm9030762] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32168931]

2. Rohacek, M.; Nickel, C.H.; Dietrich, M.; Bingisser, R. Clinical intuition ratings are associated with morbidity and hospitalisation. Int. J. Clin. Pract.; 2015; 69, pp. 710-717. [DOI: https://dx.doi.org/10.1111/ijcp.12606] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/25689155]

3. Lee, E.E.M.; Kwok, E.S.H.; Vaillancourt, C. Using emergency physicians’ abilities to predict patient admission to decrease admission delay time. Emerg. Med. J.; 2020; 37, pp. 417-422. [DOI: https://dx.doi.org/10.1136/emermed-2019-208859] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32139515]

4. Quinten, V.M.; Van Meurs, M.; Wolffensperger, A.E.; Ter Maaten, J.C.; Ligtenberg, J.J. Sepsis patients in the emergency department: Stratification using the Clinical Impression Score, Predisposition, Infection, Response and Organ dysfunction score or quick Sequential Organ Failure Assessment score?. Eur. J. Emerg. Med.; 2018; 25, pp. 328-334. [DOI: https://dx.doi.org/10.1097/MEJ.0000000000000460] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28338533]

5. Van der Vegt, A.E.; Holman, M.; Maaten, J.C.T. The value of the clinical impression in recognizing and treating sepsis patients in the emergency department. Eur. J. Emerg. Med.; 2012; 19, pp. 373-378. [DOI: https://dx.doi.org/10.1097/MEJ.0b013e32834def59] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23108381]

6. Ouchi, K.; Strout, T.; Haydar, S.; Baker, O.; Wang, W.; Bernacki, R.; Sudore, R.; Schuur, J.D.; Schonberg, M.A.; Block, S.D. et al. Association of Emergency Clinicians’ Assessment of Mortality Risk With Actual 1-Month Mortality Among Older Adults Admitted to the Hospital. JAMA Netw. Open; 2019; 2, e1911139. [DOI: https://dx.doi.org/10.1001/jamanetworkopen.2019.11139] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31517962]

7. Beglinger, B.; Rohacek, M.; Ackermann, S.; Hertwig, R.; Karakoumis-Ilsemann, J.; Boutellier, S.; Geigy, N.; Nickel, C.; Bingisser, R. Physician’s First Clinical Impression of Emergency Department Patients With Nonspecific Complaints Is Associated With Morbidity and Mortality. Medicine; 2015; 94, e374. [DOI: https://dx.doi.org/10.1097/MD.0000000000000374] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/25700307]

8. Zelis, N.; Mauritz, A.N.; Kuijpers, L.I.J.; Buijs, J.; De Leeuw, P.W.; Stassen, P.M. Short-term mortality in older medical emergency patients can be predicted using clinical intuition: A prospective study. PLoS ONE; 2019; 14, e0208741. [DOI: https://dx.doi.org/10.1371/journal.pone.0208741] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30601815]

9. Deng, C.; Ji, X.; Rainey, C.; Zhang, J.; Lu, W. Integrating Machine Learning with Human Knowledge. Iscience; 2020; 23, 101656. [DOI: https://dx.doi.org/10.1016/j.isci.2020.101656]

10. Shapshak, P. Artificial Intelligence and brain. Bioinformation; 2018; 14, pp. 38-41. [DOI: https://dx.doi.org/10.6026/97320630014038] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29497259]

11. Zou, J.; Han, Y.; So, S.-S. Overview of Artificial Neural Networks. Artif. Neural Netw.; 2008; 458, pp. 14-22. [DOI: https://dx.doi.org/10.1007/978-1-60327-101-1_2]

12. Luo, G.; Stone, B.L.; Johnson, M.D.; Tarczy-Hornoch, P.; Wilcox, A.B.; Mooney, S.D.; Sheng, X.; Haug, P.J.; Nkoy, F.L. Automating Construction of Machine Learning Models With Clinical Big Data: Proposal Rationale and Methods. JMIR Res. Protoc.; 2017; 6, e175. [DOI: https://dx.doi.org/10.2196/resprot.7757]

13. Zachariasse, J.M.; Seiger, N.; Rood, P.P.M.; Alves, C.F.; Freitas, P.; Smit, F.J.; Roukema, G.R.; Moll, H.A. Validity of the Manchester Triage System in emergency care: A prospective observational study. PLoS ONE; 2017; 12, e0170811. [DOI: https://dx.doi.org/10.1371/journal.pone.0170811] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28151987]

14. Parenti, N.; Reggiani, M.L.B.; Iannone, P.; Percudani, D.; Dowding, D. A systematic review on the validity and reliability of an emergency department triage scale, the Manchester Triage System. Int. J. Nurs. Stud.; 2014; 51, pp. 1062-1069. [DOI: https://dx.doi.org/10.1016/j.ijnurstu.2014.01.013] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/24613653]

15. Church, S.; Rogers, E.; Rockwood, K.; Theou, O. A scoping review of the Clinical Frailty Scale. BMC Geriatr.; 2020; 20, 393. [DOI: https://dx.doi.org/10.1186/s12877-020-01801-7] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33028215]

16. Rylance, J.; Baker, T.; Mushi, E.; Mashaga, D. Use of an early warning score and ability to walk predicts mortality in medical patients admitted to hospitals in Tanzania. Trans. R. Soc. Trop. Med. Hyg.; 2009; 103, pp. 790-794. [DOI: https://dx.doi.org/10.1016/j.trstmh.2009.05.004] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/19540542]

17. Kayambankadzanja, R.K.; Schell, C.O.; Nsanjama, G.; Mbingwani, I.; Mndolo, S.K.; Rylance, J.; Baker, T. Inability to Walk Predicts Death among Adult Patients in Hospitals in Malawi. Emerg. Med. Int.; 2019; 2019, 6586891. [DOI: https://dx.doi.org/10.1155/2019/6586891] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31360550]

18. Kwizera, A.; Urayeneza, O.; Mujyarugamba, P.; Meier, J.; Patterson, A.J.; Harmon, L.; Farmer, J.C.; Dünser, M.W. Sepsis in Resource-Limited Nations. The inability to walk unassisted at hospital admission as a valuable triage tool to predict hospital mortality in Rwandese patients with suspected infection. PLoS ONE; 2020; 15, e0228966. [DOI: https://dx.doi.org/10.1371/journal.pone.0228966] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32084167]

19. Bellomo, R.; Warrillow, S.; Reade, M. Why we should be wary of single-center trials. Crit. Care Med.; 2009; 37, pp. 3114-3119. [DOI: https://dx.doi.org/10.1097/CCM.0b013e3181bc7bd5] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/19789447]

20. Heal, C.; Cotterill, S.; Rowland, A.G.; Garratt, N.; Long, T.; Brown, S.; O’Connor, G.; Rishton, C.; Woby, S.; Roland, D. Inter-rater reliability of paediatric emergency assessment: Physiological and clinical features. Arch. Dis. Child.; 2021; 106, pp. 149-153. [DOI: https://dx.doi.org/10.1136/archdischild-2019-318664] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32988814]