Clinical criteria to exclude acute vascular pathology on CT angiogram in patients with dizziness

Abstract

Background

Patients presenting to the emergency department (ED) with dizziness may be imaged via CTA head and neck to detect acute vascular pathology including large vessel occlusion. We identify commonly documented clinical variables which could delineate dizzy patients with near zero risk of acute vascular abnormality on CTA.

Methods

We performed a cross-sectional analysis of adult ED encounters with chief complaint of dizziness and CTA head and neck imaging at three EDs between 1/1/2014-12/31/2017. A decision rule was derived to exclude acute vascular pathology tested on a separate validation cohort; sensitivity analysis was performed using dizzy “stroke code” presentations.

Results

Testing, validation, and sensitivity analysis cohorts were composed of 1072, 357, and 81 cases with 41, 6, and 12 instances of acute vascular pathology respectively. The decision rule had the following features: no past medical history of stroke, arterial dissection, or transient ischemic attack (including unexplained aphasia, incoordination, or ataxia); no history of coronary artery disease, diabetes, migraines, current/long-term smoker, and current/long-term anti-coagulation or anti-platelet medication use. In the derivation phase, the rule had a sensitivity of 100% (95% CI: 0.91–1.00), specificity of 59% (95% CI: 0.56–0.62), and negative predictive value of 100% (95% CI: 0.99–1.00). In the validation phase, the rule had a sensitivity of 100% (95% CI: 0.61–1.00), specificity of 53% (95% CI: 0.48–0.58), and negative predictive value of 100% (95% CI: 0.98–1.00). The rule performed similarly on dizzy stroke codes and was more sensitive/predictive than all NIHSS cut-offs. CTAs for dizziness might be avoidable in 52% (95% CI: 0.47–0.57) of cases.

Conclusions

A collection of clinical factors may be able to “exclude” acute vascular pathology in up to half of patients imaged by CTA for dizziness. These findings require further development and prospective validation, though could improve the evaluation of dizzy patients in the ED.

Introduction

Posterior circulation stroke in patients presenting with dizziness is difficult to diagnose–delayed or missed diagnosis may occur in 37% of such cases on first medical contact [1]. An underlying ischemic stroke is present in 3–5% of patients presenting dizziness to the emergency department (ED) [2]. Large vessel occlusion (LVO) is the underlying etiology in approximately a third of posterior circulation ischemic strokes [3, 4]. LVO is therefore expected in ~1% of dizzy patients presenting to the ED. The National Institutes of Health Stroke Scale (NIHSS) is the most commonly used predictor in pre-hospital/pre-CTA risk assessment for LVO, however even a NIHSS of zero does not guarantee absence of LVO [5, 6].

Stroke or vessel occlusion with low NIHSS is more common in the posterior circulation and with non-focal presentations such as dizziness [5, 7]. Other acute vascular pathologies detectable on CTA include dissection and medium/small vessel occlusion; rarely, even ruptured aneurysm may initially present as dizziness [2, 8–10]. The early detection of these abnormalities may also impact patient management [11]. Therefore, predictive tools used to reassure against the need for emergent CTA or pre-hospital diversion to centers with endovascular capability would ideally consider these entities as well. In the absence of vessel abnormalities on CTA, underlying stroke is better detected by MRI or specialized bedside maneuvers [2, 12, 13]. Excluding the need for CTA may improve diagnosis by facilitating triage to these more sensitive modalities.

We hypothesize that there is a collection of clinical variables that can delineate a subpopulation of dizzy patients in whom there is a near zero probability of acute vascular pathology detectable on CTA. Such a collection of variables could form the basis of a decision rule guiding the selection of patients for CTA versus alternative testing. In this study, we perform a retrospective analysis to derive a decision rule excluding vascular pathology in dizzy ED patients. We then validate the rule on a temporally separate validation cohort and assess applicability to a similar group of “stroke code” presentations with dizziness. We compare performance to the NIHSS and estimate the proportion of CTA exams which are potentially avoidable.

Methods

Setting and design

We performed a cross-sectional analysis based on adult patient visits to one of three EDs in a healthcare system between 1/1/2014-12/31/2017. The first ED is a comprehensive stroke center, the second is a primary stroke center, the last is a smaller community ED. Adult (age ≥18 years) patient encounters with a chief complaint of dizziness who received CTA head and neck imaging were included for the analysis. Adult “stroke code” encounters during the same time period, with NIHSS ≤ 7 and a positive review of systems for dizziness were also obtained for a sensitivity analysis in patients presenting within the treatment window with high suspicion of stroke. An NIHSS cut off 7 was chosen based on prior literature showing that patients with NIHSS > 7 have greater risk of LVO, independent of presenting with dizziness [6, 14]. Additional details of patient eligibility for CTA imaging are provided in Item 1 in S1 File.

The study was approved by the Yale University IRB with consent waived. The decision rule was developed with reference to the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) guidelines [15].

Data collection

Established vascular risk factors for stroke/LVO were identified based on a review of major recent literature and used to inform the first phase of feature selection [16–20] [Item 2 in S1 File]. Clinical data expected to be available at the time of an imaging order were extracted from the electronic medical record database (EPIC)–including but not limited to recognized risk factors. Demographic information, past medical history (PMH), review of systems (ROS), and physical exam (PE) findings were obtained. Past medical history data also captured categories of medication use, smoking, and other substance use. Physical exam findings included systems-based exam findings as well as NIHSS and Glasgow Coma Scale data. All associated CTA head and neck exam reports were obtained.

Case categorization

CTA head and neck exam reports were categorized based on the presence of acute vascular abnormality. Acute vascular pathology was defined as large vessel occlusion, smaller arterial or venous occlusion, non-occlusive dissection, and aneurysm or other vascular lesion with hemorrhage. Operational definition of LVO is provided in Item 3 in S1 File. Various etiologies were included to account for the possibility that some patients with dizziness who do not have LVO may have other acute pathology whose acute management would be altered if detected on CTA. A decision rule excluding LVO but missing other entities requiring acute intervention would be less clinically useful. Age indeterminate though potentially acute findings were categorized as “positive” cases; this categorization was performed to maximize sensitivity even at the cost of potentially reduced specificity.

Feature coding and missing data

One-hot encoding was used to covert PMH data into categorical variables. Instead of grouping past medical history elements into categories (e.g., history of cancer or history of diabetes), specific diagnosis codes were included, to assess differential risk that may arise from different severity or manifestations of medical conditions. ROS and PE findings were represented by two binary variables, one representing occurrence of a “pertinent positive” (e.g., positive review of systems for headache) the other when occurring as a “pertinent negative” (e.g., negative review of systems for nausea). Absent or missing documentation was therefore represented as a null value for both pertinent positive and pertinent negative variables. Numerical values (e.g., age, pain rating) were coded as continuous variables. Additional details of feature encoding are given in Item 4 in S1 File.

Feature selection

The study population was split into training and validation cohorts consisting of the first 75% and last 25% of included patient encounters, respectively (corresponding to encounters before and after 14:27 on 5/12/2017). Features that were rarely documented in the medical record and therefore impractical for use in a decision rule were excluded. We initially assessed the 200 most documented PMH features, 100 most commonly ROS features, all numerical vital signs, and the 100 most commonly PE features; these were drawn from an initial pool of 1500, 186, and 266 of PMH, ROS, and PE features and covered 74.5%, 96.9%, and 97.5% of instances of documentation in these categories respectively. Features were then filtered to remove those with low relationship to the target variable based on Chi-Squared test, leaving the 100 highest ranked features.

Decision rule derivation via sequential covering

A decision tree was generated of sufficient depth to categorize all cases using the CART (Classification and Regression Trees) algorithm. Features were also ranked on random forest (Gini) importance to assess ability to separate positive and negative cases not just in the single tree, but across an ensemble of trees [21]; we used 100 trees, each with access to all top 100 Chi-Squared ranked features and allowed sufficient depth to categorize all cases. Features were then extracted to a decision rule in two phases. First, established vascular risk factors with high Gini importance (rank ≤ 5) were added to the list. Second, other (less-recognized) potential predictors were identified, based on highest Gini importance and location on the decision path predicting the largest leaf node without acute findings. Continuous variables were required to have a monotonic relationship with the target variable for inclusion, to account for the potential bias of Gini criteria toward high cardinality features [22]. Each time a feature was added to the list in either phase, the decision tree was re-created using cases not excluded by the evolving decision rule. Gini importance and decision tree structure were therefore used to obtain a list of features accounting for all positive instances of the target variable (i.e., via sequential covering [23]). Lastly, features on the list describing differing manifestations of a single condition were grouped into clinically coherent categories to arrive at the final decision rule. See Item 5 in S1 File for a schematic of this process.

Decision rule validation and sensitivity analysis

The decision rule was applied to the temporally separated testing cohort, with performance compared to the NIHSS. A sensitivity analysis for hyperacute presentations was also performed, by applying the rule to “stroke code” presentations with dizziness (within the review of systems) and a NIHSS ≤ 7.

Performance measures and statistical analysis

Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated, with 95% confidence intervals (CI) using the Wilson score method [24]. The proportion of patients predicted to have a negative exam (potentially avoidable CTAs) was also reported with 95% CI using Wilson score [24].

Software tools

Data management, coding, and analysis were done with Python (Version 3.7; Python Software Foundation; Delaware, US) in the PyCharm Integrated Development Environment (Version 2020.2.2; JetBrains; Prague, Czech Republic). Statistical and data analytic packages included pandas, sklearn, dtreeviz, and tableone [25]. Manual review and categorization of CTA findings was performed in Microsoft Excel (Version 2203; Microsoft; Redmond, Washington, US).

Results

Case characteristics (training and testing cohort)

During the study period, 15,483 adult patients presented to the ED with a chief complaint of dizziness. Of these, 1,429 (9.2%), received a CTA head and neck exam, and were included for analysis. Review of CTA head and neck results revealed 47 (3.3%) cases of acute vascular pathology (n = 31, 2.2% for LVO only) [Fig 1]. This set of cases was used to produce the training and validation cohorts. Demographic and clinical characteristics of patients with and without acute vascular pathology are provided in Table 1. Details of the positive cases are given in Item 6 in S1 File.

[Figure omitted. See PDF.]

[Figure omitted. See PDF.]

The 75/25 temporal split of the study population into training and validation cohorts generated with data sets with 1072 and 357 cases, respectively. There were 41 cases of acute vascular pathology in the training set and 6 in the validation set (28 and 3 cases of LVO respectively) [Fig 1].

Decision rule list

Generation of a decision rule from the training cohort resulted in a collection of 23 features delineating a subset of encounters with zero cases of acute pathology. Summarizing these into clinical categories, we arrive at a decision rule predicting absence of acute vascular pathology in patients without a history of any of the following: stroke/transient ischemic attack (TIA), unexplained speech difficulty/ataxia or visual disturbances (i.e., possible prior stroke/TIA), dissection, coronary artery disease, diabetes, migraines, active/long-term smoking, and current long-term anti-platelet or anti-coagulant use [Table 2].

[Figure omitted. See PDF.]

Decision rule performance

In the derivation phase, the rule applied to 603 of 1072 (56%) cases, excluding all 41 cases of pathology (28 cases of LVO). In the derivation set, the rule had a sensitivity of 100% (95% CI: 0.91–1.00), specificity of 59% (95% CI: 0.56–0.62), positive predictive value of 9% (95% CI: 0.07–0.12), and negative predictive value of 100% (95% CI: 0.99–1.00). A two-by-two table of patients excluded (vs. not excluded) by the decision rule and those with (vs. without) acute vascular pathology in the derivation phase is given in Item 8 in S1 File.

Application of the decision rule to the validation cohort predicted absence of acute vascular pathology in 185 of 357 (52%) cases, excluding all 6 cases of pathology (3 cases of LVO). This corresponds to a sensitivity of 100.0% (95% CI: 0.61–1.00), specificity of 53% (95% CI: 0.48–0.58), positive predictive value of 4% (95% CI: 0.02–0.07), and negative predictive value of 100% (95% CI: 0.98–1.00). The proportion of patients with near-zero risk and potentially avoidable CTAs was 52% (CI: 0.47–0.57).

The separate cohort used for sensitivity analysis (“stroke codes,” dizziness, NIHSS ≤ 7) consisted of 81 cases with 12 acute vascular findings (10 LVO). Application of the decision rule predicted absence of acute pathology in 35 patients (43%), excluding all cases of acute pathology. Sensitivity was 1.00 (95% CI: 0.76–1.00), specificity 0.51 (95% CI: 0.39–0.62), positive predictive value 26% (95% CI: 0.16–0.40), and negative predictive value was 100% (95% CI: 0.90–1.00). The proportion of dizzy “stroke code” presents with near-zero risk and potentially avoidable CTAs was 43% (95% CI: 0.33–0.54).

Performance of the decision rule in the validation phase compared to NIHSS cut-offs is given in Table 3. Results of the derivation phase and sensitivity analysis are provided separately [Item 9 in S1 File]. Details of NIHSS documentation and performance are given in the Item 10 in S1 File.

[Figure omitted. See PDF.]

Discussion

In this study, we used retrospective data and a sequential covering approach to derive a decision rule excluding large vessel occlusion and other vascular pathology in patients presenting with dizziness. The resultant rule consists of well-recognized vascular risk factors and as well as other features identified based on random forest importance. The decision rule predicts absence of acute vascular pathology in patients who do not have any of the following past medical history features:

Stroke/transient ischemic attack (TIA), unexplained speech difficulty/ataxia or visual disturbances (i.e., possible prior stroke/TIA), dissection, coronary artery disease, diabetes, migraines, active/long-term smoking, and current long-term anti-platelet or anti-coagulant use. See Fig 2 for a visual representation of the rule.

[Figure omitted. See PDF.]

Clinical criteria are grouped into closely related categories. Further development and validation are needed prior to application in clinical contexts.

In the testing cohort, the decision rule excluded acute vascular pathology with 100% sensitivity and 53% specificity, The rule was more sensitive than conventionally used NIHSS thresholds (NIHSS ≤ 7), as well as NIHSS = 0. Similar performance of the rule, 100% sensitivity and 51% specificity, was seen when applied to “stroke code” presentations with dizziness and with relatively lower risk of anterior circulation LVO (NIHSS ≤ 7). High sensitivity suggests utility as a “rule out” tool; non-applicability of the rule does necessarily indicate need for CTA or presence of underlying pathology. The decision rule applied to 52% of cases in the validation cohort and 43% of cases in the sensitivity analysis, suggesting that nearly half of CTA head and neck examinations might be avoidable, if such a tool were sufficiently reassuring to defer neurovascular imaging.

Dizziness is a non-specific symptom that accompanies a wide range of neurologic, cardiovascular, psychiatric, and other disease processes [2]. In this study, we focused on those patients in whom dizziness is the presenting complaint, and in whom there may be no alternative presentations warranting neurovascular evaluation. For example, patients who are dizzy in the setting of trauma or thunderclap headache would receive neurovascular evaluation for other reasons.

Our decision rule suggests that documented (otherwise unexplained) speech, coordination, and vision abnormalities be considered potential prior stroke or TIA for the purposes of risk stratification. This would be consistent with prior research showing that as many as 90% of posterior circulation TIAs may be misdiagnosed on first medical contact [26]. Among established vascular risk factors, smoking status and diabetes were included in the decision rule; hypertension and hyperlipidemia were conspicuously absent. It is conceivable that these or other risk factors for stroke would provide predictive value if the rule were updated or validated with additional data sets. Medical history elements seemed to be much more strongly predictive than review of system or physical exam findings, which may reflect the pre-selection of ED visits with dizziness as a primary concern. A notable difference between the risk factors highlighted by our study and existing predictive tools is the inclusion of migraine history as a predictive factor. Migraine with aura is a well-described, though perhaps underappreciated, risk factor for both dissection and stroke [27–29]. Our study suggests potential value within prediction tools alongside more established vascular disease processes.

Prior research suggests that vascular imaging to evaluate for large vessel occlusion may be cost-effective in acute minor stroke patients even when the risk of LVO is low (0.2%) [30]. Predictive tools for vascular pathology in stroke need to have near-perfect sensitivity to appropriately defer neurovascular evaluation. The collection of risk factors highlighted by our study suggests that subpopulations of patients with dizziness may exist with near-zero risk for pathology detectable on CTA. None of the patients excluded by the decision rule (in the training, validation, or sensitivity analysis cohorts) had an acute abnormality on CTA. Future research could assess the potential role of such a decision rule in conjunction with existing bedside exams and risk stratification tools.

Strokes in the posterior circulation are less often caused by LVO than by cardiac embolism or lacunar mechanism [3]. The decision rule developed in this work may assist in distinguishing underlying etiologies by excluding patients with stenotic atherosclerosis/vasculopathy who subsequently develop vessel occlusion. CTA is less sensitive for subtle ischemic events that arise from distal embolic and other etiologies [2]. Patients with minimal risk for LVO, but still suspected to have stroke might could be prioritized for more sensitive MRI, HINTS (head impulse, nystagmus, and test of skew), or subspecialist consultation. Vascular imaging could be deferred in this cohort and performed only if alternative indications arise.

Limitations

Our study has some limitations. The first of these is the retrospective nature of data collection. Documentation in ED medical records is known to be inconsistent [31], which limits incorporation of predictive factors not appearing in the medical chart. Decision rule performance would also be impacted by medical history elements which may be present but not documented. Prospective and external validation would help address these limitations. Our data precede the COVID-19 pandemic; therefore, any potential thromboembolic risk from COVID-19 infection or related to rare vaccination side effects [32] are not captured. This possibility could be incorporated in future studies. Considering age-indeterminate findings on CTA to be potentially acute allowed us to err on the side of maximizing sensitivity, though could have reduced specificity for detection of acute abnormality. The consideration of a large and varied set of features allowed us to identify risk factors which may have been underappreciated, though could also reduce generalizability. The derivation of predictive tools for dizzy patients is complicated by the low prevalence of acute abnormality on imaging and therefore fewer positive events. We expect the need to refine the decision rule and estimates of performance with subsequent work using additional datasets. Most of the study limitations would have reduced the ability to identify patients at low risk; yet, we still found that half of patients may not have required CTA.

Conclusion

Our study suggests that a collection of clinical variables may delineate a sizeable subpopulation of dizzy patients receiving CTA in whom there is a near zero risk of acute vascular pathology including LVO. A decision rule composed of these predominantly well-recognized risk factors could apply to as many as half of patients currently evaluated by head and neck CTA. Further development and validation of this predictive tool would help guide efficient diagnosis and management of dizzy patients in the emergency department.

Supporting information

S1 File.

https://doi.org/10.1371/journal.pone.0280752.s001

(DOCX)

Citation: Tu LH, Malhotra A, Venkatesh AK, Taylor RA, Sheth KN, Yaesoubi R, et al. (2023) Clinical criteria to exclude acute vascular pathology on CT angiogram in patients with dizziness. PLoS ONE 18(3): e0280752. https://doi.org/10.1371/journal.pone.0280752

About the Authors:

Long H. Tu

Roles: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Validation, Visualization, Writing – original draft, Writing – review & editing

E-mail: [email protected]

Affiliation: Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, CT, United States of America

ORICD: https://orcid.org/0000-0002-2419-5581

Ajay Malhotra

Roles: Conceptualization, Investigation, Project administration, Supervision, Writing – review & editing

Affiliation: Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, CT, United States of America

Arjun K. Venkatesh

Roles: Conceptualization, Formal analysis, Investigation, Methodology, Supervision, Writing – review & editing

Affiliation: Department of Emergency Medicine, Yale School of Medicine, New Haven, CT, United States of America

ORICD: https://orcid.org/0000-0002-8248-0567

Richard A. Taylor

Roles: Conceptualization, Data curation, Investigation, Methodology, Resources, Supervision, Validation, Writing – review & editing

Affiliation: Department of Emergency Medicine, Yale School of Medicine, New Haven, CT, United States of America

ORICD: https://orcid.org/0000-0002-9082-6644

Kevin N. Sheth

Roles: Conceptualization, Supervision, Writing – review & editing

Affiliation: Department of Neurology, Yale School of Medicine, New Haven, CT, United States of America

Reza Yaesoubi

Roles: Conceptualization, Formal analysis, Software, Supervision, Validation, Visualization, Writing – review & editing

Affiliation: Health Policy and Management, Yale School of Public Health, New Haven, CT, United States of America

ORICD: https://orcid.org/0000-0002-9276-5750

Howard P. Forman

Roles: Conceptualization, Project administration, Supervision, Writing – review & editing

Affiliation: Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, CT, United States of America

Soundari Sureshanand

Roles: Data curation, Methodology, Resources, Writing – review & editing

Affiliation: Yale Center for Clinical Investigation, Yale School of Medicine, New Haven, CT, United States of America

Dhasakumar Navaratnam

Roles: Conceptualization, Supervision, Validation, Writing – review & editing

Affiliation: Department of Neurology, Yale School of Medicine, New Haven, CT, United States of America

References

1. Arch A.E., et al., Missed Ischemic Stroke Diagnosis in the Emergency Department by Emergency Medicine and Neurology Services. Stroke, 2016. 47(3): p. 668–73. pmid:26846858

2. Saber Tehrani A.S., et al., Diagnosing Stroke in Acute Dizziness and Vertigo: Pitfalls and Pearls. Stroke, 2018. 49(3): p. 788–795. pmid:29459396

3. Caplan L.R., et al., New England Medical Center Posterior Circulation registry. Ann Neurol, 2004. 56(3): p. 389–98. pmid:15349866

4. Rennert R.C., et al., Epidemiology, Natural History, and Clinical Presentation of Large Vessel Ischemic Stroke. Neurosurgery, 2019. 85(suppl_1): p. S4–S8. pmid:31197329

5. Heldner M.R., et al., National Institutes of Health Stroke Scale Score and Vessel Occlusion in 2152 Patients With Acute Ischemic Stroke. Stroke, 2013. 44(4): p. 1153–+. pmid:23471266

6. Smith , Accuracy of Prediction Instruments for Diagnosing Large Vessel Occlusion in Individuals With Suspected Stroke: A Systematic Review for the 2018 Guidelines for the Early Management of Patients With Acute Ischemic Stroke (vol 49, pg e111, 2018). Stroke, 2018. 49(3): p. E139–E139.

7. Martin-Schild S., et al., Zero on the NIHSS does not equal the absence of stroke. Ann Emerg Med, 2011. 57(1): p. 42–5. pmid:20828876

8. Kikkawa Y., et al., Ruptured intrameatal anterior inferior cerebellar artery aneurysm associated with obstructed internal auditory artery in a patient with sudden hearing loss and vertigo. Acta Neurochir (Wien), 2019. 161(2): p. 403–406. pmid:30560376

9. Ogasawara Y., Ito K., and Ohkuma H., Atypical Presentation of Aneurysmal Subarachnoid Hemorrhage: Incidence and Clinical Importance. J Stroke Cerebrovasc Dis, 2016. 25(5): p. 1208–1214. pmid:26935119

10. Sparaco M., Ciolli L., and Zini A., Posterior circulation ischaemic stroke—a review part I: anatomy, aetiology and clinical presentations. Neurological Sciences, 2019. 40(10): p. 1995–2006. pmid:31222544

11. Saver J.L., et al., Thrombectomy for Distal, Medium Vessel Occlusions: A Consensus Statement on Present Knowledge and Promising Directions. Stroke, 2020. 51(9): p. 2872–2884. pmid:32757757

12. Tu L.H., et al., Pilot MRI-based strategies to improve the detection of stroke in patients with dizziness/vertigo. Clin Imaging, 2022. 82: p. 234–236. pmid:34902799

13. Vanni S., et al., STANDING, a four-step bedside algorithm for differential diagnosis of acute vertigo in the Emergency Department. Acta Otorhinolaryngol Ital, 2014. 34(6): p. 419–26. pmid:25762835

14. Heldner M.R., et al., Clinical prediction of large vessel occlusion in anterior circulation stroke: mission impossible? J Neurol, 2016. 263(8): p. 1633–40. pmid:27272907

15. Collins G.S., et al., Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement. BMC Medicine, 2015. 13(1): p. 1.

16. Kleindorfer D.O., et al., 2021 Guideline for the Prevention of Stroke in Patients With Stroke and Transient Ischemic Attack: A Guideline From the American Heart Association/American Stroke Association. Stroke, 2021. 52(7): p. e364–e467. pmid:34024117

17. Boehme A.K., Esenwa C., and Elkind M.S., Stroke Risk Factors, Genetics, and Prevention. Circ Res, 2017. 120(3): p. 472–495.

18. Aigner A., et al., Contribution of Established Stroke Risk Factors to the Burden of Stroke in Young Adults. Stroke, 2017. 48(7): p. 1744–1751. pmid:28619986

19. Bi Y. and Cao F., A Dynamic Nomogram to Predict the Risk of Stroke in Emergency Department Patients With Acute Dizziness. Front Neurol, 2022. 13: p. 839042. pmid:35250839

20. Hendrix P., et al., Risk Factors for Acute Ischemic Stroke Caused by Anterior Large Vessel Occlusion. Stroke, 2019. 50(5): p. 1074–1080. pmid:31009355

21. Breiman L., Random Forests. Machine Learning, 2001. 45(1): p. 5–32.

22. Strobl C., et al., Bias in random forest variable importance measures: Illustrations, sources and a solution. BMC Bioinformatics, 2007. 8(1): p. 25. pmid:17254353

23. Molnar, C., Interpretable machine learning. 2020: Lulu. com.

24. Brown L.D., Cai T.T., and DasGupta A., Interval Estimation for a Binomial Proportion. Statistical Science, 2001. 16(2): p. 101–117.

25. Pollard T.J., et al., tableone: An open source Python package for producing summary statistics for research papers. JAMIA Open, 2018. 1(1): p. 26–31. pmid:31984317

26. Paul N.L., Simoni M., and Rothwell P.M., Transient isolated brainstem symptoms preceding posterior circulation stroke: a population-based study. Lancet Neurol, 2013. 12(1): p. 65–71. pmid:23206553

27. De Giuli V., et al., Association Between Migraine and Cervical Artery Dissection: The Italian Project on Stroke in Young Adults. JAMA Neurology, 2017. 74(5): p. 512–518. pmid:28264095

28. Øie L.R., et al., Migraine and risk of stroke. J Neurol Neurosurg Psychiatry, 2020. 91(6): p. 593–604. pmid:32217787

29. Chiang M.C., et al., Migraine with Visual aura and the Risk of Stroke- a Narrative Review. J Stroke Cerebrovasc Dis, 2021. 30(11): p. 106067. pmid:34461446

30. Wu X., et al., CT Angiography for Triage of Patients with Acute Minor Stroke: A Cost-effectiveness Analysis. Radiology, 2020. 294(3): p. 580–588.

31. Lorenzetti D.L., et al., Strategies for improving physician documentation in the emergency department: a systematic review. BMC Emerg Med, 2018. 18(1): p. 36. pmid:30558573

32. Hippisley-Cox J., et al., Risk of thrombocytopenia and thromboembolism after covid-19 vaccination and SARS-CoV-2 positive testing: self-controlled case series study. BMJ, 2021. 374: p. n1931. pmid:34446426