Use of a Diagnostic Score to Prioritize Computed Tomographic (CT) Imaging for Patients Suspected of Ischemic Stroke Who May Benefit from Thrombolytic Therapy

Abstract

Background

A shortage of computed tomographic (CT) machines in low and middle income countries often results in delayed CT imaging for patients suspected of a stroke. Yet, time constraint is one of the most important aspects for patients with an ischemic stroke to benefit from thrombolytic therapy. We set out to assess whether application of the Siriraj Stroke Score is able to assist physicians in prioritizing patients with a high probability of having an ischemic stroke for urgent CT imaging.

Methods

From the Malaysian National Neurology Registry, we selected patients aged 18 years and over with clinical features suggesting of a stroke, who arrived in the hospital 4.5 hours or less from ictus. The prioritization of receiving CT imaging was left to the discretion of the treating physician. We applied the Siriraj Stroke Score to all patients, refitted the score and defined a cut-off value to best distinguish an ischemic stroke from a hemorrhagic stroke.

Results

Of the 2176 patients included, 73% had an ischemic stroke. Only 33% of the ischemic stroke patients had CT imaging within 4.5 hours. The median door-to-scan time for these patients was 4 hours (IQR: 1;16). With the recalibrated score, it would have been possible to prioritize 95% (95% CI: 94%–96%) of patients with an ischemic stroke for urgent CT imaging.

Conclusions

In settings where CT imaging capacity is limited, we propose the use of the Siriraj Stroke Score to prioritize patients with a probable ischemic stroke for urgent CT imaging.

About the Authors:

Wen Yea Hwong

Affiliations National Clinical Research Centre, Kuala Lumpur General Hospital, Kuala Lumpur, Malaysia, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands

ORCID http://orcid.org/0000-0002-6755-920X

Michiel L. Bots

Affiliation: Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands

Sharmini Selvarajah

Affiliation: Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands

L. Jaap Kappelle

Affiliation: Department of Neurology and Neurosurgery, Brain Center Rudolf Magnus, University Medical Center Utrecht, Utrecht, the Netherlands

Zariah Abdul Aziz

Affiliation: Department of Neurology, Hospital Sultanah Nur Zahirah, Terengganu, Malaysia

Norsima Nazifah Sidek

Affiliation: Department of Pharmacy, Hospital Sultanah Nur Zahirah, Terengganu, Malaysia

Ilonca Vaartjes

* E-mail: [email protected]

Affiliation: Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands

Introduction

Use of CT imaging is considered the gold standard diagnostic test to distinguish between an ischemic and a hemorrhagic stroke.[1] In areas with inadequate resources, a majority of stroke patients do not have immediate access to CT imaging. A Malaysian national report showed that in 2010, there were 52 machines in 134 public hospitals (39%). In private hospitals, only 44% had direct access to CT imaging.[2] Access to CT imaging is also hindered by the limited availability of ambulances to transport patients. There were 793 functioning ambulances throughout the country in 2010. This equated to a low 0.28 per 10,000 population[3] when compared to the expected 1 for every 10,000 population in high income countries.[4]

This limitation of resources is a major challenge to stroke care in case of time constraints. A delay in diagnosis reduces the eligibility of patients with an ischemic stroke for thrombolytic therapy. Previously, scores to differentiate between the types of stroke were developed based on clinical parameters to accommodate for the shortage of CT machines. Among the scores available include the Siriraj Stroke Score (SSS)[5], the Guys’ Hospital Score or Allen Score[6], the Besson Score[7] and the Greek Stroke Score[8]. Although several validation studies concluded that the scores were not sensitive enough in their detection of hemorrhage to replace CT imaging[9–11], there was a consistent trend of higher predictive abilities of the SSS to rule out a hemorrhagic stroke and thus, increases the chance of detecting an ischemic stroke.[10] Nevertheless, there remains a need to assess the ability of the SSS to identify ischemic stroke patients who are potentially eligible for thrombolytic therapy in the Malaysian population. Instead of replacing CT machines, the SSS is anticipated to act as a triage for prioritization of CT imaging.

This study was therefore conducted to examine the predictive value of the Siriraj Stroke Score to identify patients who are likely to have an ischemic stroke and are potentially eligible for thrombolytic therapy. The identification of these patients would allow prioritization of urgent CT imaging and increase the number of patients that can be treated with thrombolytic therapy.

Methods

Cohort Enrollment

Between July 2009 and December 2014, 7592 patients from 14 public hospitals were registered in the Malaysian National Neurology Registry.[12] This is by far the largest and best available representation of the Malaysian population for stroke patients. Public hospitals cover a majority of total hospital admissions in the country; 66.2% of all hospital admissions in 2014 were in public facilities.[13]

Collection of data for this registry follows local routine clinical practice. For the present study, we included patients above 18 years old with signs and symptoms of stroke who arrived in the hospital 4.5 hours or less from ictus. This time frame is in accordance to local and international stroke management guidelines for the eligibility of intravenous thrombolytic therapy.[14,15] We excluded patients who received CT imaging after 15 days from ictus.

Ethical approval was obtained from the Malaysian Ministry of Health’s research ethics committee (ID: NMRR 08-1631-3189). The approval includes data collection and use of data for secondary analysis. With a waiver of informed consent, a public notice is displayed at all participating sites and participants have the option to opt out.

Types of Stroke

A diagnosis of either a hemorrhagic or a non-hemorrhagic stroke is based on CT imaging. Patients with an intracerebral hemorrhage, subarachnoid hemorrhage (SAH) or ischemia with hemorrhagic transformations were included in the category of hemorrhagic strokes. Ischemic strokes, transient ischemic attacks (TIA) and cerebral venous thrombosis were classified as non-hemorrhagic strokes. More than 90% of patients in the category of non-hemorrhagic strokes consisted of patients with an ischemic stroke, and therefore, the term ‘ischemic stroke’ will be used in subsequent sections for easier interpretation.

CT imaging was interpreted by physicians and subsequently verified by radiologists from the participating hospitals. Stroke scores were not calculated prior to the interpretation of CT imaging. There was no possibility that knowledge of the scores could have an influence on the outcome.

The Siriraj Stroke Score (SSS)

For each patient, the SSS was calculated. Clinical variables which are needed for this score are listed in Table 1. Some symptoms are not available in the registry. We substituted them with comparable variables: ‘angina’ was replaced with ‘ischemic heart disease’, and instead of ‘intermittent claudication’, we took ‘peripheral artery disease’. Data were obtained via routine examination of patients during their arrival in the hospitals. History of comorbidities was verified with their past medical records.

[Figure omitted. See PDF.]

Table 1. Variables in the Original Siriraj Stroke Score.

https://doi.org/10.1371/journal.pone.0165330.t001

Clinicians who developed the original SSS score proposed two cut-off values: <-1 for ischemic stroke and >1 for hemorrhagic stroke.[5] For patients who score between -1 and 1, a distinction between ischemic and hemorrhagic stroke cannot be made. The equation for the original score is:

Statistical Analysis

First, the proportion of missing data was evaluated for each predictor (S1 Table). We carried out multiple imputation with m = 10 (number of imputations) to reduce the extent of bias resulting from missing data.[16] For counts and descriptive statistics, we took the modes of each imputed variable per individual patient. In the event of multiple modes, the mode with highest value was taken. Multiple imputation was conducted with R version 3.1.1.[17]

Second, we fitted the SSS on our dataset. Performance of the predictive ability of the score is evaluated via discrimination: area under the curve (AUC) of receiver operating characteristics (ROC) curves is calculated[18]; and via calibration: the slope and intercept of calibration plots are estimated to assess differences between predicted and observed probabilities[19].

Thirdly, the analysis was extended with updating methods to obtain a ‘best, fitted score’.[20] We recalibrated the intercept and the slope and adjusted the regression coefficients for predictors ‘level of consciousness’ and ‘headache’. Details of the updating methods and operationalization of each predictor are shown in S1 and S2 Tables. Furthermore, performance of the recalibrated score is tested for its internal validity via bootstrapping. With bootstrap analysis, the shrinkage factor and the optimism corrected AUC are calculated.[20]

Lastly, we took different cut-off values of the recalibrated SSS to assess its diagnostic performance in distinguishing between a hemorrhagic and an ischemic stroke. We also attempted to identify possible effect modifications for the score across age categories, sex and cardiovascular risk factors including hypertension, hyperlipidemia and atrial fibrillation. As an additional analysis, we validated the recalibrated score in a cohort where the inclusion time frame was expanded to 6 hours from ictus. This is to accommodate ischemic stroke patients who were potentially eligible for intra-arterial thrombolytic therapy.[14,15]

Statistical analysis was performed using Stata version 13.0.[21]

Results

Baseline Characteristics

Of 2176 patients who were included, 57% were males. The mean age was 62 (SD: 12) years (Table 2). Cardiovascular risk factors were common: nearly three quarters of patients had hypertension (73%), 40% with diabetes mellitus and 28% had dyslipidemia. Only 4% of the included patients had a history of atrial fibrillation whereas almost 50% of them were current smokers. Mean blood pressure upon arrival was 171mmHg (SD:36) for systolic and 94mmHg (SD:21) for diastolic. The median door-to-scan time was 4 hours (IQR: 1;14) and the median time taken for patients to arrive in the emergency departments was 2 hours (IQR: 1;3).

[Figure omitted. See PDF.]

Table 2. Baseline Characteristics.

https://doi.org/10.1371/journal.pone.0165330.t002

Fitting the Siriraj Stroke Score

Initial calibration of the original SSS showed a poor fit. There was overestimation at higher observed probabilities and underestimation at lower observed probabilities. The recalibrated score, in which the intercept and weights of the individual determinants were adjusted showed a better fit with an AUC of 0.80 (95%CI: 0.78–0.83). Post bootstrap analysis, a shrinkage factor of 0.99 and an optimism for the AUC of 2.18x10^-5 were found. The optimism-corrected AUC was similar to that of the recalibrated score (results in S1 Fig and S3 Table). After simplification of the coefficients, the equation for the recalibrated score is:

Potential effect modifiers were tested and no significant interactions were found.

Use of the Recalibrated Siriraj Stroke Score in Prioritization of CT Imaging

In the present cohort, 73% (n = 1594) of the patients had an ischemic stroke. Only one third of these ischemic stroke patients (33%) who were potentially eligible for thrombolytic therapy received CT imaging within the therapeutic window period of 4.5 hours.

Table 3 provides results of the application of the recalibrated SSS for identification of an ischemic stroke. Depending on the choice of cut-off values, results differ. At a threshold of 0 (SSS equal or higher than 0 defines a hemorrhagic stroke), 95% (n = 1519) of the ischemic stroke patients would have been correctly diagnosed, thus prioritized for urgent CT imaging.

[Figure omitted. See PDF.]

Table 3. Application of the Recalibrated Siriraj Stroke Score at Different Thresholds.

https://doi.org/10.1371/journal.pone.0165330.t003

By applying the recalibrated SSS, 83% of the cohort would have been sent for urgent CT imaging. Of those, 84% would have had an ischemic stroke and 16% would have been diagnosed with a hemorrhagic stroke. Around 5% of the ischemic stroke patients would not have immediate CT imaging (Table 3).

Results from the additional analysis where the inclusion time frame was expanded from 4.5 hours to 6 hours (n = 2658) showed similar results to the above (S4 Table).

Discussion

Despite arriving within the therapeutic time for thrombolytic therapy, two-thirds of the ischemic stroke patients did not receive urgent CT imaging by 4.5 hours. This delay in diagnosis inevitably eliminate their potential eligibility for thrombolytic therapy. With the recalibrated SSS, 95% of patients with an ischemic stroke who were potentially eligible for thrombolytic therapy, could be prioritized for urgent CT imaging.

Our results on the discriminative power of the SSS were comparable with two validation studies who had reported an AUC of 0.78 (95% CI: 0.75–0.82)[22] and 0.80(95% CI not given)[23]. Previous studies including a study of smaller sample size from Malaysia reported that the original SSS has failed to achieve a high sensitivity to detect hemorrhagic stroke.[9–11,24] This is particularly so in Western countries.[25,26] Nevertheless, higher specificities for the score were found. This increases the predictive ability of the score to identify an ischemic stroke. For example, Mwita et al[10] in a systematic review reported consistently higher specificities of the SSS with a range from 65%–99% compared to its corresponding sensitivities in 18 validation studies. This is in agreement with our specificity findings. The specificity for the recalibrated score was further improved with a choice of a different cut-off value. Moreover, parallel to our aims, some of the studies[22,27] supported the use of the higher predictive rule-out ability of the SSS to detect ischemic stroke patients in areas with insufficient resources.

Previous validation studies focused on the ability of the SSS to replace CT imaging.[5,9–11,22] Here, we propose to use this score to prioritize patients who are likely to have an ischemic stroke and are potentially eligible for thrombolytic therapy for urgent CT imaging. Up to the present time, only few hospitals in the country are administering thrombolytic therapy. Reasons for this poor uptake include strict exclusion criteria such as prior use of Vitamin K antagonist, uncertainty over its benefit for elderly patients and patients with mild or very severe stroke and most importantly, a narrow therapeutic window time.[28] Although this is a similar picture globally[29], lack of infrastructure is often an additional barrier to the administration of this therapy. The present study focuses on an aspect which is common in developing regions; time delays resulting from a shortage of CT machines. Other criteria which deny patients from their eligibility for thrombolytic therapy were not taken into account in our selection of the best patients for prioritization of CT imaging.

Establishing a clinical diagnosis of an ischemic stroke with the SSS will thus, allow prioritization of the right patients for the right treatment at the right time. Moreover, there are only 5 clinical variables needed with the SSS; all easily retrieved during initial assessment of patients with symptoms of stroke. The simplicity of this score and its practical application has been commended in several studies.[5,10] Nevertheless, we are aware that the recalibrated SSS may not be easily memorized or applied without a calculator. Future implementation of this score with a possible smartphone application will be useful.

Our study had a sufficient sample size to validate the score. We performed multiple imputations to minimize possible biases from missing data. Moreover, instead of developing a new score, we chose to revise the original SSS by applying updating methods. In this respect, the score was adjusted to the new validated population without losing prior information from the development dataset.[20,30] One limitation of the study is the lack of information on stroke-mimicking diagnoses. Nevertheless, a huge majority of patients with stroke symptoms do have a diagnosis of a stroke. Previous studies showed varying rates of stroke mimics from 9–19%[31–33]. Selection bias from our selection of cohort should therefore be minimal.

There are a few important points to be emphasized with regards to the score. First, there is no intention to deprive patients from CT imaging. All suspected-for-stroke patients should receive CT imaging for diagnostic purposes. Our target patients for prioritization are ischemic stroke patients who reach the hospital within 4.5 hours because there is a tight time constraint to maximize the benefits from thrombolytic therapy. In our cohort, despite having a similar median duration of time taken to arrive at the hospital from stroke onset, patients with hemorrhagic stroke were found to have a shorter door-to-scan time in comparison to ischemic stroke patients. The median door-to-scan time was 2 hours (IQR:1;8) for the former and twice as long for the latter. While we acknowledge the urgency of a surgical procedure for patients who are suspected of a subarachnoid hemorrhage, to our best knowledge, there is no specific window period to perform surgical coiling or clipping.[34,35]

Second, the aim of this score is to guide decisions and not to overrule clinical concerns. Other factors which may influence a physician’s judgment in prioritizing patients who are suspected of a stroke for immediate CT imaging include age, other comorbidities and severity of the condition. Third, this score was recalibrated to the Asian or specifically, the Malaysian population. Differences in terms of prevalence of hemorrhagic stroke and other cardiovascular risk factors between populations may limit the generalizability of the score’s utility. The performance of this recalibrated score should be tested prior to its application in clinical practice for other populations. Fourth and most importantly, at the cut-off value chosen, 5% of ischemic stroke cases were underdiagnosed. We took the threshold of 0 to minimize this proportion of patients and at the same time, finding the best balance between false positives and false negatives. This may be less than ideal but in places that lack infrastructures, choices have to be made and we consider this as the best alternative available. Preferences of choosing specific cut-off values may differ according to the availability of resources in respective regions. With the threshold chosen, we would have to prioritize 83% of patients for urgent CT imaging. While prioritization improves efficiency in hospitals with limited CT machines, foreseeable challenges lie in hospitals without CT machines that require inter-hospital transfers for urgent CT imaging. One would be the capacity to transport these patients to hospitals with CT machines available. Should a shortage in this aspect occur, cut-off values would have to be reduced but, at the expense of a higher proportion of missed ischemic stroke cases for urgent CT imaging. Another relevant point to note is the influence of the duration for inter-hospital transfers on the eligibility for thrombolytic therapy. Having said that, about 50% of the suspected patients in our cohort arrived at the hospital within 2 hours from the onset of their symptoms. Priorities therefore, should be given to these patients once they scored below 0 (suspected ischemic stroke) for immediate stabilization and transfer for urgent CT imaging.

Conclusions

While the best option is still to have immediate access to CT imaging for every stroke patient, a shortage of CT machines would probably remain as a problem in the near future. Prioritization with the Siriraj Stroke Score aims to reduce the issues of insufficient resources. More importantly, its application is targeted to optimize stroke care especially for ischemic stroke patients with time-dependent therapy.

Supporting Information

[Figure omitted. See PDF.]

S1 Fig. Comparison of Calibration Plots Prior to (original score) and after (recalibrated score) Updating Methods.

https://doi.org/10.1371/journal.pone.0165330.s001

(TIF)

S1 Table. Proportion of Missing Data and Operationalization of Predictors.

https://doi.org/10.1371/journal.pone.0165330.s002

(DOCX)

S2 Table. Updating Methods.

https://doi.org/10.1371/journal.pone.0165330.s003

(DOCX)

S3 Table. Regression Coefficients for Each Predictor by Methods.

https://doi.org/10.1371/journal.pone.0165330.s004

(DOCX)

S4 Table. Application of the Recalibrated Siriraj Stroke Score at Different Thresholds (expanded time window of 6 hours).

https://doi.org/10.1371/journal.pone.0165330.s005

(DOCX)

Acknowledgments

We thank the Director General of Ministry of Health, Malaysia for his permission to publish this manuscript. We are grateful to Dr. Peter Zuithoff from the Julius Support for his expert statistical advices. Our sincere thanks also goes to personnel from all participating hospitals for their contribution in the data collection.

Author Contributions

1. Conceptualization: WYH MLB IV LJK.

2. Formal analysis: WYH MLB IV.

3. Methodology: WYH MLB IV.

4. Writing – original draft: WYH MLB IV.

5. Writing – review & editing: LJK SS ZAA NNS.

Citation: Hwong WY, Bots ML, Selvarajah S, Kappelle LJ, Abdul Aziz Z, Sidek NN, et al. (2016) Use of a Diagnostic Score to Prioritize Computed Tomographic (CT) Imaging for Patients Suspected of Ischemic Stroke Who May Benefit from Thrombolytic Therapy. PLoS ONE 11(10): e0165330. https://doi.org/10.1371/journal.pone.0165330

References

1. Sandercock P, Molyneux A, Warlow C. Value of computed tomography in patients with stroke: Oxfordshire Community Stroke Project. Br Med J (Clin Res Ed). 1985;290:193–7.

2. Sivasampu S, Fatihah M, Akma N, Laili M, Khalid I, Siti Z, et al. Acute Curative Hospital Services in Malaysia. In: National Healthcare Establishment and Workforce Statistics (Hospital). Kuala Lumpur; 2011. p. 5–24.

3. Arunah C, Teo A, Faizah A, Mahathar A, Tajuddin A, Khairi K, et al. Emergency and Trauma Services in Malaysian Hospitals. In: National Healthcare Establishment and Workforce Statistics (Hospital). Kuala Lumpur; 2010. p. 73–86.

4. Cadigan RT, Bugarin CE. Predicting demand for emergency ambulance service. Ann Emerg Med. 1989;18:618–21. pmid:2729686

5. Poungvarin N, Viriyavejakul A, Komontri C. Siriraj stroke score and validation study to distinguish supratentorial intracerebral haemorrhage from infarction. Br Med J. 1991;302:1565–7.

6. Allen CM. Clinical Diagnosis of the Acute Stroke Syndrome. Q J Med. 1983;52:515–23. pmid:6657914

7. Besson G, Robert C, Hommel M, Perret J. Is It Clinically Possible to Distinguish Nonhemorrhagic Infarct From Hemorrhagic Stroke? Stroke. 1995;26:1205–9. pmid:7604415

8. Efstathiou SP, Tsioulos DI, Zacharos ID, Tsiakou AG, Mitromaras AG, Mastorantonakis SE, et al. A new classification tool for clinical differentiation between haemorrhagic and ischaemic stroke. J Intern Med. 2002;252:121–9. pmid:12190887

9. Connor MD, Modi G, Warlow CP. Accuracy of the Siriraj and Guy’s Hospital Stroke Scores in urban South Africans. Stroke. 2007;38:62–8. pmid:17138946

10. Mwita C, Kajia D, Newton C, Gwer S, Etyang A. Accuracy of clinical stroke scores for distinguishing stroke subtypes in resource poor settings: A systematic review of diagnostic test accuracy. J Neurosci Rural Pract. 2014;5:330–9. pmid:25288833

11. Hui ACF, Wu B, Tang ASY, Kay R. Lack of clinical utility of the Siriraj Stroke Score. Intern Med J. 2002;32:311–4. pmid:12088348

12. Nazifah SN, Azmi IK, Hamidon BB, Looi I, Zariah A, Hanip MR. National Stroke Registry (NSR): Terengganu and Seberang Jaya experience. Med J Malaysia. 2012;67:302–4. pmid:23082422

13. Health Facts Malaysia. Ministry of Health, Malaysia. 2015.

14. Malaysian Society of Neurosciences Stroke Council. Clinical Practice Guidelines: Management of Ischaemic Stroke. 2nd Edition. Ministry of Health Malaysia, Academy of Medicine Malaysia.

15. Jauch EC, Saver JL, Adams HP, Bruno A, Connors JJB, Demaerschalk BM, et al. Guidelines for the Early Management of Patients With Acute Ischemic Stroke: A Guideline for Healthcare Professionals from the American Heart Association/American Stroke Association. Stroke. 2013;44:870–947. pmid:23370205

16. McCleary L. Using Multiple Imputation for Analysis of Incomplete Data in Clinical Research. Nurs Res. 2002;51:339–43. pmid:12352784

17. R Core Team (2014). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing; Vienna, Austria. http://www.R-project.org/.

18. Obuchowski NA. Receiver Operating Characteristic Curves and Their Use in Radiology. Radiology. 2003;229:3–8. pmid:14519861

19. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21:128–38. pmid:20010215

20. Steyerberg EW, Borsboom GJJM, van Houwelingen HC, Eijkemans MJC, Habbema JDF. Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Stat Med. 2004;23:2567–86. pmid:15287085

21. Stata Corp. Stata Statistical Software: Release 13. College Station, TX: Stata Corp LP; 2013.

22. Nouira S, Boukef R, Bouida W, Marghli S, Dridi Z, Benamou S, et al. Accuracy of Two Scores in the Diagnosis of Stroke Subtype in a Multicenter Cohort Study. Ann Emerg Med. 2009;53:373–8. pmid:18708271

23. Ozeren A, Bicakci S, Burgut R, Sarica Y, Bozdemir H. Accuracy of bedside diagnosis versus Allen and Siriraj stroke scores in Turkish patients. Eur J Neurol. 2006;13:611–5. pmid:16796585

24. Kan CH, Lee SK, Low CS, Velusamy SS, Cheong I. A validation study of the Siriraj Stroke Score. Int J Clin Pract. 2000;54:645–6. pmid:11221275

25. Weir CJ, Murray GD, Adams FG, Muir KW, Grosset DG, Lees KR. Poor accuracy of stroke scoring systems for differential clinical diagnosis of intracranial haemorrhage and infarction. Lancet. 1994;344:999–1002. pmid:7934437

26. Celani MG, Righetti E, Migliacci R, Zampolini M, Antoniutti L, Grandi FC, et al. Comparability and validity of two clinical scores in the early differential diagnosis of acute stroke. BMJ. 1994;308:1674–6. pmid:8025461

27. Goswami RP, Karmakar PS, Ghosh A. Bedside utility of clinical scoring systems in classifying stroke. Indian J Med Sci. 2013;67:137–45. pmid:24326766

28. Audebert HJ, Sobesky J. Stroke: “Time is brain” after stroke, regardless of age and severity. Nat Rev Neurol. 2014;10:675–6. pmid:25330727

29. Adeoye O, Hornung R, Khatri P, Kleindorfer D. Recombinant Tissue-Type Plasminogen Activator Use for Ischemic Stroke in the United States: A Doubling of Treatment Rates Over The Course of 5 Years. Stroke. 2011;42:1952–5. pmid:21636813

30. Janssen KJM, Moons KGM, Kalkman CJ, Grobbee DE, Vergouwe Y. Updating methods improved the performance of a clinical prediction model in new patients. J Clin Epidemiol. 2008;61:76–86. pmid:18083464

31. Hand PJ, Kwan J, Lindley RI, Dennis MS, Wardlaw JM. Distinguishing Between Stroke and Mimic at the Bedside: The Brain Attack Study. Stroke. 2006;37:769–75. pmid:16484610

32. Libman RB. Conditions That Mimic Stroke in the Emergency Department. Implication for Acute Stroke Trials. Arch Neurol. 1995;52:1119–22. pmid:7487564

33. Allder S, Moody A, Martel A, Morgan P, Delay G, Gladman J, et al. Limitations of clinical diagnosis in acute stroke. Lancet. 1999;354:1523. pmid:10551501

34. Whitfield PC, Kirkpatrick PJ. Timing of surgery for aneurysmal subarachnoid haemorrhage. Cochrane database Syst Rev. 2001;CD001697. pmid:11405999

35. Dorhout Mees SM, Molyneux AJ, Kerr RS, Algra A, Rinkel GJE. Timing of Aneurysm Treatment After Subarachnoid Hemorrhage: Relationship with Delayed Cerebral Ischemia and Poor Outcome. Stroke. 2012;43:2126–9. pmid:22700527