1. Introduction
Ischaemic stroke is the second cause of mortality in Spain in the general population and the first in women [1]. It is also the second cause of mortality worldwide and the third most common in industrialised countries [2,3]. The prevalence of this major public health problem puts significant strain on health system resources.
A large proportion of strokes (approximately 62%) are attributable to ischaemic processes, many of which are not treated with reperfusion treatment (pharmacological or mechanical) because they do not meet the strict criteria for its application [4,5]. Determining the proportion of patients undergoing reperfusion is a complex task. Among other factors, it depends on the time of onset of the stroke, the priorities and capacities of the health system and the emergency management elements available in each case. A recent study using data from 44 European countries reported a mean reperfusion rate with thrombolysis of 72.5 cases per 1000 stroke episodes; with mechanical thrombectomy, this rate fell to 19.3 cases per 1000 episodes [5,6]. Although these figures are open to improvement, it is apparent that the period from the onset of the stroke to the administration of the fibrinolytic therapy or thrombectomy provides a window of opportunity for successful treatment. Thus, during the first three hours following the onset of this condition, the application of fibrinolytics achieves a marked reduction in disability in 25% of patients. Mechanical thrombectomy, which is normally only available in specialised centres and presents a heterogeneous international distribution, also increases survival rates and drastically decreases sequelae [7,8,9,10]. However, a high proportion of strokes are not suitable for any type of reperfusion, and it is these on which our study is focused.
Various risk factors have been detected for the occurrence of an ischaemic event and for the probability of death or major sequelae [11,12]. Among other studies in this respect, Smith et al. [13] proposed a predictive model for hospital mortality following ischaemic or haemorrhagic stroke. This model had a moderate–high discriminant capacity. Other studies have also reported satisfactory predictive models to assess the risk of mortality in acute stroke [14,15,16,17,18].
In Spain, studies have been undertaken to establish the determinant factors of mortality due to ischaemic stroke, based on an analysis of hospital records. However, to our knowledge, none have proposed a predictive model based on the national database that provides the basic information obtained for each hospital admission episode (the Spanish Minimum Basic Data Set, MBDS). Analysis of data from this source would allow us to estimate the probability of mortality during the hospitalisation for acute stroke when reperfusion treatment is not provided. However, these administrative data are not amenable to analysis, and it can be a complex task to determine the severity of the stroke exclusively from the information contained in this database [19].
In addition, the aforementioned characteristics of stroke have recently led to an increasingly frequent use of machine learning (ML) or deep learning (DL) techniques. Recently, attempts have been made to predict stroke mortality using principal component analysis methods scaled to neural networks with very good predictive results [20]. In addition to stroke, other pathologies such as heart failure and sepsis are being studied using this methodology. In the case of heart failure, there are promising ML studies that validate and increase the performance of classical statistical models [21]. In the study of mortality due to sepsis, the use of convolutional neural networks is gaining ground [22].
When a stroke patient is admitted to the emergency room, the time elapsed since the event, the therapeutic measures adopted and the training of the care team are of decisive importance in reducing in-hospital mortality.
Subsidiary data compiled on admission or during the first hours of the patient’s hospital stay can be incorporated into a predictive model that can be used to estimate the probability of mortality during admission. The successful development and application of such a model would have important benefits for hospital procedures and efficacy.
On admission following a stroke, patients are rigorously evaluated by specialised personnel, who assess not only the degree of severity but also the time elapsed from the onset of symptoms to the provision of initial care. Many patients are not considered candidates for reperfusion treatment because the window for treatment has closed, because they present contraindications or because there is a high risk of haemorrhagic transformation, among other reasons. For these patients, an individualised prognostic study is required.
In view of these considerations, we aim to construct a predictive model to identify patients at greatest risk of dying during admission for ischaemic stroke when they are not eligible for reperfusion treatment. Such a model would make it possible to optimise clinical actions and improve treatment outcomes for this substantial group of patients. In a complementary analysis, we also assess the performance of ML–DL techniques.
2. Materials and Methods
2.1. Study Design and Data Source
This analytical observational study considers a cohort representing all the episodes of hospitalisation in Spain for patients with ischaemic stroke and who did not receive reperfusion therapy (diagnostic group 14 in the Spanish classification system, GRD), during the period 2008–2012. It was assumed that the attribution of cases to DRG-14 was correct in all cases.
The source of information was the Minimum Basic Data Set (MBDS) compiled by the Ministry of Health, Consumer Affairs and Social Welfare. This database contains both treatment and administrative information. The system originated in the USA and was later adopted by the European Economic Community. In Spain, the MBDS has been of mandatory implementation since 1987.
The unit of statistical analysis considered was the hospitalisation episode and not the individual; in total, 186,245 hospitalisation episodes classed as GRD-14 were analysed. Each group within this system is homogeneous by resource consumption but not in terms of disease severity. Accordingly, the processes can be grouped to enable comparability between different health regions and geographical areas.
To facilitate subsequent internal validation, the cohort was randomly segmented into two subsamples, a training set (80% of the episodes) and a test set (20% of the episodes). These subcohorts were established using a blind process of simple random sampling, and all statistical studies except the internal validation process were performed on the training set.
2.2. Variables
In all our statistical analyses, the main variable was in-hospital mortality, that is, death during hospitalisation due to an ischaemic stroke when no reperfusion treatment had been received. This variable was used to develop the intermediate and final models, as well as during the validation process.
The remaining variables were taken as independent or predictor variables and were composed of two main groups: the sociodemographic covariates of age, sex and geographic origin; and the comorbidities, mainly ischaemic heart disease, chronic bronchopneumopathy, renal and/or respiratory insufficiency, dyslipidaemia, arterial hypertension, diabetes, obesity, valvular heart disease, anaemia, arrhythmias—especially atrial fibrillation—and pneumonia.
In addition to the above, a specific analysis was performed of case management variables and those related to hospital admission characteristics. The latter included the number of diagnoses on discharge (NDD), as a proxy for the level of comorbidities, and the number of procedures at discharge (NPD), as a proxy for the diagnostic–therapeutic effort received. Other key variables were the length of stay and readmission for the same reason within one month of discharge.
The definition of readmission deserves special attention. The approximation to the number of patients and readmissions was made by identifying the same patient for a hospital and for a specific year, based on the history number, hospital code, date of birth and sex. This allowed us to identify patients treated more than once in the same hospital.
The database was subjected to a moderate screening procedure for outliers regarding the length of stay, using the formula T2 = Q3 + 1.5 (Q3 − Q1), where Q is the quartile and T2 is the length of stay for DRG-14 above which values are considered extreme. In the present case, stays exceeding 21 days were considered atypical.
2.3. Statistical Analysis
The statistical analysis was carried out on the training cohort except for the final phase, in which the results were validated on the test cohort using the hold-out method. The entire analytical procedure was performed using SPSS v.21 (IBM Corp., Armonk, NY, USA and Stata v.14 software (StataCorp LLC, College Station, TX, USA).
The initial phase of the analysis consisted of a descriptive study of the variables of interest (sex, age, NDD, NPD, readmission, mortality and comorbidities). The qualitative variables are expressed as percentages and frequency distributions, and the quantitative ones as means and standard deviations.
A bivariate analysis was performed to detect associations between mortality and each of the predictors considered. These associations are expressed as unadjusted odds ratios (ORu), accompanied by the corresponding 95% confidence interval and statistical significance.
Finally, a binary logistic regression model was developed for the dependent variable “mortality”, using the manual introduction method to force the inclusion of significant variables in the bivariate analysis, together with other appropriate variables, according to the literature. The variables were manually withdrawn from the model according to their degree of contribution to its overall performance, prioritising the withdrawal of those that were not present at the moment of hospital admission or very shortly afterwards.
The model was evaluated for its discriminant capacity using the area under the curve (AUC) or C-statistic. The calibration was evaluated by visual comparison of the risk deciles obtained using the Hosmer−Lemeshow test, since the level of significance did not provide a suitable measure, due to the large sample size; in this case, therefore, visual interpretation provides more information. Pearson’s chi-square goodness-of-fit test was also used, as it is more appropriate when a large sample size is considered [23].
Finally, the model developed on the training cohort was applied to the test cohort, by the hold-out method, to establish internal validity. The AUC values obtained for each cohort were compared using the method described by DeLong et al. [24].
To facilitate an immediate estimation of the risk of mortality and to enable rapid access to the results during treatment for patients with stroke, we created a smartphone app and a web equivalent (URL:
From the above procedures, a risk score for mortality was derived, using the coefficients of the logistic regression model, as described by Sullivan et al. [26]. In this method, each coefficient is divided by the lowest value, and the score thus obtained is rounded to the nearest integer, which is taken as the weight assigned to each factor in the final score. For each patient included in the study, we calculated the total score according to the presence or absence of each factor. The score obtained for each patient was in the range from −12 to 61 points.
A complementary analysis of the database was conducted using deep learning (DL) techniques. The most relevant variables for the model were selected using random forest (RF) [27], and the permutations test was used as a further correction mechanism [28,29]. After selecting the variables with the highest predictive capacity, a multilayer perceptron neural network was constructed, and the performances of the network and the random forest procedure per se were obtained.
3. Results
The final analysis consisted of 186,245 episodes of hospitalisation for non-reperfused stroke. Of these episodes, 80% (n = 148,891) were taken as the training cohort and 20% (n = 37,354) as the test cohort. All statistical studies except the internal validation process were performed on the former.
The patients were mainly elderly (mean age 73.92 ± 12.53 years) and the mean duration of hospital stay was 7.54 days. Just over half (53.30%) were male and 4.8% were in readmission due to ictal symptoms. In-hospital mortality during the initial admission was 6.8%. Table 1 lists the main comorbidities presented and the other variables analysed.
The proportion of outliers was 6.11% when a stay of more than 21 days was considered an atypical stay (Figure 1). After removing the outliers, the global database was composed of 186,245 episodes of hospitalisation for non-reperfused stroke.
The bivariate analysis (Table 2) revealed significant differences in several of the variables. Thus, positive associations with mortality were obtained for female sex (ORu 1.792, 95% CI: 1.720–1.787), readmission (OR 2.324, 95% CI 2.165–2.493) and among comorbidities, COPD (OR 1.188, 95% CI: 1.101–1.282), renal failure (OR 1.664, 95% CI: 1.551–1.786), atrial fibrillation (OR 2.414, 95% CI: 2.316–2.516) and ischaemic heart disease (OR 1.324, 95% CI: 1.217–1.441). On the contrary, both dyslipidaemia (OR 0.489, 95% CI: 0.466–0.514) and hypertension (OR 0.792, 95% CI: 0.760–0.826) were negatively associated with the risk of death during the admission.
The final logistic model built on the training cohort was composed of ten variables and provided an AUC of 0.742 (95% CI: 0.737–0.747). This model was also applied to the test cohort, and both presented a moderate–high and very similar discriminant capacity (Table 3).
The logistic model built from the training cohort was composed of ten variables and presented a moderate–high discriminant capacity, with an AUC of 0.742 (Table 3, Figure 2).
Application of the model to the test cohort (to verify its internal validity) produced an AUC of 0.736, 95% CI 0.727–0.746 (Figure 3). The difference in the AUC was 0.006 units and the DeLong test was significant (p < 0.001) despite the minimal differences in relation to the large sample size. The model met the classical internal validation criterion, i.e., a decrease in the C-statistic of no more than one-tenth of a unit.
The calibration was also evaluated by the Hosmer–Lemeshow test and through the visual representation of the risk deciles (Figure 4). The Pearson test [30], which is highly recommended for large sample sizes, was also applied. This test showed there were no significant differences between the values observed and those predicted by the model (Pearson χ2 ungrouped, p = 0.176); the visual interpretation of the observed and expected deciles is consistent with this result.
The risk score produced values in the range of −12 to 61 points. Its graphical representation facilitates a rapid estimate of the relation between the score on the probability curve and the risk of death (Table 4 and Figure 5).
From the perspective of DL, a random forest analysis provided the variables with the highest weights as possible predictors of mortality (Figure 6), with age, heart failure, readmission and female sex, among others, standing out. The permutation test reduced the overestimation of the RF (Figure 7) although the same variables were maintained.
Using the variables selected from the previous tests as predictors, a multilayer perceptron neural network presented an AUC of 0.651. Table 5 shows the comparative values of the different metrics obtained for the different methodologies used.
4. Discussion
4.1. Findings
In this study, we analyse information obtained from clinical–administrative databases managed by the Spanish Ministry of Health, with respect to the period 2008–2012. These data were used to develop a predictive model based on logistic regression with a moderate–high capacity to estimate the risk of mortality in patients hospitalised for an ischaemic stroke when neither thrombolysis nor thrombectomy have been performed.
The model obtained provides an accurate visual calibration in terms of the concordance between the observed values and the mortality predicted according to the risk deciles obtained by the Hosmer–Lemeshow method. To streamline the risk estimation procedure, we also developed an auxiliary web application that allows the exact risk to be estimated almost immediately [31].
As the final outcome, we obtained a score to represent the mortality risk of each patient. It can be derived quickly and easily, and we believe it will be of great use to the clinician. It can be accompanied by a graphical representation that is equally fast and easy to use, either in combination with the results of the logistic model or independently.
On the other hand, the DL-based approach showed that modern data science methodologies do not always provide better results than more traditional statistical techniques. In this sense, logistic regression showed a predictive performance equal or superior to RF and neural network methodologies.
The use of techniques such as ML or DL to tackle this type of problem is growing every day, but in the studies consulted, there is one element that tends to be constant, and that is the use of databases that are more specific and oriented towards specific pathologies than the one we have used [20,22].
One of the frequent problems when working with databases in which the outcome variable represents a very small proportion is the imbalance of the data. There is a tendency towards asymmetry and imbalance in the classification tables that makes it necessary to use metrics other than the AUC-ROC, such as accuracy, precision and recall metrics. All of these are displayed in Table 5 and denote an acceptable behaviour of the model despite the difficulties imposed by the dataset.
4.2. Comparison with Previous Studies
Numerous prediction models and mortality scores for stroke patients have been presented, but to our knowledge, none in Spain based on the MBDS or any other clinical-administrative database using such a high volume of cases as in the present study. Instead, reference is frequently made to clinical records, using only small-to-moderate sample sizes and, in most cases, without generating a complete model. On the other hand, interesting studies based on artificial intelligence, using the random forest method, have been made to predict mortality risk. However, the theoretical basis, the details of the method applied and the combined use of clinical, neuroimaging and biochemical markers substantially distance these studies from the approach we describe [32]. Moreover, the model proposed in this paper can be built to obtain a very rapid estimate, during the initial assessment of the patient, thus enhancing care efficiency.
Numerous studies have been conducted in other countries, using various sources of information to develop and validate mortality risk scores, but most of them present significant differences from our work, especially as concerns the medium- to long-term estimation of mortality risk [13,15,16,17,18,19,33].
The PREMISE score, for predicting early mortality due to stroke, was developed from a prospective register with data from 38 stroke units in Austria. This score has been validated internally and externally and provides optimal results [16,34]. From a multi-variate model, the authors derived a score that performed well in terms of discriminant capacity (AUC 0.879) and visual calibration. The information source used in this study was a group of stroke units that supplied data to a specific national register. This source, hence, was much more precise than our own (a national clinical–administrative database that presents a certain lack of exhaustiveness, as it is not designed specifically for recording details of patients who have suffered a stroke). Clinical severity instruments such as the NIH Stroke Scale/Score (NIHSS), although they make comparability more complex, also enhance predictive capacity.
Three important scores have been developed to estimate short- and medium-term mortality. Two of these (providing 30-day estimates) were produced by the Canadian Stroke Network, and the third considered in-hospital mortality in a way similar to the approach we describe (Smith et al., 2010) [13]. In the latter work, a risk score method was devised to estimate in-hospital mortality, based on a study of 274,988 patients treated in 1036 U.S. hospitals. The study data were obtained from the “Get with the Guidelines—Stroke Program” [35,36]. Using this information, the authors developed referral and validation cohorts, alternately including and omitting the severity scale of the NIHSS. This procedure generated two risk models: the one excluding the NIHSS had a discriminant capacity of 0.72, and the one including it had a discriminant capacity of 0.85. Although the latter was clearly better, in both cases, discrimination was good, as was the visual plot calibration.
The results obtained in the present study reveal a discriminant capacity (based on the C-statistic) similar to that of Smith et al. when the NIHSS severity scale was not used. In common with these authors, we not only developed a model that excluded this key variable but also created one that both incorporates estimates of in-hospital mortality and uses only the variables present at the time of admission. This feature is extremely important, since knowing the patient’s risk profile in the first few minutes of evaluation facilitates the use of personalised medicine and enables clinical circuits and schedules to be organised for optimal effectiveness and efficiency.
Notably, therefore, our model obtains a similar discriminant capacity to that offered by the one proposed by Smith et al., in both cases excluding the NIHSS variable. This is a useful achievement because our model is based on a clinical–administrative information source, which is inevitably less exhaustive than the database used in the earlier study [37]. The visual representation of the risk deciles (observed vs. expected) in the Hosmer–Lemeshow (HL) test shows our calibration data to be acceptable. However, the HL test is based on a chi-square test, which with such large sample sizes is usually non-significant. To avoid this limitation, we applied the Pearson χ2 test, implemented by Stata software [38], which confirmed the absence of significant differences in our calibration analysis, as we hoped.
Studies in this respect have also been conducted in non-industrialised countries, for example by Gondar et al. [33]. These authors analysed medical records from Ethiopia, producing a predictive model to assess the risk of in-hospital mortality due to ischaemic stroke. However, the sample size was only modest, and the multivariate analysis performed was unable to achieve an optimal model based on logistic regression. However, a subanalysis based on Cox proportional hazards showed that intercurrent infections and impaired renal function were associated with higher mortality while, as in our study, dyslipidaemia was associated with lower mortality.
Finally, a recent study conducted in Canada proposed a risk score to classify stroke severity, basing their analysis on administrative data [19]. The discriminant capacity of this score was evaluated according to the inclusion of certain critical variables, producing an AUC value of 0.82 when the Canadian Neurological Scale (CNS) with real observed data was included. This value fell to 0.76 when the CNS was used with the data estimated by the model and further still when the CNS was excluded (C = 0.69). This model has the undoubted advantage of its inclusion of critical variables (via the CNS) in the assessment of stroke mortality risk. Nevertheless, our own model, despite lacking these critical variables, obtained results similar to those of the intermediate model above, derived from the CNS.
In conclusion, having compared the outcomes of the present study with those reported in the literature in this regard, we believe our work is consistent in methodology and results with most international studies in this field and achieves results similar to those of studies that use only general clinical and administrative variables and not those of a critical nature in patients with stroke.
4.3. Strengths of the Study
We present a model for estimating the risk of mortality in patients admitted for an acute non-reperfused stroke. This model is composed of variables obtained from clinical–administrative databases and provides moderate–high discriminant capacity and good calibration. It is the first such model to be based on the Spanish Minimum Basic Data Set that considers mortality, and, moreover, it is accompanied by a smartphone app that facilitates immediate estimation of the mortality risk.
Another major strength of this study is the large sample size considered and the long duration of the data collection period (five years). In addition, it takes into account all patients admitted for stroke, nationwide, during this period.
Prior to our investigation, many such risk scores and prediction models have been developed to address the same problem, but most attempt to predict mortality at different times after hospital discharge. Furthermore, the majority are constructed using information from real-life cases, not from clinical–administrative databases.
In most of these previous studies, the patients included in the analysis were treated in hospital stroke units or, to a lesser extent, conventional neurology wards. In consequence, little research attention has been paid to the profile of patients addressed in the present study [16,17,34]. Moreover, some studies examined only non-reperfused ischaemic strokes while others addressed haemorrhagic strokes and ICU treatment in separate analyses [15]. It is important to note that stroke units make greater use of specialist procedures and therapeutic effort than is the case with patients managed conservatively, as is the case of those examined in our study [38]. Only Aylin et al. [39] used data from a database comparable to the Spanish MDBS (in their case, from the British NHS) but this analysis was applied to non-cerebral vascular pathologies. In relation to the latter, therefore, our prediction and score model can be considered totally novel. Furthermore, the creation of a web tool, accessible from any device with an Internet connection, together with the provision of a smartphone application, enable the physician to resolve a complex logistic equation almost immediately, taking into account only variables that can readily be evaluated in the first moments of patient care.
Finally, a notable strength of the study is the model’s “explainability”. The model presented is easily interpretable, and the approach using logistic regression is optimal from the point of view of explainability, as this method makes it possible to understand the way in which the predictions are made with relative simplicity. The model is also very easy to implement (in a mobile application, on the web, etc.), and it can be easily modified and re-adapted after external validation processes, using new coefficients with significant efficiency.
4.4. Potential Limitations
A potentially significant limitation of the present study is the absence of data from the NIHSS scale in our source of information, which makes it much more difficult to raise the discriminant capacity of the model. Nevertheless, moderate, efficient AUC levels are achieved for use in clinical practice.
In addition, the databases consulted do not contain sufficient information to determine the severity of the stroke upon admission. For example, in our study, it was not possible to obtain variables enabling us to reproduce or estimate the NIHSS or the CNS. This inevitably reduces the precision obtained, compared to studies that do consider these critical variables. The Spanish clinical–administrative database is not pathology specific but is designed to encode any pathological process.
The classification used in this study (ICD 9MC) predates the ICD10 version currently in use; the clinical and outcome profile of patients with reperfusion criteria (thrombectomy or thrombolysis) has changed over the years, something that is not observed for patients who are managed conservatively. In fact, increasing the time available from stroke onset for the possible application of new therapies has been the main change in stroke management.
In the present study, we have assumed a correct attribution of episodes to DRG 14; undoubtedly there is a source of variability in coding that may make this a priori assumption a limitation to some extent. Moreover, the origin of the data is very heterogeneous and difficult to integrate, as it combines information from a large number of regional databases. In relation to these characteristics of the ICD version used, no attempt has been made to validate the obtained model outside the country or to other classifications, which would have placed a significant bias on the study. Finally, our study may be subject to underreporting bias, a well-known limitation of this type of information source [40].
4.5. Implications
The study we describe proposes a method to determine the stratification of mortality risk in a patient being treated in the emergency room (with no reperfusion) after undergoing an acute ischaemic stroke. This parameter is of fundamental importance in establishing the type and urgency of healthcare to be applied.
Personalising this risk allows clinicians to optimise the pathways provided and the complementary tests performed, ensuring that critical times are not exceeded. By establishing maximum delay times before consultation with the neurology specialist and time limits for performing imaging tests or initial care, and ultimately by promoting personalised medicine targeting the specific risk presented, hospitals can optimise the care provided for patients with this pathology. Furthermore, these measures would be valid not only for urgent care but also for more routine decision-making in the hospitalisation ward.
The model proposed is not intended to replace, under any circumstances, the opinion of the expert neurologist. However, we believe it will be a valuable resource when used in conjunction with clinical scales and specialist opinion.
5. Conclusions
We propose a practical means of obtaining a predictive model of the risk of mortality during the admission of patients with non-reperfused stroke. The use of such a model would have beneficial clinical and healthcare repercussions and contribute to optimising care for this group of patients.
Finally, we emphasise that, although the discriminant capacity of this model is similar to that offered in related work, our model and risk score are based exclusively on variables that can be determined just a few minutes after admission, with evident advantages for patient care.
Conceptualisation, M.C.L.-P., J.M.G.-T. and M.-J.S.; methodology, L.T.-R. and F.R.-L.; software and validation, J.M.G.-T. and L.T.-R.; formal analysis, J.M.G.-T. and L.T.-R.; investigation, data curation and writing—original draft preparation, J.M.G.-T., M.C.L.-P., L.A.-P., M.d.M.R.S. and P.M.-S.; writing—review and editing, J.M.G.-T., M.C.L.-P., L.A.-P., J.G.-E. and P.M.-S.; supervision, J.M.G.-T.; project administration, J.G.-E.; funding acquisition, M.C.L.-P. All authors have read and agreed to the published version of the manuscript.
This work was supported by the “Fundación Progreso y Salud”, in the context of FPS 2020—R&I projects in Primary Care, Regional hospitals and CHARES. Grant number AP-0013-2020-C1-F1 and the APC was funded by the same.
The study was conducted in accordance with the guidelines of the Declaration of Helsinki and was approved by the Ethics Committee of the province of Almeria (internal protocol code 1/2001, approval date 10 February 2020).
The present study did not involve individuals and/or animals, and the information sources were databases managed by the Spanish Ministry of Health. These are totally anonymised, without traceability. Moreover, the unit of analysis was hospitalisation episodes and not specific patients. In addition, this study was approved by the Almería Provincial Ethics Committee and complies with the 2016/679 General Regulation on Personal Data Protection regarding the use of anonymised population data. Informed consent was, therefore, not required.
The data presented in this study are available on request from the corresponding author. The data are not publicly available due to ethical restrictions.
The authors express their gratitude to the Research Unit of the Torrecárdenas University Hospital and to its General Management for their continued support for research. They also thank the FIBAO Foundation and the Progreso y Salud Foundation for their collaboration in the financing of this project. We would also like to express our deep gratitude to Dafina Petrova for her help in translating technical aspects of the manuscript.
The authors declare they have no conflict of interest. The funders had no role in the design of the study; in the collection, analysis or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Enlarge this image.Figure 1. Length of stay. Cutoff point for outliers. Cutoff point for outliers: 21 days.
Enlarge this image.Figure 2. ROC curve of the mortality predictor model applied to the training cohort. AUC 0.742, 95% CI: 0.737–0.747.
Enlarge this image.Figure 3. ROC curves for the training and test cohorts. AUC Train: 0.742, 95% CI: 0.737–0.74; AUC Test: 0.736, 95% CI: 0.727–0.746. AUC Train − AUC Test = 0.006, CI difference of the means 0.002–0.005.
Enlarge this image.Figure 4. Graphical representation of the risk deciles by the Hosmer–Lemeshow test. Hosmer–Lemeshow test values: χ2 = 26.094, p = 0.01.
Enlarge this image.Figure 6. Random forests. Relevance of predictors for mortality. HF: heart failure; FA: atrial fibrillation; DLP: dyslipidaemia; HTN: arterial hypertension; IHD: ischaemic heart disease; SSPF: Symptoms suggestive of posterior fossa infarction.
Descriptive variables.
| VARIABLE | ||
|---|---|---|
| Quantitative, mean ± sd | Age | 73.92 ± 12.53 (years) |
| Stay | 7.54 ± 4.54 (days) | |
| NDD | 6.92 ± 2.95 (diagnostics) | |
| NPD | 3.27 ± 2.45 (procedures) | |
| Qualitative, n (%) | Male sex | 79,412 (53.30) |
| Re-admission | 7140 (4.80) | |
| Mortality | 10,141 (6.80) | |
| COPD | 9898 (6.60) | |
| Ischaemic heart disease | 7181 (4.80) | |
| Arterial hypertension | 95,822 (64.4) | |
| Obesity | 9358 (6.30) | |
| Renal insufficiency | 9035 (6.10) | |
| Anaemia | 7840 (5.30) | |
| Atrial fibrillation | 37,050 (24.9) | |
| Diabetes | 46,241 (31.10) | |
| Dyslipidaemia | 52,857 (35.70) | |
| Heart failure | 11,141 (7.50) | |
| Basilar artery stenosis | 660 (4.00) | |
NDD: number of diagnoses on discharge; NDP: number of procedures on discharge; COPD: chronic obstructive pulmonary disease.
Bivariate study. Mortality-associated factors in the training cohort.
| Total | Death | |||||||
| n | % | n | %* | ORu | 95% CI | p | ||
| Sex | Male | 79,412 | 53.3 | 4045 | 5.1 | 1 | ||
| Female | 69,473 | 46.7 | 6096 | 8.8 | 1.792 | 1.720–1.787 | <0.0001 | |
| Age | 24–34 | 853 | 0.6 | 8 | 0.9 | 1 | ||
| 35–44 | 2791 | 1.9 | 37 | 1.3 | 1.419 | 0.658–3.059 | 0.372 | |
| 45–54 | 9233 | 6.2 | 124 | 1.3 | 1.438 | 0.701–2.949 | 0.322 | |
| 55–64 | 18,857 | 12.7 | 381 | 2 | 2.178 | 1.078–4.402 | 0.030 | |
| 65–74 | 33,122 | 22.3 | 1196 | 3.6 | 3.957 | 1.968–7.957 | <0.001 | |
| 75–84 | 54,662 | 36.7 | 3856 | 7.1 | 8.017 | 3.993–16.095 | <0.001 | |
| >84 | 29,373 | 19.7 | 4539 | 15.5 | 19.305 | 9.616–38.758 | <0.001 | |
| Year | 2008 | 29,509 | 19.8 | 2104 | 7.1 | 1 | ||
| 2009 | 29,577 | 19.8 | 2076 | 7.0 | 0.983 | 0.923–1.047 | 0.559 | |
| 2010 | 30,019 | 20.2 | 2059 | 6.9 | 0.959 | 0.901–1.022 | 0.195 | |
| 2011 | 29,885 | 20.1 | 1966 | 6.6 | 0.917 | 0.861–0.978 | 0.008 | |
| 2012 | 29,901 | 20.1 | 1936 | 6.5 | 0.902 | 0.846–0.961 | 0.002 | |
| NDD | 0–3 | 17,460 | 11.7 | 1414 | 8.1 | 1 | ||
| 4–7 | 72,890 | 49.0 | 4686 | 6.4 | 0.78 | 0.733–0.829 | <0.0001 | |
| 8–11 | 46,083 | 31.0 | 3137 | 6.8 | 0.829 | 0.766–0.885 | <0.0001 | |
| ≥12 | 12,458 | 8.4 | 904 | 7.3 | 0.888 | 0.814–0.968 | 0.007 | |
| NPD | 0–3 | 87,960 | 59.1 | 7909 | 9.0 | 1 | ||
| 4–7 | 52,485 | 35.3 | 1930 | 3.7 | 0.386 | 0.367–0.407 | <0.0001 | |
| 8–11 | 7385 | 5 | 255 | 3.5 | 0.362 | 0.319–0.411 | <0.0001 | |
| 12–15 | 935 | 0.6 | 41 | 4.4 | 0.464 | 0.339–0.635 | <0.0001 | |
| ≥16 | 126 | 0.1 | 6 | 4.8 | 0.506 | 0.223–1.149 | 0.104 | |
| COPD | No | 138,929 | 93.3 | 9355 | 6.7 | 1 | ||
| Yes | 9898 | 6.6 | 782 | 7.9 | 1.188 | 1.101–1.282 | <0.0001 | |
| Arterialhypertension | No | 52,448 | 35.2 | 4083 | 7.8 | 1 | ||
| Yes | 95,822 | 64.4 | 6007 | 6.3 | 0.792 | 0.760–0.826 | <0.0001 | |
| Diabetes | No | 101,044 | 67.9 | 7018 | 6.9 | 1 | ||
| Yes | 46,241 | 31.1 | 3035 | 6.6 | 0.941 | 0.901–0.984 | 0.007 | |
| Anaemia | No | 140,813 | 94.6 | 9470 | 6.7 | 1 | ||
| Yes | 7840 | 5.3 | 657 | 8.4 | 1.269 | 1.168–1.378 | <0.0001 | |
| Renal failure | No | 139,807 | 93.9 | 9191 | 6.6 | 1 | ||
| Yes | 9035 | 6.1 | 947 | 10.5 | 1.664 | 1.551–1.786 | <0.0001 | |
| Atrialfibrillation | No | 11,126 | 75.0 | 5746 | 5.2 | 1 | ||
| Yes | 37,050 | 25.0 | 4310 | 11.6 | 2.414 | 2.316–2.516 | <0.0001 | |
| Dyslipidaemia | No | 95,200 | 63.9 | 7891 | 8.3 | 1 | ||
| Yes | 52,857 | 35.5 | 2239 | 4.2 | 0.489 | 0.466–0.514 | <0.0001 | |
| Ischaemic heart disease | No | 140,837 | 94.6 | 9458 | 6.7 | 1 | ||
| Yes | 7181 | 4.8 | 625 | 8.7 | 1.324 | 1.217–1.441 | <0.0001 | |
| Readmission | No | 141,751 | 95.2 | 9154 | 6.5 | 1 | ||
| Yes | 7140 | 4.8 | 987 | 13.8 | 2.324 | 2.165–2.493 | <0.0001 | |
| SSPF | No | 148,231 | 99.6 | 10,057 | 6.8 | 1 | ||
| Yes | 660 | 0.4 | 84 | 12.7 | 2.004 | 1.592–2.521 | <0.0001 | |
| n | Mean | SD | Difference of the Means | 95% CI | p | |||
| Age | Survival | 138,750 | 73.34 | 12.527 | −8.872–8.374 | |||
| Death | 10,141 | 81.96 | 9.510 | −8.623 | −8.819–8.426 | <0.0001 | ||
| Length of stay | Survival | 138,750 | 7.67 | 4.501 | 1.876–2.058 | |||
| Death | 10,141 | 5.71 | 4.700 | 1.967 | 1.873–2.062 | <0.0001 | ||
| NDD | Survival | 138,750 | 6.9224 | 2.94810 | −0.011–0.108 | |||
| Death | 10,141 | 6.8735 | 3.09802 | 0.04893 | −0.013–0.111 | 0.1240 | ||
| NPD | Survival | 138,750 | 3.3534 | 2.45031 | 1.099–1.98 | |||
| Death | 10,141 | 2.2044 | 2.21442 | 1.14901 | 1.104–1.194 | <0.0001 | ||
SSPF: symptoms suggestive of posterior fossa infarction. ORu: odds ratio unadjusted; NDD: number of diagnoses on discharge; NDP: number of procedures on discharge; UA: urgent admission; SA: scheduled admission; COPD: chronic obstructive pulmonary disease; %: percentage from total sample; %*: percentage within the deceased.
Logistic regression model to predict mortality. Training cohort.
| Logistic Regression | Observations | = | 145,400 | ||
| Log likelihood = −32,531.838 | Pseudo R2 | = | 0.0975 | ||
| Death | OR | SE | P > z | 95% CI | |
| Lower | Upper | ||||
| Age | 1.069 | 0.001 | 0.000 | 1.067 | 1.072 |
| Sex | 1.202 | 0.023 | 0.000 | 1.149 | 1.257 |
| Readmission | 2.008 | 0.038 | 0.000 | 1.862 | 2.165 |
| Ischaemic heart disease | 1.342 | 0.046 | 0.000 | 1.227 | 1.467 |
| Arterial hypertension | 0.726 | 0.023 | 0.000 | 0.695 | 0.759 |
| Diabetes mellitus | 1.105 | 0.024 | 0.000 | 1.054 | 1.158 |
| Atrial fibrillation | 1.537 | 0.023 | 0.000 | 1.471 | 1.607 |
| Dyslipidaemia | 0.638 | 0.026 | 0.000 | 0.606 | 0.671 |
| Heart failure | 1.518 | 0.034 | 0.000 | 1.421 | 1.622 |
| SSPF | 2.639 | 0.124 | 0.000 | 2.071 | 3.364 |
AUC: 0.742, 95% CI: 0.737–0.747; Pearson’s χ2 test: 0.176. SSPF: symptoms suggestive of posterior fossa infarction.
Risk scores.
| Attribute | Points |
|---|---|
| Age > 76 | 16 |
| Sex (female) | 4 |
| Readmission | 9 |
| Ischaemic heart disease | 5 |
| Hypertension | −5 |
| Diabetes | 1 |
| Atrial fibrillation | 7 |
| Dyslipidaemia | −7 |
| Heart failure | 6 |
| Symptoms of posterior fossa infarction | 14 |
Results of the different metrics according to the method of analysis.
| Balanced Accuracy | AUC ROC | Precision | Recall | |
|---|---|---|---|---|
| Logistic regression | 0.667 | 0.742 | 0.320 | 0.712 |
| Random forest | 0.669 | 0.726 | 0.326 | 0.698 |
| Neural network MLP | 0.651 | 0.651 | 0.356 | 0.863 |
References
1. Brea, A.; Laclaustra, M.; Martorell, E.; Pedragosa, À. Epidemiología de la enfermedad vascular cerebral en España. Clín. Investig. Arterioscler.; 2013; 25, pp. 211-217. [DOI: https://dx.doi.org/10.1016/j.arteri.2013.10.006]
2. Feigin, V.L.; Lawes, C.M.; Bennett, D.A.; Anderson, C.S. Stroke epidemiology: A review of population-based studies of incidence, prevalence, and case-fatality in the late 20th century. Lancet Neurol.; 2003; 2, pp. 43-53. [DOI: https://dx.doi.org/10.1016/S1474-4422(03)00266-7]
3. Krishnamurthi, R.V.; Feigin, V.L.; Forouzanfar, M.H.; Mensah, G.A.; Connor, M.; Bennett, D.A.; Moran, A.E.; Sacco, R.L.; Anderson, L.M.; Truelsen, T. et al. Global and regional burden of first-ever ischaemic and haemorrhagic stroke during 1990–2010: Findings from the Global Burden of Disease Study 2010. Lancet Glob. Health; 2013; 1, pp. e259-e281. [DOI: https://dx.doi.org/10.1016/S2214-109X(13)70089-5]
4. Global, regional, and national burden of stroke and its risk factors, 1990–2019: A systematic analysis for the Global Burden of Disease Study 2019. Lancet Neurol.; 2021; 20, pp. 795-820. [DOI: https://dx.doi.org/10.1016/S1474-4422(21)00252-0]
5. Mendelson, S.J.; Prabhakaran, S. Diagnosis and management of transient ischemic attack and acute ischemic stroke: A review. JAMA; 2021; 325, pp. 1088-1098. [DOI: https://dx.doi.org/10.1001/jama.2020.26867] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33724327]
6. Aguiar de Sousa, D.; von Martial, R.; Abilleira, S.; Gattringer, T.; Kobayashi, A.; Gallofré, M.; Fazekas, F.; Szikora, I.; Feigin, V.; Caso, V. et al. Access to and delivery of acute ischaemic stroke treatments: A survey of national scientific societies and stroke experts in 44 European countries. Eur. Stroke J.; 2019; 4, pp. 13-28. [DOI: https://dx.doi.org/10.1177/2396987318786023] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31165091]
7. Campbell, B.C.V.; Khatri, P. Stroke. Lancet; 2020; 396, pp. 129-142. [DOI: https://dx.doi.org/10.1016/S0140-6736(20)31179-X]
8. Lansberg, M.G.; Schrooten, M.; Bluhmki, E.; Thijs, V.N.; Saver, J.L. Treatment time-specific number needed to treat estimates for tissue plasminogen activator therapy in acute stroke based on shifts over the entire range of the modified Rankin Scale. Stroke; 2009; 40, pp. 2079-2084. [DOI: https://dx.doi.org/10.1161/STROKEAHA.108.540708]
9. Campbell, B.C.V.; Mitchell, P.J.; Churilov, L.; Keshtkaran, M.; Hong, K.S.; Kleinig, T.J.; Dewey, H.M.; Yassi, N.; Yan, B.; Dowling, R.J. et al. Endovascular Thrombectomy for Ischemic Stroke Increases Disability-Free Survival, Quality of Life, and Life Expectancy and Reduces Cost. Front. Neurol.; 2017; 8, 657. [DOI: https://dx.doi.org/10.3389/fneur.2017.00657]
10. Ansari, J.; Triay, R.; Kandregula, S.; Adeeb, N.; Cuellar, H.; Sharma, P. Endovascular Intervention in Acute Ischemic Stroke: History and Evolution. Biomedicines; 2022; 10, 418. [DOI: https://dx.doi.org/10.3390/biomedicines10020418]
11. Arboix, A.; García-Eroles, L.; Comes, E.; Oliveres, M.; Targa, C.; Balcells, M.; Pujadas, R.; Massons, J. Importancia del perfil cardiovascular en la mortalidad hospitalaria de los infartos cerebrales. Rev. Esp. Cardiol.; 2008; 61, pp. 1020-1029. [DOI: https://dx.doi.org/10.1157/13126042] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/18817678]
12. Boehme, A.K.; Esenwa, C.; Elkind, M.S. Stroke Risk Factors, Genetics, and Prevention. Circ. Res.; 2017; 120, pp. 472-495. [DOI: https://dx.doi.org/10.1161/CIRCRESAHA.116.308398] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28154098]
13. Smith, E.E.; Shobha, N.; Dai, D.; Olson, D.M.; Reeves, M.J.; Saver, J.L.; Hernandez, A.F.; Peterson, E.D.; Fonarow, G.C.; Schwamm, L.H. A risk score for in-hospital death in patients admitted with ischemic or hemorrhagic stroke. J. Am. Heart Assoc.; 2013; 2, e005207. [DOI: https://dx.doi.org/10.1161/JAHA.112.005207] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23525444]
14. Whisnant, J.P. Modeling of risk factors for ischemic stroke. The Willis Lecture. Stroke; 1997; 28, pp. 1840-1844. [DOI: https://dx.doi.org/10.1161/01.STR.28.9.1840] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/9303034]
15. Ho, W.M.; Lin, J.R.; Wang, H.H.; Liou, C.W.; Chang, K.C.; Lee, J.D.; Peng, T.Y.; Yang, J.T.; Chang, Y.J.; Chang, C.H. et al. Prediction of in-hospital stroke mortality in critical care unit. SpringerPlus; 2016; 5, 1051. [DOI: https://dx.doi.org/10.1186/s40064-016-2687-2]
16. Gattringer, T.; Posekany, A.; Niederkorn, K.; Knoflach, M.; Poltrum, B.; Mutzenbach, S.; Haring, H.P.; Ferrari, J.; Lang, W.; Willeit, J. et al. Predicting Early Mortality of Acute Ischemic Stroke: Score-based approach. Stroke; 2019; 50, pp. 349-356. [DOI: https://dx.doi.org/10.1161/STROKEAHA.118.022863]
17. Baptista, M.V.; van Melle, G.; Bogousslavsky, J. Prediction of in-hospital mortality after first-ever stroke: The Lausanne Stroke Registry. J. Neurol. Sci.; 1999; 166, pp. 107-114. [DOI: https://dx.doi.org/10.1016/S0022-510X(99)00117-3]
18. Lee, J.; Morishima, T.; Kunisawa, S.; Sasaki, N.; Otsubo, T.; Ikai, H.; Imanaka, Y. Derivation and validation of in-hospital mortality prediction models in ischaemic stroke patients using administrative data. Cerebrovasc. Dis.; 2013; 35, pp. 73-80. [DOI: https://dx.doi.org/10.1159/000346090]
19. Yu, A.Y.X.; Austin, P.C.; Rashid, M.; Fang, J.; Porter, J.; Hill, M.D.; Kapral, M.K. Deriving a Passive Surveillance Stroke Severity Indicator From Routinely Collected Administrative Data: The PaSSV Indicator. Circ. Cardiovasc. Qual. Outcomes; 2020; 13, e006269. [DOI: https://dx.doi.org/10.1161/CIRCOUTCOMES.119.006269]
20. Cheon, S.; Kim, J.; Lim, J. The Use of Deep Learning to Predict Stroke Patient Mortality. Int. J. Environ. Res. Public Health; 2019; 16, 1876. [DOI: https://dx.doi.org/10.3390/ijerph16111876]
21. Guo, C.-Y.; Chan, C.-H.; Chou, Y.-C.; Sung, S.-H.; Cheng, H.-M. A statistical predictive model consistent within a 5-year follow-up period for patients with acute heart failure. J. Chin. Med. Assoc.; 2020; 83, pp. 1008-1013. [DOI: https://dx.doi.org/10.1097/JCMA.0000000000000403] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32773590]
22. Viton, F.; Elbbatah, M.; Guérin, J.; Dequen, G. Multi-channel ConvNet Approach to Predict the Risk of in-Hospital Mortality for ICU Patients. Proceedings of the 1st International Conference on Deep Learning Theory and Applications—DeLTA 2020; Paris, France, 8–10 July 2020; pp. 98-102.
23. Hosmer, D.W.; Hjort, N.L. Goodness-of-fit processes for logistic regression: Simulation results. Stat. Med.; 2002; 21, pp. 2723-2738. [DOI: https://dx.doi.org/10.1002/sim.1200] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/12228887]
24. DeLong, E.R.; DeLong, D.M.; Clarke-Pearson, D.L. Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics; 1988; 44, pp. 837-845. [DOI: https://dx.doi.org/10.2307/2531595] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/3203132]
25. Chaudhary, P. Ionic Framework. Int. Res. J. Eng. Technol.; 2018; 5, pp. 3181-3185.
26. Sullivan, L.M.; Massaro, J.M.; D’Agostino, R.B., Sr. Presentation of multivariate data for clinical use: The Framingham Study risk score functions. Stat. Med.; 2004; 23, pp. 1631-1660. [DOI: https://dx.doi.org/10.1002/sim.1742]
27. Qi, Y. Random Forest for Bioinformatics. Ensemble Machine Learning: Methods and Applications; Zhang, C.; Ma, Y. Springer: Boston, MA, USA, 2012; pp. 307-323.
28. Gagnon-Bartsch, J.; Shem-Tov, Y. The classification permutation test: A flexible approach to testing for covariate imbalance in observational studies. Ann. Appl. Stat.; 2019; 13, pp. 1464-1483. [DOI: https://dx.doi.org/10.1214/19-AOAS1241]
29. de Andrade, B.M.; de Gois, J.S.; Xavier, V.L.; Luna, A.S. Comparison of the performance of multiclass classifiers in chemical data: Addressing the problem of overfitting with the permutation test. Chemom. Intell. Lab. Syst.; 2020; 201, 104013. [DOI: https://dx.doi.org/10.1016/j.chemolab.2020.104013]
30. Windmeijer, F.A.G. The asymptotic distribution of the sum of weighted squared residuals in binary choice models. Stat. Neerl.; 1990; 44, pp. 69-78. [DOI: https://dx.doi.org/10.1111/j.1467-9574.1990.tb01527.x]
31. Calculadora Ictus. Modelo Predictivo Para la Mortalidad Hospitalaria en el Ictus Isquémico No Lisado. Spain. Available online: https://calculadora-ictus.firebaseapp.com/ (accessed on 1 October 2020).
32. Fernandez-Lozano, C.; Hervella, P.; Mato-Abad, V.; Rodríguez-Yáñez, M.; Suárez-Garaboa, S.; López-Dequidt, I.; Estany-Gestal, A.; Sobrino, T.; Campos, F.; Castillo, J. et al. Random forest-based prediction of stroke outcome. Sci. Rep.; 2021; 11, 10071. [DOI: https://dx.doi.org/10.1038/s41598-021-89434-7]
33. Gebreyohannes, E.A.; Bhagavathula, A.S.; Abebe, T.B.; Seid, M.A.; Haile, K.T. In-Hospital Mortality among Ischemic Stroke Patients in Gondar University Hospital: A Retrospective Cohort Study. Stroke Res. Treat.; 2019; 2019, 7275063. [DOI: https://dx.doi.org/10.1155/2019/7275063]
34. Ntaios, G.; Georgiopoulos, G.; Koroboki, E.; Vemmos, K. External Validation of the PREMISE Score in the Athens Stroke Registry. J. Stroke Cerebrovasc. Dis.; 2019; 28, pp. 1806-1809. [DOI: https://dx.doi.org/10.1016/j.jstrokecerebrovasdis.2019.04.023]
35. LaBresh, K.A.; Reeves, M.J.; Frankel, M.R.; Albright, D.; Schwamm, L.H. Hospital treatment of patients with ischemic stroke or transient ischemic attack using the “Get With The Guidelines” program. Arch. Intern. Med.; 2008; 168, pp. 411-417. [DOI: https://dx.doi.org/10.1001/archinternmed.2007.101] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/18299497]
36. Howard, G.; Schwamm, L.H.; Donnelly, J.P.; Howard, V.J.; Jasne, A.; Smith, E.E.; Rhodes, J.D.; Kissela, B.M.; Fonarow, G.C.; Kleindorfer, D.O. et al. Participation in Get with The Guidelines-Stroke and Its Association with Quality of Care for Stroke. JAMA Neurol.; 2018; 75, pp. 1331-1337. [DOI: https://dx.doi.org/10.1001/jamaneurol.2018.2101] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30083763]
37. Patel, P.D.; Salwi, S.; Liles, C.; Mistry, A.M.; Mistry, E.A.; Fusco, M.R.; Chitale, R.V.; Shannon, C.N. Creation and Validation of a Stroke Scale to Increase Utility of National Inpatient Sample Administrative Data for Clinical Stroke Research. J. Stroke Cerebrovasc. Dis.; 2021; 30, 105658. [DOI: https://dx.doi.org/10.1016/j.jstrokecerebrovasdis.2021.105658]
38. de la Fuente, J.; García-Torrecillas, J.M.; Solinas, G.; Iglesias-Espinosa, M.M.; Garzón-Umerenkova, A.; Fiz-Pérez, J. Structural Equation Model (SEM) of Stroke Mortality in Spanish Inpatient Hospital Settings: The Role of Individual and Contextual Factors. Front. Neurol.; 2019; 10, 498. [DOI: https://dx.doi.org/10.3389/fneur.2019.00498] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31156536]
39. Aylin, P.; Bottle, A.; Majeed, A. Use of administrative data or clinical databases as predictors of risk of death in hospital: Comparison of models. BMJ; 2007; 334, 1044. [DOI: https://dx.doi.org/10.1136/bmj.39168.496366.55] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/17452389]
40. Librero, J.; Peiro, S. Do chronic diseases prevent intra-hospital mortality? Paradoxes and biases in information about hospital morbidity. Gac. Sanit.; 1998; 12, pp. 199-206. [DOI: https://dx.doi.org/10.1016/S0213-9111(98)76473-8]
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; María del Mar Rodríguez Salvador 3 ; Galván-Espinosa, José 4
; Téllez-Ramírez, Luis 5 ; Reche-Lorite, Fernando 6 ; María-José Sánchez 7
; García-Torrecillas, Juan Manuel 8
1 Internal Medicine Department, Hospital de Poniente, El Ejido, 04700 Almería, Spain;
2 Department of Neurology and Stroke Unit, Hospital Universitario Torrecárdenas, 04009 Almería, Spain;
3 Nurse in Almería Primary Care District, 04009 Almería, Spain;
4 Alejandro Otero Research Foundation (FIBAO), Hospital Universitario Torrecárdenas, 04009 Almería, Spain;
5 Biomedical Research Unit, Hospital Universitario Torrecárdenas, 04009 Almería, Spain;
6 Department of Mathematics, University of Almería, 04120 Almería, Spain;
7 Escuela Andaluza de Salud Pública, 18011 Granada, Spain;
8 Biomedical Research Unit, Hospital Universitario Torrecárdenas, 04009 Almería, Spain;