Determinants of healthcare worker turnover in intensive care units: A micro-macro multilevel analysis

Abstract

Burnout, work-life imbalance, moral distress, depression symptoms and workplace violence and bullying appeared to lead to higher nurse turnover intention and had an impact on nurse quality of life [12,17–25]. [...]social support and job satisfaction were mediators between burnout and turnover intention [21,26]. [...]we describe the multilevel regression model for modeling a quantitative group-level outcome (the turnover rate) using independent variables at both individual- and group- level, and validate our approach using a simulation study. Participants were guaranteed confidentiality and anonymity of responses. Because the study was strictly observational, did not involve medical records or biological samples, and was based on the collection of anonymous data on a voluntary basis, no further approval from an ethics committee was required by the French legislation at the time. [...]fatigue was evaluated on three measures: the Nottingham Health Profile perceived health status relative to sleep (NHP-S; coded yes or no) and to energy level (NHP-E; coded yes or no), and the current fatigue state (good or bad) of the subject at the time of the interview.

About the Authors:

Oumou Salama Daouda

Roles Formal analysis, Methodology, Software, Validation, Visualization, Writing – original draft

* E-mail: [email protected]

Affiliation: Modélisation Epidémiologie et Surveillance des Risques Sanitaires (MESuRS) Laboratory, Conservatoire National des Arts et Métiers (Cnam), Paris, France

ORCID logo https://orcid.org/0000-0001-9682-1599

Mounia N. Hocine

Roles Methodology, Supervision, Visualization, Writing – review & editing

Affiliation: Modélisation Epidémiologie et Surveillance des Risques Sanitaires (MESuRS) Laboratory, Conservatoire National des Arts et Métiers (Cnam), Paris, France

ORCID logo https://orcid.org/0000-0003-0942-7441

Laura Temime

Roles Conceptualization, Methodology, Supervision, Visualization, Writing – review & editing

Affiliation: Modélisation Epidémiologie et Surveillance des Risques Sanitaires (MESuRS) Laboratory, Conservatoire National des Arts et Métiers (Cnam), Paris, France

Introduction

High staff turnover is currently observed in hospitals worldwide. Among nurses in particular, turnover rates have been reported to be around 20% per year, and may reach as high as 40% per year in some countries [1–3]. In a French study conducted in intensive-care units (ICUs), the yearly turnover rate was estimated at 24% among nurses and 13% among auxiliary nurses [4].

Several studies suggest that turnover is related to staff and patient outcomes. Indeed, high turnover in a hospital ward may lead to increased overtime, fatigue and stress, as well as low job satisfaction, among the remaining staff [5]. It also disrupts continuity of care, leading to decreased quality and safety of care, with potentially increased rates of medication errors, falls or other nurse-sensitive outcomes including healthcare-associated infections [5]. All in all, = this is a major concern for hospital managers, as high turnover negatively impacts hospital budgets. Indeed, this leads to substantial direct and indirect costs, and varies between courtries. Using the original Nursing Turnover Cost Calculation Methodology, yearly costs associated with turnover were estimated at $48,790 in Australia, $20,561 in the US, $26,652 in Canada and $23,711 in New Zealand [1,3]. In addition, a significant proportion of turnover costs are attributed to replacement hiring and training costs, highlighting the importance of nurse retention [1]. Hence, better understanding turnover determinants in order to implement efficient nurse retaining policies is very important.

A great deal of research has been conducted to highlight the factors contributing to the nursing shortage and voluntary turnover issues. Nurse stress and fatigue levels, as well as emotional exaustion, understaffing and poor patient safety were found to be predictors of nurses intention to leave [6–14]. In addition, several studies in different countries highlight the importance of job satisfaction and favorable work environment, with ineffective working relationships with other nurses and physicians and lack of supervisory support increasing turnover intention [7,9,11,13–18]. Burnout, work-life imbalance, moral distress, depression symptoms and workplace violence and bullying appeared to lead to higher nurse turnover intention and had an impact on nurse quality of life [12,17–25]. Moreover, social support and job satisfaction were mediators between burnout and turnover intention [21,26]. Other factors such as temporal, physical, emotional and mental workloads, salaries, night shift, rewards/recognition, job strain, job control, job complexity, advancement opportunities and the ideas of feeling valued, respected and acknowledged were also found significantly associated with turnover intention [6–8,14,16,21,24,25,27,28]. However, a better and more global understanding of how individual-level and organizational-level factors interact with turnover in hospitals is still needed, in particular in the French context where studies on hospital staff turnover are scarce [29,30].

In this context, the present work aimed at developing a predictive statistical model of the turnover rate among registered nurses and auxiliary nurses in ICUs, based on French data collected by our group [4]. The turnover rate is a quantitative outcome defined at the ICU-level, which we wished to predict using both individual-level predictors (age, number of children, stress level, etc.) and ICU-level predictors (ICU size, shiftwork organization, etc.). This requires using a multi-level modeling approach [31] specifically designed to model a group-level outcome from individual- and group-level predictors. Here, we propose an extension of the approach proposed by Croon and van Veldhoven [32] to deal with such “micro-macro” problems.

The organization of the article is as follows. First, we present the data collected in French ICUs. Second, we describe the multilevel regression model for modeling a quantitative group-level outcome (the turnover rate) using independent variables at both individual- and group- level, and validate our approach using a simulation study. Finally, we present and discuss the results obtained using the data collected in the French ICUs.

Materials and methods

Study data

Study design and population.

A cross-sectional study was conducted in 30 ICUs of Paris-area hospitals of the Assistance Publique des Hôpitaux de Paris (AP-HP) between January 18, 2013 and April 2, 2013 [4]. The study population consisted of healthcare workers with patient contacts (registered nurses, auxiliary nurses, physical therapists, doctors and residents) working in the ICUs. All types of adult ICUs were included in the study (medical, surgical and polyvalent). Study data were collected using two questionnaires: a general questionnaire regarding the organizational structure of the service (completed with the nursing manager) and an individual questionnaire administered face-to-face with personnel during on-site visits to each ICU. In total, 672 healthcare workers with patient contacts (physicians, residents, registered nurses, auxiliary nurses and physical therapists) participated to the study, including 526 registered and auxiliary nurses (RANs) [4].However, here, analyses were conducted on the data obtained from RANs only as this group of healthcare workers is subject to specific organizational characteristics, not generalized across all professions.

The turnover rate, the outcome of interest, was calculated for each ICU for the year 2012 as the proportion of employees who voluntarily or involuntarily left their jobs within the ICU during that year. This outcome was computed jointly for all RANs in each ICU.

Ethical approval and inform consent.

The study protocol was elaborated in collaboration with the AP-HP Department of medical policy and the Department of care and of paramedical activities, and was approved after presentation to the Directorate General and the Committee on hygiene, safety and working conditions. Potential participants were informed of the study by a letter of information that was sent to the head doctor and nursing manager for distribution and/or posting in the unit. Verbal consent was obtained by the interviewer at the beginning of each interview. Participants were guaranteed confidentiality and anonymity of responses. Because the study was strictly observational, did not involve medical records or biological samples, and was based on the collection of anonymous data on a voluntary basis, no further approval from an ethics committee was required by the French legislation at the time.

Individual-level factors.

In the study questionnaire, the collected demographic characteristics of participants included age, gender, marital status, number of children, professional category and total commuting time to and from work. The questionnaire also included individual work-related characteristics such as experience duration in the profession and in the current position (years), perceived stress level and assessment of social support from supervisor and colleagues. Perceived stress was measured using the Perceived Stress Scale 10-item scale (PSS10) developed by Cohen and al. [33] and validated in French [34]. Social support was assessed using the Job Content Questionnaire, which measures the support of both colleagues and supervisor, developed and validated by Karasek et al. [35], and validated in French [36]. Further data relative to recent work history (shift assignment in the previous 30 days (day or night), frequency of schedule changes and overtime hours (both factors having four ordinal response categories, ranging from “never” (0) to “very often” (3)) and workload (average number of breaks, skipping a break in the previous three shifts due to workload (yes or no), and working quota) were also collected. Finally, fatigue was evaluated on three measures: the Nottingham Health Profile perceived health status relative to sleep (NHP-S; coded yes or no) and to energy level (NHP-E; coded yes or no), and the current fatigue state (good or bad) of the subject at the time of the interview. The NHP questionnaire was developped by Hunt et al. [37] and the French version was validated by Bucquet and Condon [38].

Group-level factors.

ICU-level data included information on both the physical organization of the ward (number of beds, type of ICU (medical, surgical and polyvalent) and the staff-to-patient ratio computed jointly for all RANs, during days, nights, and overall) and the work organization of the staff. The latter included the type of shift work (two 12-hour shifts or three 8-hour shifts) and information on whether intermediate care beds were included within the ICU.

Statistical modeling

General micro-macro model structure.

To predict the turnover rate Y, a quantitative group-level variable, from both individual- and ICU-level factors, we used a 2-level micro-macro model. Let G be the total number of ICUs in which the data were collected and ng the number of individuals (RANs) within the gth ICU. Let Y be the turnover, with score yg for ICU g.

For simplicity reasons, we first provide here the model adjusting for a single individual-level covariate (“micro” level) and a single ICU-level covariate (“macro” level). We thus want to predict Y by one ICU-level explanatory variable Z with score zg for group g and one individual-level explanatory variable X with score xig for individual i in group g. In the approach proposed by Croon and van Veldhoven [32], a latent ICU-level variable ξ (with score ξg within ICU g) is associated with the individual-level variable X. Hence, the scores xig are viewed as indicators of the unobserved ICU scores ξg. The relationship between ICU-level variables ξ and Z and the outcome Y may be described using a linear regression model given by the following equation:(1)

The error variable ε is assumed to be homoscedastic, that is, to have constant variance for all groups. Additionally, the relationship between the ICU-level variable ξ and the individual-level variable X may be specified by the following equation:(2)where υig is a disturbance term, which is assumed to have a constant variance for all subjects and groups and to be independent from εg and from ξg.

Model estimation with quantitative predictors.

In the case of quantitative predictors, a naive approach would be to aggregate the individual-level variable xig to the ICU level by computing its ICU-level mean and then get a simple estimate of the parameters βj based on an ordinary least square regression analysis of the regression of yg on zg and . However, this estimate is biased. The method proposed by Croon & van Veldhoven [32] consists in correcting this biased estimation by using an adjusted ICU mean instead of . This adjusted mean may be interpreted as the expected value of ξg, taking all the observed scores on the individual- and ICU-level explanatory variables in ICU g into account. The exact formula for is available in the original article of Croon and van Veldhoven [32].

The same principle for adjusting the observed ICU means before including them in a regression model applies if several different quantitative explanatory variables at the individual and ICU levels are available, as shown in the original article of Croon and van Veldhoven [32].

Correction for heteroscedasticity.

As ICU size was not constant, the final regression model was not homoscedastic. As in the original article of Croon and van Veldhoven [32], we corrected this by determining the heteroscedasticity-consistent covariance matrix estimator.

Extension to qualitative and quantitative predictors.

In our case, there is a mixture in the documented covariates; some are quantitative and others qualitative. This requires an extension of the estimation method proposed by Croon and van Veldhoven for quantitative predictors only. To that aim, we transformed the qualitative variables into quantitative variables usable in the context of regression analysis, using dummy variables. For binary variables, a single dummy variable was used. For qualitative variables with n modalities, (n−1) dummy variables were defined. The original approach proposed by Croon and van Veldhoven was then applied.

Variable selection procedure.

Variable selection was performed in two stages. In a first step, we performed a feature selection using clustering of all variables (quantitative and qualitative), separately for individual-level and ICU-level factors. The ascendant hierarchical clustering algorithm proposed in the R package ClustOfVar was used and a bootstrap approach was performed to determine the appropriate number of clusters [39]. Variables that were strongly correlated with each other were allocated to the same cluster. For each cluster, a single variable was selected based on its pertinence as a potential action lever.

In a second step, variable selection within the model was performed after computing the adjusted mean of each individual-level factor selected in the previous step. The selection was performed using regression subset selection in R applying the regsubsets function [40] based on Akaike’s Information Criterion (AIC) [41]. This function performs an exhaustive search of the best subsets of the variables predicting the outcome of interest.

Simulation study

We carried out a simulation study to validate our extension of the method proposed by Croon and van Veldhoven [32] to the situation where both qualitative and quantitative predictors are used. The objective was to estimate the accuracy of the estimation of the coefficients in the adjusted regression analysis and to examine how this accuracy was affected by the number of ICUs, the ICU size, the intraclass correlation between individual-level explanatory variable and the correlation between the explanatory variables at the ICU and individual level. A detailed presentation of all simulation scenarios is presented in S1 Appendix. All analyses were performed using R version 3.6.0 [42].

Results

Simulation study

All simulation results are provided in S1 Appendix. The results from the adjusted regression showed lower estimation biases, irrespective of the explored simulation scenario. As expected, the benefit of using the adjusted regression rather than the unadjusted one was more important for coefficients that were associated with individual-level predictors. The approach we proposed allows to estimate the regression coefficients with a relative bias below 7% for group-level factors and below 12% for individual-level factors.

Application to healthcare workers’ turnover

The analyzed data comprised 526 RANs, including 325 registered nurses and 201 auxiliary nurses, in a total of 30 distinct ICUs. The number of RANs per ICU differed, ranging from 7 to 32, with a mean of 17.5. The collected data on the 26 explanatory variables and the turnover is provided for both registered and auxiliary nurses in S1 (individual-level) and S2 (ICU-level) Tables. Among the 19 individual-level explanatory variables, eight of them showed a signficance difference between registered and auxiliary nurses. At group-level, staff-to-patient ratio and turnover rate revealed no significant difference between the two professions. The mean observed value of the turnover rate for RANs is equal to 0.19 (SD = 0.09).

The results of the feature selection (ascendant hierarchical clustering) first applied to reduce the dimension of the dataset are provided in Figs 1 and 2. The results from the bootstrap approach used to determine the stability of the partitions and therefore the number of clusters to consider are provided in S1 and S2 Figs. Sixteen clusters were identified for individual-level variables (Fig 1). The chosen variables for each cluster were: experience in profession, working quota, number of children, NHP_E, impossibility to skip a break, stress level, NHP_S, support from supervisors, overtime hours, schedules changes, commuting duration, support from colleagues, number of breaks, gender, shift assignment in the previous month and profession. Five clusters were identified for ICU-level variables (Fig 2); the chosen variables for each cluster were: type of ICU, number of beds, presence of intermediate care bed, shift work organization and staff-to patient ratio overall.

[Figure omitted. See PDF.]

Fig 1. Dendrogram of the hierarchy of the 20 variables at individual-level.

Dotted line depicts the best cut-off of clusters according to the bootstrap approach.

https://doi.org/10.1371/journal.pone.0251779.g001

[Figure omitted. See PDF.]

Fig 2. Dendrogram of the hierarchy of the 7 variables at ICU-level.

Dotted line depicts the best cut-off of clusters according to the bootstrap approach.

https://doi.org/10.1371/journal.pone.0251779.g002

Table 1 provides the results of the adjusted regression analysis to predict the turnover, based on the best model obtained after selection. This best model according to AIC included 6 individual-level variables and 4 ICU-level variables.

[Figure omitted. See PDF.]

Table 1. Results of the adjusted regression analyses of turnover on individual- and ICU-level factors.

https://doi.org/10.1371/journal.pone.0251779.t001

At the individual level, high social support from colleagues (β = -0.09, 95% confidence interval [CI] from -0.15 to -0.02) and supervisors (β = -0.02, 95%CI from -0.04 to -0.006]), as well as long durations of experience in the profession (β = -0.02, 95%CI from -0.03 to -0.01]), were significantly associated with lower rates of turnover. Conversely, number of children (β = 0.09, 95%CI from 0.03 to 0.15) and impossibility to skip a break due to workload (β = 0.21, 95%CI from 0.01 to 0.40])were associated with higher rates of turnover. At the ICU level, turnover was significantly associated with the number of beds (β = -0.01, 95%CI from -0.02 to -0.006, with lower turnover rates in large ICUs) and the type of ICU (β = -0.07, 95%CI from -0.15 to -0.005), lower turnover in polyvalent ICU vs. medical ICU). Higher turnover was also observed in ICUs where the staff-to-patient ratio was high (β = 0.07, 95%CI from 0.006 to 0.13), as well as in ICUs with intermediate care bed (β = 0.12, 95%CI from 0.04 to 0.19). ICCs of all individual-level factors retained in the model were significantly different from zero (see in S3 Table). In addition S3 Table provides standard error (SE) and the corrected standard error (SEcorr) for all covariates. The squared multiple correlation coefficient was 0.58 means that our model explains about 58% of the variation of healthcare workers’ turnover.

Discussion

While multilevel analysis has become increasingly popular over the last decades, most published studies have focused on modeling individual-level outcomes using individual- and group-level predictors. In this current article, we were in the opposite situation, called micro-macro, where the outcome is a group-level rather than individual level variable [32]. Micro-macro relations are present in a broad variety of researcher questions: in medical, social and behavioral sciences. Unfortunately, specific methods developed to deal with this kind of data are still very limited. Here, we proposed an extension of the Croon and van Veldhoven approach to group-level (macro) outcome, with a specific application to predict healthcare workers’ turnover. Based on the simulation study we performed, this extension allows to significantly reduce the bias in parameter estimation for micro-macro situations.

As suggested by Croon and van Veldhoven [32], micro-macro analyses “will yield stable and interpretable results only if […] the group means [of individual predictors] show sufficient variation that is not entirely due to within-group variation”. This theory was verified for each individual-level predictor selected in our model. For assessing the between ICU-level units variation for a micro variable, we fit a multilevel model with no explanatory variables but random effects for the macro unit (ICU level). We tested the effect of the variance of the random effect using the RLRsim package, which has a fast implementation of simulation-based tests of null hypotheses about zero variance [43,44]. For each individual-level predictor, the random effect was significant meaning that there is a variation between ICU-level units in the distribution of the individual-level predictors.

Using our approach, we identified some individual- and group-level factors determinants of RAN turnover. In particular, we found evidence that social support (from colleagues and supervisors) played a significant role on RAN turnover rate. In addition, number of children, impossibility to skip a break and long durations of experience in profession were also associated with lower RAN turnover. The latter could reflect the fact that experienced RANs are more resistant to change. All in all, our results highlight four individual and ICU-level factors less or not really present in the literature as associated to nurses turnover: the number of children, the impossibility to skip a break due to workload, the fact that the ICU is polyvalent (combining medical and surgical activities), and the presence of intermediate care beds within the ICU.

Although increasing recruitment of nurses and raising compensation may help reduce the short-term impact of turnover, researchers suggest that administrative interventions to improve the work life of RANs are more effective in the long term. Our work could provide insight into levers to improve this work life.

Moreover, Our findings are consistent with most published studies dealing with healthcare worker turnover, and in critical care in particular. Indeed, the literature highlights the importance of psychological work environment in predicting turnover [27,45,46]. Lack of social support among nurses and phycisians has been shown to be a strong predictor of turnover [7,13,16,26,47]. At the same time, low managerial support appears to favor turnover [9,14,47]. In addition, recent studies demonstrate that turnover intention is negatively correlated with years of nursing experience and years in the current position [11,48]. Job dissatisfaction or satisfaction has also frequently been reported as a determinant of intention to leave or to stay [11,13,14,21,22,27,28]. Indeed, job satisfaction of hospital nurses is closely related to work environment, structural empowerment, organizational commitment, professional commitment, job stress, patient satisfaction and patient-nurse ratios. Moreover, nurses’ health and well-being also influence experienced nurses’ decisions to leave practices [7]. In particular, even though workplace violence is associated to a higher rate of turnover intention, subjective well-being has been shown to moderate the relationship between workplace violence and nurses’ turnover, and the relationship between workplace violence and job satisfaction [26]. This moderating effect of subjective well-being has proved helpful in reducing the harm of workplace violence to nurses and in decreasing their turnover intention. Finally, it appears that nurses working conditions is associated to intention to leave, but the strength of this relationship is slightly mediated by nurse’s mental health [49].

Methodological benefits of this work

The simulation study reported here yielded promising findings for researchers in different areas faced with micro-macro data. Micro-macro analysis is indeed currently of great interest in a wide array of fields [50,51]. Several extensions of the micro-macro method proposed by Croon and van Veldhoven had been proposed before [52–54]; for example, in 2013, Bennink and al. [53] developed the case when discrete individual-level variables are used as predictors of discrete group-level outcomes. However, none of the published extensions had treated the case where there is a mixture of quantitative and qualitative predictors whether at the group or individual levels, with a quantitative group-level outcome.

Limitations and directions for future research

The goal of this work was double: (a) to extend the approach proposed by Croon and van Veldhoven, by allowing for qualitative individual- and group- level predictors; and (b) to apply the method to data on healthcare worker turnover. Based on our results, these goals were attained. However, it is important to acknowledge that there are some limitations in this study, whether in the developed method or in the data used.

First, the simulation study that demonstrated the performance of our methodological approach was based on the simplified situation where only 4 variables were included in the model. It could be interesting to assess the estimation biases obtained with a model including more than 4 variables with various distributions.

Second, the extension proposed here is applied to a real dataset in which RANs turnover (the group-level outcome) is explained by several group- and individual-level predictors. One of the limitations of this data is that the number of ICUs and the number of observations per ICU available for analysis were relatively small. It could be interesting to replicate the study with larger ICU-size and larger sample to improve the power for the study and to decrease coefficient biases.

Third, because of the small number of observations in the sample, we aggregated registered and auxiliary nurses in the analysis in order to achieve an adequate power for the study. However, results of separate analyses could be interesting to perform.

Fourth, our findings should be interpreted with caution when attempting to identify efficient actions to reduce turnover. Indeed, due to the cross-sectional nature of the study we used, these findings should be interpreted as reflecting statistical associations, rather than causal relationships [55]. Further longitudinal research is needed to prove relationships between turnover and these factors.

A final limitation of the study concerns the age of the data we used. Indeed, data were collected in 2013; unfortunately, we did not have access to more recent data on nurse turnover in French ICUs that would also document individual and organizational factors, as our data does. However, many of the factors we found associated with nurse turnover are consistent with the recent international literature. In addition, these factors (e.g. social support or having to skip breaks) remain pertinent to this day, actually even more so in the current context of the Covid-19 pandemic which has been shown to impact the careload, as well as the physical and mental health of healthcare workers [56]. However, the level of knowledge on turnover within hospitals would benefit from the collection of new longitudinal data.

Conclusion

In this study, through an extension of the method initially proposed by Croon and van Veldhoven [32], we proposed an innovative methodological framework to identify the main individual- and group-level determinants of a group-level outcome. This framework could be applied to other issues within hospital wards, such as absenteeism, hand hygiene compliance or blood exposure incident rates.

In addition, our results could help hospital decision makers facing high staff turnover, by informing them on the main factors on which they may act to try and reduce turnover intention among the staff. In particular, two axes of interventions come out of our analysis: first, ensuring that hospital staff never have to skip breaks; and second, enhancing social support among the staff. The former possibly requires increasing the staff-to-patient ratio. The latter may notably be achieved by implementing solutions that enable healthcare workers to effectively and collaboratively work together and facilitate social communication and interaction.

Supporting information

S1 Fig. Cluster partition stability at individual-level.

https://doi.org/10.1371/journal.pone.0251779.s001

(TIFF)

S2 Fig. Cluster partition stability at intensive care unit-level.

https://doi.org/10.1371/journal.pone.0251779.s002

(TIFF)

S1 Table. Description of individual-level factors (mean ± SD for quantitative factors and n(%) for qualitative factors) for registered and auxiliary nurses.

https://doi.org/10.1371/journal.pone.0251779.s003

(PDF)

S2 Table. Description of intensive care unit-level factors (mean ± SD for quantitative factors and n(%) for qualitative factors) for registered and auxiliary nurses.

https://doi.org/10.1371/journal.pone.0251779.s004

(PDF)

S3 Table. Additional results of the adjusted regression analyses of turnover on individual- and intensive care unit-level factors.

https://doi.org/10.1371/journal.pone.0251779.s005

(PDF)

S1 Appendix. Study and results of the simulation scenarios.

https://doi.org/10.1371/journal.pone.0251779.s006

(PDF)

S2 Appendix. Dictionary of categorical variables.

https://doi.org/10.1371/journal.pone.0251779.s007

(PDF)

S1 File. Original (French) version of the questionnaire.

https://doi.org/10.1371/journal.pone.0251779.s008

(PDF)

S2 File. English version of the questionnaire.

https://doi.org/10.1371/journal.pone.0251779.s009

(PDF)

S1 Data. Individual-level factors data.

https://doi.org/10.1371/journal.pone.0251779.s010

(CSV)

S2 Data. Intensive care unit-level factors data.

https://doi.org/10.1371/journal.pone.0251779.s011

(CSV)

Acknowledgments

The authors would like to thank the AP-HP and all participating ICUs and healthcare workers for their time and their contribution to this study.

Citation: Daouda OS, Hocine MN, Temime L (2021) Determinants of healthcare worker turnover in intensive care units: A micro-macro multilevel analysis. PLoS ONE 16(5): e0251779. https://doi.org/10.1371/journal.pone.0251779

References

1. Duffield CM, Roche MA, Homer C, Buchan J, Dimitrelis S. A comparative review of nurse turnover rates and costs across countries. J Adv Nurs. 2014;70: 2703–2712. pmid:25052582

2. Kim Y, Han K. Longitudinal associations of nursing staff turnover with patient outcomes in long-term care hospitals in Korea. Journal of Nursing Management. 2018;26: 518–524. pmid:29318685

3. Roche MA, Duffield CM, Homer C, Buchan J, Dimitrelis S. The rate and cost of nurse turnover in Australia. Collegian. 2015;22: 353–358. pmid:26775521

4. Jones G, Hocine M, Salomon J, Dab W, Temime L. Demographic and occupational predictors of stress and fatigue in French intensive-care registered nurses and nurses’ aides: A cross-sectional study. International Journal of Nursing Studies. 2015;52: 250–259. pmid:25443305

5. Hayes LJ, O’Brien-Pallas L, Duffield C, Shamian J, Buchan J, Hughes F, et al. Nurse turnover: A literature review–An update. International Journal of Nursing Studies. 2012;49: 887–905. pmid:22019402

6. Brunetto Y, Rodwell J, Shacklock K, Farr-Wharton R, Demir D. The impact of individual and organizational resources on nurse outcomes and intent to quit. J Adv Nurs. 2016;72: 3093–3103. pmid:27461898

7. Hayward D, Bungay V, Wolff AC, MacDonald V. A qualitative study of experienced nurses’ voluntary turnover: learning from their perspectives. J Clin Nurs. 2016;25: 1336–1345. pmid:26990238

8. Mazurenko O, Gupte G, Shan G. Analyzing U.S. nurse turnover: Are nurses leaving their jobs or the profession itself? Journal of Hospital Administration. 2015;4: 48.

9. Halter M, Boiko O, Pelone F, Beighton C, Harris R, Gale J, et al. The determinants and consequences of adult nursing staff turnover: a systematic review of systematic reviews. BMC Health Serv Res. 2017;17: 824. pmid:29246221

10. Sasso L, Bagnasco A, Catania G, Zanini M, Aleo G, Watson R, et al. Push and pull factors of nurses’ intention to leave. J Nurs Manag. 2019;27: 946–954. pmid:30614593

11. Arslan Yurumezoglu H, Kocaman G. Predictors of nurses’ intentions to leave the organisation and the profession in Turkey. J Nurs Manag. 2016;24: 235–243. pmid:25900394

12. Boamah SA, Laschinger H. The influence of areas of worklife fit and work-life interference on burnout and turnover intentions among new graduate nurses. J Nurs Manag. 2016;24: E164–174. pmid:26122304

13. Dilig-Ruiz A, MacDonald I, Demery Varin M, Vandyk A, Graham ID, Squires JE. Job satisfaction among critical care nurses: A systematic review. Int J Nurs Stud. 2018;88: 123–134. pmid:30292878

14. Perry SJ, Richter JP, Beauvais B. The Effects of Nursing Satisfaction and Turnover Cognitions on Patient Attitudes and Outcomes: A Three-Level Multisource Study. Health Serv Res. 2018;53: 4943–4969. pmid:29957888

15. Saleh U, O’Connor T, Subhi H, Alkattan R, Harbi S, Patton D. The impact of nurse managers’ leadership styles on ward staff. Br J Nurs. 2018;27: 197–203. pmid:29457941

16. Van Osch M, Scarborough K, Crowe S, Wolff AC, Reimer-Kirkham S. Understanding the factors which promote registered nurses’ intent to stay in emergency and critical care areas. J Clin Nurs. 2018;27: 1209–1215. pmid:29148125

17. Liu W, Zhao S, Shi L, Zhang Z, Liu X, Li L, et al. Workplace violence, job satisfaction, burnout, perceived organisational support and their effects on turnover intention among Chinese nurses in tertiary hospitals: a cross-sectional study. BMJ Open. 2018;8: e019525. pmid:29886440

18. Nei D, Snyder LA, Litwiller BJ. Promoting retention of nurses: A meta-analytic examination of causes of nurse turnover. Health Care Manage Rev. 2015;40: 237–253. pmid:24901298

19. Van der Heijden B, Brown Mahoney C, Xu Y. Impact of Job Demands and Resources on Nurses’ Burnout and Occupational Turnover Intention Towards an Age-Moderated Mediation Model for the Nursing Profession. Int J Environ Res Public Health. 2019;16. pmid:31195753

20. Willard-Grace R, Knox M, Huang B, Hammer H, Kivlahan C, Grumbach K. Burnout and Health Care Workforce Turnover. Ann Fam Med. 2019;17: 36–41. pmid:30670393

21. Ran L, Chen X, Peng S, Zheng F, Tan X, Duan R. Job burnout and turnover intention among Chinese primary healthcare staff: the mediating effect of satisfaction. BMJ Open. 2020;10: e036702. pmid:33033013

22. Pang Y, Dan H, Jung H, Bae N, Kim O. Depressive symptoms, professional quality of life and turnover intention in Korean nurses. Int Nurs Rev. 2020;67: 387–394. pmid:32633425

23. Kim Y, Lee E, Lee H. Association between workplace bullying and burnout, professional quality of life, and turnover intention among clinical nurses. PLoS One. 2019;14: e0226506. pmid:31860673

24. Hämmig O. Explaining burnout and the intention to leave the profession among health professionals—a cross-sectional study in a hospital setting in Switzerland. BMC Health Serv Res. 2018;18: 785. pmid:30340485

25. Moloney W, Boxall P, Parsons M, Cheung G. Factors predicting Registered Nurses’ intentions to leave their organization and profession: A job demands-resources framework. J Adv Nurs. 2018;74: 864–875. pmid:29117451

26. Duan X, Ni X, Shi L, Zhang L, Ye Y, Mu H, et al. The impact of workplace violence on job satisfaction, job burnout, and turnover intention: the mediating role of social support. Health Qual Life Outcomes. 2019;17: 93. pmid:31146735

27. Lu H, Zhao Y, While A. Job satisfaction among hospital nurses: A literature review. Int J Nurs Stud. 2019;94: 21–31. pmid:30928718

28. Hung MSY, Lam SKK. Antecedents and Contextual Factors Affecting Occupational Turnover among Registered Nurses in Public Hospitals in Hong Kong: A Qualitative Descriptive Study. Int J Environ Res Public Health. 2020;17. pmid:32481664

29. Pélissier C, Charbotel B, Fassier JB, Fort E, Fontana L. Nurses’ Occupational and Medical Risks Factors of Leaving the Profession in Nursing Homes. Int J Environ Res Public Health. 2018;15. pmid:30150568

30. El Khamali R, Mouaci A, Valera S, Cano-Chervel M, Pinglis C, Sanz C, et al. Effects of a Multimodal Program Including Simulation on Job Strain Among Nurses Working in Intensive Care Units: A Randomized Clinical Trial. JAMA. 2018;320: 1988–1997. pmid:30357264

31. Goldstein H. Multilevel Statistical Models. John Wiley & Sons; 2011.

32. Croon MA, van Veldhoven MJPM. Predicting group-level outcome variables from variables measured at the individual level: A latent variable multilevel model. Psychological Methods. 2007;12: 45–57. pmid:17402811

33. Cohen S, Kamarck T, Mermelstein R. A Global Measure of Perceived Stress. Journal of Health and Social Behavior. 1983;24: 385–396. pmid:6668417

34. Quintard B. Du stress objectif au stress perçu. In ed Bruchon-Schweitzer M. et Dantzer R. (eds). Introduction à la psychologie de la santé. Paris: Presses Universitaires de France; 1994.

35. Karasek RA. Job Demands, Job Decision Latitude, and Mental Strain: Implications for Job Redesign. Administrative Science Quarterly. 1979;24: 285–308. pmid:30315367

36. Larocque B, Brisson C, Blanchette C. [Internal consistency, factorial validity and discriminant validity of the French version of the psychological demands and decision latitude scales of the Karasek “Job Content Questionnaire”]. Rev Epidemiol Sante Publique. 1998;46: 371–381. pmid:9864766

37. Hunt SM, McEwen J, McKenna SP. Measuring health status: a new tool for clinicians and epidemiologists. J R Coll Gen Pract. 1985;35: 185–188. pmid:3989783

38. Bucquet D, Condon S. Adaptation en français du Nottingham Health Profile et caractéristiques opératoires de la version française. INSERM U164, Villejuif, France; 1988.

39. Chavent M, Kuentz V, Liquet B, Saracco L. ClustOfVar: An R Package for the Clustering of Variables. arXiv:11120295 [stat]. 2011 [cited 31 Jul 2019]. Available: http://arxiv.org/abs/1112.0295.

40. Miller TL based on F code by A. leaps: Regression Subset Selection. 2020. Available: https://CRAN.R-project.org/package=leaps.

41. Akaike H. A new look at the statistical model identification. IEEE Transactions on Automatic Control. 1974;19: 716–723.

42. R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. 2019. Available: https://www.R-project.org/.

43. Scheipl F, Bolker B. RLRsim: Exact (Restricted) Likelihood Ratio Tests for Mixed and Additive Models. 2020. Available: https://CRAN.R-project.org/package=RLRsim.

44. Scheipl F, Greven S, Küchenhoff H. Size and power of tests for a zero random effect variance or polynomial regression in additive and linear mixed models. Computational Statistics & Data Analysis. 2008;52: 3283–3299.

45. Khan N, Jackson D, Stayt L, Walthall H. Factors influencing nurses’ intentions to leave adult critical care settings. Nurs Crit Care. 2019;24: 24–32. pmid:29635820

46. Nantsupawat A, Kunaviktikul W, Nantsupawat R, Wichaikhum O-A, Thienthong H, Poghosyan L. Effects of nurse work environment on job dissatisfaction, burnout, intention to leave. Int Nurs Rev. 2017;64: 91–98. pmid:27882573

47. Tuckett A, Winters‐Chang P, Bogossian F, Wood M. ‘Why nurses are leaving the profession … lack of support from managers’: What nurses from an e-cohort study said. International Journal of Nursing Practice. 2015;21: 359–366. pmid:24571860

48. Nagaya Y. A Review of Factors Affecting Nursing Turnover in Japan. BJSTR. 2018;12.

49. Zhang Y, Punnett L, Gore R, CPH-NEW Research Team. Relationships among employees’ working conditions, mental health, and intention to leave in nursing homes. J Appl Gerontol. 2014;33: 6–23. pmid:24652941

50. Foster-Johnson L, Kromrey JD. Predicting group-level outcome variables: An empirical comparison of analysis strategies. Behav Res. 2018;50: 2461–2479. pmid:29508238

51. Schootman M, Ratnapradipa K, Loux T, McVay A, Su LJ, Nelson E, et al. Individual- and county-level determinants of high breast cancer incidence rates. Translational Cancer Research. 2019;8: S323-S333–S333.

52. Bennink M, Croon MA, Kroon B, Vermunt JK. Micro–macro multilevel latent class models with multiple discrete individual-level variables. Adv Data Anal Classif. 2016;10: 139–154.

53. Bennink M, Croon MA, Vermunt JK. Micro–macro multilevel analysis for discrete data: A latent variable approach and an application on personal network data. Sociological Methods & Research. 2013;42: 431–457.

54. Bennink M, Croon MA, Vermunt JK. Stepwise Latent Class Models for Explaining Group-Level Outcomes Using Discrete Individual-Level Predictors. Multivariate Behav Res. 2015;50: 662–675. pmid:26717125

55. Enders CK, Tofighi D. Centering predictor variables in cross-sectional multilevel models: a new look at an old issue. Psychol Methods. 2007;12: 121–138. pmid:17563168

56. Shreffler J, Petrey J, Huecker M. The Impact of COVID-19 on Healthcare Worker Wellness: A Scoping Review. West J Emerg Med. 2020;21: 1059–1066. pmid:32970555