Efficacy of the therapeutic use of video games on the depressive state of stroke patients: Protocol for systematic review and meta-analysis

Abstract

Aim

To assess the effects of virtual reality (VR) on the depressive state of patients with stroke admitted to neuro-rehabilitation units. Design: Systematic review and meta-analysis protocol.

Methods

Randomized Controlled Trials (RCTs) focusing on the effects of virtual reality on depressive state as a primary outcome will be included. Grey literature and the following databases will be consulted: PubMed, Cinahl, PsycInfo, Scopus, Embase, Cochrane Library and Web of Science. The recently revised Cochrane risk of bias tool will be used to assess the quality of included studies. Data will be extracted and meta-analyses will be performed within the specific condition of the emotional state of stroke patients admitted to neurorehabilitation units. Meta-regression and subgroup analyses will be used to identify effective modes and patterns of therapy delivery. The approach of assessment, development and evaluation of recommendations will be applied to reach a convincing conclusion.

Discussion

An accurate, transparent and standardized review process is expected to provide recommendations on the use of VR technology in the healthcare of stroke patients.

Impact

Emotional difficulties are common after stroke and have an impact on rehabilitation outcome. VR seems to have an important role in the treatment and depression in neurological patients, as it is able to improve levels of well-being, coping strategies and social relationships. The systematic review may contribute to a more convincing and specific conclusion compared to existing studies of this type.

Trial registration

Systematic review registration: CRD42022303968.

Introduction

Stroke is the second leading cause of death worldwide; it is the first leading cause of disability and the second leading cause of dementia. The high morbidity and mortality rates associated with stroke have led to the increasing creation of specialised units for its treatment [1]. Around 30% of people who survive stroke require assistance in activities of daily life (ADLs), 20% need help with ambulation, and 16% need institutional care [2]. An early multidisciplinary approach reduces high dependency rates [3].

The World Health Organization (WHO) defines depression as a condition that can often become chronic or recurrent. Those who suffer depression may feel a lack of concentration, tiredness, guilt, a lack of self-esteem and concentration, sleep or appetite disorders, a lack of interest or pleasure, sadness, hindering the individual’s ability to cope with daily life, and in its most severe form, it may even lead to suicide [4].

According to the latest published literature reviews, stroke patients have a prevalence of 19.5% of minor depression and 21.7% of major depression. Furthermore, the loss of autonomy is the most strongly correlated variable in these emotional disorders. [5]. Depression after a stroke is a serious complication that significantly restricts rehabilitation outcomes [6], being the most common neuropsychiatric consequence, with an incidence of approximately 30%–60% after the event [7].

The diagnosis of depression in neuro-rehabilitation units is difficult to manage given the characteristics of the stroke patient. These patients exhibit altered cognitive status, aphasia, agnosia, apraxia and memory disorders. Some of the signs prevalent in depression and stroke are common, such as sleep disturbances, difficulty in concentration, and appetite reduction; others make its aetiology indistinguishable, with it being unclear whether they are consequences of depression, stroke or age. Given these difficulties in diagnosis, between 50% and 80% of cases go undiagnosed [8].

Currently we can find evidence on some digital technologies that can be applied to support traditional care for various neurological dysfunctions showing benefits, not only in sensorimotor and cognitive functions, but also in other problems as pain or depression. But, when it comes to the use of Virtual Reality (VR), there is still no consensus on the types, duration, and intensity of training to assess clinical efficacy [9]. Furthermore, the use of digital technology in stroke patients has been reinforced by the Covid-19 pandemic as an opportunity to accelerate the process of digitalization in the stroke field [10].

Specifically, it has been shown that the use of Virtual Reality (VR) as a co-adjuvant therapy of neuro-rehabilitation in stroke patients with emotional disorders decreases the incidence of these disorders [11]. VR is considered a cost-effective tool for the treatment of psychological disorders. The reviewed studies claim that VR has positive effects on the affective state of patients. Emotional changes may occur gradually due to the properties and characteristics of the game (immediate feedback, challenging tasks, revaluation, etc.) and may motivate the rehabilitation process [12,13].

The reviewed studies also claim that VR as an adjunct treatment has positive effects, but no evidence supports the use of VR as a sole alternative treatment for mood disorder, anxiety, and depression [14,15].

Primary objective

This systematic review attempts to assess the effects of VR on the depressive state of patients with stroke admitted to neuro-rehabilitation units by synthesizing the RCTs available.

Review question

Primary review question.

Is the use of VR effective in the treatment of the depression state of stroke patients compared to patients who do not have this adjuvant therapy in their rehabilitation treatment?

Secondary review question.

Do the characteristics of the intervention session (number of sessions, duration of the session, individual versus group session) have an impact on your emotional state?

Methods

We plan to perform this systematic review to the procedures in the Cochrane Handbook.

We also plan to use the MeaSurement Tool to Assess systematic Reviews 2 [16], Preferred Reporting Items for Systematic reviews and Meta-Analyses [17], PRISMA-P (for protocols) 2015 [18]. The PRISMA-P checklist is shown in the S1 Appendix. This protocol has also been registered at the International Prospective Register of Systematic Reviews (PROSPERO) with the registration number as CRD42022303968. The systematic review has been funded since January 2022 and this protocol was reviewed by OML, as external expert in the field.

Data sources and search strategy

Studies will be identified through electronic searches of bibliographic databases and reference lists of articles with systematically searched in PubMed, Cinahl, PsycInfo, Embase, Cochrane Library and Web of Science will be used.

Medical subject heading (MeSH) terms related and text words will be adopted, mainly including “stroke” OR “brain infraction” AND “virtual reality” OR “virtual reality therapy”. No language restriction will be applied. Grey literature such as theses and articles located via the snowball, dissertations and conference proceeding method will be consulted. Experts will be consulted OML on the subject and RLG on the methodological level.

The search will be restricted to include documents published until January 1sr, 2022. The search strategy used for the PubMed database is shown in Table 1. The search strategy for all the databases is shown in the S2 Appendix.

[Figure omitted. See PDF.]

Criteria for study selection

For the selection of articles there will be no language restriction.

In general, studies will be screened and selected based on PICOS format as follows.

Types of studies

* Only studies with the design of randomized and non-randomized controlled trials with at least two groups in which one of them evaluates the usual intervention plus the use of virtual reality and a control group with the usual intervention will be included.

* Studies evaluating any degree of intensity and different duration of virtual reality training will be included.

Types of participants

Adults (at least 18 years of age) of any gender patients admitted to neuro-rehabilitation units due to ischemic or haemorrhagic stroke, all levels of severity, and at all stages after stroke.

Types of interventions

* We will include studies using virtual reality interventions that met the following definition “"an advanced form of human-computer interface that allows the user to ’interact’ and ’immerse’ in a computer-generated environment in a natural way" [19].

* Studies using any form of immersive or non-immersive virtual reality, and studies using commercially available game consoles will be included.

* The included studies will measure patients’ depression before and after the use of virtual reality.

* Control interventions. Conventional rehabilitation treatment without the use of VR. Given the wide range of alternative interventions, activities that include any activity designed to be therapeutic at the level of impairment, activity or participation that did not include the use of virtual reality will be considered.

Types of outcome measures

Primary outcomes: in the mental state, we can distinguish three main components: level of consciousness and alertness, cognitive function, and affective state or mood [20]. Accordingly, only articles that focus on the emotional state, specifically depression, will be included in this study. Studies which consider other indicators as a primary outcome will be excluded.

Screening procedures of eligible studies

After the initial systematic search records will managed with bibliographic manager Mendeley, duplicates will be identified and deleted. All titles identified with the literature search will be included. A peer review (NMS and TCA) will be screened and in case of disagreement, it will be assessed by a third reviewer (LRL). A justified list of those articles evaluated in full but excluded from the review will be provided. It will be presented in sufficient detail to complete a PRISMA flow chart and diagram as well as a characteristic table of excluded studies [21]. The Kappa coefficients to measure inter-rater agreement will be both calculated for the processes of titles/abstract selection and full-text screening. Specifically, the extent of between-rater agreements can be judged according to the criteria proposed by Landis and Koch [22]: 0.00–0.20 as ‘slight agreement’, 0.21–0.40 as ‘fair’, 0.41–0.60 as ‘moderate’, 0.61–0.80 as ‘substantial’, and 0.81–1.00 as ‘almost perfect agreement’. The plan of study screening and selection is available in the S1 Fig.

Data extraction

Data extraction will be performed with a pre-piloted, standardized form.

Data will be independently extracted by two reviewers (RMG and ALM) and will identify studies that meet the inclusion criteria. A third reviewer (PBG) will be resolved cases of author disagreement. We will be contacted when there was missing information. Kappa coefficients will be calculated to measure inter-rater agreement in the title/abstract selection process and to the full-text screening.

The data to be recorded will include study characteristics (type of study, authors, year of publication and journal in which it was published), characteristics of the participants (sample size, sex, mean age, nationality and loss rate), characteristics of the intervention (context in which the intervention was delivered, methodology used to deliver the intervention and costs) and the evaluation of the intervention (which scales were used in the study and at which point in the intervention as well as the data and conclusions of the study).

If after the review of the selected articles, any data essential for the review is not available, the authors will be contacted for a request.

Assessment of risk of bias

The risk of bias will be assessed through domains defined in the Cochrane Handbook for Systematic Reviews Intervention: there are five domains: domain selection bias (sequence generation, allocation concealment), domain conduct bias (blinding of participants and personnel), domain detection bias (blinding of outcome assessors), domain attrition bias (incomplete outcome data), domain reporting bias (selective reporting of results) and domain other biases [23].

The results will be shown as follows: ‘low risk’ of bias, ‘high risk’ of bias, or ‘unclear risk’ of bias. A ‘risk of bias’ graph will be included in the Cochrane review, where each of the assessments will be presented [24].

Two reviewers of the research team (RMG and ALM) will independently perform the assessment and a third reviewer (PBG) will be responsible to recheck the discrepancies or disagreement during the assessment process and make a final decision. The Kappa coefficients will also be calculated for the five assessment domains and the overall bias respectively.

Assessment of heterogeneity

Data synthesis and analysis.

According to the extracted data, descriptive statistics will be performed with SPSS 26.0 (IBM Company, Chicago, IL, USA) to explore the characteristics of the included studies, including general information, settings, participants, interventions, controls/comparators, outcomes and follow-ups.

Based on the statistics results, the afore-mentioned information will be summarized in detail. Specifically, for the intervention of VR as an adjunctive treatment for rehabilitation will be considered in addition to conventional rehabilitation. The main information will be summarized in terms of the duration each session, type of session (individual or non-individual), session frequency, etc. It is expected that the main features of the included RCTs, can be therefore clearly depicted.

We will also attempt to classify the RCTs into groups according, randomized controlled trials (RCTs) and non-randomized clinicals trials (NRCT). To ensure the homogeneity we will perform meta-analyses within each condition group. We attempt to pool the Effect sizes (ES) for functioning. Compressive Meta-Analysis (CMA) will be applied to perform meta-analysis. Standardized mean differences, or ES, will be calculated to explore the effects of VR on the depression state of patients with stroke admitted to neuro-rehabilitation units on functioning. According to Cohen (1988) [22], an ES < 0.2 is defined as negligible effect, 0.2 ≤ ES < 0.5 as small, 0.5 ≤ ES < 0.8 as medium, and ES ≥ 0.8 as large. For repeated measures, all time point meta-analysis method will be used to pool the follow-up data across the studies at different time points to discern the trend of the effect [25]. Heterogeneity will be assessed through a statistic test, I2 who is the proportion of the variability due to real differences between the estimator’s respect to the variability due to chance [26].

The data will be interpreted as follows Cochrane Handbook: 0% to 40% (may not be important), 30% to 60% (may represent moderate heterogeneity), 50% to 90% (may represent significant heterogeneity) and 75% to 100% (considerable heterogeneity) [26].

Thus, if we consider that the studies are sufficiently similar clinically, the random effects model will be applied, otherwise, the SE will be calculated on functioning in each individual study and a narrative summary will be presented.

Assessment of reporting biases.

If we find more than ten studies, we will create and examine a funnel plot to explore the possible biases of studies and publications according to the Cochrane Handbook for Systematic Reviews of Interventions. We will use Egger’s test [27] to analyse the degree of skewness in the funnel plots.

Assessment of evidence quality.

For each pooled or individual ES of functioning, the certainty of the evidence body will be independently rated by two reviewers (LRG and PBG) with GRADE [28]. Any discrepancies will be resolved by a third reviewer (RLG). We will create summary of findings tables with GRADEPro-GDT (https://gradepro.org/)

Information on all primary and secondary outcomes from our review will be included. The quality of the evidence will be assessed using five factors: 1) limitations in trial design and implementation of available trials; 2) indirect evidence; 3) unexplained heterogeneity or inconsistency of results; 4) imprecision of effect estimates; and 5) potential publication bias.

For each outcome the quality of the evidence will be classified according to the following categories: high quality (further research is very unlikely to change our confidence in the effect estimate), moderate quality (additional research is likely to have a significant impact on our confidence in the effect estimate and may change the estimate), low quality (further research is very likely to have a significant impact on our confidence in the effect estimate, and the estimate is likely to change) and very low quality (we are not sure of the estimate). Studies with high or unclear risk of performance and detection bias caused by inadequate blinding will be excluded. The subgroup analysis and investigation of heterogeneity data in the following categories will be presented: age, duration of the intervention and gender. A GRADE evidence profile will be performed at the end of the review to document all the findings for the listed primary and secondary outcomes. The certainty of the evidence from each study will be assessed using the GRADE approach, and one of the following levels will be assigned: high or tall, moderate, low and very low. The evidence will be evaluated in each of these domains as indicated in the GRADE manual. Since the certainty of the evidence is subjective, it will be discussed among the reviewers.

The effect of the intervention will be assessed using relative ratio (RR) and 95% confidence intervals for RCT or Odds Ratio (OR) and 95% confidence intervals for NRCT for the dichotomous variable, while the continuous variable will be merged to mean difference (MD) and 95% confidence intervals.

The data will be adjusted by multiplying the mean values by −1 as some scales increase with disease severity. The coefficient of between-rater Kappa will also be calculated.

Ethical considerations

This review will not involve private information from individuals and will not affect patient rights therefore does not require ethical approval. The results of this review will be disseminated through peer-reviewed publications and conference reports.

Validity and reliability

For the design and reporting of this systematic review, we will strictly follow the requirements of Cochrane Handbook [29], AMSTAR2 [16], PRISMA [17], and PRISMA-P [18] to ensure the validity and reliability.

Discussion

Over the past two decades, the number of studies on the efficacy of VR in mental health treatment has increased. The decrease in cost compared to conventional cognition, emotion, and behavioural treatments has contributed to its inclusion [30].

VR is considered a cost-effective tool for the treatment of psychological disorders and, specifically, the treatment of anxiety. Reviewed studies affirm that VR has positive effects on the affective state of patients and may be a more effective treatment than traditional anxiety treatments. The reviewed studies also claim that VR as an adjunct treatment has positive effects, but no evidence supports the use of VR as a sole alternative treatment for mood disorder, anxiety and depression [15,31–33].

Stroke patients are often isolated in their daily life from their family and society. Depression, apathy and anxiety are prevalent conditions in stroke patients, which negatively affect their rehabilitation. The use of VR in rehabilitation allows the patient to receive immediate feedback on task performance as well as visual and audible stimulation and often arouses the patient’s interest. VR generates motivation in treatment participation, and the patients have fun while actively performing tasks. It can provide stability in ADLs, thus improving the autonomy of the stroke patient [15,32–35]. In addition, to improving the functional sphere, VR used in the treatment of depression in stroke patients improves their interpersonal relationships when used in a group setting [33].

The average cost of hospitalization in specialized neurorehabilitation units is 4.348 euros per patient, with an average daily expenditure of 194 euros. Expenditure increases 4 times more in totally dependent patients when compared with more independent patients, so the cost-effectiveness of early rehabilitation after stroke are positively associated with the degree of motor disability [36].

Therefore, VR is shown as a therapeutic approach that brings benefits to the rehabilitation process of stroke patients, not only in terms of sensorimotor or cognitive functionality, but also with a positive impact on the emotional situation of these patients [37]. However, it is necessary to synthesize the evidence regarding its benefits taking into account the different types, duration and intensity of training to ensure clinical efficacy [9].

The COVID-19 emergency has highlighted the need to develop and implement new digital technologies for acute and chronic patient care [10]. This review aims specifically to clarify the scientific evidence on the impact of virtual reality on depression symptomatology.

Supporting information

S1 Fig. Plan of study screening and selection process.

https://doi.org/10.1371/journal.pone.0275740.s001

(DOCX)

S1 Appendix. The PRISMA-P checklist and PRISMA flow diagram of the study selection process.

https://doi.org/10.1371/journal.pone.0275740.s002

(DOCX)

S2 Appendix. The search strategy.

https://doi.org/10.1371/journal.pone.0275740.s003

(DOCX)

Acknowledgments

The authors would like to thank Dr. Olga Martínez-López (OML) of the University of Castilla-La Mancha for reviewing this protocol as an expert researcher in the field.

Citation: Blázquez-González P, Mirón-González R, Lendínez-Mesa A, Moro-Tejedor MN, Cobo-Sánchez JL, Mancebo-Salas N, et al. (2022) Efficacy of the therapeutic use of video games on the depressive state of stroke patients: Protocol for systematic review and meta-analysis. PLoS ONE 17(12): e0275740. https://doi.org/10.1371/journal.pone.0275740

References

1. Masjuan J, Gállego Culleré J, Ignacio García E, Mira Solves JJ, Ollero Ortiz A, Vidal de Francisco D, et al. Stroke treatment outcomes in hospitals with and without stroke units. Neurol (English Ed). 2020;35:16–23. Available from: https://doi.org/10.1016/j.nrleng.2017.06.005.

2. Biller J, Ruland S, Schneck MJ. Isquemic cerebrolvascular disease. En: Daroff RB, Jankovic J, Mazziotta JC, Pomeroy SL, editores. Bradley’s neurology in clinical practice, 2-volume set. 7a ed. Filadelfia, PA, Estados Unidos de América: Elsevier—Health Sciences Division; 12 2015. p. 920–67.

3. Garcia-Munoz C, Casuso-Holgado MJ. Efectividad de la Wii Fit Balance frente a otras intervenciones para la recuperación del equilibrio en pacientes postictus. Revisión sistemática y metaanálisis. Rev Neurol [Internet]. 2019;69(7):271–9. Disponible en: https://doi.org/10.33588/rn.6907.2019091.

4. Depression [Internet]. Who.int. [citado el 8 de septiembre de 2022]. Disponible en: https://www.who.int/news-room/fact-sheets/detail/depression.

5. Quemada JI, Mimentza N. [Los trastornos emocionales y conductuales postictus]. Inf Psiquiátricas. 2017 [cited 2022 Jan 18];228:55–61. Available from: http://www.informacionespsiquiatricas.com/admin-newsletter/uploads/docs/20170613161925.pdf.

6. Robinson RG, Jorge RE. Post-Stroke Depression: A Review. Am J Psychiatry. 2016 Mar 1;173(3):221–31. Epub 2015 Dec 18. pmid:26684921.

7. Shi Y, Liu W, Liu R, Zeng Y, Wu L, Huang S, et al. Investigation of the emotional network in depression after stroke: A study of multivariate Granger causality analysis of fMRI data. J Affect Disord. Elsevier B.V.; 2019 [cited 2022 Jan 18];249:35–44. Available from: pmid:30743020

8. Espárrago Llorca G, Castilla-Guerra L, Fernández Moreno MC, Ruiz Doblado S, Jiménez Hernández MD. Depresión post ictus: Una actualización. Neurologia. 2015 [cited 2022 Jan 18];30:23–31. Available from: https://doi.org/10.1016/j.nrl.2012.06.008.

9. Abbadessa G., Brigo F., Clerico M. et al. Digital therapeutics in neurology. J Neurol 269, 1209–1224 (2022). pmid:34018047

10. Iodice F, Romoli M, Giometto B, Clerico M, Tedeschi G, Bonavita S, et al. Stroke and digital technology: a wake-up call from COVID-19 pandemic. Neurol Sci. 2021 Mar;42(3):805–809. Epub 2021 Jan 12. pmid:33433756; PMCID: PMC7801773.

11. Moreno A, Wall KJ, Thangavelu K, Craven L, Ward E, Dissanayaka NN. A systematic review of the use of virtual reality and its effects on cognition in individuals with neurocognitive disorders. Alzheimer’s Dement Transl Res Clin Interv. 2019 [cited 2022 Jan 18];5:834–50. Available from: pmid:31799368

12. Park M, Ko MH, Oh SW, Lee JY, Ham Y, Yi H, et al. Effects of virtual reality-based planar motion exercises on upper extremity function, range of motion, and health-related quality of life: A multicenter, single-blinded, randomized, controlled pilot study. J Neuroeng Rehabil. 2019;16:1–13. Available from: https://doi.org/10.1186/s12984-019-0595-8.

13. López-Martín O, Segura Fragoso A, Rodríguez Hernández M, Dimbwadyo Terrer I, Polonio-López B. Efectividad de un programa de juego basado en realidad virtual para la mejora cognitiva en la esquizofrenia. Gac Sanit. 2016 [cited 2022 Jan 18];30:133–6. Available from: pmid:26611240

14. Rodríguez KV, Pardo JC, Abdellah LD, Martín SM, Tomás JFÁ de. Gamificación: papel del juego en las aplicaciones digitales en salud. FMC—Form Médica Contin Aten Primaria [Internet]. 2015;22(7):369–74. Disponible en: https://doi.org/10.1016/j.fmc.2015.05.002.

15. Song G Bin, Park EC. Effect of virtual reality games on stroke patients’ balance, gait, depression, and interpersonal relationships. J Phys Ther Sci. 2015;27:2057–60. Available from: pmid:26311925

16. Shea B, Reeves R, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ; 2017;358. Available from: pmid:28935701

17. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. Ann Intern Med; 2009 [cited 2022 Jan 18];151:264–9. Available from: pmid:19622511

18. Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev; 2015;4:1–9. Available from: pmid:25554246

19. Palacios-Navarro G, Hogan N. Head-mounted display-based therapies for adults post-stroke: A systematic review and meta-analysis. Sensors (Switzerland) 2021; 21: 1–24. Available from: pmid:33562657

20. Ríos Flórez J.A; Jiménez Zuloaga PJ. Activación de las redes neuronales del arte y la creatividad en la rehabilitación neuropsicológica. Cuad Hispanoam Psicol. 2015;15:47–60. Available from: https://dialnet.unirioja.es/descarga/articulo/5689501.pdf.

21. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, et al. The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration. PLoS Med. 2009 [cited 2022 Jan 18];6:e1000100. Available from: pmid:19621070

22. Landis R., Koch G. An Application of Hierarchical Kappa-type Statistics in the Assessment of Majority Agreement among Multiple Observers. Biometrics 1977; 33: 363–374. Available from: https://www.jstor.org/stable/2529786. pmid:884196

23. Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: A revised tool for assessing risk of bias in randomised trials. BMJ. 2019 [cited 2022 Jan 18];366:1–8. Available from: pmid:31462531

24. Cohen J. Statistical power analysis for behavioural sciences (2nd ed.). 2 nd. New York: Lawrence Erlbaum Associates, 1988[cited 2022 Jan 18]65(3):145–153.

25. Peters JL, Mengersen KL. Meta-analysis of repeated measures study designs: Meta-analysis of repeated measures study designs. J Eval Clin Pract [Internet]. 2008;14(5):941–50. Available from: https://doi.org/10.1111/j.1365-2753.2008.01010.x.

26. Higgins JPT, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat. Med. 2002 [cited 2022 Jan 18]. 21(11), 1539–58. Available from: pmid:12111919

27. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997 Sep 13;315(7109):629–34. pmid:9310563

28. Furlan AD, Pennick V, Bombardier C, Van Tulder M. 2009 updated method guidelines for systematic reviews in the Cochrane Back Review Group. Spine (Phila Pa 1976). Spine (Phila Pa 1976); 2009 [cited 2022 Jan 18];34:1929–41. Available from: pmid:19680101

29. Higgins J, Green S. Cochrane Handbook for Systematic Reviews of Interventions [Manual Cochrane de revisiones sistemáticas de intervenciones, in Spanish]. Cochrane Collab 2011; 1–639.

30. Shiban Y, Brütting J, Pauli P, Mühlberger A. Fear reactivation prior to exposure therapy: Does it facilitate the effects of VR exposure in a randomized clinical sample? J Behav Ther Exp Psychiatry. Elsevier Ltd; 2015 [cited 2022 Jan 18];46:133–40. Available from: https://pubmed.ncbi.nlm.nih.gov/25460259/https://doi.org/10.1016/j.jbtep.2014.09.009

31. Anderson KR, Woodbury ML, Phillips K, Gauthier L V. Virtual reality video games to promote movement recovery in stroke rehabilitation: A guide for clinicians. Arch Phys Med Rehabil. 2015 [cited 2022 Jan 18];96:973–6. Available from: pmid:25910856

32. Fodor LA, Coteţ CD, Cuijpers P, Szamoskozi Ş, David D, Cristea IA. The effectiveness of virtual reality based interventions for symptoms of anxiety and depression: A meta-Analysis. Sci Rep. 2018 [cited 2022 Jan 18];8:1–13. Available from: https://doi.org/10.1038/s41598-018-28113-6.

33. Lindner P, Hamilton W, Miloff A, Carlbring P. How to treat depression with low-intensity virtual reality interventions: Perspectives on translating cognitive behavioral techniques into the virtual reality modality and how to make anti-depressive use of virtual reality–unique experiences. Front Psychiatry. 2019 [cited 2022 Jan 18];10:1–6. Available from: https://doi.org/10.3389/fpsyt.2019.00792.

34. Zawadzka E, Domańska Ł. Assessment of select dimensions of patients’ emotional functioning at different time periods after stroke. Appl Neuropsychol. 2014 [cited 2022 Jan 18];21:87–93. Available from: pmid:24826501

35. Zeng N, Pope Z, Pope Z, Lee J, Gao Z. Virtual Reality Exercise for Anxiety and Depression: A Preliminary Review of Current Research in an Emerging Field. J Clin Med. 2018 [cited 2022 Jan 18];7:42. Available from: pmid:29510528

36. Angerova Y, Marsalek P, Chmelova I, et al. Cost and cost-effectiveness of early inpatient rehabilitation after stroke varies with initial disability: The Czech Republic perspective. Int J Rehabil Res 2020[cited 2022 Jan 18]; 0: 376–382.Available from: pmid:32991353

37. Lohse KR, Hilderman CGE, Cheung KL, Tatla S, Van der Loos HFM (2014) Virtual Reality Therapy for Adults Post-Stroke: A Systematic Review and Meta-Analysis Exploring Virtual Environments and Commercial Games in Therapy. PLoS ONE 9(3): e93318. pmid:24681826