Efficacy of Virtual Reality Exposure Therapy in the Treatment of Specific Phobias: A Systematic Review

Headnote

ABSTRACT

Specific phobia is defined as a notable and continuous fear of a certain object or a situation that particularly impairs daily life and functioning. It is also one of the most common psychological disorders. Exposure-based interventions are commonly used in the treatment of specific phobias. However, some limitations of the standard methods require the need for alternative approaches. In light of this, the use of virtual reality technology in psychotherapy has become increasingly widespread in recent years and is now integrated with exposure therapy. Virtual reality provides real-time interaction using the computer-generated three-dimensional environment via variety oftechnological tools. Applications of virtual reality in exposure therapy have proven to be an important intervention method, especially in the psychopathologies such as specific phobias. In this study, virtual reality exposure therapy for the treatment of specific phobias is systematically reviewed. Inclusion criteria were taken into account in the scanning performed in APA (PsycINFO), EBSCO, PubMed, Scopus, Web of Science, Science Direct, Ulakbim ve TRDizin databses following the PRISMA method. Following that, ten randomized controlled trials, which included adult participants meeting the diagnostic criteria for specific phobia, examined the efficacy of virtual reality and exposure therapy compared to the control group, and the ones with full text could be accessed, were included in this systematic review. The included studies were evaluated in terms of sample attributes, primary measurement tools, research design, characteristics of the intervention and its efficacy. In general, the findings indicate the efficacy of virtual reality exposure therapy in reducing specific phobia symptoms. Follow-up studies support the long-term maintenance of the results.

Keywords: Phobia, fear, virtual reality, exposure therapy, systematic review

ÖZ

Özgül fobi, günlük yaşamı ve işlevselliǧi önemli derecede olumsuz etkileyen belirli nesne ya da duruma yönelik belirgin ve sürekli bir korku olarak tanımlanmaktadır. Aynı zamanda, en sık görülen psikolojik bozukluklar arasında olduǧu bildirilmektedir. Özgül fobilerin tedavisinde maruz bırakma temelli müdahalelerin yaygın olarak kullanıldıǧı bilinmekle birlikte standart yöntemlerin bazı sınırlılıklarının bulunması farklı yaklaşımlara ihtiyaç duyulduǧuna işaret etmektedir. Bu çerçevede, son yıllarda psikoterapi baǧlamında sanal gerçeklik teknolojisinin kullanımı giderek yaygınlaşmış ve maruz bırakma terapisi ile bütünleştirilmiştir. Bilgisayar tarafından oluşturulan üç boyutlu ortamla çeşitli teknolojik araçlar aracılıǧıyla gerçek zamanlı etkileşim saǧlayan sanal gerçekliǧin, maruz bırakma terapisi ile kullanımının özellikle özgül fobiler gibi psikopatolojilerin tedavisinde önemli bir müdahale yöntemi haline geldiǧi görülmektedir. Bu çalışma kapsamında, özgül fobilerin tedavisinde sanal gerçeklik ile maruz bırakma müdahaleleri sistematik olarak gözden geçirilmiştir. PRISMA yöntemi izlenerek APA (PsycINFO), EBSCO, PubMed, Scopus, Web of Science, Science Direct, Ulakbim ve TRDizin veri tabanlarında yapılan taramada dahil etme ölçütleri kapsamında; özgül fobi tanı kriterlerini karşılayan yetişkin katılımcıların yer aldıǧı, sanal gerçeklik ile maruz bırakma terapisinin kontrol grubuna kıyasla etkinliǧinin incelendiǧi ve tam metnine ulaşılabilen 10 seçkisiz kontrollü çalışma bu sistematik derlemeye dahil edilmiştir. Dahil edilen makaleler örneklem özellikleri, temel ölçüm araçları, araştırma deseni, müdahalenin özellikleri ve etkisi açısından deǧerlendirilmiştir. Bulgular, sanal gerçeklik ile maruz bırakma terapisinin, özgül fobi belirtilerini azaltmadaki etkinliǧine işaret etmektedir. Yapılan izlem çalışmaları tedavi sonuçlarının uzun vadede korunduǧuna yönelik destek saǧlamaktadır.

Anahtar sözcükler: Fobi, korku, sanal gerçeklik, maruz bırakma terapisi, sistematik gözden geçirme

Introduction

According to the Diagnostic and Statistical Manual of Mental Disorders (DSM-5), specific phobia is an anxiety disorder defined by clinically significant and persistent fear or anxiety towards a specific object or situation that negatively affects people's functionality (American Psychiatric Association [APA] 2013). In DSM-5, the diagnosis of specific phobia is classified into five types according to the source of fear or anxiety: blood-injection-injury type (blood, dental, etc.), animal type (spider, dog, etc.), natural environment type (height, storm, etc.), situational type (flying, driving, etc.) and unspecified subtypes (drowning, vomiting, etc.) (APA 2013). However, it is accepted that the fear of specific objects or situations in individuals diagnosed with specific phobia is disproportionate in the sociocultural context compared to the real danger it may cause (Antony and Barlow 1998, APA 2013). Epidemiologic studies report that specific phobias are the most common anxiety disorder diagnosis (Kessler et al. 2012). In a large-scale study conducted with a total of 124,902 adult participants from twenty-two countries, the lifetime prevalence of specific phobia was reported to be approximately 7.4% (Wardenaar et al. 2017). This rate is 9.8% in women and 4.9% in men. All specific phobia subtypes are more common in women (Sigström et al. 2011). The lifetime prevalence of specific phobia in Turkey is estimated to be 11% and the difference between genders is similar with international studies (Ertan 2008).

Behavioral theory to explain specific phobias focuses on conditioning processes. According to Mowrer's (1947) two-factor model, specific phobias are considered as a conditioned response that develops after a threatening experience and is maintained by avoidance behavior. According to this model, in the first stage, fear occurs through classical conditioning in which a previously neutral object/situation is paired with a feared object/situation. This conditioning can occur through modeling or verbal instruction in addition to a direct experience. The second stage involves operant conditioning, in which avoidance behavior is reinforced because it reduces anxiety and provides short-term relief, and thus the person's fears persist. As long as the feared stimulus is not encountered and the feared consequences do not materialize, people do not learn that the situation is not as scary as imagined (Eaton et al. 2018). However, most people who have a directly threatening experience do not develop specific phobias. Risk factors such as genetic predisposition and cognitions with negative content are suggested to be effective in whether specific phobia will develop in the context of a conditioning experience (Kendler et al. 2002, Mineka and Sutton 2006).

One of the recent approaches in the treatment of specific phobias is exposure-based interventions. These interventions, especially experiential exposure, are evidence-based practices whose effectiveness in the treatment of specific phobias is supported by research (Chambless et al. 1998, APA Presidential Task Force on Evidence-Based Practice 2006). In these interventions, a hierarchy of anxiety is typically created by the therapist and the client regarding the conditions that may lead to anxiety/fear. In line with this hierarchy, the client is exposed to these conditions until he/she reports a significant decrease in anxiety level. Thus, it is assumed that new associations related to the conditioned stimuli will be learned and thus the conditioned fear response will be extinguished (Vervliet et al. 2013). There is a general consensus that exposure-based interventions are preferred in the treatment of specific phobias (Nahtan and Gorman 2015). However, individuals diagnosed with specific phobias generally do not tend to seek treatment and only 10% to 25% of these individuals receive treatment (Magee et al. 1996, Eaton et al. 2018). A possible explanation for this might be that, due to the nature of the diagnosis, individuals do not experience a problem as long as they do not encounter certain objects or situations. In addition, the fact that avoidance behaviors are effective by reducing the level of fear/anxiety in the short term is considered to be an important factor preventing treatment seeking (Eaton et al. 1991; Wardenaar et al. 2017, Eaton et al. 2018). On the other hand, approximately 17% to 27% of treatment-seekers reportedly refuse to receive experiential exposure therapy or discontinue treatment in the future (Garcia-Palacios et al. 2001, 2007). Although experiential exposure is widely accepted in the treatment of specific phobias (Choy et al. 2007, Barlow et al. 2015, Nathan and Gorman 2015), high rates of treatment dropout and low rates of treatment acceptance make this assumption controversial. In addition, there are some limitations in the application of experiential exposure therapy in specific phobias. For example, it may not always be possible to create and apply blood-injection or medical procedures that are the source of fear in the clinical environment in experiential exposure therapy that can be applied for blood-injection-injury type phobia (Jiang et al. 2020). Similarly, in animal-type phobias, there are possible limitations such as the transportation of the feared animal/insect species to the therapy environment and the difficulty in controlling these creatures by the clinician (Neudeck and Einsle 2012). Therefore, there is a need for different approaches to eliminate these limitations in the implementation of exposure-based interventions.

The use of virtual reality technology in psychotherapy applications is expected to be an alternative to standard methods in the context of the aforementioned limitations. Virtual reality application enables the person to interact with this environment in real time with various technological tools in a three-dimensional environment created by a computer (Maples-Keller et al. 2017). This interaction is realized as a result of the integration of sensory inputs such as visual, auditory and tactile through tools such as visual displays, body tracking devices and motion sensors (Anderson et al. 2001). In this way, the person actively explores the virtual environment and experiences fear or anxiety in the closest way to reality (Powers and Emmelkamp 2008). Virtual reality is integrated with the cognitive-behavioral-based exposure technique in the field of psychotherapy. This intervention, called virtual reality exposure, aims to reduce the level of anxiety by repeatedly and systematically exposing people to the stimuli and/or situations they fear in a safe environment, as in the traditional exposure technique (Hoffman and Smits 2008). Unlike the traditional practice, in this intervention, people with a diagnosis of specific phobia are exposed to the virtual equivalent of that stimulus instead of the real stimulus that arouses fear and/or anxiety (Pull 2008). Thus, it is assumed that people will get used to the stimulus that causes fear and/or anxiety after the intervention (Rothbaum and Hodges 1999, Wiederhold and Wiederhold 2004). In this framework, studies on exposure intervention with virtual reality were first conducted with individuals diagnosed with phobia. Although the application has been used for the first time in the treatment of people with flight phobia (North and North 1994, North et al. 1996), the first study evaluating the effectiveness of the therapy was performed by North and North (1996) with participants with agoraphobia. Recently, the place of the intervention in the treatment of specific phobias has been addressed more broadly, including bloodinjection-injury (Jiang et al. 2020), animal (Götestam 2002, Suso-Ribera et al. 2019), natural environment (Donker et al. 2019) and situational types (Kaussner et al. 2020, Trappey and Trappey 2020).

The idea that virtual reality exposure therapy could be an alternative to standard methods has led to studies on the acceptability of this method by participants. Some findings reveal that more than 80% of people diagnosed with specific phobia may prefer virtual reality-based exposure compared to experiential exposure (GarciaPalacios et al. 2001, 2007). This preference seems to be made on the grounds that participants are afraid of encountering the real object/situation and that virtual reality provides a safer environment (Garcia-Palacios et al. 2007). However, virtual reality technology also provides convenience in terms of applying exposure in different conditions (Vansteenwegen et al. 2006) and is increasingly considered as a low-cost and accessible application (Donker et al. 2019). Considering these factors, integrating virtual reality technology with exposure therapy is predicted to be an effective treatment approach.

The main aim of this study was to determine the effectiveness of virtual reality exposure therapy in adults with specific phobia compared to control groups. In the literature, there are systematic reviews that address different types of anxiety-based disorders together, covering case report studies, and including various research designs such as multiple levels of baseline (Botella et al. 2017, Oing and Prescott, 2018, Wechsler et al. 2019, Freitas et al. 2021, Krzystanek et al. 2021, Albakri et al. 2022). Unlike other systematic review studies, this study aimed to examine only interventions focusing on the treatment of specific phobias. However, the highest level of evidence for the effectiveness of the intervention in psychotherapy research comes from the systematic review of randomized controlled trials (Chambless and Hollon 1998, Spring 2007). Therefore, the current systematic review was limited to randomized controlled trials comparing the effectiveness of virtual reality exposure therapy with control groups in specific phobias. When the current literature is examined, it is noteworthy that virtual reality exposure is frequently applied in the treatment of anxiety-related disorders. Considering its recent increasing use, especially in specific phobias (Geraets et al. 2021, Krzystanek et al. 2021), evaluating the effectiveness of the studies on the subject is deemed to be important. The research question of this study was determined within the framework of PICO (S) criteria (P: population/participants), (I: interventions/interventions), (C: comparators/comparison groups), (O: outcomes/results) and (S: study designs) (Richardson et al. 1995). Accordingly, the question "What is the effect on specific phobia symptoms in randomized controlled trials examining the effect of exposure therapy with virtual reality compared to the control group for adults who meet the diagnostic criteria for specific phobia or who are above the cut-off score on scales related to the diagnosis?" was addressed.

Method

This study was conducted in accordance with PRISMA (Preferred Reporting Items for Systematic Reviews and MetaAnalyses) criteria (Moher et al. 2015). Accordingly, APA (PsycINFO), EBSCO, PubMed, Scopus, Web of Science, Science Direct, Ulakbim and TRDizin databases were searched in Turkish and English languages without using any date range restrictions. Keywords were selected according to Medical Subject Headings (MeSH). The keywords "virtual reality AND phobia", "virtual reality AND fear", "virtual reality exposure therapy AND phobia", "virtual reality exposure therapy AND phobia", "virtual reality exposure therapy AND fear" were used in the search. As a result of the search performed between August 2022 and September 2022, a total of 833 studies were reached, including APA (PsychINFO) (n=98), EBSCO (n=57), PubMed (n=148), Scopus (n=170), Web of Science (n=158), Science Direct (n=193), Ulakbim (n=7) and TRDizin (n=2). When repeated studies were excluded, the title and abstract sections of the remaining 424 studies were independently evaluated by both authors within the scope of the inclusion criteria and a joint decision was reached. The studies included in the review were determined in accordance with PICO (S) criteria. In this context, randomized controlled trials involving adults who met the diagnostic criteria for specific phobia or who were above the cut-off score on scales related to the diagnosis; in which virtual reality exposure therapy was applied; in which a control group comparison was made; and in which the effect on specific phobia symptoms was examined were included in this review. Based on inclusion criteria, 349 studies did not meet these criteria. Consequently, 75 studies with full text were analyzed according to the inclusion and exclusion criteria. Among these studies, a total of 65 studies were excluded, which included participants who did not meet the criteria for specific phobia (n=6), did not apply exposure therapy with virtual reality (n=15), did not compare the effectiveness of exposure therapy with virtual reality with the control group (n=23), and were not conducted in a randomized controlled design (n=21). Overall, 10 research articles were included in the current study. The PRISMA flow diagram is depicted in Figure 1.

Results

The articles included in the current systematic review were evaluated in terms of sample characteristics, basic measurement tools, research design, intervention characteristics and effects. General information about the included studies is presented in Table 1.

Sample

The sample sizes in the studies included in the review ranged from 23 (Garcia-Palacios et al. 2002) to 193 (Donker et al. 2019). In all studies, the samples consisted of adult participants between the ages of 18 and 65. The number of female participants was higher in all studies. The lowest rate of female participants was 60% (Wiederhold et al. 2000, Gujjar et al. 2019) and the highest was 96.98% (Michaliszyn et al. 2010); the lowest rate of male participants was 3.02% (Michaliszyn et al. 2010) and the highest was 40% (Wiederhold et al. 2002, Gujjjar et al. 2019).

In all studies reviewed, the participants met the diagnostic criteria for specific phobia according to DSM-IV or DSM-5 or demonstrated symptoms of specific phobia by being above the cut-off score in the measurement tools related to the diagnosis. Spider phobia was addressed in four of the studies (40%). In all of these studies, inclusion criteria included the inability of participants to physically interact with a live spider in a behavioral avoidance task. In addition, participants had to meet the criteria for spider-type specific phobia according to DSM-IV (Garcia-Palacios et al. 2002, Michaliszyn et al. 2010) and DSM-5 (Miloff et al. 2019) and/or the Fear of Spiders Questionnaire. In all three studies on flight phobia (30%), participants had to meet the diagnostic criteria for specific phobia according to DSM-IV. Similarly, in a study aiming to intervene in blood-injectioninjury phobia (10%), participants who met the relevant diagnostic criteria according to DSM-5 were included in the study. Finally, participants who met the criteria according to the Modified Dental Anxiety Scale in a study (10%) addressing dental phobia and the Acrophobia Questionnaire in a study (10%) aiming to intervene in height phobia were included in this study within the scope of the inclusion criteria.

Research Designs

In all studies within the scope of the subject, participants were randomly assigned to intervention and control groups. In all intervention groups, exposure therapy with virtual reality was applied individually. Unlike the others, in one study (10%) (Donker et al. 2019), the intervention was carried out in the participant's own environment with a cognitive behavioral therapy-based smartphone application installed on the participants' personal phones and simple virtual reality goggles suitable for home use, while in another study (10%) (Wiederhold et al. 2002), there were two prerequisites with and without physiological feedback.

Practices for the control groups also varied among the studies. In three studies (30%) (Garcia-Palacios et al. 2002, Donker et al. 2019, Jiang et al. 2020), a waiting list control group was formed by taking measurements before and after the intervention, similar to the intervention group, without applying any intervention to the participants throughout the process. In one of the two studies with a placebo control group, participants were given an information brochure about the diagnosis (Gujjar et al. 2019), while in the other, only psychoeducation was applied (Minns et al. 2019). In two studies (20%), the standard treatment control group included experiential exposure (Miloff et al. 2019) and systematic desensitization (Wiederhold et al. 2002) interventions including imaginal exposure. Finally, three studies (30%) included a standard treatment control group (Rothbaum et al. 2000, Rothbaum et al. 2006, Michaliszyn et al. 2010) in which both waiting list and experiential exposure were applied.

Eight studies reviewed included a follow-up measurement after the intervention. In four of these, a follow-up measurement was taken once three months after the intervention ended (Wiederhold et al. 2002, Michaliszyn et al. 2010, Donker et al. 2019, Jiang et al. 2020 ). In one study, follow-up measurements were taken three and six months after the intervention (Gujjar et al. 2019), and in another study three and 12 months after the intervention (Miloff et al. 2019). Finally, two studies reported that follow-up measurements were taken 6 and 12 months after the end of the intervention (Rothbaum et al. 2000, Rothbaum et al. 2006).

Basic Measurement Tools Used

The effectiveness of exposure therapy with virtual reality has been evaluated with various measurement tools. The Fear of Spiders Questionnaire and the Behavioral Avoidance Task were used as the main measurement tools in all studies dealing with spider phobia. While phobia of flying was assessed with The Questionnaire on Attitudes Toward Flying and The Fear of Flying Inventory, in a different study (Wiederhold et al., 2002), SelfSurvey of Stress Responses and State-Trait Anxiety Inventory were used to measure anxiety symptoms. In addition to these self-report instruments, Wiederhold et al. (2002) utilized a computer-based biofeedback system to assess physiological symptoms such as body resistance, peripheral body temperature, heart rate and respiratory rate. Anxiety level towards dental situations was assessed with the Visual Analogue Scale for Anxiety, Dental Fear Survey and Modified Dental Anxiety Scale (Gujjar et al. 2019). The Acrophobia Questionnaire was used to assess phobia of heights (Donker et al. 2019). Finally, Jiang et al. (2020) evaluated the symptoms of the participants with the Medical Fear Survey in medical situations including injection/bloodletting, examination/symptoms, sharp objects, blood and injury.

Characteristics of Interventions

In four studies addressing spider phobia, virtual reality exposure was applied through a headset that allowed participants to explore the virtual environment by tracking their head movements. In these studies, in which the spider image in the virtual environment got closer and closer to the participants (Garcia-Palacios et al. 2002, Michaliszyn et al. 2010, Minns et al. 2019) and increased in realism (Miloff et al. 2019), the anxiety stages consisted of at least three and at most eight stages. In two of the studies, the intervention was completed in one session (Miloff et al. 2019, Minns et al. 2019). In another study, a total of eight sessions were conducted, with each anxiety stage in a separate session (Michaliszyn et al. 2010). In the study conducted by Garcia-Palacios et al. (2002), the number of sessions varied between three and 10. The session durations in the studies ranged from approximately 30 minutes to 180 minutes. When the studies are analyzed in terms of practitioner characteristics, two clinical psychologists were reported in one study (Garcia-Palacios et al. 2002) and five doctoral students in the psychology department in another study (Michaliszyn et al. 2010). While one study (Minns et al. 2019) did not provide information about the number and characteristics of the practitioners, Miloff et al. (2019) stated that the therapists were present only for technical problems, and that a virtual therapist and a spider expert carried out the intervention.

In two of the studies involving intervention for flight phobia (Rothbaum et al. 2000, Rothbaum et al. 2006), a seat belt and a seat with a speaker placed under it were used to create a realistic flight experience. The aim was to provide sound and vibration during the flight at the appropriate time through the speaker. In a second study (Wiederhold et al. 2002), a seat with a speaker was used to increase the realism of the flight experience. In these studies, the virtual reality environment was created with a device that can monitor the head movements of the participants and transfer their viewing angles to the virtual environment. In two studies (Rothbaum et al. 2000, Rothbaum et al. 2006), there were six stages in which the airplane moved from standby to take-off and the weather conditions gradually deteriorated. As part of the intervention, breathing exercise, cognitive restructuring and thought stopping for ruminative thoughts were applied followed by four sessions of exposure technique. In the study of Wiederhold et al. (2002), after the breathing exercise and fear hierarchy were created, exposure was completed in six sessions. In these studies, session durations varied between 30 minutes and 90 minutes.

Donker et al. (2019) developed a smartphone application for the treatment of height phobia, in which participants can manage the process completely by themselves and experience the virtual environment with a 360° perspective with virtual reality glasses. In this application, six phases, each lasting between five and 40 minutes, were completed in three weeks. The application was narrated by a virtual therapist.

Gujjar et al. (2019), in their intervention for dental phobia, created a virtual environment with a virtual reality simulator and a head-mounted screen that allows participants to explore the virtual environment when they change their viewing angle. To increase realism, cotton wool soaked in clove oil was placed around the dentist's chair to create a standard odor related to the environment. In the virtual environment, the stages of a dentist approaching the participant, seeing the syringe and dental drill, and then hearing the sound of the drill were presented in a single session. The session duration was approximately 40 minutes.

In a study on Blood-Injection-Injury phobia (Jiang et al. 2020), a virtual reality intervention; an interactive 360° virtual reality environment including a smartphone, virtual reality glasses and different medical conditions was created with a software program. In the single-session intervention lasting 90 minutes in total, first, various medical procedures were hierarchically ranked by the participants in terms of anxiety level. Then, the intervention was initiated with the least anxiety-inducing procedure. The intervention was implemented by clinical psychologists.

Effectiveness of Interventions

The results of studies on the treatment of spider phobia indicated that virtual reality intervention was more effective than the waitlist (Garcia-Palacios et al. 2002) and placebo control group (Minns et al. 2019) in reducing spider fear level and avoidance behaviors. Miloff et al. (2019) reported that both virtual reality and experiential exposure were effective in reducing spider avoidance behaviors. Participants in the virtual reality exposure group experienced significantly less improvement; however, the difference between the groups decreased at the 12month follow-up measurement. Finally, Michaliszyn et al. (2010) revealed that virtual reality exposure and experiential exposure significantly reduced fear of spiders and avoidance behavior compared to the waitlist control group; however, experiential exposure was more effective on the improvement in negative beliefs about spiders. The effects were maintained at the three-month follow-up measurement.

When the results of the intervention for flight phobia were reviewed, both the virtual reality and experiential exposure treatment groups were more effective in reducing flight phobia symptoms compared to the waiting list control group; however, there was no significant difference between the two intervention groups (Rothbaum et al. 2000, Rothbaum et al. 2006). While the gains were maintained at six (Rothbaum et al. 2000, Rothbaum et al. 2006) and 12 months (Rothbaum et al. 2006) follow-up measurements, the rate of boarding an airplane was higher in the virtual reality group (79% - 81%) compared to the standard treatment group (69% - 70%). In Wiederhold et al.'s (2002) study, both the virtual reality exposure group (two conditions with and without physiological feedback) and the systematic desensitization group significantly decreased their fear and anxiety levels about flying. However, no significant difference was found between the interventions. At the three-month follow-up measurement, all participants (100%) in the condition including the virtual reality exposure intervention and physiological feedback, 80% of those in the condition without physiological feedback, and 10% of those in the standard treatment group reported being able to board an airplane.

The data from the study on height phobia (Donker et al. 2019) suggested that Cognitive-Behavioral Therapybased virtual reality exposure provided a significantly greater reduction in height phobia symptoms compared to the waitlist control group from the post-intervention measurement to the three-month follow-up measurement. In the intervention for dental anxiety and avoidance behaviors, virtual reality exposure was significantly more effective than the placebo control group. Treatment benefits were maintained at three- and six-month follow-up measurements (Gujjar et al. 2022). In blood-injection-injury phobia, virtual reality exposure was significantly more effective in reducing the fear of injection, injury and fainting compared to the waiting list control group. There was no significant difference between the groups at the level of fear of sharp objects, hospitals, injury or blood (Jiang et al. 2020). Treatment benefits were maintained at the three-month follow-up measurement.

Discussion

Technological developments in recent years have enabled the development of new methods in psychotherapy practices. One of the prominent applications in this sense is exposure therapy based on virtual reality technology, which is used in the treatment of various psychological problems. Exposure therapy with virtual reality is increasingly used especially in the treatment of specific phobias. This study conducted a systematic literature review on virtual reality-based exposure therapy for the treatment of individuals with different types of specific phobias. Based on the review, 10 studies that were determined to meet the inclusion criteria were examined in terms of sample characteristics, study designs, assessment tools, characteristics of intervention approaches and the effects of interventions.

The sample size of the studies differed between the studies. However, only one study (Garcia-Palacios et al. 2002) had less than 30 participants. In general, the inclusion of larger sample groups in the studies allows for more meaningful inferences in terms of the generalizability of the effect of the interventions applied (Page and Coxon 2016). The rate of female participants was higher in all the studies evaluated. Considering the results of prevalence studies (Ertan 2008, Sigström et al. 2011; Wardenaar et al. 2017), which reveal that the prevalence of specific phobias is higher in women (Dickinson et al. 2012), this finding is plausible. However, more balanced studies are warranted in terms of gender distribution, as the proportion of male participants was generally below 10% in the studies examined.

Four of the studies (40%) consisted of participants diagnosed with spider phobia and three (30%) with flight phobia. The remaining three studies (30%) included participants diagnosed with height phobia, blood-injection phobia and dental phobia, each diagnostic group in separate studies. Therefore, virtual reality exposure therapy appears to be a flexible intervention approach that can be applied to different types of problems. Since the studies were applied to different types of specific phobias, the measurement tools used to include the participants in the study and to evaluate the effect of the intervention also differed. In all studies, both selfreport measures and clinician assessments were used. Wiederhold et al. (2002) included physiological measurements (skin resistance, heart rate, etc.) in addition to self-report measures and clinician assessments. Therefore, physiological measurements are crucial in the holistic evaluation of the effectiveness of the intervention. While measurements were performed before and after the intervention in all studies, only two studies (Garcia-Palacios et al. 2002, Minns et al. 2019) did not include follow-up measurements. In all the remaining studies, follow-up measurements were taken for a minimum of 3 and a maximum of 12 months. In long-term follow-up studies, the validity of the results may be negatively affected due to the possibility of participant loss (Kristman et al. 2004, von Allmen et al. 2015). On the other hand, follow-up measurements in randomized controlled trials are important in terms of both detecting the long-term effect of the intervention and evaluating possible outcomes that are difficult to observe immediately after the intervention (LlewellynBennett et al. 2016). In this sense, longer follow-up measurements are required to reveal the effects of these interventions.

Virtual reality-based interventions in all the studies reviewed were carried out through various software to create a virtual reality environment, virtual reality goggles, headsets and screens. The aim was to provide the participants with stimuli for different senses, including visual, tactile, olfactory and auditory, to experience fear/anxiety in the closest way to reality. The addition of different sensory inputs in virtual reality exposure therapy has been found to increase the participant's level of experiencing the virtual environment realistically and the duration of remembering the objects in the environment (Dinh et al. 1999). While visual and auditory stimuli were provided in all studies included in this review, some studies also used olfactory (Gujjar et al. 2019) and tactile (Garcia-Palacios et al. 2002, Gujjar et al. 2019) stimuli to increase the effect of the intervention. However, these studies do not include an assessment of whether sensory stimuli enhance the effect of the intervention. Therefore, it is crucial to determine the specific effect of different types of sensory stimuli in future studies. In addition, in the included studies, participants were exposed to the feared object or situation either in a self-directed manner or under the guidance of an expert. In two studies (Donker et al. 2019, Miloff et al. 2019), participants completed the procedure themselves without the need for an expert during the therapy. This shows that virtual reality-based exposure intervention has become easier and more accessible. Particularly for individuals who have privacy concerns about receiving face-to-face therapy, virtual reality-based exposure intervention is likely to be an alternative for accessing treatment.

As part of the inclusion criteria, exposure therapy with virtual reality was applied in all intervention groups, while the effectiveness of the intervention was evaluated with different types of control groups. This is of significance in terms of interpreting the results of the intervention. In the five studies in which no active intervention was applied to the control group, participants in the intervention group had significantly greater reductions in height phobia, fear of spiders and spider avoidance, dental anxiety and avoidance of dental situations, injection, injury and fainting symptoms compared to the control group. Although virtual reality exposure was generally effective in reducing specific phobia symptoms compared to control groups without active treatment, the absence of not only the intervention but also a therapeutic context in research with such control groups leads to difficulties in interpreting the findings (Karlsson and Bergmark 2015). The therapeutic context includes elements based on the relationship between the therapist and the client (e.g. therapeutic alliance, empathy, etc.), which are reported to have a significant contribution to treatment outcomes (Norcross and Wampold 2011). Therefore, in studies using waiting list and placebo control groups, it may be difficult to determine the specific effect of both the intervention and the relationship-based elements between therapist and client on treatment outcomes (Wampold 2001, Elkins 2012). On the other hand, in studies where the control group received standardized treatment, both research groups included these elements that could potentially affect the therapeutic relationship. Therefore, it is becoming clearer whether the results obtained as a result of the intervention depend on the intervention techniques applied (Karlsson and Bergmark 2015).

Five studies comparing the effectiveness of virtual reality exposure intervention with standard treatment control groups, such as experiential exposure and systematic desensitization, pointed to the effectiveness of both interventions in reducing symptoms of fear of spiders, spider avoidance and flight phobia. The findings suggest that the effectiveness of virtual reality exposure in specific phobias is similar to that of standard treatments. In this sense, the fact that virtual reality-based exposure was found to be comparable to experiential exposure may reflect that the participants experienced the virtual reality environment realistically. This is supported by the finding that realistic visual and auditory stimuli in virtual reality exposure are associated with more favorable therapy outcomes (Price et al. 2011). The fact that the virtual environment allows for more focused and controlled exposure may be an essential factor in the reduction of symptoms. On the other hand, in one study (Miloff et al. 2019), standard treatment was reported to be more effective than virtual reality-based intervention; however, this difference decreased in the follow-up measurement. In this respect, the improvements observed in the follow-up measurements of the participants in the virtual reality-based intervention group are noteworthy. For example, in studies on flight phobia (Rothbaum et al. 2000, Wiederhold et al. 2002, Rothbaum et al. 2006), participants in the intervention group had a higher rate of boarding an airplane in the long term. The ease of virtual reality-based intervention in terms of diversity in creating exposure contexts for researchers may be related to the increase in the positive effects of the interventions in the long term. Studies examining this relationship indicate that exposure in more than one context with virtual reality may reduce the relapse rate by increasing generalization (Vansteenwegen et al. 2006, Shiban et al. 2013, Dunsmoor et al. 2014). As a conclusion, exposure in more than one context is recommended to maintain the gains achieved in treatment and thus reduce the likelihood of relapse.

The reviewed research findings suggest that virtual reality exposure therapy is effective in reducing specific phobia symptoms. Studies have demonstrated that virtual reality-based interventions for specific phobias have some advantages over standard face-to-face interventions. One prominent advantage is that the virtual reality environment allows for a higher level of control compared to real-life conditions. For example, in imaginal exposure, the kind of imagery the participant creates is unknown. However, the decrease in the ability to form imagery due to aging makes it difficult to apply these interventions in elderly people (Grenier et al. 2015). Especially in animal-type specific phobias, the ability to appropriately control the movements of stimuli provides convenience for both participants and therapists (Suso-Ribera et al. 2019). Another advantage of the virtual reality-based application is its economical use of time and space. For example, while organizing experiential exposure in an office environment can be quite challenging, conditions may not always be favorable for out-ofoffice exposure (Boeldt et al. 2019). In addition, in an exposure intervention for flight phobia, therapists reported both temporal and financial difficulties such as buying plane tickets, going to the airport, flying for a certain period of time (Rothbaum et al. 2000). In some cases, these advantages are assumed to gain even more importance as repeated exposure to fear/anxiety situations is necessary.

The literature review shows that studies done up to 20 years ago (Garcia-Palacios et al. 2002) reported that it was costly to create a virtual reality environment. Today, software and hardware for creating virtual reality are more cost-effective; in addition, the quantity and quality of virtual reality content is gradually increasing (Boeldt et al. 2019). In this sense, exposure therapy with virtual reality, whose effectiveness in specific phobias is supported, is expected to become more widespread. Virtual reality-based applications can be considered advantageous in terms of making the treatment more accessible, especially for individuals whose access to therapy is limited for various reasons.

This systematic review study has some limitations. These limitations include the fact that only English language studies were found and that studies on individuals in childhood and adolescence were not included. In addition, the fact that only 10 studies with randomized controlled design were found is an indication that more studies are needed. On the other hand, although the high level of control in the conditions of the studies in this design enables attributing the treatment results to the intervention rather than other factors, it constitutes a limitation in terms of the generalizability of the findings to the real world (Depp and Lebowitz 2007). For this reason, future studies that include reviews of studies with different designs are anticipated to contribute to the literature.

Conclusion

The studies included in this systematic review indicate the effectiveness of virtual reality exposure therapy in different types of specific phobia symptoms (spider phobia, blood-injection phobia, flight phobia, dental phobia). In other words, virtual reality exposure therapy is comparable to standard therapy methods and more effective than control conditions without treatment. Considering the advantages such as creating a safe, controllable and reproducible environment, expanding the use of virtual reality application in the field of psychotherapy may be beneficial. Finally, it is noteworthy that there is no randomized controlled design study in Turkey and there is a need for further research on this subject.

Sidebar

Address for Correspondence: Merve Demir, Aydın Adnan Menderes University Faculty of Humanities and Social Sciences Department of Psychology, Aydın, Türkiye E-mail: [email protected]

Received: 21.10.2022 | Accepted: 15.01.2023

Authors Contributions: The author(s) have declared that they have made a significant scientific contribution to the study and have assisted in the preparation or revision of the manuscript

Peer-review: Externally peer-reviewed.

Conflict of Interest: No conflict of interest was declared.

Financial Disclosure: No financial support was declared for this study.

References

References

Albakri G, Bouaziz R, Alharthi W, Kammoun S, Al-Sarem M, Saeed F et al. (2022) Phobia exposure therapy using virtual and augmented reality: a systematic review. Appl Sci, 12:1672.

APA (2013) Diagnostic and Statistical Manual of Mental Disorders, 5th ed. Washington D.C., American Psychiatric Association.

American Psychological Association, Presidential Task Force on Evidence-Based Practice (2006) Evidence-based practice in psychology. Am Psychol, 61:271-285.

Anderson PL, Rothbaum BO, Hodges L (2001) Virtual reality: using the virtual world to improve quality of life in the real world. Bull Menninger Clin, 65:78-91.

Antony MM, Barlow DH (2002) Specific phobias. In Anxiety and its Disorders - The Nature and Treatment of Anxiety and Panic (Ed. DH Barlow): 380-417. London, Guilford Press.

Barlow DH, Conklin LR, Bentley KH (2015) Psychological treatments for panic disorders, phobias, and social and generalized anxiety disorders. In A Guide to Treatments That Work (Eds PE Nathan, JM Gorman): 409-461. New York, Oxford University Press.

Boeldt D, McMahon E, McFaul M, Greenleaf W (2019) Using virtual reality exposure therapy to enhance treatment of anxiety disorders: identifying areas of clinical adoption and potential obstacles. Front Psychiatry, 10:773.

Botella C, Fernández-Álvarez J, Guillén V, García-Palacios A, Baños R (2017) Recent progress in virtual reality exposure therapy for phobias: a systematic review. Curr Psychiatry Rep, 19:42.

Chambless DL, Baker MJ, Baucom DH, Beutler LE, Calhoun KS, Daiuto A (1998) Update on empirically validated therapies, II. Clin Psychol, 51:3-16.

Choy Y, Fyer AJ, Lipsitz JD (2007) Treatment of specific phobia in adults. Clin Psychol Rev, 27:266-286.

Cohen DC (1977) Comparison of self-report and overt-behavioral procedures for assessing acrophobia. Behav Ther, 8:17-23.

Depp C, Lebowitz BD (2007) Clinical trials: bridging the gap between efficacy and effectiveness. Int Rev Psychiatry, 19:531539.

Dickinson ER, Adelson, JL, Owen J (2012) Gender balance, representativeness, and statistical power in sexuality research using undergraduate student samples. Arch Sex Behav, 41:325-327.

Dinh HQ, Walker N, Hodges LF, Song C, Kobayashi A (1999) Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments (p. 222-228). IEEE Comput Soc, Houston.

Donker T, Cornelisz I, Van Klaveren C, Van Straten A, Carlbring P, Cuijpers P et al. (2019) Effectiveness of self-guided appbased virtual reality cognitive behavior therapy for acrophobia: a randomized clinical trial. JAMA Psychiatry, 76:682-690.

Dunsmoor JE, Ahs F, Zielinski DJ, LaBar KS (2014) Extinction in multiple virtual reality contexts diminishes fear reinstatement in humans. Neurobiol Learn Mem, 113:157-164.

Eaton WW, Bienvenu OJ, Miloyan B (2018) Specific phobias. Lancet Psychiatry, 5:678-686.

Eaton WW, Dryman A, Weissman MM (1991) Panic and phobia. In Psychiatric Disorders in America: The Epidemiologic Catchment Area Study (Eds LA Robins, DA Regier):155-179. Los Angeles, CA, Free Press.

Elkins DN (2012) Toward a common focus in psychotherapy research. Psychotherapy, 49:450-454.

Ertan T (2008) Psikiyatrik bozukluklarin epidemiyolojisi. Türkiye'de Sık Karşılaşılan Psikiyatrik Hastalıklar Sempozyumu (Eds M Uǧur, Í Balcıoǧlu, N Kocabaşoǧlu):25-30. İstanbul, İstanbul Üniversitesi.

Freitas JRS, Velosa VHS, Abreu LTN, Jardim RL, Santos JAV, Peres B et al. (2021) Virtual reality exposure treatment in phobias: a systematic review. Psychiatr Q, 92:1685-1710.

Garcia-Palacios A, Botella C, Hoffman H, Fabregat S (2007) Comparing acceptance and refusal rates of virtual reality exposure vs. in vivo exposure by patients with specific phobias. Cyberpsychol Behav, 10:722-724.

Garcia-Palacios A, Hoffman HG, See SK, Tsai A, Botella C (2001) Redefining therapeutic success with virtual reality exposure therapy. Cyberpsychol Behav, 4:341-348.

Garcia-Palacios A, Hoffman H, Carlin A, Furness III TA, Botella C (2002) Virtual reality in the treatment of spider phobia: a controlled study. Behav Res Ther, 40:983-993.

Geraets CN, Van der Stouwe EC, Pot-Kolder R, Veling W (2021) Advances in immersive virtual reality interventions for mental disorders: a new reality?. Curr Opin Psychol, 41:40-45.

Götestam KG (2002) One session group treatment of spider phobia by direct or modelled exposure. Cogn Behav Ther, 31:1824.

Grenier S, Forget H, Bouchard S, Isere S, Belleville S, Potvin O et al. (2015) Using virtual reality to improve the efficacy of cognitive-behavioral therapy (CBT) in the treatment of late-life anxiety: preliminary recommendations for future research. Int Psychogeriatr, 27:1217-1225.

Gujjar KR, van Wijk A, Kumar R, de Jongh A (2019) Efficacy of virtual reality exposure therapy for the treatment of dental phobia in adults: a randomized controlled trial. J Anxiety Disord, 62:100-108.

Hofmann SG, Smits JA (2008) Cognitive-behavioral therapy for adult anxiety disorders: a meta-analysis of randomized placebo-controlled trials. J Clin Psychiatry, 69:621-632.

Hood HK, Antony MM (2012) Evidence-based assessment and treatment of specific phobias in adults. In Intensive OneSession Treatment of Specific Phobias (Eds TE Davis III, TH Ollendick, LG Öst): 19-41. New York, NY, Springer.

Humphris GM, Morrison T, Lindsay SJE (1995) The modified dental anxiety scale: validation and United Kingdom norms. Community Dent Health, 12:143-50.

Jiang MY, Upton E, Newby JM (2020) A randomised wait-list controlled pilot trial of one-session virtual reality exposure therapy for blood-injection-injury phobias. J Affect Disord, 276:636-645.

Karlsson P, Bergmark A (2015) Compared with what? an analysis of control-group types in cochrane and campbell reviews of psychosocial treatment efficacy with substance use disorders. Addiction, 110:20-428.

Kaussner Y, Kuraszkiewicz AM, Schoch S, Markel P, Hoffmann S, Baur-Streubel R et al. (2020) Treating patients with driving phobia by virtual reality exposure therapy - a pilot study. PLoS One, 15:e0226937.

Kendler KS, Myers J, Prescott CA (2002) The etiology of phobias: an evaluation of the stress-diathesis model. Arch Gen Psychiatry, 59:242-248.

Kessler RC, Petukhova M, Sampson, NA, Zaslavsky AM, Wittchen HU (2012) Twelve-month and lifetime prevalence and lifetime morbid risk of anxiety and mood disorders in the United States. Int J Methods Psychiatr Res, 21:169-184.

Kristman V, Manno M, Côté P (2004) Loss to follow-up in cohort studies: how much is too much?. Eur J Epidemiol, 19:751760.

Krzystanek M, Surma S, Stokrocka M, Romańczyk M, Przybyło J, Krzystanek N et al. (2021) Tips for effective implementation of virtual reality exposure therapy in phobias - a systematic review. Front Psychiatry, 12:737351.

Llewellyn-Bennett R, Bowman L, Bulbulia R (2016) Post-trial follow-up methodology in large randomized controlled trials: a systematic review protocol. Syst Rev, 5:214.

Magee WJ, Eaton WW, Wittchen HU, McGonagle KA, Kessler RC (1996) Agoraphobia, simple phobia, and social phobia in the National Comorbidity Survey. Arch Gen Psychiatry 53:159-168.

Maples-Keller JL, Bunnell BE, Kim SJ, Rothbaum BO (2017) The use of virtual reality technology in the treatment of anxiety and other psychiatric disorders. Harv Rev Psychiatry, 25:103-113.

Marks IM (1978) Behavioral psychotherapy of adult neurosis. In Handbook of Psychotherapy and Behavior Change (Eds SL Garfield, AE Bergin):493-547). Chichester, Wiley.

Michaliszyn D, Marchand A, Bouchard S, Martel MO, Poirier-Bisson J (2010) A randomized, controlled clinical trial of in virtuo and in vivo exposure for spider phobia. Cyberpsychol Behav Soc Netw, 13:689-695.

Miloff A, Lindner P, Dafgård P, Deak S, Garke M, Hamilton W et al. (2019) Automated virtual reality exposure therapy for spider phobia vs. in-vivo onesession treatment: a randomized non-inferiority trial. Behav Res Ther, 118:130-140.

Mineka S, Sutton J (2006) Contemporary learning theory perspectives on the etiology of fears and phobias. Fear and Learning: From Basic Processes to Clinical Implications (Eds MG Craske, D Hermans, D Vansteenwegen): 75-97). Washington DC, American Psychological Association.

Minns, S, Levihn-Coon A, Carl E, Smits JA, Miller W, Howard D et al. (2019) Immersive 3D exposure-based treatment for spider fear: a randomized controlled trial. J Anxiety Disord, 61:37-44.

Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M et al. (2015) Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev, 4:1.

Nathan PE, Gorman JM (2015) A Guide to Treatments That Work. New York, Oxford University Press.

Neudeck P, Einsle F (2012) Dissemination of exposure therapy in clinical practice: how to handle the barriers?. In Exposure Therapy: Rethinking the Model - Refining the Method (Eds. P Neudeck, HU Wittchen):23-34. New York, NY, Springer.

Norcross JCW, Bruce E (2011) Evidence-based therapy relationships: research conclusions and clinical practices. Psychotherapy, 48:98-102.

North M, North S (1994) Virtual environments and psychological disorders. Electronic Journal of Virtual Culture, 2:37-42.

North MM, North SM, Coble JR (1996) Effectiveness of virtual environment desensitization in the treatment of agoraphobia. Presence (Camb), 5:346-352.

Oing, T, Prescott J (2018) Implementations of virtual reality for anxiety-related disorders: systematic review. JMIR Serious Games, 6:e10965.

Page S, Coxon M (2016) Virtual reality exposure therapy for anxiety disorders: small samples and no controls?. Front Psychol, 7:326.

Powers MB, Emmelkamp PM (2008) Virtual reality exposure therapy for anxiety disorders: a meta-analysis. J Anxiety Disord, 22:561-569.

Pull CB (2008) Recent trends in the study of specific phobias. Curr Opin Psychiatry, 21:43-50.

Price M, Mehta N, Tone EB, Anderson PL (2011) Does engagement with exposure yield better outcomes? Components of presence as a predictor of treatment response for virtual reality exposure therapy for social phobia. J Anxiety Disord, 25:763-770.

Richardson WS, Wilson MC, Nishikawa J, Hayward RS (1995) The well-built clinical question: a key to evidence-based decisions. ACP J Club, 123:A12-A13.

Rothbaum BO, Anderson P, Zimand E, Hodges L, Lang D, Wilson J (2006) Virtual reality exposure therapy and standard (in vivo) exposure therapy in the treatment of fear of flying. Behav Ther, 37:80-90.

Rothbaum BO, Hodges LF (1999) The use of virtual reality exposure in the treatment of anxiety disorders. Behav Modif, 23:507-525.

Rothbaum BO, Hodges L, Smith S, Lee JH, Price L (2000) A controlled study of virtual reality exposure therapy for the fear of flying. J Consult Clin Psychol, 68:1020.

Shiban Y, Pauli P, Mühlberger A (2013) Effect of multiple context exposure on renewal in spider phobia. Behav Res Ther, 51:68-74.

Sigström R, Östling S, Karlsson B, Waern M, Gustafson D, Skoog I (2011) A population-based study on phobic fears and DSM-IV specific phobia in 70-year olds. J Anxiety Disord, 25:148-153.

Spring B (2007) Evidence-based practice in clinical psychology: what it is, why it matters; what you need to know. J Clin Psychol, 63:611-631.

Suso-Ribera C, Fernández-Álvarez J, García-Palacios A, Hoffman HG, Bretón-López J, Banos RM et al. (2019) Virtual reality, augmented reality, and in vivo exposure therapy: a preliminary comparison of treatment efficacy in small animal phobia. Cyberpsychol Behav Soc Netw, 22: 31-38.

Steinman SA, Teachman BA (2011) Cognitive processing and acrophobia: validating the Heights Interpretation Questionnaire? J Anxiety Disord, 25:896-902.

Szymanski J, O'Donohue W (1995) Fear of spiders questionnaire. J Behav Ther Exp Psychiatry, 26:31-34.

Trappey A, Trappey CV, Chang CM, Kuo RR, Lin AP, Nieh CH (2020) Virtual reality exposure therapy for driving phobia disorder: system design and development. Appl Sci, 10:4860.

Wampold BE (2001) The Great Psychotherapy Debate: Models, Methods, and Findings. Mahwah, NJ, Lawrence Erlbaum.

Wardenaar KJ, Lim C, Al-Hamzawi AO, Alonso J, Andrade LH, Benjet C et al. (2017) The cross-national epidemiology of specific phobia in the World Mental Health Surveys? Psychol Med, 47:1744-1760.

Wechsler TF, Kümpers F, Mühlberger A (2019) Inferiority or even superiority of virtual reality exposure therapy in phobias? - a systematic review and quantitative meta-analysis on randomized controlled trials specifically comparing the efficacy of virtual reality exposure to gold standard in vivo exposure in agoraphobia, specific phobia, and social phobia. Front Psychol, 10:1758.

Wiederhold BK, Jang DP, Gevirtz RG, Kim SI, Kim IY, Wiederhold MD (2002) The treatment of fear of flying: a controlled study of imaginal and virtual reality graded exposure therapy. IEEE Trans Inf Technol Biomed, 6:218-223.

Wiederhold BK, Wiederhold MD (2004) Virtual Reality Therapy for Anxiety Disorders: Advances in Evaluation and Treatment. New York, American Psychological Association Press.

Vansteenwegen D, Vervliet B, Hermans D, Beckers T, Baeyens F, Eelen P (2006) Stronger renewal in human fear conditioning when tested with an acquisition retrieval cue than with an extinction retrieval cue. Behav Res Ther, 44:1717-1725.

Vervliet B, Craske MG, Hermans D (2013) Fear extinction and relapse: state of the art. Annu Rev Clin Psychol, 9:215-248.

Von Allmen RS, Weiss S, Tevaearai HT, Kuemmerli C, Tinner C, Carrel TP et al. (2015) Completeness of follow-up determines validity of study findings: results of a prospective repeated measures cohort study. PloS One, 10:e0140817.