Introduction
Between 30% and 50% of people attending primary care (PC) in Spain have mental health problems, with a large majority demonstrating mild-to-moderate anxiety and depressive symptoms, usually work-related [1,2]. Only 5% of these people will eventually require specialized mental health care [3]. Nevertheless, access to specialized mental health care can take up to 3 months because of the high number of referrals and lack of resources [4-6], including personnel, consultation times, and lack of training of general practitioners (GPs) to address mental health problems [7]. Considering this complex and problematic challenge, during the last decade, the Spanish National Health System has launched a Primary Care Mental Health Support Program (PCMHSP) composed of mental health specialists, including psychiatrists, psychologists, and nurses, who dedicate part of their workweek to PC [8]. Nonetheless, the care demand still far exceeds the resources currently available at PC, representing a global challenge across the international community [9,10]. Consequently, collaborative initiatives are increasingly proposed to address this issue [11].
The unresolved delay in receiving specialized attention added to the lack of resources in PC, may lead to an increased and unnecessary risk of exacerbation of symptoms, affecting the quality of life of patients and potentially increasing sick leaves, and, in the worst-case scenario, irreparable consequences such as suicide [12]. Evidence shows that up to 30% of patients with depression reported not receiving any type of care in the previous year [13]. This situation has led to a substantial increase in the unnecessary and premature prescription of psychotropic drugs (up to 50% of people attending PC) [14-17], and superfluous referrals to specialized care and a lack of early detection of severe mental health problems.
In addition to this already challenging situation, the COVID-19 pandemic has caused a considerable impact on the mental health of the general population, with increases of > 25% in both anxiety and depressive disorders [18]. This has exacerbated the existing problems of mental health care in PC [19-21], and also impacted health care workers, especially those in the front line [22-25]. In particular, health care workers have reported increased rates of anxiety, depression, and posttraumatic stress disorder (PTSD) symptoms [26-28], with potentially severe mental health consequences [25,29,30]. This increase in mental health problems in health care workers is related to an overload of work and includes symptoms such as mental and physical exhaustion, depersonalization, diminished sense of personal accomplishment, and reduced professional functioning, thus fulfilling the criteria of work-related burnout [31,32], which is usually comorbid with symptoms of anxiety and depression such as fatigue, insomnia, and sadness [33,34]. In fact, work-related burnout is a major issue among the general population, particularly in health care workers [35], which can lead to increased sick leaves with a severe economic impact [2], or even major consequences, such as suicide.
This sudden worldwide increase in the demand for mental health care has proven challenging, with essential mental health services being disrupted [36]. Digital psychological tools have been proposed as a potential solution [37-39]. In fact, some digital interventions through smartphone apps have proven effective in improving anxiety [40], depressive symptoms [41], and work-related burnout [42-44], especially in PC settings [45,46], and also for people with severe mental disorders such as bipolar disorder [47,48].
More recently, chatbots (virtual assistants with artificial intelligence), by directly implementing Natural Language Processing (NLP) techniques and powered by artificial intelligence, have been shown to increase engagement, usability, and effectiveness of these interventions by offering a more friendly and personalized approach to users [49,50]. In people with PTSD (eg, war veterans), the anonymous nature of chatbots has also effectively increased the disclosure of traumatic events [51]. This may be of particular interest to health care workers who have been exposed to extreme and continuous pressure during the COVID-19 pandemic. In addition, NLP techniques have proven useful for both the design of smartphone apps [52] and for identifying differential patient profiles to tailor specific psychological interventions according to these characteristics [53]. Apps and chatbots allow continuous psychological intervention that adapts to the needs, schedules, and status of the patient in a dynamic way based on the progression of the patient, with the possibility of capturing active or passive data related to symptom evolution [54,55].
The primary care digital support tool in mental health (PRESTO) project [56] intends to mitigate the growing problem of mental health problems in PC and among health care workers by developing a digital decision support platform combining machine-learning severity prediction models (phase 1) with a smartphone-based app for screening, monitoring, and delivering evidence-based psychological interventions to reduce anxiety, depressive symptoms, and work-related burnout (phase 2).
In this work, we present the pilot results of phase 2: the development and studies of feasibility and potential effectiveness in PC patients and health care workers. The primary aim of this study was to assess the feasibility of the intervention. Secondary aims included assessing (1) its use in screening and monitoring anxiety and depressive symptoms and work-related burnout, (2) the potential effectiveness of the intervention in reducing them, and (3) the ability of the intervention to detect suicide risk.
Methods
The “Vickybot” Intervention: Development and Functions
We developed a chatbot intended to screen and monitor anxiety and depressive symptoms and work-related burnout while providing evidence-based psychological interventions. User-centered development strategies have been adopted with evidence of improved user retention and satisfaction with the end product and increased chances of engagement in long-term real-world clinical settings [57]. Considering the preferences and recommendations of patients and health care workers presented in the protocol [56], the research team outlined the priority aspects of the digital tool as well as the psychological contents to be included. Its main functions included were as follows (Figure 1):
Enlarge this image.Figure 1. Scheme of the main functions of the intervention Vickybot. Please see Multimedia Appendix 1 for a larger version.
To consolidate the development and evaluate the feasibility and potential effectiveness of the intervention, the following phases were conducted sequentially.
Setup Phase
Over the course of 2 weeks, 40 healthy controls (HCs) used Vickybot and were asked to test the different functionalities without specific instructions (eg, responding to self-assessments, accessing the psychological modules, setting reminders, etc). A technical test was performed to evaluate the stability and reliability of the data transmission between devices and servers. The tolerance of calls per minute to the server was verified, and logs of bugs were also collected. HCs were recruited via advertisements, and psychiatric disorders were excluded through semistructured interviews.
Simulation Phase
A total of 17 HCs, all participants from the previous setup phase, were instructed to use the Vickybot for 2 more weeks and simulate different clinical situations indicated by the research team to elicit specific functions from the chatbot. Clinical simulations were designed to identify specific clinical profiles (severe-moderate-mild anxiety and depressive symptoms, work-related burnout, isolated symptoms, suicidal thoughts, etc), as shown in Table 1. The data recorded on the server were correlated with the patterns identified in each clinical situation. In addition, the treatment algorithm for customizing the psychological modules was tested and calibrated for each simulation. Finally, the baseline patterns of each simulator were compared with their own follow-up outcomes as a function of time to assess variability and longitudinal registers. After the simulation phase, the HCs were asked for their insights into the functions that they thought could be improved. Subjective user-engagement indicators (UEI; acceptability, usability, and satisfaction) were assessed using the Mobile Health App Usability Questionnaire (MAUQ) [70,71] (Multimedia Appendix 2), which includes 18 statements to determine the usability, acceptability, and satisfaction of mental health interventions.
Table 1. Clinical simulations and functions of the intervention assessed.
| User | Clinical situation to be simulated | Specific functions to be tested | Specific psychological modules to be activated |
| 1 | Severe anxiety symptoms | Self-assessments, treatment algorithm, modules | 2. Anxiety; 8. Overthinking; 9. Irritability; and 10. Sleep disturbance |
| 2 | Moderate anxiety symptoms | Self-assessments, treatment algorithm, modules | 2. Anxiety; 8. Overthinking; 9. Irritability; and 10. Sleep disturbance |
| 3 | Severe depressive symptoms | Self-assessments, treatment algorithm, modules | 1. Depressed mood; 3. Apathy-anhedonia; 4. Depressive cognitions; 5. Suicidal thoughts; 6. Restlessness; 7. Decreased concentration; and 10. Sleep disturbance |
| 4 | Moderate depressive symptoms without suicidal thoughts | Self-assessments, treatment algorithm, modules | 1. Depressed mood; 3. Apathy-anhedonia; 4. Depressive cognitions; 5. Suicidal thoughts; 6. Restlessness; 7. Decreased concentration; and 10. Sleep disturbance |
| 5 | Work-related burnout with moderate anxiety symptoms | Self-assessments, treatment algorithm, modules | 2. Anxiety; 8. Overthinking; 9. Irritability; 10. Sleep disturbance; 11. Work-related stress; and 12. Burnout |
| 6 | Work-related burnout with moderate depressive symptoms | Self-assessments, treatment algorithm, modules | 11. Work-related stress and 12. Burnout |
| 7 | Depressive symptoms with suicidal thoughts | Emergency alerts of suicidal thoughts (item-9 of the PHQ-9a and chatbot NLPb-intent detection), automatic alert to the research team | 5. Suicidal thoughts |
| 8 | Lack of anxiety-depressive symptoms and lack of work-related burnout | Tools and “null” function of the treatment algorithm | N/Ac |
| 9 | Insomnia (solely) | Self-assessments, treatment algorithm, modules | 10. Sleep disturbance |
| 10 | Recurring concerns (solely) | Self-assessments, treatment algorithm, modules | 8. Overthinking |
| 11 | Cognitive problems (solely) | Self-assessments, treatment algorithm, modules | 7. Decreased concentration |
| 12 | Irritability (solely) | Self-assessments, treatment algorithm, modules | 9. Irritability |
| 13 | Depressive cognitions such as guilt or lack of confidence or self-esteem (solely) | Self-assessments, treatment algorithm, modules | 4. Depressive cognitions |
| 14 | Apathy (solely) | Self-assessments, treatment algorithm, modules | 3. Apathy-anhedonia |
| 15 | Insomnia and irritability (both solely) | Self-assessments, specific combination of the treatment algorithm, modules | 9. Irritability and 10. Sleep disturbance |
| 16 | Depressive cognitions and recurring concerns (both solely) | Self-assessments, specific combination of the treatment algorithm, modules | 4. Depressive cognitions and 8. Overthinking |
| 17 | User with erratic behavior (contradictory self-assessment feedback, insults, skips the proposed modules, etc) | Self-assessments, treatment algorithm, modules; insult intent | N/A |
aPHQ-9: Patient Health Questionnaire-9.
bNLP: Natural Language Processing.
cN/A: not applicable.
Feasibility and Potential Effectiveness Study: Participants, Procedure, and Measures
Inclusion criteria were as follows: (1) people referred to the PCMHSP from the GP or health care workers from Hospital Clínic de Barcelona referred to specialized mental health care from the Department of Occupational Health, (2) aged between 18 and 65 years, (3) have a compatible smartphone and sufficient digital skills to use a chatbot (access, main functions use, etc), and (4) have accepted and signed the informed consent for study participation. Exclusion criteria were as follows: (1) previous or present diagnosis of a severe mental disorder and (2) risk of suicide detected during clinical consultations before the use of the chatbot.
Participation in the study was offered during a clinical consultation with either the GP (for people referred to the PCMHSP) or the Department of Occupational Health (for health care workers). Subsequently, our research team contacted potential participants, explained the details and conditions of the study, and resolved doubts. If participants consented to participate, they received an SMS invitation with access to a survey from Hospital Clínic de Barcelona, where they signed the informed consent form for participation, facilitated sociodemographic and clinical data, and were given access to Vickybot (Figure 2). All the participants used their personal phones, and no economic compensation was provided for their participation.
Enlarge this image.Figure 2. Recruitment procedure for the feasibility and potential effectiveness study.
After accessing Vickybot (Figure 1), the chatbot presented its functions and asked users to perform a baseline self-assessment of anxiety-depressive symptoms and work-related burnout. According to the results of the first self-assessment, a treatment plan consisting of psychological modules was generated for each user. In the following month, users were prompted to interact with the chatbot and perform their treatment plans sequentially using personalized reminders. Users were also advised to complete the self-assessments every 2 weeks by receiving a preconfigured reminder.
At the end of the feasibility and effectiveness study (1 month after the first self-assessment), similar to the simulation phase, users were prompted to assess their experience with the different functions of the chatbot. Subjective UEI (acceptability, usability, and satisfaction) was assessed using the MAUQ (Multimedia Appendix 2), and objective UEI (completion, adherence, compliance, and engagement) was retrieved from the servers. The feasibility was assessed based on a combination of subjective and objective UEIs.
Statistical Analyses
Statistical analyses were conducted using IBM SPSS Statistics for Windows Version 28.0 (IBM Corp). The analyses of all subjects were considered until their last interaction with the chatbot. Sociodemographic and clinical data of the sample, as well as details on the general use of the intervention and the specific functions used, were evaluated using descriptive analyses. The Shapiro-Wilk test and visual inspection of Q-Q plots were used to assess whether continuous variables were normally distributed.
The main outcomes for potential effectiveness were changes in anxiety-depressive symptoms and work-related burnout measured with self-assessment scales using paired 2-tailed t tests or the Wilcoxon signed-rank test. Subgroup analyses were conducted to assess the influence of baseline symptoms and the effect of engagement on intervention effectiveness. In addition, to assess the relationship between use and effectiveness, we performed Pearson correlations on the number of times that modules were completed and the days of use with the change in clinical scales’ scores. The results of the MAUQ at the end of the first month of use (acceptability, usability, and satisfaction), as well as other objective-use UEIs (completion, adherence, compliance, and engagement) and feasibility, were presented using descriptive analyses and analyzed according to the definitions recently proposed by the International Society for Bipolar Disorders Big Data Task Force [72]. Statistical significance was set at 2-tailed P≤.05.
Ethics Approval and Consent for Publication
The PRESTO project was approved by the Clinical Research Ethics Committee at the Hospital Clínic of Barcelona (HCB/2020/0735). All data were collected and stored in encrypted and secure servers following the guidelines and standards of the 2018 European General Data Protection Regulation and were solely managed by project researchers. The project is in line with the ethical standards of European experts on personalized interventions and precision psychiatry [73]. All participants provided written informed consent before inclusion in the study.
Results
Setup Phase
A total of 40 HCs tested Vickybot simultaneously. They used different functions of the intervention, with all functions tested by at least 7 users. Data were transmitted and registered correctly, and the servers proved reliable with a rapidly increasing number of users.
Simulation Phase
A total of 17 HCs (n=13, 76% female; mean age 36.5, SD 9.7 years) simulated different clinical situations while using Vickybot. Table 1 shows the specific clinical simulations and respective functions of the interventions to be assessed.
All simulators signed up, performed the first self-assessment, and obtained a treatment plan with specific psychological modules. According to the respective simulations, 98.8% of the expected modules were recommended to the users (Table 1). Only 1 module was not offered in the respective treatment plan (user 4) because it had a lower-than-expected score on the PHQ-9. Furthermore, 59% (10/17) of users were offered “extra” modules, which were not expected in the simulation. This was mostly because of high scores on the clinical scales by simulators who were asked to report specific symptoms solely (users 9-16). The expected simulated severity of anxiety and depressive symptoms was achieved by 75% of the users (1-4) with 1 scoring below the expected severity (user 4). In total, 98.5% of the expected functions (Table 1) were tested by simulators according to each clinical situation with the expected outcomes. All simulators showed a progressive reduction in scales’ scores according to the clinical simulation of symptom reduction. This was not the case for 1 user (user 17) who was instructed to simulate “erratic behavior.” Suicidal alerts were correctly activated by expected users (4 and 7) and received by the research team. Overall, 53% (9/17) of users set up “weekly objectives,” of which 67% (6/9) rated their daily completion. Moreover, 65% (11/17) of users set reminders, and 53% (9/17) of users registered an audio file after self-assessment.
A total of 65% (11/17) of HCs responded to the MAUQ after the simulation phase, with a mean score of 6.39 (SD 0.36) on a scale from 1 (strongly disagree) to 7 (strongly agree). All simulators (34/34, 100%) agreed with 14 of 18 statements of the MAUQ questionnaire. Most patients (72%-90%) agreed with the remaining 4 of the 18 statements. Vickybot showed high usability, satisfaction, and acceptability among the simulators. The most recurrently perceived weak points to be improved were the lack of reminders, the user’s lack of capacity to configure or personalize them, the lack of incentives to answer the self-assessments or access the psychological modules, and not being understood by the chatbot many times.
Feasibility and Effectiveness Study
Overview
More than 300 patients ( > 150 from PC and > 150 from health care workers) were offered participation in the study by their GPs or personnel from the Department of Occupational Health, respectively. After being contacted and receiving instructions from our research team, 130 patients (55 from PC and 75 health care workers) received an SMS invitation. A total of 34 patients (15 from PC and 19 from health care workers; 26.2% (34/130) of those who received the SMS invitation) signed up and performed the first self-assessments, with 100% (34/34) of patients presenting with low-to-severe anxiety symptoms, 94% (32/34) with depressive symptoms, and 65% (22/34) reported work-related burnout.
Table 2 provides a summary of the sample characteristics. Most of the patients were female (26/34, 77%). Mean age was 35.3 (SD 10.1) years. A minority of participants used alcohol or cannabis ( < 6%). More than 30% of the patients had medical comorbidities (17/34, 47%) or were prescribed nonpsychiatric drugs (13/34, 38%). Overall, 88% (30/34) had current psychiatric diagnoses, most commonly anxiety (18/34, 53%), followed by adjustment and depressive disorders (7/34, 21%), and 82% (28/34) had currently prescribed psychiatric drugs.
A total of 56% (19/34) of patients used Vickybot for several days, with a mean of 15.3 (SD 17.6) days of use. Furthermore, 27% (9/34) of patients completed the second self-assessment after 2 weeks of use, and 6% (2/34) completed the third self-assessment after 4 weeks of use. The longitudinal self-assessment scores are presented in Table 3. The Shapiro-Wilk test and visual examination of the Q-Q plots showed that the GAD-7 and PHQ-9 scores had a normal distribution, whereas the distribution of work-related burnout scores deviated from normality.
Table 2. Characteristics of the sample (N=34).
| Sociodemographic variables | Value | |||
| Age (years), mean (SD) | 35.3 (10.12) | |||
| Sex (female), n (%) | 26 (77) | |||
| Medical record (yes), n (%) | ||||
| Medical comorbidities | 17 (47) | |||
| Nonpsychiatric drugs prescribed | 13 (38) | |||
| Current drug use (yes), n (%) | ||||
| Cannabis | 2 (6) | |||
| Alcohol | 1 (3) | |||
| Mental health record, n (%) | ||||
| Past psychiatric diagnosis | 11 (29) | |||
| Current psychiatric diagnosisa | 30 (88) | |||
| Anxiety disorders | 18 (53) | |||
| Adjustment disorders | 7 (21) | |||
| Depressive disorders | 7 (21) | |||
| Personality disorders | 1 (3) | |||
| Eating disorders | 1 (3) | |||
| Psychiatric drugs prescribed, n (%) | ||||
| Past psychiatric drugs prescribed | 6 (18) | |||
| Current psychiatric drugs prescribed | 28 (82) | |||
| Antidepressants | 23 (68) | |||
| Benzodiazepines | 10 (29) | |||
| Trazodone or mirtazapine | 6 (18) | |||
| Low-dose amitryptiline | 2 (6) | |||
| Other | 4 (12) | |||
aDiagnoses were of low to moderate severity.
Table 3. Longitudinal self-assessment scores.
| First (baseline) (n=34) | Second (2 weeks) (n=9) | Third (4 weeks) (n=2) | |
| GAD-7a, mean (SD) | 13.4 (4.9) | 13.0 (4.7) | 10.5 (6.4) |
| PHQ-9b, mean (SD) | 13.2 (6.6) | 12.7 (7.4) | 11.0 (7.1) |
| Work-related burnout, median (IQR) | 2.0 (0.25-2.0) | 0 (0.0-1.52) | 0 (—c) |
aGAD-7: Generalized Anxiety Disorder 7-item.
bPHQ-9: Patient Health Questionnaire-9.
cNot available.
Potential Effectiveness of the Intervention
No significant differences between the means of the first and second self-assessments were found for depressive (t8=0.40; P=.70) or anxiety (t8=1.00; P=.34) symptoms. This lack of significant difference was maintained for the third self-assessment. Work-related burnout scores were significantly lower after 2 weeks of using the Vickybot (z=−2.07; P=.04), with a moderate effect size (r=0.32; Table 3).
To assess the effect of baseline symptoms, we conducted subanalyses of participants with mild-to-moderate depression (PHQ-9, 5-14) and anxiety (GAD-7, 5-14) symptoms. No significant differences were found in depressive (t4=0.33; P=.75) or anxiety (t4=0.00; P=.99) symptoms between the means of the first and second self-assessments.
To assess whether the effect of the intervention was influenced by the use of the chatbot (dose-related effects of the intervention), we correlated the number of times that modules were performed (active treatment) with the change in the clinical scales’ scores. No significant associations were found for depressive symptoms (r=−0.48; P=.16), anxiety symptoms (r=0.12; P=.76), or work-related burnout (r=0.55; P=.13). We also correlated the “days using the chatbot” with the change in the clinical scales’ scores, and no significant associations were found for depressive symptoms (r=−0.23; P=.58), anxiety symptoms (r=−0.41; P=.36), or work-related burnout (r=0.38; P=.39). Despite the nonsignificant results, there was a trend toward a reduction in anxiety and depressive symptoms with greater chatbot use. To assess the effect of “clinically significant doses” of the intervention, we conducted subanalyses of patients with ≥50% (moderate to high) engagement. No significant differences between the scores of the first and second self-assessments were found for depressive symptoms (t6=−0.82; P=.44), anxiety symptoms (t6=0.33; P=.75), or work-related burnout (z=−1.89; P=.06).
Functions Used
All patients obtained a treatment plan with specific psychological modules that were recommended based on the first self-assessment. A total of 74% (25/34) of patients had access to their treatment plans and had performed at least one psychological module. In total, 112 modules were performed by users (mean 4.5, SD 2.4). The most frequently performed modules were depressed mood and anxiety (32 and 31 times, respectively), followed by work-related stress and burnout (17 and 15 times, respectively). Modules corresponding to depressive cognition; suicidal thoughts, decreased concentrations and irritability were not performed by any of the users. Within the modules, 100% (34/34) of patients who accessed the modules performed psychoeducation, 41% (14/34) the weekly objective, 24% (8/34) sense-based therapy, 18% (6/34) mindfulness, and 6% (2/34) metaphors. Moreover, 24% (8/34) of patients set up reminders for their “weekly objective,” and 50% (4/8) rated their daily completion; 65% (22/34) of patients set a reminder, 41% (14/34) registered an audio file after the first self-assessment, and 6% (2/34) registered new audio files after the second and third self-assessments.
Overall, 9% (3/34) of patients had activated suicide alerts. Vickybot recommended that they go to the psychiatric emergency department and provided emergency phone numbers, and the research team contacted them by phone call within 8 hours. All 3 patients responded to and acknowledged the call. Before the call, 1 of the contacted patients had overdosed on benzodiazepines and had suicidal intentions; therefore, he was urgently referred to the emergency department with a successful outcome. The other 2 contacted patients were referred to outpatient specialized mental health care services, as their current suicidal risk was deemed low by the clinical evaluation.
User-Engagement Indicators
With regard to subjective UEIs, 24% (8/34) patients responded to the MAUQ (Multimedia Appendix 2) 1 month after using Vickybot, with a mean score of 6.53 (SD 0.34) on a scale from 1 (strongly disagree) to 7 (strongly agree). All patients (34/34, 100%) agreed to 15 of 18 statements from the MAUQ questionnaire. Most patients (27/34, 80%) agreed with the remaining 3 of the 18 statements. All patients agreed on the 12th MAUQ statement, evaluating “satisfaction” with the chatbot, and also on the 18th statement, evaluating the “acceptability” of the intervention. Vickybot showed high usability, satisfaction, and acceptability. The most usefully perceived functions of the chatbot were (in order of rating from best to worst): “self-assessments,” “personalized treatment plan,” “detection of suicidal thoughts,” “contents of the psychological modules,” and the “weekly objective.”
Concerning objective UEIs, 24% (8/34) patients completed the study at 1 month, highlighting a low “completers rate.” Low adherence was also observed as only 6% (2/34) of patients used the chatbot after 4 weeks. Patients used the chatbot actively for 42.5% days of those expected (30 days), thus showing a low “compliance.” Patients used the chatbot actively for 23% (42/136) of the expected weeks (at least 5-7 days of each week, out of 4 weeks expected), meaning that “engagement” was also low. Overall, the results showed that Vickybot had low completion rates, adherence, compliance, and engagement.
Conversely, Vickybot showed high subjective UEIs (usability, satisfaction, and acceptability) but low objective-UEI (completion, adherence, compliance, and engagement). Considering the contrast between subjective and objective UEIs (perceived usefulness and actual use, respectively), Vickybot was deemed moderately feasible.
Discussion
Principal Findings
This study details the development and evaluation of the feasibility and potential effectiveness of a chatbot for anxiety, depressive symptoms, and work-related burnout among both patients from PC and health care workers. This intervention forms part of a larger project that aims to generate a digital platform (PRESTO) combining machine-learning severity prediction models with a smartphone-based app for screening, monitoring, and delivering psychological interventions [56]. The smartphone-based intervention has been envisaged to cover some of the most concerning demands from the PC system and also to provide personalized help to health care workers, as both have been affected by the COVID-19 pandemic. These multiple purposes require stepwise and solid growth as well as the collaboration and coordination of specialists in different areas.
The chatbot was useful in screening anxiety and depressive symptoms and work-related burnout in patients from PC and health care workers using self-administered scales. Although there were no statistically significant reductions in anxiety or depressive symptoms in the follow-up self-assessments, there was a significant reduction in work-related burnout (present in > 60% of our sample). Similar to other digital solutions [74,75], Vickybot pointed toward potential effectiveness in reducing work-related burnout. Further research with increased statistical power is required to confirm these findings. There is a need to tackle this growing issue with measures from occupational and health care systems, and digital tools could offer a potential solution to this highly prevalent problem. Finally, the chatbot allowed for accurate detection and prompt intervention in emergency situations (suicidal thoughts).
It is known that health care workers and people referred to specialized mental health care from PC are distinct in several aspects, including medical literacy and work-related conditions. Despite the differences between both groups, they also share several common aspects, such as a high prevalence of depressive and anxious symptoms. Even though the causes of work-related burnout are particularly and highly specific (ie, stress and lack of support at the workplace), its manifestations include mostly anxiety and depressive symptoms such as fatigue, insomnia, and sadness. Moreover, even though health care workers share a common work environment, they are highly heterogeneous in their backgrounds and work situations, including people from medical staff, nursing staff, nursing assistants, administrative workers, management, cleaning staff, security, and logistical support. In fact, during the development phase, these considerations were debated and taken into account, but we found that despite the distinct characteristics between the 2 groups, their problems and preferences for a mental health app were highly similar. In this study, subanalyses between health care workers and PC patients regarding UEIs and potential effectiveness outcomes were performed, but no differences were found. This finding reinforces the hypothesis that these 2 groups have more common mental health problems than specific differences.
It should be noted that severe anxiety and depressive symptoms (GAD-7 or PHQ-9 ≥15) were observed in 44% (15/34) and 42% (14/34) of the patients, respectively, and 24% (8/34) presented both severe anxiety and depression simultaneously. In addition to the reduced sample size and low engagement, the high severity of symptoms in > 40% of the sample may explain the lack of reduction in anxiety and depression symptoms. Digital interventions have only been found to be effective in the reduction of mild-to-moderate depressive symptoms [41], but not for severe symptoms. Therefore, consideration of the characteristics of the target population is of utmost importance when designing digital interventions. In our case, we managed to identify patients with severe anxiety and depression, which, it could be argued, are probably not suited for this kind of intervention, as they may require face-to-face visits with mental health specialists. In this sense, according to the literature, chatbots may be useful screening tools for mental health symptoms by providing a “humanized” environment while preserving anonymization [76,77].
Detecting patients with high severity is a core problem faced by overloaded PC systems. Nowadays, the prioritization system has been shown to be ineffective [78], and digital solutions that can screen for severity may be suitable for solving this problem. Vickybot was useful and accurate in detecting both patients with higher severity in addition to people with suicidal thoughts, while also offering on-time tools for immediate help (ie, emergency services) and alerting the research team to evaluate and propose tailored interventions. Suicide is probably the most striking unsolved mental health problem [79], and COVID-19 consequences are likely to worsen this situation [80-83]. Suicidal behaviors are the most alarming emergency situations that can be encountered; thus, tools that allow prompt detection and immediate interventions are crucial. Digital solutions may fulfill these requirements [84], with some interventions already showing positive evidence [85]. In fact, because this is a very sensitive issue, consensus statements on how to tackle suicide with digital technologies have recently come to light [86], and conclusive remarks regarding the future consensus on the applicability of digital solutions in mental health in both research and clinical practice are yet to come.
Some smartphone-based interventions have shown efficacy in outcomes such as well-being, quality of life, or perceived stress, which are considered more important than symptom reduction by patients [87]. Likewise, interventions such as psychoeducation empowered patients with strategies to cope with symptoms. As such, even when symptoms are not reduced, enhancing patients’ perceptions of control and self-management are key interventions for improving their quality of life [88-90]. Studies on digital health interventions should therefore focus on these parameters and not only on symptom reduction to gain further insight into their effectiveness and real-world application.
Moreover, smartphone-based solutions usually include functions such as self-assessment of symptoms, longitudinal monitoring, detection of emergency situations (eg, suicidal thoughts), and specific tools based on psychological therapies (eg, CBT or mindfulness). It is still not clear which of the former functions are more suitable for each patient profile in terms of their severity of anxiety and depression, specific symptom presentation, or sociodemographic characteristics [91]. The subanalyses and correlations performed in this study showed a nonsignificant trend toward a reduction in anxiety and depressive symptoms and work-related burnout with greater use of the chatbot. However, the small sample size in this longitudinal assessment may have limited the results. Future studies with increased statistical power are required to shed light on the remaining unanswered questions.
Regarding UEIs, subjective perceptions of use (acceptability, usability, and satisfaction) were highly positive, both for HCs during the simulation phase and for patients during the feasibility and effectiveness studies. Patients especially valued the ability to perform mental health self-assessments and personalization of treatments, as well as the chatbot’s ability to detect urgent situations and offer immediate help. This contrasts with the low objective-use metrics (completion, adherence, compliance, and engagement). This was probably because the chatbot had a rigid and sequential structure that users had to follow (based on the usual clinical evaluation and treatment procedures), thus lacking flexibility of use, which is key to user retention. This was shown by the initial use of the chatbot on day 1, with all users completing the first self-assessments and accessing at least some modules of the treatment plan, followed by a gradual reduction using the chatbot, leaving many modules incomplete. Furthermore, some modules were not accessed by any user. Other factors that may have lowered engagement outcomes are probably the lack of incentives offered by the intervention (eg, paucity of personalized reminders) or the limitations of the natural-language understanding of the chatbot, as stated in the comments of the simulation phase. A decrease in engagement in the use of digital tools over time has been reported in the literature [48,92,93]. For instance, studies with similar digital interventions reported equivalent results on engagement, with most participants only using the app once or twice [94], or only 20% of participants providing the required information in the app during the study period, with a gradual loss of entries after day 1 [95]. This suggests that engagement is a widespread problem with digital interventions and that it is key to develop strategies to mitigate it. Interventions designed for brief interactions have shown higher engagement outcomes [96]. This suggests the need to adapt smartphone-based interventions to the ways people use these technologies (usually briefly and concisely) [41]. On the basis of users’ comments and the evidence in the literature, we propose that future versions of the chatbot should aim to be more flexible by offering users alternative flows of use at all times and also include gamification with user incentives, adding personalization strategies to the psychological modules and reminders, and enhancing the chatbot’s interpretation and response capacities with NLP techniques.
Limitations
First, the limited sample size of the feasibility and effectiveness study, the low engagement with the intervention, and the high severity of anxiety and depressive symptoms of many of the patients included may have precluded the possibility of finding the potential effectiveness of the intervention in reducing anxiety and depressive symptoms. Accordingly, further research with increased sample sizes, particularly of patients with mild-to-moderate anxiety and depressive symptoms, and longer follow-ups are required to determine the effectiveness of the intervention.
Second, we did not assess PTSD symptoms, which are prevalent among health care workers [23], and other relevant outcomes, such as well-being, quality of life, and perceived stress [87]. We suggest that future studies consider these aspects to achieve a holistic perception of the intervention by users.
Third, although Vickybot succeeded in cross-sectionally screening and establishing the severity of anxiety and depressive symptoms and work-related burnout, it failed to provide follow-up (biweekly) information on the evolution of most of the sample. Notably, obtaining cross-sectional information on symptom severity is key for successful triage in PC settings and for identifying emergency situations (eg, suicidal thoughts) in which the intervention was useful. However, the key goal of the PRESTO project is to provide self-management tools for people with anxiety and depressive symptoms. Moreover, only 26.2% (34/130) of patients who received the SMS invitation signed up and performed the first self-assessments. This represents a loss of almost 3 of 4 potential participants, which is in line with other studies showing similar losses of potential participants in the initial phases [94]. We believe that this was because of access and download difficulties, hence, the future versions of the chatbot should simplify this process as much as possible. To achieve this, in-app sign-up (eg, ensuring all steps can be conducted within the app itself) and reducing the amount of data required from participants before access may provide a solution to reduce the complexity of this process. Altogether, accessibility to the chatbot and its functions need to be redesigned considering user preferences while also keeping in mind the relevance of monitoring patient evolution.
Finally, there is still a lack of consensus on how to assess the feasibility of smartphone-based interventions, especially chatbots [88,97]. In addition, UEIs still require consistent and replicable standard definitions and valid measures with solid thresholds [98]. In this study, we applied the definitions and metrics recommended by the International Society for Bipolar Disorders [72]. Although these guidelines are based on digital interventions for bipolar disorder, most concepts may be applicable to other types of interventions. In recent years, platforms and databases of mental health smartphone-based interventions have been developed [99,100], including information on the characteristics of the intervention, functions included, population for whom they are directed, evidence-based validation, professional support, and expert reviews. Future directions point toward international expert consensus, providing homogeneous valid indications on digital solutions’ regulation [101] and clinical validation [102].
Future lines of the PRESTO project include (1) enhancing the chatbot NLP component, including natural-language understanding and generation capacities; (2) integrating the chatbot into a smartphone app to facilitate accessibility and incorporate the aforementioned functions; (3) performing focus groups with patients and professionals to detect needs and points of improvement to increase engagement; and finally (4) integrating the app into a digital platform to be implemented in the PC system.
Conclusions
The chatbot was useful in screening for the presence and severity of anxiety and depressive symptoms. Although anxiety and depressive symptoms were not significantly reduced, there were significant reductions in work-related burnout on follow-up self-assessments, thus suggesting the potential effectiveness of Vickybot. Emergency situations were accurately identified and prompt interventions with successful outcomes were provided. Subjective perceptions of use (acceptability, usability, and satisfaction) were high, in contrast to low objective-use metrics (completion, adherence, compliance, and engagement), and the feasibility of the intervention was moderate. Our results are promising but suggest the need to adapt and enhance the smartphone-based solution to improve engagement. Finally, a consensus on how to report user engagement indicators and validate digital solutions, especially for chatbots, is required.
Acknowledgments
We are grateful to all participants. GA is supported by a Rio Hortega 2021 grant (CM21/00017) from the Spanish Ministry of Health financed by the Instituto de Salud Carlos III (ISCIII) and cofinanced by Fondo Social Europe Plus. MS was supported by a grant from the Baszucki Brain Research Fund. AM is supported by the Agència de Gestió d’Ajudes Universitàries I de Investigació—PANDÈMIES 2020 grant (PI047003) from the Generalitat de Catalunya. IG thanks the support of the Spanish Ministry of Science and Innovation (PI19/00954) integrated into the Plan Nacional de I+D+I and cofinanced by the ISCIII-Subdirección General de Evaluación y el Fondos Europeos de la Unión Europea (FEDER, Fondo Social Europe, Next Generation European Union or Plan de Recuperación Transformación y Resiliencia_PRTR); the Instituto de Salud Carlos III; the CIBER of Mental Health (CIBERSAM); and the Secretaria d’Universitats i Recerca del Departament d’Economia i Coneixement (2017 SGR 1365), CERCA Programme or Generalitat de Catalunya as well as the Fundació Clínic per la Recerca Biomèdica (Pons Bartran 2022-FRCB_PB1_2022). AHY’s independent research was funded by the National Institute for Health Research Biomedical Research Centre in South London and Maudsley National Health Service Foundation Trust and King’s College London. The views expressed are those of the authors and not necessarily those of the National Health Service, National Institute for Health and Care Research, or Department of Health. JR is supported by a Miguel Servet II contract (CPII19/00009), funded by ISCIII and cofunded by the European Social Fund “Investing in your future.” CT has been supported through a “Miguel Servet” postdoctoral contract (CPI14/00175) and a Miguel Servet II contract (CPII19/00018) and thanks the support of the Spanish Ministry of Innovation and Science (PI17/01066 and PI20/00344), funded by the Instituto de Salud Carlos III and cofinanced by the European Union (FEDER) “Una manera de hacer Europa.” AMA thanks the support of the Spanish Ministry of Science and Innovation (PI18/00789, PI21/00787) integrated into the Plan Nacional de I+D+I and cofinanced by ISCIII-Subdirección General de Evaluación and the Fondo Europeo de Desarrollo Regional (FEDER); the ISCIII; the CIBER of Mental Health (CIBERSAM); the Secretaria d’Universitats i Recerca del Departament d’Economia i Coneixement (2017 SGR 1365); the CERCA Programme; and the Departament de Salut de la Generalitat de Catalunya for the Pla estratègic de recerca I innovació en salut (PERIS) grant SLT006/17/00177. AM thanks the support of the Spanish Ministry of Science and Innovation (PI19/00672) integrated into the Plan Nacional de I+D+I and cofinanced by the ISCIII-Subdirección General de Evaluación and the FEDER. GF is supported by a fellowship from “La Caixa” Foundation (ID 100010434)—fellowship code—LCF/BQ/DR21/11880019. SA has been supported by a Sara Borrell contract (CD20/00177), funded by ISCIII and founded by the European Social Fund “Investing in your future.” EV thanks the support of the Spanish Ministry of Science, Innovation and Universities (PI15/00283, PI18/00805, PI19/00394, PI21/00787, and CPII19/00009) integrated into the Plan Nacional de I+D+I and cofinanced by the ISCIII-Subdirección General de Evaluación and the FEDER; the ISCIII; the CIBER of Mental Health (CIBERSAM); the Secretaria d’Universitats i Recerca del Departament d’Economia i Coneixement (2017 SGR 1365), and the CERCA Programme or Generalitat de Catalunya. We would like to thank the Departament de Salut de la Generalitat de Catalunya for the PERIS grant SLT006/17/00357. DHM´s research was supported by Juan Rodés JR18/00021 granted by the ISCIII. The PRESTO project has been funded by Fundació Clínic per a la Recerca Biomèdica through the Pons Bartran 2020 grant (PI046549). The development of a version of the digital solution adapted to health workers is funded by the Spanish Foundation for Psychiatry and Mental Health, Spanish Psychiatric Society, and Spanish Society of Biological Psychiatry (PI046813). The enhancement of the digital solution with Natural Language Processing techniques in a chatbot user-interface in collaboration with the text mining technologies in the health domain of the Barcelona Supercomputing Center is funded by the Agència de Gestió d’Ajudes Universitàries I de Investigació—PANDÈMIES 2020 grant (PI047003), from La Generalitat de Catalunya.
Data Availability
The data supporting the findings of this study are available upon request from the corresponding author.
Authors' Contributions
Study planning, project conception and coordination was done by GA, DHM, JB. Development of the digital solution was conducted by MPT, XS, MS, AM, IM, AC, JPE, ME, ARR, FP, FJV, CRT, ME, GV, CS, AR, CL, VO, LP, SQ, BS, AMA, JG, RN, VV, CPC, MFC, MV, GA, and DHM. Recruitment was done by MS, DHM, MC, IM, IG, VR, ES, SM, MSC, AMA, RN, and GA. Data extraction was done by MS and GA. Data analysis was conducted by JR, AS, DHM, and GA. GA, MS, and DHM were involved with manuscript preparation. Manuscript revision was done by all authors.
Conflicts of Interest
GA received continuing medical education (CME)-related honoraria or consulting fees from Janssen-Cilag, Lundbeck or Otsuka, and Angelini, with no financial or other relationships relevant to the subject of this paper. MC has received grants and served as a consultant, adviser, or CME speaker for the following entities: Lundbeck, Esteve, and Pfizer, with no financial or other relationships relevant to the subject of this paper. IG has received grants and serves as a consultant, adviser, or CME speaker for the following identities: Adamed, Angelini, Casen Recordati, Ferrer, Janssen-Cilag, Lundbeck, Lundbeck-Otsuka, Luye, and SEI Health care outside the submitted work. RZ collaborated with EMIS PLC, Depsee Ltd, and EMOTRA; received honoraria from Lundbeck and Janssen; is a coinvestigator for a Livanova-funded study; and provides private health care. AY has received honoraria for lectures and advisory boards for all major pharmaceutical companies with drugs used in affective and related disorders, with no financial or other relationships relevant to the subject of this paper. HL-P has received grants for training from Lundbeck with no financial or other relationships relevant to the subject of this paper. AM received grants and served as a consultant, adviser, or CME speaker for the following entities: Angelini, Idorsia, Lundbeck, Pfizer, and Takeda, with no financial or other relationships relevant to the subject of this paper. EV has received research support from or served as consultant, adviser or speaker for AB-Biotics, Abbott, Abbvie, Adamed, Angelini, Biogen, Celon, Dainippon Sumitomo Pharma, Ferrer, Gedeon Richter, GH Research, Glaxo SimthKline, Janssen, Lundbeck, Organon, Otsuka, Rovi, Sage pharmaceuticals, Sanofi-Aventis, Shire, Sunovion, Takeda, and Viatris, out of the submitted work. DHM has received CME-related honoraria and served as a consultant for Abbott, Angelini, Ethypharm Digital Therapy, and Janssen-Cilag, with no financial or other relationships relevant to the subject of this paper.
Multimedia Appendix 1
Scheme of the main functions of the intervention Vickybot.
Roca M, Gili M, Garcia-Garcia M, Salva J, Vives M, Garcia Campayo J, et al. Prevalence and comorbidity of common mental disorders in primary care. J Affect Disord 2009 Dec;119(1-3):52-58.
Ruiz-Rodríguez P, Cano-Vindel A, Muñoz Navarro R, Medrano L, Moriana JA, Buiza Aguado C, Grupo de Investigación PsicAP. Impacto económico y carga de los trastornos mentales comunes en España: una revisión sistemática y crítica. Ansiedad Estrés 2017 Jul;23(2-3):118-123.
González-Blanch C, Umaran-Alfageme O, Cordero-Andrés P, Muñoz-Navarro R, Ruiz-Rodríguez P, Medrano LA, Grupo de Investigación PsicAP. Tratamiento psicológico de los trastornos emocionales en Atención Primaria: el manual de tratamiento transdiagnóstico del estudio PsicAP. Ansiedad Estrés 2018 Jan;24(1):1-11.
Serrano-Blanco A, Palao D, Luciano JV, Pinto-Meza A, Luján L, Fernández A, et al. Prevalence of mental disorders in primary care: results from the diagnosis and treatment of mental disorders in primary care study (DASMAP). Soc Psychiatry Psychiatr Epidemiol 2010 Feb;45(2):201-210.
Fernández A, Haro JM, Martinez-Alonso M, Demyttenaere K, Brugha TS, Autonell J, et al. Treatment adequacy for anxiety and depressive disorders in six European countries. Br J Psychiatry 2007 Feb;190:172-173
Muñoz-Navarro R, Cano-Vindel A, Ruiz-Rodríguez P, Adrián Medrano L, González-Blanch C, Moriana JA, et al. Modelo jerárquico de diagnóstico y derivación de los trastornos mentales comunes en centros de atención primaria. Una propuesta a partir del ensayo clínico PsicAP. Ansiedad Estrés 2017 Jul;23(2-3):124-129.
Latorre Postigo JM, Navarro Bravo B, Parra Delgado M, Salguero JM, Mae Wood C, Cano Vindel A. Evaluación e intervención de los problemas de ansiedad y depresión en atención primaria: un problema sin resolver. Rev Clin Med Fam 2012 Feb;5(1):37-45
Pérez-Ibáñez O, García-Pedrajas C, Grases N, Marsó E, Cots I, Aparicio S, et al. Programa de soporte a la primaria: un análisis descriptivo y de concordancia diagnóstica entre médico/a de atención primaria y psicólogo/a. Rev Asoc Esp Neuropsiq 2015 Jun;35(126):257-266.
Puyat JH, Kazanjian A, Goldner EM, Wong H. How often do individuals with major depression receive minimally adequate treatment? A population-based, data linkage study. Can J Psychiatry 2016 Jul;61(7):394-404.
Wiegner L, Hange D, Björkelund C, Ahlborg Jr G. Prevalence of perceived stress and associations to symptoms of exhaustion, depression and anxiety in a working age population seeking primary care--an observational study. BMC Fam Pract 2015 Mar 19;16:38
Strawbridge R, McCrone P, Ulrichsen A, Zahn R, Eberhard J, Wasserman D, et al. Care pathways for people with major depressive disorder: a European Brain Council Value of Treatment study. Eur Psychiatry 2022 Jun 15;65(1):1-21
Cayuela L, Sánchez Gayango A, Sánchez-Trincado PA, Rodríguez-Domínguez S, Velasco Quiles AA, Cayuela A. Suicide mortality in Spain (1984-2018): age-period-cohort analysis. Rev Psiquiatr Salud Ment (Engl Ed) (forthcoming) 2020 Jul 13.
Gabilondo A, Rojas-Farreras S, Rodríguez A, Fernández A, Pinto-Meza A, Vilagut G, et al. Use of primary and specialized mental health care for a major depressive episode in Spain by ESEMeD respondents. Psychiatr Serv 2011 Feb;62(2):152-161.
Vedia Urgell C, Bonet Monne S, Forcada Vega C, Parellada Esquius N. Estudio de utilización de psicofármacos en atención primaria. Aten Primaria 2005 Sep 30;36(5):239-245
Baos Vicente V. Prescripción de psicofármacos en atención primaria. Aten Primaria 2005 Sep 30;36(5):246-247
Carrasco-Garrido P, Hernández-Barrera V, Jiménez-Trujillo I, Esteban-Hernández J, Álvaro-Meca A, López-de Andrés A, et al. Time trend in psychotropic medication use in Spain: a nationwide population-based study. Int J Environ Res Public Health 2016 Nov 24;13(12):1177
Anmella G, Sanabra M, Primé-Tous M, Segú X, Solanes A, Ruíz V, et al. Antidepressants overuse in primary care: prescription trends between 2010-2019 in Catalonia. Rev Psiquiatr Salud Ment (Engl Ed) (forthcoming) 2022 Dec 14.
COVID-19 Mental Disorders Collaborators. Global prevalence and burden of depressive and anxiety disorders in 204 countries and territories in 2020 due to the COVID-19 pandemic. Lancet 2021 Nov 06;398(10312):1700-1712
Balanzá-Martínez V, Kapczinski F, de Azevedo Cardoso T, Atienza-Carbonell B, Rosa AR, Mota JC, et al. The assessment of lifestyle changes during the COVID-19 pandemic using a multidimensional scale. Rev Psiquiatr Salud Ment (Engl Ed) 2021;14(1):16-26
Vieta E, Pérez V, Arango C. Psychiatry in the aftermath of COVID-19. Rev Psiquiatr Salud Ment (Engl Ed) 2020;13(2):105-110
Fullana MA, Hidalgo-Mazzei D, Vieta E, Radua J. Coping behaviors associated with decreased anxiety and depressive symptoms during the COVID-19 pandemic and lockdown. J Affect Disord 2020 Oct 01;275:80-81
Dosil Santamaría M, Ozamiz-Etxebarria N, Redondo Rodríguez I, Jaureguizar Alboniga-Mayor J, Picaza Gorrotxategi M. Psychological impact of COVID-19 on a sample of Spanish health professionals. Rev Psiquiatr Salud Ment (Engl Ed) 2021;14(2):106-112
Alonso J, Vilagut G, Mortier P, Ferrer M, Alayo I, Aragón-Peña A, MINDCOVID Working group. Mental health impact of the first wave of COVID-19 pandemic on Spanish healthcare workers: a large cross-sectional survey. Rev Psiquiatr Salud Ment (Engl Ed) 2021;14(2):90-105
Mortier P, Vilagut G, Alayo I, Ferrer M, Amigo F, Aragonès E, MINDCOVID Working group. Four-month incidence of suicidal thoughts and behaviors among healthcare workers after the first wave of the Spain COVID-19 pandemic. J Psychiatr Res 2022 May;149:10-17
Mortier P, Vilagut G, Ferrer M, Serra C, Molina JD, López-Fresneña N, MINDCOVID Working Group. Thirty-day suicidal thoughts and behaviors among hospital workers during the first wave of the Spain COVID-19 outbreak. Depress Anxiety 2021 May;38(5):528-544
Blekas A, Voitsidis P, Athanasiadou M, Parlapani E, Chatzigeorgiou AF, Skoupra M, et al. COVID-19: PTSD symptoms in Greek health care professionals. Psychol Trauma 2020 Oct;12(7):812-819.
Kang L, Li Y, Hu S, Chen M, Yang C, Yang BX, et al. The mental health of medical workers in Wuhan, China dealing with the 2019 novel coronavirus. Lancet Psychiatry 2020 Mar;7(3):e14
Lai J, Ma S, Wang Y, Cai Z, Hu J, Wei N, et al. Factors associated with mental health outcomes among health care workers exposed to coronavirus disease 2019. JAMA Netw Open 2020 Mar 02;3(3):e203976
Valdés-Florido MJ, López-Díaz Á, Palermo-Zeballos FJ, Martínez-Molina I, Martín-Gil VE, Crespo-Facorro B, et al. Reactive psychoses in the context of the COVID-19 pandemic: clinical perspectives from a case series. Rev Psiquiatr Salud Ment (Engl Ed) 2020;13(2):90-94
Anmella G, Fico G, Roca A, Gómez-Ramiro M, Vázquez M, Murru A, et al. Unravelling potential severe psychiatric repercussions on healthcare professionals during the COVID-19 crisis. J Affect Disord 2020 Aug 01;273:422-424
Maslach C, Schaufeli WB, Leiter MP. Job burnout. Annu Rev Psychol 2001;52:397-422.
Burn-out an "occupational phenomenon": International Classification of Diseases. World Health Organization. 2019 May 28. URL: https://www.who.int/news/item/28-05-2019-burn-out-an-occupational-phenomenon-international-classification-of-diseases [accessed 2022-10-06]
Glise K, Ahlborg Jr G, Jonsdottir IH. Course of mental symptoms in patients with stress-related exhaustion: does sex or age make a difference? BMC Psychiatry 2012 Mar 12;12:18
Metlaine A, Sauvet F, Gomez-Merino D, Elbaz M, Delafosse JY, Leger D, et al. Association between insomnia symptoms, job strain and burnout syndrome: a cross-sectional survey of 1300 financial workers. BMJ Open 2017 Jan 13;7(1):e012816
The Lancet. Physician burnout: a global crisis. Lancet 2019 Jul 13;394(10193):93.
COVID-19 disrupting mental health services in most countries, WHO survey. World Health Organization. 2020 Oct 5. URL: https://www.who.int/news/item/05-10-2020-covid-19-disrupting-mental-health-services-in-most-countries-who-survey [accessed 2020-10-20]
Hollander JE, Carr BG. Virtually perfect? Telemedicine for COVID-19. N Engl J Med 2020 Apr 30;382(18):1679-1681.
Chen Q, Liang M, Li Y, Guo J, Fei D, Wang L, et al. Mental health care for medical staff in China during the COVID-19 outbreak. Lancet Psychiatry 2020 Apr;7(4):e15-e16
Schuh Teixeira AL, Spadini AV, Pereira-Sanchez V, Ojeahere MI, Morimoto K, Chang A, et al. The urge to implement and expand telepsychiatry during the COVID-19 crisis: early career psychiatrists' perspective. Rev Psiquiatr Salud Ment (Engl Ed) 2020;13(3):174-175
Firth J, Torous J, Nicholas J, Carney R, Rosenbaum S, Sarris J. Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. J Affect Disord 2017 Aug 15;218:15-22
Firth J, Torous J, Nicholas J, Carney R, Pratap A, Rosenbaum S, et al. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry 2017 Oct;16(3):287-298
Roy A, Druker S, Hoge EA, Brewer JA. Physician anxiety and burnout: symptom correlates and a prospective pilot study of app-delivered mindfulness training. JMIR Mhealth Uhealth 2020 Apr 01;8(4):e15608
Pandya SP. Meditation app alleviates burnout and builds resilience for chaplains in hospices for older adults in Asian and African cities. J Health Care Chaplain 2021;27(3):129-145.
Pace TW, Zeiders KH, Cook SH, Sarsar ED, Hoyt LT, Mirin NL, et al. Feasibility, acceptability, and preliminary efficacy of an app-based meditation intervention to decrease firefighter psychological distress and burnout: a one-group pilot study. JMIR Form Res 2022 Jun 08;6(6):e34951
Magee JC, Adut S, Brazill K, Warnick S. Mobile app tools for identifying and managing mental health disorders in primary care. Curr Treat Options Psychiatry 2018 Sep;5(3):345-362
Hoffman L, Benedetto E, Huang H, Grossman E, Kaluma D, Mann Z, et al. Augmenting mental health in primary care: a 1-year study of deploying smartphone apps in a multi-site primary care/behavioral health integration program. Front Psychiatry 2019 Feb 28;10:94
Hidalgo-Mazzei D, Mateu A, Reinares M, Murru A, Del Mar Bonnín C, Varo C, et al. Psychoeducation in bipolar disorder with a SIMPLe smartphone application: feasibility, acceptability and satisfaction. J Affect Disord 2016 Aug;200:58-66.
Hidalgo-Mazzei D, Reinares M, Mateu A, Nikolova VL, Bonnín CD, Samalin L, et al. OpenSIMPLe: a real-world implementation feasibility study of a smartphone-based psychoeducation programme for bipolar disorder. J Affect Disord 2018 Dec 01;241:436-445.
Fitzpatrick KK, Darcy A, Vierhile M. Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial. JMIR Ment Health 2017 Jun 06;4(2):e19
Vaidyam AN, Wisniewski H, Halamka JD, Kashavan MS, Torous JB. Chatbots and conversational agents in mental health: a review of the psychiatric landscape. Can J Psychiatry 2019 Jul;64(7):456-464
Lucas GM, Rizzo A, Gratch J, Scherer S, Stratou G, Boberg J, et al. Reporting mental health symptoms: breaking down barriers to care with virtual human interviewers. Front Robot AI 2017 Oct 12;4:51.
Petersen CL, Halter R, Kotz D, Loeb L, Cook S, Pidgeon D, et al. Using natural language processing and sentiment analysis to augment traditional user-centered design: development and usability study. JMIR Mhealth Uhealth 2020 Aug 07;8(8):e16862
Fukuoka Y, Lindgren TG, Mintz YD, Hooper J, Aswani A. Applying natural language processing to understand motivational profiles for maintaining physical activity after a mobile app and accelerometer-based intervention: the mPED randomized controlled trial. JMIR Mhealth Uhealth 2018 Jun 20;6(6):e10042
Torous J. Measuring progress in measurement-based care with smartphone tools. Acta Psychiatr Scand 2019 Oct;140(4):293-294.
Wisniewski H, Torous J. Digital navigators to implement smartphone and digital tools in care. Acta Psychiatr Scand 2020 Apr;141(4):350-355
Anmella G, Primé-Tous M, Segú X, Solanes A, Ruíz V, Martín-Villalba I, et al. PRimary carE digital Support ToOl in mental health (PRESTO): design, development and study protocols. Rev Psiquiatr Salud Ment (Engl Ed) (forthcoming) 2021 Apr 29.
Bowen DJ, Kreuter M, Spring B, Cofta-Woerpel L, Linnan L, Weiner D, et al. How we design feasibility studies. Am J Prev Med 2009 May;36(5):452-457
Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006 May 22;166(10):1092-1097.
García-Campayo J, Zamorano E, Ruiz MA, Pardo A, Pérez-Páramo M, López-Gómez V, et al. Cultural adaptation into Spanish of the generalized anxiety disorder-7 (GAD-7) scale as a screening tool. Health Qual Life Outcomes 2010 Jan 20;8:8
Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001 Sep;16(9):606-613
Diez-Quevedo C, Rangil T, Sanchez-Planell L, Kroenke K, Spitzer RL. Validation and utility of the patient health questionnaire in diagnosing mental disorders in 1003 general hospital Spanish inpatients. Psychosom Med 2001;63(4):679-686.
Maslach C, Jackson SE, Leiter MP. Maslach burnout inventory: third edition. In: Zalaquett CP, Wood RJ, editors. Evaluating Stress: A Book of Resources. Palo Alto, CA, USA: Consulting Psychologists Press; 1986.
Aguayo R, Vargas C, de la Fuente EI, Lozano LM. A meta-analytic reliability generalization study of the Maslach Burnout Inventory. Int J Clin Heal Psychol 2011;11(2):343-361.
Luis Martorell J. Psicoterapiascuelas y Conceptos Básicos. 2da edición. Madrid, Spain: Pirámide Publisher; 2014.
Barlow DH. Clinical Handbook of Psychological Disorders: A Step-by-Step Treatment Manual. 6th edition. New York, NY, USA: Guilford Press; 2021.
Díaz MI, Ruiz MÁ, Villalobos A. Manual de Técnicas y Terapias Cognitivo Conductuales. Paris, France: Desclée de Brouwer; 2017.
Linehan MM. Skills Training Manual for Treating Borderline Personality Disorder. DBT Skills Training Manual. 2nd edition. New York, NY, USA: Guilford Press; 2015.
Wilson KG, Luciano SC. Terapia de aceptacion y compromiso (ACT) (Psicologia / Psychology). Spanish edition. Madrid, Spain: Pirámide Publisher; 2007.
Lehrhaupt L, Meibert P. Mindfulness-Based Stress Reduction: The MBSR Program for Enhancing Health and Vitality. Novato, CA, USA: New World Library; 2017.
Zhou L, Bao J, Setiawan IM, Saptono A, Parmanto B. The mHealth app usability questionnaire (MAUQ): development and validation study. JMIR Mhealth Uhealth 2019 Apr 11;7(4):e11500
Muro-Culebras A, Escriche-Escuder A, Martin-Martin J, Roldán-Jiménez C, De-Torres I, Ruiz-Muñoz M, et al. Tools for evaluating the content, efficacy, and usability of mobile health apps according to the consensus-based standards for the selection of health measurement instruments: systematic review. JMIR Mhealth Uhealth 2021 Dec 01;9(12):e15433
Anmella G, Faurholt-Jepsen M, Hidalgo-Mazzei D, Radua J, Passos IC, Kapczinski F, et al. Smartphone-based interventions in bipolar disorder: systematic review and meta-analyses of efficacy. A position paper from the International Society for Bipolar Disorders (ISBD) Big Data Task Force. Bipolar Disord 2022 Sep;24(6):580-614
Fusar-Poli P, Manchia M, Koutsouleris N, Leslie D, Woopen C, Calkins ME, PSMD EBRA cluster (annex 1). Ethical considerations for precision psychiatry: a roadmap for research and clinical practice. Eur Neuropsychopharmacol 2022 Oct;63:17-34
Michelsen C, Kjellgren A. The effectiveness of web-based psychotherapy to treat and prevent burnout: controlled trial. JMIR Form Res 2022 Aug 11;6(8):e39129
Thomas Craig KJ, Willis VC, Gruen D, Rhee K, Jackson GP. The burden of the digital environment: a systematic review on organization-directed workplace interventions to mitigate physician burnout. J Am Med Inform Assoc 2021 Apr 23;28(5):985-997
Hungerbuehler I, Daley K, Cavanagh K, Garcia Claro H, Kapps M. Chatbot-based assessment of employees' mental health: design process and pilot implementation. JMIR Form Res 2021 Apr 21;5(4):e21678
Giunti G, Isomursu M, Gabarron E, Solad Y. Designing depression screening chatbots. Stud Health Technol Inform 2021 Dec 15;284:259-263.
Wittchen HU, Mühlig S, Beesdo K. Mental disorders in primary care. Dialogues Clin Neurosci 2003 Jun;5(2):115-128
Knipe D, Padmanathan P, Newton-Howes G, Chan LF, Kapur N. Suicide and self-harm. Lancet 2022 May 14;399(10338):1903-1916.
Porras-Segovia A, Baca-García E, Courtet P, Wainberg M, Oquendo MA. If suicide were COVID-19: a neglected cause of premature death. J Clin Psychiatry 2021 Feb 23;82(2):20l13702
Courtet P, Olié E. Suicide in the COVID-19 pandemic: what we learnt and great expectations. Eur Neuropsychopharmacol 2021 Sep;50:118-120.
Pompili M. Can we expect a rise in suicide rates after the COVID-19 pandemic outbreak? Eur Neuropsychopharmacol 2021 Nov;52:1-2
Pérez V, Elices M, Vilagut G, Vieta E, Blanch J, Laborda-Serrano E, et al. Suicide-related thoughts and behavior and suicide death trends during the COVID-19 in the general population of Catalonia, Spain. Eur Neuropsychopharmacol 2022 Mar;56:4-12
Barrigon ML, Courtet P, Oquendo M, Baca-García E. Precision medicine and suicide: an opportunity for digital health. Curr Psychiatry Rep 2019 Nov 28;21(12):131.
Torok M, Han J, Baker S, Werner-Seidler A, Wong I, Larsen ME, et al. Suicide prevention using self-guided digital interventions: a systematic review and meta-analysis of randomised controlled trials. Lancet Digit Health 2020 Jan;2(1):e25-e36
Nock MK, Kleiman EM, Abraham M, Bentley KH, Brent DA, Buonopane RJ, et al. Consensus statement on ethical and safety practices for conducting digital monitoring studies with people at risk of suicide and related behaviors. Psychiatr Res Clin Pract 2021;3(2):57-66
Faurholt-Jepsen M, Frost M, Christensen EM, Bardram JE, Vinberg M, Kessing LV. The effect of smartphone-based monitoring on illness activity in bipolar disorder: the MONARCA II randomized controlled single-blinded trial. Psychol Med 2020 Apr;50(5):838-848.
Jang S, Kim JJ, Kim SJ, Hong J, Kim S, Kim E. Mobile app-based chatbot to deliver cognitive behavioral therapy and psychoeducation for adults with attention deficit: a development and feasibility/usability study. Int J Med Inform 2021 Jun;150:104440.
Lakhtakia T, Torous J. Current directions in digital interventions for mood and anxiety disorders. Curr Opin Psychiatry 2022 Mar 01;35(2):130-135.
Bonnín CD, Solé B, Reinares M, García-Estela A, Samalin L, Martínez-Arán A, et al. Does cognitive impairment in bipolar disorder impact on a SIMPLe app use? J Affect Disord 2021 Mar 01;282:488-494.
Ly KH, Trüschel A, Jarl L, Magnusson S, Windahl T, Johansson R, et al. Behavioural activation versus mindfulness-based guided self-help treatment administered through a smartphone application: a randomised controlled trial. BMJ Open 2014 Jan 09;4(1):e003440
Lattie EG, Schueller SM, Sargent E, Stiles-Shields C, Tomasino KN, Corden ME, et al. Uptake and usage of IntelliCare: a publicly available suite of mental health and well-being apps. Internet Interv 2016 May;4(2):152-158
García-Estela A, Cantillo J, Angarita-Osorio N, Mur-Milà E, Anmella G, Pérez V, et al. Real-world implementation of a smartphone-based psychoeducation program for bipolar disorder: observational ecological study. J Med Internet Res 2022 Feb 02;24(2):e31565
Kinderman P, Hagan P, King S, Bowman J, Chahal J, Gan L, et al. The feasibility and effectiveness of Catch It, an innovative CBT smartphone app. BJPsych Open 2016 May;2(3):204-209
Chan NP, Lai AY, Choy HK, Cheung DY, Wan AN, Cheng VY, et al. Feasibility and potential effectiveness of a smartphone zero-time exercise intervention for promoting physical activity and fitness in patients with coronary heart disease: a pilot randomized controlled trial. Front Public Health 2022 Jul 14;10:865712
Mohr DC, Tomasino KN, Lattie EG, Palac HL, Kwasny MJ, Weingardt K, et al. IntelliCare: an eclectic, skills-based app suite for the treatment of depression and anxiety. J Med Internet Res 2017 Jan 05;19(1):e10
To QG, Green C, Vandelanotte C. Feasibility, usability, and effectiveness of a machine learning-based physical activity chatbot: quasi-experimental study. JMIR Mhealth Uhealth 2021 Nov 26;9(11):e28577
Ng MM, Firth J, Minen M, Torous J. User engagement in mental health apps: a review of measurement, reporting, and validity. Psychiatr Serv 2019 Jul 01;70(7):538-544
One Mind PsyberGuide | A Mental Health App Guide. URL: https://onemindpsyberguide.org/ [accessed 2022-02-01]
Mobile Health Index and Navigation Database, App Evaluation Resources from the Division of Digital Psychiatry at BIDMC. URL: https://mindapps.org/ [accessed 2022-02-01]
Diao JA, Venkatesh KP, Raza MM, Kvedar JC. Multinational landscape of health app policy: toward regulatory consensus on digital health. NPJ Digit Med 2022 May 11;5(1):61
Torous J, Andersson G, Bertagnoli A, Christensen H, Cuijpers P, Firth J, et al. Towards a consensus around standards for smartphone apps and digital mental health. World Psychiatry 2019 Feb;18(1):97-98
© 2023. This work is licensed under https://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Sanabra, Miriam
; Primé-Tous, Mireia
; Segú, Xavier
; Cavero, Myriam
; Morilla, Ivette
; Grande, Iria
; Ruiz, Victoria
; Mas, Ariadna
; Martín-Villalba, Inés
; Caballo, Alejandro
; Julia-Parisad Esteva
; Rodríguez-Rey, Arturo
; Piazza, Flavia
; Valdesoiro, Francisco José
; Rodriguez-Torrella, Claudia
; Espinosa, Marta
; Virgili, Giulia
; Sorroche, Carlota
; Ruiz, Alicia
; Solanes, Aleix
; Radua, Joaquim
; Also, María Antonieta
; Sant, Elisenda
; Murgui, Sandra
; Sans-Corrales, Mireia
; Young, Allan H
; Vicens, Victor
; Blanch, Jordi
; Caballeria, Elsa
; López-Pelayo, Hugo
; López, Clara
; Olivé, Victoria
; Pujol, Laura
; Quesada, Sebastiana
; Solé, Brisa
; Torrent, Carla
; Martínez-Aran, Anabel
; Guarch, Joana
; Navinés, Ricard
; Murru, Andrea
; Fico, Giovanna
; de Prisco, Michele
; Oliva, Vicenzo
; Amoretti, Silvia
; Pio-Carrino, Casimiro
; Fernández-Canseco, María
; Villegas, Marta
; Vieta, Eduard
; Hidalgo-Mazzei, Diego