Long-Term Effects of Internet-Based Cognitive Behavioral Therapy on Depression Prevention Among University Students: Randomized Controlled Factorial Trial

Abstract

Background:Internet-based cognitive behavioral therapy (iCBT) shows promise in the prevention of depression. However, the specific iCBT components that contribute to its effectiveness remain unclear.

Objective:We aim to evaluate the effects of iCBT components in preventing depression among university students.

Methods:Using a smartphone cognitive behavioral therapy (CBT) app, we randomly allocated university students to the presence or absence of 5 different iCBT components: self-monitoring, behavioral activation, cognitive restructuring, assertiveness training, and problem-solving. The active intervention lasted 8 weeks but the app remained accessible through the follow-up. The primary outcome was the onset of a major depressive episode (MDE) between baseline and the follow-up after 52 weeks, as assessed with the computerized World Health Organization Composite International Diagnostic Interview. Secondary outcomes included changes in the 9-item Patient Health Questionnaire, 7-item General Anxiety Disorder, and CBT Skills Scale.

Results:During the 12-month follow-up, 133 of 1301 (10.22%) participants reported the onset of an MDE. There were no significant differences in the incidence of MDEs between the groups with or without each component (hazard ratios ranged from 0.85, 95% CI 0.60‐1.20, for assertiveness training to 1.26, 95% CI 0.88‐1.79, for self-monitoring). Furthermore, there were no significant differences in the changes on the 9-item Patient Health Questionnaire, 7-item General Anxiety Disorder, or for CBT Skills Scale between component allocation groups. However, significant reductions in depression and anxiety symptoms were observed among all participants at the 52-week follow-up.

Conclusions:In this study, we could not identify any specific iCBT components that were effective in preventing depression or the acquisition of CBT skills over the 12-month follow-up period, but all participants with and without intervention of each iCBT component demonstrated significant improvements in depressive and anxiety symptoms. Further research is needed to explore the potential impact of frequency of psychological assessments, nonspecific intervention effects, natural change in the mental state, and the baseline depression level.

Trial Registration:UMINCTR UMIN000031307; https://center6.umin.ac.jp/cgi-open-bin/ctr_e/ctr_view.cgi?recptno=R000035735

International Registered Report Identifier (IRRID):RR2-10.1186/s13063-018-2719-z

Introduction

Major depressive disorder (MDD) is one of the most common mental disorders among university students with an estimated 12-month prevalence of 18.5% [1]. MDD can disrupt academic, interpersonal, and social functioning and can lead to suicide [2]. The suicide risk among people with MDD is more than 20 times greater than in the general population [2]. Worldwide, the World Health Organization (WHO) reports that suicide is the fourth leading cause of death among 15-29 year olds [3]. According to Ministry of Health, Labor and Welfare Suicide Countermeasures Promotion Office, in Japan, between 2020 and 2022, over 1000 students per year lost their lives to suicide, and suicide was the leading cause of death among those aged 15 to 34 years [4]. To improve functioning and to reduce suicide risk, preventing major depressive episodes (MDEs) among university students is especially important.

Several psychological interventions have shown promise for preventing depression [5] and cognitive behavioral therapy (CBT)–based strategies make up the bulk of the evidence [6,7]. Further, 1 recent systematic review showed that psychological interventions, mostly CBT-based, reduce the chances of depression incidence by 19% [8]. Furthermore, CBT-based depression prevention interventions are proven efficacious among child and adolescent populations [7]. Additionally, internet-based cognitive behavioral therapy (iCBT) has demonstrated consistent efficaciousness in treating depression [9,10]. iCBT is highly desirable because of its ease of accessibility and lower costs than face-to-face interventions [11].

However, the efficaciousness of iCBT has been examined as a package of several components [9]. In these packages, participants can receive specific modules from a menu of CBT skills, including psychoeducation, self-monitoring (SM), behavioral activation (BA), cognitive restructuring (CR), assertiveness training (AT), and problem-solving (PS). Hence, the efficacy of individual components for preventing depression remains unknown. By focusing on the effective components, iCBT could be conducted more efficiently, further boosting the scalability among university students.

In this large factorial trial, we aimed to identify the efficacious components of iCBT for preventing depression in the long-term. Our previous analysis of acute phase effects on depressive symptoms indicated an improvement in symptoms regardless of iCBT component allocation [12]. However, the long-term effects of each iCBT component on MDE incidence is still unclear. If reducing the occurrence of MDEs is due to specific iCBT components learned over the course of a year, this would indicate the necessary intervention ingredients needed to reduce long-term MDD progression.

Methods

Design

We conducted the Healthy Campus Trial, a fully factorial trial of 5 iCBT components: SM, BA, CR, AT, and PS. This study was designed as a fully factorial randomized controlled trial. The original protocol [13] explains the finer details of the Healthy Campus Trial.

Participants

Undergraduate or graduate students from 5 universities in Japan were recruited between September 2018 and May 2021. Students aged between 18 and 39 years, possessing a smartphone, were introduced to online screening with the 9-item Patient Health Questionnaire (PHQ-9). To examine the effect of smartphone CBT on those with and without mild mental symptoms, based on the screening PHQ-9 scores, we included a random tenth of students scoring 4 or less on the PHQ-9 and all of the subthreshold depressive students with PHQ-9 scores between 5 and 9, or between 10 and 14 with suicidal ideation for less than half of the days. We were able to include only 1 in 10 of the students with no to very mild symptoms because including them all would have made the total sample size 10 times greater, and our budget and manpower could not afford this. Since the focus of the acute phase intervention [12,13] was on those with subthreshold depression, we prioritized their inclusion, while for the long-term outcome, we also wished to examine the effect of the smartphone CBT on those with or without mild symptoms. We excluded students under treatment by any mental health professionals. No student was diagnosed with an MDE at baseline.

Interventions

All participants received a psychoeducation module. With psychoeducation, participants learned the importance of monitoring their own mental state through the trial duration using the smartphone app “Resilience Training,” which was developed for this study. After psychoeducation and baseline PHQ-9 assessment, iCBT combinations were randomly allocated for each participant among 32 combinations (with or without each of the 5 iCBT components, hence 2⁵=32 combinations, see Multimedia Appendix 1). Participants could be either assigned to or not assigned to each one individually. As a result, some participants might end up with zero iCBT components after the psychoeducation, while others could receive all 5. We used block randomization stratified by participant university and baseline PHQ-9 score (4 or less versus 5 or more). Each component was supposed to take 1 week. All participants, regardless of the number of the allocated components, were expected to finish the app in 8 weeks. Participants work on each component for 7‐10 days. They were able to review any assigned components as often as they liked during the 8 weeks and fill in as many additional worksheets as they wished. The participants had to complete (read all the lessons and complete at least one worksheet) each component before they could advance to the next component. Participants received digital gift cards worth US $9 for completing self-checks in weeks 4 and 24 and US $18 in weeks 8 and 52. In addition, research staff sent regular emails to encourage all participants to move forward with the program and reminder emails to fill in questionnaires if required. These emails did not include any specific guidance on the content of iCBT.

The contents of the five iCBT components were as follows:

SM: Participants monitored their own reactions to different situations (ie, feelings, thoughts, behaviors, and physical reactions) on mind maps and learned the association between each reaction.

BA: Participants recorded their activities with facilitation of new challenges through gamification in the “Action Marathon.”

CR: Participants recorded automatic thoughts and situations in a mind map. They were challenged to broaden their thoughts using several items, including telephone calls from imagined friends, a thought credibility percentage calculator, and a “Serenity Prayer”–modeled self-talk script.

AT: Participants learned assertive communication methods and recorded the content of real conversations, the situations, and the people involved.

PS: Participants set goals regarding difficult situations, raised ideas for solving associated problems, and evaluated the merits and demerits of each idea.

All CBT component lessons were delivered through the “Resilience Training” app.

Measurements

The primary outcome was time to the first MDE by 52 weeks post randomization. The occurrence of an MDE was evaluated by the computerized version of the WHO Composite International Diagnostic Interview (CIDI) version 3.0 depression section, which has demonstrated good reliability [14,15]. The secondary outcomes included depressive symptoms, anxiety symptoms, and 5 CBT skills, as measured by the PHQ-9 [16,17], 7-item General Anxiety Disorder (GAD-7) [18,19], and CBT Skills Scale [20], respectively. The reliability of each questionnaire is well supported by previous evidence, including our study in which the CBT Skills Scale was developed and validated [20]. The timing for each questionnaire is detailed in the original protocol [13]. For the current analysis, the PHQ-9 was measured at baseline and at weeks 1 through 8, followed by every 4 weeks thereafter, up to week 52. The GAD-7 and the CBT Skills Scale were measured at baseline, and at the fourth, eighth, 24th, and 52nd weeks of this study. The CIDI results were obtained at week 52. To ensure participant safety, those with high levels of depression and suicidal ideation measured by the PHQ-9 were monitored and advised via email to seek professional care. All questionnaires were administered using the Japanese version.

Statistical Analysis

We used SAS (version 9.4, SAS Institute) and R (version 4.2.1, R Foundation for Statistical Computing) for all analyses. We performed Cox regression analyses to investigate whether the presence or absence of each CBT component was related to the occurrence of an MDE. The model involves 5 iCBT components, baseline PHQ-9 scores, university, age, and sex as explanatory variables. Allocation of each iCBT component was defined based on an intention to treat (ITT). We plotted Kaplan-Meier survival curves for a graphical representation of MDE-free survival for each iCBT component. Interval censorings were addressed with Anderson-Bergman adjustment methods [21,22]. In addition, changes in PHQ-9, GAD-7, and CBT Skills Scale were repeatedly measured up to 52 weeks and analyzed with mixed models for repeated measures [23]. The correlation matrix of repeated outcomes was assumed to be unstructured. For explanatory variables, 5 iCBT components, week, iCBT component by week interaction, university, age, and baseline scores were modeled. The between-group effect size was estimated by dividing the estimated mean difference by the observed SD of week 52 scores, which is the most widely used definition of standardized difference in clinical trials of psychiatry [24]. In addition, the within-group effect size was estimated by dividing the estimated mean change by the baseline SD.

Ethical Considerations

This study involves human participants and has been approved by the Ethics Committee of the Kyoto University School of Medicine (C1357). All data obtained from participants were deidentified, and participants provided informed consent before participating in this study.

Results

Demographics

Figure 1 presents the CONSORT diagram. Of the 5063 students assessed for eligibility, 1627 students provided informed consent and were randomly allocated to 1 of the 32 iCBT combinations (Multimedia Appendix 1). Finally, 1626 students were included in this study, after 1 student withdrew consent and refused the use of their data for analysis. Table 1 tabulates the demographic characteristics of the participants. The mean age was 21.5 (SD 3.0) years, and 933 (57.38%) were women. The mean PHQ-9 and GAD-7 scores were 6.4 (SD 3.4) and 5.5 (SD 3.4), respectively. Participants were allocated to each module (SM: n=808; BA: n=817; CR: n=811; AT: n=814; PS: n=811). The characteristics of the participants per component are described in Multimedia Appendix 2, and the number of participants allocated to each of the 32 combinations is described in Multimedia Appendix 1.

Figure 1. CONSORT diagram. CIDI: Composite International Diagnostic Interview; CONSORT: Consolidated Standards of Reporting Trials; IC: informed consent; PHQ-9: 9-item Patient Health Questionnaire.

Table 1.Baseline characteristics of total participants for 1-year analysis (N=1626).

				Participants
Demographics
Age (years), mean (SD)				21.5 (2.99)
Sex (female), n (%)				933 (57.38)
Undergraduate, n (%)				1250 (76.88)
Married, n (%)				33 (2.03)
Living alone (apart from family), n (%)				1002 (61.62)
Part-time employment, n (%)				1261 (77.55)
Smoking experience, n (%)				94 (5.78)
Drinking alcohol regularly, n (%)				665 (40.90)
Exercise opportunity, n (%)				1014 (62.36)
History of psychiatric or psychological treatment, n (%)				184 (11.32)
History of major depressive episode in past year (CIDIa), n (%)				151 (9.29)
Cognitive and behavioral skills, mean (SD)
SMb skills				8.03 (3.26)
BAc skills				10.53 (4.33)
CRd skills				8.71 (3.44)
ATe skills				9.60 (3.55)
PSf skills				10.6 (3.09)
Clinical characteristic, mean (SD)
PHQ-9g				6.38 (3.40)
GAD-7h				5.47 (3.35)
Function, mean (SD)
WHO-HPQi Presenteeism				6.30 (1.58)

^aCIDI: Composite International Diagnostic Interview.

^bSM: self-monitoring.

^cBA: behavioral activation.

^dCR: cognitive restructuring.

^eAT: assertiveness training.

^fPS: problem-solving.

^gPHQ-9: 9-item Patient Health Questionnaire.

^hGAD-7: 7-item General Anxiety Disorder.

ⁱWHO-HPQ: World Health Organization Health and Work Performance Questionnaire.

Primary Analysis

Of the 1626 study participants, 1301 (80.01%) responded to the 52-week follow-up survey. Among the responders, 133 (10.22%) reported incident MDE by week 52. The number of completions and their percentages for each assigned module were recorded (SM: 743/808, 92.6%; BA: 703/817, 86%; CR: 688/811, 84.8%; AT: 680/814, 83.8%; PS: 666/811, 82.1%). Table 2 summarizes the results of the multivariable Cox regression analysis of each CBT component on MDE occurrence. The hazard ratios (95% CI) of presence over absence were 1.26 (0.88‐1.79) for SM, 1.09 (0.77‐1.54) for BA, 1.18 (0.83‐1.68) for CR, 0.85 (0.60‐1.20) for AT, and 1.18 (0.82‐1.68) for PS. Multimedia Appendix 3 shows the full results of the analysis with coefficients for the included covariates. The results of the univariable Cox regression analysis (Multimedia Appendix 4) showed similar results to those of the multivariable analyses. Figure 2 presents the Kaplan-Meier survival curves for the occurrence of an MDE for each of the 5 iCBT components. We did not find a significant difference in the occurrence of an MDE in each iCBT component.

Table 2.Summary of the results of multiple cox regression analyses of each CBT^a component on occurrence of an MDE^b the results of the analysis for each component are shown in Multimedia Appendix 3.^c

Component	Cases with MDE, n/N (completed cases)		HRd for incident MDE
	Presence	Absence	HR	SE	95% CI			P value
SMe	75/808 (638)	58/818 (663)	1.26	0.18	0.88-1.79			.21
BAf	69/817 (648)	64/809 (653)	1.09	0.18	0.77-1.54			.63
CRg	72/811 (652)	61/815 (649)	1.18	0.18	0.83-1.68			.37
ATh	62/814 (647)	71/812 (654)	0.85	0.18	0.60-1.20			.35
PSi	71/811 (646)	62/815 (655)	1.18	0.18	0.82-1.68			.37

^aCBT: cognitive behavioral therapy.

^bMDE: major depressive episode

^cAge, sex, baseline 9-item Patient Health Questionnaire point, and recruition site (4 universities) were adjusted in the Cox regression analysis.

^dHR: hazard ratio.

^eSM: self-monitoring.

^fBA: behavioral activation.

^gCR: cognitive restructuring.

^hAT: assertiveness training.

ⁱPS: problem-solving.

Figure 2. Survival curves for MDEs for the 5 CBT components. AT: assertiveness training; BA: behavioral activation; CBT: cognitive behavioral therapy; CR: cognitive restructuring; MDE: major depressive episode; PS: problem-solving; S(t): survival function (major depressive episode–free survival); SM: self-monitoring.

Secondary Analysis

Table 3 shows the estimated least squares mean change scores of the PHQ-9 for participants allocated to the presence or absence of each component using the mixed models for repeated measures. Compared to the baseline, the estimated least squares mean change scores ranged between −1.77 and −1.97 at the 52-week follow-up for both the presence and absence groups across all iCBT components. Regardless of the allocated iCBT component, average depressive symptoms were reduced from baseline. However, no significant differences in change scores were found in the comparison between the presence group and the absence group across all iCBT components. Analysis of the GAD-7 showed similar results for anxiety symptoms, as shown in Table 4. Table 5 shows an analysis of the relevant CBT skills at week 52 for each CBT component. There was no difference in changes in CBT skills between the presence or absence of each iCBT component.

Table 3.Summary of the results of differences of change scores of PHQ-9^a at 52 weeks using MMRM^b CBT^c components, week, CBT component by week interaction, university, age, and baseline scores as fixed effects, while participants were introduced as a random effect. Effect size was estimated by division of the estimated mean changes by the observed SD of baseline scores.

Component		Participants, n	Estimated least squares mean change scores of PHQ-9 within groups (95% CI)	Within-group effect size for baseline-week 52 change scores of PHQ-9 (95% CI)	Estimated differences of baseline-week 52 change scores between groups (95% CI)	Effect size for differences of baseline-week 52 change scores between groups (95% CI)	P values
SMd					−0.09 (−0.28 to 0.4)	−0.02 (−0.06 to 0.09)	.81
	Presence	808	−1.91 (−2.43 to −1.39)	−0.69 (−0.87 to −0.50)
	Absence	818	−1.83 (−2.35 to −1.31)	−0.66 (−0.84 to −0.47)
BAe					−0.14 (−0.43 to 0.25)	−0.03 (−0.09 to 0.05)	.71
	Presence	817	−1.94 (−2.46 to −1.41)	−0.70 (−0.88 to −0.51)
	Absence	809	−1.8 (−2.32 to −1.28)	−0.65 (−0.83 to −0.46)
CRf					0.21 (−0.29 to 0.39)	0.04 (−0.06 to 0.08)	.58
	Presence	811	−1.77 (−2.28 to −1.25)	−0.64 (−0.82 to −0.45)
	Absence	815	−1.97 (−2.50 to −1.45)	−0.71 (−0.90 to −0.52)
ATg					0.21 (−0.16 to 0.52)	0.04 (−0.04 to 0.11)	.57
	Presence	814	−1.77 (−2.29 to −1.24)	−0.64 (−0.82 to −0.45)
	Absence	812	−1.97 (−2.49 to −1.45)	−0.71 (−0.90 to −0.52)
PSh					−0.09 (−0.30 to 0.38)	−0.02 (−0.06 to 0.08)	.80
	Presence	811	−1.92 (−2.43 to −1.40)	−0.69 (−0.87 to −0.50)
	Absence	815	−1.82 (−2.35 to −1.3)	−0.65 (−0.84 to −0.47)

^aPHQ-9: 9-item Patient Health Questionnaire.

^bMMRM: mixed models for repeated measures.

^cCBT: cognitive behavioral therapy.

^dSM: self-monitoring.

^eBA: behavioral activation.

^fCR: cognitive restructuring.

^gAT: assertiveness training.

^hPS: problem-solving.

Table 4.Summary of the results of differences of change scores of GAD-7^a at 52 weeks using MMRM^b CBT^c components, week, CBT component by week interaction, university, age, and baseline scores as fixed effects, while participants were introduced as a random effect. Effect size was estimated by division of the estimated mean changes by the observed SD of baseline scores.

Component		Participants, n	Estimated least squares mean change scores of GAD-7 within groups (95% CI)	Within-group effect size for baseline-week 52 change score of GAD-7 (95% CI)	Estimated differences of baseline-week 52 change scores between groups (95% CI)	Effect size for differences of baseline-week 52 change scores between groups (95% CI)	P values
SMd					0.27 (−0.46 to 1)	0.08 (−0.14 to 0.3)	.47
	Presence	808	−1.28 (−1.82 to −0.74)	−0.38 (−0.55 to −0.22)
	Absence	818	−1.55 (−2.08 to −1.02)	−0.47 (−0.62 to −0.31)
BAe					−0.32 (−1.04 to 0.41)	−0.09 (−0.31 to 0.12)	.39
	Presence	817	−1.57 (−2.11 to −1.04)	−0.47 (−0.63 to −0.31)
	Absence	809	−1.26 (−1.79 to −0.72)	−0.38 (−0.54 to −0.22)
CRf					0.30 (−0.43 to 1.02)	0.09 (−0.13 to 0.31)	.42
	Presence	811	−1.27 (−1.79 to −0.74)	−0.38 (−0.54 to −0.22)
	Absence	815	−1.56 (−2.11 to −1.02)	−0.47 (−0.63 to −0.31)
ATg					0.14 (−0.59 to 0.86)	0.04 (−0.18 to 0.26)	.71
	Presence	814	−1.35 (−1.89 to −0.81)	−0.41 (−0.57 to −0.24)
	Absence	812	−1.48 (−2.01 to −0.95)	−0.44 (−0.60 to −0.29)
PSh					−0.01 (−0.73 to 0.72)	0 (−0.22 to 0.22)	.99
	Presence	811	−1.42 (−1.95 to −0.88)	−0.43 (−0.59 to −0.26)
	Absence	815	−1.41 (−1.95 to −0.87)	−0.42 (−0.59 to −0.26)

^aGAD-7: 7-item General Anxiety Disorder.

^bMMRM: mixed models for repeated measures.

^cCBT: cognitive behavioral therapy.

^dSM: self-monitoring.

^eBA: behavioral activation.

^fCR: cognitive restructuring.

^gAT: assertiveness training.

^hPS: problem-solving.

Table 5.Summary of the estimated least squares mean differences of change in corresponding CBT^a skills at 52 weeks using MMRM^b in addition to CBT component, week, CBT component by week interaction, university, age, and baseline scores as fixed effects, while participants were introduced as a random effect.

Component	Estimated least squares mean differences of change in corresponding CBT skill (95% CI)
SMc	0.23 (−0.28 to 0.75)
BAd	−0.44 (−1.10 to 0.22)
CRe	−0.1 (−0.58 to 0.38)
ATf	0.14 (−0.39 to 0.66)
PSg	0.24 (−0.15 to 0.64)

^aCBT: cognitive behavioral therapy.

^bMMRM: mixed models for repeated measures.

^cSM: self-monitoring.

^dBA: behavioral activation.

^eCR: cognitive restructuring.

^fAT: assertiveness training.

^gPS: problem-solving.

Discussion

Principal Findings

We conducted a fully factorial randomized trial of 5 iCBT skills to reduce the incidence of MDEs among university students. We did not observe significant preventive effects attributable to individual iCBT components in depression prevention. Secondary analyses showed no significant difference in change scores for PHQ-9, GAD-7, and CBT skills, regardless of the presence or absence of each iCBT component either. Participants exhibited improvements in depression and anxiety symptoms from the baseline to the 52 weeks, irrespective of the specific iCBT component allocation.

The significantly better PHQ-9 and GAD-7 scores compared to baseline at 52 weeks, irrespective of the iCBT component, replicate the results from our previous 8-week short-term analysis [12]. Multiple extra-intervention factors may have contributed to the general improvement effects. First, the frequent psychological assessments administered to all participants may have introduced a significant nonspecific therapeutic effect. This hypothesis is supported by findings from a prior meta-analysis of antidepressant efficacy trials, which demonstrated that follow-up assessments significantly influence placebo responses [25]. In this study, frequent follow-up visits significantly impacted depressive scale scores, where the extra follow-up visits accounted for approximately 34%‐44% of the overall placebo response observed over the course of the trials. These results emphasize that the therapeutic effect of these assessments is not only statistically significant but also cumulative and proportional to the number of interactions, thereby suggesting a potent nonspecific therapeutic effect stemming from the assessment frequency itself [25].

In our study, a rigorous schedule of assessments was implemented, occurring as many as 19 times throughout the duration of this study (specifically at weeks 1-8, 12, 16, 20, 24, 28, 32, 36, 40, 44, 48, and 52). This high frequency of assessments might have played a significant role in mediating a therapeutic effect, potentially confounding the distinct impacts attributable to each individual iCBT component. It is important to note, however, that all participants in this study were subjected to the same number of assessments, making it challenging to isolate the effects of the assessments from those of the iCBT interventions themselves. Consequently, while it is tempting to speculate about the influence of frequent assessments, such speculation cannot be conclusively confirmed given the uniform assessment protocol applied across all participants.

Given these considerations, it is recommended that future research should explore the implications of varying the frequency of assessments by including experimental groups subjected to differing assessment schedules. This approach would allow for a more refined analysis of how frequent interactions might influence outcomes, thereby enabling a more accurate assessment of the true efficacy of the iCBT components. By distinguishing the effects of the intervention from those of the assessment frequency, future studies could yield more reliable and generalizable findings.

In addition to frequent assessments, application help notifications coupled with encouragement emails may have produced extra-intervention improvement effects. Regardless of allocated iCBT components, participants were instructed to keep the app installed until the end of the 52-week follow-up. This instruction also conveyed crisis management information, such as where to go for help in case of serious problems. Providing emergency contact information was reported to reduce rates of repeated self-harm [26]. Thus, it is possible that the crisis management information in our study could have contributed positively to mental health conditions. Future studies should consider incorporating groups with different frequencies of self-assessments to derive more reliable results.

Together with the frequent follow-up assessments and application help notifications, university grade level could have affected the results. A longitudinal study reported that depressive and anxiety symptoms were better in the fourth year of university than in the first 3 years [27]. This trend toward improvement may mask the effect of our iCBT intervention. Similarly, baseline depression level may have affected our results. As depression symptoms become milder, placebo effects become more prominent relative to the intervention. This assertion is supported by a meta-analysis examining the effects of antidepressants compared to placebo [28]. If we targeted university students with more severe depression symptoms, we may have a greater ability to detect intervention effects. On the other hand, another study assessing the effects of an iCBT program with healthy participants showed a significant improvement in subthreshold depressive symptoms [10]. Therefore, our failure to identify the effects of the intervention may not be attributed solely to the depressive severity of the participants.

The latter half of our study unfolded during the COVID-19 pandemic, accompanied by extensive social restrictions. This period has been scrutinized across various studies, notably in a systematic review of epidemiological research which reported a temporary surge in depression and anxiety within the first 3 months of the pandemic [29]. However, our own secondary analysis, using the same data set [30], did not find a statistically significant link between depressive symptoms, as measured by the PHQ-9, and the implementation of 4 distinct states of emergency in Japan aimed at mitigating the spread of the virus. Although our results suggest no direct correlation between these emergency measures and depressive symptoms, the overarching impact of the pandemic—its indirect influence on mental health through other unmeasured factors—remains a plausible explanation for variations in our findings. Consequently, we acknowledge the complexity of the pandemic’s effects, which may interact with psychological outcomes in nuanced and varied ways that are not immediately apparent in our data.

The validated CBT Skills Scale [20] was used to measure the main 5 CBT skills. The results at 52 weeks showed no significant difference in change scores with or without each of the iCBT components. This finding was unexpected as our previously reported short-term analysis revealed significant improvement of CR and AT skills [12]. The 52-week results lack of significant differences may be due to the attenuation of skills learned during the first 8-week intervention period. However, some studies reported that the effect of CBT lasts longer than 46 months as an adjunct to pharmacotherapy for treatment-resistant depression [31]. These studies focused only on symptoms, not on CBT skills. Our report is the first to assess the persistence of CBT skills using our original CBT Skills Scale. Further research is needed to determine how long CBT skills are usually maintained. It is possible that maintenance CBT may be effective in maintaining CBT skills, despite the reported fact that maintenance CBT on depression prevention is superior to psychoeducation only among patients with a high risk of depression relapse [32].

Limitations

There are several limitations in our study. First, our participant pool was limited to 2 prefectures in Japan (ie, Kyoto and Aichi). Nonetheless, this well-powered multisite trial experienced minimal attrition and included 3 types of universities, national, public, and private, which attracted students from all regions of Japan. Additionally, we ensured adequate allocation concealment during randomization, resulting in a well-distributed sample. Nevertheless, caution is advised when generalizing these results. Second, our study did not account for potentially significant protective factors, such as good academic performance and financial stability. Although our research used a fully factorial design intended to evenly distribute unknown confounders across study groups, we cannot definitively exclude the influence of unmeasured variables. Third, our measures of MDEs, mental state, and CBT skills relied on self-administered questionnaires that, despite their established reliability, are susceptible to biases such as social-desirability. This susceptibility potentially obscures the true symptoms of depression, and the accuracy of the PHQ-9 may also be compromised by the repeated measurement effect. To mitigate these limitations and enhance assessment accuracy, face-to-face evaluations conducted by trained professionals would be more ideal. Finally, the validated CBT Skills Scale used in our study may not have been sensitive enough to capture subtle differences in skill acquisition and retention over the extensive follow-up period. While the scale is robust for assessing general CBT competency, it may not adequately reflect incremental changes or the long-term retention of specific skills. Future research should use such tools to ensure a more precise and sensitive evaluation of the long-term skill acquisition and retention of CBT skills.

Conclusions

This study did not show the specific impacts of individual components of iCBT on preventing MDEs. However, participants’ scores on the PHQ-9 and GAD-7 generally improved over the year during their involvement in this study. In addition to the statistical phenomenon of regression to the mean and the natural course of the conditions, the psychoeducation given to all participants, the frequent psychological assessments, and the experience of being involved in this study itself may have contributed to some alleviation of depression and anxiety symptoms among university students. Future iCBT optimization trials should consider psychological assessment frequency, the impact of common intervention elements, the natural history of the mental state of the target group, and the baseline depression level.

Acknowledgments

This work was supported in part by JST SPRING (grant JPMJSP2110) and the World Health Organization Centre for Health Development (WHO Kobe Centre—WKC: K22001). Support was also given through grant-in-aid from Japan Society for the Promotion of Science to MS (21K03049), RT (22K21171), and TI (18K18643); Japan Agency for Medical Research and Development (AMED; dk0307085) and WHO Mental Health Research (2021/HQ/WKC/0013) to TAF; and Suzuken Memorial Foundation, KDDI Foundation, and Pfizer Health Research Foundation to TU.

Conflicts of Interest

TU reports receiving lecture fees from Sumitomo Pharma and Shionogi Pharma. MS reports personal fees from SONY outside of this submitted work. NS reports receiving lecture fees from Mochida Pharma and Sumitomo Pharma. AT reports personal fees from Sumitomo Dainippon Pharma, Eisai, Janssen Pharmaceutical, Meiji-Seika Pharma, Mitsubishi Tanabe Pharma, Otsuka, Takeda Pharmaceutical, and Shionogi. HN received a research fund from GlaxoSmithKline and consulting fees from GlaxoSmithKline, Ono Pharmaceutical, Sony, and Kowa, outside of this submitted work. TAF reports personal fees from Boehringer-Ingelheim, Daiichi Sankyo, DT Axis, Kyoto University Original, Shionogi, SONY, and UpToDate, and a grant from DT Axis and Shionogi, outside of this submitted work. In addition, TAF has a patent 7448125 and a pending patent 2022-082495 and has licensed intellectual properties for Kokoro-app to Mitsubishi-Tanabe.

------------

Abbreviations

AT: assertiveness training

BA: behavioral activation

CBT: cognitive behavioral therapy

CR: cognitive restructuring

GAD-7: 7-item General Anxiety Disorder

iCBT: internet-based cognitive behavioral therapy

ITT: intention-to-treat

MDD: major depressive disorder

MDE: major depressive episode

PHQ-9: 9-item Patient Health Questionnaire

PS: problem-solving

SM: self-monitoring

WHO: World Health Organization

------------

Edited by John Torous; submitted 24.01.24; peer-reviewed by Jun Wang, Patricia Frazier; final revised version received 11.06.24; accepted 12.06.24; published 24.09.24.

References

Auerbach RP, Mortier P, Bruffaerts R, et al. WHO world mental health surveys international college student project: prevalence and distribution of mental disorders. J Abnorm Psychol. Oct 2018;127(7):623-638.

Lépine JP, Briley M. The increasing burden of depression. Neuropsychiatr Dis Treat. 2011;7(Suppl 1):3-7.

Suicide worldwide in 2019: global health estimates World Health Organization. 2021. URL: https://iris.who.int/bitstream/handle/10665/341728/9789240026643-eng.pdf?sequence=1 [Accessed 2024-08-06]

Ministry of Health, Labour and Welfare 2022 white paper on suicide prevention. 2022. URL: https:/​/www.​mhlw.go.jp/​stf/​seisakunitsuite/​bunya/​hukushi_kaigo/​seikatsuhogo/​jisatsu/​jisatsuhakusyo2022.​html [Accessed 2024-08-06]

Caldwell DM, Davies SR, Hetrick SE, et al. School-based interventions to prevent anxiety and depression in children and young people: a systematic review and network meta-analysis. Lancet Psychiatry. Dec 2019;6(12):1011-1020.

Hetrick SE, Cox GR, Witt KG, Bir JJ, Merry SN. Cognitive behavioural therapy (CBT), third-wave CBT and interpersonal therapy (IPT) based interventions for preventing depression in children and adolescents. Cochrane Database Syst Rev. Aug 9, 2016;2016(8):CD003380.

Zhang Q, Wang J, Neitzel A. School-based mental health interventions targeting depression or anxiety: a meta-analysis of rigorous randomized controlled trials for school-aged children and adolescents. J Youth Adolesc. Jan 2023;52(1):195-217.

Cuijpers P, Pineda BS, Quero S, et al. Psychological interventions to prevent the onset of depressive disorders: a meta-analysis of randomized controlled trials. Clin Psychol Rev. Feb 2021;83:101955.

Furukawa TA, Suganuma A, Ostinelli EG, et al. Dismantling, optimising, and personalising internet cognitive behavioural therapy for depression: a systematic review and component network meta-analysis using individual participant data. Lancet Psychiatry. Jun 2021;8(6):500-511.

Karyotaki E, Efthimiou O, Miguel C, et al. Internet-based cognitive behavioral therapy for depression: a systematic review and individual patient data network meta-analysis. JAMA Psychiatry. Apr 1, 2021;78(4):361-371.

Baumann M, Stargardt T, Frey S. Cost-utility of internet-based cognitive behavioral therapy in unipolar depression: a Markov model simulation. Appl Health Econ Health Policy. Aug 2020;18(4):567-578.

Sakata M, Toyomoto R, Yoshida K, et al. Components of smartphone cognitive-behavioural therapy for subthreshold depression among 1093 university students: a factorial trial. Evid Based Ment Health. Dec 2022;25(e1):e18-e25.

Uwatoko T, Luo Y, Sakata M, et al. Healthy campus trial: a multiphase optimization strategy (MOST) fully factorial trial to optimize the smartphone cognitive behavioral therapy (CBT) app for mental health promotion among university students: study protocol for a randomized controlled trial. Trials. Jul 4, 2018;19(1):353.

Haro JM, Arbabzadeh-Bouchez S, Brugha TS, et al. Concordance of the ComposIte International Diagnostic Interview Version 3.0 (CIDI 3.0) with standardized clinical assessments in the WHO world mental health surveys. Int J Methods Psychiatr Res. 2006;15(4):167-180.

Shimoda H, Inoue A, Tsuno K, Kawakami N. One-year test-retest reliability of a Japanese web-based version of the WHO composite international diagnostic interview (CIDI) for major depression in a working population. Int J Methods Psychiatr Res. Sep 2015;24(3):204-212.

Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. Sep 2001;16(9):606-613.

Muramatsu K, Miyaoka H, Kamijima K, et al. The Patient Health Questionnaire, Japanese version: validity according to the Mini-International Neuropsychiatric Interview-Plus. Psychol Rep. Dec 2007;101(3 Pt 1):952-960.

Muramatsu K, Muramatsu Y, Miyaoka H. Validation and utility of a Japanese version of the GAD-7. In: Panminerva Medica 20th World Congress on Psychosomatic Medicine Abstracts Book. CiNii Research; 2009:79.

Spitzer RL, Kroenke K, Williams JBW, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. May 22, 2006;166(10):1092-1097.

Sakata M, Toyomoto R, Yoshida K, et al. Development and validation of the Cognitive Behavioural Therapy Skills Scale among college students. Evid Based Ment Health. May 2021;24(2):70-76.

Anderson-Bergman C. icenReg: regression models for interval censored data in R. J Stat Softw. Nov 2017;81(12):1-12.

Anderson-Bergman C. An efficient implementation of the EMICM algorithm for the interval censored NPMLE. J Comput Graph Stat. Apr 3, 2017;26(2):463-467.

Mallinckrodt CH, Clark WS, David SR. Accounting for dropout bias using mixed-effects models. J Biopharm Stat. 2001;11(1-2):9-21.

Luo Y, Funada S, Yoshida K, Noma H, Sahker E, Furukawa TA. Large variation existed in standardized mean difference estimates using different calculation methods in clinical trials. J Clin Epidemiol. Sep 2022;149:89-97.

Posternak MA, Zimmerman M. Therapeutic effect of follow-up assessments on antidepressant and placebo response rates in antidepressant efficacy trials: meta-analysis. Br J Psychiatry. Apr 2007;190:287-292.

Scott A, Guo B. For which strategies of suicide prevention is there evidence of effectiveness? World Health Organization. 2012. URL: https://iris.who.int/bitstream/handle/10665/363870/9789289057233-eng.pdf?sequence=1 & isAllowed=y [Accessed 2024-08-07]

Gao W, Ping S, Liu X. Gender differences in depression, anxiety, and stress among college students: A longitudinal study from China. J Affect Disord. Feb 15, 2020;263:292-300.

Stone MB, Yaseen ZS, Miller BJ, Richardville K, Kalaria SN, Kirsch I. Response to acute monotherapy for major depressive disorder in randomized, placebo controlled trials submitted to the US Food And Drug Administration: individual participant data analysis. BMJ. Aug 2, 2022;378:e067606.

Salanti G, Peter N, Tonia T, et al. The impact of the COVID-19 pandemic and associated control measures on the mental health of the general population: a systematic review and dose-response meta-analysis. Ann Intern Med. Nov 2022;175(11):1560-1571.

Shiraishi N, Sakata M, Toyomoto R, et al. Dynamics of depressive states among university students in Japan during the COVID-19 pandemic: an interrupted time series analysis. Ann Gen Psychiatry. Oct 10, 2023;22(1):38.

Wiles NJ, Thomas L, Turner N, et al. Long-term effectiveness and cost-effectiveness of cognitive behavioural therapy as an adjunct to pharmacotherapy for treatment-resistant depression in primary care: follow-up of the CoBalT randomised controlled trial. Lancet Psychiatry. Feb 2016;3(2):137-144.

Stangier U, Hilling C, Heidenreich T, et al. Maintenance cognitive-behavioral therapy and manualized psychoeducation in the treatment of recurrent depression: a multicenter prospective randomized controlled trial. Am J Psychiatry. Jun 2013;170(6):624-632.