Internet and Computer-Based Cognitive Behavioral Therapy for Anxiety and Depression in Youth: A Meta-Analysis of Randomized Controlled Outcome Trials

Abstract

Background

Anxiety and depression in children and adolescents are undertreated. Computer- and Internet-based cognitive behavioral treatments (cCBT) may be an attractive treatment alternative to regular face-to-face treatment.This meta-analysis aims to evaluate whether cCBT is effective for treating symptoms of anxiety and depression in youth.

Methods and Findings

We conducted systematic searches in bibliographical databases (Pubmed, Cochrane controlled trial register, PsychInfo) up to December 4, 2013. Only randomized controlled trials in which a computer-, Internet- or mobile-based cognitive behavioral intervention targeting either depression, anxiety or both in children or adolescents up to the age of 25 were compared to a control condition were selected. We employed a random-effects pooling model in overall effect analyses and a mixed effect model for sub-group analyses. Searches resulted in identifying 13 randomized trials, including 796 children and adolescents that met inclusion criteria. Seven studies were directed at treating anxiety, four studies at depression, and two were of a transdiagnostic nature, targeting both anxiety and depression. The overall mean effect size (Hedges’ g) of cCBT on symptoms of anxiety or depression at post-test was g=0.72 (95% CI:0.55-0.90, numbers needed to be treated (NNT)=2.56). Heterogeneity was low (I²=20.14%, 95% CI: 0-58%). The superiority of cCBT over controls was evident for interventions targeting anxiety (g=0.68; 95% CI: 0.45-0.92; p < .001; NNT=2.70) and for interventions targeting depression (g=0.76; 95% CI: 0.41-0.12; p < .001; NNT=2.44) as well as for transdiagnostic interventions (g=0.94; 95% CI: 0.23-2.66; p < .001; NNT=2.60).

Conclusions

Results provide evidence for the efficacy of cCBT in the treatment of anxiety and depressive symptoms in youth. Hence, such interventions may be a promising treatment alternative when evidence based face-to-face treatment is not feasible. Future studies should examine long-term effects of treatments and should focus on obtaining patient-level data from existing studies, to perform an individual patient data meta-analysis.

About the Authors:

David Daniel Ebert

* E-mail: [email protected]

Affiliations Leuphana University, Innovation Incubator, Division Health Trainings Online, Lueneburg, Germany, Friedrich-Alexander University Nuremberg-Erlangen, Department of Psychology, Clinical Psychology and Psychotherapy, Erlangen, Germany, Department of Clinical Psychology and EMGO Institute for Health and Care Research, VU University, Amsterdam, the Netherlands

Anna-Carlotta Zarski

Affiliation: Leuphana University, Innovation Incubator, Division Health Trainings Online, Lueneburg, Germany

Helen Christensen

Affiliation: Black Dog Institute, Sydney, Australia

Yvonne Stikkelbroek

Affiliation: Center for Child and Adolescent Studies, University of Utrecht, the Netherlands

Pim Cuijpers

Affiliations Leuphana University, Innovation Incubator, Division Health Trainings Online, Lueneburg, Germany, GGZ inGeest, Regional Mental Health Service Centre, VU University Medical Centre, Amsterdam, the Netherlands, Department of Clinical Psychology and EMGO Institute for Health and Care Research, VU University, Amsterdam, the Netherlands

Matthias Berking

Affiliations Leuphana University, Innovation Incubator, Division Health Trainings Online, Lueneburg, Germany, Friedrich-Alexander University Nuremberg-Erlangen, Department of Psychology, Clinical Psychology and Psychotherapy, Erlangen, Germany

Heleen Riper

Affiliations Leuphana University, Innovation Incubator, Division Health Trainings Online, Lueneburg, Germany, GGZ inGeest, Regional Mental Health Service Centre, VU University Medical Centre, Amsterdam, the Netherlands, Department of Clinical Psychology and EMGO Institute for Health and Care Research, VU University, Amsterdam, the Netherlands

Introduction

Depression and anxiety are common in children and adolescents (in the following referred to collectively as youth) [1–3] and often co-occur [4, 5]. Both disorders are associated with substantial burdens [6, 7] an increased risk for other mental disorders [8–10] and they often tend to persist in adulthood [11–13].

In the past decades, a number of psychological treatments for anxiety and depression in youth have been developed with demonstrated efficacy in a substantial number of clinical trials [14–18]. Cognitive behavioral therapy (CBT) is generally regarded as the treatment of choice for depression and anxiety in youth [19].

However, up to 80% of children and adolescents with mental health needs receive no treatment [20–24]. The reasons include not only a lack of treatment availability, but also reticence to seek help because of perceived stigma associated with mental illness, discomfort discussing mental health problems, and / or a preference for self-help [25].

Using the computer or the Internet to provide CBT may overcome some of the limitations of traditional treatment services. Advantages of computer and Internet-based CBT (cCBT) include (1) availability (2) anonymity, (3) accessibility at any time and place, (4) flexibility in self-direction and self-pacing, (5) reduced travel time and costs for both participants and clinicians, and (6) the appeal of interactivity and visual attractiveness of Internet-based programs [26, 27, 28]. Given the Internet savviness of the younger generations, these advantages might be even more relevant in youths than in adults.

cCBT is effective in the treatment of depressive [29] and anxiety disorders [30] in adults. For example, a systematic review of 19 randomized controlled trials evaluating Internet-based and other computerized interventions for adult depression symptoms in N = 2996 individuals found a mean effect size of d = 0.56. In another meta-analyses on self-help interventions for anxiety disorders, a pooled mean effect size of d = .90 was found across 27 randomized controlled trials evaluating Internet-based treatments for anxiety [31].

Less is known, however, about the effectiveness of cCBT for anxiety and depression in youth. In a recent systematic review [28], only three randomized controlled trials were included. Hence, the authors could not calculate pooled effect sizes using meta-analytic techniques. With the recent addition of several new published trials, we performed a meta-analysis to evaluate the efficacy of cCBT for anxiety and depression in youth (up to the age of 25) in comparison to a non-active control condition within randomized controlled trials.

Methods

Study selection

Several strategies were used to identify relevant studies. Two independent assessors (Ebert & Zarski) searched three major bibliographical databases (PubMed, PsycInfo, Cochrane) by combining terms indicative of each of the disorders with terms indicative of psychological treatment and randomized controlled trials. The searches were performed December 04, 2013. We also checked the references of the identified studies for earlier publications. Details of the searches are provided in Fig. 1.

[Figure omitted. See PDF.]

Fig 1. Flow Chart.

https://doi.org/10.1371/journal.pone.0119895.g001

We included (a) randomized controlled trials in which (b) a computer or Internet- or mobile-based (c) cognitive behavioural intervention (d) targeting either depression, anxiety or both (e) in children or adolescents up to the age of 25 (f) were compared to a non-active control condition (wait-list; placebo). We only included studies (g) that used an anxiety or depression measure as the primary outcome measure and (h) in which participants were selected because of elevated levels of depressive/anxiety symptoms based on either a standardized self-report measure or a formal diagnosis of major depressive or any anxiety disorder (including OCD), and (i) effect sizes could be calculated from the information provided in the paper. If effect sizes could not be calculated, authors were contacted to retrieve the necessary information. Studies were not included when they had only an active treatment comparison (e.g., face-to-face CBT) or only included participants older than 18 years. Comorbid general medical and psychiatric or substance/alcohol use disorders were not used as an exclusion criterion. No language restrictions were applied. Studies that evaluated interventions in which the Internet/computer was not the primary treatment modality (e.g., blended treatments with face-to-face psychotherapy and some online elements) were excluded.

Risk of bias assessment and data extraction

Risk of bias assessment and data extraction was conducted independently by two assessors (Ebert & Zarski). We assessed the validity of included studies using four criteria of the Risk of Bias assessment tool, developed by the Cochrane Collaboration [32] (adequate generation of allocation sequence, the concealment of allocation to conditions, masking of assessors, and the missing data handling (the assessment was positive when intention-to-treat analyses were conducted)). We examined the relationship between risk of bias and the effect size by performing meta-regression techniques. In these analyses, the total bias score was entered as the dependent variable. We compared effect sizes of studies rated as low-risk of bias (fulfilling all four assessed criteria) compared to studies with some risk of bias (fulfilling less than four assessed criteria). We also identified and coded additional aspects of the included studies, relating to the study design, participants and interventions.

Study design

Variables related to the study design included the target condition (anxiety vs. depression vs. transdiagnostic) and the comparison condition (no treatment vs. placebo). The comparison group was coded as placebo control group if cCBT was compared to an active condition with no intended therapeutic properties that would be expected to produce change in the target diagnosis [33].

We also coded the outcome informant (youth, parent, teacher), that is, whether the information about symptoms were derived from the youth report, parent report, or teacher report. We further coded the type of outcome assessment (self-report vs. observer based) and whether the studies included a follow-up (yes/no; duration).

Participant characteristics

In line with previous meta-analyses on psychotherapy in children and adolescents [16, 34, 35], we classified studies as child, adolescents or mixed studies. Studies in which all participants were 13 years-old or younger were classified as child, studies in which all participants were over 13 years-old were classified as adolescent, studies that included both children and adolescents were classified as mixed. The mean age was also coded, and the percentage of boys was included. We further coded whether studies included only participants with a confirmed diagnosis of depression or an anxiety disorder using an established diagnostic interview (yes/no).

Intervention characteristics

Interventions were classified as with or without parental involvement (yes/no) and with or without professional guidance (therapeutic or simply administrative) (yes/no). We also coded the technology on which the intervention was delivered (computer: cCBT; Internet: iCBT; mobile: mCBT), the setting in which the intervention was conducted (at home/not at home, e.g., clinic, school, university), as well as the number of treatment sessions.

Meta-analysis

For each comparison between cCBT for anxiety or depression or both and a control group, we calculated the effect size indicating the difference between the two groups in change from baseline to post-test (Hedges’ g). To calculate pooled mean effect sizes, we used the Comprehensive Meta-Analysis (CMA) software (Biostat, Inc.). In the main outcome analysis, we examined the overall mean effects of cCBT on symptoms of depression or anxiety. To generate a mean effect size, we only used instruments that related to the principal measure of the disorder. If interventions were transdiagnostic, targeting both anxiety and depression, we used the pooled effect size of these outcomes. If more than one measure per outcome was used, the mean of the effect sizes was calculated, so that each study provided only one effect size. Given that youth depression and anxiety symptoms highly correlate [4, 5] and in light of the current debate over whether depression and anxiety require separate treatments or can be treated by combined interventions [16, 36], we also examined the overall effects of all interventions on symptoms of anxiety and depression in a separate analysis.

We expected considerable heterogeneity among the studies, hence we employed a random-effects pooling model (DerSimonian-Laird) in all analyses. Given the difficulty of interpreting the standardized mean difference (Hedges’ g) from a clinical perspective, we transformed these values into the number-needed-to-treated (NNT) using the formula provided by Kraemer and Kupfer [37]. The NNT indicates the number of patients that must be treated to generate one additional positive outcome [38].

As a test of homogeneity of effect sizes, we calculated the I² statistic as an indicator of heterogeneity in percentages [39, 40]. We calculated 95% confidence intervals around I² [39] using the non-central chi-squared-based approach within the heterogi module for Stata [41]. We also calculated the Q statistic but only reported whether the result was significant.

Publication bias was tested by visually inspecting the funnel plot on primary outcome measures. We also conducted Egger’s test to quantify the bias captured by the funnel plot and to test whether it was significant [42]. The Duval and Tweedie trim-and-fill analysis [43] was performed to further verify an unbiased estimate of the pooled effect size. This method calculates an estimation of the number of missing studies and the effect that these studies might have had on its outcome.

We further conducted a series of subgroup-analyses, according to the mixed-effect model [44]. In this model, studies within subgroups are pooled with the random-effects model, while tests for significant differences between subgroups are conducted with the fixed-effects model.

Results

Selection and inclusion of studies

Thirteen studies met inclusion criteria for the meta-analysis of cCBT for depression and anxiety in youth. Fig. 1 presents a PRISMA flowchart describing the inclusion process. Two reviewers (Ebert & Zarski) independently reviewed studies for inclusion or exclusion. Agreement between raters on inclusion was 100%.

Characteristics of included studies

The 13 studies on cCBT for symptoms of anxiety and depression in youth included 796 participants, 420 in the treatment groups and 376 in the control groups. Selected characteristics of the included studies are displayed in Table 1. Seven studies were directed at treating anxiety, four at depression, two were transdiagnostic and targeted both depression and anxiety. Two studies were aimed at children (< 13 years), six studies at adolescents (≥ 13 years), five studies had a mixed sample. About half of the studies (n = 6) included only participants with a diagnosis, confirmed by an established diagnostic interview. The intervention in the majority of studies could be completed in the home of participants (n = 9).

[Figure omitted. See PDF.]

Table 1. Selected characteristics of randomised controlled studies examining the effects of cCBT and iCBT for anxiety and depression in children and adolescents.

https://doi.org/10.1371/journal.pone.0119895.t001

A non-treatment comparison was used in 10 studies, the other three studies applied a placebo control. The majority relied on a multi-method outcome assessment approach (i.e., observer-based outcome interview and self-report measure, n = 8). Only five studies used only self-report measures. Outcome informants were both youths and parents in seven studies and youth-only in six studies. Studies using youth-only as outcome informant were mainly directed at adolescents (n = 4), two studies on mixed-age range samples. Many studies included a follow-up assessment. However, only two studies assessed a follow-up in both the intervention and control group [44, 45]. Thus, we were not able to examine long-term effects of treatments.

In most studies, the intervention was delivered via the Internet (n = 8), followed by computer (n = 5). The number of treatment sessions ranged from 6 to 14. Six studies were classified as brief treatment (6–7 sessions) and seven as long treatment (≥ 8 sessions). Parents were involved in the intervention within six studies and the majority of interventions included guidance by a professional (n = 11). Four Studies were conducted in the USA and Australia, respectively, two studies were conducted in New Zealand, two studies in Sweden and one study was conducted in the UK and the Netherlands, respectively. Thus, all studies were conducted in high-income countries. Cohen's kappa for agreement between independent raters was very good (κ = .84).

Risk of bias

Overall risk of bias was low. Eleven studies reported an adequate sequence generation. Twelve studies reported blinding of outcome assessors or used only self-report outcomes, whereas five did not report blinding. Only eight studies reported the information needed to clarify whether allocation to conditions was performed by an independent (third) party. Ten studies reported adequate handling of missing data, using intention-to-treat principles. Seven studies met all four quality criteria, four studies met three of four criteria, and two studies met only one criterion. Inter-rater reliability between independent raters on the risk of bias was very good (κ = 0. 94).

Effects of cCBT on symptoms of anxiety/depression when compared to a control group

The overall mean effect size of cCBT on symptoms of anxiety or depression when compared to a control group at post-test was g = .72 (95% CI: 0.55 to 0.90; p <. 001; Table 2). This value corresponded to the number needed to be treated to achieve one additional positive outcome of 2.56. Heterogeneity was low (I² = 20.14%, 95% CI: 0% to 58%). Fig. 2 provides a forest plot of the effect sizes per study and the corresponding 95% CI.

[Figure omitted. See PDF.]

Table 2. Effects of computer and internet-based CBT for anxiety and depression in youth when compared with control groups at post-test: Hedges’ g.

https://doi.org/10.1371/journal.pone.0119895.t002

[Figure omitted. See PDF.]

Fig 2. Forest Plot.

https://doi.org/10.1371/journal.pone.0119895.g002

Excluding studies that also assessed participants above 18 years of age [51, 57, 58] resulted in an effect size of 0.61 (95% CI: 0.55 to 0.90). Excluding the study with the lowest effect size [46] further reduced heterogeneity (I² = 0, 95% CI: 0% to 58%), while excluding the study with the highest effect size [51] did not change heterogeneity substantially (I² = 15.11%, 95% CI: 0% to 55%).

Including also anxiety outcomes of interventions targeting depression, the effect size on anxiety was g = 0.65 (95% CI 0.40 to 0.90) with an NNT of 2.82. Including also depression outcomes of interventions targeting anxiety resulted in a pooled effect size of g = .56 (95% CI 0.31 to 0.82), NNT = 3.25.

Subgroup and moderator analyses

We conducted a series of subgroup analyses (Table 2). Effects in subgroups were all in favor of cCBT and all significantly different from zero. Moderator analyses revealed that age group significantly moderated treatment outcome, with studies aimed at adolescent achieving better results (g = 0.95, 95% CI 0.76 to 1.17) compared to studies aimed at children (g = 0.51, 95% CI: 0.11 to 0.92) and mixed samples (g = 0.48, 95% CI 0.25 to 0.71). Given that some meta-analyses on treatments of anxiety disorders in youth [15] categorized children up to 13 years of age as children, we repeated the analysis classifiyng the study of Khanna [49] as directed at children. This result did not produce a different outcome pattern, with slighty higher effects for children (g = 0.56, 95% CI 0.21 to 0.91, NNT = 3.25) but still a significant difference in effectiveness between age groups (p = .007). We found that target condition, confirmation of disorder using a diagnostic interview, parental involvement, or risk of bias was not significantly associated with the size of the effect (Table 2).

Publication bias

The inspection of the funnel plot and Egger’s test indicated some possible publication bias. However, after adjustment for missing studies using the Duval-Tweedie trim and fill procedure (four imputed studies) Hedge’s g for the main outcome analyses was 0.64 (95% CI 0.46 to 0.82), corresponding to an NNT of 2.86.

Discussion

We found cCBT for youth was associated with significant and moderate to large effects on symptoms of anxiety and depression, with an NNT of 2.56. Overall, risk of bias was low and the seven studies that met all quality criteria showed a significant effect comparable with the overall effect size. Heterogeneity was low in most analyses, suggesting that most studies pointed in the same direction, with no major outliers. We found some indications for publication bias, but adjusting effect sizes resulted in no major changes.

Effects sizes were slightly lower than those found for cCBT for anxiety and depressive disorders in adults (g = 0.88) [30], were comparable to those found in recent meta-analyses on traditional face-to-face CBT for anxiety disorders in youth (0.66) [15] and somewhat higher as those found for CBT for depression in youth (0.35) [16]. However, we also included studies with mixed samples that also included participants above the age of 18. Removing these studies resulted in an overall ES of 0.61 and NNT of about 2.99, still in range of ESs of face-to-face CBT. The comparable effects to face-to-face psychotherapy are consistent with research on cCBT for psychiatric and somatic disorders in adults, showing no differences between internet-based CBT and face-to-face treatments [59].

We also found significant, moderate but somewhat smaller effects compared to the main-outcome analysis on symptoms of depression when including also outcomes of interventions targeting anxiety. Likewise, we also found significant and moderate effects on symptoms of anxiety when also including outcomes of interventions targeting depression. These findings are consistent with growing evidence indicating that (a) youth depression and anxiety are closely associated [4, 5], (b) psychotherapeutic treatment for youth can have significant effects on comorbid problems [16], and (c) may be treated simultaneously [60]. In fact our results support such an assumption, with significant and large effects (g = 0.94, NNT = 2.02) for those treatments targeting both disorders at the same time [51, 53].

Participant age was the only significant moderator of treatment effects. The finding was consistent also when using different classifications of children (< 13 vs. ≤ 13 years of age). In line with the results of the most recent meta-analyses on psychotherapy for anxiety disorders in youth [15], we found that treatments for children were significant but with smaller effects compared to treatments for adolescents. However, meta-analyses on psychotherapy for depression in youth have not found such an association [16].

We found no association between the inclusion of parents and better treatment outcomes. Although it is often argued that parental involvement is important in the treatment of anxiety and depressive disorders in youth [61], our results are consistent with the meta-analyses of Reynolds and colleagues, showing no additional benefit of treatments with parental involvement [15].

When interpreting results of this study, several limitations should be considered.

First, 11 of 13 studies reviewed reported no follow-up assessment with treatment versus control comparison. Hence, we could not examine long-term effects of treatments. This shortcoming points to the need for future studies with follow-up assessments. Second, the interventions evaluated in the studies were very heterogeneous regarding format of treatment, ranging from therapist focused group-based chat interventions based on face-to-face treatment manuals [57] to unguided more serious gaming-based approaches [45, 53]. Given the large variability and the limited amount of studies available, we could not examine the differential effectiveness of treatment formats. Third, also because of the limited number of studies, we were not able to perform subgroup analyses with adequate power separately for depression and anxiety or other potential subgroup analyses of interest. Fourth, most studies targeting depression excluded participants with severe depression. Consequently, our results should not be generalized to youths with severe depression. Fifths, the present study did not search for unpublished studies. Thus it may be possible that there are unpublished studies that are not included here, which may lead to an over- or under-estimation of the true intervention effect sizes. Sixths, true levels of heterogeneity in meta-analyses cannot be detected with currently available statistical methods [62]. Although we applied a number of state-of-the art methods to estimate heterogeneity (i.e. I² including 95%-CIs, Q statistics, sensitivity analysis excluding studies with the lowest/highest effect size), we cannot rule out that heterogeneity exits and the reported effect sizes are over- or underestimated. Finally, all studies were conducted in high-income countries. The generalizability of the study findings for low- and middle income countries is therefore limited.

Findings from the present study provide evidence for the efficacy of cCBT in the treatment of anxiety and depressive symptoms in youth. Hence, such intervention might be a promising treatment alternative when evidence based face-to-face treatment is not available or simply not wished. Applying cCBT at large scale may thus serve as one (out of several) strategies to bridge the enormous gap between the need and provision of evidence-based treatments for anxiety and depression in children and adolescents. Given that access to evidence-based psychotherapeutic treatment is not only a problem in high, but even more in low- and middle-income countries [54], future studies should evaluate the potential of cCBT effectiveness for youths in non-high income countries. Subsequent studies should also examine the long-term effectiveness of cCBT and evaluate potential negative effects of cCBT [63]. Findings also indicate that more research is necessary to draw clear conclusions with regard to moderators. Finally, this meta-analysis was limited to studies including participants with elevated symptoms, excluding studies directed at the prevention rather than treatment of symptoms (e.g. [64]). Given the potential scalability of Internet-based treatments, there might be a large potential for their use in the prevention of common mental health disorders [65, 66], which should also be examined in future studies.

Supporting Information

[Figure omitted. See PDF.]

S1 PRISMA Checklist. PRISMA Checklist.

https://doi.org/10.1371/journal.pone.0119895.s001

(DOC)

S1 Text. Review Protocol.

https://doi.org/10.1371/journal.pone.0119895.s002

(DOC)

Author Contributions

Conceived and designed the experiments: DDE HR. Performed the experiments: DDE PC HR. Analyzed the data: DDE HR PC. Contributed reagents/materials/analysis tools: DDE MB. Wrote the paper: DDE AZ HC PC YS MB HR.

Citation: Ebert DD, Zarski A-C, Christensen H, Stikkelbroek Y, Cuijpers P, Berking M, et al. (2015) Internet and Computer-Based Cognitive Behavioral Therapy for Anxiety and Depression in Youth: A Meta-Analysis of Randomized Controlled Outcome Trials. PLoS ONE 10(3): e0119895. https://doi.org/10.1371/journal.pone.0119895

References

1. Costello EJ, Mustillo S, Erkanli A, Keeler G, Angold A. Prevalence and development of psychiatric disorders in childhood and adolescence. Arch Gen Psychiatry. 2003;60(8):837–44. pmid:12912767

2. Ford T, Goodman R, Meltzer H. The british child and adolescent mental health survey 1999: the prevalence of DSM-IV disorders. J Am Acad Child Adolesc Psychiatry. 2003;42(10):1203–11. pmid:14560170

3. Merikangas KR, He JP, Burstein M, Swanson SA, Avenevoli S, Cui L, et al. Lifetime prevalence of mental disorders in U.S. adolescents: results from the national comorbidity survey replication—adolescent supplement (NCS-A). J Am Acad Child Adolesc Psychiatry. 2010;49(10):980–9. pmid:20855043

4. Brady EU, Kendall PC. Comorbidity of anxiety and depression in children and adolescents. Psychol Bull. 1992;111(2):244–55. pmid:1557475

5. Axelson DA, Birmaher B. Relation between anxiety and depressive disorders in childhood and adolescence. Depress Anxiety. 2001;14(2):67–78. pmid:11668659

6. Angold A, Messer SC, Stangl D, Farmer EM, Costello EJ, Burns BJ. Perceived parental burden and service use for child and adolescent psychiatric disorders. Am J Public Health. 1998;88(1):75–80. pmid:9584037

7. Bodden DHM, Dirksen CD, Bögels SM. Societal burden of clinically anxious youth referred for treatment: a cost-of-illness study. J Abnorm Child Psychol. 2008;36(4):487–97. pmid:18214667

8. Angold A, Costello EJ. Depressive comorbidity in children and adolescents: empirical, theoretical, and methodological issues. Am J Psychiatry. 1993;150(12):1779–91. pmid:8238631

9. Hofstra MB, van der Ende J, Verhulst FC. Child and adolescent problems predict DSM-IV disorders in adulthood: a 14-year follow-up of a dutch epidemiological sample. J Am Acad Child Adolesc Psychiatry. 2002;41(2):182–9. pmid:11837408

10. Orvaschel H, Lewinsohn PM, Seeley JR. Continuity of psychopathology in a community sample of adolescents. J Am Acad Child Adolesc Psychiatry. 1995;34(11):1525–35. pmid:8543521

11. Woodward LJ, Fergusson DM. Life course outcomes of young people with anxiety disorders in adolescence. J Am Acad Child Adolesc Psychiatry. 2001;40(9):1086–93. pmid:11556633

12. Parker G, Roy K, Hadzi-Pavlovic D, Mitchell P, Wilhelm K. Distinguishing early and late onset non-melancholic unipolar depression. J Affect Disord. 2003;74(2):131–8. pmid:12706514

13. Rao U, Ryan ND, Birmaher B, Dahl RE, Williamson DE, Kaufman J, et al. Unipolar depression in adolescents: clinical outcome in adulthood. J Am Acad Child Adolesc Psychiatry. 1995;34(5):566–78. pmid:7775352

14. James AC, James G, Cowdrey FA, Soler A, Choke A. Cognitive behavioural therapy for anxiety disorders in children and adolescents. Cochrane database Syst Rev. 2013;6:CD004690. pmid:23733328

15. Reynolds S, Wilson C, Austin J, Hooper L. Effects of psychotherapy for anxiety in children and adolescents: a meta-analytic review. Clin Psychol Rev. 2012;32(4):251–62. pmid:22459788

16. Weisz JR, McCarty CA, Valeri SM. Effects of psychotherapy for depression in children and adolescents: a meta-analysis. Psychol Bull. 2006;132(1):132–49. pmid:16435960

17. Weisz JR, Kuppens S, Eckshtain D, Ugueto AM, Hawley KM, Jensen-Doss A. Performance of evidence-based youth psychotherapies compared with usual clinical care: a multilevel meta-analysis. JAMA psychiatry. 2013;70(7):750–61. pmid:23754332

18. Klein JB, Jacobs RH, Reinecke MA. Cognitive-behavioral therapy for adolescent depression: a meta-analytic investigation of changes in effect-size estimates. J Am Acad Child Adolesc Psychiatry. 2007;46(11):1403–13. pmid:18049290

19. Compton SN, March JS, Brent D, Albano AM, Weersing VR, Curry J. Cognitive-behavioral psychotherapy for anxiety and depressive disorders in children and adolescents: an evidence-based medicine review. J Am Acad Child Adolesc Psychiatry. 2004;43(8):930–959. pmid:15266189

20. Burns BJ, Costello EJ, Angold A, Tweed D, Stangl D, Farmer EM, et al. Children’s mental health service use across service sectors. Health Aff (Millwood). 1995;14(3):147–59. pmid:7498888

21. Essau CA. Frequency and patterns of mental health services utilization among adolescents with anxiety and depressive disorders. Depress Anxiety. 2005;22(3):130–7.

22. Zachrisson HD, Rödje K, Mykletun A. Utilization of health services in relation to mental health problems in adolescents: a population based survey. BMC Public Health. 2006;6:34. pmid:16480522

23. Sawyer MG, Arney FM, Baghurst PA, Clark JJ, Graetz BW, Kosky RJ, et al. The mental health of young people in Australia: key findings from the child and adolescent component of the national survey of mental health and well-being. Aust N Z J Psychiatry. 2001;35(6):806–14. pmid:11990891

24. Kataoka SH. Unmet need for mental health care among U.S. children: variation by ethnicity and insurance status. Am J Psychiatry. 2002;159(9):1548–1555. pmid:12202276

25. Gulliver A, Griffiths KM, Christensen H. Perceived barriers and facilitators to mental health help-seeking in young people: a systematic review. BMC Psychiatry. 2010;10(1):113.

26. Lin J, Ebert DD, Lehr D, Berking M, Baumeister H. [Internet based cognitive behavioral interventions: state of the art and implementation possibilities in rehabilitation]. Die Rehabilitation, 52(3), 2013:52(3):155–63 pmid:23761204

27. Cunningham MJ, Wuthrich VM, Rapee RM, Lyneham HJ, Schniering CA, Hudson JL. The Cool Teens CD-ROM for anxiety disorders in adolescents: a pilot case series. Eur Child Adolesc Psychiatry. 2009;18(2):125–9. pmid:18563472

28. Calear AL, Christensen H. Review of internet-based prevention and treatment programs for anxiety and depression in children and adolescents. Med J Aust. 2010;192(11 Suppl):S12–4.

29. Hedman E, Ljótsson B, Lindefors N. Cognitive behavior therapy via the Internet: a systematic review of applications, clinical efficacy and cost-effectiveness. Expert Rev Pharmacoecon Outcomes Res. 2012;12(6):745–64. pmid:23252357

30. Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One. 2010;5(10):e13196. pmid:20967242

31. Haug T, Nordgreen T, Öst LG, Havik OE. Self-help treatment of anxiety disorders: a meta-analysis and meta-regression of effects and potential moderators. Clin Psychol Rev. 2012;32(5):425–45. pmid:22681915

32. Higgins JM, Altman DG. Assessing Risk of Bias in included studies. In: Cochrane Handbook for Systematic Reviews of Interventions: John Wiley & Sons, Ltd. pp; 2008:187–241.

33. Kazdin AE. Research Design in Clinical Psychology (4th Edition) 4th (fourth) Edition by Kazdin, Alan E. (2002). Pearson; 1000.

34. Weisz JR, Weiss B, Han SS, Granger DA, Morton T. Effects of psychotherapy with children and adolescents revisited: a meta-analysis of treatment outcome studies. Psychol Bull. 1995;117(3):450–68. pmid:7777649

35. Michael KD, Crowley SL. How effective are treatments for child and adolescent depression? A meta-analytic review. Clin Psychol Rev. 2002;22(2):247–69. pmid:11806021

36. Bennett SM, Ehrenreich ÆJT. Emotion Regulation in Youth with Emotional Disorders: Implications for a Unified Treatment Approach. Psychol Rev. 2009:234–254.

37. Kraemer HC, Kupfer DJ. Size of treatment effects and their importance to clinical research and practice. Biol Psychiatry. 2006;59(11):990–6.

38. Laupacis A, Sackett DL, Roberts RS. An assessment of clinically useful measures of the consequences of treatment. N Engl J Med. 1988;318(26):1728–33. pmid:3374545

39. Ioannidis JPA, Patsopoulos NA, Evangelou E. Uncertainty in heterogeneity estimates in meta-analyses. BMJ. 2007;335(7626):914–6. pmid:17974687

40. Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557–60. pmid:12958120

41. Orsini N, Higgins J, Bottai M, Buchan I. Heterogi: Stata module to quantify heterogeneity in a Meta-analysis. 2013. Available at: http://econpapers.repec.org/RePEc:boc:bocode:s449201. Accessed December 15, 2013.

42. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315(7109):629–34. pmid:9310563

43. Duval S, Tweedie R. Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics. 2000;56(2):455–63.

44. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to Meta-Analysis. Chichester, UK: John Wiley & Sons, Ltd. 2009.

45. Stasiak K, Hatcher S, Frampton C, Merry SN. A pilot double blind randomized placebo controlled trial of a prototype computer-based cognitive behavioural therapy program for adolescents with symptoms of depression. Behav Cogn Psychother. 2012;1–17.

46. Makarushka MM, Unruh D, Seeley JR, Hodges S. Efficacy of an internet-based intervention targeted to adolescents with subthreshold depression [dissertation]. Oregon, USA: University of Oregon; 2011.

47. Fleming T, Dixon R, Frampton C, Merry S. A pragmatic randomized controlled trial of computerized CBT (SPARX) for symptoms of depression among adolescents excluded from mainstream education. Behav Cogn Psychother. 2012;40(5):529–41. pmid:22137185

48. Keller M. An internet cognitive-behavioral skills-based program for child anxiety—Udini. Dissertation. 2009. Available at: http://udini.proquest.com/view/an-internet-cognitive-behavioral-pqid:1973587951/.

49. Khanna MS, Kendall PC. Computer-assisted cognitive behavioral therapy for child anxiety: results of a randomized clinical trial. J Consult Clin Psychol. 2010;78(5):737–45. pmid:20873909

50. March S, Spence SH, Donovan CL. The efficacy of an internet-based cognitive-behavioral therapy intervention for child anxiety disorders. J Pediatr Psychol. 2009;34(5):474–87. pmid:18794187

51. Sethi S, Campbell AJ, Ellis LA. The use of computerized self-help packages to treat adolescent depression and anxiety. J Technol Hum Serv. 2010;28(3):144–160.

52. Spence SH, Donovan CL, March S, Gamble A, Anderson RE, Prosser S, et al. A randomized controlled trial of online versus clinic-based CBT for adolescent anxiety. J Consult Clin Psychol. 2011;79(5):629–42. pmid:21744945

53. Stallard P, Richardson T, Velleman S, Attwood M. Computerized CBT (Think, Feel, Do) for depression and anxiety in children and adolescents: outcomes and feedback from a pilot randomized controlled trial. Behav Cogn Psychother. 2011;39(3):273–84. pmid:21272393

54. Storch EA, Caporino NE, Morgan JR, Lewin AB, Rojas A, Brauer L, et al. Preliminary investigation of web-camera delivered cognitive-behavioral therapy for youth with obsessive-compulsive disorder. Psychiatry Res. 2011;189(3):407–12. pmid:21684018

55. Tillfors M, Andersson G, Ekselius L, Furmark T, Lewenhaupt S, Karlsson A, et al. A randomized trial of internet-delivered treatment for social anxiety disorder in high school students. Cogn Behav Ther. 2011;40(2):147–157. pmid:25155815

56. Wuthrich VM, Rapee RM, Cunningham MJ, Lyneham HJ, Hudson JL, Schniering C. A randomized controlled trial of the Cool Teens CD-ROM computerized program for adolescent anxiety. J Am Acad Child Adolesc Psychiatry. 2012;51(3):261–70. pmid:22365462

57. Van der Zanden R, Kramer J, Gerrits R, Cuijpers P. Effectiveness of an online group course for depression in adolescents and young adults: a randomized trial. J Med Internet Res. 2012;14(3):e86. pmid:22677437

58. Tillfors M, Carlbring P, Furmark T, Lewenhaupt S, Spak M, Eriksson A, et al. Treating university students with social phobia and public speaking fears: Internet delivered self-help with or without live group exposure sessions. Depress Anxiety. 2008;25(8).

59. Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. (In press). Internet-based vs. face-to-face cognitive behaviour therapy for psychiatric and somatic disorders: A systematic review and meta-analysis. World Psychiatry.

60. Ehrenreich JT, Goldstein CM, Wright LR, Barlow DH. Development of a Unified Protocol for the Treatment of Emotional Disorders in Youth. Child Fam Behav Ther. 2009;31(1):20–37.

61. Barrett PM, Dadds MR, Rapee RM. Family treatment of childhood anxiety: a controlled trial. J Consult Clin Psychol. 1996;64(2):333–42. pmid:8871418

62. Kontopantelis E, Springate DA, Reeves D. A re-analysis of the Cochrane Library data: the dangers of unobserved heterogeneity in meta-analyses. 2013. PloS One, 8(7).

63. Rozental A, Andersson G, Boettcher J, Ebert DD, Cuijpers P, Knaevelsrud C, et al. Consensus statement on defining and measuring negative effects of Internet interventions. Internet Interv. 2014;1(1):12–19.

64. Calear AL, Christensen H, Mackinnon A, Griffiths KM, O’Kearney R. The YouthMood Project: a cluster randomized controlled trial of an online cognitive behavioral program with adolescents. J Consult Clin Psychol. 2009;77(6):1021–32. pmid:19968379

65. van Zoonen K, Buntrock C, Ebert DD, Smit F, Reynolds CF, Beekman A, et al. Preventing the onset of major depressive disorder: A meta-analytic review of psychological interventions. Int. J. Epidemiol. 2014;43(2):318–29. pmid:24760873

66. Buntrock C, Ebert DD, Lehr D, Cuijpers P, Riper H, Smit F, et al. Evaluating the efficacy and cost-effectiveness of web-based indicated prevention of major depression: design of a randomised controlled trial. BMC Psychiatry 2014;14(1):25. pmid:24552142