Intensive behavioural interventions based on applied behaviour analysis (ABA) for young children with autism: A cost-effectiveness analysis

Abstract

Background

The economic and social costs of autism are significant. This study evaluates the cost-effectiveness of early intensive Applied Behaviour Analysis (ABA)-based interventions for autistic pre-school children in the UK.

Methods

A de novo economic analysis was developed in Microsoft Excel comparing early intensive ABA-based interventions compared with treatment as usual (TAU). The analysis used 15.5-year time horizon, with costs and benefits discounted a 3.5%. The model structure was based on cohort structure to capture changes in adaptive behaviour and cognitive ability over time. The analysis was informed by an individual patient data (IPD) meta-analysis of available evidence.

Results

Adopting a public sector perspective, early intensive ABA-based therapies were associated with greater incremental costs and greater benefits. When pessimistic assumptions were made regarding the long-term effects of treatment incremental costs were £46,103 and incremental quality-adjusted life years (QALYs) were 0.24, resulting in an incremental cost-effectiveness ratio (ICER) of £189,122 per quality-adjusted life year (QALY). When optimistic assumptions were made about long-term effects, incremental costs were £39,233 with incremental benefits of 0.84 QALYs. The resulting ICER was £46,768 per QALY. Scenario analyses emphasised the importance of assumptions made regarding adult outcomes and type of school attended, both of which significantly affect the results of the analysis.

Conclusions

The results of this economic analysis suggest that early intensive ABA-based interventions are unlikely to represent value for money, based on a £20,000 to £30,000 per QALY threshold typically adopted to inform UK healthcare funding decisions. However, important gaps in the available evidence, limit the strength of the conclusions that can be drawn from the presented analysis. Further research, focusing on the trajectory of autistic children following intervention is likely to be highly beneficial to resolving some of these uncertainties.

Introduction

Autism spectrum disorder (henceforth referred to as autism) has significant social and economic impacts for individuals, their families and wider society [1–3]. Although the skills and needs of autistic children and their families are highly variable, they can have a profound effect on children’s development into adulthood [4, 5]. For example, the available evidence suggests that autistic adults are likely to have poorer educational attainment, lower rates of employment, are less likely to be living independently, and experience higher rates of mental and physical health problems compared with the general population and adults with other disabilities [6–8]. The estimated total costs to the United Kingdom (UK) of supporting autistic people and related conditions has been estimated at between £32.1 and £34.4 billion per year, with higher lifetime care costs reported for individuals with co-occurring intellectual disability [9].

Early intensive applied behaviour analysis (ABA) based interventions include a range of interventions that aim to positively impact a child’s developmental by shifting a child’s developmental trajectory through early interventions [10] and are typically delivered to young autistic children for several years on a one-to-one basis, for between 20 to 50 hours per week [11].

Early intensive ABA-based interventions have been variably defined to include a range of specific interventional approaches but minimally include early intensive behavioural interventions (EIBI) as first described by Lovaas in the late 1960s [10]. EIBI, as developed by Lovaas, is a behavioural interventional approach based on the principles of ABA and emphases teaching skills through structured task completion and rewards. EIBI has subsequently been adapted and developed to incorporate the principles of ABA within a more naturalistic and developmentally informed framework. Collectively known as Naturalistic Developmental Behavioural Interventions (NDBIs) [12], these techniques combine behavioural based techniques with child-led and incidental teaching. Specific examples include Pivotal Response Treatment (PRT) [13] and the Early Start Denver Model (ESDM) [14]. For the purposes of this study, we use ‘early intensive ABA-based interventions’ in the broad sense to refer to both EIBI and NDBI based approaches. Early intensive ABA-based interventions are not routinely delivered in the UK as part of state early years provision. There is, however, some provision for school-age children in parts of the country, as well as private provision for those able to self-fund [15–18]. Expanding early intensive ABA-based intervention provision would require significant investment on behalf of UK local authorities who are responsible for providing regional education and social care services. It may also require investment from clinical commissioner groups and local National Health Services (NHS) trusts who are responsible for regional health care expenditure. Several economic analyses have been conducted considering the value of implementing early intensive ABA-based interventions [19–21]. These studies suggest these interventions are highly cost-effective. However, none consider a UK perspective and they make several strong assumptions, particularly regarding effectiveness and long-term outcomes associated with early intensive ABA-based interventions. Most of the previous studies also only account for costs and do not consider the value of health improvements and other benefits.

The objective of this study was to evaluate the potential effectiveness and cost-effectiveness of early intensive ABA-based interventions in pre-school children from a UK perspective as part of a National Institute for Health Research (NIHR) funded health technology assessment (HTA). The full technical report [22] including an individual patient data (IPD) meta-analysis of the available primary research evidence on the effectiveness of early intensive ABA-based interventions is reported elsewhere [23]. This study presents the assessment of the cost-effectiveness of early intensive ABA-based intervention in a UK context. Notwithstanding the studies reported above, we are aware that cost-effectiveness analysis has been rarely conducted to assess interventions for treatment management of autism. For those unfamiliar with the underlying principles, we recommend readers consult one of the many introductory texts [24–26].

As part of the HTA, the economic evaluation utilised data from the IPD meta-analysis which evaluated the effectiveness of early intensive applied behaviour analysis (ABA)-based interventions for preschool autistic children compared with treatment as usual (TAU). Details of the IPD-meta-analysis including a justification of the adopted inclusion criteria and methods reported in full in the technical reports as well as in associated publication [22]. In brief, the IPD-meta-analysis found that early intensive ABA-based interventions produced statistically significant improvements in cognitive ability and adaptive behaviour after two years as compared to treatment as usual (including eclectic interventions), with limited evidence of effects on other outcomes. The studies that informed these estimates were, however, found to be at high risk of bias, with all studies using non-randomised designs and in many cases convenience samples. Further, while the nature of the interventions meant that blinding of education staff and participants was not possible, outcome assessors were also often not blinded to intervention. Importantly from the perspective of the economic analysis, most studies also lacked long-term follow-up data, meaning there was minimal evidence to support effects beyond two years.

Methods

The economic analysis sought to compare early intensive ABA-based interventions with TAU which may include eclectic interventions in a population of pre-school autistic children. To assess and compare the cost-effectiveness of early intensive ABA-based interventions and TAU we considered both expected costs and utility (effectiveness) associated with them. Early intensive ABA-based interventions were assumed to represent any type of intensive ABA-based intervention and did not distinguish between subtypes (e.g. EIBI vs NDBIs). This was due to lack of evidence to support differential effects between subtypes [22]. TAU was modelled to represent current provision for young children with autism in the UK.

The de novo analysis was designed and developed in collaboration with UK and international experts and was also able to draw on an advisory group that included representation from autistic people, parents and practitioners (see S1 File). To reflect the fact that autism is a long-term neurodevelopmental condition and is likely to accrue benefits and have cost implications that extend beyond the healthcare system, the economic analysis was undertaken from both health and social care services, and a public sector perspective which included costs accruing to the education sector. The health and social care services perspective included all health benefits along with costs incurred by the health sector, such as any direct NHS medical costs and costs of social care. The public sector perspective added a broader range of costs borne by the state, including, for example, costs of providing education.

Expected utilities were expressed in quality-adjusted life years (QALYs) which are a function of quality and length of life [27], to allow for a direct comparison with other health care interventions and for incremental cost-effectiveness ratio (ICER; additional quality-adjusted life years (QALY) divided by additional costs) to be estimated. The advantage of this approach is that it allows the result of the economic analysis to benchmarked against willingness to pay (WTP) criteria commonly used to inform decisions on the value of healthcare technologies in the UK. To benchmark the potential cost-effectiveness of early intensive ABA-based interventions, ICERs were compared with the UK National Institute for Health and Care Excellence (NICE) published decision rules, which outline a WTP threshold used in the NHS (£20,000 to £30,000 per QALY) and set the maximum ICER acceptable for new expenditure in the NHS [28]. We, however, note that alternative willingness to pay thresholds may be relevant to implementation decision in this context because at least some of the funding and benefits of autism interventions lie outside the health sector. There are, however, no set WTP for other sectors as cost-utility analysis is not routinely used in other sectors e.g. education. Costs and outcomes were discounted using a 3.5% annual discount rate, in line with current NICE guidelines [28].

Model structure

The economic analysis was constructed in Microsoft Excel^® and used a simple cohort approach. Under this approach mean cognitive ability scores (IQ) and Vineland Adaptive Behaviour scores (VABs) for the cohort are used to predict model outcomes [29]. These included educational costs, social care and medical costs as well as quality of life measured in QALYs. The focus on cognitive ability and adaptative behaviour reflects the limitations of the available effectiveness which primarily reports on changes in these two outcome measures [30]. An overview of the model structure is described in Fig 1. The analysis uses a cycle length of one month (representing the shortest time period over which outcomes were measured) with a half-cycle correction applied to correct for the fact that events can occur at any point in a cycle. Changes in mean cognitive ability and adaptative behaviour scores were therefore incorporated as mean changes per month.

[Figure omitted. See PDF.]

The time horizon considered in the analysis was assumed to be 15.5 years, to represent the period from age 3 years to the average age children leave secondary education (18.5 years). The relatively short time horizon of 15.5 years was selected because of considerable uncertainty about the long-term effects of early intensive ABA-based interventions and the lack of epidemiological evidence linking short and long-term outcomes [23].

The time horizon in the main analysis was divided into three phases representing different stages of an individual’s childhood. The first phase “pre-school” (up to 4.5 years old) reflects the period when early interventions are delivered, while the second “school-age” is split into two sub-phases covering the period in which children attend primary (4.5 to 11.5 years old) and secondary education (11.5 to 18.5 years old). To accommodate educational placement and the associated resource implications, the analysis distinguished between three education settings: mainstream, mainstream with support, and special education provision. Estimation of the proportion of children receiving each type of education was determined at the beginning of the primary and secondary phase of the model and estimated using logistic regression models. These determined the proportion of children in each educational setting based on the cognitive ability and adaptative behaviour scores of the cohort, with separate models fitted for primary and secondary age children, see Table 1 for the coefficients applied and data sources.

[Figure omitted. See PDF.]

Inputs

To populate the inputs used in the economic analysis we conducted a series of formal systematic reviews and undertook additional primary data analysis to develop and populate the economic analysis. Details of each review and data sources are provided in the full technical report [22]. An overview of the structure of the economic analysis and input parameters is outlined below. A summary table of all input parameters used in the analysis are reported in Table 1.

Uncertainty in parameter inputs was explored in deterministic sensitivity and scenario analysis as well as in probabilistic sensitivity analysis (PSA) which represent input parameters as distributions around the mean estimate. The PSA was undertaken using Monte Carlo sampling methods, using 10,000 samples. The choice of distribution for individual parameters was selected in accordance with their statistical suitability, see Table 1.

Starting characteristics except age were drawn from the IPD meta-analysis, these included biological sex and the outcome measures: Autism Diagnostic Observation Schedule (ADOS) [41], cognitive ability (IQ) and adaptive behaviour (VABS). Starting age was set to 3 to ensure consistency and differed only slightly from the average baseline age (3.16 years) reported by studies included in the IPD meta-analysis.

Treatment effect.

In the TAU arm of the economic analysis, changes in cognitive ability and adaptive behaviour scores were modelled using the four studies identified in the systematic review that had follow-up duration greater than two years [31–34]; these were then interpolated assuming a linear trend. These predicted small declines in cognitive and adaptive behaviour measures and aligned with much of the epidemiological research which show small declines in scores over time [42–48].

To model the treatment effect, in the early intensive ABA-based arm, cognitive ability and adaptive behaviour scores were modelled by applying the treatment effect derived from the IPD meta-analysis [23]. Cognitive ability and adaptative behaviour scores used in the early intensive ABA-based interventions arm of the economic analysis were therefore the sum of the score predicted for TAU plus the treatment effect for intensive ABA.

The treatment effect was modelled in two phases: a short-term phase, covering the first two years and a long-term phase, covering two years and onwards. This distinction was made to reflect that most of the studies included in the IPD meta-analysis had no more than two years of follow-up. In the first two years (up to cycle 24), the economic analysis applied the treatment effect at one year and two years from the IPD meta-analysis (see Table 1 for input values). Evidence on the longer-term effects of early intensive ABA-based interventions on cognitive ability and adaptive behaviour scores, and in particular the degree to which any early benefits are sustained, is very limited. Very few studies identified in the systematic review reporting outcomes post end of therapy and there have been relatively few attempts to follow up children after conclusion of the short-term studies, with a few notable exceptions [49–51]. To reflect this uncertainty in long-term effects of treatment both optimistic and a pessimistic long-term scenarios were explored.

Under the optimistic scenario, the treatment effect at 2 years was assumed to persist throughout the time horizon of the analysis, while under the pessimistic scenario it was assumed that the treatment effect dissipated over time, such that at seven years no additional treatment effect with ABA-based interventions remained. This time limit reflects the maximum period of follow- up by a study included in the IPD meta-analysis [34] acknowledging that any further changes in the treatment effect are unknown. In both scenarios, these increases/decreases were modelled as a linear trend because evidence from intermediate time points was unavailable.

Mortality

Several epidemiological studies have demonstrated that people with autism experience reduced life expectancy relative to the general population. To account for this, autism specific mortality rates were applied using relative risks reported in Hirvikoski et al. [52] and applied to general population rates reported by the UK Office of National Statistics. Mortality was, however, not linked to intervention effectiveness.

Health-related quality of life

A systematic search of studies reporting utility scores of autistic children was undertaken and identified several studies- full details of this review are reported in the full technical [22]. Of the studies identified, only one reported utility scores in a way that could be linked meaningfully with the outcomes reported in the IPD meta-analysis (this was primarily because reported values did not differentiate between different levels of severity/ability). This study recruited 224 children (aged 4–17 years) with autism and related conditions, including children with co-existing intellectual disability [37]. Quality of life was estimated at each time point based on reported algorithms, using adaptive behaviour, age, baseline cognitive ability (IQ) and baseline autism symptom severity (ADOS scores) as predictors of quality of life. Quality of life scores therefore changed in accordance with changes in cognitive ability, adaptive behaviour scores and age over time. In the lifetime horizon scenario analysis, the age parameter was held constant at 18 years and age-related decrements applied to account for the natural effects of ageing on utility scores. These decrements were calculated based on published vales using a published algorithm of utility scores in autistic children [53].

Resource use

The base price year was 2016/17 as this was the most recent year of publication for Personal Social Services Research Unit (PSSRU) [38] available and inflation indices for 2017/2018 were not available at the time. Prices reported in alternative cost years were inflated using inflation indices reported in PSSRU.

For the purposes of this study, early intensive ABA-based intervention was defined as consisting of 30 hours of one-to-one sessions [23]. The duration of treatment was set at 24 months as this aligns with the maximum follow-up period available for the majority of the clinical research studies included in the IPD meta-analysis [23]. It is, however, acknowledged that in practice children may continue to receive ABA-based treatment, including top-up therapy, through the school years. The modelled assumptions may therefore underestimate the true costs of ABA-based interventions. The economic analysis also does not account for discontinuation rates as no study reported compliance of fidelity to task.

Costs for early intensive ABA-based interventions were derived from published sources [38, 39] and set at £36,682 per annum. Costs of TAU were primarily based on UK national funding structures and information obtained from three local authorities on special educational needs (SEN) funding. The estimated cost of nursery provision was £8,634 per annum. The analysis also considered local authority provision of TAU intervention with costs based on funding information provided by the City of York and Newcastle local authorities [54].

UK costs associated with each type of school education and the costs for social and medical (NHS) care were drawn from a re-analysis of IPD obtained from Barrett et al [36] in a sample of adolescents with autism and Asperger’s syndrome. These costs were then linked to outcomes in the economic analysis using a regression equation in which cognitive ability (IQ) and adaptive behaviour scores (VABs) were used to predict total social care and medical costs. UK care costs in adulthood were based primarily on values reported in Buescher et al. [9] and values reported in PSSRU [38].

Sensitivity and scenario analysis

Uncertainty in parameter inputs was explored in deterministic sensitivity and scenario analysis as well as in probabilistic sensitivity analysis. Details of which are explained below.

Deterministic sensitivity analysis.

One-way deterministic sensitivity analysis was performed to explore the impact of single parameters on the results of the economic analysis. In this analysis each parameter was varied according to its 95% CI or standard error, while holding all other parameters constant. All parameters with uncertainty were included in the sensitivity analyses, this excluded time horizon and discount rates, which were assumed to be fixed. For a detailed list of the parameters varied and range of variation tested in the one-way univariate sensitivity analysis, see Table 1.

Scenario analysis.

Two scenario analysis were explored key assumptions made the in the base-case economic analysis.

The first explored the data used assess the impact of each intervention on education outcomes. In the base-case analysis education outcomes are modelled using an indirect link in which cognitive ability and adaptive behaviour scores are used to predict education outcomes. This approach allows for a consistency in the studies used to inform the relative effectiveness of early ABA-based interventions. Several studies, however, report direct evidence on the impact of early ABA-based interventions and therefore can be used to directly inform education outcomes avoiding the need for an indirect link. Scenario analysis therefore explores using these studies to inform education placement. In this scenario analysis, children are assumed to attend the same type of education throughout childhood (see Table 1 for proportions used).

The second scenario analysis explored a lifetime time horizon. This acknowledges that early interventions may have long-lasting and potentially permanent effects on an individual that last not only through childhood but into adulthood. In this scenario an additional phase was added to the structure of the economic analysis, with five levels of independence: “Completely independent”, “Mostly independent”, “Some independence”, “Mostly dependent” and “Completely dependent”. The definitions used for each category were based on those used in Howlin et al., [4] see technical report for details [22].

Independence levels were determined on entering the adult phase of the analysis based on adaptive behaviour scores at 18.5 years of age. Due to the lack of available data regarding changes in independence over time, it was assumed that independence levels remained constant throughout adulthood. The validity of this assumption is uncertain, as it does not include any consideration of an individual’s potential for change, or the impact of any existing or additional health and/or mental health needs.

Evidence linking adult levels of independence to childhood cognitive ability and adaptive behaviour is extremely limited. Although several studies report strong correlations between adult cognitive ability, adaptive behaviour scores and adult levels of independence, data are rarely reported in a way that allows predictions about adult outcomes to be made [47, 55–57]. Farley et al., [56] however, reports mean adaptive behaviour scores for several different independence levels. Using this summary, we were able to create simulated IPD to which a logistic regression model with independence levels as coefficients was fitted. This algorithm was then used to predict adult outcomes. The details of the methodology used are reported in the full technical report [22]. The coefficients generated from this analysis are reported in Table 1.

Probabilistic sensitivity analysis.

Uncertainty in parameter inputs was also explored in a probabilistic sensitivity analysis which represents input parameters as distributions around the mean estimate. Distributions for each parameter input were informed by the 95% CI, and additional literature. The probabilistic analysis ran the analysis using10,000 samples and was undertaken using simple Monte Carlo sampling methods (this involves random draws from the distributions for each iteration of the economic analysis). The choice of distribution for individual parameters was selected in accordance with their statistical suitability, see Table 1. The probability that early intensive ABA-based interventions are cost-effective was also investigated by considering the proportion of iterations where early the ICER was below the willingness to pay threshold.

Results

Deterministic analysis

Table 2 presents the base-case results of the economic analysis using both a health and social care services payer perspective and a wider public sector perspective. In all scenarios, in line with the IPD-meta-analysis results [22], the analysis predicted that the use of early intensive ABA-based interventions was associated with improved outcomes. The magnitude of accrued benefits differed substantially depending upon the assumptions made about the durability of the treatment effect, with optimistic scenarios predicting QALY benefits more than three times greater than under the pessimistic scenarios. While incremental costs were always positive due to the implementation costs of early intensive ABA-based interventions, their magnitude differed substantially across scenarios. This was driven by the degree of cost offsets (due to lower medical or educational costs), resulting from improved outcomes, and was a function of the perspective taken. This was a consequence of the limited scope for costs offsets when adopting a health and social care services payer perspective and as such incremental costs are lowest when the broader public sector is adopted.

[Figure omitted. See PDF.]

Using the UK NICE decision rules to benchmark the results of the cost-effectiveness analysis, early intensive ABA-based interventions would not be considered cost-effective in either the optimistic or pessimistic scenario. Under a health and social care services payer perspective, the pessimistic scenario suggests that early intensive ABA-based interventions would need to generate a further 1.68 QALYs or £50,547 in additional cost savings (not captured by the economic analysis) to meet the maximum NICE threshold of £30,000 per QALY. In the optimistic scenario, early intensive ABA-based interventions would only be cost-effective at a threshold of £30,000 per QALY if there were a further 1.07 QALYs or £32,117 in additional cost savings not captured by the analysis.

When adopting a public sector perspective, the size of the additional QALY benefits or cost savings required is lower. In the pessimistic scenario, early intensive ABA-based interventions would need to generate either a further 1.22 QALYs worth of additional health or non-health benefits or a further £36,608 in additional costs savings to be cost-effective at this threshold. In the optimistic scenario, this is reduced to either 0.37 QALYs worth of benefits or £11,126 in cost savings.

Sensitivity analysis

Figs 2 and 3 present the results of the deterministic sensitivity analysis in the form of a tornado diagram. This summarises the results of the 10 most influential parameters on the ICER, with each bar representing the variation in the ICER for that parameter. These analyses indicate early intensive ABA-based interventions remained cost-ineffective across a large range of parameter values. The magnitude of the treatment effect and cost of specialised education were the key drivers of the analysis.

[Figure omitted. See PDF.]

[Figure omitted. See PDF.]

Scenario analysis

Results of two scenario analysis are presented in Table 3 assuming a public sector perspective.

[Figure omitted. See PDF.]

The first scenario assumes a lifetime horizon in which adult outcomes are included. The second replaces the regression analysis used to predict education placement with observed data on education outcomes from three studies included in the systematic review and IPD meta-analysis. [10, 34, 58].

The impact of adopting a longer time horizon depends upon assumptions made about the durability of the treatment effect. Under the pessimistic scenario, where the treatment effect dissipates to zero after seven years, the ICER does not differ from the base-case analysis. In contrast, in the optimistic scenario, early intensive ABA-based interventions were found to dominate TAU, now generating greater benefits and lower incremental costs. This is because the longer time horizon allows greater scope benefits to be accrued as well as resulting in additional cost-savings.

The impact of using the direct evidence on education outcomes is significant in both the pessimistic and optimistic scenarios, in both cases, the resulting ICER is substantially lower than in the base-case analysis as a consequence of reduced incremental costs. The ICER in the pessimistic scenario, however, remains above the NICE threshold of £30,000 per QALY gained.

Probabilistic sensitivity analysis

The results of the probabilistic analysis are presented in Table 4 and broadly correspond with those of the deterministic analysis (see Table 2).

[Figure omitted. See PDF.]

The degree of decision uncertainty around these estimates is illustrated in Fig 4. This presents the cost-effectiveness acceptability curve (CEAC), which describes the probability that early intensive ABA-based interventions are cost-effective at different WTP thresholds [59]. The probability that early intensive ABA-based interventions are cost-effective when taking a health and social care services payer perspective, remains close to zero up to a threshold of £84,000 per QALY in the pessimistic scenario and up to £25,000 per QALY in the optimistic scenario. When a public sector perspective is adopted, the probability that early intensive ABA-based interventions are cost-effective begins to depart from zero at a threshold of around £30,000 per QALY under the pessimistic scenario and £1,000 per QALY in the optimistic scenario.

[Figure omitted. See PDF.]

Discussion

This study presents the first assessment of the cost-effectiveness of early intensive ABA-based intervention from UK payer perspective, using data from a recently completed IPD meta-analysis [23].

Results indicated that early intensive ABA-based intervention might generate higher incremental QALYs than TAU intervention. Under both optimistic and pessimistic scenarios, the magnitude of these benefits is relatively small, given the additional costs of implementing early intensive ABA-based therapies. Interpretation of whether these benefits represent value for money is dependent on the willingness to pay threshold considered. In our analysis we have considered a £20,000 to £30, 000 threshold, which are commonly employed by NICE in the context of UK healthcare decisions. However, it is important to acknowledge that these thresholds do not necessarily apply to autism, owing to the associated impacts upon multiple sectors and therefore may not be appropriate for decision making. These decision rules do, however, provide a useful benchmark, indicating that even when adopting a public sector perspective, early intensive ABA-based interventions would not meet the decision criteria for implementation in the UK. This is irrespective of the assumptions made about the about the long-term effectiveness of early intensive ABA-based interventions.

The results of our analyses contrast sharply with those reported in previous economic evaluations which have overwhelmingly concluded that early intensive ABA interventions are highly cost-effective generating substantial cost savings [19–21]. The difference in our results is driven in large part by differences in assumptions made regarding the persistence of any treatment effect and the time horizon. Previous economic evaluations have tended to use longer life horizons and to assume that the benefits of early intensive ABA-based treatments persist for the lifetime of the analysis. These differences highlight the importance of these assumptions and align with the scenario analysis we present which shows that the cost-effectiveness of early intensive ABA-based interventions is likely contingent upon treatment effects persisting into adulthood.

In interpreting the results of our analyses, it is also important to acknowledge that there are likely to be substantive logistical hurdles to implementing early intensive ABA-based interventions in the UK, which may incur additional costs not accounted for in the current analysis. These include the challenges with recruiting and (re)training the very large number of therapists that would be required to implement this intervention across the UK, a difficulty aggravated by both the intensity of the intervention and the fact that the intervention is delivered on either a one to one or small group basis. Diagnosis of children within an appropriate time frame also represents a significant challenge. Although in some areas access to interventions is determined by a needs-based assessment rather than a full ASD diagnostic evaluation assessment, implementing wide scale provision of an early ASD specific intervention would likely require a significant expansion of diagnostic services. Current diagnostic patterns would mean that a substantial number of autistic children would not have access to treatment due to not being diagnosed until later childhood [60]. These challenges while probably not insurmountable would need significant irrevocable investment in services. This should be considered carefully when evaluating the risks and benefits of implementing early intensive ABA-based interventions, particularly in light of the significant uncertainties in the supporting effectiveness evidence.

Limitations

The limitations of this analysis reflect the limitations of the underpinning research evidence. The narrow focus on cognitive ability and adaptive behaviour in existing effectiveness studies mean that the economic analysis cannot account for effects on other outcomes and consequently may not fully account for the benefits (or harms) of early intensive ABA-based interventions. The IPD meta-analysis also identified important weaknesses in the available research evidence [22]. Specifically, it raised concerns about the reliability of treatment effect estimates, with substantive weaknesses in the methods used; all studies were considered to be at high risk of bias. This has important implications for interpreting the results of the economic analyses if the benefits of early ABA-based interventions have not been clearly established and therefore it is not clear whether any quality of life benefits would actually be realised in practice. Further this analysis was also not able to consider relevant subgroups of children, as no substantive evidence of differential effects was established in the IPD meta-analysis.

Importantly, in common with most early intervention evaluation studies, there is little reliable longer-term follow-up data from children who have received early intensive ABA-based intervention to inform assumptions about the durability of initial effect and whether any comparative benefits of therapy relative to TAU are retained through childhood and even into adulthood. This is further compounded by limited epidemiological evidence on the prognostic value of the outcomes typically collected in the effectiveness evidence (cognitive ability and adaptive behaviour) [61–63] and a general lack of evidence about how (and if) therapy can alter the course of a child’s education, and impact on adult outcomes [35, 49] As demonstrated in the scenario analysis, considering longer term effects, resolving the uncertainties associated with extrapolating outcomes is crucial to more accurately assessing the cost-effectiveness of early interventions including intensive ABA-based interventions. Moreover, there is a need to better define what constitutes a good outcome for autistic adults and we note that recent literature has highlighted that functional outcomes such as employment status and independent living are not well correlated with the subjective well-being of individuals [64, 65]. The simplistic approach adopted in the presented scenario analysis may therefore poorly reflect the benefits of early intervention and the priorities of the autism community.

More generally, there is uncertainty regarding several input parameters as well as omissions that may impact on the validity of the analysis. For example, the HRQoL, data used were sourced from a study recruiting children who had less severe autism symptoms and included fewer children with cognitively impairments. These difference in the population may be important as the goals and ambitions relevant to children with more severe impairments may be quite different to a more able population and, as such, the way improvements in outcomes are valued may differ substantially. We were also not able to include (due to lack of appropriate data) HRQoL improvements that may accrue to carers. This may be an important omission in the context of early autism interventions. There is sizable literature demonstrating that parents of autistic children have lower HRQoL and significant potential for positive spillover effects [66–68].

Costing data on interventions is also likely to be subject to a high degree of uncertainty. Costings were based broadly on the early intensive ABA-based interventions considered in the IPD meta-analysis and what constitutes current TAU in the UK. There are, however, several factors that are likely to mean that these costings may not fully reflect the actual costs of provision. These include significant variations in the way in which early ABA-based interventions can be delivered (e.g. greater use of group settings), as well as significant variations in what constitutes current TAU in the UK.

Suggested research priorities

The substantive uncertainties associated with evaluating the cost-effectiveness of early intensive ABA-based interventions may in part be resolved by new high-quality research studies that address the concerns identified in the IPD meta-analysis regarding the internal validity of the identified effectiveness evidence. Specifically, such studies should make use of pre-specified intervention evaluation protocols including an RCT design and blinded outcome assessment.

For future intervention evaluation trials, careful consideration and decisions about the choice of relevant outcomes for the autism community together with the use of reliable outcome measures will need to be prioritised. The autism community experts, researchers and practitioners consulted as part of the advisory group for this NIHR funded study highlighted the limitations of cognitive ability and adaptive behaviour to capture both benefits and any potential adverse impacts of the intervention. Concern was also expressed about the relevance of these outcomes for the autism community and for best practice. The development of a set of core outcome measures relevant for children with autism/ASD under the age of five years and their families would be extremely beneficial in this regard and would also facilitate the sharing of findings across clinical trials [63].

As highlighted above, little is known about the timeframes over which both benefits and harms from early intervention may become apparent. The length of follow-up of any future studies of early intervention in autism needs to be considered carefully as there is substantial uncertainty regarding the durability or otherwise of early benefits. Ideally, the longest possible follow-up is desirable, but recognising both financial and pragmatic constraints, other types of research may be needed to address these uncertainties. This could include, retrospective observations studies, or planned follow-up of children recruited to existing cohort studies.

Conclusions

Based on the existing evidence, the results of this economic analysis indicate that early intensive ABA-based interventions are unlikely to represent value for money, using the current thresholds typically adopted by NICE to inform UK healthcare funding decisions. The presented economic analysis, however, highlights substantive uncertainties in several key inputs and assumptions which are likely to impact significantly on the cost-effectiveness of early intensive ABA-based interventions making it impossible to draw definite conclusions from the present study.

Future studies addressing these key uncertainties including high-quality comparative research studies of the short and longer-term impact of evidence-based early interventions may allow for more nuanced and robust estimates of the cost-effectiveness of such interventions for young autistic children.

Supporting information

S1 File. Advisory group membership.

https://doi.org/10.1371/journal.pone.0270833.s001

(DOCX)

Acknowledgments

We’d like to thank Mike Clarke, Bernard Fleming, Patricia Howlin, Emma Jenner, Helen McConachie, Anne McLaren, Dean McMillan, Damian Milton, Roo Philip, Peter Szatmari, Emma Truelove, Colin Wilson, Anne-Marie Wood, and Suzy Yardley who all provided support as part of our Advisory Group. We are particularly grateful to Patricia Howlin, Helen McConachie, and Peter Szatmari who, as part of the steering group, gave expert advice throughout the project and assisted us with the writing of this article and the final report. Special thanks to Tristram Smith who sadly died part way through this project, and who provided advice as part of our advisory group.

Citation: Hodgson R, Biswas M, Palmer S, Marshall D, Rodgers M, Stewart L, et al. (2022) Intensive behavioural interventions based on applied behaviour analysis (ABA) for young children with autism: A cost-effectiveness analysis. PLoS ONE 17(8): e0270833. https://doi.org/10.1371/journal.pone.0270833

About the Authors:

Robert Hodgson

Roles: Conceptualization, Data curation, Formal analysis, Funding acquisition, Methodology, Project administration, Resources, Supervision, Validation, Writing – original draft, Writing – review & editing

E-mail: [email protected]

Affiliation: Centre for Reviews and Dissemination, University of York, York, United Kingdom

https://orcid.org/0000-0001-6962-2893

Mousumi Biswas

Roles: Conceptualization, Data curation, Formal analysis, Funding acquisition, Methodology, Writing – review & editing

Affiliation: Bristol Myers Squibb, Lawrence Township, New Jersey, United States of America

Stephen Palmer

Roles: Conceptualization, Funding acquisition, Methodology, Supervision, Writing – review & editing

Affiliation: Centre for Health Economics, University of York, York, United Kingdom

David Marshall

Roles: Conceptualization, Funding acquisition, Writing – review & editing

Affiliation: Centre for Reviews and Dissemination, University of York, York, United Kingdom

https://orcid.org/0000-0001-5969-9539

Mark Rodgers

Roles: Conceptualization, Funding acquisition, Writing – review & editing

Affiliation: Centre for Reviews and Dissemination, University of York, York, United Kingdom

Lesley Stewart

Roles: Conceptualization, Funding acquisition, Writing – review & editing

Affiliation: Centre for Reviews and Dissemination, University of York, York, United Kingdom

Mark Simmonds

Roles: Conceptualization, Funding acquisition, Writing – review & editing

Affiliation: Centre for Reviews and Dissemination, University of York, York, United Kingdom

Dheeraj Rai

Roles: Conceptualization, Funding acquisition, Writing – review & editing

Affiliation: Centre for Academic Mental Health, School of Social and Community Medicine, University of Bristol, Bristol, United Kingdom

Ann Le Couteur

Roles: Conceptualization, Funding acquisition, Supervision, Writing – review & editing

Affiliation: Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, United Kingdom

References

1. Baird G, Simonoff E, Pickles A, Chandler S, Loucas T, Meldrum D, et al. Prevalence of disorders of the autism spectrum in a population cohort of children in South Thames: The Special Needs and Autism Project (SNAP). Lancet. 2006 Jul 15;368(9531):210–5. pmid:16844490

2. Brugha T, McManus S, Bankart J, Scott F, Purdon S, Smith J, et al. Epidemiology of autism spectrum disorders in adults in the community in England. Arch Gen Psychiatry. 2011 May;68(5):459–65. pmid:21536975

3. Lyall K, Croen L, Daniels J, Fallin M, Ladd-Acosta C, Lee B, et al. The changing epidemiology of autism spectrum disorders. Ann Rev Public Health. 2017 Mar 20;38:81–102. pmid:28068486

4. Howlin P, Goode S, Hutton J, Rutter M. Adult outcome for children with autism. J Child Psychol Psychiatry. 2004 Feb;45(2):212–29. pmid:14982237

5. Howlin P, Moss P. Adults with autism spectrum disorders. Can J Psychiatry. 2012 May;57(5):275–83. pmid:22546059

6. Croen L, Zerbo O, Qian Y, Massolo M, Rich S, Sidney S, et al. The health status of adults on the autism spectrum. Autism. 2015 Oct;19(7):814–23. pmid:25911091

7. Rai D, Heuvelman H, Dalman C, Culpin I, Lundberg M, Carpenter P, et al. Association between autism spectrum disorders with or without intellectual disability and depression in young adulthood. JAMA Netw Open. 2018 Aug 3;1(4):e181465. pmid:30646131

8. Selten J, Lundberg M, Rai D, Magnusson C. Risks for nonaffective psychotic disorder and bipolar disorder in young people with autism spectrum disorder: A population-based study. JAMA Psychiatry. 2015 May;72(5):483–9. pmid:25806797

9. Buescher A, Cidav Z, Knapp M, Mandell D. Costs of autism spectrum disorders in the United Kingdom and the United States. JAMA Pediatr. 2014 Aug;168(8):721–8. pmid:24911948

10. Lovaas O. Behavioral treatment and normal educational and intellectual functioning in young autistic children. J Consult Clin Psychol. 1987 Feb;55(1):3–9. pmid:3571656

11. Dawson G. Early behavioral intervention, brain plasticity, and the prevention of autism spectrum disorder. Dev Psychopathol. 2008;20(3):775–803. pmid:18606031

12. Schreibman L, Dawson G, Stahmer AC, Landa R, Rogers SJ, McGee GG, et al. Naturalistic developmental behavioral interventions: Empirically validated treatments for autism spectrum disorder. J Autism Dev Disord. 2015;45:2411–28. pmid:25737021

13. Koegel L, Koegel R, Harrower J, Carter C. Pivotal response intervention I: Overview of approach. Res Pract Persons Severe Disabl. 1999;24(3):174–85.

14. Rogers S, Dawson G. Early Start Denver Model for young children with autism: Promoting language, learning, and engagement: Guilford Press; 2010.

15. ABA4ALL. ABA4ALL. 2018 [cited 2018 20th November]; https://www.abaa4all.com/aba-in-the-uk.

16. Beam ABA services. Beam ABA services. 2018 [cited 2018 20th November]; https://ourprices.beamaba.com/.

17. The Owl Centre. The Owl Centre for Children’s Independent Therapy. 2018 [cited 2018 20th November]; http://www.theowltherapycentre.co.uk/faqs-help/fees.html.

18. Eikeseth S, Hayward D, Gale C. UK Young Autism Project: Information pack. 2018 [cited 2018 20th November]; http://www.ukyap.org/_userfiles/pages/file/02%20Information%20Pack%20June%202017.pdf.

19. Peters-Scheffer N, Didden R, Korzilius H, Matson J. Cost comparison of early intensive behavioral intervention and treatment as usual for children with autism spectrum disorder in the Netherlands. Res Dev Disabil. [Comparative Study Research Support, Non-U.S. Gov’t]. 2012 Nov-Dec;33(6):1763–72. pmid:22705454

20. Chasson G, Harris G, Neely W. Cost comparison of early intensive behavioral intervention and special education for children with autism. J Child Fam Stud. 2007 Jun;16(3):401–13.

21. Penner M, Rayar M, Bashir N, Roberts S, Hancock-Howard R, Coyte P. Cost-effectiveness analysis comparing pre-diagnosis autism spectrum disorder (ASD)-targeted intervention with Ontario’s Autism Intervention Program. J Autism Dev Disord. 2015 Sep;45(9):2833–47. pmid:25936527

22. Rodgers M, Marshall D, Simmonds M, Le Couteur A, Biswas M, Wright K, et al. Intensive behavioural interventions based on applied behaviour analysis (ABA) for young autistic children: A systematic review and cost-effectiveness analysis. Health Technol Assess. 2020;24(35). pmid:32686642

23. Rodgers M, Simmonds M, Marshall D, Hodgson R, Stewart LA, Rai D, et al. Intensive behavioural interventions based on applied behaviour analysis for young children with autism: An international collaborative individual participant data meta-analysis. Autism. 2021 Jan 22:1362361320985680. pmid:33482692

24. Brazier J, Ratcliffe J, Saloman JA, Tsuchiya A. Measuring and valuing health benefits for economic evaluation. Second edition. ed 2017.

25. Neumann PJ, Sanders GD, Russell LB, Siegel JEG, Theodore G. Cost effectiveness in health and medicine. Second edition. ed 2016.

26. Drummond M, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes. Fourth edition / Michael F. Drummond, Mark J. Sculpher, Karl Claxton, Greg L. Stoddart, George W. Torrance. ed 2015.

27. Phillips C. What is a QALY?: Newmarket: Hayward Group Ltd; 2009.

28. National Institute for Health and Clinical Excellence. Guide to the methods of technology appraisal 2013. London: NICE; 2013.

29. Sonnenberg F, Beck J. Markov models in medical decision making: A practical guide. Med Decis Making. 1993 Oct-Dec;13(4):322–38. pmid:8246705

30. Sparrow S, Balla D, Cicchetti D, Harrison P, Doll E. Vineland Adaptive Behavior Scales. Vineland-3. 3rd ed. London: Pearson Education; 1984.

31. Eikeseth S, Smith T, Jahr E, Eldevik S. Outcome for children with autism who began intensive behavioral treatment between ages 4 and 7: A comparison controlled study. Behav Modif. 2007 May;31(3):264–78. pmid:17438342

32. Estes A, Munson J, Rogers S, Greenson J, Winter J, Dawson G. Long-term outcomes of early intervention in 6-year-old children with autism spectrum disorder. J Am Acad Child Adolesc Psychiatry. 2015;54(7):580–7. pmid:26088663

33. Kovshoff H, Hastings R, Remington B. Two-year outcomes for children with autism after the cessation of Early Intensive Behavioral Intervention. Behav Modif. 2011 Sep;35(5):427–50. pmid:21586502

34. Magiati I, Moss J, Charman T, Howlin P. Patterns of change in children with autism spectrum disorders who received community based comprehensive interventions in their pre-school years: A seven year follow-up study. Res Autism Spect Dis. 2011 Jul-Sep;5(3):1016–27.

35. Smith D, Hayward D, Gale C, Eikeseth S, Klintwall L. Treatment gains from Early and Intensive Behavioral Intervention (EIBI) are maintained 10 years later. Behav Modif. 2019 2020/02/21:0145445519882895. pmid:31619051

36. Barrett B, Mosweu I, Jones C, Charman T, Baird G, Simonoff E, et al. Comparing service use and costs among adolescents with autism spectrum disorders, special needs and typical development. Autism. 2015 Jul;19(5):562–9. pmid:24913778

37. Payakachat N, Tilford J, Kuhlthau K, van Exel N, Kovacs E, Bellando J, et al. Predicting health utilities for children with autism spectrum disorders. Autism Res. 2014 Dec;7(6):649–63. pmid:25255789

38. Curtis L, Burns A. Unit Costs of Health and Social Care 2017. Canterbury: Personal Social Services Research Unit, University of Kent; 2017.

39. Griffith G, Fletcher R, Hastings R. A national UK census of Applied Behavior Analysis school provision for children with autism. Res Autism Spect Dis. 2012 Apr-Jun;6(2):798–805.

40. Knapp M, Romeo R, Beecham J. Economic cost of autism in the UK. Autism. [Comparative Study Research Support, Non-U.S. Gov’t]. 2009 May;13(3):317–36.

41. Lord C, Risi S, Lambrecht L, Cook E, Leventhal B, DiLavore P, et al. The Autism Diagnostic Observation Schedule—Generic: A standard measure of social and communication deficits associated with the spectrum of autism. J Autism Dev Disord. 2000;30(3):205–23. pmid:11055457

42. Billstedt E, Gillberg C, Gillberg C. Autism after adolescence: population-based 13- to 22-year follow-up study of 120 individuals with autism diagnosed in childhood. J Autism Dev Disord. 2005 Jun;35(3):351–60. pmid:16119476

43. Billstedt E, Gillberg I, Gillberg C. Autism after adolescence: population-based 13- to 22-year follow-up study of 120 individuals with autism diagnosed in childhood Journal of autism and developmental disorders (vol 35, pg 351, 2005). J Autism Dev Disord. 2007 Oct;37(9):1822-.

44. Billstedt E, Gillberg I, Gillberg C. Aspects of quality of life in adults diagnosed with autism in childhood: A population-based study. Autism. 2011 Jan;15(1):7–20. pmid:20923888

45. Cederlund M, Hagberg B, Billstedt E, Gillberg I, Gillberg C. Asperger syndrome and autism: A comparative longitudinal follow-up study more than 5 years after original diagnosis. J Autism Dev Disord. 2008 Jan;38(1):72–85. pmid:17340200

46. Bishop S, Farmer C, Thurm A. Measurement of nonverbal IQ in autism spectrum disorder: Scores in young adulthood compared to early childhood. J Autism Dev Disord. 2015 Apr;45(4):966–74. pmid:25239176

47. Anderson D, Liang J, Lord C. Predicting young adult outcome among more and less cognitively able individuals with autism spectrum disorders. J Child Psychol Psychiatry. 2014 01 May;55(5):485–94. pmid:24313878

48. Gillespie-Lynch K, Sepeta L, Wang Y, Marshall S, Gomez L, Sigman M, et al. Early childhood predictors of the social competence of adults with autism. J Autism Dev Disord. 2012 Feb;42(2):161–74. pmid:22187106

49. Perry A, Koudys J, Prichard A, Ho H. Follow-up study of youth who received EIBI as young children. Behav Modif. 2017 2020/02/21;43(2):181–201. pmid:29241355

50. Smith DP, Hayward DW, Gale CM, Eikeseth S, Klintwall L. Treatment Gains from Early and Intensive Behavioral Intervention (EIBI) are Maintained 10 Years Later. Behav Modif. 2021 Jul;45(4):581–601. pmid:31619051

51. Waters CF, Dickens MA, Thurston SW, Lu X, Smith T. Sustainability of Early Intensive Behavioral Intervention for Children With Autism Spectrum Disorder in a Community Setting. Behav Modif. 2020 Jan;44(1):3–26. pmid:30009626

52. Hirvikoski T, Mittendorfer-Rutz E, Boman M, Larsson H, Lichtenstein P, Bolte S. Premature mortality in autism spectrum disorder. Brit J Psychiat. 2016 Mar;208(3):232-+. pmid:26541693

53. Szende A, Janssen B, Cabases J. Self-reported population health: An international perspective based on EQ-5D. Rotterdam: Springer Netherlands; 2014. https://www.springer.com/gp/book/9789400775954.

54. Horner R, Johns L. Personal communication. 2018.

55. Howlin P, Magiati I. Autism spectrum disorder: Outcomes in adulthood. Curr Opin Psychiatr. 2017 Mar;30(2):69–76. pmid:28067726

56. Farley M, McMahon W, Fombonne E, Jenson W, Miller J, Gardner M, et al. Twenty-year outcome for individuals with autism and average or near-average cognitive abilities. Autism Res. [Research Support, Non-U.S. Gov’t]. 2009 Apr;2(2):109–18. pmid:19455645

57. Rowley E, Chandler S, Baird G, Simonoff E, Pickles A, Loucas T, et al. The experience of friendship, victimization and bullying in children with an autism spectrum disorder: Associations with child characteristics and school placement. Res Autism Spect Dis. 2012;6(3):1126–34.

58. Smith T, Groen A, Wynn J. Randomized trial of intensive early intervention for children with pervasive developmental disorder. Am J Ment Retard. 2000 Jul;105(4):269–85.

59. Fenwick E, Byford S. A guide to cost-effectiveness acceptability curves. Brit J Psychiat. 2005;187(2):106–8. pmid:16055820

60. Brett D, Warnell F, McConachie H, Parr JR. Factors affecting age at ASD diagnosis in UK: No evidence that diagnosis age has decreased between 2004 and 2014. J Autism Dev Disord. 2016 Jun;46(6):1974–84. pmid:27032954

61. Howlin P, Moss P, Savage S, Rutter M. Social outcomes in mid- to later adulthood among individuals diagnosed with autism and average nonverbal IQ as children. J Am Acad Child Psy. 2013 Jun;52(6):572–81. pmid:23702446

62. Lord C, Bishop S, Anderson D. Developmental trajectories as autism phenotypes. American journal of medical genetics Part C, Seminars in medical genetics. 2015;169(2):198–208. pmid:25959391

63. McConachie H, Parr J, Glod M, Hanratty J, Livingstone N, Oono I, et al. Systematic review of tools to measure outcomes for young children with autism spectrum disorder. Health Technol Assess. 2015 Jun;19(41):1-+. pmid:26065374

64. McCauley JB, Pickles A, Huerta M, Lord C. Defining Positive Outcomes in More and Less Cognitively Able Autistic Adults. Autism Res. 2020 Sep;13(9):1548–60. pmid:32851813

65. Pickles A, McCauley JB, Pepa LA, Huerta M, Lord C. The adult outcome of children referred for autism: typology and prediction from childhood. J Child Psychol Psyc. 2020 Jul;61(7):760–7. pmid:31957035

66. Hoefman R, Payakachat N, van Exel J, Kuhlthau K, Kovacs E, Pyne J, et al. Caring for a child with autism spectrum disorder and parents’ quality of life: application of the CarerQol. J Autism Dev Disord. 2014 Aug;44(8):1933–45. pmid:24577786

67. Kuhlthau K, Payakachat N, Delahaye J, Hurson J, Pyne JM, Kovacs E, et al. Quality of life for parents of children with autism spectrum disorders. Res Autism Spect Dis. 2014 Oct;8(10):1339–50.

68. Vasilopoulou E, Nisbet J. The quality of life of parents of children with autism spectrum disorder: a systematic review. Res Autism Spect Dis. 2016 Mar;23:36–49.