The Effectiveness of RAADS-R as a Screening Tool for Adult ASD Populations

Abstract

Adult referrals to specialist autism spectrum disorder diagnostic services have increased in recent years, placing strain on existing services. It was proposed that the Ritvo Autism Asperger’s Diagnostic Scale could be used as a screening tool, in order to identify and prioritise patients most likely to receive an ASD diagnosis. This study evaluates the validity of the RAADS-R as a screening tool for ASD in an adult population. Retrospective case note analysis was used to evaluate the efficacy of the RAADS-R as a screening tool to predict ASD diagnostic outcomes in 50 service users of a NHS specialist autism service. Results indicate no association between RAADS-R scores and clinical diagnostic outcome, suggesting the RAADS-R is not an effective screening tool for identifying service users most likely to receive an ASD diagnosis. In conclusion, used as a self-report measure pre-full diagnostic assessment, the RAADS-R lacks predictive validity and is not a suitable screening tool for adults awaiting autism assessments. Future research should aim to identify reliable screening tools for this purpose.

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. Introduction

Autism spectrum disorders (ASD) are lifelong neurodevelopmental conditions characterised by a clinically significant impairment in reciprocal social interaction and communication, alongside restricted interest and repetitive behaviours [1]. Although ASD is typically diagnosed in childhood, there has been a marked increase in the number of adults referred for assessment over recent years [2], which has placed greater demand on local services and resources. Because of this demand, the time for diagnosis is rather long and one report found 29% of adults with autism and 46% of those with Asperger’s disorder did not receive a diagnosis until adulthood [3]. ASD diagnosis for adults is often expensive in both time and resources; assessments are lengthy and subjective in nature. Adding to the complexity, the diagnostic procedure should be based on multidisciplinary observations which include evaluation of current functioning and behaviours, together with a detailed history taking [4]. This process can sometimes be further complicated in adult populations due to difficulties obtaining an accurate early history, in differentiating autistic symptoms from learned behaviour or compensation strategies, other conditions, or mental health disorders which may lead to misdiagnosis [5–12].

There have been a number of key pieces of legislation which have focused on improving services for adults with ASD. “The Autism Act” [13] and subsequent “Strategy for Adults with Autism” [14] have been instrumental in setting out strategies for improved diagnostic provision and support. However, despite this, waiting times for diagnostic assessments of ASD in adults are still lengthy. Therefore, there is a clear necessity to alleviate the pressures on diagnostic services by screening the waiting lists to identify and prioritise those at a greater probability of receiving a diagnosis using standardised methods [15].

Currently, NICE guidelines recommend the use of the Autism Quotient (AQ) [16] for screening purposes [4]. The AQ is a 50-item self-report questionnaire, which was originally developed as a means to quantify ASD symptoms. It is commonly used in research studies for participant inclusion and exclusion purposes [17] and reports high sensitivity and specificity [16, 18]. Clinically, it has been noted to be useful in the screening process by identifying the extent of autistic symptoms an individual demonstrates [19], although there is good evidence to suggest it has weaknesses [20–22].

Similarly, the Ritvo Autism Asperger’s Diagnostic Scale-Revised (RAADS-R) is an 80-item self-report questionnaire, based on the ICD-10 and DSM-5 diagnostic criteria [2]. It is designed to be used in ASD diagnostic assessments for adults and has proven good specificity and sensitivity in previous studies [23]. The RAADS-R is recommended by NICE for use in the assessment of ASD in adult populations [4]. However, it is not clear how effective the RAADS-R is as a tool for screening purposes.

Although various screening tools are available for identifying ASD in adult populations, few have been evaluated for use in outpatient settings [17]. Considering that diagnostic services are bound by long and lengthy waiting lists, this study aims to evaluate the predictive validity of the RAADS-R as a self-report screening tool for patients referred for an ASD assessment to the Adult ADHD and Autism Service, South West Yorkshire Partnership NHS Foundation Trust. The implications of which could aid with informing the pathway to diagnosis for patients, thus contributing to more effective service.

2. Methods

2.1. Participants

The sample consisted of fifty adults, 70% of the sample identified as male (n = 35), 28% as female (n = 14), and 2% as transgender (n = 1), with a mean age of 32.8 years (SD± = 10.3). Patients were assessed between May 2013 and April 2016, and none had a confirmed ASD diagnosis upon referral. The adult ADHD and autism service are specialists in diagnosing ADHD and autism in adulthood. Patients are referred to the service by health care professionals, who deem it appropriate based on patient history and current difficulties. Inclusion criteria dictated that participants did not have learning disability, were over the age of 18 years (no upper cut off), had good comprehension of the English language, and had IQ within normal range, i.e., >70.

2.2. Design

Data were gathered retrospectively from records of the adult Autism Service, South West Yorkshire Partnership NHS Foundation Trust. As a part of the referral process, each participant completed a RAADS-R questionnaire. For the purposes of the adult autism service, the RAADS-R was used as a self-report measure. The study required access to sensitive information and the computerised health records of patients who had been discharged from the service. Therefore, ethical approval was sought from Leeds Beckett University. The study was also registered as a service evaluation within South West Yorkshire Partnership NHS Foundation Trust, in accordance with trust protocol. Service user’s data were taken from RiO (electronic medical records system) and anonymised prior to analysis.

2.3. Measures

The RAADS-R [2] is designed to contribute to the diagnostic assessment of ASD in adults during the screening process. It is designed for use with higher-functioning individuals of adult age. It consists of 80 questions in total, divided into four subscales measuring as follows: social relatedness (39 items), circumscribed interest (14 items), language (4 items), and sensory-motor symptoms (20 items) and is designed to be delivered by a trained clinician. Participants are asked to rate the degree to which the preceding statements, measuring both autistic and normative behaviours, are present in their life on a Likert scale ranging from “True now and when I was young” = 3; “only true now” = 2; “true only when I was young” = 1; or “never true” = 0. Threshold score of 65 has to be met to determine that ASD is likely to be present. The questionnaire has been found to have a sensitivity of 97%, specificity of 100% (at a cutoff of >65), and test-retest reliability of r = 0.987 [2].

2.4. Procedure

Following referral, the RAADS-R and AQ was sent to service users to complete at home and return by post as a part of the first step in the clinical evaluation. Upon return, scores were entered into the secure electronic patient record system, RiO, by health care professionals who were familiar with the scoring procedure. The results of the RAADS-R were used by clinicians in the diagnostic screening process (and were made available to researchers retrospectively). The researchers compared the RAADS-R score with the clinical diagnostic outcome of the service user, which was also made available through RiO. This information enabled the researchers to categorise service users into two groups: those who received an ASD diagnosis and those who did not. To reach a positive diagnosis of ASD, the participant has to be screened as likely to have ASD using the AAA [19], classified having autism spectrum or autism using ADOS-2 [24] and their full psychiatric history discussed at an multidisaplinary team meeting (MDT) consisting of experts in ASD.

Data were analysed using SPSS version 27; analysis was performed to establish any significant differences between the RAADS-R scores of those who received a clinically determined diagnosis, compared to those who did not. A receiver operating characteristic (ROC) curve was then calculated to establish the quality of the RAADS-R for predicting the diagnostic outcome amongst adults referred for an ASD assessment.

3. Results

The sample consisted of 50 service users, all of whom were referred to the service for diagnostic assessment of ASD. There were no differences found for age or sex between ASD and non-ASD outcome groups (ns). 70% of the sample was identified as male (n = 35), 28% as female (n = 14), and 2% as transgender (n = 1), with a mean age of 32.8 years (SD± = 10.3). Overall, 17 patients (34%) received a final diagnostic outcome of ASD by clinical consensus of a specialist multidisciplinary team. For those who were identified as males, the diagnostic rate was 37.1%; for females, it was 28.6%; and for transgender, it was 0%. In terms of meeting the threshold score for RAADS-R, 49 patients (98%) scored above the diagnostic threshold (>65) (median = 146, range = 214). Subgroup analysis showed that males tended to score lower on the RAADS-R (median = 138, range = 209) than females (median = 154, range = 146), but not at a significant level U = 309.5, $p=0.55$. There was no difference between RAADS-R scores for patients who received an ASD diagnosis (median = 138, range = 123) and those who did not (median = 154, range = 214).

3.1. Sensitivity, Specificity, and Predictive Values

The RAADS-R demonstrated 100% sensitivity in detecting the presence of ASD in those who received a clinical diagnosis, alongside 3.03% specificity in detecting the absence of ASD in those who did not receive a clinical diagnosis. Positive predictive value (PPV) determined that if a patient scored above the RAADS-R cutoff (>65), they have a 34.7% chance of receiving a clinical diagnosis. Negative predictive value (NPV) determined that 100% of those who did not score above the threshold did not receive a clinical diagnosis.

3.2. Receiver Operating Characteristic

The validity of the RAADS-R in predicting an ASD diagnosis in this adult sample was calculated using an ROC curve (Figure 1). The area under the curve (AUC) can be interpreted as a measure of predictive validity by indicating the predictive value of a diagnostic test. The AUC was an unacceptable level of discriminaative ability (AUC = 0.45; 95% CI, 0.285 to 0.612), in line with the null hypothesis.

[figure omitted; refer to PDF]

4. Discussion

The use of evidence-based assessment is crucial in providing effective and efficient health services; thus, it is imperative that the credibility of screening assessments is explored. The implementation of screening tools which can accurately identify those who are most likely to be diagnosed with ASD would facilitate efficient use of resources and reduced waiting times, which produce quicker diagnostic outcomes and access to interventions.

The aim of the study was to evaluate the effectiveness of the RAADS-R as a screening tool for predicting the presence of ASD in an adult population of service users referred to a specialist ASD diagnostic service. The results suggest that when used as a self-report screening tool, the RAADS-R was unable to discriminate between ASD and non-ASD cohorts. Troublingly, scores on the RAADS-R only had a 3.03% chance of detecting the absence of ASD in our sample, rendering the assessment futile. The results here are disappointing given the recommendation by NICE for the inclusion of the RAADS-R assessment in the diagnostic pathway [4].

Whilst there is some support for the use of the RAADS-R in terms of test-retest reliability [25] and facilitating referrals in psychiatric and psychology services (albeit not in isolation) [26], as well as evidence of its ability to differentiate between ASD and eating disorder cohorts [27], other research supports what we have found here. For instance, Sizoo and Horwitz [17] examined the predictive validity of several assessments commonly used as screening tools in ASD clinical practice, including the AQ and RAADS-R. They concluded poor predictive validity in both questionnaires, noting that they were unable to predict diagnostic outcomes amongst adults referred for ASD assessments, where the mean score for neurotypical controls sat 27 points higher than the cutoff score [17], which is similar to the finding here that, descriptively, the non-ASD outcome group scored greater on average than the ASD outcome group. Furthermore, Kember and Williams [28] explored the validity of the RAADS-14 [29] (a shortened version of the RAADS-R) in a population of New Zealand adults, finding high levels of sensitivity alongside low levels of specificity, concluding that the RAADS-14 is not a reliable measure of self-reported ASD as the assessment results in a very high rate of false positives, which is in line with our results. Also, in a study exploring the validity of popular ASD measures in 93 adults who received an ASD assessment at a specialist service, Conner et al. [30] found those who received a clinical diagnosis, scored higher on the ADOS than those who did not receive a diagnosis. However, no differences between outcome groups were evident on the AQ or the RAADS-R assessment. All three assessments showed poor sensitivity and specificity levels. A recent systematic review explored structured questionnaires as diagnostic measures recommended by NICE guidelines [31]. The authors found sensitivity and specificity of assessments are reduced in those referred to services for diagnostic assessment compared to populations who have a previously confirmed diagnosis, with discrimination between ASD and other mental health conditions particularly limited.

Confounds such as inappropriate referrals or comorbid conditions could have contributed to findings here and those of similar studies. It is likely that high levels of comorbidity in ASD exist within these cohorts, and it has been suggested that due to high levels of comorbidity in referrals, assessments such as the RAADS-R, which are consistently demonstrating low specificity, are not reliable indicators of which patients should receive a full diagnostic assessment as a priority [31], and we would concur. Evidence strongly suggests that ASD often exists alongside symptoms of other conditions or disorders [32–38]. Anxiety and depression, in particular, may “mimic” some ASD symptoms, thereby resulting in an increased false positive rate on the ASD assessments [39, 40]. Due to an increased awareness of ASD, these cofounding symptoms may contribute to inappropriate referrals from health care professionals; as ASD is considered in advance of other conditions such as OCD or attachment disorders. These symptoms, in turn, could be interpreted by the RAADS-R as autistic in nature, thereby leading to false positive outcomes. This is evident in previous studies, for example Sizoo et al. [17] suggested that referred adults with nonautistic psychiatric problems scored higher in the autism spectrum, but not as high as patients with ASD. Eriksson et al. [29] found good sensitivity and specificity of the RAADS in an ASD cohort compared to a general population control group. However, specificity reduced to 46% when differentiating between ASD and a clinical control group (ADHD). Also, in a study exploring the validity of the French version of the RAADS-R, Picot et al. [41] found limited support for the RAADS-R, suggesting that it is useful in helping clinicians discriminate between ASD and non-ASD groups. However the RAADS-R had a high false positive rate in a psychiatric disorder group (inclusion criteria: diagnosis of either mood disorder, schizophrenia, bipolar disorder, or anxiety disorder confirmed by a clinician consensus opinion according to ICD-10 or DSM-IV criteria). Furthermore, in a large study exploring the use of questionnaires for social and communication disorders in 738 service users of various disorders, over half of the sample met the recommended threshold for ASD as determined by the RAADS-R assessment, resulting in a high false positive rate. This leads the authors to advice that the cutoff score of >65 is too low and should not be used. Alternatively, they recommend a higher threshold score of >120 [26]. Also, there is an argument that the RAADS-R is not a sensitive enough measure to detect milder variations of ASD. Horwitz and Schoevers [42] suggest that this is because the RAADS-R assessment fails to cover a whole range of behavioural issues that may be relevant to milder forms of ASD, and therefore, it is not robust enough to capture the autism spectrum. This is an important issue considering the clinical picture of ASD is complex and is varied in presentation.

Health services are under exceeding pressure to deliver efficient care for service users. By administering screening tools, services can facilitate effective systems by identifying those who are more likely to receive a clinical diagnosis, using a standardised method [15]. Our findings and those discussed above, confirm that symptoms of ASD can only be reliably identified or differentiated from other conditions following specialist clinical assessment from an ASD practitioner. Yet, guidelines advocate using screening assessments to build a clear picture of the clinical need. It is important to make clear that these aids are not intended to be diagnostic. Indeed, they are used to aid and guide clinicians in diagnosis, and ideally, they should always be validated against the “gold standard” assessments for that population [43]. The idea of concurrent validity is relevant as the RAADS-R and RAADS-14 have shown a strong positive correlation with scores on the AQ and AQ-10 [25, 28] and a strong negative correlation with the empathy quotient (EQ) [28]. In general, this should be deemed supportive of the effectiveness of the RAADS-R. However, there are serious questions surrounding the validity of the AQ as a screening measure for ASD populations [20, 22, 39] which is of considerable concern for tools measuring against the AQ.

Due to the lengthy waiting lists, expert screening of service users prior to assessments is a logical solution. Future research should aim to identify reliable and robust ASD screening methods for use in an adult outpatient population. Not only could these be used by specialist services to reduce waiting list times and to prioritise those identified as most likely to receive an ASD diagnoses but also the screening tools could be used by GP’s to assist them with referral decisions and reduce the number of inappropriate referrals to diagnostic services. Some suggest that future screening questionnaires used in outpatient services could aim to include an element of qualitative data, whereby after each question, there could be a prompt for the service user to provide “additional information.” Ritvo et al. [2] noted that, during the assessment, the clinician would note down any additional information provided by the respondent, which contributed to the eventual diagnostic prediction. The spontaneous information provided by the service user could prove a valuable addition to the screening and may bridge the gap between a self-report and clinical-led assessment, thereby resulting in more realistic results.

It must be highlighted that one possible factor in determining results here is the way the RAADS-R was administered in this cohort. The RAADS-R was not designed to be completed without face-to-face support; indeed, the authors recommend it should be completed as a self-report measure, but within the presence of a trained clinician [2]. In the current study, however, the questionnaire was used in a self-report format, completed by the service user alone. This may have led to service users to provide misleading answers due to not fully understanding the questions or lack of clarity on how to use the rating scale. We recognise that it is likely that the clinician would be able to respond to service user questions or elaborate more on the question details, which could have been reflected in the results. However, whilst this should be considered, a recent study found only 6% of patients were unable to complete the RAADS-R and AQ questionnaire by themselves and required assistance [26]. Further research would be required to investigate this possibility. In general, it is pertinent to mention here that self-report measures in general may be particularly problematic in ASD populations as self-insight from self-report measures may not be reliable [44]. A particular strength of our study is that it was carried out in a setting that the RAADS-R assessment was intended for. As Brugha et al. [26] allude, often studies that assess sensitivity and specificity are carried out in different populations. They compare people known to have the condition against control or general populations that are highly unlikely to have the condition, which leads to an overestimated validity of the assessment. Here, we employed a clinical sample which is representative of typical referral cases. It is recognised that future studies would benefit from larger samples.

5. Conclusions

Despite the literature supporting the use of the RAADS-R for predicting ASD diagnoses in adult populations, the results of this study support the null hypothesis, i.e., the RAADS-R was not able to differentiate between service users who receive an ASD diagnosis after full assessment and those who do not. When used as a self-report tool, the RAADS-R had no clinical value. It is of importance to keep in mind that whilst assessments such as the RAADS-R are not designed to make a highly accurate prediction of diagnostic outcome, they are intended for use in this way in clinical practice. Further research is required to identify appropriate screening tools which could be used in an outpatient adult population to sufficiently differentiate between ASD, mild ASD, and non-ASD cohorts.

Ethical Approval

This research study was conducted retrospectively from data obtained for clinical purposes, in line with local guidelines.

References

[1] American Psychiatric Association, Diagnostic and Statistical Manual of Mental Disorder, 2013.

[2] R. A. Ritvo, E. R. Ritvo, D. Guthrie, M. J. Ritvo, D. H. Hufnagel, W. McMahon, B. Tonge, D. Mataix-Cols, A. Jassi, T. Attwood, J. Eloff, "The ritvo autism asperger diagnostic scale-revised (RAADS-R): a scale to assist the diagnosis of autism spectrum disorder in adults: an international validation study," Journal of Autism and Developmental Disorders, vol. 41 no. 8, pp. 1076-1089, DOI: 10.1007/s10803-010-1133-5, 2011.

[3] J. Barnard, Ignored or Ineligible? the Reality for Adults with Autism Spectrum Disorders, 2001.

[4] National Institute for Health and Care Excellence, Autism Spectrum Disorder in Adults: Diagnosis and Management (Guideline CG142), 2016.

[5] J. A. Bastiaansen, H. Meffert, S. Hein, P. Huizinga, C. Ketelaars, M. Pijnenborg, A. Bartels, R. Minderaa, C. Keysers, A. de Bildt, "Diagnosing autism spectrum disorders in adults: the use of Autism Diagnostic Observation Schedule (ADOS) module 4," Journal of Autism and Developmental Disorders, vol. 41 no. 9, pp. 1256-1266, DOI: 10.1007/s10803-010-1157-x, 2011.

[6] S. Barlati, G. Deste, M. Gregorelli, A. Vita, "Autistic traits in a sample of adult patients with schizophrenia: prevalence and correlates," Psychological Medicine, vol. 49 no. 1, pp. 140-148, DOI: 10.1017/s0033291718000600, 2019.

[7] S. Berthoz, E. L. Hill, "The validity of using self-reports to assess emotion regulation abilities in adults with autism spectrum disorder," European Psychiatry, vol. 20 no. 3, pp. 291-298, DOI: 10.1016/j.eurpsy.2004.06.013, 2005.

[8] F. De Crescenzo, V. Postorino, M. Siracusano, A. Riccioni, M. Armando, P. Curatolo, L. Mazzone, "Autistic symptoms in schizophrenia spectrum disorders: a systematic review and meta-analysis," Frontiers in Psychiatry, vol. 10,DOI: 10.3389/fpsyt.2019.00078, 2019.

[9] U. Frith, Autism: Explaining the Enigma, 2003.

[10] O. T. Leyfer, S. E. Folstein, S. Bacalman, N. O. Davis, E. Dinh, J. Morgan, H. Tager-Flusberg, J. E. Lainhart, "Comorbid psychiatric disorders in children with autism: interview development and rates of disorders," Journal of Autism and Developmental Disorders, vol. 36 no. 7, pp. 849-861, DOI: 10.1007/s10803-006-0123-0, 2006.

[11] C. Gillberg, A Guide to Asperger Syndrome, 2002.

[12] L. Fusar-Poli, "Missed diagnoses and misdiagnoses of adults with autism spectrum disorder," European Archives of Psychiatry and Clinical Neuroscience, vol. 89,DOI: 10.1007/s00406-020-01189-w, 2020.

[13] U. Parliament, The Autism Act, 2009.

[14] D. o. Health, Think Autism: Fulfilling and Rewarding Lives, the Strategy for Adults with Autism in England: An Update, 2010.

[15] F. P. Glascoe, "Screening for developmental and behavioral problems," Mental Retardation and Developmental Disabilities Research Reviews, vol. 11 no. 3, pp. 173-179, DOI: 10.1002/mrdd.20068, 2005.

[16] S. Baron-Cohen, "The autism-spectrum quotient (AQ): evidence from Asperger syndrome/high-functioning autism, males and females, scientists and mathematicians," Journal of Autism and Developmental Disorders, vol. 31 no. 1, 2001.

[17] B. B. Sizoo, E. Horwitz, J. Teunisse, C. Kan, C. Vissers, E. Forceville, A. Van Voorst, H. Geurts, "Predictive validity of self-report questionnaires in the assessment of autism spectrum disorders in adults," Autism, vol. 19 no. 7, pp. 842-849, DOI: 10.1177/1362361315589869, 2015.

[18] A. Wakabayashi, S. Baron-Cohen, S. Wheelwright, Y. Tojo, "The Autism-Spectrum Quotient (AQ) in Japan: a cross-cultural comparison," Journal of Autism and Developmental Disorders, vol. 36 no. 2, pp. 263-270, DOI: 10.1007/s10803-005-0061-2, 2006.

[19] S. Baron-Cohen, S. Wheelwright, J. Robinson, M. Woodbury-Smith, "The adult asperger assessment (AAA): a diagnostic method," Journal of Autism and Developmental Disorders, vol. 35 no. 6, pp. 807-819, DOI: 10.1007/s10803-005-0026-5, 2005.

[20] H. Kenny, A. J. Stansfield, "How useful are the Adult Asperger Assessment and AQ-10 within an adult clinical population of all intellectual abilities?," Advances in Autism, vol. 2 no. 3, pp. 118-130, DOI: 10.1108/aia-03-2016-0009, 2016.

[21] L. Fusar-Poli, A. Ciancio, A. Gabbiadini, V. Meo, F. Patania, A. Rodolico, G. Saitta, L. Vozza, A. Petralia, M. S. Signorelli, E. Aguglia, "Self-reported autistic traits using the AQ: a comparison between individuals with ASD, psychosis, and non-clinical controls," Brain Sciences, vol. 10 no. 5,DOI: 10.3390/brainsci10050291, 2020.

[22] M. Adamou, S. L. Jones, S. Wetherhill, "AAA screening in adults with ASD: a retrospective cohort study," Advances in Autism, vol. 76,DOI: 10.1108/aia-10-2020-0059, 2021.

[23] R. A. Ritvo, E. R. Ritvo, D. Guthrie, A. Yuwiler, M. J. Ritvo, L. Weisbender, "A scale to assist the diagnosis of autism and asperger’s disorder in adults (RAADS): a pilot study," Journal of Autism and Developmental Disorders, vol. 38 no. 2, pp. 213-223, DOI: 10.1007/s10803-007-0380-6, 2008.

[24] C. Lord, S. Risi, L. Lambrecht, E. H. Cook, B. L. Leventhal, P. C. DiLavore, A. Pickles, M. Rutter, "The autism diagnostic observation schedule-generic: a standard measure of social and communication deficits associated with the spectrum of autism," Journal of Autism and Developmental Disorders, vol. 30 no. 3, pp. 205-223, 2000.

[25] L. M. J. Andersen, K. Näswall, I. Manouilenko, L. Nylander, J. Edgar, R. A. Ritvo, E. Ritvo, S. Bejerot, "The Swedish version of the ritvo autism and asperger diagnostic scale: revised (RAADS-R). A validation study of a rating scale for adults," Journal of Autism and Developmental Disorders, vol. 41 no. 12, pp. 1635-1645, DOI: 10.1007/s10803-011-1191-3, 2011.

[26] T. Brugha, F. Tyrer, A. Leaver, S. Lewis, S. Seaton, Z. Morgan, S. Tromans, K. van Rensburg, "Testing adults by questionnaire for social and communication disorders, including autism spectrum disorders, in an adult mental health service population," International Journal of Methods in Psychiatric Research, vol. 29 no. 1,DOI: 10.1002/mpr.1814, 2020.

[27] D. Vagni, D. Moscone, S. Travaglione, A. Cotugno, "Using the Ritvo Autism Asperger Diagnostic Scale-Revised (RAADS-R) disentangle the heterogeneity of autistic traits in an Italian eating disorder population," Research in Autism Spectrum Disorders, vol. 32, pp. 143-155, DOI: 10.1016/j.rasd.2016.10.002, 2016.

[28] S. M. Kember, M. N. Williams, "Autism in Aotearoa: is the RAADS-14 a valid tool for a New Zealand population?," European Journal of Psychological Assessment, vol. 73, 2020.

[29] J. M. Eriksson, L. M. Andersen, S. Bejerot, "RAADS-14 Screen: validity of a screening tool for autism spectrum disorder in an adult psychiatric population," Molecular Autism, vol. 4 no. 1,DOI: 10.1186/2040-2392-4-49, 2013.

[30] C. M. Conner, R. D. Cramer, J. J. McGonigle, "Examining the diagnostic validity of autism measures among adults in an outpatient clinic sample," Autism in Adulthood, vol. 1 no. 1, pp. 60-68, DOI: 10.1089/aut.2018.0023, 2019.

[31] S. Wigham, J. Rodgers, T. Berney, A. Le Couteur, B. Ingham, J. R. Parr, "Psychometric properties of questionnaires and diagnostic measures for autism spectrum disorders in adults: a systematic review," Autism, vol. 23 no. 2, pp. 287-305, DOI: 10.1177/1362361317748245, 2018.

[32] E. Fombonne, "Epidemiology of autistic disorder and other pervasive developmental disorders," Journal of Clinical Psychiatry, vol. 66 no. 10, 2005.

[33] M.-C. Lai, C. Kassee, R. Besney, S. Bonato, L. Hull, W. Mandy, P. Szatmari, S. H. Ameis, "Prevalence of co-occurring mental health diagnoses in the autism population: a systematic review and meta-analysis," The Lancet Psychiatry, vol. 6 no. 10, pp. 819-829, DOI: 10.1016/s2215-0366(19)30289-5, 2019.

[34] T. Lugnegård, M. U. Hallerbäck, C. Gillberg, "Asperger syndrome and schizophrenia: overlap of self-reported autistic traits using the Autism-spectrum Quotient (AQ)," Nordic Journal of Psychiatry, vol. 69 no. 4, pp. 268-274, DOI: 10.3109/08039488.2014.972452, 2015.

[35] A. Mannion, G. Leader, "Comorbidity in autism spectrum disorder: a literature review," Research in Autism Spectrum Disorders, vol. 7 no. 12, pp. 1595-1616, DOI: 10.1016/j.rasd.2013.09.006, 2013.

[36] M. Romero, J. M. Aguilar, Á. Del-Rey-Mejías, F. Mayoral, M. Rapado, M. Peciña, M. Á. Barbancho, M. Ruiz-Veguilla, J. P. Lara, "Psychiatric comorbidities in autism spectrum disorder: a comparative study between DSM-IV-TR and DSM-5 diagnosis," International Journal of Clinical and Health Psychology, vol. 16 no. 3, pp. 266-275, DOI: 10.1016/j.ijchp.2016.03.001, 2016.

[37] L. Tebartz van Elst, "High-functioning autism spectrum disorder as a basic disorder in adult psychiatry and psychotherapy: psychopathological presentation, clinical relevance and therapeutic concepts," European Archives of Psychiatry and Clinical Neuroscience, vol. 263 no. 2, pp. S189-S196, DOI: 10.1007/s00406-013-0459-3, 2013.

[38] H. Westwood, I. Eisler, W. Mandy, J. Leppanen, J. Treasure, K. Tchanturia, "Using the autism-spectrum quotient to measure autistic traits in anorexia nervosa: a systematic review and meta-analysis," Journal of Autism and Developmental Disorders, vol. 46 no. 3, pp. 964-977, DOI: 10.1007/s10803-015-2641-0, 2016.

[39] K. L. Ashwood, N. Gillan, J. Horder, H. Hayward, E. Woodhouse, F. S. McEwen, J. Findon, H. Eklund, D. Spain, C. E. Wilson, T. Cadman, S. Young, V. Stoencheva, C. M. Murphy, D. Robertson, T. Charman, P. Bolton, K. Glaser, P. Asherson, E. Simonoff, D. G. Murphy, "Predicting the diagnosis of autism in adults using the Autism-Spectrum Quotient (AQ) questionnaire," Psychological Medicine, vol. 46 no. 12, pp. 2595-2604, DOI: 10.1017/s0033291716001082, 2016.

[40] F. G. Happé, H. Mansour, P. Barrett, T. Brown, P. Abbott, R. A. Charlton, "Demographic and cognitive profile of individuals seeking a diagnosis of autism spectrum disorder in adulthood," Journal of Autism and Developmental Disorders, vol. 46 no. 11, pp. 3469-3480, DOI: 10.1007/s10803-016-2886-2, 2016.

[41] M.-C. Picot, C. Michelon, H. Bertet, E. Pernon, D. Fiard, R. Coutelle, M. Abbar, J. Attal, A. Amestoy, P. Duverger, A. R. Ritvo, E. R. Ritvo, A. Baghdadli, "The French version of the revised ritvo autism and asperger diagnostic scale: a psychometric validation and diagnostic accuracy study," Journal of Autism and Developmental Disorders, vol. 51 no. 1, pp. 30-44, DOI: 10.1007/s10803-020-04518-z, 2021.

[42] E. H. Horwitz, R. A. Schoevers, C. E. J. Ketelaars, C. C. Kan, A. M. D. N. van Lammeren, Y. Meesters, A. A. Spek, S. Wouters, J. P. Teunisse, L. Cuppen, A. A. J. Bartels, E. Schuringa, H. Moorlag, D. Raven, D. Wiersma, R. B. Minderaa, C. A. Hartman, "Clinical assessment of ASD in adults using self- and other-report: psychometric properties and validity of the Adult Social Behavior Questionnaire (ASBQ)," Research in Autism Spectrum Disorders, vol. 24, pp. 17-28, DOI: 10.1016/j.rasd.2016.01.003, 2016.

[43] L. D. Maxim, R. Niebo, M. J. Utell, "Screening tests: a review with examples," Inhalation Toxicology, vol. 26 no. 13, pp. 811-828, DOI: 10.3109/08958378.2014.955932, 2014.

[44] S. L. Bishop, M. M. Seltzer, "Self-reported autism symptoms in adults with autism spectrum disorders," Journal of Autism and Developmental Disorders, vol. 42 no. 11, pp. 2354-2363, DOI: 10.1007/s10803-012-1483-2, 2012.