The Validity and Reliability of Autism Behavior Checklist in Iran

Headnote

Objectives: The aim of this study was to evaluate the psychometric features of the Persian version of the Autism Behavior Checklist (ABC).

Method: The International Quality of Life Assessment (IQOLA) approach was used to translate the English ABC into Persian. A total sample of 184 parents of children including 114 children with autism disorder (mean age =7.21, SD =1.65) and 70 typically developing children (mean age = 6.82, SD =1.75) completed the ABC. Internal consistency, test-retest reliability, concurrent and discriminant validity, and cut-offscore were assessed.

Results: The results of this study revealed that the Persian version of the ABC has an acceptable degree of internal consistency (.73). Test-retest comparisons using interclass correlation confirmed the instrument's time stability (.83). The instrument's concurrent validity with Gilliam Autism Rating Scale (GARS) was verified; the correlation between total scores was .94. In the discriminant validity, the autism group had significantly higher scores compared to the normal group. Receiver Operating Characteristic (ROC) analysis revealed that individuals with total scores below 25 are less likely to be in the autism group.

Conclusion: The Persian version of the ABC can be used as an initial screening tool in clinical contexts.

Keywords: Autism, Autism Behavior Checklist, Persian, Reliability, Validity

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder characterized by deficits in social communication and social interaction, as well as restricted, repetitive patterns of behavior, interests or activities(1). In DSM-IV, autism spectrum disorders include autism disorder, Pervasive Developmental Disorder Not otherwise Specified (PDD-NOS), Asperger syndrome, Childhood Disintegrative Disorder (CDD) and Rett's disorder (2), but in DSM-V, all of these disorders have been categorized under the term "autism spectrum disorder" (1) .

Several studies reported that the prevalence of ASD increased dramatically in recent years (3-8). These prevalence rates of ASD have a range from 250 per 10000 (9) to one in 88 children (4). Developing the new assessment instruments may be one factor that contributes to the raising of ASD prevalence. In addition, increase in the awareness about autism spectrum disorder may also contribute to this phenomenon (8, 10).

Early diagnosis of autism is crucial because a series of studies have demonstrated that early identification and early intervention leads to more positive outcomes in communication, social interaction and cognitive development (11-13).

There are some ASD screening tools such as Autism Behavior Checklist (ABC) (14), the Baby and Infant Screen for Children with Autism Traits (BISCUIT) (15), the Checklist for Autism in Toddlers (CHAT) (16), the Gilliam Autism Rating Scale (GARS) (17), the Childhood Autism Rating Scales (CARS) (18), the Autism Diagnostic Observation Schedule (ADOS) (19) and the Autism Diagnostic Interview-Revised (ADI-R) (20). However, there is only one validated instrument available in Persian for screening and assessing children with autism. Ahmadi et al. (201 1) investigated the psychometric features of the Gilliam Autism Rating Scale (GARS) for identifying children with autism in Iran. They used the GARS to interview 100 mothers of children with autism (mean age = 8.28 years) and 100 mothers of typical children. They reported that the GARS is a reliable instrument for screening and identifying children with autism (21).

The ASIEP-2 is a screening instrument used to evaluate children with autism and to create educational plans for these children. It consists of five subscales: the Autism Behavior Checklist (ABC), the Sample of Vocal Behavior, Interaction Assessment, Educational Assessment, and Learning Rate Prognosis (22).

Autism behavior checklist (ABC) is a well-established instrument used to screen and diagnose autism (23), and it has been used successfully in differential diagnosis of autism (24). Krug et al. (1980) selected the behavior items from nine sources: the criteria outlined by Kanner, Lovaas et al., the British Working Party's Checklist or Creak's Nine Points, Rimland's Form E-2, the BRIAAC, Rendle- Short and Clancy's Checklist, and Lotter's Checklist (22). Also, it has been used by health professionals in various countries (25, 26). The ABC has been considered useful in the screening of these children (27, 28). ABC is a popular instrument for identifying children with autism spectrum disorder because of its simple use, scoring, and low cost (29).

The psychometric properties of the ABC have been investigated by Krug et al. (1980). They noted that in a new sample of 62 individuals with autism, 86% received total ABC scores within 1 SD from the standardization sample mean and the remaining 14% had scores within 1.5 SDs (14). The split-half reliability was also reported to be 0.87(14) .

In addition, Volkmar et al. (1988) reported the split-half reliability of the total scale to be 0.74 and split-half reliabilities of the subscales as ranging from 0.30 to 0.70 (28).

Sturmey et al. (1992) reported that in their study, coefficient alpha for the ABC total score was .87(30). The validity of the scale was also examined in a study carried out by Miranda-Linné and Melin (1997). They compared the total score of the scale between speaking and nonspeaking autistic individuals. The mean scores obtained in the two groups were both lower than the 68-point cutoffproposed by Krug et al. (1980). In their study, they confirmed the proposals of other authors to decrease the 68-point cutoffscore, considering it too high to correctly identify children with autism, and suggested a cutoffscore of 54 (31) .

In total, there is a crucial need to have valid and reliable screening instruments for children with autism spectrum disorders in Iran. Therefore, the purpose of this study was to translate ABC into Persian and establish the reliability and validity of this instrument in Persian. In addition, we used the Gilliam Autism Rating Scale (GARS) for examining ABC concurrent validity.

Material and Methods

Participants

Using convenience sampling method, a total number of 189 parents of children aged 4 to 10 years old including 114 children with autism spectrum disorder (mean age = 7.21, SD = 1.65) and 70 typically developing children (mean age = 6.82, SD=1.75) participated in this study. T-test revealed no significant differences in the chronological age of the groups. Children with autism disorder were diagnosed by child psychiatrists according to DSM-IV Text revision; these children were recruited from 5 autism-specific schools in Tehran and from speech therapy clinics at Tehran University of Medical Sciences. Normally developing children were recruited from Tehran's kindergartens and schools. Children with physical disorders, blindness, and deafness or with language disorders were excluded.

The study was approved by the Medical Ethics Committee of Tehran University of Medical Sciences.

Instruments

Autism Behavior Checklist

The ABC contains 57 items in five areas: Sensory, Relating, Body and Object use, Language, and Social and Self-help skills. Each item is scored from 1 to 4 and the total score is obtained by adding the weight of the different areas. Krug et al. (1980) assigned the cutoffpoint of 68 as a score to correctly classify children who were suspected of having autism; a score of 68 and above was associated with a high probability of clinical diagnosis. In the standardization sample, 90% of the sample who received ABC scores higher than 68 also had a previous diagnosis of autism. In contrast, 95% of the sample who received ABC scores lower than 53 were not diagnosed as autistic by clinicians (14).

Gilliam Autism Rating Scale-2

Gilliam Autism Rating Scale-2 is a behavioral checklist designed to identify autism in 3-22 year old individuals. GARS-2 contains 42 items involving three subscales: Stereotyped Behavior, Communication, and Social Interaction. Items are rated on a four point Likert scale ranging from never observed (0) to frequently observed (3). GARS is designed to be answered by parents or teachers. This scale does not need special training (32). Based on Cronbach Alpha, the reliability of the test indicates alpha coefficient of 0.89 in Iran (21).

Procedure

At first, permission to translate and evaluate the psychometric features of the ABC was obtained from Pro-Ed, the publisher of the instrument. The original version of the profile was translated into Persian according to International Quality of Life Assessment (IQOLA) approach. First, the checklist was translated into Persian language by two independent Persian professionals familiar with special education. The forward translations were compared and discussed in a group meeting of the two translators and two of the authors. Differences were discussed until consensus was reached about the final Persian version. Then, in order to examine the equivalence of this translated version with the original version, back-translation to English was done by a Persian-English bilingual professional. Third, a committee of 10 professionals including 6 speech and language pathologist and 4 child psychiatrists were asked to confirm the validity of the translation and made revisions to the Persian version. The Persian version of ABC was then administrated to 10 mothers (5 mothers of children with autism and 5 mothers of typically developing children) to provide a qualitative testing of readability and comprehension. This qualitative testing revealed no problem with the Persian Version. Like the ABC, the Persian adaptation consists of 57 items, which are divided into 5 subscales: Sensory, Relating, Body and Object use, Language, and Social and Self-help skills.

The purpose and procedure of the study were explained to all mothers, and written informed consent was obtained .

The ABC was administered to all of mothers in the form an interview. The mothers answered yes or no regarding the presence of a given behavior. To examining concurrent validity between ABC and the GARS, the GARS was administered to only 45 mothers of verbal children with autism (mean age = 7.06, SD = 1.25), because the GARS has a communication subtest. Test-retest reliability was collected from 50 mothers of children with autism spectrum disorder (mean age = 7.25, SD = 1.62) with an interval of two weeks.

Data Analysis

All statistical analyses were conducted using SPSS version 18.0. To calculate internal consistency, Cronbach's alpha coefficient was used and test-retest was analyzed by interclass correlation. The concurrent validity of the ABC was evaluated by calculating the correlation coefficient between the total scores obtained from the ABC and the GARS. Discriminant validity of the ABC was assessed by performing independent t-test between the autism group and normally developing children. Receiver operating characteristic (ROC) curve analysis was used to determine optimal cutoffvalues for differentiation between autism and normally developing children based on the total score of ABC.

Results

The internal consistency reliability of the items on the ABC was investigated using the Cronbach's alpha coefficient. Cronbach's alpha for the total score of 57 items was .73 and the item-total correlation ranged from 0.35 to 0.75 (Table 1). Internal consistency of the ABC subscales is displayed in Table 1.

Subscale Correlations

The correlation between each subscale with each of other four subscales is demonstrated in Table2 .

Test-Retest Reliability:

To determine the test-retest reliability of the ABC, we used Intraclass Correlation Coefficient (ICC) with two weeks interval. The stability was.83 (n = 20). The correlation between subscales is displayed in Table3.

Concurrent Validity

The ABC and the GARS total scores were correlated at .94, and subscale correlations between instruments ranged from .37 to .92.

Discriminant Validity

To examine the instrument's discriminant validity, independent t-test was performed between the two groups. Total scores and subscale scores were compared between children with autism spectrum disorder and normally developing children. There was a significant difference in total scores and subscale scores between the two groups; autism group had a significantly higher scores than the normal group (p<.001). These results are displayed in Table 4.

ROC Curve

Receiver operating characteristics (ROC) were examined to find optimal cut-offscores to indicate a diagnosis of autism. The optimal cut-offscore for screening autism was 25, associated with .97 sensitivity and .98 specificity.

Discussion

The purpose of this study was to evaluate the psychometric features of the Persian version of Autism Behavior Checklist (14). Reliability and validity were examined, and the cut-offpoint for the autism spectrum disorders was calculated. In the current study, Cronbach's alpha was 0.73 for the total scale, similar to the internal consistency estimate of .74 reported by Volkmar et al. (28), and lower than that suggested by other authors which was .87 (14). Cronbach's alpha for subscales ranged from 0.35 to 0.67 which is similar to that reported by Volkmar et al. which ranged from .30 to .70 (28).

Test-retest reliability of the ABC was .83 in this study which is acceptable and suggests that the results of the ABC are stable over time and enables the professionals to confidently interpret the results from the ABC. The correlation between the total scores of the ABC and the GARS was .94, which confirmed the concurrent validity of the ABC with the GARS. This study showed the validity of the ABC in discriminating children with autism from children with normal development. These results are in accordance with the original ABC development sample and other studies (14, 25, 27, 33).

The total score for the ABC is the sum of the five subscale scores. Higher scores indicate more autistic behavior symptoms (14). Using 68 as a cutoffscore which was recommended by Krug et al. (14), only 46% of children with autism were classified as autistic. ROC analysis determined that individuals with the total scores below 25 are less likely to have autism. These results are in accordance with some researches which also have questioned the accuracy of the recommended cutoffscores and suggested to lower the cutoffscore; these suggestions range from 39 to 54 (28, 31, 33-35). The cut-offscore proposed by this study is lower than other studies. Krug et al. were concerned about the applicability of the ABC to high-functioning children with autism (14), and according to Yirmia, this instrument is not appropriate for use on school-age autistic children (35). Therefore, recruiting children from autism specific schools may make the obtained total scores lower. Second, all children used rehabilitation services and this led to the reduction of symptoms..

We suggest future studies to evaluate psychometric features of other screening and assessment tools for autism spectrum disorders in Iran.

Limitation

The results of this study should be interpreted with the following main limitation. Because we used a convenience sampling method, representation of the general population was limited (36).

Conclusion

According to these findings and similar to other studies, the ABC is not appropriate for school age children, but it can be used as an initial screening tool at clinics. For more accurate assessment, there is a need to use other valid and reliable instruments.

Acknowledgment

We are grateful to all our participants and their parents and to the staffof autism-specific schools in Tehran, speech therapy clinics at Tehran University of Medical Sciences and Tehran's kindergartens for their administration support. We appreciate Tehran University of Medical Science for providing the financial support.

Conflict of interest

The authors report no declaration of interest.

References

References

1. American Psychiatric Association. Diagnostic and statistical manual of mental disorders, 5th edition. Washington, DC: American Psychiatric Association; 2013.

2. American Psychiatric Association. Diagnostic and statistical manual of mental disorders, 4th edition. Washington, DC: American Psychiatric Association;1994.

3. Matson JL, LoVullo SV, Trends and topics in autism spectrum disorders research. Research in Autism Spectrum Disorders 2009; 3: 252-257.

4. Chakrabarti S, Fombonne E. Pervasive developmental disorders in preschool children: confirmation of high prevalence. American Journal of Psychiatry 2005: 162: 1133-1141.

5. Leung C, et al., Development of a preschool developmental assessment scale for assessment of developmental disabilities. Research in developmental disabilities, 2010. 31.

6. Nicholas JS, et al. Prevalence and Characteristics of Children With Autism-Spectrum Disorders. Annals of Epidemiology, 2008. 18: 130-136.

7. Smith KRM, Matson JL. Behavior problems: Differences among intellectually disabled adults with co-morbid autism spectrum disorders and epilepsy. Research in Developmental Disabilities 2010. 31: 1062-1069.

8. Matson JL, Kozlowski AM. The increasing prevalence of autism spectrum disorders. Research in Autism Spectrum Disorders, 2011; 5: 418-425.

9. Ghanizadeh A. A preliminary study on screening prevalence of pervasive developmental disorder in schoolchildren in Iran. Journal of autism and developmental disorders 2008; 38: 759-763.

10. Johnson, C.P. and S.M. Myers, Identification and evaluation of children with autism spectrum disorders. Pediatrics 2007; 120 11. : 1183-1215.

12. Matson JL, Smith KR. Current status of intensive behavioral interventions for young children with autism and PDD-NOS. Research in Autism Spectrum Disorders, 2008. 2: 60-74.

13. Ben Itzchak, E., et al., Cognitive, behavior and intervention outcome in young children with autism. Research in Developmental Disabilities 2008; 29: 447-458.

14. Rogers SJ, Vismara LA. Evidence-based comprehensive treatments for early autism. Journal of Clinical Child & Adolescent Psychology 2008; 37: 8-38.

15. Krug DA, Arick J, Almond P. Behavior checklist for identifying severely handicapped individuals with high levels of autistic behavior. Journal of Child Psychology and Psychiatry 1980; 21: 221-229.

16. Matson JL, et al. Reliability and item content of the Baby and Infant Screen for Children with aUtIsm Traits (BISCUIT): Parts 1-3. Research in Autism Spectrum Disorders 2009; 30: 336-344.

17. Baron-Cohen S, Allen J, Gillberg C. Can autism be detected at 18 months? The needle, the haystack, and the CHAT. The British Journal of Psychiatry 1992 ; 61: 839-843.

18. Gilliam JE, Gilliam Autism Rating Scale: Examiner's Manual. 1995.

19. Schopler E, et al. Toward objective classification of childhood autism: Childhood Autism Rating Scale (CARS). Journal of autism and developmental disorders 1980; 10: 91-103.

20. Lord C, et al. The Autism Diagnostic Observation Schedule-Generic: A standard measure of social and communication deficits associated with the spectrum of autism. Journal of autism and developmental disorders 2000; 30: 205-223.

21. Lord C, Rutter M, Le Couteur A. Autism Diagnostic Interview-Revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of autism and developmental disorders 1994; 24: 659-685.

22. Ahmadi SJ, Hemmatian ST, Khalili Z. [Investigation of the psychometric features of the GARS (persion)]. Research in Cognitive and Behavioral Journal 2011; 1: 87-104.

23. Krug DA, Arick JR, Almond PJ. Autism screening instrument for education planning. 1980: ASIEP Education Company.

24. Rellini E, et al. Childhood Autism Rating Scale (CARS) and Autism Behavior Checklist (ABC) correspondence and conflicts with DSM-IV criteria in diagnosis of autism. Journal of autism and developmental disorders 2004; 34: 703-708.

25. Teal MB, Wiebe MJ, A validity analysis of selected instruments used to assess autism. Journal of Autism and Developmental Disorders 1986; 16: 485-494.

26. Marteleto MRF, Pedromônico MRM. Validity of autism behavior checklist (ABC): preliminary study. Revista Brasileira de Psiquiatria 2005; 27: 295-301.

27. Karabekiroglu K, Aman MG. Validity of the aberrant behavior checklist in a clinical sample of toddlers. Child psychiatry and human development 2009; 40: 99-110.

28. Eaves RC, Milner B. The criterion-related validity of the childhood autism rating scale and the autism behavior checklist. Journal of Abnormal Child Psychology 1993; 21: 481-491.

29. Volkmar FR, et al. An evaluation of the autism behavior checklist. Journal of Autism and Developmental Disorders 1988; 18: 81-97.

30. Oro AB, Navarro-Calvillo ME, Esmer C. Autistic Behavior Checklist (ABC) and Its Applications, in Comprehensive Guide to Autism 2014, Springer: 2787-2798.

31. Sturmey P, Matson JL, Sevin JA. Brief report: Analysis of the internal consistency of three autism scales. Journal of Autism and Developmental Disorders 1992; 22: 321-328.

32. Miranda-Linne FM, Melin L. A comparison of speaking and mute individuals with autism and autistic-like conditions on the Autism Behavior Checklist. Journal of autism and developmental disorders 1997; 27: 245-264.

33. Gilliam JE, GARS-2: Gilliam autism rating scale. 2006: Pro-ed.

34. Sevin JA, et al. A comparison and evaluation of three commonly used autism scales. Journal of Autism and Developmental Disorders 1991; 21: 417-432.

35. Wadden NP, Bryson SE, Rodger RS. A closer look at the Autism Behavior Checklist: Discriminant validity and factor structure. Journal of Autism and Developmental Disorders 1991; 21: 529-541.

36. Yirmiya NM, Sigman, Freeman BJ, Comparison between diagnostic instruments for identifying high-functioning children with autism. Journal of Autism and Developmental Disorders 1994; 24: 281-291.

37. Birnbaum MH,.Human research and data collection via the Internet. Annu. Rev. Psychol 2004; 55: 803-832.