Using Expert Panels to Examine the Content Validity and Inter-Rater Reliability of the ABLLS-R

Abstract

The assessment literature cites several instruments used to assess the skills of children with an autism spectrum disorder (ASD) diagnosis, but many lack adequate empirical support for their psychometric properties. The Assessment of Basic Language and Learning Skills-Revised (ABLLS-R) is a popular assessment used by clinicians to measure the skills of children with ASD. Despite its widespread use, the ABLLS-R contains limited research on its psychometric properties. The current study sought to extend the recent research on the psychometric properties of the ABLLS-R by using data obtained from two separate panels of expert raters to evaluate its content validity and the inter-rater reliability of its scores. Our results demonstrate evidence of content validity as at least five out of six expert panel members rated 441 out of the 544 ABLLS-R items (or 81% of the items in the assessment) as “essential.” We also found evidence of excellent inter-rater reliability (intraclass correlation coefficient = .95, p < .001) across the ABLLS-R scores obtained from a second panel of expert raters. These findings extend the existing literature and further document the ABLLS-R as a valid instrument that yields reliable scores.