Content area
Full Text
J Autism Dev Disord (2014) 44:31543160 DOI 10.1007/s10803-014-2177-8
ORIGINAL PAPER
Reliability of the ADI-R for the Single Case-Part II: Clinical Versus Statistical Signicance
Domenic V. Cicchetti Catherine Lord
Kathy Koenig Ami Klin Fred R. Volkmar
Published online: 5 July 2014 Springer Science+Business Media New York 2014
Abstract In an earlier investigation, the authors assessed the reliability of the ADI-R when multiple clinicians evaluated a single case, here a female 3 year old toddler suspected of having an autism spectrum disorder (Cicchetti et al. in J Autism Dev Disord 38:764770, 2008). Applying the clinical criteria of Cicchetti and Sparrow (Am J Men Def 86:127137, 1981); and those of Cicchetti et al. (Child Neuropsychol 126137, 1995): 74 % of the ADI-R items showed 100 % agreement; 6 % showed excellent agreement; 7 % showed good agreement; 3 % manifested average agreement; and the remaining 10 % evidenced poor agreement. In this follow-up investigation, the authors described and applied a novel method for determining levels of statistical signicance of the reliability coefcients obtained in the earlier investigation. It is based upon a modication of the Z test for comparing a given level of inter-examiner reliability with a lower limit value of 70 % (Dixon and Massey in Introduction to statistical analysis. McGraw-Hill, New York, 1957). Results indicated that every item producing a clinically acceptable level of inter-examiner reliability was also statistically signicant. However, the reverse was not true, since a number of the
items with statistically signicant reliability levels did not reach levels of agreement that were clinically meaningful. This indicated that clinical signicance was an accurate marker of statistical signicance. The generalization of these ndings to other areas of diagnostic interest and importance is also examined.
Keywords Autism Clinical signicance Statistical
signicance Inter-rater reliability Multiple raters, one
patient
Introduction
In the prototypic research design for assessing the reliability of a given test instrument, two well-trained evaluators independently examine a large clinically representative sample of respondents. A much less frequently utilized design is one in which a well-trained group of multiple examiners evaluates a single case.
In a previous article in this Journal, the authors investigated the inter-examiner reliability of the ADI-R when multiple clinicians evaluated a single case (Cicchetti et al. 2008). The subject was a three...