Content area
Full Text
Abstract. One potentially feasible tool for use in the formative assessment of social behavior is the direct behavior rating, yet empirical support for the reliability of its use is limited. In this study, generalizability theory was used to provide preliminary psychometric data regarding the generalizability and dependability of the direct behavior rating to measure the social behavior of preschoolers. Two typical preschool behaviors (works to resolve conflicts, interacts cooperatively with peers) were selected for investigation within the direct behavior rating created for this study. Overall, results varied depending on which behavior was rated and the number of raters whose ratings were considered. The results suggested that a fairly substantial proportion of measurement variance was attributable to the different raters, and that the four raters varied in their mean level of ratings within and across the 15 students. In addition, although the actual number of days was dependent on the number of ratings collected per day, results suggested direct behavior ratings are likely to approximate or exceed reliability-like coefficients of .70 after 7 ratings are collected across 4-7 days, and .90 after 10 ratings. Limitations, future directions, and implications are discussed.
In applied settings, both effective and efficient assessment procedures are needed to facilitate good decision-making about the academic and social behavior of students (Chafouleas, Riley-Tillman, & Sugai, in press). Although reliable and valid tools readily exist for accomplishing this task (e.g., curriculum-based assessment, systematic direct observation), existing measures are not without flaws. First, feasibility of use, particularly in a formative fashion, can be an issue in settings often faced with limited resources. second, to date, greater attention has been directed toward the study of tools for assessing academic behavior (e.g., curriculumbased assessment) than for assessing social behavior. This is unfortunate given increasing evidence suggesting a strong reciprocal connection between problem behavior and academic difficulties (e.g., Lane, O'Shaughnessy, Lambros, Gresham, & Beebe-Frankenberger, 2002; Nelson, Benner, & Gonzalez, 2003; Torgesen et al., 1999). Thus, greater attention should be directed to the study of effective and efficient methods for assessing social behavior. The purpose of this study was to provide preliminary psychometric data regarding the generalizability and dependability of a direct behavior rating (DBR) for assessing social behavior of preschoolers.
Defining a Need to Develop Formative Measures...