Content area
Abstract
Individual faculty assessments of resident competency are complicated by inconsistent application of standards, lack of reliability, and the "halo" effect.
We determined whether the addition of faculty group assessments of residents in an ambulatory clinic, compared with individual faculty-of-resident assessments alone, have better reliability and reduced halo effects.
This prospective, longitudinal study was performed in the outpatient continuity clinics of a large internal medicine residency program.
Faculty-on-resident and group faculty-on-resident assessment scores were used for comparison.
Overall mean scores were significantly higher for group than individual assessments (3.92±0.51 vs. 3.83±0.38, p=0.0001). Overall inter-rater reliability increased when combining group and individual assessments compared to individual assessments alone (intraclass correlation coefficient, 95% CI=0.828, 0.785-0.866 vs. 0.749, 0.686-0.804). Inter-item correlations were less for group (0.49) than individual (0.68) assessments.
This study demonstrates improved inter-rater reliability and reduced range restriction (halo effect) of resident assessment across multiple performance domains by adding the group assessment method to traditional individual faculty-on-resident assessment. This feasible model could help graduate medical education programs achieve more reliable and discriminating resident assessments.[PUBLICATION ABSTRACT]