Test Security and Item Exposure Control for Computer-Based Examinations: Performance of a Computerized Classification Test for Professional Certification

Abstract

A client of American College Testing, Inc. (ACT) decided to implement a computer-based testing program to replace their paper-pencil format for professional certification. This paper reports on the results of the developed test after 1 year's use, especially as the results relate to test security issues. ACT research shows that a variable length computerized classification test using the sequential probability ratio test worked well in retaining the estimated classification accuracy of the paper-pencil test. The developed examination consisted of a minimum of 100 scored items with a maximum of 120 scored items, with 25 pretest items administered throughout the examination. Scale scores were obtained on the total test and two subtest content domains using maximum likelihood estimates of theta after testing had ended. As of February 2000, 1,751 candidates had taken the examination, and of this number, 1,524 (87%) had passed the examination. These initial results, with about half of the expected candidate population tested, suggest that the item exposure control parameters that were calculated during the initial simulations worked appropriately. This conclusion is based on the assumption that the previous paper-pencil test-taking behavior will continue with the computerized version in that higher ability test-takers will take the examination in the fall and the less able will sit for the examination in the winter or spring. (SLD)