Content area
Full text
Accepted: April l6, 2018
Introduction
The effective use of Objective Structured Clinical Examina- tions (OSCEs) requires careful attention to the construction of clinical scenarios and to the way clinical performance will be measured.
Assessments must be aligned to the curriculum and when constructing an OSCE station, care must be taken to set a sce- nario which tests the desired learning outcomes. This allows the student to demonstrate evidence of the clinical skills they have acquired and will create measures of clinical perfor- mance likely to be valid and reliable. However, there are many sources of variability which can influence the general- izability and reliability of the scores. For example, the asses- sors' judgment, students' ability and the sampling of tasks/cases.
An assessor's judgment may lose validity if the tasks within a station are so complex and extensive, resulting in examiner 'demoralization and fatigue'.1 Such errors will dis- tort the measurement and threaten the validity and reliability of the examination. If an OSCE is planned prudently with 'a common conceptualization' of performance among assessors when they are observing and rating students,2 performance ratings will accurately reflect the clinical skills of students.
The judgments that assessors make may also be prone to measurement error leading to issues of inter-rater reliability i.e., would the student have achieved the same performance rating in the "abdominal" station if a different examiner had rated them?
Performance ratings are susceptible to the various types of internal and external errors which can contribute to less reliable OSCE performance ratings. An example of an inter- nal error would be variations in the level of student's interest and motivation, whereas differences in circuits, student gen- der, the ethnicity of standardised patients and students,3 sites and examiners are considered as external error factors.4 Since the subjectivity in ratings can be a major source of error in OSCEs, one solution to minimise such error and improve OSCE-rating reliability is to obtain ratings from multiple assessors in each station (i.e., each student is rated by two or more independently judging assessors). However, to im- prove the reliability of an OSCE it is better to increase the number of stations with one examiner present than have fewer stations with two examiners assessing5 and is more fea- sible in practice....