Establishing a minimum passing score for the rating scale in simulation training for surfactant administration using the LISA procedure

Abstract

Introduction

In a previous study, a rating scale for simulation training on surfactant administration using the LISA (Less Invasive Surfactant Administration) procedure was developed and validated. Our objective was to determine a minimum passing score for this rating scale to use it so that it could be used for normative and certifying evaluation.

Methods

The LISA scale comprises 8 categories and 25 items. It was developed and agreed upon by a panel of 12 LISA procedure experts, and subsequently validated through simulation sessions involving 40 learners. Two independent assessors evaluated these 40 simulations. The Cronbach’s alpha score for this scale is 0.72, the R-squared value is 0.99, and the intra-class correlation coefficient is 0.92. Three different methods were employed to establish the minimum passing score: the Angoff method, the Borderline group method, and the Contrasting Group method. For the Angoff method, we enlisted 5 experts from the panel who developed the rating scale. In the Borderline Group method, the two assessors evaluated the 40 simulations following the prescribed methodology. For the Contrasting Group method, the outcomes of various simulation sessions were analyzed.

Results

Using the Angoff method, the minimum passing score was determined to be 35 out of 50, equivalent to 70.6% (standard deviation: 15.8%). Employing the Borderline Group method yielded a minimum passing score of 31.70 out of 50, i.e., 63.4%. Finally, utilizing the Contrasting Group method, the minimum passing score was found to be 33 out of 50, or 66%.

Conclusion

The three methods employed resulted in varying minimum passing scores. A higher score is likely to ensure enhanced safety and quality of patient care, while also facilitating the learner’s progression in simulator training. We recommend considering a minimum passing score of 35 out of 50.