Content area
Full Text
Introduction
The National Board of Medical Examiners (NBME) item-writing manual is a mainstay of guidance for question writers aiming to produce high-quality questions. It gives advice for developing many multiple-choice question formats, including multiple true/false, single best answer (SBA) and extended-matching questions (EMQs). Since the item-writing manual was commissioned, most medical examinations of knowledge have adopted the SBA format. The reasons SBAs have become so widely adopted are varied, but include the pitfalls of the other formats. Multiple true/false questions are the easiest to construct but may comprise simply a series of sometimes unrelated statements. However, they rarely reflect the real-life practices of medical diagnosis and management, which come in multiple shades of grey. EMQs assess application of knowledge rather than simple recall but are hard to write well. It is exceptionally difficult to construct EMQs with a suitable number of distractors applicable to all scenarios without undue redundancy. In contrast, well-written SBA questions can concisely assess the application of clinical knowledge, sampling widely from potentially hundreds of learning outcomes in a single easily marked assessment. These characteristics have led them to become the most widely used multiple-choice question format in medical schools. 1-5
The NBME item-writing manual provides valuable advice on constructing SBA questions. 1 Here, we highlight and elaborate upon its key points and offer additional advice derived from writing experience.
Some general points
The assessors' work for multiple-choice question examinations is almost entirely in their preparation, rather than in their marking. SBA papers can be prepared many months in advance and avoid the postexam frenzy of essay marking. However, it can be very challenging to produce high-quality, discriminating SBA questions that ensure fair, valid and reliable assessments.
The most clinically applicable SBA questions present a scenario, with relevant and plausible options (at least to the mind of a borderline candidate); the 'best' answer might be judged as 80% correct and the distractors perhaps 20-30% correct. While students in clinical practice obviously do not have the prompt of possible options, SBAs do encourage students to work with conditional probabilities that compare to real clinical practice.
The language and scientific content of questions must show clarity, precision and economy to ensure that they test only knowledge and application of knowledge rather than the interpretation...