- ExamEval
- Item Writing Flaws
- Single Best Answer
Single Best Answer: When Multiple Options Seem Correct
Questions with Ranked Choices Are Preferred
Experts recommend a "single best answer" multiple-choice question format for assessing student knowledge, particularly in health sciences education. This format requires students to "rank" or "order" answer choices from least to most appropriate and ultimately select the best answer out of multiple plausible choices.
With this format, a common student complaint is that multiple answers seem equally correct. This problem typically arises when question writers attempt to create nuanced distinctions between good and better options, but fail to make these distinctions clear and defensible. Students who possess solid content knowledge may identify multiple plausible answers and struggle to determine which one the test creator intended as "best."
When a question has multiple plausible answers, it introduces construct-irrelevant variance. The question no longer measures content knowledge alone, but also the student's ability to guess the instructor's intended answer. This undermines the validity of the assessment.
Consider the following example:
Impact on Students
When students encounter questions with multiple defensible answers, they experience several negative effects:
- Confusion and Frustration: Students who have mastered the content may become confused and frustrated when they cannot identify a single best answer.
- Second-Guessing: The ambiguity can cause students to second-guess their knowledge and change correct answers to incorrect ones.
- Loss of Confidence: Over time, exposure to poorly written questions can erode students' confidence in their own knowledge and in the fairness of the assessment process.
To avoid these issues, it is crucial to ensure that every question has a single, clear, and defensible best answer. This can be achieved through careful question writing, peer review, and post-exam analysis.
Examples of Single Best Answer Problems in Health Sciences Education
In this example, there is no nuance at all -- there are truly two equally correct answers, making the question flawed:
Correcting Flaws with Single Best Answer Question Formats
When two answers are equally correct (as in the statin example above), the answer choices should be revised so that only a single answer choice is correct.
More commonly, however, is that a question is ambiguous or not specific enough to justify one answer being "more" correct over another. This problem can be addressed in several ways:
-
Provide Specific Context: Include enough situational detail to make one option clearly superior. Specify patient characteristics, timing, clinical scenarios, or other relevant factors.
-
Use Qualifying Language: Employ terms like "most appropriate," "priority," "immediate," or "essential" when there are clear hierarchies among options.
-
Specify a Goal in the Question Stem: Instead of stating "which is most appropriate", provide a more specific question stem lead-in, such as "Which is most appropriate to reduce long-term mortality" or "Which is most appropriate to relieve the symptoms of dysuria".
Multiple defensible answers in exam questions frustrate students and compromise assessment validity. ExamEval provides AI-powered analysis that automatically detects ambiguous answer choices and recommends specific improvements to ensure each question has one defensible correct answer, helping health professions educators create more reliable and fair assessments.
References
- National Board of Medical Examiners (NBME). Item-Writing Guide. Philadelphia, PA: National Board of Medical Examiners; February 2021.
- Haladyna TM, Downing SM, Rodriguez MC. A review of multiple-choice item-writing guidelines for classroom assessment. Appl Meas Educ. 2002;15(3):309-334. doi:10.1207/S15324818AME1503_5