- ExamEval
- Item Writing Flaws
- Absolute Terms
"Always" and "Never" Are Red Flags in Multiple-Choice Questions
Why Absolute Terms are Problematic
Answer choices containing absolute terms like "always", "never", "all", "none", and "must" are rarely the correct answer in a multiple-choice assessment. Particularly in the medical field, where there are few "absolute" truths, test-wise students are likely to eliminate these answer choices even if they don't actually have a full understanding of the concept being assessed.
Consider the following fictitious example. Even without having any knowledge of the Veritaserum potion, a test-wise student can make an educated guess on the correct answer:
In this example, a test-wise student's exam performance will reflect their skill in identifying and exploiting this item-writing flaw, rather than their true understanding of the subject matter. As a result, their scores may overestimate their actual competence.
Absolute terms compromise assessment validity by allowing students to eliminate answer choices based on test-taking strategies rather than content knowledge. This flaw introduces construct-irrelevant variance, as scores may reflect test wiseness instead of true understanding.
Examples of Absolute Terms in Health Sciences Education
In health sciences education, the use of absolute terms is particularly problematic because clinical practice is nuanced and rarely governed by absolutes. For example, a question stating that a specific drug "always" causes a certain side effect or that a particular symptom is "never" associated with a disease fails to account for patient variability and exceptions. As Tarrant et al. found, such flaws are common in high-stakes nursing exams, undermining their ability to measure clinical judgment accurately.
Correcting This Item Writing Flaw
In most cases, an absolute term ("always", "never", etc.) can be replaced with wording that indicates a less black-and-white qualifier:
- Replace "Always required" with "Typically indicated".
- Replace "Never appropriate" with "Rarely suitable".
- Replace "All patients need" with "Most patients benefit from".
- Replace "Cannot be done" with "Generally not recommended".
By removing absolute terms from answer choices, exam questions more accurately assess student understanding and professional competence. This approach ensures that assessments measure true knowledge rather than test-taking strategies.
Identifying and removing absolute terms can be challenging, especially in large exams or when multiple writers are involved. AI-powered exam analysis platforms, such as ExamEval, can automate the detection of absolute terms and suggest improved wording for exam questions.
References
- National Board of Medical Examiners (NBME). Item-Writing Guide. Philadelphia, PA: National Board of Medical Examiners; February 2021.
- Haladyna TM, Downing SM, Rodriguez MC. A review of multiple-choice item-writing guidelines for classroom assessment. Appl Meas Educ. 2002;15(3):309-334. doi:10.1207/S15324818AME1503_5
- Tarrant M, Knierim A, Hayes SK, Ware J. The frequency of item writing flaws in multiple-choice questions used in high stakes nursing assessments. Nurse Educ Pract. 2006;6(6):354-363. doi:10.1016/j.nepr.2006.07.002