ExamEval
Item Writing Flaws
Combination Answer Format

Combination, Complex, or K-Type Question Formats

By Sean P. Kane, PharmD, BCPS

Published June 18, 2025

What is a Combination, Complex, or K-Type Multiple-Choice Question Format?

A combination, complex, or K-type question is a multiple-choice item where some or all of the answer choices are combinations of other options (e.g., "A and B," "A, B, and C," or "A and C only"). These questions function as a more complicated version of true/false items, requiring students to evaluate multiple statements and their logical relationships within a single question.

Combination format questions are a significant flaw in assessment design because they add logistical complexity on top of the core concept being tested, increasing the cognitive load on students. Instead of purely testing knowledge of the subject matter, these questions also test the ability to perform logical operations. As a result, even students who understand the content may be penalized if they struggle with the logical structure, leading to assessment results that do not accurately reflect true understanding.

Critical Issue

Combination format questions introduce construct-irrelevant variance by testing logistical processing and working memory alongside content knowledge. This dual demand means that a student's score may not accurately reflect their true understanding of the material, but rather their ability to handle complex logical structures under pressure.

Cognitive Load Problems

Research in cognitive psychology demonstrates that combination formats impose excessive demands on working memory. Students must simultaneously:

Read and understand the question stem.
Evaluate the correctness of each individual option.
Hold these evaluations in memory.
Analyze the combination choices to determine which one accurately reflects their evaluations.

This multi-step process increases cognitive load, making it more likely for students to make errors due to processing difficulties rather than knowledge gaps. These formats can disadvantage students who are not skilled in logical reasoning, even if they know the subject matter.

Example of Combination Format Flaw in Health Sciences Education

Flawed Question

A patient with community-acquired pneumonia typically presents with which of the following signs and symptoms?
I. Fever
II. Productive cough
III. Bradycardia

A. I and II only ✓

B. I and III only

C. II and III only

D. I, II, and III

This combination format requires students to evaluate multiple statements and determine correct combinations, shifting focus and cognitive load from clinical knowledge to logical reasoning.

Corrected Question

A patient with community-acquired pneumonia most commonly presents with which of the following clinical findings?

A. Fever and productive cough ✓

B. Bradycardia and hypothermia

C. Dry cough and normal temperature

D. Chest pain and rapid heart rate

This question directly assesses knowledge of pneumonia presentation without requiring complex logical operations. It focuses on the most common combination of symptoms rather than testing ability to evaluate multiple separate statements.

False Sophistication

While combination formats may appear to create more rigorous or challenging questions, they often represent a form of false sophistication. The complexity arises from the format itself rather than from the depth or nuance of the content being assessed. True assessment rigor comes from questions that require deep conceptual understanding and application of knowledge, not from those that demand intricate logical processing. In a review by Haladyna and colleagues, K-type questions are more difficult but do not improve the question's discrimination index.

Improving Complex or Combination Question Formats

Convert to Single Best Answer: Rewrite so each choice stands alone rather than requiring combinations.
Use Clinical Scenarios: Frame questions in realistic contexts to assess application, not logical manipulation.
Target One Learning Objective: Focus each question on a specific concept or skill.
Use Clear Language: Avoid complex logical structures that confuse students.
Create Plausible Distractors: Develop incorrect choices that reflect common misconceptions.
Split Complex Ideas: Use separate questions for related concepts rather than combining them.

Combination, complex, or K-type question formats are a common item-writing flaw that can be difficult to spot and correct. ExamEval uses advanced AI to automatically detect and flag these problematic formats, helping educators create clearer, more valid assessments. Try ExamEval to ensure your exams are free from confusing combination questions and focused on measuring true student understanding.

References

National Board of Medical Examiners (NBME). Item-Writing Guide. Philadelphia, PA: National Board of Medical Examiners; February 2021.
Haladyna TM, Downing SM, Rodriguez MC. A review of multiple-choice item-writing guidelines for classroom assessment. Appl Meas Educ. 2002;15(3):309-334. doi:10.1207/S15324818AME1503_5