How to Design Better Multiple Choice Questions
There are numerous challenges when we talk about writing effective multiple-choice questions. Nevertheless, writing a good multiple-choice question is essential and must not be ignored or taken lightly. A poorly written multiple-choice question can bring serious problems to an assessment, like compromising assessment data, confusing participants, or causing legal and organizational issues.
There are two parts to multiple-choice questions (MCQs): a stem and multiple alternatives. A stem indicates the problem or question. Below the question is a list of multiple alternatives that consist of a key which is the right answer and a few distractors that are reasonable but wrong answers to that question. The MCQ response method requires students to indicate which alternative best answers the question or completes it.
In the following article, we will discuss several important elements of writing effective multiple-choice questions so you can begin improving them right away.
Ways to Design Effective Multiple Choice Questions
The use of MCQs as a method of assessment has numerous advantages. A few advantages of having MCQ assessments are that they are less time-consuming, easy to mark, and can be scored through computers. It is especially beneficial for classes that have a great number of students enrolled. By designing MCQs well, they can cover a wide range of subject areas and objectives, which provide a means for teachers as well as the students to assess their abilities objectively.
Developing an Appropriate Stem for Your MCQ’s
As discussed earlier, a stem is a statement or the actual problem of the question. Here are a few ways through which you can develop better stems:
Make Your Stems Meaningful
Make sure your stem provides some meaning on its own and should pose a clear problem. It becomes easier to pay attention to the learning outcome when you have a strong and clear problem/stem. An unclear or a failing stem, however, is more likely to assess students’ abilities to draw concussions from obscure and unclear descriptions as opposed to direct measurement of whether the learning targets have been met.
Avoid Adding Irrelevant Information in the Stem
When designing a stem, make sure it doesn’t contain any irrelevant or confusing content. If you do so, it will reduce the validity and readability of the test results.
Be Careful of Using too Many Negatively Stated Stems
It is best to use affirmative sentences when creating stems. However, you can still state them negatively only in circumstances when it is necessary to achieve significant learning outcomes.
This point is especially important because items with negative phrasing are often difficult for students to understand. If a negative statement is necessary to convey an important learning outcome, for example, identifying risky clinical or laboratory practices, make the negative element stand out by capitalizing, underlining, or Italicizing it.
Make Your Stem a Question or Partial Sentence
Question stems are preferable since they allow students to concentrate on finding the answer instead of focusing on the martial statement and trying to recall each alternative sentence in their working memory. The cognitive workload increases if the stem has a missing initial or interior part. Therefore, it is better to avoid such construction.
Developing an Appropriate Alternative for Your MCQ’s
Alternatives, as mentioned earlier, are a list of possible answers to the stem. It consists of one right and 2-4 wrong answers known as distractors. Let’s find out how you can come up with some effective alternatives for your MCQs.
Make Your Alternatives Plausible
The main aim of incorrect alternatives is to act as distractors. Students who failed to achieve the learning outcome select those distractors, whereas students who achieved them, ignore the distractors and choose the right answer.
The use of implausible alternatives does not serve as a distractor and should therefore be avoided by the MCQs developer. It is most likely that students' common errors will serve as a good source of distractions.
Your Alternatives Must be Clear and Concise
When the alternative is overly wordy, they help in the measurement of students’ reading abilities rather than assessing whether the targets have been met.
State Mutually Exclusive Alternatives
In general, mutual exclusivity refers to events that can’t occur at the same time. Alternatives that have too much overlapping content are believed to be quite tricky elements by the test-taker. However, overuse of these tricky items may result in diminishing respect and trust for the evaluation process.
State Homogenous Alternatives in Your MCQs
Homogenous alternatives make the test effective and appropriately assess whether or not the learning outcomes have been achieved. On the contrary, heterogeneous alternatives give students a clue about the correct answer.
Leave No Room for Hints
The best alternatives are the ones that provide no clues about the correct answer to the students. Smart test takers are aware of the subtle clues that specify the right answer, for example, differences in language, length, format, and grammar in the alternative answers.
Therefore, alternatives should focus on these factors:
- Maintain consistency in grammar
- Are in a parallel form
- Have similar length
- Use the same language (e.g., a language unlike textbooks or a language similar to textbooks)
Avoid Using “None of the Above” and “All of the Above” Alternatives
In the case of an answer that includes "all of the above", students who can only identify two or more alternatives will mark the correct answer even if they are not sure about the other alternatives. Similarly, when test developers add “none of the above” as an alternative to the stem, students who can eliminate one choice can also omit another. Whatever the case may be, students can reach a correct conclusion by using partial knowledge.
Use Logical Order to Present the Alternatives
An orderly presentation of alternatives is recommended (e.g., numerical or alphabetical) in order to avoid favored positions.
Have a Variety of Alternatives until they are Plausible
It does not matter how many alternatives you add to the items as long as they are all plausible. In general, there is not much difference between items containing two, three, and four distractors in terms of difficulty, discrimination, and test score reliability.
Examples of Good Stems and Alternatives
Avoiding Negatively Stated Stems
Good Example:
After how many days does leap year come?
a. One
b. Two
c. Three
d. Four
Bad Example:
Which of the following is not an example of a mammal?
a. Cat
b. Elephant
c. Giraffe
d. Penguins
Avoiding Opinion-Based Questions
Good Example:
Which character plays the leading role of Venom?
a. Chris Hemsworth
b. Chris Pratt
c. Eddie Brock
d. Ryan Reynolds
Bad Example:
Which country is worth visiting?
a. Greece
b. Italy
c. Switzerland
d. Vietnam
Phrasing the Stem as a Question
Good Example:
What is the capital of China?
a. Beijing
b. Shanghai
c. Shenzhen
d. Wuhan
Bad Example:
Mammal who can fly is:
a. Bat
b. Ostrich
c. Peacock
d. Pigeon
Formatting Distractors Vertically
Good Example:
Which is the largest animal in the world?
a. Blue Whale
b. Elephant
c. Giraffe
d. Hippopotamus
Bad Example:
Which is the largest animal in the world?
a. Blue Whale b. Elephant c. Giraffe d. Hippopotamus
Practices to Follow
You will need to answer Yes to the following questions in order to construct valid multiple choice questions:
- Can students respond to this item if they possess the prior knowledge required?
- Does this question cover the important concept that your learners should have learned while taking the lecture?
- Have you used appropriate vocabulary for the students when stating the item?
- Does your question pose the appropriate level of thinking for:
- The content addressed?
- The student’s position in terms of novice to expert continuum?
- Learning objectives of the subject?
Tips for Writing Effective Multiple Choice Questions
Here are a few more tips you can use to create engaging and effective Multiple Choice questions that will help in the measurement of learning objectives.
- Limit the number of answers you list. It is more likely that a student will remember the wrong answer if they see a huge list of answers. Additionally, students are more likely to spend a great deal of time on questions with many alternatives, limiting the amount of content they are able to cover on the test. It is best to stick to 3-4 options, a great balance between efficiency and quality.
- Do not ask trick questions. Riddles, brain teaser questions, and gimmicky questions confuse and frustrate the learner and aren’t something that gets students engaged in learning. Even though it may seem attractive and tempting to add questions that encourage students to put extra effort and spend a little more time thinking for the right answer, tricky questions may confuse students—even the ones who are giving their best and doing great on other sections of the test—and ultimately be harmful than being productive.
- Format your questions simply. “None of the above” “all of the above,” and multi-answer alternatives (such as “B, C, and D”) can be manipulated. The list of correct responses is narrowed as soon as a student eliminates one of the possible answers, creating artificially higher test scores.
- Don't make tests too hard, but challenge them enough to keep students engaged. Taking challenging quizzes discourages future study and makes the material seem unbreakable. It is also believed that students may commit bad data to memory when taking a difficult test because the answers we select during the test are likely to stay in our minds. When students are attentive, they should get around 70 - 80 percent of the questions answered rightly. If they take the test any less, they may not be able to learn what is required.
- Following up with feedback is essential. Reviews are crucial. A post-test review not only provides the teacher with insight into skills that students may be having difficulty with but also provides the students with an opportunity to master the test again. The response to a multiple-choice test can enhance its positive impact on learning and significantly reduce its negative impact. It also encourages students to do their best and improve in future tests.
Benefits of Multiple Choice Questions
Numerous reasons make multiple choice questions appealing to instructors. The multiple choice test is very practical in classroom settings where there are many students - especially from a time management perspective. Some of the other advantages are:
Read our blog on 15 Qualities You Must Have to Be a Good Instructional Designer for more details.
Versatility
There are various levels of learning outcomes that multiple choice tests can measure, ranging from basic recall to analysis, evaluation, and application. Due to the fact that students can choose from a number of options, however, multiple choice items are obviously limited to topics that can be tested. For example, they cannot be used to test students' abilities to organize thoughts or to explain or express ideas and concepts clearly.
Reliability
The reliability of a test is determined by its ability to measure a learning outcome consistently. As compared with true/false questions, multiple choice questions are less vulnerable to guessing, thus they are a more reliable form of assessment. Increased reliability is achieved by having a greater number of MCQs focused on one learning objective. Multiple choice items, by contrast, are scored objectively, which eliminates the problems with inconsistency in scoring that can be associated with essay questions.
Validity
In order for a test to be valid, it must accurately measure the results it claims to measure. Multiple choice items tend to enable students to answer them more quickly than essay questions, which means that multiple choice tests are generally able to cover a broader scope of course content, which makes them more valid.
Bottomline
In creating multiple choice questions to assess higher-order thinking, create questions that emphasize Bloom's taxonomy's higher levels of cognition. If the stem requires students to apply course principles, analyze a problem, or evaluate alternatives, it is more likely to require higher-level thinking and thus test students' thinking capabilities.
This type of item has the advantage that it can be scored quickly, giving students quick feedback and allowing efficient ways to assess large numbers of students over a wide range of content. The drawback of multiple choice questions is that they take time to write well, especially if they are measuring higher-order thinking.
Read our article on Instructional Design Models for Employee Training for further details on Bloom’s Taxonomy.