Z. Naqvi  ( Department for Educational Development,The Aga Khan University, Karachi. )
R. Ahmed  ( Department for Educational Development,The Aga Khan University, Karachi )

Currently, MCQs is perhaps the most widely used mode of testing that is reliable, valid1 and can test application of basic sciences for understanding of clinical problems. The dilemma to construct MCQ tests that validly reflect individual performance in making (a doctor?) is ever increasing especially with emphasis of medical educationists on relevance of instruction and assessment. Generalizability analyses indicate that performance in one topic area does not predict performance in other areas very well.2 To summarise, most authors believe that a good test question should address important content and also be well structured. Therefore, to evaluate skills in medical students, it is preferable to sample more presenting complaints rather than to sample more items within a small number of presenting complaints. The next phase of this discussion will be directed at determining whether there are benefits to using the current multiple choice formats.

According to Norman2 MCQs are expressly designed to assess knowledge. He further writes that although many authors believe that MCQs merely test retention of isolated facts, well-designed MCQs can assess higher levels of learning, understanding and application of the learnt information.3

Almost all educationsts believe that all effective assessments should be valid and reliable. MCQs ability to cover large areas increases the reliability of the tool over almost all other modes of testing.4

MCQs, for purpose of description can be broadly divided into two families of items: The true and false variety (requiring response to all items) and the One Best Answer (single response) questions.5 The terms Multiple True/False and Best Answer deserve wider applicability than at present when discussing multiple choice questions (MCQs). They describe the truthfulness of the choices and distinguish between the question types

For the items in these questions the examinee has to make decision about the cut-off point - extent of 'trueness' for being correct. This provokes guessing, as most of the items are neither completely true nor entirely false. To prevent extensive guessing, True and false items have to be covered with negative marking which deducts marks from the right choices for incorrect responses.6,7 This masks the actual strengths and weaknesses of students.

Which of the following is true about tuberculosis?
1. It occurs commonly in women
2. It is seldom associated with acute pain in joints
3. It may be associated with chondrocalcinosis
4. It is hereditary in many cases
5. It responds well to Rifampicin
The options in this set are heterogeneous and deal with miscellaneous facts; the stem is unfocussed, vague and does not provoke thinking without looking at the options. The incorrect responses are not totally wrong and may be called less correct than the keyed answers. This makes reaching a consensus on the correct response very difficult even for a group of experts, making objective interpretation of the results complex.

Easy to make
Each set tests many aspects of the same topic

Tests mostly recall
- Pre and post examination review is ambiguous (A group of experts cannot usually agree on correct answer5)
- Width of the paper is decreased (100 items test 20 - 25 topics)
- Options have to be absolutely correct or false (no shades of gray are permissible)
- Presence of cues in the stem or the options may help the test-wise candidate
- Negative marking is essential to prevent indiscriminate guessing.

In its standard format the question consists of a stem followed by five alternate options.3 An intelligently (well) constructed one Best Answer Question, provides a valid and reliable assessment of the students' competence in the desired "content" area. This reduces the probability of guessing to 20% (50% in True/False) and may also increase the width of topics that can be assessed.

A 75-year-old man has problem in rising from a sitting position and straightening his trunk. He has no difficulty in flexing his trunk. The most probable site of injury is:
A. gluteus maximus
B. gluteus minimum
C. hamstrings
D. iliopsoas
E. obturator internus

- Increased breadth of testing (100 items can assess 100 different topics)
- Carefully made questions may test interpretation and some levels of problem solving
- Post Examination review provides a comprehensive analysis of the learning issues and misconceptions amongst students.

- Relatively difficult to construct
- May overtly test recognition only
- May direct students' learning approaches negatively.

According to Holsgrove6 multiple-choice questions (Single Best Questions) is among the most common components of medical exams. They are potentially reliable and easy to mark accurately. Their validity leaves much to be desired. Lowe7 writes that for the new MRCS examination, it became important that the test evolved away from the template of multiple true-false questions to test mental processes that are not simple factual recall, and the single best type was introduced. Furthermore, Holsgrove6 considers the common practice of negative marking callous which is essential for true and false variety. Additionally, Pamphlett and Farnill8 report that although women students depicted higher levels of anxiety than men in negatively marked multiple choice tests, levels for both genders were indicative of a moderately stressful situation.
Finally, Case and Swanson5,9 recommend that True and False Questions/items should be avoided as far as possible. They feel that although item-writers find these easier to write, careful reviews in committees reveal that the items contained ambiguities that were in many instances difficult to remove. Besides they feel that Single Best Questions can assess higher levels of learning as compared to the true and false variety.

1. Braddom CL. A brief guide to writing better test questions. Am J Phys Med Rehab 1997; 76:514-16.
2. Norman G. Multiple choice questions. in. McMasters University's evaluation methods: a resource hand book, 1995, pp. 47-54.
3. Case SM, Swanson DB, Stillman PL. Evaluating diagnostic pattern recognition: the psychometric characteristics of a new item format. Proceedings of Annual Conference of Research in medical education.; 1998;27:3-8.
4. Schultheis NM. Writing cognitive educational objectives and multiple-choice test questions. Am J Health Sys Pharmacol 1998; 55:2397-401.
5. Case SM, Swanson DB. Constructing written test questions for the basic and clinical sciences, 1996. http://www.nbme.org/itmwrgd1.htm
6. Holsgrove G. (1993, 1992) Guide to postgraduate exams: multiple-choice questions. Medical Colleges of St. Bartholomew's, London. British Journal of Hospital Medicine, 1993;48:757-61.
7. Lowe DG, Foulkes J, Russell RC. BS to MRCS at the RCS: philosophy, format and future. Annals of Royal College of Surgeons England. 1998;80:213-18.
8. Pamphlett R, Farnill D. Effect of anxiety on performance in multiple choice examination. Medical Education 1995;29:297-302.
9. PremadasaI G. A reappraisal of the use of multiple choice questions. Medical Teacher. 1993;15:237-42.