TYPES OF ITEM WRITING FLAWS IN MULTIPLE CHOICE QUESTIONS IN MEDICAL EDUCATION AND THEIR EFFECT ON PASSING RATE OF STUDENTS

Authors

  • Sarmud Latif Awan AJK Medical College, Muzaffarabad
  • Shagufta Manzoor Department of Anatomy, AJK Medical College, Muzaffarabad, Pakistan
  • Irum Gillani Department of Community Medicine, AJK Medical College, Muzaffarabad, Pakistan
  • Ziyad Afzal Kayani Department of Surgery, AJK Medical College, Muzaffarabad, Pakistan
  • Humza Farooq Department of Surgery, Shifa International Hospital, Islamabad, Pakistan
  • Sahar Khurshid Department of Surgery, Abbas Institute of Medical Sciences, Muzaffarabad, Pakistan

Keywords:

MCQs, flawed items, Academic achievements, Medical Education, Undergraduate

Abstract

Background: The most popular written exam in health professions education is the multiple-choice style question (MCQ). This study sought to understand the effects of multiple-choice writing errors on students' academic performance in medical education. Methods: This descriptive study was done from December 2017 to June 2019 in Azad Jammu Kashmir (AJK) Medical College, Muzaffarabad. Ten block examinations were included. The item review committee reviewed all MCQs for flaws. Two tests from each class of MBBS were taken. The original tests containing all items were labeled as flawed tests and the result of each flawed test was evaluated. The students were graded into high, moderate and low achievement groups with scores of more than 79.9, between 50–79.9 and less than 50 percent respectively. Flawed items were then removed from the tests by the review committee and the scores of each test (standard test) were determined and compared with flawed tests and its effects were assessed in three achieving groups of students. Optical mark reading (OMR) classic-4 software was used for post-exam analysis and data were analyzed by using SPSS-25.  Results: The passing rate ranged from 68.18% to 90.82% in flawed and 75.54% to 93.69% in standard tests. Most standard tests (7) had higher passing rates than flawed tests. Conclusion: Tests containing in-house developed MCQs have frequent item writing flaws and their inclusion in assessment did affect the passing rates of students but were not statistically significant.

Pak J Physiol 2023;19(4):49–52

Downloads

Download data is not yet available.

References

Downing SM, Yudkowsky R, (Eds). Assessment in health professions education. New York: Routledge; 2009.p.149–84.

Palmer, EJ, Devitt, PG. Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper. BMC Med Educ 2007;7:49.

Schuwirth LW, van der Vleuten, CP. Different written assessment methods: what can be said about their strengths and weaknesses? Med Educ 2004;38(9):974–9.

DiBattista D, Kurzawa, L. Examination of the quality of multiple-choice items on classroom tests. Can J Scholarsh Teach Learn 2011;2(2):4.

Case SM, Swanson DB. Constructing written test questions for the basic and clinical sciences. National Board of Medical Examiners: Philadelphia, PA; 1998.

Haladyna TM, Downing SM, Rodriguez MC. A review of multiple-choice item-writing guidelines for classroom assessment. Appl Meas Educ 2002;15(3):309–34.

Downing SM. Construct-irrelevant variance and flawed test questions: Do multiple-choice item-writing principles make any difference? Acad Med 2002;77(10 Suppl):S103–4.

Downing SM. The effects of violating standard item writing principles on tests and students: the consequences of using flawed test items on achievement examinations in medical education. Adv Health Sci Educ Theory Pract 2005;10:133–43.

Tarrant M, Ware J. Impact of item-writing flaws in multiple-choice questions on student achievement in high-stakes nursing assessments. Med Educ 2008;42(2):198–206.

Wadi M, Rahim AFA, Yusoff MSB, Baharuddin, KA. The effect of MCQ vetting on students’ examination performance. Educ Med J 2014;6(2):e16–26.

Przymusza?a P, Piotrowska K, Lipski D, Marciniak R, Cerbin-Koczorowska M. Guidelines on writing multiple choice questions: A well-received and effective faculty development intervention. SAGE Open 2020;10(3):2158244020947432.

Fayyaz Khan H, Farooq Danish K, Saeed Awan A, Anwar M. Identification of technical item flaws leads to improvement of the quality of single best Multiple Choice Questions. Pak J Med Sci 2013;29(3):715–8.

Downloads

Published

31-12-2023

How to Cite

1.
Awan SL, Manzoor S, Gillani I, Kayani ZA, Farooq H, Khurshid S. TYPES OF ITEM WRITING FLAWS IN MULTIPLE CHOICE QUESTIONS IN MEDICAL EDUCATION AND THEIR EFFECT ON PASSING RATE OF STUDENTS. Pak J Phsyiol [Internet]. 2023 Dec. 31 [cited 2024 Feb. 27];19(4):49-52. Available from: https://pjp.pps.org.pk/index.php/PJP/article/view/1587