EFFECT OF ITEM WRITING FLAWS IN MULTIPLE CHOICE QUESTIONS ON STUDENTS’ ACHIEVEMENT GROUPS AND RELIABILITIES OF TESTS IN AZAD JAMMU KASHMIR MEDICAL COLLEGE

Authors

  • Sarmud Latif Awan Department of Surgery, AJK Medical College, Muzaffarabad, Pakistan
  • Shagufta Manzoor Department of Anatomy, AJK Medical College, Muzaffarabad, Pakistan
  • Irum Gillani Department of Community Medicine, AJK Medical College, Muzaffarabad, Pakistan
  • Ziyad Afzal Kayani Department of Surgery, AJK Medical College, Muzaffarabad, Pakistan
  • Sahar Khurshid Department of Surgery, Abbas Institute of Medical Sciences, Muzaffarabad, Pakistan
  • Humza Farooq Department of Surgery, Shifa International Hospital, Islamabad, Pakistan

Keywords:

Academic achievements, flawed items, MCQs, Medical Education, Undergraduate

Abstract

BACKGROUND: MCQs can test different levels of cognitive knowledge from recall to analysis. This study was to determine the item writing flaws in multiple choice questions and their effects on students’ academic achievements and test reliabilities. Methods: This study was conducted at the Azad Jammu Kashmir (AJK) Medical College Muzaffarabad from December 2017 to June 2019. Ten end-of-block exams were used in this study. The item review committee went over every MCQ. Two tests from each class of MBBS were taken. The initial tests without being reviewed were deemed to be flawed tests. The outcomes of each test were assessed, and the students were ranked into three achievement levels high, moderate, and low based on their scores of >79.9, between 50 and 79.9, and <50%, respectively. After removal of flawed items from the exams, the scores of each test (the standard test) were calculated, compared to the flawed tests, and its impacts were evaluated among three achievement groups. The post-exam analysis of test items was done on optical mark reading (OMR) classic-4 programme and data analysis on SPSS-25.  Results: In one test i.e. test no 6 the null hypothesis was rejected with p<0.05. However, in all other tests null hypothesis could not be rejected on statistical basis. Moderate achievers and low achievers had statistically significant correlation (p<0.05). The flawed tests had better reliabilities with statistically significant (p<0.05). Conclusion:   Flawed MCQs had negative impact on high and moderate achievers. Reliabilities of flawed test were greater than standard tests.

Downloads

Download data is not yet available.

References

Downing SM, Written tests. In: Downing SM, Yudkowsky R. (Eds). Assessment in health professions education. New York: Routledge; 2009.pp 149–84.

Palmer EJ, Devitt PG. Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper. BMC Med Educ 2007;7:49.

Schuwirth LW, van der Vleuten CP. Different written assessment methods: what can be said about their strengths and weaknesses? Med Educ 2004;38(9):974–9.

DiBattista D, Kurzawa L. Examination of the quality of multiple-choice items on classroom tests. Can J Scholarsh Teach Learn 2011;2(2):4.

Case SM, Swanson DB. Constructing written test questions for the basic and clinical sciences. Philadelphia: National Board of Medical Examiners; 1998.

Haladyna TM, Downing SM, Rodriguez MC. A Review of Multiple-Choice Item-Writing Guidelines for Classroom Assessment. Appl Meas Educ 2002;15(3):309–33.

Downing SM. Construct-irrelevant variance and flawed test questions: Do multiple-choice item-writing principles make any difference? Acad Med 2002;77(10 Suppl):S103–4.

Downing SM. The effects of violating standard item writing principles on tests and students: the consequences of using flawed test items on achievement examinations in medical education. Adv Health Sci Educ Theory Pract 2005;10(2):133–43.

Tarrant M, Ware J. Impact of item-writing flaws in multiple-choice questions on student achievement in high-stakes nursing assessments. Med Educ 2008;42(2):198–206.

Axelson RD, Kreiter CD. Reliability. In: Downing SM, Yudkowsky R. (Eds). Assessment in health professions education. New York: Routledge; 2009.pp 57–73.

Downloads

Published

31-03-2024

How to Cite

1.
Awan SL, Manzoor S, Gillani I, Kayani ZA, Khurshid S, Farooq H. EFFECT OF ITEM WRITING FLAWS IN MULTIPLE CHOICE QUESTIONS ON STUDENTS’ ACHIEVEMENT GROUPS AND RELIABILITIES OF TESTS IN AZAD JAMMU KASHMIR MEDICAL COLLEGE. Pak J Phsyiol [Internet]. 2024 Mar. 31 [cited 2024 Jun. 16];20(1):41-4. Available from: https://pjp.pps.org.pk/index.php/PJP/article/view/1586

Issue

Vol. 20 No. 1 (2024): Pakistan Journal of Physiology

Section

Original Article

License

Copyright (c) 2024 Sarmud Latif Awan, Shagufta Manzoor, Irum Gillani, Ziyad Afzal Kayani, Sahar Khurshid, Humza Farooq

Creative Commons License

This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License.

Pakistan Journal of Physiology, Pak J Physiol, PJP is FREE for research and academic purposes. It can be freely downloaded and stored, printed, presented, projected, cited and quoted with full reference of, and acknowledgement to the author(s) and the PJP. The contents are published with an international CC-BY-ND-4.0 License.