A Comparison of the Effectiveness between Multiple-Choice and Essay Writing Exams in Writing Competency Assessment
Keywords:
an essay writing exam, a multiple-choice exam, writing competencyAbstract
Objectives: This research aimed to study the effectiveness of multiple choice exams when used to assess writing skills compared to essay writing exams.
Methods: The subjects of the study were 79 third-year students who were studying English as their minor subject at the Faculty of Arts, Silpakorn University in 2021 and taking the writing course 412221 Reading and Writing. Two types of exams were used in the study: a multiple choice exam and an essay writing exam. A cause & effect essay was used in both multiple choice and essay writing exams. The 30-item multiple choice exam covered essay elements and organization, grammar points, and vocabulary. As for the essay writing exam, students were instructed to write one essay on one of the four given topics.
Results: Results showed that the students performed better in the multiple choice exam, scoring an average of 21.0253 points while in the essay writing exam, the students received 19.3418 points. Looking at each area of assessment individually, students achieved a marginally better performance in the grammar and vocabulary topics of the multiple choice exam, while they received a slightly higher score in the essay elements of the essay writing exam. However, except for the vocabulary part, the difference in the score of the remaining parts was not statistically significant. In addition, the difficulty index mean comparison showed that both types of exam were of comparable difficulty and the discrimination index of the multiple choice exam and essay writing exam were comparable with no statistical significance, 0.2743 and 0.2848 respectively. One contributing factor to this may be the use of the essay writing exam analytic rubric, as mistakes other than what were included in the analytic rubrics were not recorded. Furthermore, the analytic rubrics had a fixed maximum number of points that could be deducted.
Application of this study: The results suggest that multiple choice exams, if constructed well and covered all the topics that are taught in class, can be used as a tool in evaluating writing competency and can have comparable effectiveness to essay writing exams.
References
Boyle, A. P. (2019). Writing Better Essay Exams. IDEA Paper, 76. Retrieved 15 July 2022, from https://ideacontent.blob.core.windows.net/content/sites/2/2020/01/Writing_Better_Essay_Exams_IDEA_Paper_76.pdf
Brame, C. J. (n.d.). Writing Good Multiple Choice Test Questions | Center for Teaching | Vanderbilt University. Vanderbilt Center for Teaching. Retrieved 17 July 2022, from https://cft.vanderbilt.edu/guides-sub-pages/writing-good-multiple-choice-test-questions/
Brown, H. D. (2001). Teaching by principles: An interactive approach to language pedagogy. 2nd ed. Addison Wesley Longman.
Cahill, D. R., & Leonard, R. J. (1999). Missteps and masquerade in American medical academy: Clinical anatomists call for action. Clinical Anatomy, 12, 220-222.
Cameron, B. (2021). The Benefits of Essay Writing. Time Business News. Retrieved 15 July 2022, from https://timebusinessnews.com/the-benefits-of-essay-writing/
DePaul University. (n.d.). Types of Rubrics. DePaul University Resources. Retrieved 15 July 2022, from https://resources.depaul.edu/teaching-commons/teaching-guides/feedback-grading/rubrics/Pages/types-of-rubrics.aspx#holistic
Designing Multiple-Choice Questions | Centre for Teaching Excellence. (n.d.). University of Waterloo. Retrieved 14 July 2022, from https://uwaterloo.ca/centre-for-teaching-excellence/catalogs/tip-sheets/designing-multiple-choice-questions
Erturk, S., van Tilburg, W. A. P., & Igou, E. R. (2022). Off the mark: Repetitive marking undermines essay evaluations due to boredom. Motivation and Emotion, 46, 264-275.
Frey, B. (2018). The SAGE encyclopedia of educational research, measurement, and evaluation (Vols. 1-4). SAGE Publications.
Ghalib, K., & Al-Hattami, A. A. (2015). Holistic versus Analytic Evaluation of EFL Writing: A Case Study. English Language Teaching, 8(7).
Gonzalez, J. (2014, May 1). Know Your Terms: Holistic, Analytic, and Single-Point Rubrics. Cult of Pedagogy. Retrieved 15 July 2022, from https://www.cultofpedagogy.com/holistic-analytic-single-point-rubrics/
Graham, S. (2019). Changing How Writing Is Taught. Review of Research in Education, 43(1), 277-303.
Hamp-Lyons, L. (1989). Raters respond to rhetoric in writing. In H.W. Dechert & Raupauch (Eds.), Interlingual processes, pp. 229-244, Gunter Narr.
Hillocks Jr., G. (1995). Teaching writing as reflective practice. Teachers College Press.
How effective is multiple choice… | Durrington Research School. (2019). Research Schools Network. Retrieved 14 July 2022, from https://researchschool.org.uk/durrington/news/how-effective-is-multiple-choice-quizzing-a-very-effective-b-effective-c-ineffective
Hyland, K. (2002). Activity and evaluation: Reporting practices in academic writing. In J. Flowerdew (Ed.), Academic discourse. pp.115-130, Longman.
Hyland, K. (2003). Second Language Writing. Cambridge University Press.
Jacobs, H. L., Zingraf, S. A., Wormuth, D. R., Hartfiel, V. F., & Hughey, J. B. (1981). Testing ESL composition: A practical approach. Newbury House.
Lee, S., & Schmidgall, J. (2020). The Importance of English Writing Skills in the International Workplace.
Little, J., & Bjotk, E. (2012). The Persisting Benefits of Using Multiple-Choice Tests as Learning Events. Retrieved 14 July 2022, from https://www.researchgate.net/publication/267028787_The_Persisting_Benefits_of_Using_Multiple-Choice_Tests_as_Learning_Events
Mousavi, H. S. & Kashefian-Naeeini, S. (2011). Academic writing problems of Iranian post-graduate students at national university of Malaysia (UKM). European Journal of Social Sciences, 23(4), 593-603.
Luo, S., & Zhang, X. (2011). Multiple-choice Item and Its Backwash Effect on Language Teaching in China. Theory and Practice in Language Studies, 1(4), 423-425.
Nakamura, Y. (2004). A comparison of holistic and analytic scoring methods in the assessment of writing. The Interface Between Interlanguage, Pragmatics and Assessment: Proceedings of the 3rd Annual JALT Pan-SIG Conference.
Nodoushan, M. A. S. (2007). Thinking on the write path. Training Journal, 37-40.
Nodoushan, M. A. S. (2014). Assessing Writing: A Review of the Main Trends. Studies in English Language and Education, 1(2), 119-129.
Nordquist, R. (2019). Holistic Grading (Composition). ThoughtCo. Retrieved 15 July 2022, from https://www.thoughtco.com/holistic-grading-composition-1690838
Park, T. (2003). Scoring Procedures for Assessing Writing. Columbia University Libraries, 3(1), 1-3.
Patton, M. Q. (1987). Qualitative Evaluation Methods. SAGE Publications.
Schaefer, E. (2008). Rater bias patterns in an EFL writing assessment. Language Testing, 25(4), 465-493.
Sim, M. A. (2010). Some thoughts on writing skills. Annals of Faculty of Economics, 1(1), 127-133.
Steele, C. W. (1997, Fall). Essays - well worth the effort. College Teaching, 45(4).
Stough, L. (1993). Research on Multiple-Choice Questions: Implications for Strategy Instruction. Retrieved 14 July 2022, from https://www.researchgate.net/publication/234623778_Research_on_Multiple-Choice_Questions_Implications_for_Strategy_Instruction
Tisi, J., Whitehouse, G., Maughan, S., & Burdett, N. (2013). A Review of Literature on Marking Reliability Research (Report for Ofqual). Retrieved 15 July 2022, from https://www.nfer.ac.uk/publications/mark01/mark01.pdf
Tozoglu, D., Tozoglu, M. D., Gurses, A., & Dogar, C. (2004). The Students’ Perceptions: Essay versus Muiple-choice Type Exams. Journal of Baltic Science Education, 2, 52-59.
Tuckman, B. W. (1993). “The essay test: a look at the advantages and disadvantages”. Nassp Bulletin, 77, 55, 20-26.
University of North Carolina at Chapel Hill. (n.d.). Essay Exams. UNC Writing Center. Retrieved 15 July 2022, from https://writingcenter.unc.edu/tips-and-tools/essay-exams/
Walstad, W., & Becker, W. E. (1994). Achievement Differences on Multiple-Choice and Essay Tests in Economics. College of Business Faculty Publications University of Nebraska Lincoln, 84(2), 193-196.
Weigle, S. C. (2002). Assessing writing. Cambridge University Press.
Weimer, M. (2018, February 21). Multiple-Choice Tests: Revisiting the Pros and Cons. Faculty Focus. Retrieved 17 July 2022, from https://www.facultyfocus.com/articles/educational-assessment/multiple-choice-tests-pros-cons/
Wiseman, S. (2012). A Comparison of the Performance of Analytic vs. Holistic Scoring Rubrics to Assess L2 Writing. Iranian Journal of Language Testing, 2(1), 59-92.
Zeng, X. (2018, November). A Study on the Improvement of English Writing Competence for College Students. Journal of Language Teaching and Research, 9(6), 1334-1348.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Journal of Arts and Thai Studies
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
