An Initial Study on Understanding the Effect of Question Surface Features on Students' Responses

Abstract

Assessment is one of the key feedback mechanisms in measuring the effectiveness of the teaching and learning process of science education. One of the challenges in evaluating through the assessments is that students often focus on surface features of the questions such as the length, vocabulary, or the phrasing, rather than on underlying scientific principles of the question. According to educational theories, the wording of the questions has a substantial impact on the students’ performance during the science examinations. However, in the current science education context, there is only a limited number of research studies available which provides an insight into the relationship between students’ performance and the question features. The objective of the current study is to investigate the effect of the surface features of the questions on students’ performance. Herein, we have investigated how wording of an examination question could affect students’ performance. The study was conducted as a part of the Chemistry for Technology course (sample size =86) at the Faculty of Computing and Technology, University of Kelaniya, Sri Lanka. At the final examination, two different types of constructed-response questions were given to students. One was a Direct question which included solving a question using a basic chemistry concept and simple arithmetic calculations. Second version (Wordy question); included the same data as in the Direct version but with a related scenario. In the Wordy question, students had to go through several sentences to pick relevant data to solve the problem. It is expected that, the scenario given in the Wordy version could lead to a deeper understanding of the question and thereby result in better students’ performance. Marks obtained for the two versions were averaged and compared to investigate whether there is any significance of the wording towards the performance of students. Average mark for the Wordy question yielded to be 43% while that of the Direct question was 63%. Moreover, the attempt rate (no of student who had attempted) was significantly low as 39% for the Wordy version of the question. According to the performance of the students it was clear that the students meet a considerable difficulty in the understanding the Wordy questions. It can be concluded that the use of items such as syntactically complex sentences in examinations could decrease the performance of the students. Moreover, significantly lower attempt rate observed in the case of the Wordy questions suggests that the students tend to skip lengthy questions even without trying to identify the basic scientific principle lying behind. Hence it can be concluded that the surface features such as the length, type of words can play a key role in students’ performance at the examinations. Currently, further studies are being conducted to gain more insight into this.