Question paper analysis with Natural Language Processing

Abstract

“Art of Paper Setting” is very popular terminology when it is come to education examination process. As it is an “Art”, teachers should passionate enough to prepare a better question paper which will reflect the educational objectives. There are few steps involved in the process of paper setting and analysis of the paper is the most important element among those steps as it is only indicator of the alignment of questions with intended objectives. When it comes to the analysis process, human intelligence can analyze questions more easily. But implementing similar intelligent systems with computer intelligence is a real challenge. Therefore the purpose of this research is to build a computer intelligent system which can analyze and classify questions. When it is come to classification standards, Bloom’s Taxonomy is a world recognized cognitive skills classification standard. Therefore this standard was used as the guide for the questions categorization of question papers. In the analysis phase, natural language processing techniques were used to analyze the raw text. With these techniques, first the row texts were processed and then the meaningful features of the questions such as verb similarity stem pattern similarity and stem meaning similarity were extracted. Next with machine learning techniques, a model (the brain of the system) was trained by feeding extracted question features. For the model training, several classification algorithms such as Multinomial Naive Bayes Classifier, Bernoulli Naive Bayes Classifier, Logistic Regression Classifier, Stochastic Gradient Descent Classifier, C-Support Vector Classifier and Linear Support Vector Classifier were used. Accuracy levels of each and every classification algorithms were measured with changing the size of the training data set and the optimum algorithm was selected for model training. Finally the model was trained with the optimum algorithm and that model was used to classify the unseen questions. The ultimate model was fine tuned to gain 80% classification accuracy.