Enhancing usability in learning management systems: Exploring future prospects with virtual assistants

Abstract

In Sri Lankan universities, the Learning Management System (LMS) plays a major role in online program content development and management. Amidst the COVID-19 crisis, Sri Lankan university students had to rely heavily on LMS for all academic tasks. The academic workload, a prominent stress factor for university students, coupled with potential shortcomings in the Learning Management System (LMS), can result in diminished engagement, heightened stress levels, and adverse effects on the overall educational experience. Therefore, the main objective of this research is to evaluate the usability of existing learning management systems. The second objective is to discuss the characteristics of virtual assistants that can cater to Sri Lankan university students. The study takes a quantitative research approach, as its primary data is collected through an online questionnaire created from Google Forms and answered by 160 undergraduates from the Faculty of Science at the University of Kelaniya. Likert-Scale questions measured the frequency of engagement activities with the LMS, agreement with usability factors, and importance of features for a virtual assistant. The convenience sampling method was chosen due to the accessible participant pool and as a preliminary step for a future comprehensive study. The data was validated and analysed using SPSS software. Descriptive analysis and hypothesis testing with a single sample-sign test were carried out to observe the statistical significance of active user engagement in the LMS and the usability factors for the LMS. Additionally, the analysis used a machine learning algorithm called Random Forest Classifier in Python to identify the importance of characteristics and challenges for virtual assistants. This study considered eight features that were used as independent variables while using the preference for a virtual assistant as the dependent variable for the model. Based on the data analysis, users generally engage actively with the LMS. However, some features like forums, discussions, video conferencing, and chat need to be more utilised due to limited user awareness. Students express positive views about the current LMS's accessibility and satisfaction levels, which meet acceptable Human-Computer Interaction (HCI) standards. The importance of features found by the machine learning model emphasizes the importance of simplicity and relatability for effective virtual assistants. Future research could benefit from a stratified random sampling method that provides better coverage of the population, including students from all faculties, improves the accuracy of data analysis and the machine learning model for a broader representation, and extends insights into learners’ experiences.