Abstract:
The educational advising process most often consists of repeated queries related to institutional policies, academic progression, career pathways, and industry placements. In the higher education system, this procedure is usually initiated by various learners directing the same questions toward a limited number of advisors, which results in the advising process being reliant on the availability of the individuals and their hectic work schedules. Hence, this study introduces a feasible mechanism of a chatbot-based advising system to bridge this identified gap between learner requirements and resource availability by automating the prescriptive advising process beyond the traditionally available methods. Most existing systems provide a rule-based approach with limited pre-defined intent-response structures, resulting in several identified usability shortcomings. In response, this study utilizes an opensource Large Language Model (LLM) combined with a custom knowledge base to address critical aspects needed for a chatbot-based advising system, such as personalization, conversational memory, and ease of maintenance. The system is built around three major components: an admin panel for advisors, a conversational user interface (CUI) for learners, and an easy-to-maintain custom knowledge base. It uses the traditional form of information distributed to students through handbooks, guidelines, and course outlines to create a custom knowledge base which is then utilized to answer the user's queries through a semantic similarity algorithm. This work contributes (1) a prototype of a chatbot-based advising system for higher educational institutes in Sri Lanka, (2) the application of Large Language Models, vector databases, and semantic similarity in the design of the system, and (3) the results of evaluating the system's functionality and performance metrics through comprehensive test cases and a comparative analysis against the existing approaches. As identified, the proposed system showcased a response accuracy rate of 89% proving that this novel approach of a component-based architecture excels in performance when compared to similar approaches.
