| Popis: |
The development and testing of a virtual assistant (VA) designed to enhance information retrieval and support in an academic environment are presented in this paper, with the Retrieval-Augmented Generation (RAG) approach being utilized alongside Google’s VERTEX AI Palm-2 model. A novel integration of RAG with contextual learning is introduced in this study, specifically for applications in university contact centers, where accuracy and relevance are considered paramount. The effectiveness of the VA was evaluated through user testing, focusing on two primary hypotheses: first, that the VA can achieve accurate interpretation and response to queries with context-based information, and second, that the VA minimizes potential harm from erroneous responses. In total, 187 participants were involved in the testing, and a diverse set of inquiries was utilized, resulting in 561 query–response interactions that were analyzed. It was shown that contextual data significantly reduced hallucinations and increased response accuracy, thereby underscoring the value of the RAG method in applications requiring high levels of specificity. Furthermore, the study provides empirical insights into the impact of AI-generated hallucinations and response inconsistencies, particularly about structured or procedural data. A framework for mitigating these challenges in future implementations is also offered. The scalability and adaptability of the RAG method in specialized academic contexts are demonstrated in this work, with broader implications for integrating AI-driven VAs across educational and professional domains being highlighted. |
