Volltext-Downloads (blau) und Frontdoor-Views (grau)

Supporting chatbot intent recognition with sentiment analysis

  • Developing an intelligent chatbot that can imitate human-to-human interaction has become important in recent years. For this reason, many studies have been conducted to evaluate the quality of chatbots. Furthermore, various approaches and tools, such as sentiment analysis, have been created to improve the performance of chatbots. This study examines previous research to identify the quality dimensions used to measure chatbots performance in order to develop a general chatbot assessment model that evaluates and compares chatbots quality. The developed evaluation model measures ten chatbot quality dimensions. This model is based on user experience. It requires human testers to interact with the chatbot to test its functioning and then a quantitative approach is used to collect data from user testing by conducting a survey with these testers. In this survey, they are instructed to evaluate the quality of the chatbot using a questionnaire that contains the items needed to evaluate each dimension. This study also investigates whether sentiment analysis can improve the quality of chatbots and, if so, to identify the dimensions improved with sentiment analysis. For this reason, two chatbot versions are implemented using the Rasa framework (one that cannot detect sentiment and the other that analyzes sentiment and responds accordingly). Following that, we used our evaluation model to evaluate and compare the two chatbot versions with two groups of participants by conducting a survey. In this survey, each group tested the functioning of one version. Then, both groups were instructed to use the items of the evaluation model to evaluate the version they tested. The goal of this survey was to evaluate the validity and reliability of the items used in the evaluation model to evaluate chatbots, and also to determine if sentiment analysis improved the chatbot quality by comparing survey results between the two groups. The results show that items used in the assessment model to evaluate chatbots are valid and reliable. The findings also indicate that sentiment analysis improves the chatbot’s quality. However, it improves the quality of some dimensions but not the majority of them.

Download full text files

Export metadata

Additional Services

Search Google Scholar


Author:Mohamed Alaa Eddine Cherni
Referee:Fatih Gedikli, Anne Stockem-Novo
Document Type:Master's Thesis
Year of Completion:2022
Date of final exam:2022/07/28
Release Date:2022/09/01
Page Number:107
Institutes:Fachbereich 1 - Institut Informatik
DDC class:300 Sozialwissenschaften / 330 Wirtschaft
Licence (German):License LogoNo Creative Commons