Repository with supplementary material for the paper entitled Unveiling Quality in Chatbot Conversations: Quantitative Analysis of Chatbot Requirements.
Abstract: As conversational assistants and natural language interfaces proliferate, the demand for a precise understanding of quality software requirements for chatbots becomes increasingly critical. In this work, we adopted a quantitative methodology, scrutinizing a dataset composed of conversational requirements from a diverse range of agile projects for chatbot development, and identified meaningful patterns in the language and structure utilized. Our investigation led to significant findings, revealing the importance of structured documentation, conversation flow, and user interaction in the development of chatbots, with the most desired quality attributes being capability, naturalness, straightforwardness, and clarity. In addition, a significant emphasis was placed on feature development and meeting acceptance criteria. The research also illuminated the iterative nature of chatbot development, with a recurrent presence of verbs related to improvement or refactoring. While less pronounced, the roles of documentation and testing in ensuring chatbot quality and effectiveness were also noted. This work provides valuable insights into chatbot requirements management and the significance of quality attributes in chatbot development.
Files:
- extract_issues.py: script to mine issues from the repositories;
- issues_chatbot.csv: issues retrieved by extract_issues.py;
- filter_issues.py: script to filter issues with conversational requirements;
- issues_conversacionais.csv: issues retrieved by filter_issues.py;
- analyzing-chatbot-issues.ipynb: notebook with data analysis;