A Comprehensive Methodology for Evaluating Conversation-Based Interfaces to Relational Databases (C-BIRDs)
| Autor: | Fathi Gasir, Majdi Owda, Amani Yousef Owda |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
Information retrieval
Process (engineering) Relational database business.industry Computer science media_common.quotation_subject Interface (computing) 05 social sciences Usability law.invention Task (project management) law 0502 economics and business CLARITY 050211 marketing 0501 psychology and cognitive sciences Conversation business 050107 human factors Natural language media_common |
| Zdroj: | Advances in Intelligent Systems and Computing ISBN: 9783030551865 IntelliSys (2) |
| DOI: | 10.1007/978-3-030-55187-2_17 |
| Popis: | Evaluation can be defined as a process of determining the significance of a research output. This is usually done by devising a well-structured study on this output using one or more evaluation measures in which a careful inspection is performed. This paper presents a review of evaluation techniques for Conversational Agents (CAs) and Natural Language Interfaces to Databases (NLIDBs). It then introduces the developed customized evaluation methodology for Conversation-Based Interface to Relational Databases (C-BIRDs). The evaluation methodology created has been divided into two groups of measures. The first is based on quantitative measures, including two measures: task success and dialogue length. The second group is based on a number of qualitative measures, including: prototype ease of use, naturalness of system responses, positive/negative emotion, appearance, text on screen, organization of information, and error message clarity. Then an elaboration is carried out on the devised methodology by adding a discussion and recommendations on the sample size, the experimental setup and the scaling in order to provide a comprehensive evaluation methodology for C-BIRDs. In conclusion the evaluation methodology created is better way for identifying the strengths and weaknesses of C-BIRDs in comparison to the usage of single measure evaluations. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|