Multimodal input fusion for companion technology
| Autor: | Schüssel, Felix |
|---|---|
| Přispěvatelé: | Weber, Michael, Minker, Wolfgang |
| Jazyk: | angličtina |
| Rok vydání: | 2018 |
| Předmět: |
Benutzeroberfl��che
Companion technology Evidential reasoning Pattern recognition systems User interfaces (Computer systems) Multimodal input fusion Human-computer interaction Multimodal interaction Individual user adaptation Benutzeroberfläche Interactive multimedia ddc:004 DDC 004 / Data processing & computer science Datenfusion |
| DOI: | 10.18725/oparu-5660 |
| Popis: | Today���s computer systems provide a vast variety of functionalities but at the same time lack an intelligent and natural user interface. The idea of companion technology is to complement a system with an intelligent interface that adapts itself to the individual user and its context of use. One important task of the interface is to take inputs of all available devices and determine what the user has intended to do. This process is called multimodal input fusion. Being a topic of research for over 30 years now, it has still failed broad employment. The question is: What are the known approaches lacking, especially regarding their use as companion technology? From the state of the art, two major weaknesses are identified: the insufficient handling of uncertainties considering that a variety of sensors provide inputs, and no means of adaptation to the individual user to reduce error rates and increase the overall robustness. This work���s contribution is a fully implemented multimodal input fusion that employs evidential reasoning and individual user adaptation to provide a comprehensive and robust decision on all available inputs. It is not about progressing specific input technologies and devices, but to make the most of what they provide on a generic level. Based on an existing idea of applying evidential reasoning, an input fusion is developed that provides functionalities of combination, disambiguation, reinforcement and conflict detection. A theoretical evaluation proves the benefits of the formalism compared to the usually employed n-best lists. Additional benefits arise when the realized component is embedded into a companion system, closely cooperating with a multimodal fission component in charge of arbitrating the available output modalities. Two perspectives are taken regarding individual user adaptation. First, the user���s decision to adopt a specific input modality is investigated. A user study identifies yet unknown predictors, like personality traits and technology affinity. Though useful in system design, the complexity of influencing factors hinders direct application for input fusion. Taking a system perspective, however, temporal data provided by input sensors is known to bear important information and research promised its usefulness for some time. Nevertheless, no practical application has been presented to date. In this work, an original approach to individual user adaptation is developed based on the notion of temporal interaction histories, to complement and extend the input fusion with individually adaptive error recognition capabilities. An evaluation study proves a significant decrease of error rates compared to previous ideas suggested in literature. As a comprehensive concept of individually adaptive input fusion, the presented work can not only be used within companion systems, but in any type of system as a technology for realizing robust multimodal inputs. Thus, it could be the basis for multimodal interfaces to reach a maturity at which they in fact advance HCI for end users. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|