Modeling of Human Visual Attention in Multiparty Open-World Dialogues
Autor: | Giampiero Salvi, Kalin Stefanov, Hedvig Kjellström, Jonas Beskow, Dimosthenis Kontogiorgos |
---|---|
Rok vydání: | 2019 |
Předmět: |
Computer science
Heuristic Orientation (computer vision) Speech recognition media_common.quotation_subject 05 social sciences Context (language use) 02 engineering and technology Gaze Task (project management) Human-Computer Interaction Artificial Intelligence 0202 electrical engineering electronic engineering information engineering Feedforward neural network 020201 artificial intelligence & image processing 0501 psychology and cognitive sciences Active listening Function (engineering) 050107 human factors media_common |
Zdroj: | ACM Transactions on Human-Robot Interaction. 8:1-21 |
ISSN: | 2573-9522 |
Popis: | This study proposes, develops, and evaluates methods for modeling the eye-gaze direction and head orientation of a person in multiparty open-world dialogues, as a function of low-level communicative signals generated by his/hers interlocutors. These signals include speech activity, eye-gaze direction, and head orientation, all of which can be estimated in real time during the interaction. By utilizing these signals and novel data representations suitable for the task and context, the developed methods can generate plausible candidate gaze targets in real time. The methods are based on Feedforward Neural Networks and Long Short-Term Memory Networks. The proposed methods are developed using several hours of unrestricted interaction data and their performance is compared with a heuristic baseline method. The study offers an extensive evaluation of the proposed methods that investigates the contribution of different predictors to the accurate generation of candidate gaze targets. The results show that the methods can accurately generate candidate gaze targets when the person being modeled is in a listening state. However, when the person being modeled is in a speaking state, the proposed methods yield significantly lower performance. |
Databáze: | OpenAIRE |
Externí odkaz: |