Figurines, a multimodal framework for tangible storytelling
| Autor: | Maxime Portaz, Dominique Vaufreydaz, Laurence Boissieux, Maxime Garcia, Rémi Ronfard, Marie-Paule Cani, Adela Barbulescu, Antoine Begault |
|---|---|
| Přispěvatelé: | Interaction située avec les objets et environnements intelligents (PERVASIVE), Inria Grenoble - Rhône-Alpes, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire d'Informatique de Grenoble (LIG ), Institut polytechnique de Grenoble - Grenoble Institute of Technology (Grenoble INP )-Centre National de la Recherche Scientifique (CNRS)-Université Grenoble Alpes [2016-2019] (UGA [2016-2019])-Institut polytechnique de Grenoble - Grenoble Institute of Technology (Grenoble INP )-Centre National de la Recherche Scientifique (CNRS)-Université Grenoble Alpes [2016-2019] (UGA [2016-2019]), Laboratoire d'Informatique de Grenoble (LIG ), Institut polytechnique de Grenoble - Grenoble Institute of Technology (Grenoble INP )-Centre National de la Recherche Scientifique (CNRS)-Université Grenoble Alpes [2016-2019] (UGA [2016-2019]), Intuitive Modeling and Animation for Interactive Graphics & Narrative Environments (IMAGINE ), Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Institut polytechnique de Grenoble - Grenoble Institute of Technology (Grenoble INP )-Laboratoire Jean Kuntzmann (LJK ), Institut polytechnique de Grenoble - Grenoble Institute of Technology (Grenoble INP )-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-Université Grenoble Alpes [2016-2019] (UGA [2016-2019])-Centre National de la Recherche Scientifique (CNRS)-Université Grenoble Alpes [2016-2019] (UGA [2016-2019]), Service Expérimentation et Développement (SED [Grenoble]), Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria), Laboratoire d'informatique de l'École polytechnique [Palaiseau] (LIX), École polytechnique (X)-Centre National de la Recherche Scientifique (CNRS), SED [Grenoble], Centre National de la Recherche Scientifique (CNRS)-École polytechnique (X) |
| Jazyk: | angličtina |
| Rok vydání: | 2017 |
| Předmět: |
Engineering
[SCCO.COMP]Cognitive science/Computer science 02 engineering and technology Representation (arts) RGB-D sensor computer.software_genre Session (web analytics) Motion (physics) 0202 electrical engineering electronic engineering information engineering Multimodal data fusion 0501 psychology and cognitive sciences Narrative 050107 human factors IMU sensor Multimedia business.industry Puppetry Interpretation (philosophy) 05 social sciences ComputingMilieux_PERSONALCOMPUTING [INFO.INFO-CV]Computer Science [cs]/Computer Vision and Pattern Recognition [cs.CV] 020207 software engineering Storytelling business computer Gesture |
| Zdroj: | WOCCI 2017-6th Workshop on Child Computer Interaction at ICMI 2017-19th ACM International Conference on Multi-modal Interaction WOCCI 2017-6th Workshop on Child Computer Interaction at ICMI 2017-19th ACM International Conference on Multi-modal Interaction, Nov 2017, Glasgow, United Kingdom. pp.52-57, ⟨10.21437/WOCCI.2017-9⟩ WOCCI |
| DOI: | 10.21437/WOCCI.2017-9⟩ |
| Popis: | Author version; International audience; This paper presents Figurines, an offline framework for narrative creation with tangible objects, designed to record storytelling sessions with children, teenagers or adults. This framework uses tangible diegetic objects to record a free narrative from up to two storytellers and construct a fully annotated representation of the story. This representation is composed of the 3D position and orientation of the figurines, the position of decor elements and interpretation of the storytellers' actions (facial expression, gestures and voice). While maintaining the playful dimension of the storytelling session, the system must tackle the challenge of recovering the free-form motion of the figurines and the storytellers in uncontrolled environments. To do so, we record the storytelling session using a hybrid setup with two RGB-D sensors and figurines augmented with IMU sensors. The first RGB-D sensor completes IMU information in order to identify figurines and tracks them as well as decor elements. It also tracks the storytellers jointly with the second RGB-D sensor. The framework has been used to record preliminary experiments to validate interest of our approach. These experiments evaluate figurine following and combination of motion and storyteller's voice, gesture and facial expressions. In a make-believe game, this story representation was re-targeted on virtual characters to produce an animated version of the story. The final goal of the Figurines framework is to enhance our understanding of the creative processes at work during immersive storytelling. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|