Object and Text-guided Semantics for CNN-based Activity Recognition
| Autor: | Sungmin Eum, Heesung Kwon, Clare R. Voss, Claire Bonial, Christopher Reale |
|---|---|
| Rok vydání: | 2018 |
| Předmět: |
Text corpus
FOS: Computer and information sciences Training set Computer science business.industry Computer Vision and Pattern Recognition (cs.CV) Cognitive neuroscience of visual object recognition Computer Science - Computer Vision and Pattern Recognition Multi-task learning 02 engineering and technology 010501 environmental sciences Object (computer science) Semantics computer.software_genre 01 natural sciences Convolutional neural network Activity recognition 0202 electrical engineering electronic engineering information engineering Task analysis 020201 artificial intelligence & image processing Artificial intelligence business computer Natural language processing 0105 earth and related environmental sciences |
| Zdroj: | ICASSP |
| DOI: | 10.48550/arxiv.1805.01818 |
| Popis: | Many previous methods have demonstrated the importance of considering semantically relevant objects for carrying out video-based human activity recognition, yet none of the methods have harvested the power of large text corpora to relate the objects and the activities to be transferred into learning a unified deep convolutional neural network. We present a novel activity recognition CNN which co-learns the object recognition task in an end-to-end multitask learning scheme to improve upon the baseline activity recognition performance. We further improve upon the multitask learning approach by exploiting a text-guided semantic space to select the most relevant objects with respect to the target activities. To the best of our knowledge, we are the first to investigate this approach. Comment: Submitted to ICIP 2018 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|