SoGAR: Self-supervised Spatiotemporal Attention-based Social Group Activity Recognition

Autor:	Chappa, Naga VS Raviteja, Nguyen, Pha, Nelson, Alexander H, Seo, Han-Seok, Li, Xin, Dobbs, Page Daniel, Luu, Khoa
Rok vydání:	2023
Předmět:	Computer Science - Computer Vision and Pattern Recognition
Druh dokumentu:	Working Paper
Popis:	This paper introduces a novel approach to Social Group Activity Recognition (SoGAR) using Self-supervised Transformers network that can effectively utilize unlabeled video data. To extract spatio-temporal information, we created local and global views with varying frame rates. Our self-supervised objective ensures that features extracted from contrasting views of the same video were consistent across spatio-temporal domains. Our proposed approach is efficient in using transformer-based encoders to alleviate the weakly supervised setting of group activity recognition. By leveraging the benefits of transformer models, our approach can model long-term relationships along spatio-temporal dimensions. Our proposed SoGAR method achieved state-of-the-art results on three group activity recognition benchmarks, namely JRDB-PAR, NBA, and Volleyball datasets, surpassing the current numbers in terms of F1-score, MCA, and MPCA metrics. Comment: Under review for PR journal; 32 pages, 7 figures. arXiv admin note: text overlap with arXiv:2303.12149
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2305.06310 Zobrazit plný text záznamu View this record from Arxiv