Feature fusion over hyperbolic graph convolution networks for video summarisation

Autor: GuangLi Wu, ShengTao Wang, ShiPeng Xu
Jazyk: angličtina
Rok vydání: 2024
Předmět:
Zdroj: IET Computer Vision, Vol 18, Iss 1, Pp 150-164 (2024)
Druh dokumentu: article
ISSN: 1751-9640
1751-9632
DOI: 10.1049/cvi2.12232
Popis: Abstract A novel video summarisation method called the Hyperbolic Graph Convolutional Network (HVSN) is proposed, which addresses the challenges of summarising edited videos and capturing the semantic consistency of video shots at different time points. Unlike existing methods that use linear video sequences as input, HVSN leverages Hyperbolic Graph Convolutional Networks (HGCNs) and an adaptive graph convolutional adjacency matrix network to learn and aggregate features from video shots. Moreover, a feature fusion mechanism based on the attention mechanism is employed to facilitate cross‐module feature fusion. To evaluate the performance of the proposed method, experiments are conducted on two benchmark datasets, TVSum and SumMe. The results demonstrate that HVSN achieves state‐of‐the‐art performance, with F1‐scores of 62.04% and 50.26% on TVSum and SumMe, respectively. The use of HGCNs enables the model to better capture the complex spatial structures of video shots, and thus contributes to the improved performance of video summarisation.
Databáze: Directory of Open Access Journals
Externí odkaz: