A point‐image fusion network for event‐based frame interpolation

Autor: Chushu Zhang, Wei An, Ye Zhang, Miao Li
Jazyk: angličtina
Rok vydání: 2024
Předmět:
Zdroj: IET Computer Vision, Vol 18, Iss 4, Pp 439-447 (2024)
Druh dokumentu: article
ISSN: 1751-9640
1751-9632
DOI: 10.1049/cvi2.12220
Popis: Abstract Temporal information in event streams plays a critical role in event‐based video frame interpolation as it provides temporal context cues complementary to images. Most previous event‐based methods first transform the unstructured event data to structured data formats through voxelisation, and then employ advanced CNNs to extract temporal information. However, voxelisation inevitably leads to information loss, and processing the sparse voxels introduces severe computation redundancy. To address these limitations, this study proposes a point‐image fusion network (PIFNet). In our PIFNet, rich temporal information from the events can be directly extracted at the point level. Then, a fusion module is designed to fuse complementary cues from both points and images for frame interpolation. Extensive experiments on both synthetic and real datasets demonstrate that our PIFNet achieves state‐of‐the‐art performance with high efficiency.
Databáze: Directory of Open Access Journals
Externí odkaz: