Feature Re-Learning with Data Augmentation for Video Relevance Prediction
Autor: | Leimin Zhang, Chaoxi Xu, Xirong Li, Jianfeng Dong, Xun Wang, Gang Yang |
---|---|
Jazyk: | angličtina |
Rok vydání: | 2020 |
Předmět: |
FOS: Computer and information sciences
Computer science Computer Vision and Pattern Recognition (cs.CV) Feature extraction Computer Science - Computer Vision and Pattern Recognition Context (language use) 02 engineering and technology Machine learning computer.software_genre Convolutional neural network Ranking (information retrieval) Computer Science - Information Retrieval 020204 information systems 0202 electrical engineering electronic engineering information engineering Relevance (information retrieval) business.industry Computer Science Applications Visualization Computational Theory and Mathematics Feature (computer vision) Affine transformation Artificial intelligence business computer Information Retrieval (cs.IR) Information Systems |
Popis: | Predicting the relevance between two given videos with respect to their visual content is a key component for content-based video recommendation and retrieval. Thanks to the increasing availability of pre-trained image and video convolutional neural network models, deep visual features are widely used for video content representation. However, as how two videos are relevant is task-dependent, such off-the-shelf features are not always optimal for all tasks. Moreover, due to varied concerns including copyright, privacy and security, one might have access to only pre-computed video features rather than original videos. We propose in this paper feature re-learning for improving video relevance prediction, with no need of revisiting the original video content. In particular, re-learning is realized by projecting a given deep feature into a new space by an affine transformation. We optimize the re-learning process by a novel negative-enhanced triplet ranking loss. In order to generate more training data, we propose a new data augmentation strategy which works directly on frame-level and video-level features. Extensive experiments in the context of the Hulu Content-based Video Relevance Prediction Challenge 2018 justify the effectiveness of the proposed method and its state-of-the-art performance for content-based video relevance prediction. accepted by IEEE Transactions on Knowledge and Data Engineering (TKDE) |
Databáze: | OpenAIRE |
Externí odkaz: |