Adversarial Learning-Based Semantic Correlation Representation for Cross-Modal Retrieval
| Autor: | Xinpan Yuan, Shichao Zhang, Jiayu Song, Yang Wang, Chengyuan Zhang, Xiaofeng Zhu, Lei Zhu |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
Computer science
business.industry Machine learning computer.software_genre Computer Science Applications Correlation Modal Hardware and Architecture Signal Processing Media Technology Artificial intelligence Canonical correlation Intramodal dispersion business Feature learning computer Classifier (UML) Software Subspace topology Semantic gap |
| Zdroj: | IEEE MultiMedia. 27:79-90 |
| ISSN: | 1941-0166 1070-986X |
| DOI: | 10.1109/mmul.2020.3015764 |
| Popis: | Cross-modal retrieval has become a hot issue in past years. Many existing works pay attentions on correlation learning to generate a common subspace for cross-modal correlation measurement, and others use adversarial learning technique to abate the heterogeneity of multimodal data. However, very few works combine correlation learning and adversarial learning to bridge the intermodal semantic gap and diminish cross-modal heterogeneity. This article proposes a novel cross-modal retrieval method, named Adversarial Learning based Semantic COrrelation Representation (ALSCOR), which is an end-to-end framework to integrate cross-modal representation learning, correlation learning, and adversarial. Canonical correlation analysis model, combined with VisNet and TxtNet, is proposed to capture cross-modal nonlinear correlation. Besides, intramodal classifier and modality classifier are used to learn intramodal discrimination and minimize the intermodal heterogeneity. Comprehensive experiments are conducted on three benchmark datasets. The results demonstrate that the proposed ALSCOR has better performance than the state of the arts. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|