Deep canonical correlation analysis with progressive and hypergraph learning for cross-modal retrieval
| Autor: | Anni Cai, Leiquan Wang, Jie Shao, Fei Su, Zhicheng Zhao |
|---|---|
| Rok vydání: | 2016 |
| Předmět: |
Hypergraph
Image View business.industry Cognitive Neuroscience Pattern recognition 02 engineering and technology 010501 environmental sciences Similarity measure Semantics computer.software_genre 01 natural sciences Computer Science Applications Image (mathematics) Artificial Intelligence 0202 electrical engineering electronic engineering information engineering Embedding 020201 artificial intelligence & image processing Relevance (information retrieval) Artificial intelligence Data mining business Canonical correlation computer 0105 earth and related environmental sciences Mathematics |
| Zdroj: | Neurocomputing. 214:618-628 |
| ISSN: | 0925-2312 |
| DOI: | 10.1016/j.neucom.2016.06.047 |
| Popis: | This paper deals with the problem of modeling Internet images and associated texts for cross-modal retrieval such as text-to-image retrieval and image-to-text retrieval. We start with deep canonical correlation analysis (DCCA), a deep approach for mapping text and image pairs into a common latent space. We first propose a novel progressive framework and embed DCCA in it. In our progressive framework, a linear projection loss layer is inserted before the nonlinear hidden layers of a deep network. The training of linear projection and the training of nonlinear layers are combined to ensure that the linear projection is well matched with the nonlinear processing stages and good representations of the input raw data are learned at the output of the network. Then we introduce a hypergraph semantic embedding (HSE) method, which extracts latent semantics from texts, into DCCA to regularize the latent space learned by image view and text view. In addition, a search-based similarity measure is proposed to score relevance of image-text pairs. Based on the above ideas, we propose a model, called DCCA-PHS, for cross-modal retrieval. Experiments on three publicly available data sets show that DCCA-PHS is effective and efficient, and achieves state-of-the-art performance for unsupervised scenario. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Full Text from ScienceDirect