Cross-modal transfer with neural word vectors for image feature learning
| Autor: | Takayuki Kurozumi, Tetsuya Kinebuchi, Go Irie, Taichi Asami, Shuhei Tarashima |
|---|---|
| Rok vydání: | 2017 |
| Předmět: |
business.industry
Computer science Feature extraction Image processing Pattern recognition 02 engineering and technology 010501 environmental sciences Machine learning computer.software_genre 01 natural sciences Convolutional neural network Kernel (image processing) Feature (computer vision) 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Word2vec Artificial intelligence business computer Image retrieval Feature learning Word (computer architecture) 0105 earth and related environmental sciences |
| Zdroj: | ICASSP |
| DOI: | 10.1109/icassp.2017.7952690 |
| Popis: | Neural word vector (NWV) such as word2vec is a powerful text representation tool that can encode extensive semantic information into compact vectors. This ability poses an interesting question in relation to image processing research - Can we learn better semantic image features from NWVs? We empirically explore this question in the context of semantic content-based image retrieval (CBIR). In this paper, we consider cross-modal transfer learning (CMT) to improve initial convolutional neural network (CNN) image features by using NWVs. We first show that NWVs can improve semantic CBIR performance compared to classical word vectors, even if it is with simple CMT models, i.e., canonical correlation analysis (CCA). Next, inspired by a characteristic property of NWVs, we propose a new CMT model and demonstrate that it can improve CBIR performance even further. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|