Supervising topic models with Gaussian processes
| Autor: | Melih Kandemir, Reyyan Yeniterzi, Taygun Kekec |
|---|---|
| Přispěvatelé: | Özyeğin University, Kandemir, Melih, Yeniterzi, Reyyan |
| Jazyk: | angličtina |
| Rok vydání: | 2018 |
| Předmět: |
Topic model
Computer science Inference Latent Dirichlet allocation Gaussian processes Linear classifier 02 engineering and technology Latent variable 010501 environmental sciences Nonparametric Bayesian inference Machine learning computer.software_genre 01 natural sciences Data modeling symbols.namesake Artificial Intelligence 0202 electrical engineering electronic engineering information engineering Gaussian process 0105 earth and related environmental sciences business.industry Dimensionality reduction Supervised topic models ComputingMethodologies_PATTERNRECOGNITION Signal Processing symbols 020201 artificial intelligence & image processing Computer Vision and Pattern Recognition Artificial intelligence Language model business Variational inference computer Software Curse of dimensionality |
| Popis: | Due to copyright restrictions, the access to the full text of this article is only available via subscription. Topic modeling is a powerful approach for modeling data represented as high-dimensional histograms. While the high dimensionality of such input data is extremely beneficial in unsupervised applications including language modeling and text data exploration, it introduces difficulties in cases where class information is available to boost up prediction performance. Feeding such input directly to a classifier suffers from the curse of dimensionality. Performing dimensionality reduction and classification disjointly, on the other hand, cannot enjoy optimal performance due to information loss in the gap between these two steps unaware of each other. Existing supervised topic models introduced as a remedy to such scenarios have thus far incorporated only linear classifiers in order to keep inference tractable, causing a dramatical sacrifice from expressive power. In this paper, we propose the first Bayesian construction to perform topic modeling and non-linear classification jointly. We use the well-known Latent Dirichlet Allocation (LDA) for topic modeling and sparse Gaussian processes for non-linear classification. We combine these two components by a latent variable encoding the empirical topic distribution of each document in the corpus. We achieve a novel variational inference scheme by adapting ideas from the newly emerging deep Gaussian processes into the realm of topic modeling. We demonstrate that our model outperforms other existing approaches such as: (i) disjoint LDA and non-linear classification, (ii) joint LDA and linear classification, (iii) joint non-LDA linear subspace modeling and linear classification, and (iv) non-linear classification without topic modeling, in three benchmark data sets from two real-world applications: text categorization and image tagging. Dutch Organization for Scientific Research (NWO) |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|