Online Bayesian Sparse Learning with Spike and Slab Priors
| Autor: | Jennifer Neville, Kai Zhang, Shandian Zhe, Shikai Fang, Kuang-Chih Lee |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
Artificial neural network
Computer science business.industry Bayesian probability Feature extraction 02 engineering and technology Machine learning computer.software_genre 01 natural sciences Regularization (mathematics) 010104 statistics & probability 020204 information systems Prior probability 0202 electrical engineering electronic engineering information engineering Feature (machine learning) Artificial intelligence 0101 mathematics Online algorithm business computer Shrinkage Interpretability |
| Zdroj: | ICDM |
| DOI: | 10.1109/icdm50108.2020.00023 |
| Popis: | In many applications, a parsimonious model is often preferred for better interpretability and predictive performance. Online algorithms have been studied extensively for building such models in big data and fast evolving environments, with a prominent example, FTRL-proximal [1]. However, existing methods typically do not provide confidence levels, and with the usage of $L_{1}$ regularization, the model estimation can be undermined by the uniform shrinkage on both relevant and irrelevant features. To address these issues, we developed OLSS, a Bayesian online sparse learning algorithm based on the spike-and-slab prior. OLSS achieves the same scalability as FTRL-proximal, but realizes appealing selective shrinkage and produces rich uncertainty information, such as posterior inclusion probabilities and feature weight variances. On the tasks of text classification and click-through-rate (CTR) prediction for Yahoo!'s display and search advertisement platforms, OLSS often demonstrates superior predictive performance to the state-of-the-art methods in industry, including Vowpal Wabbit [2] and FTRL-proximal. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|