Hierarchical mixtures of Unigram models for short text clustering: the role of Beta-Liouville priors

Autor:	Bilancia, Massimo, Magro, Samuele
Rok vydání:	2024
Předmět:	Statistics - Machine Learning Computer Science - Machine Learning Statistics - Computation
Druh dokumentu:	Working Paper
Popis:	This paper presents a variant of the Multinomial mixture model tailored for the unsupervised classification of short text data. Traditionally, the Multinomial probability vector in this hierarchical model is assigned a Dirichlet prior distribution. Here, however, we explore an alternative prior - the Beta-Liouville distribution - which offers a more flexible correlation structure than the Dirichlet. We examine the theoretical properties of the Beta-Liouville distribution, focusing on its conjugacy with the Multinomial likelihood. This property enables the derivation of update equations for a CAVI (Coordinate Ascent Variational Inference) variational algorithm, facilitating the approximate posterior estimation of model parameters. Additionally, we propose a stochastic variant of the CAVI algorithm that enhances scalability. The paper concludes with data examples that demonstrate effective strategies for setting the Beta-Liouville hyperparameters. Comment: 47 pages, 4 figures. Submitted
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2410.21862 Zobrazit plný text záznamu View this record from Arxiv