Zobrazeno 1 - 10
of 1 559
pro vyhledávání: '"Noise Contrastive Estimation"'
This paper studies a family of estimators based on noise-contrastive estimation (NCE) for learning unnormalized distributions. The main contribution of this work is to provide a unified perspective on various methods for learning unnormalized distrib
Externí odkaz:
http://arxiv.org/abs/2409.18209
Autor:
Hugger, Johannes, Uhlmann, Virginie
Soft targets combined with the cross-entropy loss have shown to improve generalization performance of deep neural networks on supervised classification tasks. The standard cross-entropy loss however assumes data to be categorically distributed, which
Externí odkaz:
http://arxiv.org/abs/2404.14076
Noise-contrastive estimation (NCE) is a popular method for estimating unnormalised probabilistic models, such as energy-based models, which are effective for modelling complex data distributions. Unlike classical maximum likelihood (ML) estimation th
Externí odkaz:
http://arxiv.org/abs/2402.16688
Tactics, Techniques and Procedures (TTPs) represent sophisticated attack patterns in the cybersecurity domain, described encyclopedically in textual knowledge bases. Identifying TTPs in cybersecurity writing, often called TTP mapping, is an important
Externí odkaz:
http://arxiv.org/abs/2401.10337
Recent research has developed several Monte Carlo methods for estimating the normalization constant (partition function) based on the idea of annealing. This means sampling successively from a path of distributions that interpolate between a tractabl
Externí odkaz:
http://arxiv.org/abs/2310.03902
Autor:
Feeney, Patrick, Hughes, Michael C.
The information noise-contrastive estimation (InfoNCE) loss function provides the basis of many self-supervised deep learning methods due to its strong empirical results and theoretic motivation. Previous work suggests a supervised contrastive (SupCo
Externí odkaz:
http://arxiv.org/abs/2309.14277
A crucial design decision for any robot learning pipeline is the choice of policy representation: what type of model should be used to generate the next set of robot actions? Owing to the inherent multi-modal nature of many robotic tasks, combined wi
Externí odkaz:
http://arxiv.org/abs/2309.05803
Autor:
Zach, Christopher
By using the underlying theory of proper scoring rules, we design a family of noise-contrastive estimation (NCE) methods that are tractable for latent variable models. Both terms in the underlying NCE loss, the one using data samples and the one usin
Externí odkaz:
http://arxiv.org/abs/2304.02473
Autor:
Xu, Nathaniel
Training energy-based models (EBMs) with noise-contrastive estimation (NCE) is theoretically feasible but practically challenging. Effective learning requires the noise distribution to be approximately similar to the target distribution, especially i
Externí odkaz:
http://arxiv.org/abs/2211.02650
Self-supervised learning is an increasingly popular approach to unsupervised learning, achieving state-of-the-art results. A prevalent approach consists in contrasting data points and noise points within a classification task: this requires a good no
Externí odkaz:
http://arxiv.org/abs/2301.09696