A precise bare simulation approach to the minimization of some distances. Foundations
| Autor: | Michel Broniatowski, Wolfgang Stummer |
|---|---|
| Přispěvatelé: | Laboratoire de Probabilités, Statistiques et Modélisations (LPSM (UMR_8001)), Université Paris Diderot - Paris 7 (UPD7)-Sorbonne Université (SU)-Centre National de la Recherche Scientifique (CNRS), Friedrich-Alexander Universität Erlangen-Nürnberg (FAU) |
| Rok vydání: | 2021 |
| Předmět: |
FOS: Computer and information sciences
Hellinger distance Euclidean norms Computer Science - Information Theory Information Theory (cs.IT) Ali-Silvey-Morimoto type relative entropy Shannon entropy Tsallis (cross) entropies generalized maximum entropy method Library and Information Sciences Jensen-Shannon divergence/distance Computer Science Applications Cressie-Read measures fuzzy divergences Kullback-Leibler information distance importance sampling power divergences Renyi entropies [MATH.MATH-ST]Mathematics [math]/Statistics [math.ST] alpha-divergences Renyi divergences Bhattacharyya coefficient Bhattacharyya distance Information Systems |
| DOI: | 10.48550/arxiv.2107.01693 |
| Popis: | In information theory -- as well as in the adjacent fields of statistics, machine learning, artificial intelligence, signal processing and pattern recognition -- many flexibilizations of the omnipresent Kullback-Leibler information distance (relative entropy) and of the closely related Shannon entropy have become frequently used tools. To tackle corresponding constrained minimization (respectively maximization) problems by a newly developed dimension-free bare (pure) simulation method, is the main goal of this paper. Almost no assumptions (like convexity) on the set of constraints are needed, within our discrete setup of arbitrary dimension, and our method is precise (i.e., converges in the limit). As a side effect, we also derive an innovative way of constructing new useful distances/divergences. To illustrate the core of our approach, we present numerous solved cases. The potential for widespread applicability is indicated, too; in particular, we deliver many recent references for uses of the involved distances/divergences and entropies in various different research fields (which may also serve as an interdisciplinary interface). Comment: v3: considerably shortened and restructured version of v1/v2; 64 pages + 7 pages supplement. This work is accepted by the journal "IEEE Transactions on Information Theory", and is available in early-access form at https://ieeexplore.ieee.org/document/9925151 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|