Distribution-Dependent Weighted Union Bound †
| Autor: | Sandro Ridella, Luca Oneto |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2021 |
| Předmět: |
Computer Science::Machine Learning
General Physics and Astronomy distribution-dependent weights lcsh:Astrophysics 0102 computer and information sciences 02 engineering and technology 01 natural sciences Article Combinatorics Set (abstract data type) Empirical error statistical learning theory Bounding overwatch weighted union bound lcsh:QB460-466 Distribution-dependent weights Finite number of hypothesis Statistical learning theory Union bound Weighted union bound 0202 electrical engineering electronic engineering information engineering lcsh:Science Mathematics union bound State (functional analysis) Function (mathematics) lcsh:QC1-999 finite number of hypothesis Distribution (mathematics) 010201 computation theory & mathematics A priori and a posteriori lcsh:Q 020201 artificial intelligence & image processing lcsh:Physics |
| Zdroj: | Entropy Volume 23 Issue 1 Entropy, Vol 23, Iss 101, p 101 (2021) |
| ISSN: | 1099-4300 |
| Popis: | In this paper, we deal with the classical Statistical Learning Theory&rsquo s problem of bounding, with high probability, the true risk R(h) of a hypothesis h chosen from a set H of m hypotheses. The Union Bound (UB) allows one to state that PLR^(h),&delta qh&le R(h)&le UR^(h),&delta ph&ge 1&minus &delta where R^(h) is the empirical errors, if it is possible to prove that P{R(h)&ge L(R^(h),&delta )}&ge and P{R(h)&le U(R^(h),&delta when h, qh, and ph are chosen before seeing the data such that qh,ph&isin [0,1] and &sum h&isin H(qh+ph)=1. If no a priori information is available qh and ph are set to 12m, namely equally distributed. This approach gives poor results since, as a matter of fact, a learning procedure targets just particular hypotheses, namely hypotheses with small empirical error, disregarding the others. In this work we set the qh and ph in a distribution-dependent way increasing the probability of being chosen to function with small true risk. We will call this proposal Distribution-Dependent Weighted UB (DDWUB) and we will retrieve the sufficient conditions on the choice of qh and ph that state that DDWUB outperforms or, in the worst case, degenerates into UB. Furthermore, theoretical and numerical results will show the applicability, the validity, and the potentiality of DDWUB. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
| Nepřihlášeným uživatelům se plný text nezobrazuje | K zobrazení výsledku je třeba se přihlásit. |