On Margins and Derandomisation in PAC-Bayes
| Autor: | Biggs, Felix, Guedj, Benjamin |
|---|---|
| Přispěvatelé: | University College of London [London] (UCL), Computer science department [University College London] (UCL-CS), The Inria London Programme (Inria-London), University College of London [London] (UCL)-University College of London [London] (UCL)-Institut National de Recherche en Informatique et en Automatique (Inria), Inria-CWI (Inria-CWI), Centrum Wiskunde & Informatica (CWI)-Institut National de Recherche en Informatique et en Automatique (Inria), MOdel for Data Analysis and Learning (MODAL), Laboratoire Paul Painlevé - UMR 8524 (LPP), Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-École polytechnique universitaire de Lille (Polytech Lille), Department of Computer science [University College of London] (UCL-CS), Laboratoire Paul Painlevé (LPP), Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille-Centre National de la Recherche Scientifique (CNRS)-Université de Lille, Sciences et Technologies-Inria Lille - Nord Europe, Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille)-École polytechnique universitaire de Lille (Polytech Lille), Felix Biggs gratefully acknowledges the support of the CDT for Foundational Artificial Intelligence through UKRI grant EP/S021566/1. Benjamin Guedj acknowledges partial support by the U.S. Army Research Laboratory and the U.S. Army Research Office, and by the U.K. Ministry of Defence and the U.K. Engineering and Physical Sciences Research Council (EPSRC) under grant number EP/R013616/1, Benjamin Guedj also acknowledges partial support from the French National Agency for Research, grants ANR18-CE40-0016-01 and ANR- 18-CE23-0015-02., ANR-18-CE40-0016,BEAGLE,Apprentissage PAC-bayésien agnostique(2018), ANR-18-CE23-0015,APRIORI,Une Perspective PAC-Bayésienne de l'Apprentissage de Représentations(2018) |
| Jazyk: | angličtina |
| Rok vydání: | 2021 |
| Předmět: |
FOS: Computer and information sciences
Computer Science - Machine Learning [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG] [STAT.ML]Statistics [stat]/Machine Learning [stat.ML] FOS: Mathematics Mathematics - Statistics Theory Statistics Theory (math.ST) [STAT.TH]Statistics [stat]/Statistics Theory [stat.TH] Machine Learning (cs.LG) |
| Zdroj: | AISTATS 2022-25th International Conference on Artificial Intelligence and Statistics AISTATS 2022-25th International Conference on Artificial Intelligence and Statistics, Mar 2022, Valencia, Spain |
| Popis: | We give a general recipe for derandomising PAC-Bayesian bounds using margins, with the critical ingredient being that our randomised predictions concentrate around some value. The tools we develop straightforwardly lead to margin bounds for various classifiers, including linear prediction -- a class that includes boosting and the support vector machine -- single-hidden-layer neural networks with an unusual \(\erf\) activation function, and deep ReLU networks. Further, we extend to partially-derandomised predictors where only some of the randomness is removed, letting us extend bounds to cases where the concentration properties of our predictors are otherwise poor. 23 pages |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|