Meta-learning for symbolic hyperparameter defaults
| Autor: | Gijsbers, P., Pfisterer, F., Rijn, J.N. van, Bischl, B., Vanschoren, J., Chicano, F. |
|---|---|
| Přispěvatelé: | Chicano, F. |
| Jazyk: | angličtina |
| Rok vydání: | 2021 |
| Předmět: |
FOS: Computer and information sciences
Hyperparameter Computer Science - Machine Learning Optimization problem Meta learning (computer science) Grammar business.industry Computer science media_common.quotation_subject Evolutionary algorithm Machine Learning (stat.ML) Machine learning computer.software_genre Metalearning Machine Learning (cs.LG) Statistics - Machine Learning Hyperparameter optimization Artificial intelligence Heuristics business computer media_common |
| Zdroj: | GECCO '21: Proceedings of the genetic and evolutionary computation conference companion, 151-152. New York: ACM STARTPAGE=151;ENDPAGE=152;TITLE=GECCO '21: Proceedings of the genetic and evolutionary computation conference companion GECCO Companion |
| Popis: | Hyperparameter optimization in machine learning (ML) deals with the problem of empirically learning an optimal algorithm configuration from data, usually formulated as a black-box optimization problem. In this work, we propose a zero-shot method to meta-learn symbolic default hyperparameter configurations that are expressed in terms of the properties of the dataset. This enables a much faster, but still data-dependent, configuration of the ML algorithm, compared to standard hyperparameter optimization approaches. In the past, symbolic and static default values have usually been obtained as hand-crafted heuristics. We propose an approach of learning such symbolic configurations as formulas of dataset properties from a large set of prior evaluations on multiple datasets by optimizing over a grammar of expressions using an evolutionary algorithm. We evaluate our method on surrogate empirical performance models as well as on real data across 6 ML algorithms on more than 100 datasets and demonstrate that our method indeed finds viable symbolic defaults. Pieter Gijsbers and Florian Pfisterer contributed equally to the paper. V1: Two page GECCO poster paper accepted at GECCO 2021. V2: The original full length paper (8 pages) with appendix |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|