Bayesian Optimization in a Billion Dimensions via Random Embeddings
| Autor: | David Matheson, Frank Hutter, Nando de Freitas, Ziyu Wang, Masrour Zoghi |
|---|---|
| Rok vydání: | 2016 |
| Předmět: |
FOS: Computer and information sciences
Mathematical optimization 021103 operations research Computer science Bayesian optimization 0211 other engineering and technologies Machine Learning (stat.ML) 02 engineering and technology Solver Field (computer science) Machine Learning (cs.LG) Computer Science - Learning Dimension (vector space) Statistics - Machine Learning Artificial Intelligence 0202 electrical engineering electronic engineering information engineering Embedding 020201 artificial intelligence & image processing Categorical variable Integer programming Curse of dimensionality |
| Zdroj: | Journal of Artificial Intelligence Research. 55:361-387 |
| ISSN: | 1076-9757 |
| DOI: | 10.1613/jair.4806 |
| Popis: | Bayesian optimization techniques have been successfully applied to robotics, planning, sensor placement, recommendation, advertising, intelligent user interfaces and automatic algorithm configuration. Despite these successes, the approach is restricted to problems of moderate dimension, and several workshops on Bayesian optimization have identified its scaling to high-dimensions as one of the holy grails of the field. In this paper, we introduce a novel random embedding idea to attack this problem. The resulting Random EMbedding Bayesian Optimization (REMBO) algorithm is very simple, has important invariance properties, and applies to domains with both categorical and continuous variables. We present a thorough theoretical analysis of REMBO. Empirical results confirm that REMBO can effectively solve problems with billions of dimensions, provided the intrinsic dimensionality is low. They also show that REMBO achieves state-of-the-art performance in optimizing the 47 discrete parameters of a popular mixed integer linear programming solver. Comment: 33 pages |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|