Bayesian Optimization in High-dimension via a Combination of Kriging sub-models
| Autor: | Appriou, Tanguy, Rullière, Didier, Gaudrie, David |
|---|---|
| Přispěvatelé: | Laboratoire d'Informatique, de Modélisation et d'Optimisation des Systèmes (LIMOS), Ecole Nationale Supérieure des Mines de St Etienne (ENSM ST-ETIENNE)-Centre National de la Recherche Scientifique (CNRS)-Université Clermont Auvergne (UCA)-Institut national polytechnique Clermont Auvergne (INP Clermont Auvergne), Université Clermont Auvergne (UCA)-Université Clermont Auvergne (UCA), École des Mines de Saint-Étienne (Mines Saint-Étienne MSE), Institut Mines-Télécom [Paris] (IMT), Institut Henri Fayol (FAYOL-ENSMSE), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT), Département Génie mathématique et industriel (FAYOL-ENSMSE), Ecole Nationale Supérieure des Mines de St Etienne (ENSM ST-ETIENNE)-Institut Henri Fayol, Stellantis - PSA Centre Technique de Vélizy, UQ@Paris-Saclay |
| Jazyk: | angličtina |
| Rok vydání: | 2023 |
| Předmět: | |
| Zdroj: | MASCOT-NUM2023 MASCOT-NUM2023, Apr 2023, LE CROISIC, France |
| Popis: | International audience; Numerical optimization has been widely used to solve design engineering problems in recent years. However, due to the large number of numerical simulations required, the optimization process involved is often computationally expensive. In order to accelerate the optimization process, strategies based on surrogate models are often used. One such approach, efficient global optimization (EGO) [1], based on Kriging surrogate models [2], has been successfully applied to a number of real-worlds applications in low-dimensions (less than 20).However, in engineering design optimization, the designs are often parametrized by more than 50 shape parameters. The ordinary Kriging method scales poorly for high-dimensional problems, and building a good Kriging model is met with various setbacks. One of the main challenges is related to length-scale hyperparameter optimization. Most Kriging models consider one length-scale hyperparameter per dimension which all need to be optimized simultaneously. This is typically done by maximizing the log-likelihood of the model. For high-dimensional problems, this optimization is problematic due to the exponential growth of the search space with the dimension, to the shape of the log-likelihood function and to its computational cost, and to over-fitting issues when there are too few observations. Several papers address these difficulties by reducing the dimension of the problem, for instance by embedding the design space into a lower dimensional space [3,4], or by considering simplifying hypothesis such as additive models [5].In this paper, we propose a new method for high-dimensional Kriging models which bypasses the length-scales optimization by combining Kriging sub-models with fixed length-scales. Contrarily to other approaches, this does not rely on dimension reduction and preserves the correlation between all input variables. Furthermore, the combination has a closed-form expression and does not require any inner optimization. We also describe a novel approach to obtain suitable bounds for the lengthscales of the sub-models, and we compare different weighting schemes for the approximation of high-dimensional test functions. Finally, we present a way to obtain the variance of the combination for any weighting methods and we apply our combined model to high-dimensional EGO. We show that the classical Kriging approach using maximum likelihood estimation fails to properly optimize the length-scale hyperparameters and that our combination of sub-models with fixed length-scales successfully build more accurate surrogate models for EGO. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|