A Comparison of Regularization Methods for Gaussian Processes
| Autor: | Le Riche, Rodolphe, Mohammadi, Hossein, Durrande, Nicolas, Touboul, Eric, Bay, Xavier |
|---|---|
| Přispěvatelé: | Breuil, Florent, Département Génie mathématique et industriel (FAYOL-ENSMSE), Ecole Nationale Supérieure des Mines de St Etienne (ENSM ST-ETIENNE)-Institut Henri Fayol, Laboratoire d'Informatique, de Modélisation et d'Optimisation des Systèmes (LIMOS), Ecole Nationale Supérieure des Mines de St Etienne (ENSM ST-ETIENNE)-Université Clermont Auvergne [2017-2020] (UCA [2017-2020])-Centre National de la Recherche Scientifique (CNRS), École des Mines de Saint-Étienne (Mines Saint-Étienne MSE), Institut Mines-Télécom [Paris] (IMT), Institut Henri Fayol (FAYOL-ENSMSE), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT), Centre National de la Recherche Scientifique (CNRS), Ecole Nationale Supérieure des Mines de St Etienne-Institut Henri Fayol, Ecole Nationale Supérieure des Mines de St Etienne-Université Clermont Auvergne [2017-2020] (UCA [2017-2020])-Centre National de la Recherche Scientifique (CNRS) |
| Jazyk: | angličtina |
| Rok vydání: | 2017 |
| Předmět: | |
| Zdroj: | 2017 SIAM Conference on Optimization 2017 SIAM Conference on Optimization, May 2017, Vancouver, Canada |
| Popis: | International audience; Gaussian Processes (GPs) are classical probabilistic models to represent the results of experiments on grids of points. They have numerous applications, in particular in nonlinear global optimization when the experiments (typically PDE simulations) are costly. GPs require the inversion of a covariance matrix. There are many situations, in particular optimization, when the density of experiments becomes higher in some regions of the search space, which makes the covariance matrix ill-conditionned, an issue which is handled in general through regularization techniques. Today, the need to better understand and improve regularization remains.The two most classical regularization methods are i) pseudoinverse (PI) and ii) adding a small positive constant to the main diagonal (which is called nugget regularization). This work provides new algebraic insights into PI and nugget regularizations. It is proven that pseudoinverse regularization averages the output values and makes the variance null at redundant points. On the opposite, nugget regularization lacks interpolation properties but preserves a non-zero variance at every point. However, these two regularization techniques become similar as the nugget value decreases. A new distribution-wise GP is then introduced which interpolates Gaussian distributions instead of data points and mitigates the drawbacks of pseudoinverse and nugget regularized GPs. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|