Novel First Order Bayesian Optimization with an Application to Reinforcement Learning
| Autor: | Chandramouli Kamanchi, K J Prabuchandran, Santosh Penubothula, Shalabh Bhatnagar |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
Mathematical optimization
Parallelizable manifold Artificial Intelligence Computer science Bayesian optimization 0202 electrical engineering electronic engineering information engineering Leverage (statistics) Reinforcement learning 020201 artificial intelligence & image processing Statistical model 02 engineering and technology First order |
| Zdroj: | Applied Intelligence. 51:1565-1579 |
| ISSN: | 1573-7497 0924-669X |
| Popis: | Zeroth Order Bayesian Optimization (ZOBO) methods optimize an unknown function based on its black-box evaluations at the query locations. Unlike most optimization procedures, ZOBO methods fail to utilize gradient information even when it is available. On the other hand, First Order Bayesian Optimization (FOBO) methods exploit the available gradient information to arrive at better solutions faster. However, the existing FOBO methods do not utilize a crucial information that the gradient is zero at the optima. Further, the inherent sequential nature of the FOBO methods incur high computational cost limiting their wide applicability. To alleviate the aforementioned difficulties of FOBO methods, we propose a relaxed statistical model to leverage the gradient information that directly searches for points where gradient vanishes. To accomplish this, we develop novel acquisition algorithms that search for global optima effectively. Unlike the existing FOBO methods, the proposed methods are parallelizable. Through extensive experimentation on standard test functions, we compare the performance of our methods over the existing methods. Furthermore, we explore an application of the proposed FOBO methods in the context of policy gradient reinforcement learning. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Full text from SpringerLink