Taming Hyper-parameters in Deep Learning Systems
| Autor: | Guo Li, Alexandros Koliousis, Andrei-Octavian Brabete, Luo Mai, Peter Pietzuch |
|---|---|
| Rok vydání: | 2019 |
| Předmět: |
0303 health sciences
Thesaurus (information retrieval) Computer science business.industry Deep learning 030303 biophysics 010501 environmental sciences Machine learning computer.software_genre 01 natural sciences 03 medical and health sciences Hyper parameters General Earth and Planetary Sciences Artificial intelligence business Implementation computer 0105 earth and related environmental sciences General Environmental Science |
| Zdroj: | ACM SIGOPS Operating Systems Review. 53:52-58 |
| ISSN: | 0163-5980 |
| Popis: | Deep learning (DL) systems expose many tuning parameters ("hyper-parameters") that affect the performance and accuracy of trained models. Increasingly users struggle to configure hyper-parameters, and a substantial portion of time is spent tuning them empirically. We argue that future DL systems should be designed to help manage hyper-parameters. We describe how a distributed DL system can (i) remove the impact of hyper-parameters on both performance and accuracy, thus making it easier to decide on a good setting, and (ii) support more powerful dynamic policies for adapting hyper-parameters, which take monitored training metrics into account. We report results from prototype implementations that show the practicality of DL system designs that are hyper-parameter-friendly. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|