Evolving a Deep Neural Network Training Time Estimator
| Autor: | Simon See, Frédéric Pinel, Sébastien Varrette, Pascal Bouvry, Emmanuel Kieffer, Jianxiong Yin, Christian Hundt |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
Computer science
Computer Science::Neural and Evolutionary Computation Population Evolutionary algorithm 02 engineering and technology 010501 environmental sciences Machine learning computer.software_genre 01 natural sciences Genetic algorithm 0202 electrical engineering electronic engineering information engineering education Representation (mathematics) 0105 earth and related environmental sciences Computer science [C05] [Engineering computing & technology] Network architecture education.field_of_study Artificial neural network business.industry Deep learning Estimator Sciences informatiques [C05] [Ingénierie informatique & technologie] 020201 artificial intelligence & image processing Artificial intelligence business computer |
| Zdroj: | Communications in Computer and Information Science ISBN: 9783030419127 OLA |
| DOI: | 10.1007/978-3-030-41913-4_2 |
| Popis: | We present a procedure for the design of a Deep Neural Net- work (DNN) that estimates the execution time for training a deep neural network per batch on GPU accelerators. The estimator is destined to be embedded in the scheduler of a shared GPU infrastructure, capable of providing estimated training times for a wide range of network architectures, when the user submits a training job. To this end, a very short and simple representation for a given DNN is chosen. In order to compensate for the limited degree of description of the basic network representation, a novel co-evolutionary approach is taken to fit the estimator. The training set for the estimator, i.e. DNNs, is evolved by an evolutionary algorithm that optimizes the accuracy of the estimator. In the process, the genetic algorithm evolves DNNs, generates Python-Keras programs and projects them onto the simple representation. The genetic operators are dynamic, they change with the estimator’s accuracy in order to balance accuracy with generalization. Results show that despite the low degree of information in the representation and the simple initial design for the predictor, co-evolving the training set performs better than near random generated population of DNNs. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|