Details

Autor(en) / Beteiligte

• Pinel, Frédéric
• Yin, Jian-xiong
• Hundt, Christian
• Kieffer, Emmanuel
• Varrette, Sébastien
• Bouvry, Pascal
• See, Simon
• Dorronsoro, Bernabé
• Ruiz, Patricia
• de la Torre, Juan Carlos
• Urda, Daniel
• Talbi, El-Ghazali

Titel

Evolving a Deep Neural Network Training Time Estimator

Ist Teil von

• Optimization and Learning, p.13-24

Ort / Verlag

Cham: Springer International Publishing

Link zum Volltext

Quelle

Alma/SFX Local Collection

Beschreibungen/Notizen

• We present a procedure for the design of a Deep Neural Network (DNN) that estimates the execution time for training a deep neural network per batch on GPU accelerators. The estimator is destined to be embedded in the scheduler of a shared GPU infrastructure, capable of providing estimated training times for a wide range of network architectures, when the user submits a training job. To this end, a very short and simple representation for a given DNN is chosen. In order to compensate for the limited degree of description of the basic network representation, a novel co-evolutionary approach is taken to fit the estimator. The training set for the estimator, i.e. DNNs, is evolved by an evolutionary algorithm that optimizes the accuracy of the estimator. In the process, the genetic algorithm evolves DNNs, generates Python-Keras programs and projects them onto the simple representation. The genetic operators are dynamic, they change with the estimator’s accuracy in order to balance accuracy with generalization. Results show that despite the low degree of information in the representation and the simple initial design for the predictor, co-evolving the training set performs better than near random generated population of DNNs.