Details

Autor(en) / Beteiligte

• Herrera Segura, Carolina
• Montoya, Edison
• Tapias, Diego

Titel

Subaging in underparametrized deep neural networks

Ist Teil von

• Machine learning: science and technology, 2022-09, Vol.3 (3), p.35013

Ort / Verlag

Bristol: IOP Publishing

Erscheinungsjahr

2022

Quelle

EZB Electronic Journals Library

Beschreibungen/Notizen

• Abstract We consider a simple classification problem to show that the dynamics of finite–width Deep Neural Networks in the underparametrized regime gives rise to effects similar to those associated with glassy systems, namely a slow evolution of the loss function and aging. Remarkably, the aging is sublinear in the waiting time (subaging) and the power–law exponent characterizing it is robust to different architectures under the constraint of a constant total number of parameters. Our results are maintained in the more complex scenario of the MNIST database. We find that for this database there is a unique exponent ruling the subaging behavior in the whole phase.