Details

Autor(en) / Beteiligte

• Xia, Shuhao
• Shi, Yuanming

Titel

Learning Shallow Neural Networks via Provable Gradient Descent with Random Initialization

Ist Teil von

• ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019, p.5616-5620

Ort / Verlag

IEEE

Erscheinungsjahr

2019

Quelle

IEEE Electronic Library (IEL)

Beschreibungen/Notizen

• This paper presents the provable gradient descent algorithm with random initialization for learning a two-layer neural network with quadratic activation functions. Specifically, we focus on the under-parameterized regime where the number of hidden units is smaller than the dimension of the inputs. We reveal that the randomly initialized gradient descent for the nonconvex neural network training problem is able to enter a local region that enjoys strong convexity and strong smoothness within a few iterations, and then provably converges to a globally optimal model at a linear rate.