Spurious valleys in one-hidden-layer neural network optimization landscapes



Neural networks provide a rich class of high-dimensional, non-convex optimization problems. Despite their non-convexity, gradient-descent methods often successfully optimize these models. This has motivated a recent spur in research attempting to characterize properties of their loss surface that may explain such success.


Luca Venturi
Afonso S. Bandeira
Joan Bruna