FollowtheCompressedLeader:FasterOnlineLearningofEigenvectorsandFasterMMWUZeyuanAllen-Zhu1YuanzhiLi2Abstractsymmetricandsatisfies0AkI.1Boththeseassump-tionsareforthesakeofsimplicityandcanberelaxed.2...
FollowtheMovingLeaderinDeepLearningShuaiZheng1JamesT.Kwok1Abstractever,trainingdeepnetworksisdifficultastheoptimiza-tioncansufferfrompathologicalcurvatureandgetstuckDeepnetworksarehighlynonlinearan...