ExponentiallyManyLocalMinimainQuantumNeuralNetworksXuchenYou12XiaodiWu12Abstractso-calledvariationalquantummethod(e.g.,Peruzzoetal.(2014)),arethemajorcandidatesofapplicationsthatcanbeQuantumNeuralN...
UniquePropertiesofFlatMinimainDeepNetworksRotemMulayoff1TomerMichaeli1Abstractmodels(Gunasekaretal.,2018b)anddeepnonlinearnet-workswithhomogeneousactivationfunctions(Lyu&Li,Itiswellknownthat(stocha...
NormalizedFlatMinima:ExploringScaleInvariantDefinitionofFlatMinimaforNeuralNetworksUsingPAC-BayesianAnalysisYusukeTsuzuku12IsseiSato12MasashiSugiyama21Abstractbleexplanationforthegeneralizationabil...
Information-TheoreticLocalMinimaCharacterizationandRegularizationZhiweiJia1HaoSu1Abstractinitionsof“flatness/sharpness”havebeenintroducedandanalyzed(Keskaretal.,2017;Neyshaburetal.,2018;2017;Rece...
Passed&Spurious:DescentAlgorithmsandLocalMinimainSpikedMatrix-TensorModelsStefanoSaraoMannelli1FlorentKrzakala2PierfrancescoUrbani1LenkaZdeborova´1AbstractRecentyearsbroughtapopularlineofresearchi...
GradientDescentFindsGlobalMinimaofDeepNeuralNetworksSimonS.Du1JasonD.Lee2HaochuanLi34LiweiWang54XiyuZhai6AbstractThesecondmysteriousphenomenonintrainingdeepneuralnetworksis“deepernetworksareharder...
SpuriousLocalMinimaareCommoninTwo-LayerReLUNeuralNetworksItaySafran1OhadShamir1Abstractlearning,andtensordecomposition,donothavespuriouslocalMinimaundersuitableassumptions,inwhichcaselo-Weconsidert...
SharpMinimaCanGeneralizeForDeepNetsLaurentDinh1RazvanPascanu2SamyBengio3YoshuaBengio14Abstractapproximatecertainfunctions(e.g.Montufaretal.,2014;Raghuetal.,2016).Otherworks(e.gDauphinetal.,2014;Des...
NoSpuriousLocalMinimainNonconvexLowRankProblems:AUnifiedGeometricAnalysisRongGe1ChiJin2YiZheng1Abstract2016;Parketal.,2016)andmatrixcompletion(Geetal.,2016)havewell-behavedoptimizationlandscape:all...