WGANwithanInfinitelyWideGeneratorHasNoSpuriousStationaryPointsAlbertNo1TaeHoYoon2SehyunKwon2ErnestK.Ryu2AbstractfinitelyWidegeneratortrainedwithstochasticgradientascent-descent.Specifically,weshowt...
UniquePropertiesofFlatMinimainDeepNetworksRotemMulayoff1TomerMichaeli1Abstractmodels(Gunasekaretal.,2018b)anddeepnonlinearnet-workswithhomogeneousactivationfunctions(Lyu&Li,Itiswellknownthat(stocha...
GoWide,ThenNarrow:EfficientTrainingofDeepThinNetworksDennyZhou1MaoYe2ChenChen1TianjianMeng1MingxingTan1XiaodanSong1QuocLe1QiangLiu2DaleSchuurmans1Abstractfeletal.,2019;Brownetal.,2020).Toenlargeamo...
NeuralNetworksShouldBeWideEnoughtoLearnDisconnectedDecisionRegionsQuynhNguyen1MaheshChandraMukkamala1MatthiasHein2AbstractThefirstimportantresultsaretheuniversalapproximationtheorems(Cybenko,1989;H...
TheLossSurfaceofDeepandWideNeuralNetworksQuynhNguyen1MatthiasHein1Abstractdoesnotencounterproblemswithsuboptimallocalmin-ima.However,astheauthorsadmitthemselvesin(Good-Whiletheoptimizationproblembe...