Variational(Gradient)EstimateoftheScoreFunctioninEnergy-basedLatentVariableModelsFanBao12KunXu1ChongxuanLi1LanqingHong2JunZhu1BoZhang1Abstractsivepower(Salakhutdinov&Hinton,2009;Baoetal.,2020)andto...
UnderstandingtheDynamicsofGradientFlowinOverparameterizedLinearModelsSalmaTarmoun12GuilhermeFranc¸a13BenjaminHaeffele14Rene´Vidal14Abstractathoroughanalysisofthedynamicsofoptimizationmeth-ods,suc...
UnbiasedGradientEstimationinUnrolledComputationGraphswithPersistentEvolutionStrategiesPaulVicol12LukeMetz2JaschaSohl-Dickstein2AbstractZipser,1989;Tallec&Ollivier,2017a;Mujikaetal.,2018;Benzingetal...
TemporalDifferenceLearningasGradientSplittingRuiLiu1AlexOlshevsky2AbstractTDusesdifferencesinpredictionsoversuccessivetimestepstodrivethelearningprocess,withthepredictionatTemporaldifferencelearnin...
StraighttotheGradient:LearningtoUseNovelTokensforNeuralTextGenerationXiangLin1SimengHan1ShafiqJoty12Abstractrization(Seeetal.,2017),imagecaptioning(Melas-Kyriazietal.,2018;Wang&Chan,2019)andmachine...
StabilityandConvergenceofStochasticGradientClipping:BeyondLipschitzContinuityandSmoothnessVienV.Mai1MikaelJohansson1Abstractproblemsareatthecoreofmanymachine-learningappli-cations,andareoftensolved...
StabilityandGeneralizationofStochasticGradientMethodsforMinimaxProblemsYunwenLei1ZhenhuanYang2TianbaoYang3YimingYing2Abstracting(Goodfellowetal.,2014),robustoptimization(Chenetal.,2017;Namkoong&Duc...
SGLB:StochasticGradientLangevinBoostingAlekseiUstimenko1LiudmilaProkhorenkova123AbstractLangevindynamics(SGLD),whichisapowerfuliterativeoptimizationalgorithm(Raginskyetal.,2017).ItturnsThispaperint...
RobustPolicyGradientagainstStrongDataCorruptionXuezhouZhang1YidingChen1JerryZhu1WenSun2Abstracthighlynoisydata,suchasautonomousdriving,quantitativetrading,ormedicaldiagnosis.Westudytheproblemofrobu...
Progressive-ScaleBoundaryBlackboxAttackviaProjectiveGradientEstimationJiaweiZhang1LinyiLi2HuichenLi2XiaoluZhang3ShuangYang4BoLi2Abstract1.IntroductionBoundarybasedblackboxattackhasbeenrec-Blackboxa...
PrivateAdaptiveGradientMethodsforConvexOptimizationHilalAsi12JohnDuchi23AlirezaFallah41OmidJavidbakht5KunalTalwar5AbstractopingprivatevariantsofstochasticGradientdescent(SGD),wherealgorithmsguarant...
Positive-NegativeMomentum:ManipulatingStochasticGradientNoisetoImproveGeneralizationZekeXie1LiYuan2ZhanxingZhu3MasashiSugiyama41AbstractItiswell-knownthatstochasticGradientnoise(SGN)instochasticopt...
PolicyGradientBayesianRobustOptimizationforImitationLearningZaynahJaved1DanielS.Brown1SatvikSharma1JerryZhu1AshwinBalakrishna1MarekPetrik2AncaD.Dragan1KenGoldberg1Abstracthuman-designedrewardfuncti...
PhasicPolicyGradientKarlCobbe1JacobHilton1OlegKlimov1JohnSchulman1Abstractcanbeusedtobetteroptimizetheother.WeintroducePhasicPolicyGradient(PPG),are-However,therearealsodisadvantagestosharingnetwor...
PAGE:ASimpleandOptimalProbabilisticGradientEstimatorforNonconvexOptimizationZhizeLi1HongyanBao1XiangliangZhang1PeterRichta´rik1Abstract(Jain&Kar,2017).Drivenbytheappliedsuccessofdeepneuralnetworks...
OopsITookAGradient:ScalableSamplingforDiscreteDistributionsWillGrathwohl12KevinSwersky2MiladHashemi2DavidDuvenaud12ChrisJ.Maddison1AbstractFigure1.Ourapproachvisualized.Oftendiscretedistributionsar...
OnlinePolicyGradientforModelFre√eLearningofLinearQuadraticRegulatorswithTRegretAsafCassel1TomerKoren12AbstractModel-basedmethods,whichperformplanningbasedonasystemidentificationprocedurethatestima...
OnLearnabilityviaGradientMethodforTwo-LayerReLUNeuralNetworksinTeacher-StudentSettingShuntaAkiyama1TaijiSuzuki12Abstractforthegeneralizationaspect.Inthisstudy,wetacklethesetwoproblemsinateacher-stu...
OntheProofofGlobalConvergenceofGradientDescentforDeepReLUNetworkswithLinearWidthsQuynhNguyen1Abstracttrainingdata,thentheoutputatlayerlisgivenbyWegiveasimpleprooffortheglobalconver-genceofgradien...
LearningGradientFieldsforMolecularConformationGenerationChenceShi12ShitongLuo3MinkaiXu12JianTang145Abstractamorenaturalandintrinsicrepresentationformoleculesistheirthree-dimensionalstructures,where...