StochasticVariance-ReducedCubicRegularizedNewtonMethodsDongruoZhou1PanXu1QuanquanGu1Abstracttheirextensions(Ghadimi&Lan,2013;Reddietal.,2016a;Allen-Zhu&Hazan,2016;Ghadimi&Lan,2016).Never-Weproposea...
StochasticProximalAlgorithmsforAUCMaximizationMichaelNatoleJr.1YimingYing1SiweiLyu2Abstractformeasuringtheperformanceinthesetasks.Inpartic-ular,minimizationoftheranklossinbipartiterankingisStochast...
StochasticPCAwith2and1RegularizationPooryaMianjy1RamanArora1Abstractsubspacethatmaximizesthevarianceindataprojectedontothesubspace,i.e.E[Ux2],amongallk-dimensionalsub-Werevisitconvexrelaxationbased...
Shampoo:PreconditionedStochasticTensorOptimizationVineetGupta1TomerKoren1YoramSinger21AbstractNocedal,1980)thatcanbeusedwheneversecond-orderinformationisunavailableortooexpensivetocompute.Precondit...
SADAGRAD:StronglyAdaptiveStochasticGradientMethodsZaiyiChen12YiXu2EnhongChen1TianbaoYang2Abstractiterations.Ithasreceivedtremendousinterestsforsolvingbigdatalearningproblems(e.g.,see(Deanetal.,2012...
RiemannianStochasticRecursiveGradientAlgorithmHiroyukiKasai1HiroyukiSato2BamdevMishra3Abstractmanniangradientdescentmethod,whichcalculatestheR1ie∑mnannianfullgradientestimation,i.e.,gradf(w)=Stoch...
Projection-FreeOnlineOptimizationwithStochasticGradient:FromConvexitytoSubmodularityLinChen12ChristopherHarshaw13HamedHassani4AminKarbasi12Abstract1.IntroductionOnlineoptimizationhasbeenasuccessful...
OptimalDistributedLearningwithMulti-passStochasticGradientMethodsJunhongLin1VolkanCevher1AbstractTheclassicalalgorithmstoperformlearningtaskareregular-izedalgorithms,suchasKRR,kernelprincipalcompon...
Low-RankRiemannianOptimizationonPositiveSemidefiniteStochasticMatriceswithApplicationstoGraphClusteringAhmedDouik1BabakHassibi1overthesetofsymmetricpositivesemidefinitestochas-Abstractticmatrices.S...
LightweightStochasticOptimizationforMinimizingFiniteSumswithInfiniteDataShuaiZheng1JamesT.Kwok1Abstractiandgaresmoothandconvex.Stochasticgradientde-scent(SGD)(Robbins&Monro,1951)anditsvariantsVaria...
KatyushaX:PracticalMomentumMethodforStochasticSum-of-NonconvexOptimizationZeyuanAllen-Zhu1Abstractsolvingthem.Theproblemofminimizingsum-of-nonconvexAtleasttracingbackto2015,Shalev-Shwartz(2016)iden...
FasterDerivative-FreeStochasticAlgorithmforSharedMemoryMachinesBinGu1ZhouyuanHuo1ChengDeng2HengHuang12Abstractsianmatrixoritsapproximations)oftheobjectivefunction.Forexample,Hogwild!(Rechtetal.,201...
FastVarianceReductionMethodwithStochasticBatchSizeXuanqingLiu1Cho-JuiHsieh12AbstractHereweassumeeachfi(w)isaµ-stronglyconvex,L-smoothfunction,theregularizationtermg(w)isconvexInthispaperwestudyafa...
FastStochasticAUCMaximizationwithO(1/n)-ConvergenceRateMingruiLiu1XiaoxuanZhang1ZaiyiChen2XiaoyuWang3TianbaoYang1Abstract1982).Comparedwithmisclassificationrate,AUCismorefavorableintheapplicationsw...
EscapingSaddleswithStochasticGradientsHadiDaneshmand1JonasKohler1AurelienLucchi1ThomasHofmann1Abstractthroughaconservativestepsize(Nesterov,2013)orviaexplicitvariance-reductiontechniques(Johnson&Zh...
DissipativityTheoryforAcceleratingStochasticVarianceReduction:AUnifiedAnalysisofSVRGandKatyushaUsingSemidefiniteProgramsBinHu1StephenWright1LaurentLessard1AbstractMonro,1951;Bottou&LeCun,2003).Rece...
Data-DependentStabilityofStochasticGradientDescentIljaKuzborskij1ChristophH.Lampert2Abstractticeonemightnotevenreachaminimum,yetneverthelessobservesexcellentperformance.Weestablishadata-dependentno...
ContinuousandDiscrete-timeAcceleratedStochasticMirrorDescentforStronglyConvexFunctionsPanXu1TianhaoWang2QuanquanGu1Abstractloss,squarelossandstronglyconvexregularizerssuchas`2normandKullback-Leible...
Black-BoxVariationalInferenceforStochasticDifferentialEquationsThomasRyder12AndrewGolightly1A.StephenMcGough2DennisPrangle1Abstracterrorinvolvedconvergestozeroasthegridbecomesfiner.Parameterinferen...
AugmentandReduce:StochasticInferenceforLargeCategoricalDistributionsFranciscoJ.R.Ruiz12MichalisK.Titsias3AdjiB.Dieng2DavidM.Blei2Abstractwithmanyclasses,recommendationsystemswithmanyitems,andlangua...