Quasi-GlobalMomentum:AcceleratingDecentralizedDeepLearningonHeterogeneousDataTaoLin1SaiPraneethKarimireddy1SebastianU.Stich1MartinJaggi1Abstractiskeptlocally(nevertransmittedduringtraining).Decentr...
OptimalComplexityinDecentralizedTrainingYuchengLu1ChristopherDeSa1AbstractTable1.Designchoiceofcentralizationanddecentralizationindifferentlayersofaparallelmachinelearningsystem.TheprotocolDecentra...
LearningFairPoliciesinDecentralizedCooperativeMulti-AgentReinforcementLearningMatthieuZimmer1ClaireGlanois1UmerSiddique1PaulWeng12Abstractcurrentmainfocusisontheirperformancewithrespecttothetotal(o...
KD3A:UnsupervisedMulti-SourceDecentralizedDomainAdaptationviaKnowledgeDistillationHao-zheFeng1ZhaoyangYou2MinghaoChen1TianyeZhang1MinfengZhu1FeiWu2ChaoWu3WeiChen1Abstracttotrainthemodel.However,due...
DecentralizedSingle-TimescaleActorCriticonZero-SumTwo-PlayerStochasticGamesHongyiGuo1ZuyueFu1ZhuoranYang2ZhaoranWang1AbstractasMarkovdecisionprocess(Puterman,2014,MDP),whereanagentaimstolearnanopti...
DecentralizedRiemannianGradientDescentontheStiefelManifoldShixiangChen1AlfredoGarcia1MingyiHong2ShahinShahrampour1AbstractagentoptimizationproblemWeconsiderdistributednon-convexoptimization1nwherea...
Cross-GradientAggregationforDecentralizedLearningfromNon-IIDDataYasamanEsfandiari1SinYongTan1ZhanhongJiang2AdityaBalu1EthanHerron1ChinmayHegde3SoumikSarkar1AbstractFigure1.Algorithmoverview.Inthepr...
ConsensusControlforDecentralizedDeepLearningLingjingKong1TaoLin1AnastasiaKoloskova1MartinJaggi1SebastianU.Stich1Abstractlocalmini-batchgradientscomputedondifferentsubsetsofthedata,forthelatersynchr...
Beyondlog2(T)RegretforDecentralizedBanditsinMatchingMarketsSoumyaBasu1KarthikAbinavSankararaman2AbishekSankararaman3Abstractbanditsisdedicatedtounderstandingalgorithmicprinciplesintheinterplayofcom...
ADOM:AcceleratedDecentralizedOptimizationMethodforTime-VaryingNetworksDmitryKovalev1EgorShulgin1PeterRichta´rik1AlexanderRogozin23AlexanderGasnikov23Abstract⇒⇒WeproposeADOM–anacceleratedmethodf...
AsynchronousDecentralizedOptimizationWithImplicitStochasticVarianceReductionKentaNiwa12GuoqiangZhang3W.BastiaanKleijn4NoboruHarada12HiroshiSawada1AkinoriFujino1AbstractOurgoalistofacilitateMLmodelt...
AHybridVariance-ReducedMethodforDecentralizedStochasticNon-ConvexOptimizationRanXin1UsmanA.Khan2SoummyaKar1Abstractistosolve,vialocalcomputationandcommunication,thefollowingoptimizationproblem:This...
TheNon-IIDDataQuagmireofDecentralizedMachineLearningKevinHsieh12AmarPhanishayee1OnurMutlu32PhillipB.Gibbons2AbstractTheseconstraintsmotivatetheneedforMLtrainingoverwidelydistributeddata(decentraliz...
QuantizedDecentralizedStochasticLearningoverDirectedGraphsHosseinTaheri1AryanMokhtari2HamedHassani3RamtinPedarsani1AbstractDecentralizedlearningistooptimizeafunctionf:Rd→Rdefinedastheaverageoffunc...
OptimallySolvingTwo-AgentDecentralizedPOMDPsUnderOne-SidedInformationSharingYuxuanXie1JillesS.Dibangoye1OlivierBuffet2Abstractalongwiththeirdoubleexponentialgrowthwithagentsandtimeexplaintheworst-c...
Moniqua:ModuloQuantizedCommunicationinDecentralizedSGDYuchengLu1ChristopherDeSa1Abstract(Lietal.,2014a;b)ortheMPIAllReduceoperation(Groppetal.,1999).Suchadesign,however,putsheavypressureRunningStoc...
AUnifiedTheoryofDecentralizedSGDwithChangingTopologyandLocalUpdatesAnastasiaKoloskova1NicolasLoizou2SadraBoreiri1MartinJaggi1SebastianU.Stich1Abstractetal.,2016;2017;Kairouzetal.,2019)hasemerged,bu...
DecentralizedExplorationinMulti-ArmedBanditsRaphaëlFéraud1RédaAlami1RomainLaroche2Abstractviceisconnectingtotheapplication,theapplicationpresentsanoptiontotheuserofthedevice.TheaimistomaximizeWe...
DecentralizedStochasticOptimizationandGossipAlgorithmswithCompressedCommunicationAnastasiaKoloskova1SebastianU.Stich1MartinJaggi1Abstracttionwithneighboringdevices.Thiscoversforinstancetheclassicse...
TowardsMoreEfficientStochasticDecentralizedLearning:FasterConvergenceandSparseCommunicationZebangShen12AryanMokhtari3TengfeiZhou1PeilinZhao4HuiQian1Abstractvariableofnoden,theproblemofinterestisRec...