FederatedLearningunderArbitraryCommunicationPatternsDmitriiAvdyukhin1ShivaPrasadKasiviswanathan2AbstractThecanonicalfederatedlearningprobleminvolveslearningasingle,globalstatisticalmodelfromdatasto...
1-bitAdam:CommunicationEfficientLarge-ScaleTrainingwithAdam’sConvergenceSpeedHanlinTang12ShaoduoGan3AmmarAhmadAwan1SamyamRajbhandari1ConglongLi1XiangruLian2JiLiu2CeZhang3YuxiongHe1Abstract1.Introd...
Moniqua:ModuloQuantizedCommunicationinDecentralizedSGDYuchengLu1ChristopherDeSa1Abstract(Lietal.,2014a;b)ortheMPIAllReduceoperation(Groppetal.,1999).Suchadesign,however,putsheavypressureRunningStoc...
LearningEfficientMulti-agentCommunication:AnInformationBottleneckApproachRundongWang1XuHe1RunshengYu1Weiqiu1BoAn1ZinoviRabinovich1Abstractcorrelationthatbenefitsagroup’scooperation.Therefore,manyr...
ImprovingtheSampleandCommunicationComplexityforDecentralizedNon-ConvexOptimization:JointGradientEstimationandTrackingHaoranSun1SongtaoLu2MingyiHong1Abstract1.IntroductionManymodernlarge-scalemachin...
ImprovedCommunicationCostinDistributedPageRankComputation–ATheoreticalStudySiqiangLuo1AbstractNeumannetal.,2011;Ponzetto&Strube,2007).GivenagraphGofnnodesandaprobabilityα∈(0,1),thePageRankisawid...
TradingRedundancyforCommunication:SpeedingupDistributedSGDforNon-convexOptimizationFarzinHaddadpour1MohammadMahdiKamani2MehrdadMahdavi1ViveckR.Cadambe1Abstractchinelearningmodelssuchasdeepneuralnet...
TarMAC:TargetedMulti-AgentCommunicationAbhishekDas1‹ThéophileGervet2JoshuaRomoff23DhruvBatra13DeviParikh13MichaelRabbat23JoellePineau23Abstractandswifttransport,toteamsofrobotsonsearch-and-rescue...
OntheLinearSpeedupAnalysisofCommunicationEfficientMomentumSGDforDistributedNon-ConvexOptimizationHaoYu1RongJin1SenYang1Abstractwherefi(x)=E⇠i⇠Di[Fi(x;⇠i)]aresmoothnon-convexfunctionswithpossibly...
OntheComputationandCommunicationComplexityofParallelSGDwithDynamicBatchSizesforStochasticNon-ConvexOptimizationHaoYu1RongJin1Abstractvariables⇣⇠Drefertoinstance-labelpairsandF(x;⇣)referstothecor...
DistributedLearningwithSublinearCommunication1121JayadevAcharyaChristopherDeSaDylanJ.FosterKarthikSridharanAbstractwhere"(H,N)isadecreasingfunctionofN.Thispaperfocusesondistributedstatisticallearni...
CommunicationComplexityinLocallyPrivateDistributionEstimationandHeavyHittersJayadevAcharya1ZitengSun1AbstractWasserman&Zhou,2010;Wainwrightetal.,2012;Chaud-hurietal.,2011)).DifferentialPrivacy(DP)(...
DistributedNonparametricRegressionunderCommunicationConstraintsYuanchengZhu1JohnLafferty2Abstractaggregatedfromthemultiplelocationstoformthefinalesti-mate.However,theCommunicationbetweenmachinesmay...
DistributedMeanEstimationwithLimitedCommunicationAnandaTheerthaSuresh1FelixX.Yu1SanjivKumar1H.BrendanMcMahan2Abstractthemeansofallclustersineachupdatestep.Similarly,forPCA,ifdatasamplesaredistribut...