WidthProvablyMattersinOptimizationforDeepLinearNeuralNetworksSimonS.Du1WeiHu2Abstractconvergestoglobalminimumunderfurtherassumptionsonbothdataandglobalminimum.TheseresultsrequireWeprovethatforanL-l...
TransferinDeepReinforcementLearningUsingSuccessorFeaturesandGeneralisedPolicyImprovementAndre´Barreto1DianaBorsa1JohnQuan1TomSchaul1DavidSilver1MatteoHessel1DanielMankowitz1AugustinZˇ´ıdek1Re´...
ToUnderstandDeepLearningWeNeedtoUnderstandKernelLearning†MikhailBelkin1SiyuanMa1SoumikMandal1AbstractDeeplearningwillbedifficultuntilmoretractable“shallow”kernelmethodsarebetterunderstood.Genera...
StrongerGeneralizationBoundsforDeepNetsviaaCompressionApproachSanjeevArora1RongGe2BehnamNeyshabur3YiZhang1AbstractfueledresearchinthisareabyshowingexperimentallythatstandardarchitecturesusingSGDand...
StructuredControlNetsforDeepReinforcementLearningMarioSrouji1JianZhang2RuslanSalakhutdinov12AbstractInrecentyears,DeepReinforcementLearningFigure1.TheproposedStructuredControlNet(SCN)forpolicyhasma...
StrassenNets:DeepLearningwithaMultiplicationBudgetMichaelTschannen1AranKhanna2AnimaAnandkumar23Abstractandreducingthenumericalprecisionofweightsandactiva-tions(seeSection1.1foradetailedoverview).Al...
StabilizingGradientsforDeepNeuralNetworksviaEfficientSVDParameterizationJiongZhang1QiLei1InderjitS.Dhillon12Abstract1.IntroductionVanishingandexplodinggradientsaretwooftheDeepneuralnetworkshaveachi...
Spotlight:OptimizingDevicePlacementforTrainingDeepNeuralNetworksYuanxiangGao12LiChen1BaochunLi1AbstractSample...TrainingDeepneuralnetworks(DNNs)requiresTrainingtimeanincreasingamountofcomputationre...
SplineFiltersForEnd-to-EndDeepLearningRandallBalestriero1RomainCosentino1Herve´Glotin2RichardBaraniuk1AbstractWhileprovidingafullyautomatedapproach,DNNs’per-formancesdependonthenumberofperturbati...
SoftActor-Critic:Off-PolicyMaximumEntropyDeepReinforcementLearningwithaStochasticActorTuomasHaarnoja1AurickZhou1PieterAbbeel1SergeyLevine1Abstractnetworksholdsthepromiseofautomatingawiderangeofdeci...
RegretMinimizationforPartiallyObservableDeepReinforcementLearningPeterJin1KurtKeutzer1SergeyLevine1Abstractfunction-basedmethods.Somepolicygradientmethodssuchasadvantageactor-critic(Mnihetal.,2016)...
Pseudo-taskAugmentation:FromDeepMultitaskLearningtoIntrataskSharing—andBackElliotMeyerson12RistoMiikkulainen12Abstract2018).DeepMTLreliesontrainingsignalsfrommultipledatasetstotrainDeepstructureth...
QMIX:MonotonicValueFunctionFactorisationforDeepMulti-AgentReinforcementLearningTabishRashid1MikayelSamvelyan2ChristianSchroederdeWitt1GregoryFarquhar1JakobFoerster1ShimonWhiteson1Abstract(a)5Marine...
PredictandConstrain:ModelingCardinalityinDeepStructuredPredictionNatalyBrukhim1AmirGloberson1Abstractcardinality(Tarlowetal.2012;Tarlowetal.2010;Milchetal.2008;Swerskyetal.2012;Guptaetal.2007).Name...
prDeep:RobustPhaseRetrievalwithaFlexibleDeepNetworkChristopherA.Metzler1PhilipSchniter2AshokVeeraraghavan1RichardG.Baraniuk1AbstractPRalgorithmswerefirstdevelopedintheearly1970sandhavebeencontinuou...
OptimizationLandscapeandExpressivityofDeepCNNsQuynhNguyen1MatthiasHein2AbstractTable1.Themaximumwidthofalllayersinseveralstate-of-the-artCNNarchitecturescomparedwiththesizeofImageNetdatasetWeanalyz...
OntheOptimizationofDeepNetworks:ImplicitAccelerationbyOverparameterizationSanjeevArora12NadavCohen2EladHazan13AbstractGiventhelongstandingconsensusonexpressivenessvs.op-timizationtrade-offs,thispap...
NotAllSamplesAreCreatedEqual:DeepLearningwithImportanceSamplingAngelosKatharopoulos12Franc¸oisFleuret12Abstractmodel.Tothisend,weproposeanovelimportancesamplingschemethatacceleratesthetrainingofan...
MentorNet:LearningData-DrivenCurriculumforVeryDeepNeuralNetworksonCorruptedLabelsLuJiang1ZhengyuanZhou2ThomasLeung1Li-JiaLi1LiFei-Fei12Abstractonthecleantestdata.Althoughlearningmodelsonweaklylabel...
LearningtoReweightExamplesforRobustDeepLearningMengyeRen12WenyuanZeng12BinYang12RaquelUrtasun12Abstractdifferentforms.Classimbalanceinthetrainingsetisaverycommonexample.Inapplicationssuchasobjectde...