WhiteningforSelf-SupervisedRepresentationLearningAleksandrErmolov1AliaksandrSiarohin1EnverSangineto1NicuSebe1Abstractofwordsinasentenceisusedtolearnalanguagemodel(Mikolovetal.,2013a;b;Devlinetal.,2...
WhiteningandSecondOrderOptimizationBothMakeInformationintheDatasetUnusableDuringTraining,andCanReduceorPreventGeneralizationNehaS.Wadia1DanielDuckworth2SamuelS.Schoenholz2EthanDyer2JaschaSohl-Dicks...