Linear Transformers Are Secretly Fast Weight Programmers

LinearTransformersAreSecretlyFastWeightProgrammersImanolSchlag∗1KazukiIrie∗1Ju¨rgenSchmidhuber1Abstractfieldnetwork(Ramsaueretal.,2021;Krotov&Hopfield,2016;Demircigiletal.,2017).Itextendsaformof...