LeveragingLanguagetoLearnProgramAbstractionsandSearchHeuristicsCatherineWong1KevinEllis2JoshuaB.Tenenbaum13JacobAndreas1AbstractmodelsusingnaturalLanguagesupervision.InLAPS,lan-guageguideslearningo...
LearningTransferableVisualModelsFromNaturalLanguageSupervisionAlecRadford1JongWookKim1ChrisHallacy1AdityaRamesh1GabrielGoh1SandhiniAgarwal1GirishSastry1AmandaAskell1PamelaMishkin1JackClark1Gretchen...
Grey-boxExtractionofNaturalLanguageModelsSantiagoZanella-Béguelin1ShrutiTople1AndrewPaverd12BorisKöpf1Abstractquery-responsepairsobtainedviathemodel’sinferenceAPI,thuseffectivelycircumventingthe...
GroundingLanguagetoEntitiesandDynamicsforGeneralizationinReinforcementLearningAustinW.Hanjie1VictorZhong2KarthikNarasimhan1Abstractunseenentitiesordynamics(Narasimhanetal.,2018;Zhongetal.,2020).Whi...
Few-shotLanguageCoordinationbyModelingTheoryofMindHaoZhu1GrahamNeubig1YonatanBisk1Abstractprocess,withawidevarietyofworksexaminingcommuni-cationbetweenagentsviaeithercompletelyartificialemer-Nomani...
ALanguageforCounterfactualGenerativeModelsZennaTavares1JamesKoppel1XinZhang2RiaDas1ArmandoSolar-Lezama1AbstractFigure1:Aspeedingdriver(Left:driver’sview)crashesintoapedestrian(yellow)emergingfromb...
StructuralLanguageModelsofCodeUriAlon1RoySadaka1OmerLevy23EranYahav1Abstractvlinetal.,2017;Ellisetal.,2019),whileotherrecentap-proachesgeneratecodeingeneralLanguageslikeJavaandWeaddresstheproblemof...
REALM:Retrieval-AugmentedLanguageModelPre-TrainingKelvinGuu1KentonLee1ZoraTung1PanupongPasupat1Ming-WeiChang1AbstractFigure1.REALMaugmentsLanguagemodelpre-trainingwithaneuralknowledgeretrieverthatr...
RecurrentHierarchicalTopic-GuidedRNNforLanguageGenerationDandanGuo1BoChen1RuiyingLu1MingyuanZhou2Abstractlatentrepresentation,theytypicallytreateachdocumentasabagofwords(BoW),ignoringwordorder(Grif...
UNILMv2:Pseudo-MaskedLanguageModelsforUnifiedLanguageModelPre-TrainingHangboBao1LiDong2FuruWei2WenhuiWang2NanYang2XiaodongLiu2YuWang2SonghaoPiao1JianfengGao2MingZhou2Hsiao-WuenHon2Abstract?1?2?3...
ExplainableandDiscourseTopic-awareNeuralLanguageUnderstandingYatinChaudhary12HinrichSchütze2PankajGupta1Abstracttributionoverwordsinvocabulary.Beyondadocumentrepresentation,topicmodelsalsoofferint...
EmergenceofSeparableManifoldsinDeepLanguageRepresentationsJonathanMamou1HangLe2MiguelADelRio2CoryStephenson1HanlinTang1YoonKim3SueYeonChung24Abstract1.IntroductionandRelatedWorkDeepneuralnetworks(D...
CounteringLanguageDriftwithSeededIteratedLearningYuchenLu1SoumyeSinghal1FlorianStrub2OlivierPietquin3AaronCourville14Abstractrisetoinconsistentbehaviorsingoal-orientedLanguageset-tings,suchasquesti...
MASS:MaskedSequencetoSequencePre-trainingforLanguageGenerationKaitaoSong1XuTan2TaoQin2JianfengLu1Tie-YanLiu2Abstractwhilepre-traininghasplentyofdata(Girshicketal.,2014;Szegedyetal.,2015;Ouyangetal....
ImprovingNeuralLanguageModelingviaAdversarialTrainingDilinWang1ChengyueGong1QiangLiu1AbstractUnfortunately,amajorchallengeintraininglargescaleRNN-basedLanguagemodelsistheirtendencytooverfit;Recentl...
DeepResidualOutputLayersforNeuralLanguageGenerationNikolaosPappas1JamesHenderson1Abstractbeddingstocapturethesimilaritystructureoftheoutputlabelspace,sothatdataforsimilarlabelscanhelpclassi-Manytas...
AStatisticalInvestigationofLongMemoryinLanguageandMusicAlexanderGreaves-Tunnell1ZaidHarchaoui1Abstractundoubtedlyhelpful,suchheuristicsarerarelydefinedwithrespecttoanunderlyingmathematicalorstatist...
LanguageModelingwithGatedConvolutionalNetworksYannN.Dauphin1AngelaFan1MichaelAuli1DavidGrangier1Abstractoutperformclassicaln-gramLanguagemodels(Kneser&Ney,1995;Chen&Goodman,1996).Theseclassicalmod-...