SparseBERT Rethinking the Importance Analysis in Self-attention

SparseBERT:RethinkingtheImportanceAnalysisinSelf-attentionHanShi1JiahuiGao2XiaozheRen3HangXu3XiaodanLiang4ZhenguoLi3JamesT.Kwok1AbstractincludetheBERT(Devlinetal.,2019),whichachievesstate-of-the-ar...