Efficient Training of BERT by Progressively Stacking

EfficientTrainingofBERTbyProgressivelyStackingLinyuanGong1DiHe1ZhuohanLi1TaoQin2LiweiWang13Tie-YanLiu2Abstractespeciallyindomainsthatrequireparticularexpertise.Unsupervisedpre-trainingiscommonlyuse...