Poolingformer Long Document Modeling with Pooling Attention

Poolingformer:LongDocumentModelingwithPoolingAttentionHangZhang12YeyunGong3YelongShen4WeishengLi5JianchengLv1NanDuan3WeizhuChen4Abstract(a)Single-levellocalattention(b)Two-levelpoolingattentionInth...