| TensorCoder: Dimension-Wise Attention via Tensor Representation for Natural Language Modeling | Jul 28, 2020 | Language ModelingLanguage Modelling | —Unverified | 0 |
| I-BERT: Inductive Generalization of Transformer to Arbitrary Context Lengths | Jun 18, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| GMAT: Global Memory Augmentation for Transformers | Jun 5, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| Masked Language Modeling for Proteins via Linearly Scalable Long-Context Transformers | Jun 5, 2020 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Segatron: Segment-aware Transformer for Language Modeling and Understanding | Jun 2, 2020 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Position Masking for Language Models | Jun 2, 2020 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning | Apr 29, 2020 | AllHellaSwag | —Unverified | 0 |
| UHH-LT at SemEval-2020 Task 12: Fine-Tuning of Pre-Trained Transformer Networks for Offensive Language Detection | Apr 23, 2020 | Domain AdaptationGeneral Classification | —Unverified | 0 |
| XGPT: Cross-modal Generative Pre-Training for Image Captioning | Mar 3, 2020 | Data AugmentationDenoising | —Unverified | 0 |
| ImageBERT: Cross-modal Pre-training with Large-scale Weak-supervised Image-Text Data | Jan 22, 2020 | Image RetrievalImage-text matching | —Unverified | 0 |
| Retrieval Augmented Language Model Pre-Training | Jan 1, 2020 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Pre-Training of Deep Bidirectional Protein Sequence Representations with Structural Information | Nov 25, 2019 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| Distilling Knowledge Learned in BERT for Text Generation | Nov 10, 2019 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| Contextual Grounding of Natural Language Entities in Images | Nov 5, 2019 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| Emerging Cross-lingual Structure in Pretrained Language Models | Nov 4, 2019 | Cross-Lingual TransferLanguage Modeling | —Unverified | 0 |
| On the Cross-lingual Transferability of Monolingual Representations | Oct 25, 2019 | Cross-Lingual Question AnsweringLanguage Modeling | CodeCode Available | 0 |
| UNITER: Learning UNiversal Image-TExt Representations | Sep 25, 2019 | Image-text matchingImage-text Retrieval | —Unverified | 0 |
| AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models | Sep 19, 2019 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| Specializing Unsupervised Pretraining Models for Word-Level Semantic Similarity | Sep 5, 2019 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives | Sep 3, 2019 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-training | Aug 16, 2019 | Image-text matchingImage-text Retrieval | —Unverified | 0 |
| BERT Masked Language Modeling for Co-reference Resolution | Aug 1, 2019 | General ClassificationLanguage Modeling | —Unverified | 0 |
| Selfie: Self-supervised Pretraining for Image Embedding | Jun 7, 2019 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| Adversarial Generation and Encoding of Nested Texts | Jun 1, 2019 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Unsupervised Domain Adaptation of Contextualized Embeddings for Sequence Labeling | Apr 4, 2019 | Domain AdaptationLanguage Modeling | CodeCode Available | 0 |