| TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue | Apr 15, 2020 | Dialogue State TrackingIntent Detection | CodeCode Available | 1 |
| ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators | Mar 23, 2020 | GPULanguage Modeling | CodeCode Available | 1 |
| Self-Supervised Log Parsing | Mar 17, 2020 | Anomaly DetectionFault Detection | CodeCode Available | 2 |
| Talking-Heads Attention | Mar 5, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| XGPT: Cross-modal Generative Pre-Training for Image Captioning | Mar 3, 2020 | Data AugmentationDenoising | —Unverified | 0 |
| REALM: Retrieval-Augmented Language Model Pre-Training | Feb 10, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| ImageBERT: Cross-modal Pre-training with Large-scale Weak-supervised Image-Text Data | Jan 22, 2020 | Image RetrievalImage-text matching | —Unverified | 0 |
| Retrieval Augmented Language Model Pre-Training | Jan 1, 2020 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Pre-Training of Deep Bidirectional Protein Sequence Representations with Structural Information | Nov 25, 2019 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| Distilling Knowledge Learned in BERT for Text Generation | Nov 10, 2019 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| Contextual Grounding of Natural Language Entities in Images | Nov 5, 2019 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| Emerging Cross-lingual Structure in Pretrained Language Models | Nov 4, 2019 | Cross-Lingual TransferLanguage Modeling | —Unverified | 0 |
| On the Cross-lingual Transferability of Monolingual Representations | Oct 25, 2019 | Cross-Lingual Question AnsweringLanguage Modeling | CodeCode Available | 0 |
| UNITER: Learning UNiversal Image-TExt Representations | Sep 25, 2019 | Image-text matchingImage-text Retrieval | —Unverified | 0 |
| UNITER: UNiversal Image-TExt Representation Learning | Sep 25, 2019 | Image-text matchingImage-text Retrieval | CodeCode Available | 1 |
| AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models | Sep 19, 2019 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| Specializing Unsupervised Pretraining Models for Word-Level Semantic Similarity | Sep 5, 2019 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives | Sep 3, 2019 | Language ModelingLanguage Modelling | —Unverified | 0 |
| LXMERT: Learning Cross-Modality Encoder Representations from Transformers | Aug 20, 2019 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-training | Aug 16, 2019 | Image-text matchingImage-text Retrieval | —Unverified | 0 |
| BERT Masked Language Modeling for Co-reference Resolution | Aug 1, 2019 | General ClassificationLanguage Modeling | —Unverified | 0 |
| Selfie: Self-supervised Pretraining for Image Embedding | Jun 7, 2019 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| Adversarial Generation and Encoding of Nested Texts | Jun 1, 2019 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Mask-Predict: Parallel Decoding of Conditional Masked Language Models | Apr 19, 2019 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Unsupervised Domain Adaptation of Contextualized Embeddings for Sequence Labeling | Apr 4, 2019 | Domain AdaptationLanguage Modeling | CodeCode Available | 0 |