| TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue | Apr 15, 2020 | Dialogue State TrackingIntent Detection | CodeCode Available | 1 |
| ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators | Mar 23, 2020 | GPULanguage Modeling | CodeCode Available | 1 |
| Self-Supervised Log Parsing | Mar 17, 2020 | Anomaly DetectionFault Detection | CodeCode Available | 2 |
| Talking-Heads Attention | Mar 5, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| XGPT: Cross-modal Generative Pre-Training for Image Captioning | Mar 3, 2020 | Data AugmentationDenoising | —Unverified | 0 |
| REALM: Retrieval-Augmented Language Model Pre-Training | Feb 10, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| ImageBERT: Cross-modal Pre-training with Large-scale Weak-supervised Image-Text Data | Jan 22, 2020 | Image RetrievalImage-text matching | —Unverified | 0 |
| Retrieval Augmented Language Model Pre-Training | Jan 1, 2020 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Pre-Training of Deep Bidirectional Protein Sequence Representations with Structural Information | Nov 25, 2019 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| Distilling Knowledge Learned in BERT for Text Generation | Nov 10, 2019 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
