| COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining | Feb 16, 2021 | Contrastive LearningGPU | CodeCode Available | 1 |
| GradInit: Learning to Initialize Neural Networks for Stable and Efficient Training | Feb 16, 2021 | Image ClassificationLanguage Modeling | CodeCode Available | 1 |
| DOBF: A Deobfuscation Pre-Training Objective for Programming Languages | Feb 15, 2021 | Code SearchCode Translation | CodeCode Available | 1 |
| End-to-end Audio-visual Speech Recognition with Conformers | Feb 12, 2021 | Audio-Visual Speech RecognitionAutomatic Speech Recognition (ASR) | CodeCode Available | 1 |
| Proof Artifact Co-training for Theorem Proving with Language Models | Feb 11, 2021 | Automated Theorem ProvingImitation Learning | CodeCode Available | 1 |
| Unsupervised Extractive Summarization using Pointwise Mutual Information | Feb 11, 2021 | ArticlesExtractive Summarization | CodeCode Available | 1 |
| AuGPT: Auxiliary Tasks and Data Augmentation for End-To-End Dialogue with Pre-Trained Language Models | Feb 9, 2021 | DiversityEnd-To-End Dialogue Modelling | CodeCode Available | 1 |
| Unifying Vision-and-Language Tasks via Text Generation | Feb 4, 2021 | Conditional Text GenerationDecoder | CodeCode Available | 1 |
| Phoneme-BERT: Joint Language Modelling of Phoneme Sequence and ASR Transcript | Feb 1, 2021 | intent-classificationIntent Classification | CodeCode Available | 1 |
| Generative Spoken Language Modeling from Raw Audio | Feb 1, 2021 | DecoderLanguage Modeling | CodeCode Available | 1 |
