| JavaBERT: Training a transformer-based model for the Java programming language | Oct 20, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| NormFormer: Improved Transformer Pretraining with Extra Normalization | Oct 18, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| DS-TOD: Efficient Domain Specialization for Task Oriented Dialog | Oct 15, 2021 | dialog state trackingLanguage Modeling | CodeCode Available | 0 |
| Dict-BERT: Enhancing Language Model Pre-training with Dictionary | Oct 13, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| Maximizing Efficiency of Language Model Pre-training for Learning Representation | Oct 13, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Multi-Modal Pre-Training for Automated Speech Recognition | Oct 12, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Contextualized Semantic Distance between Highly Overlapped Texts | Oct 4, 2021 | Domain AdaptationLanguage Modeling | CodeCode Available | 0 |
| Image BERT Pre-training with Online Tokenizer | Sep 29, 2021 | image-classificationImage Classification | —Unverified | 0 |
| Predicting Attention Sparsity in Transformers | Sep 24, 2021 | DecoderLanguage Modeling | —Unverified | 0 |
| MLIM: Vision-and-Language Model Pre-training with Masked Language and Image Modeling | Sep 24, 2021 | Image ReconstructionLanguage Modeling | —Unverified | 0 |
