| Cross-lingual Transfer Learning for Pre-trained Contextualized Language Models | Jan 1, 2021 | Cross-Lingual TransferLanguage Modeling | —Unverified | 0 |
| Domain-slot Relationship Modeling using a Pre-trained Language Encoder for Multi-Domain Dialogue State Tracking | Jan 1, 2021 | Dialogue State TrackingLanguage Modeling | —Unverified | 0 |
| Context-Aware Temperature for Language Modeling | Jan 1, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| BROS: A Pre-trained Language Model for Understanding Texts in Document | Jan 1, 2021 | DecoderDiversity | —Unverified | 0 |
| Towards Practical Second Order Optimization for Deep Learning | Jan 1, 2021 | Click-Through Rate PredictionCPU | —Unverified | 0 |
| Transformer-QL: A Step Towards Making Transformer Network Quadratically Large | Jan 1, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Translation Memory Guided Neural Machine Translation | Jan 1, 2021 | DecoderLanguage Modeling | —Unverified | 0 |
| Non-iterative Parallel Text Generation via Glancing Transformer | Jan 1, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Refine and Imitate: Reducing Repetition and Inconsistency in Dialogue Generation via Reinforcement Learning and Human Demonstration | Jan 1, 2021 | Dialogue GenerationLanguage Modeling | —Unverified | 0 |
| TaskSet: A Dataset of Optimization Tasks | Jan 1, 2021 | Diversityimage-classification | CodeCode Available | 0 |
| Pretrain Knowledge-Aware Language Models | Jan 1, 2021 | Knowledge ProbingLanguage Modeling | —Unverified | 0 |
| Synthesizer: Rethinking Self-Attention for Transformer Models | Jan 1, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| MONGOOSE: A Learnable LSH Framework for Efficient Neural Network Training | Jan 1, 2021 | Efficient Neural NetworkLanguage Modeling | —Unverified | 0 |
| Subformer: A Parameter Reduced Transformer | Jan 1, 2021 | Abstractive Text SummarizationDecoder | —Unverified | 0 |
| SEQUENCE-LEVEL FEATURES: HOW GRU AND LSTM CELLS CAPTURE N-GRAMS | Jan 1, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Memory Representation in Transformer | Jan 1, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Adding Recurrence to Pretrained Transformers | Jan 1, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Discovering Autoregressive Orderings with Variational Inference | Jan 1, 2021 | Code GenerationImage Captioning | CodeCode Available | 1 |
| Block Skim Transformer for Efficient Question Answering | Jan 1, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| K-PLUG: KNOWLEDGE-INJECTED PRE-TRAINED LANGUAGE MODEL FOR NATURAL LANGUAGE UNDERSTANDING AND GENERATION | Jan 1, 2021 | ChatbotDecoder | CodeCode Available | 1 |
| Learning Chess Blindfolded | Jan 1, 2021 | Domain ProbingGame of Chess | —Unverified | 0 |
| Representation and Bias in Multilingual NLP: Insights from Controlled Experiments on Conditional Language Modeling | Jan 1, 2021 | FairnessLanguage Modeling | —Unverified | 0 |
| On the use of linguistic similarities to improve Neural Machine Translation for African Languages | Jan 1, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Transformer protein language models are unsupervised structure learners | Jan 1, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| ROMUL: Scale Adaptative Population Based Training | Jan 1, 2021 | Data Augmentationimage-classification | —Unverified | 0 |
| Universal Sentence Representations Learning with Conditional Masked Language Model | Jan 1, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Syntactic Relevance XLNet Word Embedding Generation in Low-Resource Machine Translation | Jan 1, 2021 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Not All Memories are Created Equal: Learning to Expire | Jan 1, 2021 | AllLanguage Modeling | CodeCode Available | 1 |
| SCoRe: Pre-Training for Context Representation in Conversational Semantic Parsing | Jan 1, 2021 | Dialogue State TrackingLanguage Modeling | —Unverified | 0 |
| The Pile: An 800GB Dataset of Diverse Text for Language Modeling | Dec 31, 2020 | DiversityLanguage Modeling | CodeCode Available | 2 |
| Unified Mandarin TTS Front-end Based on Distilled BERT Model | Dec 31, 2020 | Knowledge DistillationLanguage Modeling | CodeCode Available | 1 |
| XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders | Dec 31, 2020 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Studying Strategically: Learning to Mask for Closed-book QA | Dec 31, 2020 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Verb Knowledge Injection for Multilingual Event Processing | Dec 31, 2020 | Event ExtractionLanguage Modeling | —Unverified | 0 |
| Shortformer: Better Language Modeling using Shorter Inputs | Dec 31, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| AraGPT2: Pre-Trained Transformer for Arabic Language Generation | Dec 31, 2020 | ArticlesLanguage Modeling | CodeCode Available | 1 |
| Directed Beam Search: Plug-and-Play Lexically Constrained Language Generation | Dec 31, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| ERNIE-Doc: A Retrospective Long-Document Modeling Transformer | Dec 31, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| CoCoLM: COmplex COmmonsense Enhanced Language Model with Discourse Relations | Dec 31, 2020 | Knowledge GraphsLanguage Modeling | CodeCode Available | 0 |
| AraELECTRA: Pre-Training Text Discriminators for Arabic Language Understanding | Dec 31, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Enhancing Pre-trained Language Model with Lexical Simplification | Dec 30, 2020 | DiversityGeneral Classification | —Unverified | 0 |
| Can Sequence-to-Sequence Models Crack Substitution Ciphers? | Dec 30, 2020 | DeciphermentLanguage Identification | —Unverified | 0 |
| SemGloVe: Semantic Co-occurrences for GloVe from BERT | Dec 30, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding | Dec 29, 2020 | Document Image ClassificationDocument Layout Analysis | CodeCode Available | 0 |
| Generating Adversarial Examples in Chinese Texts Using Sentence-Pieces | Dec 29, 2020 | Language ModelingLanguage Modelling | —Unverified | 0 |
| CMV-BERT: Contrastive multi-vocab pretraining of BERT | Dec 29, 2020 | Contrastive LearningLanguage Modeling | —Unverified | 0 |
| Generating Query Focused Summaries from Query-Free Resources | Dec 29, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| General Mechanism of Evolution Shared by Proteins and Words | Dec 28, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| Universal Sentence Representation Learning with Conditional Masked Language Model | Dec 28, 2020 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Assessment of the Relative Importance of different hyper-parameters of LSTM for an IDS | Dec 26, 2020 | Intrusion DetectionLanguage Modeling | —Unverified | 0 |