| GradInit: Learning to Initialize Neural Networks for Stable and Efficient Training | Feb 16, 2021 | Image ClassificationLanguage Modeling | CodeCode Available | 1 |
| COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining | Feb 16, 2021 | Contrastive LearningGPU | CodeCode Available | 1 |
| DOBF: A Deobfuscation Pre-Training Objective for Programming Languages | Feb 15, 2021 | Code SearchCode Translation | CodeCode Available | 1 |
| End-to-end Audio-visual Speech Recognition with Conformers | Feb 12, 2021 | Audio-Visual Speech RecognitionAutomatic Speech Recognition (ASR) | CodeCode Available | 1 |
| Unsupervised Extractive Summarization using Pointwise Mutual Information | Feb 11, 2021 | ArticlesExtractive Summarization | CodeCode Available | 1 |
| Proof Artifact Co-training for Theorem Proving with Language Models | Feb 11, 2021 | Automated Theorem ProvingImitation Learning | CodeCode Available | 1 |
| AuGPT: Auxiliary Tasks and Data Augmentation for End-To-End Dialogue with Pre-Trained Language Models | Feb 9, 2021 | DiversityEnd-To-End Dialogue Modelling | CodeCode Available | 1 |
| Unifying Vision-and-Language Tasks via Text Generation | Feb 4, 2021 | Conditional Text GenerationDecoder | CodeCode Available | 1 |
| Phoneme-BERT: Joint Language Modelling of Phoneme Sequence and ASR Transcript | Feb 1, 2021 | intent-classificationIntent Classification | CodeCode Available | 1 |
| Generative Spoken Language Modeling from Raw Audio | Feb 1, 2021 | DecoderLanguage Modeling | CodeCode Available | 1 |
| LESA: Linguistic Encapsulation and Semantic Amalgamation Based Generalised Claim Detection from Online Content | Jan 28, 2021 | Argument MiningLanguage Modeling | CodeCode Available | 1 |
| PolyLM: Learning about Polysemy through Language Modeling | Jan 25, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| EGFI: Drug-Drug Interaction Extraction and Generation with Fusion of Enriched Entity and Sentence Information | Jan 25, 2021 | ClassificationDrug–drug Interaction Extraction | CodeCode Available | 1 |
| CPT: Efficient Deep Neural Network Training via Cyclic Precision | Jan 25, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| PalmTree: Learning an Assembly Language Model for Instruction Embedding | Jan 21, 2021 | Boundary DetectionCode Search | CodeCode Available | 1 |
| Persistent Anti-Muslim Bias in Large Language Models | Jan 14, 2021 | Adversarial TextLanguage Modeling | CodeCode Available | 1 |
| Implicit Unlikelihood Training: Improving Neural Text Generation with Reinforcement Learning | Jan 11, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processing | Jan 9, 2021 | Dependency ParsingLanguage Modeling | CodeCode Available | 1 |
| Multitask Learning for Emotion and Personality Detection | Jan 7, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| PhoNLP: A joint multi-task learning model for Vietnamese part-of-speech tagging, named entity recognition and dependency parsing | Jan 5, 2021 | Dependency ParsingLanguage Modeling | CodeCode Available | 1 |
| Outline to Story: Fine-grained Controllable Story Generation from Cascaded Events | Jan 4, 2021 | Keyword ExtractionLanguage Modeling | CodeCode Available | 1 |
| KM-BART: Knowledge Enhanced Multimodal BART for Visual Commonsense Generation | Jan 2, 2021 | Knowledge GraphsLanguage Modeling | CodeCode Available | 1 |
| CDLM: Cross-Document Language Modeling | Jan 2, 2021 | Citation RecommendationCoreference Resolution | CodeCode Available | 1 |
| Subformer: Exploring Weight Sharing for Parameter Efficiency in Generative Transformers | Jan 1, 2021 | Abstractive Text SummarizationLanguage Modeling | CodeCode Available | 1 |
| Discovering Autoregressive Orderings with Variational Inference | Jan 1, 2021 | Code GenerationImage Captioning | CodeCode Available | 1 |
| BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Bangla | Jan 1, 2021 | Document ClassificationLanguage Modeling | CodeCode Available | 1 |
| Not All Memories are Created Equal: Learning to Expire | Jan 1, 2021 | AllLanguage Modeling | CodeCode Available | 1 |
| WARP: Word-level Adversarial ReProgramming | Jan 1, 2021 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| K-PLUG: KNOWLEDGE-INJECTED PRE-TRAINED LANGUAGE MODEL FOR NATURAL LANGUAGE UNDERSTANDING AND GENERATION | Jan 1, 2021 | ChatbotDecoder | CodeCode Available | 1 |
| Shortformer: Better Language Modeling using Shorter Inputs | Dec 31, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Unified Mandarin TTS Front-end Based on Distilled BERT Model | Dec 31, 2020 | Knowledge DistillationLanguage Modeling | CodeCode Available | 1 |
| AraGPT2: Pre-Trained Transformer for Arabic Language Generation | Dec 31, 2020 | ArticlesLanguage Modeling | CodeCode Available | 1 |
| AraELECTRA: Pre-Training Text Discriminators for Arabic Language Understanding | Dec 31, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Generating Query Focused Summaries from Query-Free Resources | Dec 29, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning | Dec 22, 2020 | Generalization BoundsLanguage Modeling | CodeCode Available | 1 |
| RealFormer: Transformer Likes Residual Attention | Dec 21, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training | Dec 18, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Binary Black-box Evasion Attacks Against Deep Learning-based Static Malware Detectors with Adversarial Byte-Level Language Model | Dec 14, 2020 | Deep LearningFeature Engineering | CodeCode Available | 1 |
| Extracting Training Data from Large Language Models | Dec 14, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Towards Neural Programming Interfaces | Dec 10, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Fusing Context Into Knowledge Graph for Commonsense Question Answering | Dec 9, 2020 | Common Sense ReasoningKnowledge Graphs | CodeCode Available | 1 |
| TAP: Text-Aware Pre-training for Text-VQA and Text-Caption | Dec 8, 2020 | Caption GenerationLanguage Modeling | CodeCode Available | 1 |
| Pre-training Protein Language Models with Label-Agnostic Binding Pairs Enhances Performance in Downstream Tasks | Dec 5, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Multi-Task Learning for Knowledge Graph Completion with Pre-trained Language Models | Dec 1, 2020 | Knowledge Graph CompletionKnowledge Graphs | CodeCode Available | 1 |
| End-to-End Automatic Speech Recognition for Gujarati | Dec 1, 2020 | Automatic Speech RecognitionAutomatic Speech Recognition (ASR) | CodeCode Available | 1 |
| Kungfupanda at SemEval-2020 Task 12: BERT-Based Multi-TaskLearning for Offensive Language Detection | Dec 1, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Try to Substitute: An Unsupervised Chinese Word Sense Disambiguation Method Based on HowNet | Dec 1, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Retrieving Skills from Job Descriptions: A Language Model Based Extreme Multi-label Classification Framework | Dec 1, 2020 | Extreme Multi-Label ClassificationLanguage Modeling | CodeCode Available | 1 |
| CPM: A Large-scale Generative Chinese Pre-trained Language Model | Dec 1, 2020 | Cloze TestLanguage Modeling | CodeCode Available | 1 |
| SentiX: A Sentiment-Aware Pre-Trained Model for Cross-Domain Sentiment Analysis | Dec 1, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 1 |