| Toward Efficient Language Model Pretraining and Downstream Adaptation via Self-Evolution: A Case Study on SuperGLUE | Dec 4, 2022 | Common Sense Reasoningcoreference-resolution | —Unverified | 0 |
| Global memory transformer for processing long documents | Dec 3, 2022 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Nonparametric Masked Language Modeling | Dec 2, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Comparison Study Between Token Classification and Sequence Classification In Text Classification | Nov 25, 2022 | ClassificationLanguage Modeling | —Unverified | 0 |
| Seeing What You Miss: Vision-Language Pre-training with Semantic Completion Learning | Nov 24, 2022 | cross-modal alignmentImage-text Retrieval | CodeCode Available | 1 |
| Self-supervised vision-language pretraining for Medical visual question answering | Nov 24, 2022 | Contrastive LearningImage-text matching | CodeCode Available | 1 |
| Unified Multimodal Model with Unlikelihood Training for Visual Dialog | Nov 23, 2022 | Answer GenerationChatbot | CodeCode Available | 1 |
| Enhancing Crisis-Related Tweet Classification with Entity-Masked Language Modeling and Multi-Task Learning | Nov 21, 2022 | Hierarchical Multi-label ClassificationLanguage Modeling | CodeCode Available | 0 |
| Leveraging per Image-Token Consistency for Vision-Language Pre-training | Nov 20, 2022 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Embracing Ambiguity: Improving Similarity-oriented Tasks with Contextual Synonym Knowledge | Nov 20, 2022 | Entity LinkingLanguage Modeling | —Unverified | 0 |
| HanTrans: An Empirical Study on Cross-Era Transferability of Chinese Pre-trained Language Model | Nov 1, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| Reduce, Reuse, Recycle: Improving Training Efficiency with Distillation | Nov 1, 2022 | image-classificationImage Classification | —Unverified | 0 |
| CodeEditor: Learning to Edit Source Code with Pre-trained Models | Oct 31, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| Leveraging Label Correlations in a Multi-label Setting: A Case Study in Emotion | Oct 28, 2022 | Emotion RecognitionLanguage Modeling | CodeCode Available | 1 |
| Retrieval Oriented Masking Pre-training Language Model for Dense Passage Retrieval | Oct 27, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 2 |
| Towards Unifying Reference Expression Generation and Comprehension | Oct 24, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| Generative Prompt Tuning for Relation Classification | Oct 22, 2022 | ClassificationLanguage Modeling | CodeCode Available | 1 |
| SpaBERT: A Pretrained Language Model from Geographic Data for Geo-Entity Representation | Oct 21, 2022 | Entity LinkingEntity Typing | —Unverified | 0 |
| InforMask: Unsupervised Informative Masking for Language Model Pretraining | Oct 21, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Deep Bidirectional Language-Knowledge Graph Pretraining | Oct 17, 2022 | Common Sense ReasoningKnowledge Graphs | CodeCode Available | 2 |
| Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training | Oct 14, 2022 | HallucinationImage Augmentation | CodeCode Available | 0 |
| Mixture of Attention Heads: Selecting Attention Heads Per Token | Oct 11, 2022 | Computational EfficiencyLanguage Modeling | CodeCode Available | 1 |
| MAP: Multimodal Uncertainty-Aware Vision-Language Pre-training Model | Oct 11, 2022 | Contrastive LearningImage-text matching | CodeCode Available | 1 |
| Revisiting and Advancing Chinese Natural Language Understanding with Accelerated Heterogeneous Knowledge Pre-training | Oct 11, 2022 | GPUKnowledge Graphs | CodeCode Available | 0 |
| The Effectiveness of Masked Language Modeling and Adapters for Factual Knowledge Injection | Oct 3, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| KUL@SMM4H’22: Template Augmented Adaptive Pre-training for Tweet Classification | Oct 1, 2022 | Data AugmentationLanguage Modeling | —Unverified | 0 |
| A Closer Look at Parameter Contributions When Training Neural Language and Translation Models | Oct 1, 2022 | Causal Language ModelingLanguage Modeling | —Unverified | 0 |
| Taking Actions Separately: A Bidirectionally-Adaptive Transfer Learning Method for Low-Resource Neural Machine Translation | Oct 1, 2022 | Generative Adversarial NetworkLanguage Modeling | —Unverified | 0 |
| Towards Making the Most of Pre-trained Translation Model for Quality Estimation | Oct 1, 2022 | DenoisingLanguage Modeling | —Unverified | 0 |
| Bidirectional Language Models Are Also Few-shot Learners | Sep 29, 2022 | DenoisingLanguage Modeling | —Unverified | 0 |
| IDIAPers @ Causal News Corpus 2022: Efficient Causal Relation Identification Through a Prompt-based Few-shot Approach | Sep 8, 2022 | Event Causality IdentificationLanguage Modeling | CodeCode Available | 0 |
| TransPolymer: a Transformer-based language model for polymer property predictions | Sep 3, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Learning Better Masking for Better Language Model Pre-training | Aug 23, 2022 | DenoisingLanguage Modeling | CodeCode Available | 0 |
| Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks | Aug 22, 2022 | AllCross-Modal Retrieval | CodeCode Available | 0 |
| GRIT-VLP: Grouped Mini-batch Sampling for Efficient Vision and Language Pre-training | Aug 8, 2022 | Image-text matchingLanguage Modeling | CodeCode Available | 1 |
| Towards No.1 in CLUE Semantic Matching Challenge: Pre-trained Language Model Erlangshen with Propensity-Corrected Loss | Aug 5, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 4 |
| Masked Vision and Language Modeling for Multi-modal Representation Learning | Aug 3, 2022 | cross-modal alignmentLanguage Modeling | —Unverified | 0 |
| Augmenting Vision Language Pretraining by Learning Codebook with Visual Semantics | Jul 31, 2022 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Boosting Point-BERT by Multi-choice Tokens | Jul 27, 2022 | Few-Shot LearningLanguage Modeling | CodeCode Available | 0 |
| Unsupervised pre-training of graph transformers on patient population graphs | Jul 21, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| STT: Soft Template Tuning for Few-Shot Adaptation | Jul 18, 2022 | Few-Shot LearningLanguage Modeling | —Unverified | 0 |
| Multilinguals at SemEval-2022 Task 11: Complex NER in Semantically Ambiguous Settings for Low Resource Languages | Jul 14, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 0 |
| GPTs at Factify 2022: Prompt Aided Fact-Verification | Jun 29, 2022 | Fact VerificationLanguage Modeling | —Unverified | 0 |
| SemMAE: Semantic-Guided Masking for Learning Masked Autoencoders | Jun 21, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| General Framework for Reversible Data Hiding in Texts Based on Masked Language Modeling | Jun 21, 2022 | Language ModelingLanguage Modelling | —Unverified | 0 |
| SSM-DTA: Breaking the Barriers of Data Scarcity in Drug-Target Affinity Prediction | Jun 20, 2022 | Drug DiscoveryLanguage Modeling | CodeCode Available | 1 |
| Zero-Shot Video Question Answering via Frozen Bidirectional Language Models | Jun 16, 2022 | Fill MaskLanguage Modeling | CodeCode Available | 1 |
| LAVENDER: Unifying Video-Language Understanding as Masked Language Modeling | Jun 14, 2022 | DecoderLanguage Modeling | CodeCode Available | 1 |
| GLIPv2: Unifying Localization and Vision-Language Understanding | Jun 12, 2022 | 2D Object DetectionContrastive Learning | CodeCode Available | 4 |
| VL-BEiT: Generative Vision-Language Pretraining | Jun 2, 2022 | image-classificationImage Classification | —Unverified | 0 |
