| Merging Text Transformer Models from Different Initializations | Mar 1, 2024 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| NextLevelBERT: Masked Language Modeling with Higher-Level Representations for Long Documents | Feb 27, 2024 | Document ClassificationLanguage Modeling | CodeCode Available | 1 |
| CodeArt: Better Code Models by Attention Regularization When Symbols Are Lacking | Feb 19, 2024 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Interpretation of Intracardiac Electrograms Through Textual Representations | Feb 2, 2024 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Endowing Protein Language Models with Structural Knowledge | Jan 26, 2024 | Drug DesignLanguage Modeling | CodeCode Available | 1 |
| ECAMP: Entity-centered Context-aware Medical Vision Language Pre-training | Dec 20, 2023 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Labrador: Exploring the Limits of Masked Language Modeling for Laboratory Data | Dec 9, 2023 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| GeoLM: Empowering Language Models for Geospatially Grounded Language Understanding | Oct 23, 2023 | ArticlesContrastive Learning | CodeCode Available | 1 |
| FATA-Trans: Field And Time-Aware Transformer for Sequential Tabular Data | Oct 20, 2023 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| FiLM: Fill-in Language Models for Any-Order Generation | Oct 15, 2023 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| PepMLM: Target Sequence-Conditioned Generation of Therapeutic Peptide Binders via Span Masked Language Modeling | Oct 5, 2023 | BenchmarkingLanguage Modeling | CodeCode Available | 1 |
| A Multi-Task Semantic Decomposition Framework with Task-specific Pre-training for Few-Shot NER | Aug 28, 2023 | Contrastive Learningfew-shot-ner | CodeCode Available | 1 |
| Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning | Aug 23, 2023 | In-Context LearningLanguage Modeling | CodeCode Available | 1 |
| Pairing interacting protein sequences using masked language modeling | Aug 14, 2023 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Stochastic positional embeddings improve masked image modeling | Jul 31, 2023 | Language ModellingMasked Language Modeling | CodeCode Available | 1 |
| Pushing the Limits of Unsupervised Unit Discovery for SSL Speech Representation | Jun 15, 2023 | Automatic Speech RecognitionClustering | CodeCode Available | 1 |
| Generate to Understand for Representation | Jun 14, 2023 | Contrastive LearningGPU | CodeCode Available | 1 |
| Global and Local Semantic Completion Learning for Vision-Language Pre-training | Jun 12, 2023 | cross-modal alignmentImage-text Retrieval | CodeCode Available | 1 |
| On the Difference of BERT-style and CLIP-style Text Encoders | Jun 6, 2023 | Image GenerationLanguage Modeling | CodeCode Available | 1 |
| Preserving Pre-trained Features Helps Calibrate Fine-tuned Language Models | May 30, 2023 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| Rethinking Masked Language Modeling for Chinese Spelling Correction | May 28, 2023 | DiversityDomain Generalization | CodeCode Available | 1 |
| ESCOXLM-R: Multilingual Taxonomy-driven Pre-training for the Job Market Domain | May 20, 2023 | De-identificationLanguage Modeling | CodeCode Available | 1 |
| Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Model | May 19, 2023 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
| DinoSR: Self-Distillation and Online Clustering for Self-supervised Speech Representation Learning | May 17, 2023 | ClusteringLanguage Modeling | CodeCode Available | 1 |
| Fine-grained Audible Video Description | Mar 27, 2023 | Language ModelingLanguage Modelling | CodeCode Available | 1 |
