| Rolling-Unrolling LSTMs for Action Anticipation from First-Person Video | May 4, 2020 | Action AnticipationAction Recognition | CodeCode Available | 1 | 5 |
| Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data | Mar 30, 2021 | Change DetectionSelf-Supervised Learning | CodeCode Available | 1 | 5 |
| HIQL: Offline Goal-Conditioned RL with Latent States as Actions | Jul 22, 2023 | Reinforcement Learning (RL)Unsupervised Pre-training | CodeCode Available | 1 | 5 |
| TabTransformer: Tabular Data Modeling Using Contextual Embeddings | Dec 11, 2020 | tabular-classificationUnsupervised Pre-training | CodeCode Available | 1 | 5 |
| BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrievers | Apr 29, 2024 | RetrievalUnsupervised Pre-training | CodeCode Available | 1 | 5 |
| D^2LV: A Data-Driven and Local-Verification Approach for Image Copy Detection | Nov 13, 2021 | Copy DetectionUnsupervised Pre-training | CodeCode Available | 1 | 5 |
| MultiTalent: A Multi-Dataset Approach to Medical Image Segmentation | Mar 25, 2023 | Image SegmentationLesion Segmentation | CodeCode Available | 1 | 5 |
| A Further Study of Unsupervised Pre-training for Transformer Based Speech Recognition | May 20, 2020 | speech-recognitionSpeech Recognition | CodeCode Available | 1 | 5 |
| A Transformer-based Framework for Multivariate Time Series Representation Learning | Oct 6, 2020 | Computational EfficiencyGeneral Classification | CodeCode Available | 1 | 5 |
| SEED: Self-supervised Distillation For Visual Representation | Jan 12, 2021 | Knowledge DistillationSelf-Supervised Learning | CodeCode Available | 1 | 5 |
| Self-Supervised Visual Representation Learning with Semantic Grouping | May 30, 2022 | Contrastive LearningInstance Segmentation | CodeCode Available | 1 | 5 |
| SimIPU: Simple 2D Image and 3D Point Cloud Unsupervised Pre-Training for Spatial-Aware Visual Representations | Dec 9, 2021 | Contrastive LearningUnsupervised Pre-training | CodeCode Available | 1 | 5 |
| FreePoint: Unsupervised Point Cloud Instance Segmentation | May 11, 2023 | Instance SegmentationSegmentation | CodeCode Available | 1 | 5 |
| SPACE: Your Genomic Profile Predictor is a Powerful DNA Foundation Model | Jun 2, 2025 | Mixture-of-ExpertsUnsupervised Pre-training | CodeCode Available | 1 | 5 |
| TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task | Apr 30, 2020 | RelationRelation Extraction | CodeCode Available | 1 | 5 |
| Pre-training strategies and datasets for facial representation learning | Mar 30, 2021 | 3D Face Reconstruction3D Facial Landmark Localization | CodeCode Available | 1 | 5 |
| DOBF: A Deobfuscation Pre-Training Objective for Programming Languages | Feb 15, 2021 | Code SearchCode Translation | CodeCode Available | 1 | 5 |
| DocILE Benchmark for Document Information Localization and Extraction | Feb 11, 2023 | Key Information ExtractionUnsupervised Pre-training | CodeCode Available | 1 | 5 |
| Don't Stop Pretraining? Make Prompt-based Fine-tuning Powerful Learner | May 2, 2023 | SentenceUnsupervised Pre-training | CodeCode Available | 1 | 5 |
| Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biases | Oct 28, 2020 | BIG-bench Machine LearningFace Recognition | CodeCode Available | 1 | 5 |
| Initialization and Regularization of Factorized Neural Layers | May 3, 2021 | Knowledge DistillationModel Compression | CodeCode Available | 1 | 5 |
| Drop your Decoder: Pre-training with Bag-of-Word Prediction for Dense Passage Retrieval | Jan 20, 2024 | DecoderPassage Retrieval | CodeCode Available | 1 | 5 |
| Unsupervised Pre-Training on Patient Population Graphs for Patient-Level Predictions | Mar 23, 2022 | Disease PredictionImputation | CodeCode Available | 1 | 5 |
| Improving Transformer-based Speech Recognition Using Unsupervised Pre-training | Oct 22, 2019 | speech-recognitionSpeech Recognition | CodeCode Available | 1 | 5 |
| PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training | Jun 9, 2021 | reinforcement-learningReinforcement Learning (RL) | CodeCode Available | 1 | 5 |
| ELECTRIcity: An Efficient Transformer for Non-Intrusive Load Monitoring | Apr 11, 2022 | Non-Intrusive Load MonitoringUnsupervised Pre-training | CodeCode Available | 1 | 5 |
| CATE: Computation-aware Neural Architecture Encoding with Transformers | Feb 14, 2021 | AutoMLNeural Architecture Search | CodeCode Available | 1 | 5 |
| Unsupervised Pre-Training For Data-Efficient Text-to-Speech On Low Resource Languages | Mar 28, 2023 | Data Augmentationtext-to-speech | CodeCode Available | 1 | 5 |
| Self-Supervised Pre-Training Boosts Semantic Scene Segmentation on LiDAR Data | Sep 5, 2023 | Scene SegmentationUnsupervised Pre-training | CodeCode Available | 0 | 5 |
| Accelerating Training of Transformer-Based Language Models with Progressive Layer Dropping | Oct 26, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 0 | 5 |
| Self-Supervised Modality-Agnostic Pre-Training of Swin Transformers | May 21, 2024 | Computed Tomography (CT)Unsupervised Pre-training | CodeCode Available | 0 | 5 |
| Self-training and Pre-training are Complementary for Speech Recognition | Oct 22, 2020 | speech-recognitionSpeech Recognition | CodeCode Available | 0 | 5 |
| Decoupling Exploration and Exploitation for Unsupervised Pre-training with Successor Features | May 4, 2024 | Unsupervised Pre-training | CodeCode Available | 0 | 5 |
| Data-dependent Initializations of Convolutional Neural Networks | Nov 21, 2015 | image-classificationImage Classification | CodeCode Available | 0 | 5 |
| BERT and PALs: Projected Attention Layers for Efficient Adaptation in Multi-Task Learning | Feb 7, 2019 | Multi-Task LearningNatural Language Inference | CodeCode Available | 0 | 5 |
| Curriculum-Based Self-Training Makes Better Few-Shot Learners for Data-to-Text Generation | Jun 6, 2022 | Data-to-Text GenerationText Generation | CodeCode Available | 0 | 5 |
| An Analysis of Unsupervised Pre-training in Light of Recent Advances | Dec 20, 2014 | Data AugmentationImage Classification | CodeCode Available | 0 | 5 |
| RMDL: Random Multimodel Deep Learning for Classification | May 3, 2018 | ClassificationDeep Learning | CodeCode Available | 0 | 5 |
| A Large-Scale Study on Unsupervised Spatiotemporal Representation Learning | Apr 29, 2021 | Representation LearningSelf-Supervised Action Recognition | CodeCode Available | 0 | 5 |
| Contextual embedding and model weighting by fusing domain knowledge on Biomedical Question Answering | Jun 26, 2022 | Cloze TestMachine Reading Comprehension | CodeCode Available | 0 | 5 |
| PUNR: Pre-training with User Behavior Modeling for News Recommendation | Apr 25, 2023 | News RecommendationUnsupervised Pre-training | CodeCode Available | 0 | 5 |
| Generalized 3D Self-supervised Learning Framework via Prompted Foreground-Aware Feature Contrast | Mar 11, 2023 | 3D Semantic SegmentationContrastive Learning | CodeCode Available | 0 | 5 |
| Post Training in Deep Learning with Last Kernel | Nov 14, 2016 | Deep LearningUnsupervised Pre-training | CodeCode Available | 0 | 5 |
| COLA: COarse LAbel pre-training for 3D semantic segmentation of sparse LiDAR datasets | Feb 14, 2022 | 3D Semantic SegmentationAutonomous Driving | CodeCode Available | 0 | 5 |
| Neural Grammatical Error Correction Systems with Unsupervised Pre-training on Synthetic Data | Aug 1, 2019 | Grammatical Error CorrectionUnsupervised Pre-training | CodeCode Available | 0 | 5 |
| Pre-train and Learn: Preserve Global Information for Graph Neural Networks | Oct 27, 2019 | AttributeNode Classification | CodeCode Available | 0 | 5 |
| CochCeps-Augment: A Novel Self-Supervised Contrastive Learning Using Cochlear Cepstrum-based Masking for Speech Emotion Recognition | Feb 10, 2024 | Contrastive LearningEmotion Recognition | CodeCode Available | 0 | 5 |
| MML: Maximal Multiverse Learning for Robust Fine-Tuning of Language Models | Nov 5, 2019 | Unsupervised Pre-training | CodeCode Available | 0 | 5 |
| Unsupervised Pre-Training of Image Features on Non-Curated Data | May 3, 2019 | ClusteringSelf-Supervised Image Classification | CodeCode Available | 0 | 5 |
| Learning Deep Representations Using Convolutional Auto-encoders with Symmetric Skip Connections | Nov 28, 2016 | DenoisingGeneral Classification | CodeCode Available | 0 | 5 |
