| Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data | Mar 30, 2021 | Change DetectionSelf-Supervised Learning | CodeCode Available | 1 | 5 |
| TabTransformer: Tabular Data Modeling Using Contextual Embeddings | Dec 11, 2020 | tabular-classificationUnsupervised Pre-training | CodeCode Available | 1 | 5 |
| HIQL: Offline Goal-Conditioned RL with Latent States as Actions | Jul 22, 2023 | Reinforcement Learning (RL)Unsupervised Pre-training | CodeCode Available | 1 | 5 |
| Bag of Tricks and A Strong baseline for Image Copy Detection | Nov 13, 2021 | Copy DetectionUnsupervised Pre-training | CodeCode Available | 1 | 5 |
| Behavior From the Void: Unsupervised Active Pre-Training | Mar 8, 2021 | Atari GamesReinforcement Learning (RL) | CodeCode Available | 1 | 5 |
| D^2LV: A Data-Driven and Local-Verification Approach for Image Copy Detection | Nov 13, 2021 | Copy DetectionUnsupervised Pre-training | CodeCode Available | 1 | 5 |
| A Further Study of Unsupervised Pre-training for Transformer Based Speech Recognition | May 20, 2020 | speech-recognitionSpeech Recognition | CodeCode Available | 1 | 5 |
| Exact solutions to the nonlinear dynamics of learning in deep linear neural networks | Dec 20, 2013 | Unsupervised Pre-training | CodeCode Available | 1 | 5 |
| Self-Supervised Visual Representation Learning with Semantic Grouping | May 30, 2022 | Contrastive LearningInstance Segmentation | CodeCode Available | 1 | 5 |
| Semi-supervised 3D shape segmentation with multilevel consistency and part substitution | Apr 19, 2022 | SegmentationSemantic Segmentation | CodeCode Available | 1 | 5 |
| Self-training Improves Pre-training for Natural Language Understanding | Oct 5, 2020 | Data AugmentationFew-Shot Learning | CodeCode Available | 1 | 5 |
| TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task | Apr 30, 2020 | RelationRelation Extraction | CodeCode Available | 1 | 5 |
| PTGB: Pre-Train Graph Neural Networks for Brain Network Analysis | May 20, 2023 | Transfer LearningUnsupervised Pre-training | CodeCode Available | 1 | 5 |
| BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrievers | Apr 29, 2024 | RetrievalUnsupervised Pre-training | CodeCode Available | 1 | 5 |
| SPACE: Your Genomic Profile Predictor is a Powerful DNA Foundation Model | Jun 2, 2025 | Mixture-of-ExpertsUnsupervised Pre-training | CodeCode Available | 1 | 5 |
| FreePoint: Unsupervised Point Cloud Instance Segmentation | May 11, 2023 | Instance SegmentationSegmentation | CodeCode Available | 1 | 5 |
| DOBF: A Deobfuscation Pre-Training Objective for Programming Languages | Feb 15, 2021 | Code SearchCode Translation | CodeCode Available | 1 | 5 |
| DocILE Benchmark for Document Information Localization and Extraction | Feb 11, 2023 | Key Information ExtractionUnsupervised Pre-training | CodeCode Available | 1 | 5 |
| Don't Stop Pretraining? Make Prompt-based Fine-tuning Powerful Learner | May 2, 2023 | SentenceUnsupervised Pre-training | CodeCode Available | 1 | 5 |
| Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biases | Oct 28, 2020 | BIG-bench Machine LearningFace Recognition | CodeCode Available | 1 | 5 |
| Spatiotemporal Contrastive Video Representation Learning | Aug 9, 2020 | Action RecognitionContrastive Learning | CodeCode Available | 1 | 5 |
| Drop your Decoder: Pre-training with Bag-of-Word Prediction for Dense Passage Retrieval | Jan 20, 2024 | DecoderPassage Retrieval | CodeCode Available | 1 | 5 |
| The CLEAR Benchmark: Continual LEArning on Real-World Imagery | Jan 17, 2022 | Continual Learningimage-classification | CodeCode Available | 1 | 5 |
| Improving Transformer-based Speech Recognition Using Unsupervised Pre-training | Oct 22, 2019 | speech-recognitionSpeech Recognition | CodeCode Available | 1 | 5 |
| Unsupervised Pre-Training on Patient Population Graphs for Patient-Level Predictions | Mar 23, 2022 | Disease PredictionImputation | CodeCode Available | 1 | 5 |
| ELECTRIcity: An Efficient Transformer for Non-Intrusive Load Monitoring | Apr 11, 2022 | Non-Intrusive Load MonitoringUnsupervised Pre-training | CodeCode Available | 1 | 5 |
| CATE: Computation-aware Neural Architecture Encoding with Transformers | Feb 14, 2021 | AutoMLNeural Architecture Search | CodeCode Available | 1 | 5 |
| Korean-Specific Dataset for Table Question Answering | Jan 17, 2022 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| Self-training and Pre-training are Complementary for Speech Recognition | Oct 22, 2020 | speech-recognitionSpeech Recognition | CodeCode Available | 0 | 5 |
| Self-Supervised Pre-Training Boosts Semantic Scene Segmentation on LiDAR Data | Sep 5, 2023 | Scene SegmentationUnsupervised Pre-training | CodeCode Available | 0 | 5 |
| Accelerating Training of Transformer-Based Language Models with Progressive Layer Dropping | Oct 26, 2020 | Language ModelingLanguage Modelling | CodeCode Available | 0 | 5 |
| Decoupling Exploration and Exploitation for Unsupervised Pre-training with Successor Features | May 4, 2024 | Unsupervised Pre-training | CodeCode Available | 0 | 5 |
| Data-dependent Initializations of Convolutional Neural Networks | Nov 21, 2015 | image-classificationImage Classification | CodeCode Available | 0 | 5 |
| BERT and PALs: Projected Attention Layers for Efficient Adaptation in Multi-Task Learning | Feb 7, 2019 | Multi-Task LearningNatural Language Inference | CodeCode Available | 0 | 5 |
| Curriculum-Based Self-Training Makes Better Few-Shot Learners for Data-to-Text Generation | Jun 6, 2022 | Data-to-Text GenerationText Generation | CodeCode Available | 0 | 5 |
| An Analysis of Unsupervised Pre-training in Light of Recent Advances | Dec 20, 2014 | Data AugmentationImage Classification | CodeCode Available | 0 | 5 |
| A Large-Scale Study on Unsupervised Spatiotemporal Representation Learning | Apr 29, 2021 | Representation LearningSelf-Supervised Action Recognition | CodeCode Available | 0 | 5 |
| RMDL: Random Multimodel Deep Learning for Classification | May 3, 2018 | ClassificationDeep Learning | CodeCode Available | 0 | 5 |
| Contextual embedding and model weighting by fusing domain knowledge on Biomedical Question Answering | Jun 26, 2022 | Cloze TestMachine Reading Comprehension | CodeCode Available | 0 | 5 |
| PUNR: Pre-training with User Behavior Modeling for News Recommendation | Apr 25, 2023 | News RecommendationUnsupervised Pre-training | CodeCode Available | 0 | 5 |
| Self-Supervised Modality-Agnostic Pre-Training of Swin Transformers | May 21, 2024 | Computed Tomography (CT)Unsupervised Pre-training | CodeCode Available | 0 | 5 |
| SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discovery | Nov 12, 2019 | Drug DiscoveryLanguage Modeling | CodeCode Available | 0 | 5 |
| Post Training in Deep Learning with Last Kernel | Nov 14, 2016 | Deep LearningUnsupervised Pre-training | CodeCode Available | 0 | 5 |
| Pre-train and Learn: Preserve Global Information for Graph Neural Networks | Oct 27, 2019 | AttributeNode Classification | CodeCode Available | 0 | 5 |
| COLA: COarse LAbel pre-training for 3D semantic segmentation of sparse LiDAR datasets | Feb 14, 2022 | 3D Semantic SegmentationAutonomous Driving | CodeCode Available | 0 | 5 |
| Neural Grammatical Error Correction Systems with Unsupervised Pre-training on Synthetic Data | Aug 1, 2019 | Grammatical Error CorrectionUnsupervised Pre-training | CodeCode Available | 0 | 5 |
| CochCeps-Augment: A Novel Self-Supervised Contrastive Learning Using Cochlear Cepstrum-based Masking for Speech Emotion Recognition | Feb 10, 2024 | Contrastive LearningEmotion Recognition | CodeCode Available | 0 | 5 |
| MML: Maximal Multiverse Learning for Robust Fine-Tuning of Language Models | Nov 5, 2019 | Unsupervised Pre-training | CodeCode Available | 0 | 5 |
| Unsupervised Pre-Training of Image Features on Non-Curated Data | May 3, 2019 | ClusteringSelf-Supervised Image Classification | CodeCode Available | 0 | 5 |
| m2caiSeg: Semantic Segmentation of Laparoscopic Images using Convolutional Neural Networks | Aug 23, 2020 | AnatomyData Augmentation | CodeCode Available | 0 | 5 |
