Unsupervised Pre-training

Pre-training a neural network using unsupervised (self-supervised) auxiliary tasks on unlabeled data.

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 51–75 of 265 papers

Title	Date	Tasks	Status	Hype	Score
MultiTalent: A Multi-Dataset Approach to Medical Image Segmentation	Mar 25, 2023	Image SegmentationLesion Segmentation	CodeCode Available	1	5
METRA: Scalable Unsupervised RL with Metric-Aware Abstraction	Oct 13, 2023	Reinforcement Learning (RL)Unsupervised Pre-training	CodeCode Available	1	5
BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrievers	Apr 29, 2024	RetrievalUnsupervised Pre-training	CodeCode Available	1	5
Bag of Tricks and A Strong baseline for Image Copy Detection	Nov 13, 2021	Copy DetectionUnsupervised Pre-training	CodeCode Available	1	5
Behavior From the Void: Unsupervised Active Pre-Training	Mar 8, 2021	Atari GamesReinforcement Learning (RL)	CodeCode Available	1	5
D^2LV: A Data-Driven and Local-Verification Approach for Image Copy Detection	Nov 13, 2021	Copy DetectionUnsupervised Pre-training	CodeCode Available	1	5
The CLEAR Benchmark: Continual LEArning on Real-World Imagery	Jan 17, 2022	Continual Learningimage-classification	CodeCode Available	1	5
Unified Multi-modal Unsupervised Representation Learning for Skeleton-based Action Understanding	Nov 6, 2023	Action UnderstandingRepresentation Learning	CodeCode Available	1	5
Patient Contrastive Learning: a Performant, Expressive, and Practical Approach to ECG Modeling	Apr 9, 2021	Contrastive LearningElectrocardiography (ECG)	CodeCode Available	1	5
OBoW: Online Bag-of-Visual-Words Generation for Self-Supervised Learning	Dec 21, 2020	object-detectionObject Detection	CodeCode Available	1	5
PEAC: Unsupervised Pre-training for Cross-Embodiment Reinforcement Learning	May 23, 2024	reinforcement-learningReinforcement Learning	CodeCode Available	1	5
ES-Net: An Efficient Stereo Matching Network	Mar 5, 2021	Autonomous DrivingDepth Estimation	CodeCode Available	1	5
ELECTRIcity: An Efficient Transformer for Non-Intrusive Load Monitoring	Apr 11, 2022	Non-Intrusive Load MonitoringUnsupervised Pre-training	CodeCode Available	1	5
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks	Dec 20, 2013	Unsupervised Pre-training	CodeCode Available	1	5
HIQL: Offline Goal-Conditioned RL with Latent States as Actions	Jul 22, 2023	Reinforcement Learning (RL)Unsupervised Pre-training	CodeCode Available	1	5
End-to-End Training of Neural Retrievers for Open-Domain Question Answering	Jan 2, 2021	Natural QuestionsOpen-Domain Question Answering	CodeCode Available	1	5
DOBF: A Deobfuscation Pre-Training Objective for Programming Languages	Feb 15, 2021	Code SearchCode Translation	CodeCode Available	1	5
DocILE Benchmark for Document Information Localization and Extraction	Feb 11, 2023	Key Information ExtractionUnsupervised Pre-training	CodeCode Available	1	5
Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learning	May 29, 2023	Autonomous DrivingDecoder	CodeCode Available	1	5
FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learning	Feb 4, 2021	Federated LearningUnsupervised Pre-training	CodeCode Available	1	5
Pre-training strategies and datasets for facial representation learning	Mar 30, 2021	3D Face Reconstruction3D Facial Landmark Localization	CodeCode Available	1	5
Drop your Decoder: Pre-training with Bag-of-Word Prediction for Dense Passage Retrieval	Jan 20, 2024	DecoderPassage Retrieval	CodeCode Available	1	5
Rethinking Semi-supervised Learning with Language Models	May 22, 2023	Pseudo LabelSemi-Supervised Text Classification	CodeCode Available	1	5
Semi-supervised 3D shape segmentation with multilevel consistency and part substitution	Apr 19, 2022	SegmentationSemantic Segmentation	CodeCode Available	1	5
Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals	Feb 11, 2021	ClusteringObject	CodeCode Available	1	5

Show:10 25 50

← PrevPage 3 of 11Next →

All datasets Measles UCI measles

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	15RDLs	Accuracy (%)	95	—	Unverified
2	9RDLs	Accuracy (%)	94	—	Unverified
3	3 RMDL	Accuracy (%)	93	—	Unverified
4	CNN	Accuracy (%)	73	—	Unverified
5	RMDL	Accuracy (%)	0.1	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	RMDL (30 RDLs)	Sensitivity (VEB)	90.69	—	Unverified
2		Sensitivity	89.1	—	Unverified
3	RMDL 3 RDLs	Sensitivity	0.87	—	Unverified