SOTAVerified

Self-Supervised Learning

Self-Supervised Learning is proposed for utilizing unlabeled data with the success of supervised learning. Producing a dataset with good labels is expensive, while unlabeled data is being generated all the time. The motivation of Self-Supervised Learning is to make use of the large amount of unlabeled data. The main idea of Self-Supervised Learning is to generate the labels from unlabeled data, according to the structure or characteristics of the data itself, and then train on this unsupervised data in a supervised manner. Self-Supervised Learning is wildly used in representation learning to make a model learn the latent features of the data. This technique is often employed in computer vision, video processing and robot control.

Source: Self-supervised Point Set Local Descriptors for Point Cloud Registration

Image source: LeCun

Papers

Showing 751800 of 5044 papers

TitleStatusHype
Effective Self-supervised Pre-training on Low-compute Networks without DistillationCode1
SimPer: Simple Self-Supervised Learning of Periodic TargetsCode1
Exploring The Role of Mean Teachers in Self-supervised Masked Auto-EncodersCode1
CCC-wav2vec 2.0: Clustering aided Cross Contrastive Self-supervised learning of speech representationsCode1
Multi-task Self-supervised Graph Neural Networks Enable Stronger Task GeneralizationCode1
Improving Label-Deficient Keyword Spotting Through Self-Supervised PretrainingCode1
Differentiable Raycasting for Self-supervised Occupancy ForecastingCode1
Attention Distillation: self-supervised vision transformer students need more guidanceCode1
Spectral Augmentation for Self-Supervised Learning on GraphsCode1
Multimodality Multi-Lead ECG Arrhythmia Classification using Self-Supervised LearningCode1
The Efficacy of Self-Supervised Speech Models for Audio RepresentationsCode1
Dive into Self-Supervised Learning for Medical Image Analysis: Data, Models and TasksCode1
CUTS: A Deep Learning and Topological Framework for Multigranular Unsupervised Medical Image SegmentationCode1
Attentive Symmetric Autoencoder for Brain MRI SegmentationCode1
NeRF-SOS: Any-View Self-supervised Object Segmentation on Complex ScenesCode1
Self-Supervised Learning of Phenotypic Representations from Cell Images with Weak LabelsCode1
Self-Supervised Learning with an Information Maximization CriterionCode1
MetaMask: Revisiting Dimensional Confounder for Self-Supervised LearningCode1
Spatial-then-Temporal Self-Supervised Learning for Video CorrespondenceCode1
Multi-Modal Masked Autoencoders for Medical Vision-and-Language Pre-TrainingCode1
Improving Self-Supervised Learning by Characterizing Idealized RepresentationsCode1
Continual Learning, Fast and SlowCode1
ChemBERTa-2: Towards Chemical Foundation ModelsCode1
Multi-modal Masked Autoencoders Learn Compositional Histopathological RepresentationsCode1
Self-Supervised Pretraining for 2D Medical Image SegmentationCode1
Supervised Contrastive Learning with Hard Negative SamplesCode1
Self-Supervised Pyramid Representation Learning for Multi-Label Visual Analysis and BeyondCode1
CounTR: Transformer-based Generalised Visual CountingCode1
Refine and Represent: Region-to-Object Representation LearningCode1
Light-weight probing of unsupervised representations for Reinforcement LearningCode1
Augmenting Reinforcement Learning with Transformer-based Scene Representation Learning for Decision-making of Autonomous DrivingCode1
Self-Supervised Endoscopic Image Key-Points MatchingCode1
Improving Knowledge-aware Recommendation with Multi-level Interactive Contrastive LearningCode1
Heterogeneous Graph Masked AutoencodersCode1
Relational Self-Supervised Learning on GraphsCode1
A Hybrid Self-Supervised Learning Framework for Vertical Federated LearningCode1
SensorSCAN: Self-Supervised Learning and Deep Clustering for Fault Diagnosis in Chemical ProcessesCode1
Self-Supervised Vision Transformers for Malware DetectionCode1
Self-Supervised Learning for Anomalous Channel Detection in EEG Graphs: Application to Seizure AnalysisCode1
Consistency-based Self-supervised Learning for Temporal Anomaly LocalizationCode1
Non-Contrastive Self-Supervised Learning of Utterance-Level Speech RepresentationsCode1
TransDSSL: Transformer based Depth Estimation via Self-Supervised LearningCode1
COCOA: Cross Modality Contrastive Learning for Sensor DataCode1
SdAE: Self-distillated Masked AutoencoderCode1
Contrastive UCB: Provably Efficient Contrastive Self-Supervised Learning in Online Reinforcement LearningCode1
Self-supervised learning with rotation-invariant kernelsCode1
Self-Supervised Hypergraph Transformer for Recommender SystemsCode1
Dive into Big Model TrainingCode1
Self-supervised contrastive learning of echocardiogram videos enables label-efficient cardiac disease diagnosisCode1
Hyper-Representations for Pre-Training and Transfer LearningCode1
Show:102550
← PrevPage 16 of 101Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Pretraining: NoneImages & Text57.5Unverified
2Pretraining: ShEDImages & Text54.3Unverified
3Pretraining: e-MixImages & Text48.9Unverified
#ModelMetricClaimedVerifiedStatus
1ResNet50Accuracy91.7Unverified
2ResNet18Accuracy91.02Unverified
3MV-MRAccuracy89.67Unverified
#ModelMetricClaimedVerifiedStatus
1ResNet50average top-1 classification accuracy93.89Unverified
2ResNet18average top-1 classification accuracy92.58Unverified
#ModelMetricClaimedVerifiedStatus
1ResNet50average top-1 classification accuracy72.51Unverified
2ResNet18average top-1 classification accuracy69.31Unverified
#ModelMetricClaimedVerifiedStatus
1CorInfomax (ResNet50)Top-1 Accuracy82.64Unverified
2CorInfomax (ResNet18)Top-1 Accuracy80.48Unverified
#ModelMetricClaimedVerifiedStatus
1ResNet50average top-1 classification accuracy51.84Unverified
2ResNet18average top-1 classification accuracy51.67Unverified
#ModelMetricClaimedVerifiedStatus
1CorInfomax (ResNet18)Top-1 Accuracy93.18Unverified
#ModelMetricClaimedVerifiedStatus
1CorInfomax (ResNet18)Top-1 Accuracy71.61Unverified
#ModelMetricClaimedVerifiedStatus
1Hybrid BYOL-S/CvTAccuracy67.2Unverified
#ModelMetricClaimedVerifiedStatus
1CorInfomax (ResNet50)Top-1 Accuracy54.86Unverified