SOTAVerified

Self-Supervised Learning

Self-Supervised Learning is proposed for utilizing unlabeled data with the success of supervised learning. Producing a dataset with good labels is expensive, while unlabeled data is being generated all the time. The motivation of Self-Supervised Learning is to make use of the large amount of unlabeled data. The main idea of Self-Supervised Learning is to generate the labels from unlabeled data, according to the structure or characteristics of the data itself, and then train on this unsupervised data in a supervised manner. Self-Supervised Learning is wildly used in representation learning to make a model learn the latent features of the data. This technique is often employed in computer vision, video processing and robot control.

Source: Self-supervised Point Set Local Descriptors for Point Cloud Registration

Image source: LeCun

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 3801–3850 of 5044 papers

TitleStatusHype
Equivariant Contrastive LearningCode1
10 Security and Privacy Problems in Large Foundation Models—0
Learning Deep Representation with Energy-Based Self-Expressiveness for Subspace Clustering—0
Hyper-Representations: Self-Supervised Representation Learning on Neural Network Weights for Model Characteristic PredictionCode1
OMASGAN: Out-of-Distribution Minimum Anomaly Score GAN for Sample Generation on the BoundaryCode1
Self-Supervised Learning Disentangled Group Representation as FeatureCode1
MEmoBERT: Pre-training Model with Prompt-based Learning for Multimodal Emotion Recognition—0
Intermediate Layers Matter in Momentum Contrastive Self Supervised LearningCode1
Self-supervised EEG Representation Learning for Automatic Sleep StagingCode1
Robust Contrastive Learning Using Negative Samples with Diminished SemanticsCode1
GenURL: A General Framework for Unsupervised Representation Learning—0
Towards artificial general intelligence via a multimodal foundation modelCode1
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech ProcessingCode3
Pairwise Half-graph Discrimination: A Simple Graph-level Self-supervised Strategy for Pre-training Graph Neural Networks—0
Understanding the Role of Self-Supervised Learning in Out-of-Distribution Detection Task—0
Directional Self-supervised Learning for Heavy Image Augmentations—0
Self-supervised similarity search for large scientific datasetsCode1
2nd Place Solution for SODA10M Challenge 2021 -- Continual Detection Track—0
Contrastive Neural Processes for Self-Supervised LearningCode1
Distance-wise Prototypical Graph Neural Network in Node Imbalance ClassificationCode1
A Simple Baseline for Low-Budget Active LearningCode1
Self-Supervised Visual Representation Learning Using Lightweight Architectures—0
CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIPCode1
Depth360: Self-supervised Learning for Monocular Depth Estimation using Learnable Camera Distortion Model—0
Knowledge distillation from language model to acoustic model: a hierarchical multi-task learning approach—0
Zero-Shot Physics-Guided Deep Learning for Subject-Specific MRI Reconstruction—0
SSFD: Self-Supervised Feature Distance as an MR Image Reconstruction Quality Metric—0
Constrained Mean Shift for Representation Learning—0
SSAST: Self-Supervised Audio Spectrogram TransformerCode2
Speech Representation Learning Through Self-supervised Pretraining And Multi-task Finetuning—0
Mitigating Memorization of Noisy Labels via Regularization between RepresentationsCode1
Self-Supervised Monocular Depth Estimation with Internal Feature FusionCode1
SentimentArcs: A Novel Method for Self-Supervised Sentiment Analysis of Time Series Shows SOTA Transformers Can Struggle Finding Narrative ArcsCode1
TLDR: Twin Learning for Dimensionality ReductionCode1
Understanding Dimensional Collapse in Contrastive Self-supervised LearningCode1
DECAR: Deep Clustering for learning general-purpose Audio RepresentationsCode0
Unsupervised Representation Learning for Binary Networks by Joint Classifier LearningCode1
SleepPriorCL: Contrastive Representation Learning with Prior Knowledge-based Positive Mining and Adaptive Temperature for Sleep Staging—0
Multilingual Speech Recognition using Knowledge Transfer across Learning Processes—0
Conformer-Based Self-Supervised Learning for Non-Speech Audio Tasks—0
Continual Learning on Noisy Data Streams via Self-Purified ReplayCode0
Self-Supervised Learning by Estimating Twin Class DistributionsCode1
The Impact of Spatiotemporal Augmentations on Self-Supervised Audiovisual Representation Learning—0
Attentive and Contrastive Learning for Joint Depth and Motion Field Estimation—0
NoisyActions2M: A Multimedia Dataset for Video Understanding from Noisy LabelsCode0
Decoupled Contrastive LearningCode1
Multi-Modal Pre-Training for Automated Speech Recognition—0
UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-TrainingCode1
Relative Molecule Self-Attention TransformerCode1
Towards Safer Transportation: a self-supervised learning approach for traffic video deraining—0
Show:102550
← PrevPage 77 of 101Next →
All datasetsDABSSTL-10CIFAR10cifar100ImageNet-100 (TEMI Split)TinyImageNetCIFAR-10CIFAR-100CREMA-DTiny ImageNet

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Pretraining: NoneImages & Text57.5—Unverified
2Pretraining: ShEDImages & Text54.3—Unverified
3Pretraining: e-MixImages & Text48.9—Unverified
#ModelMetricClaimedVerifiedStatus
1ResNet50Accuracy91.7—Unverified
2ResNet18Accuracy91.02—Unverified
3MV-MRAccuracy89.67—Unverified
#ModelMetricClaimedVerifiedStatus
1ResNet50average top-1 classification accuracy93.89—Unverified
2ResNet18average top-1 classification accuracy92.58—Unverified
#ModelMetricClaimedVerifiedStatus
1ResNet50average top-1 classification accuracy72.51—Unverified
2ResNet18average top-1 classification accuracy69.31—Unverified
#ModelMetricClaimedVerifiedStatus
1CorInfomax (ResNet50)Top-1 Accuracy82.64—Unverified
2CorInfomax (ResNet18)Top-1 Accuracy80.48—Unverified
#ModelMetricClaimedVerifiedStatus
1ResNet50average top-1 classification accuracy51.84—Unverified
2ResNet18average top-1 classification accuracy51.67—Unverified
#ModelMetricClaimedVerifiedStatus
1CorInfomax (ResNet18)Top-1 Accuracy93.18—Unverified
#ModelMetricClaimedVerifiedStatus
1CorInfomax (ResNet18)Top-1 Accuracy71.61—Unverified
#ModelMetricClaimedVerifiedStatus
1Hybrid BYOL-S/CvTAccuracy67.2—Unverified
#ModelMetricClaimedVerifiedStatus
1CorInfomax (ResNet50)Top-1 Accuracy54.86—Unverified