Unsupervised Semantic Segmentation

Models that learn to segment each image (i.e. assign a class to every pixel) without seeing the ground truth labels.

( Image credit: SegSort: Segmentation by Discriminative Sorting of Segments )

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 51–75 of 95 papers

Title	Date	Tasks	Status	Hype
Network-Free, Unsupervised Semantic Segmentation With Synthetic Images	Jan 1, 2023	SegmentationSemantic Segmentation	—Unverified	0
Unsupervised Object Localization: Observing the Background to Discover Objects	Dec 15, 2022	Instance SegmentationObject	CodeCode Available	1
Extracting Semantic Knowledge from GANs with Unsupervised Learning	Nov 30, 2022	Image GenerationImage Segmentation	—Unverified	0
Rethinking Alignment and Uniformity in Unsupervised Semantic Segmentation	Nov 26, 2022	Representation LearningSegmentation	—Unverified	0
Copy-Pasting Coherent Depth Regions Improves Contrastive Learning for Urban-Scene Segmentation	Nov 25, 2022	Contrastive LearningDepth Estimation	CodeCode Available	0
Peekaboo: Text to Image Diffusion Models are Zero-Shot Segmentors	Nov 23, 2022	SegmentationSemantic Segmentation	CodeCode Available	1
Unsupervised Image Semantic Segmentation through Superpixels and Graph Neural Networks	Oct 21, 2022	Image SegmentationSegmentation	—Unverified	0
Perceptual Grouping in Contrastive Vision-Language Models	Oct 18, 2022	Object LocalizationRepresentation Learning	CodeCode Available	1
ACSeg: Adaptive Conceptualization for Unsupervised Semantic Segmentation	Oct 12, 2022	Image SegmentationSemantic Segmentation	—Unverified	0
What the DAAM: Interpreting Stable Diffusion Using Cross Attention	Oct 10, 2022	DenoisingDescriptive	CodeCode Available	2
Latent Space Unsupervised Semantic Segmentation	Jul 22, 2022	Change Point DetectionSegmentation	—Unverified	0
Unsupervised Semantic Segmentation with Self-supervised Object-centric Representations	Jul 11, 2022	ClusteringObject	CodeCode Available	1
ReCo: Retrieve and Co-segment for Zero-shot Transfer	Jun 14, 2022	RetrievalSegmentation	CodeCode Available	1
Discovering Object Masks with Transformers for Unsupervised Semantic Segmentation	Jun 13, 2022	ObjectSegmentation	CodeCode Available	1
SERE: Exploring Feature Self-relation for Self-supervised Transformer	Jun 10, 2022	RelationSelf-Supervised Learning	CodeCode Available	1
Self-Supervised Visual Representation Learning with Semantic Grouping	May 30, 2022	Contrastive LearningInstance Segmentation	CodeCode Available	1
Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localization	May 16, 2022	graph partitioningSegmentation	CodeCode Available	2
Self-Supervised Learning of Object Parts for Semantic Segmentation	Apr 27, 2022	Community DetectionImage Segmentation	CodeCode Available	1
Unsupervised Hierarchical Semantic Segmentation with Multiview Cosegmentation and Clustering Transformers	Apr 25, 2022	ClusteringSegmentation	CodeCode Available	1
SegDiscover: Visual Concept Discovery via Unsupervised Semantic Segmentation	Apr 22, 2022	Semantic SegmentationUnsupervised Semantic Segmentation	—Unverified	0
Self-supervised Semantic Segmentation Grounded in Visual Concepts	Mar 25, 2022	Representation LearningSegmentation	—Unverified	0
Drive&Segment: Unsupervised Semantic Segmentation of Urban Scenes via Cross-modal Distillation	Mar 21, 2022	Image SegmentationSegmentation	CodeCode Available	1
Dense Siamese Network for Dense Unsupervised Learning	Mar 21, 2022	Self-Supervised LearningSemantic Segmentation	CodeCode Available	1
Unsupervised Semantic Segmentation by Distilling Feature Correspondences	Mar 16, 2022	FormSemantic Segmentation	CodeCode Available	2
Fully Self-Supervised Learning for Semantic Segmentation	Feb 24, 2022	ClusteringSegmentation	—Unverified	0

Show:10 25 50

← PrevPage 3 of 4Next →

All datasets COCO-Stuff-27 Cityscapes test PASCAL VOC 2012 val Potsdam-3 COCO-Stuff-3 ImageNet-S-50 COCO-Stuff-171 COCO-Stuff-81 SUIM COCO-Stuff-15 ACDC (Adverse Conditions Dataset with Correspondences)Cityscapes val

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	SGSeg	Clustering [Accuracy]	55.7	—	Unverified
2	DynaSeg - FSF (ResNet-18 FPN)	Clustering [mIoU]	54.1	—	Unverified
3	SAN	Clustering [Accuracy]	52	—	Unverified
4	DiffCut	Clustering [mIoU]	49.1	—	Unverified
5	PiCIE	Clustering [Accuracy]	48.1	—	Unverified
6	HCL (ViT-S/8)	Linear Classifier [mIoU]	47.4	—	Unverified
7	HCL (ViT-S/16)	Linear Classifier [mIoU]	45.8	—	Unverified
8	CAUSE (DINOv2, ViT-B/14)	Clustering [mIoU]	45.3	—	Unverified
9	Ours (SlotCon)	Clustering [Accuracy]	42.36	—	Unverified
10	CAUSE (ViT-B/8)	Clustering [mIoU]	41.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GraPix	Pixel Accuracy	64.89	—	Unverified
2	CUPS	mIoU	26.8	—	Unverified
3	ViCE	mIoU	25.2	—	Unverified
4	EAGLE (DINO, ViT-B/8)	mIoU	22.1	—	Unverified
5	EQUSS	mIoU	22	—	Unverified
6	PriMaPs-EM + STEGO (DINO ViT-B/8)	mIoU	21.6	—	Unverified
7	STEGO	mIoU	21	—	Unverified
8	EAGLE (DINO, ViT-S/8)	mIoU	19.7	—	Unverified
9	PriMaPs-EM (DINO ViT-S/8)	mIoU	19.4	—	Unverified
10	HP	mIoU	18.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAUSE (iBOT, ViT-B/16)	Clustering [mIoU]	53.4	—	Unverified
2	CAUSE (ViT-B/8)	Clustering [mIoU]	53.3	—	Unverified
3	CAUSE (DINOv2, ViT-B/14)	Clustering [mIoU]	53.2	—	Unverified
4	MaskDistill+CRF	Clustering [mIoU]	48.9	—	Unverified
5	Leopart (ViT-B/8)	Clustering [mIoU]	47.2	—	Unverified
6	HCL (ViT-S/8)	Clustering [mIoU]	46.3	—	Unverified
7	MaskDistill	Clustering [mIoU]	45.8	—	Unverified
8	MaskContrast (Saliency)	Clustering [mIoU]	44.2	—	Unverified
9	HCL (ViT-S/16)	Clustering [mIoU]	43.2	—	Unverified
10	Leopart (ViT-S/16)	Clustering [mIoU]	41.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PriMaPs-EM+HP (DINO ViT-B/8)	Accuracy	83.3	—	Unverified
2	EAGLE (DINO, ViT-B/8)	Accuracy	83.3	—	Unverified
3	HP	Accuracy	82.4	—	Unverified
4	EQUSS	Accuracy	82	—	Unverified
5	PriMaPs-EM (DINO ViT-B/8)	Accuracy	80.5	—	Unverified
6	STEGO	Accuracy	77	—	Unverified
7	InfoSeg	Pixel Accuracy	71.6	—	Unverified
8	IIC	Accuracy	45.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SAN	Pixel Accuracy	80.3	—	Unverified
2	SGSeg	Pixel Accuracy	74.6	—	Unverified
3	InfoSeg	Pixel Accuracy	73.8	—	Unverified
4	InMARS	Pixel Accuracy	73.1	—	Unverified
5	AC	Pixel Accuracy	72.9	—	Unverified
6	IIC	Pixel Accuracy	72.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PASS (+Saliency map)	mIoU (test)	42.3	—	Unverified
2	PASS	mIoU (test)	32	—	Unverified
3	MaskContrast (+Saliency map)	mIoU (test)	24.2	—	Unverified
4	PiCIE (Supervised pretrain)	mIoU (test)	17.6	—	Unverified
5	MDC (Supervised pretrain)	mIoU (test)	14.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAUSE-TR (ViT-S/8)	mIoU	15.2	—	Unverified
2	TransFGU (ViT-S/8)	mIoU	11.93	—	Unverified
3	PiCIE (ResNet-50)	mIoU	5.6	—	Unverified
4	IIC (ResNet-50)	mIoU	2.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAUSE-TR (ViT-S/8)	mIoU	21.2	—	Unverified
2	CAUSE-MLP (ViT-S/8)	mIoU	19.1	—	Unverified
3	TransFGU (ViT-S/8)	mIoU	12.7	—	Unverified
4	MaskContrast (ResNet-50)	mIoU	3.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	DatUS (ViT-B/8) + OC	Pixel Accuracy	69.98	—	Unverified
2	GraPix + AUT	Pixel Accuracy	65.48	—	Unverified
3	DatUS (ViT-B/8)	Pixel Accuracy	64.67	—	Unverified
4	GraPix	Pixel Accuracy	64.06	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	InfoSeg	Pixel Accuracy	38.8	—	Unverified
2	InMARS	Pixel Accuracy	31	—	Unverified
3	IIC	Pixel Accuracy	27.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Segmenter ViT-S/16	mIoU	16.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Segmenter ViT-S/16	mIoU	21.8	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	DenseSiam	mIoU	16.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	InfoSeg	Pixel Accuracy	69.6	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Segmenter ViT-S/16	mIoU	14.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PASS	mIoU (test)	11	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PASS	mIoU (test)	18.1	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Segmenter ViT-S/16	mIoU	18.9	—	Unverified