Panoptic Segmentation

Panoptic Segmentation is a computer vision task that combines semantic segmentation and instance segmentation to provide a comprehensive understanding of the scene. The goal of panoptic segmentation is to segment the image into semantically meaningful parts or regions, while also detecting and distinguishing individual instances of objects within those regions. In a given image, every pixel is assigned a semantic label, and pixels belonging to "things" classes (countable objects with instances, like cars and people) are assigned unique instance IDs. ( Image credit: Detectron2 )

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 101–125 of 462 papers

Title	Date	Tasks	Status	Hype
On Point Affiliation in Feature Upsampling	Jul 17, 2023	DecoderDepth Estimation	CodeCode Available	1
Pair then Relation: Pair-Net for Panoptic Scene Graph Generation	Jul 17, 2023	Graph GenerationPanoptic Scene Graph Generation	CodeCode Available	1
Towards accurate instance segmentation in large-scale LiDAR point clouds	Jul 6, 2023	ClusteringInstance Segmentation	CodeCode Available	1
PANet: LiDAR Panoptic Segmentation with Sparse Instance Proposal and Aggregation	Jun 27, 2023	Autonomous DrivingInstance Segmentation	CodeCode Available	1
Primitive Generation and Semantic-related Alignment for Universal Zero-Shot Segmentation	Jun 19, 2023	Instance SegmentationPanoptic Segmentation	CodeCode Available	1
PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentation	Jun 16, 2023	3D Panoptic SegmentationAutonomous Driving	CodeCode Available	1
Efficient Multi-Task Scene Analysis with RGB-D Transformers	Jun 8, 2023	Panoptic SegmentationScene Classification	CodeCode Available	1
1st Place Solution for PVUW Challenge 2023: Video Panoptic Segmentation	Jun 7, 2023	Autonomous DrivingPanoptic Segmentation	CodeCode Available	1
PhenoBench -- A Large Dataset and Benchmarks for Semantic Image Interpretation in the Agricultural Domain	Jun 7, 2023	Instance SegmentationPanoptic Segmentation	CodeCode Available	1
DFormer: Diffusion-guided Transformer for Universal Image Segmentation	Jun 6, 2023	DecoderDenoising	CodeCode Available	1
BUOL: A Bottom-Up Framework with Occupancy-aware Lifting for Panoptic 3D Scene Reconstruction From A Single Image	Jun 1, 2023	3D Panoptic Segmentation3D Reconstruction	CodeCode Available	1
Revisiting the Encoding of Satellite Image Time Series	May 3, 2023	Image Segmentationobject-detection	CodeCode Available	1
CLUSTSEG: Clustering for Universal Segmentation	May 3, 2023	Instance SegmentationPanoptic Segmentation	CodeCode Available	1
RT-K-Net: Revisiting K-Net for Real-Time Panoptic Segmentation	May 2, 2023	GPUInstance Segmentation	CodeCode Available	1
EDAPS: Enhanced Domain-Adaptive Panoptic Segmentation	Apr 27, 2023	Domain AdaptationInstance Segmentation	CodeCode Available	1
A Review of Panoptic Segmentation for Mobile Mapping Point Clouds	Apr 27, 2023	Instance SegmentationPanoptic Segmentation	CodeCode Available	1
AutoFocusFormer: Image Segmentation off the Grid	Apr 24, 2023	Image SegmentationInstance Segmentation	CodeCode Available	1
An Instance Segmentation Dataset of Yeast Cells in Microstructures	Apr 15, 2023	Cell SegmentationImage Segmentation	CodeCode Available	1
Instance Neural Radiance Field	Apr 10, 2023	3D Instance SegmentationInstance Segmentation	CodeCode Available	1
FinnWoodlands Dataset	Apr 3, 2023	Autonomous DrivingDepth Completion	CodeCode Available	1
Position-Guided Point Cloud Panoptic Segmentation Transformer	Mar 23, 2023	Instance SegmentationPanoptic Segmentation	CodeCode Available	1
CoDEPS: Online Continual Learning for Depth Estimation and Panoptic Segmentation	Mar 17, 2023	Continual LearningDepth Estimation	CodeCode Available	1
SCPNet: Semantic Scene Completion on Point Cloud	Mar 13, 2023	3D Semantic Scene CompletionKnowledge Distillation	CodeCode Available	1
ElC-OIS: Ellipsoidal Clustering for Open-World Instance Segmentation on LiDAR Data	Mar 8, 2023	Autonomous NavigationClustering	CodeCode Available	1
Improving Sketch Colorization using Adversarial Segmentation Consistency	Jan 20, 2023	ColorizationImage Segmentation	CodeCode Available	1

Show:10 25 50

← PrevPage 5 of 19Next →

All datasets COCO test-dev Cityscapes val COCO minival ADE20K val Mapillary val Cityscapes test LaRS S3DIS Area5 ScanNetV2 Indian Driving Dataset KITTI Panoptic Segmentation PanNuke

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Mask DINO (single scale)	PQ	59.5	—	Unverified
2	kMaX-DeepLab (single-scale)	PQ	58.5	—	Unverified
3	Mask2Former (Swin-L)	PQ	58.3	—	Unverified
4	Panoptic SegFormer (Swin-L)	PQ	56.2	—	Unverified
5	Panoptic SegFormer (PVTv2-B5)	PQ	55.8	—	Unverified
6	CMT-DeepLab (single-scale)	PQ	55.7	—	Unverified
7	K-Net (Swin-L)	PQ	55.2	—	Unverified
8	MaskConver (ResNet50, single-scale)	PQ	53.6	—	Unverified
9	MaskFormer (Swin-L)	PQ	53.3	—	Unverified
10	Panoptic FCN* (Swin-L)	PQ	52.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ViT-P (OneFormer, InternImage-H)	PQ	70.8	—	Unverified
2	Panoptic FCN* (Swin-L, Cityscapes-fine)	PQst	70.6	—	Unverified
3	OneFormer (ConvNeXt-L, single-scale, 512x1024, Mapillary Vistas-pretrained)	PQ	70.1	—	Unverified
4	Panoptic-DeepLab (SWideRNet [1, 1, 4.5], Mapillary Vistas, multi-scale)	PQ	69.6	—	Unverified
5	OneFormer (ConvNeXt-L, single-scale)	PQ	68.51	—	Unverified
6	Panoptic-DeepLab (SWideRNet [1, 1, 4.5], Mapillary Vistas, single-scale)	PQ	68.5	—	Unverified
7	Axial-DeepLab-XL (Mapillary Vistas, multi-scale)	PQ	68.5	—	Unverified
8	kMaX-DeepLab (single-scale)	PQ	68.4	—	Unverified
9	OneFormer (ConvNeXt-XL, single-scale)	PQ	68.4	—	Unverified
10	AFF-Base (single-scale, point-based Mask2Former)	PQ	67.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	HyperSeg (Swin-B)	PQ	61.2	—	Unverified
2	OneFormer (InternImage-H,single-scale)	PQ	60	—	Unverified
3	OpenSeeD (SwinL, single-scale)	PQ	59.5	—	Unverified
4	UMG-CLIP-E/14	PQ	59.5	—	Unverified
5	MasK DINO (SwinL,single-scale)	PQ	59.4	—	Unverified
6	EoMT (DINOv2-g, single-scale, 1280x1280)	PQ	59.2	—	Unverified
7	UMG-CLIP-L/14	PQ	58.9	—	Unverified
8	Panoptic FCN* (Swin-L, single-scale)	PQth	58.5	—	Unverified
9	DiNAT-L (single-scale, Mask2Former)	PQ	58.5	—	Unverified
10	ViT-Adapter-L (single-scale, BEiTv2 pretrain, Mask2Former)	PQ	58.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	OneFormer (InternImage-H, emb_dim=256, single-scale, 896x896)	PQ	54.5	—	Unverified
2	ViT-P (OneFormer, DiNAT-L, single-scale, 1280x1280, COCO_pretrain)	PQ	54	—	Unverified
3	OpenSeed(SwinL, single scale, 1280x1280)	PQ	53.7	—	Unverified
4	OneFormer (DiNAT-L, single-scale, 1280x1280, COCO-Pretrain)	PQ	53.4	—	Unverified
5	EoMT (DINOv2-g, single-scale, 1280x1280, COCO pre-trained)	PQ	52.8	—	Unverified
6	X-Decoder (Davit-d5, Deform, single-scale, 1280x1280)	PQ	52.4	—	Unverified
7	ViT-P (OneFormer, DiNAT-L, single-scale, 1280x1280)	PQ	51.9	—	Unverified
8	OneFormer (DiNAT-L, single-scale, 1280x1280)	PQ	51.5	—	Unverified
9	OneFormer (Swin-L, single-scale, 1280x1280)	PQ	51.4	—	Unverified
10	kMaX-DeepLab (ConvNeXt-L, single-scale, 1281x1281)	PQ	50.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	OneFormer (DiNAT-L, single-scale)	PQ	46.7	—	Unverified
2	OneFormer (ConvNeXt-L, single-scale)	PQ	46.4	—	Unverified
3	Panoptic FCN* (Swin-L, single-scale)	PQ	45.7	—	Unverified
4	Panoptic-DeepLab (SWideRNet-(1, 1, 4.5), multi-scale)	PQ	44.8	—	Unverified
5	Panoptic FCN* (ResNet-50-FPN)	PQst	42.3	—	Unverified
6	Mask2Former + Intra-Batch Supervision (ResNet-50)	PQ	42.2	—	Unverified
7	Axial-DeepLab-L (multi-scale)	PQ	41.1	—	Unverified
8	EfficientPS	PQ	40.6	—	Unverified
9	Panoptic-DeepLab (X71)	PQ	40.5	—	Unverified
10	AdaptIS (ResNeXt-101)	PQ	40.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	OneFormer (ConvNeXt-L, single-scale, Mapillary Vistas-Pretrained)	PQ	68	—	Unverified
2	Panoptic-DeepLab (SWideRNet [1, 1, 4.5], Mapillary, multi-scale)	PQ	67.8	—	Unverified
3	EfficientPS	PQ	67.1	—	Unverified
4	Axial-DeepLab-XL (Mapillary Vistas, multi-scale)	PQ	66.6	—	Unverified
5	kMaX-DeepLab (single-scale)	PQ	66.2	—	Unverified
6	Panoptic-Deeplab	PQ	65.5	—	Unverified
7	EfficientPS (Cityscapes-fine)	PQ	62.9	—	Unverified
8	COPS (ResNet-50)	PQ	60	—	Unverified
9	SOGNet (ResNet-50)	PQ	60	—	Unverified
10	Dynamically Instantiated Network	PQ	55.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Mask2Former (Swin-B)	PQ	41.7	—	Unverified
2	Panoptic FPN (ResNet-50)	PQ	40.1	—	Unverified
3	Mask2Former (Swin-T)	PQ	39.2	—	Unverified
4	Panoptic FPN (ResNet-101)	PQ	38.7	—	Unverified
5	Mask2Former (ResNet-50)	PQ	37.6	—	Unverified
6	Mask2Former (ResNet-101)	PQ	37.2	—	Unverified
7	Panoptic Deeplab (ResNet-50)	PQ	34.7	—	Unverified
8	MaX-DeepLab	PQ	31.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SuperCluster	PQ	50.1	—	Unverified
2	PointGroup (Xiang 2023)	PQ	42.3	—	Unverified
3	KPConv (Xiang 2023)	PQ	41.8	—	Unverified
4	MinkowskiNet (Xiang 2023)	PQ	39.2	—	Unverified
5	PointNet++ (Xiang 2023)	PQ	24.6	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	OneFormer3D	PQ	71.2	—	Unverified
2	PanopticNDT (10cm)	PQ	59.19	—	Unverified
3	SuperCluster	PQ	58.7	—	Unverified
4	PanopticFusion (with CRF)	PQ	33.5	—	Unverified
5	SceneGraphFusion (NN mapping)	PQ	31.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	EfficientPS	PQ	51.1	—	Unverified
2	Seamless	PQ	48.5	—	Unverified
3	UPSNet	PQ	47.1	—	Unverified
4	Panoptic FPN	PQ	46.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	EfficientPS	PQ	43.7	—	Unverified
2	Seamless	PQ	42.2	—	Unverified
3	UPSNet	PQ	39.9	—	Unverified
4	Panoptic FPN	PQ	39.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	LKCell	PQ	50.8	—	Unverified
2	CellViT-SAM-H	PQ	50.62	—	Unverified
3	TSFD	PQ	50.4	—	Unverified
4	NuLite-H	PQ	49.81	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	OneFormer3D	PQ	71.2	—	Unverified
2	SuperCluster	PQ	58.7	—	Unverified
3	PanopticFusion	PQ	33.5	—	Unverified
4	SceneGraphFusion	PQ	31.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Exchanger+Mask2Former	PQ	52.6	—	Unverified
2	Exchanger+Unet+PaPs	PQ	47.8	—	Unverified
3	U-TAE + PaPs	PQ	40.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	VAN-B6*	PQ	58.2	—	Unverified
2	PFPN (ideal number of groups)	PQ	42.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAFuser (Swin-T)	PQ	59.7	—	Unverified
2	MUSES (Mask2Former /w 4xSwin-T)	PQ	53.6	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	EMSANet (2x ResNet-34 NBt1D, PanopticNDT version, finetuned)	PQ	51.15	—	Unverified
2	EMSANet	PQ	47.38	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	P3Former	PQ	0.65	—	Unverified
2	DS-Net	PQ	0.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MasQCLIP	PQ	23.3	—	Unverified