Panoptic Segmentation

Panoptic Segmentation is a computer vision task that combines semantic segmentation and instance segmentation to provide a comprehensive understanding of the scene. The goal of panoptic segmentation is to segment the image into semantically meaningful parts or regions, while also detecting and distinguishing individual instances of objects within those regions. In a given image, every pixel is assigned a semantic label, and pixels belonging to "things" classes (countable objects with instances, like cars and people) are assigned unique instance IDs. ( Image credit: Detectron2 )

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 176–200 of 462 papers

Title	Date	Tasks	Status	Hype
DFormer: Diffusion-guided Transformer for Universal Image Segmentation	Jun 6, 2023	DecoderDenoising	CodeCode Available	1
BUOL: A Bottom-Up Framework with Occupancy-aware Lifting for Panoptic 3D Scene Reconstruction From A Single Image	Jun 1, 2023	3D Panoptic Segmentation3D Reconstruction	CodeCode Available	1
SimpSON: Simplifying Photo Cleanup with Single-Click Distracting Object Segmentation Network	May 28, 2023	Panoptic SegmentationSegmentation	CodeCode Available	0
SAD: Segment Any RGBD	May 23, 2023	3D Panoptic SegmentationOpen Vocabulary Semantic Segmentation	CodeCode Available	2
Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Model	May 18, 2023	Image GenerationLanguage Modeling	CodeCode Available	2
Asynchronous Events-based Panoptic Segmentation using Graph Mixer Neural Network	May 5, 2023	Panoptic SegmentationRobotic Grasping	CodeCode Available	0
Revisiting the Encoding of Satellite Image Time Series	May 3, 2023	Image Segmentationobject-detection	CodeCode Available	1
CLUSTSEG: Clustering for Universal Segmentation	May 3, 2023	Instance SegmentationPanoptic Segmentation	CodeCode Available	1
RT-K-Net: Revisiting K-Net for Real-Time Panoptic Segmentation	May 2, 2023	GPUInstance Segmentation	CodeCode Available	1
EDAPS: Enhanced Domain-Adaptive Panoptic Segmentation	Apr 27, 2023	Domain AdaptationInstance Segmentation	CodeCode Available	1
A Review of Panoptic Segmentation for Mobile Mapping Point Clouds	Apr 27, 2023	Instance SegmentationPanoptic Segmentation	CodeCode Available	1
AutoFocusFormer: Image Segmentation off the Grid	Apr 24, 2023	Image SegmentationInstance Segmentation	CodeCode Available	1
Ensembling Instance and Semantic Segmentation for Panoptic Segmentation	Apr 20, 2023	Instance SegmentationPanoptic Segmentation	—Unverified	0
ProPanDL: A Modular Architecture for Uncertainty-Aware Panoptic Segmentation	Apr 17, 2023	Panoptic Segmentationscoring rule	—Unverified	0
Intra-Batch Supervision for Panoptic Segmentation on High-Resolution Images	Apr 17, 2023	Panoptic Segmentation	CodeCode Available	0
An Instance Segmentation Dataset of Yeast Cells in Microstructures	Apr 15, 2023	Cell SegmentationImage Segmentation	CodeCode Available	1
Instance Neural Radiance Field	Apr 10, 2023	3D Instance SegmentationInstance Segmentation	CodeCode Available	1
Video-kMaX: A Simple Unified Approach for Online and Near-Online Video Panoptic Segmentation	Apr 10, 2023	Panoptic SegmentationScene Understanding	—Unverified	0
SegGPT: Segmenting Everything In Context	Apr 6, 2023	Few-Shot Semantic SegmentationIn-Context Learning	CodeCode Available	4
Uncertainty estimation in Deep Learning for Panoptic segmentation	Apr 4, 2023	Deep Learningimage-classification	—Unverified	0
FinnWoodlands Dataset	Apr 3, 2023	Autonomous DrivingDepth Completion	CodeCode Available	1
FreeSeg: Unified, Universal and Open-Vocabulary Image Segmentation	Mar 30, 2023	Image SegmentationInstance Segmentation	—Unverified	0
4D Panoptic Segmentation as Invariant and Equivariant Field Prediction	Mar 28, 2023	4D Panoptic SegmentationAutonomous Driving	—Unverified	0
You Only Segment Once: Towards Real-Time Panoptic Segmentation	Mar 26, 2023	DecoderPanoptic Segmentation	CodeCode Available	2
Position-Guided Point Cloud Panoptic Segmentation Transformer	Mar 23, 2023	Instance SegmentationPanoptic Segmentation	CodeCode Available	1

Show:10 25 50

← PrevPage 8 of 19Next →

All datasets COCO test-dev Cityscapes val COCO minival ADE20K val Mapillary val Cityscapes test LaRS S3DIS Area5 ScanNetV2 Indian Driving Dataset KITTI Panoptic Segmentation PanNuke

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Mask DINO (single scale)	PQ	59.5	—	Unverified
2	kMaX-DeepLab (single-scale)	PQ	58.5	—	Unverified
3	Mask2Former (Swin-L)	PQ	58.3	—	Unverified
4	Panoptic SegFormer (Swin-L)	PQ	56.2	—	Unverified
5	Panoptic SegFormer (PVTv2-B5)	PQ	55.8	—	Unverified
6	CMT-DeepLab (single-scale)	PQ	55.7	—	Unverified
7	K-Net (Swin-L)	PQ	55.2	—	Unverified
8	MaskConver (ResNet50, single-scale)	PQ	53.6	—	Unverified
9	MaskFormer (Swin-L)	PQ	53.3	—	Unverified
10	Panoptic FCN* (Swin-L)	PQ	52.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	ViT-P (OneFormer, InternImage-H)	PQ	70.8	—	Unverified
2	Panoptic FCN* (Swin-L, Cityscapes-fine)	PQst	70.6	—	Unverified
3	OneFormer (ConvNeXt-L, single-scale, 512x1024, Mapillary Vistas-pretrained)	PQ	70.1	—	Unverified
4	Panoptic-DeepLab (SWideRNet [1, 1, 4.5], Mapillary Vistas, multi-scale)	PQ	69.6	—	Unverified
5	OneFormer (ConvNeXt-L, single-scale)	PQ	68.51	—	Unverified
6	Panoptic-DeepLab (SWideRNet [1, 1, 4.5], Mapillary Vistas, single-scale)	PQ	68.5	—	Unverified
7	Axial-DeepLab-XL (Mapillary Vistas, multi-scale)	PQ	68.5	—	Unverified
8	kMaX-DeepLab (single-scale)	PQ	68.4	—	Unverified
9	OneFormer (ConvNeXt-XL, single-scale)	PQ	68.4	—	Unverified
10	AFF-Base (single-scale, point-based Mask2Former)	PQ	67.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	HyperSeg (Swin-B)	PQ	61.2	—	Unverified
2	OneFormer (InternImage-H,single-scale)	PQ	60	—	Unverified
3	OpenSeeD (SwinL, single-scale)	PQ	59.5	—	Unverified
4	UMG-CLIP-E/14	PQ	59.5	—	Unverified
5	MasK DINO (SwinL,single-scale)	PQ	59.4	—	Unverified
6	EoMT (DINOv2-g, single-scale, 1280x1280)	PQ	59.2	—	Unverified
7	UMG-CLIP-L/14	PQ	58.9	—	Unverified
8	Panoptic FCN* (Swin-L, single-scale)	PQth	58.5	—	Unverified
9	DiNAT-L (single-scale, Mask2Former)	PQ	58.5	—	Unverified
10	ViT-Adapter-L (single-scale, BEiTv2 pretrain, Mask2Former)	PQ	58.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	OneFormer (InternImage-H, emb_dim=256, single-scale, 896x896)	PQ	54.5	—	Unverified
2	ViT-P (OneFormer, DiNAT-L, single-scale, 1280x1280, COCO_pretrain)	PQ	54	—	Unverified
3	OpenSeed(SwinL, single scale, 1280x1280)	PQ	53.7	—	Unverified
4	OneFormer (DiNAT-L, single-scale, 1280x1280, COCO-Pretrain)	PQ	53.4	—	Unverified
5	EoMT (DINOv2-g, single-scale, 1280x1280, COCO pre-trained)	PQ	52.8	—	Unverified
6	X-Decoder (Davit-d5, Deform, single-scale, 1280x1280)	PQ	52.4	—	Unverified
7	ViT-P (OneFormer, DiNAT-L, single-scale, 1280x1280)	PQ	51.9	—	Unverified
8	OneFormer (DiNAT-L, single-scale, 1280x1280)	PQ	51.5	—	Unverified
9	OneFormer (Swin-L, single-scale, 1280x1280)	PQ	51.4	—	Unverified
10	kMaX-DeepLab (ConvNeXt-L, single-scale, 1281x1281)	PQ	50.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	OneFormer (DiNAT-L, single-scale)	PQ	46.7	—	Unverified
2	OneFormer (ConvNeXt-L, single-scale)	PQ	46.4	—	Unverified
3	Panoptic FCN* (Swin-L, single-scale)	PQ	45.7	—	Unverified
4	Panoptic-DeepLab (SWideRNet-(1, 1, 4.5), multi-scale)	PQ	44.8	—	Unverified
5	Panoptic FCN* (ResNet-50-FPN)	PQst	42.3	—	Unverified
6	Mask2Former + Intra-Batch Supervision (ResNet-50)	PQ	42.2	—	Unverified
7	Axial-DeepLab-L (multi-scale)	PQ	41.1	—	Unverified
8	EfficientPS	PQ	40.6	—	Unverified
9	Panoptic-DeepLab (X71)	PQ	40.5	—	Unverified
10	AdaptIS (ResNeXt-101)	PQ	40.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	OneFormer (ConvNeXt-L, single-scale, Mapillary Vistas-Pretrained)	PQ	68	—	Unverified
2	Panoptic-DeepLab (SWideRNet [1, 1, 4.5], Mapillary, multi-scale)	PQ	67.8	—	Unverified
3	EfficientPS	PQ	67.1	—	Unverified
4	Axial-DeepLab-XL (Mapillary Vistas, multi-scale)	PQ	66.6	—	Unverified
5	kMaX-DeepLab (single-scale)	PQ	66.2	—	Unverified
6	Panoptic-Deeplab	PQ	65.5	—	Unverified
7	EfficientPS (Cityscapes-fine)	PQ	62.9	—	Unverified
8	COPS (ResNet-50)	PQ	60	—	Unverified
9	SOGNet (ResNet-50)	PQ	60	—	Unverified
10	Dynamically Instantiated Network	PQ	55.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Mask2Former (Swin-B)	PQ	41.7	—	Unverified
2	Panoptic FPN (ResNet-50)	PQ	40.1	—	Unverified
3	Mask2Former (Swin-T)	PQ	39.2	—	Unverified
4	Panoptic FPN (ResNet-101)	PQ	38.7	—	Unverified
5	Mask2Former (ResNet-50)	PQ	37.6	—	Unverified
6	Mask2Former (ResNet-101)	PQ	37.2	—	Unverified
7	Panoptic Deeplab (ResNet-50)	PQ	34.7	—	Unverified
8	MaX-DeepLab	PQ	31.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SuperCluster	PQ	50.1	—	Unverified
2	PointGroup (Xiang 2023)	PQ	42.3	—	Unverified
3	KPConv (Xiang 2023)	PQ	41.8	—	Unverified
4	MinkowskiNet (Xiang 2023)	PQ	39.2	—	Unverified
5	PointNet++ (Xiang 2023)	PQ	24.6	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	OneFormer3D	PQ	71.2	—	Unverified
2	PanopticNDT (10cm)	PQ	59.19	—	Unverified
3	SuperCluster	PQ	58.7	—	Unverified
4	PanopticFusion (with CRF)	PQ	33.5	—	Unverified
5	SceneGraphFusion (NN mapping)	PQ	31.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	EfficientPS	PQ	51.1	—	Unverified
2	Seamless	PQ	48.5	—	Unverified
3	UPSNet	PQ	47.1	—	Unverified
4	Panoptic FPN	PQ	46.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	EfficientPS	PQ	43.7	—	Unverified
2	Seamless	PQ	42.2	—	Unverified
3	UPSNet	PQ	39.9	—	Unverified
4	Panoptic FPN	PQ	39.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	LKCell	PQ	50.8	—	Unverified
2	CellViT-SAM-H	PQ	50.62	—	Unverified
3	TSFD	PQ	50.4	—	Unverified
4	NuLite-H	PQ	49.81	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	OneFormer3D	PQ	71.2	—	Unverified
2	SuperCluster	PQ	58.7	—	Unverified
3	PanopticFusion	PQ	33.5	—	Unverified
4	SceneGraphFusion	PQ	31.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Exchanger+Mask2Former	PQ	52.6	—	Unverified
2	Exchanger+Unet+PaPs	PQ	47.8	—	Unverified
3	U-TAE + PaPs	PQ	40.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	VAN-B6*	PQ	58.2	—	Unverified
2	PFPN (ideal number of groups)	PQ	42.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CAFuser (Swin-T)	PQ	59.7	—	Unverified
2	MUSES (Mask2Former /w 4xSwin-T)	PQ	53.6	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	EMSANet (2x ResNet-34 NBt1D, PanopticNDT version, finetuned)	PQ	51.15	—	Unverified
2	EMSANet	PQ	47.38	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	P3Former	PQ	0.65	—	Unverified
2	DS-Net	PQ	0.56	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MasQCLIP	PQ	23.3	—	Unverified