Fully Convolutional Networks for Panoptic Segmentation
Yanwei Li, Hengshuang Zhao, Xiaojuan Qi, LiWei Wang, Zeming Li, Jian Sun, Jiaya Jia
Code Available — Be the first to reproduce this paper.
ReproduceCode
- github.com/yanwei-li/PanopticFCNOfficialIn paperpytorch★ 403
- github.com/Jia-Research-Lab/PanopticFCNOfficialIn paperpytorch★ 403
- github.com/dvlab-research/panopticfcnpytorch★ 403
- github.com/dvlab-research/msadpytorch★ 131
- github.com/Jia-Research-Lab/MSADpytorch★ 131
- github.com/DdeGeus/PanopticFCN-IBSpytorch★ 3
Abstract
In this paper, we present a conceptually simple, strong, and efficient framework for panoptic segmentation, called Panoptic FCN. Our approach aims to represent and predict foreground things and background stuff in a unified fully convolutional pipeline. In particular, Panoptic FCN encodes each object instance or stuff category into a specific kernel weight with the proposed kernel generator and produces the prediction by convolving the high-resolution feature directly. With this approach, instance-aware and semantically consistent properties for things and stuff can be respectively satisfied in a simple generate-kernel-then-segment workflow. Without extra boxes for localization or instance separation, the proposed approach outperforms previous box-based and -free models with high efficiency on COCO, Cityscapes, and Mapillary Vistas datasets with single scale input. Our code is made publicly available at https://github.com/Jia-Research-Lab/PanopticFCN.
Tasks
Benchmark Results
| Dataset | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| Cityscapes val | Panoptic FCN* (ResNet-FPN) | PQ | 61.4 | — | Unverified |
| Cityscapes val | Panoptic FCN* (Swin-L, Cityscapes-fine) | PQst | 70.6 | — | Unverified |
| Cityscapes val | Panoptic FCN* (ResNet-50-FPN) | PQst | 66.6 | — | Unverified |
| COCO minival | Panoptic FCN* (Swin-L, single-scale) | PQth | 58.5 | — | Unverified |
| COCO minival | Panoptic FCN* (ResNet-50-FPN) | PQ | 44.3 | — | Unverified |
| COCO test-dev | Panoptic FCN* (Swin-L) | PQ | 52.7 | — | Unverified |
| COCO test-dev | Panoptic FCN*++ (DCN-101-FPN) | PQ | 47.5 | — | Unverified |
| Mapillary val | Panoptic FCN* (ResNet-50-FPN) | PQst | 42.3 | — | Unverified |
| Mapillary val | Panoptic FCN* (Swin-L, single-scale) | PQ | 45.7 | — | Unverified |
| Mapillary val | Panoptic FCN* (ResNet-FPN) | PQ | 36.9 | — | Unverified |