HM: Hybrid Masking for Few-Shot Segmentation
Seonghyeon Moon, Samuel S. Sohn, Honglu Zhou, Sejong Yoon, Vladimir Pavlovic, Muhammad Haris Khan, Mubbasir Kapadia
Code Available — Be the first to reproduce this paper.
ReproduceCode
- github.com/moonsh/hm-hybrid-maskingOfficialIn paperpytorch★ 6
Abstract
We study few-shot semantic segmentation that aims to segment a target object from a query image when provided with a few annotated support images of the target class. Several recent methods resort to a feature masking (FM) technique to discard irrelevant feature activations which eventually facilitates the reliable prediction of segmentation mask. A fundamental limitation of FM is the inability to preserve the fine-grained spatial details that affect the accuracy of segmentation mask, especially for small target objects. In this paper, we develop a simple, effective, and efficient approach to enhance feature masking (FM). We dub the enhanced FM as hybrid masking (HM). Specifically, we compensate for the loss of fine-grained spatial details in FM technique by investigating and leveraging a complementary basic input masking method. Experiments have been conducted on three publicly available benchmarks with strong few-shot segmentation (FSS) baselines. We empirically show improved performance against the current state-of-the-art methods by visible margins across different benchmarks. Our code and trained models are available at: https://github.com/moonsh/HM-Hybrid-Masking
Tasks
Benchmark Results
| Dataset | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| COCO-20i (1-shot) | HSNet (HM, ResNet-50) | Mean IoU | 44.3 | — | Unverified |
| COCO-20i (1-shot) | ASNet (HM, ResNet-50) | Mean IoU | 44.7 | — | Unverified |
| COCO-20i (1-shot) | ASNet (HM, ResNet-101) | Mean IoU | 45.9 | — | Unverified |
| COCO-20i (1-shot) | HSNet (HM, ResNet-101) | Mean IoU | 46.5 | — | Unverified |
| COCO-20i (1-shot) | VAT (HM, ResNet-50) | Mean IoU | 43.2 | — | Unverified |
| COCO-20i (5-shot) | ASNet (HM, ResNet-101) | Mean IoU | 50.6 | — | Unverified |
| COCO-20i (5-shot) | HSNet (HM, ResNet-101) | Mean IoU | 50.6 | — | Unverified |
| COCO-20i (5-shot) | HSNet (HM, ResNet-50) | Mean IoU | 49.4 | — | Unverified |
| COCO-20i (5-shot) | ASNet (HM, ResNet-50) | Mean IoU | 48.4 | — | Unverified |
| COCO-20i (5-shot) | VAT (HM, ResNet-50) | Mean IoU | 48.3 | — | Unverified |
| COCO-20i -> Pascal VOC (1-shot) | HSNet (HM, ResNet-101) | Mean IoU | 66.5 | — | Unverified |
| COCO-20i -> Pascal VOC (1-shot) | HSNet (HM, ResNet-50) | Mean IoU | 65.2 | — | Unverified |
| COCO-20i -> Pascal VOC (1-shot) | VAT (HM, ResNet-50) | Mean IoU | 65.1 | — | Unverified |
| COCO-20i -> Pascal VOC (5-shot) | VAT (HM, ResNet-50) | Mean IoU | 69.7 | — | Unverified |
| COCO-20i -> Pascal VOC (5-shot) | HSNet (HM, ResNet-50) | Mean IoU | 69.7 | — | Unverified |
| COCO-20i -> Pascal VOC (5-shot) | HSNet (HM, ResNet-101) | Mean IoU | 70.9 | — | Unverified |
| FSS-1000 (1-shot) | HSNet (HM, ResNet-50) | Mean IoU | 87.1 | — | Unverified |
| FSS-1000 (1-shot) | VAT (HM, ResNet-101) | Mean IoU | 90.2 | — | Unverified |
| FSS-1000 (1-shot) | VAT (HM, ResNet-50) | Mean IoU | 89.4 | — | Unverified |
| FSS-1000 (1-shot) | HSNet (HM, ResNet-101) | Mean IoU | 87.8 | — | Unverified |
| FSS-1000 (5-shot) | HSNet (HM, ResNet-50) | Mean IoU | 88 | — | Unverified |
| FSS-1000 (5-shot) | VAT (HM, ResNet-101) | Mean IoU | 90.5 | — | Unverified |
| FSS-1000 (5-shot) | VAT (HM, ResNet-50) | Mean IoU | 89.9 | — | Unverified |
| FSS-1000 (5-shot) | HSNet (HM, ResNet-101) | Mean IoU | 88.5 | — | Unverified |
| PASCAL-5i (1-Shot) | HSNet (HM, ResNet-50) | Mean IoU | 65 | — | Unverified |
| PASCAL-5i (1-Shot) | VAT (HM, ResNet-50) | Mean IoU | 65.8 | — | Unverified |
| PASCAL-5i (1-Shot) | HSNet (HM, ResNet-101) | Mean IoU | 66.7 | — | Unverified |
| PASCAL-5i (1-Shot) | VAT (HM, ResNet-101) | Mean IoU | 67.8 | — | Unverified |
| PASCAL-5i (5-Shot) | HSNet (HM, ResNet-50) | Mean IoU | 67.1 | — | Unverified |
| PASCAL-5i (5-Shot) | VAT (HM, ResNet-50) | Mean IoU | 68.2 | — | Unverified |
| PASCAL-5i (5-Shot) | HSNet (HM, ResNet-101) | Mean IoU | 69.3 | — | Unverified |
| PASCAL-5i (5-Shot) | VAT (HM, ResNet-101) | Mean IoU | 70.9 | — | Unverified |