UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation

Code

• github.com/MrGiovanni/UNetPlusPlus
  OfficialIn paperpytorch★ 0
• github.com/frgfm/Holocron
  pytorch★ 328
• github.com/Burf/tfdetection
  tf★ 56
• github.com/mrgiovanni/dissertation
  none★ 14
• github.com/albertsokol/pneumothorax-detection-unet
  tf★ 0
• github.com/2023-MindSpore-1/ms-code-118
  mindspore★ 0
• github.com/reyvaz/steel-defect-segmentation
  tf★ 0
• github.com/reyvaz/pneumothorax_detection
  tf★ 0
• github.com/rizalmaulanaa/robustness_of_prob_u_net
  tf★ 0
• github.com/sauravmishra1710/UNet-Plus-Plus---Brain-Tumor-Segmentation
  tf★ 0

Abstract

The state-of-the-art models for medical image segmentation are variants of U-Net and fully convolutional networks (FCN). Despite their success, these models have two limitations: (1) their optimal depth is apriori unknown, requiring extensive architecture search or inefficient ensemble of models of varying depths; and (2) their skip connections impose an unnecessarily restrictive fusion scheme, forcing aggregation only at the same-scale feature maps of the encoder and decoder sub-networks. To overcome these two limitations, we propose UNet++, a new neural architecture for semantic and instance segmentation, by (1) alleviating the unknown network depth with an efficient ensemble of U-Nets of varying depths, which partially share an encoder and co-learn simultaneously using deep supervision; (2) redesigning skip connections to aggregate features of varying semantic scales at the decoder sub-networks, leading to a highly flexible feature fusion scheme; and (3) devising a pruning scheme to accelerate the inference speed of UNet++. We have evaluated UNet++ using six different medical image segmentation datasets, covering multiple imaging modalities such as computed tomography (CT), magnetic resonance imaging (MRI), and electron microscopy (EM), and demonstrating that (1) UNet++ consistently outperforms the baseline models for the task of semantic segmentation across different datasets and backbone architectures; (2) UNet++ enhances segmentation quality of varying-size objects -- an improvement over the fixed-depth U-Net; (3) Mask RCNN++ (Mask R-CNN with UNet++ design) outperforms the original Mask R-CNN for the task of instance segmentation; and (4) pruned UNet++ models achieve significant speedup while showing only modest performance degradation. Our implementation and pre-trained models are available at https://github.com/MrGiovanni/UNetPlusPlus.

Tasks

Benchmark Results

Reproductions