SPCTNet: A Series-Parallel CNN and Transformer Network for 3D Medical Image Segmentation

Abstract

Medical image segmentation is crucial for lesion localization and surgical navigation. Recent advancements in medical image segmentation have been driven by Convolutional Neural Networks (CNNs) and Transformers. However, CNNs have limitations in capturing long-range dependencies due to their weight sharing and localized receptive fields, posing challenges in handling varying organ shapes. While Transformers offer an alternative with global receptive fields, their spatial and computational complexity is particularly high, especially for 3D medical images. To address this issue, we propose a novel series-parallel network that combines convolution and self-attention for 3D medical image segmentation. We utilize a serial 3D CNN as the encoder to extract multi-level feature maps, which are fused via a feature pyramid network. Subsequently, we adopt four parallel Transformer branches to capture global features. To efficiently model long-range information, we introduce patch self-attention, which divides the input into non-overlapping patches and computes attention between corresponding pixels across patches. Experimental evaluations on 3D MRI prostate and left atrial segmentation tasks confirm the superior performance of our network compared to other CNN and Transformer-based networks. Notably, our method achieves higher segmentation accuracy and faster inference speed.

Tasks

Reproductions