Refine-FPN: Instance Segmentation Based on a Non-local Multi-feature Aggregation Mechanism

Authors：Li, Xiaolian; Zhu, Lei^*; Wang, Wenwu; Yang, Ke

Source：NEURAL PROCESSING LETTERS, 2023, 55(3): 3411-3428.

DOI：10.1007/s11063-022-11016-z

• Instance segmentation
• Feature Pyramid Network
• Feature extraction
• Feature fusion
• Attention mechanism

Rational use of multilevel structures of deep networks to extract multiscale features is crucial for instance segmentation. The Feature Pyramid Network (FPN) is a classical architecture that enriches the semantic information of multiscale objects. However, inherent defects in FPN structure are bound to cause loss of information during feature extraction and feature fusion. In this paper, we propose a feature pyramid structure (called Refine-FPN) based on a non-local multi-feature aggregation operation, a module that integrates multi-scale feature to rely on attention mechanisms to improve pyramid feature representation. The algorithm enriches the feature details of feature layers by aggregating multiple features to form a contextual global feature representation. By replacing FPN with Refine-FPN in the Mask R-CNN, our model improved the performance of the mask AP by 0.6% and 0.5% on the COCO dataset, when using ResNet-50 and ResNet-101 as the backbone, respectively. Moreover, it is friendly to integrate the proposed method into other popular architectures. For example, equipping the Cascade Mask R-CNN with Refine-FPN achieves an improvement of 0.5% and 0.4% mask AP under ResNet-50 and ResNet-101, respectively.