Multi-Projection Fusion and Refinement Network for Salient Object Detection in 360° Omnidirectional Image

作者：Cong, Runmin; Huang, Ke; Lei, Jianjun^*; Zhao, Yao; Huang, Qingming; Kwong, Sam

来源：IEEE Transactions on Neural Networks and Learning Systems, 2023.

DOI：10.1109/TNNLS.2022.3233883

• 360? omnidirectional image
• cube-unfolding (CU)
• dynamic weighting
• filtration and refinement (FR)
• salient object detection (SOD)

Salient object detection (SOD) aims to determine the most visually attractive objects in an image. With the devel-opment of virtual reality (VR) technology, 360(?) omnidirectional image has been widely used, but the SOD task in 360(?) omni-directional image is seldom studied due to its severe distortions and complex scenes. In this article, we propose a multi-projection fusion and refinement network (MPFR-Net) to detect the salient objects in 360(?) omnidirectional image. Different from the existing methods, the equirectangular projection (EP) image and four corresponding cube-unfolding (CU) images are embedded into the network simultaneously as inputs, where the CU images not only provide supplementary information for EP image but also ensure the object integrity of cube-map projection. In order to make full use of these two projection modes, a dynamic weighting fusion (DWF) module is designed to adaptively integrate the features of different projections in a complementary and dynamic manner from the perspective of inter and intrafeatures. Furthermore,

• 单位
  北京交通大学; 山东大学; 中国科学院研究生院; 天津大学