摘要
Scene graph generation (SGG) aims to detect objects and predict their pairwise relationships within an image. Current SGG methods typically utilize graph neural networks (GNN,$) to acquire context information between objects/relationships. Despite their effectiveness, however, current SGG methods only assume scene graph homophily while ignoring heterophily. Accordingly, in this paper, we propose a novel Heterophily Learning Network (HL-Net) to comprehensively explore the homophily and heterophily between objects/relationships in scene graphs. More specifically, HL-Net comprises the following 1) an adaptive reweighting transformer module, which adaptively integrates the information from different layers to exploit both the heterophily and homophily in objects; 2) a relationship feature propagation module that efficiently explores the connections between relationships by considering heterophily in order to refine the relationship representation; 3) a heterophily-aware message-passing scheme to further distinguish the heterophily and homophily between objects/relationships, thereby facilitating improved message passing in graphs. We conducted extensive experiments on two public datasets: Visual Genome (VG) and Open Images (OI). The experimental results demonstrate the superiority of our proposed HL-Net over existing state-of-the-art approaches. In more detail, HL-Net outperforms the second-best competitors by 2.1% on the VG dataset for scene graph classification and 1.2% on the 10 dataset for the final score.
- GPS-Net: Graph Property Sensing Network for Scene Graph GenerationLin, Xin; Ding, Changxing; Zeng, Jinquan; Tao, DachengIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), ELECTR NETWORK, 2020-06-14 To 2020-06-19.被引 98
- RU-Net: Regularized Unrolling Network for Scene Graph GenerationLin, Xin; Ding, Changxing*; Zhang, Jing; Zhan, Yibing; Tao, DachengIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, 2022-06-18 To 2022-06-24.
- Scene Graph Generation using Depth-based Multimodal NetworkChen, Lianggangxu*; Lu, Jiale; Wang, Changbo; He, GaoqiIEEE International Conference on Multimedia and Expo (ICME), Brisbane, AUSTRALIA, 2023-07-10 To 2023-07-14.
