ScholarMate
English简体中文繁體中文
客服热线:400-1616-289
登录注册
期刊论文

Improving rare relation inferring for scene graph generation using bipartite graph network

Lu, Jiale; Chen, Lianggangxu; Guan, Haoyue; Lin, Shaohui; Gu, Chunhua; Wang, Changbo; He, Gaoqi*univ shanghai sci & technol
Computer Vision and Image Understanding2024-2DOI: 10.1016/j.cviu.2023.103901
8阅读
引用收藏
分享

摘要

Scene graph generation (SGG) task is plagued by insufficient and long-tailed training samples. Thus external text knowledge has been introduced into SGG task, which augments the training dataset. However, rare relations or low-frequency events, such as , are still hard to be discovered since they also rarely appear in external texts. Addressed these issues, this paper proposes a model-agnostic Bipartite Graph Network with Dual-Group Message Passing (DG-BGN). It extends the relation space of each object pair in training dataset based on the similarity of different object pairs, which cooperates external text knowledge and internal visual information. Specifically, the main framework of DG-BGN consists of three parts, that is, bipartite graph construction (BGC), dual-group message propagation (DG-MP), and pseudo label generation (PLG). For BGC, alignment between external texts and visual dataset is performed to mine information from multimodal resources. Taking object pair as the index of the grouping, dual bipartite graph containing intra/inter-group graph is established to represent predicate label probability distribution of object pair instances. Among them, inter-group bipartite graph is built based on the similarity between object pair instance. For DG-MP, efficient message passing is running on both kinds of bipartite graph through Graph Convolution Neural Networks to refine the graphs, in order to let each object pair instance learn from its similar instances. Finally in PLG, pseudo labels, which are used to train SGG models in fully supervised way, are obtained by aggregating these refined graphs. Systematic experiments on Visual Genome dataset and Conceptual Captions show that our method performs better on discovering rare relations.

学科与关键词

关键词
Text-supervised scene graph generationBipartite graph networkRare relation inferring
相关
加载中…
  • GPS-Net: Graph Property Sensing Network for Scene Graph Generation
    Lin, Xin; Ding, Changxing; Zeng, Jinquan; Tao, DachengIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), ELECTR NETWORK, 2020-06-14 To 2020-06-19.被引 98
  • Knowledge-Enhanced Scene Graph Generation with Multimodal Relation Alignment (Student Abstract)
    Fu, Ze; Feng, Junhao; Zheng, Changmeng; Cai, Yi*36th AAAI Conference on Artificial Intelligence / 34th Conference on Innovative Applications of Artificial Intelligence / 12th Symposium on Educational Advances in Artificial Intelligence, ELECTR NETWORK, 2022-02-22 To 2022.
  • Scene Graph Generation using Depth-based Multimodal Network
    Chen, Lianggangxu*; Lu, Jiale; Wang, Changbo; He, GaoqiIEEE International Conference on Multimedia and Expo (ICME), Brisbane, AUSTRALIA, 2023-07-10 To 2023-07-14.
查看全部 →

发表信息

论文状态
公开发表
期刊
Computer Vision and Image Understanding
发表日期
2024-02
卷期
第 239 卷
页码
103901
DOI
10.1016/j.cviu.2023.103901
语言
ENGLISH
成果收录
SCIE
资助标注
Natural Science Foundation of Chongqing, China [CSTB2022NSCQ-MSX0552]; National Natural Science Foundation of China [62002121, 62102151, 62072183]; Shanghai Sailing Program [21YF1411200]; Shanghai Science and Technology Commission [21511100700, 22511104600]; Open Project Program of the State Key Lab of CADCG; CCF-Tencent Rhino-Bird Open Research Fund

作者

  • L

    Lu, Jiale (第一作者)

  • C

    Chen, Lianggangxu

查看全部 7 位作者收起作者
  • G

    Guan, Haoyue

  • L

    Lin, Shaohui

  • G

    Gu, Chunhua

    univ shanghai sci & technol

  • W

    Wang, Changbo

  • H

    He, Gaoqi (通讯)

登录解锁更多功能

  • 收藏这篇,随时在“我的 · 收藏”找回
  • 认领你的成果,生成推广素材
  • 关注作者获取最新成果
  • 加入项目组、协同写作
注册 / 登录

产品服务

  • 科研之友
  • 创新城
  • 科创云

服务支持

  • 帮助中心
  • 隐私政策
  • 服务条款

联系方式

在线客服:【】
客服热线:400-1616-289
电子邮箱:support@scholarmate.com

关注或下载科研之友

微信二维码
微信公众号
客户端下载二维码
下载客户端
科研成果科研人员科研机构科研动态爱瑞思软件

©2026 深圳市科研之友网络服务有限公司

公安备案图标粤公网安备 44030502000213
粤ICP备 16046710 号粤B2-20110417