ScholarMate
English简体中文繁體中文
客服热线:400-1616-289
登录注册
期刊论文

A question-guided multi-hop reasoning graph network for visual question answering

Xu, Zhaoyang; Gu, Jinguang; Liu, Maofu; Zhou, Guangyou; Fu, Haidong; Qiu, Chen*wuhan univ sci & technol; cent china normal univ
INFORMATION PROCESSING & MANAGEMENT2023-3DOI: 10.1016/j.ipm.2022.103207
3被引4阅读
引用收藏
分享

摘要

Visual Question Answering (VQA) requires reasoning about the visually-grounded relations in the image and question context. A crucial aspect of solving complex questions is reliable multi-hop reasoning, i.e., dynamically learning the interplay between visual entities in each step. In this paper, we investigate the potential of the reasoning graph network on multi-hop reasoning questions, especially over 3 "hops."We call this model QMRGT: A Question-Guided Multi-hop Reasoning Graph Network. It constructs a cross-modal interaction module (CIM) and a multi-hop reasoning graph network (MRGT) and infers an answer by dynamically updating the inter-associated instruction between two modalities. Our graph reasoning module can apply to any multi-modal model. The experiments on VQA 2.0 and GQA (in fully supervised and O.O.D settings) datasets show that both QMRGT and pre-training V&L models+MRGT lead to improvement on visual question answering tasks. Graph-based multi-hop reasoning provides an effective signal for the visual question answering challenge, both for the O.O.D and high-level reasoning questions.

学科与关键词

关键词
Visual question answeringMulti-hop reasoningReasoning graph network
相关
加载中…
  • Graph-Based Multi-Interaction Network for Video Question Answering
    Gu, Mao; Zhao, Zhou*; Jin, Weike; Hong, Richang; Wu, FeiIEEE Transactions on Image Processing, 2021, 30: 2758-2770.被引 29
  • Multi-Modal Alignment of Visual Question Answering Based on Multi-Hop Attention Mechanism
    Xia, Qihao; Yu, Chao*; Hou, Yinong; Peng, Pingping; Zheng, Zhengqi; Chen, WenElectronics, 2022, 11(11): 1778.被引 4
  • ClueReader: Heterogeneous Graph Attention Network for Multi-Hop Machine Reading Comprehension
    Gao, Peng*; Gao, Feng; Wang, Peng; Ni, Jian-Cheng; Wang, Fei; Fujita, HamidoElectronics, 2023, 12(14): 3183.
查看全部 →

发表信息

论文状态
公开发表
期刊
INFORMATION PROCESSING & MANAGEMENT
发表日期
2023-03
卷期
第 60 卷, 第 2 期
页码
103207
DOI
10.1016/j.ipm.2022.103207
语言
ENGLISH
成果收录
SSCI · SCIE
资助标注
National Natural Science Foundation of China [U1836118]; Fundamental Research Funds for the Central Universities [61972173]; [CCNU22QN015]

作者

  • X

    Xu, Zhaoyang (第一作者)

    wuhan univ sci & technol

  • G

    Gu, Jinguang

    wuhan univ sci & technol

查看全部 6 位作者收起作者
  • L

    Liu, Maofu

    wuhan univ sci & technol

  • Z

    Zhou, Guangyou

    cent china normal univ

  • F

    Fu, Haidong

    wuhan univ sci & technol

  • Q

    Qiu, Chen (通讯)

    wuhan univ sci & technol

登录解锁更多功能

  • 收藏这篇,随时在“我的 · 收藏”找回
  • 认领你的成果,生成推广素材
  • 关注作者获取最新成果
  • 加入项目组、协同写作
注册 / 登录

产品服务

  • 科研之友
  • 创新城
  • 科创云

服务支持

  • 帮助中心
  • 隐私政策
  • 服务条款

联系方式

在线客服:【】
客服热线:400-1616-289
电子邮箱:support@scholarmate.com

关注或下载科研之友

微信二维码
微信公众号
客户端下载二维码
下载客户端
科研成果科研人员科研机构科研动态爱瑞思软件

©2026 深圳市科研之友网络服务有限公司

公安备案图标粤公网安备 44030502000213
粤ICP备 16046710 号粤B2-20110417