ScholarMate
English简体中文繁體中文
客服热线:400-1616-289
登录注册
期刊论文

Graph-Based Multi-Interaction Network for Video Question Answering

Gu, Mao; Zhao, Zhou*; Jin, Weike; Hong, Richang; Wu, Fei合肥工业大学; 浙江大学
IEEE Transactions on Image Processing2021DOI: 10.1109/TIP.2021.3051756
29被引6阅读
引用收藏
分享

摘要

Video question answering is an important task combining both Natural Language Processing and Computer Vision, which requires a machine to obtain a thorough understanding of the video. Most existing approaches simply capture spatio-temporal information in videos by using a combination of recurrent and convolutional neural networks. Nonetheless, most previous work focus on only salient frames or regions, which normally lacks some significant details, such as potential location and action relations. In this paper, we propose a new method called Graph-based Multi-interaction Network for video question answering. In our model, a new attention mechanism named multi-interaction is designed to capture both element-wise and segment-wise sequence interactions simultaneously, which can be found between and inside the multi-modal inputs. Moreover, we propose a graph-based relation-aware neural network to explore a more fine-grained visual representation, which could explore the relationships and dependencies between objects spatially and temporally. We evaluate our method on TGIF-QA and other two video QA datasets. The qualitative and quantitative experimental results show the effectiveness of our model, which achieves state-of-the-art performance.

学科与关键词

关键词
VisualizationKnowledge discoveryCatsSemanticsTask analysisImage segmentationAdaptation modelsVideo question answeringmulti-interactiongraph-based relation-aware neural network
相关
加载中…
  • Location-Aware Graph Convolutional Networks for Video Question Answering
    Huang, Deng; Chen, Peihao; Zeng, Runhao; Du, Qing; Tan, Mingkui*; Gan, Chuang34th AAAI Conference on Artificial Intelligence / 32nd Innovative Applications of Artificial Intelligence Conference / 10th AAAI Symposium on Educational Advances in Artificial Intelligence, New York, NY, 2020-02-07 To 2020-02-12.被引 63
  • Differentiated Attention with Multi-modal Reasoning for Video Question Answering
    Yao, Shentao; Li, Kun*; Xing, Kun; Wu, Kewei; Xie, Zhao; Guo, DanIEEE International Conference on Electrical Engineering, Big Data and Algorithms (EEBDA), Changchun, PEOPLES R CHINA, 2022-02-25 To 2022-02-27.
  • A question-guided multi-hop reasoning graph network for visual question answering
    Xu, Zhaoyang; Gu, Jinguang; Liu, Maofu; Zhou, Guangyou; Fu, Haidong; Qiu, Chen*INFORMATION PROCESSING & MANAGEMENT, 2023, 60(2): 103207.被引 3
查看全部 →

发表信息

论文状态
公开发表
期刊
IEEE Transactions on Image Processing
发表日期
2021
卷期
第 30 卷
页码
2758-2770
DOI
10.1109/TIP.2021.3051756
语言
ENGLISH
期刊收录
NSL
期刊等级
1区
成果收录
SCIE
资助标注
National Natural Science Foundation of China [61625107, 61836002, 62072397, 61732007, 61932009]; National Key Research and Development Program of China [2019YFA0706200]; Zhejiang Natural Science Foundation [LR19F020006]; Major Scientifc Research Project of Zhejiang Lab [2018EC0ZX01]

作者

  • G

    Gu, Mao (第一作者)

    zhejiang univ

  • Z

    Zhao, Zhou (通讯)

    zhejiang univ

查看全部 5 位作者收起作者
  • J

    Jin, Weike

    zhejiang univ

  • H

    Hong, Richang

    hefei univ technol

  • W

    Wu, Fei

    zhejiang univ

登录解锁更多功能

  • 收藏这篇,随时在“我的 · 收藏”找回
  • 认领你的成果,生成推广素材
  • 关注作者获取最新成果
  • 加入项目组、协同写作
注册 / 登录

产品服务

  • 科研之友
  • 创新城
  • 科创云

服务支持

  • 帮助中心
  • 隐私政策
  • 服务条款

联系方式

在线客服:【】
客服热线:400-1616-289
电子邮箱:support@scholarmate.com

关注或下载科研之友

微信二维码
微信公众号
客户端下载二维码
下载客户端
科研成果科研人员科研机构科研动态爱瑞思软件

©2026 深圳市科研之友网络服务有限公司

公安备案图标粤公网安备 44030502000213
粤ICP备 16046710 号粤B2-20110417