ScholarMate
English简体中文繁體中文
客服热线:400-1616-289
登录注册
期刊论文

MRN: Moment Relation Network for Natural Language Video Localization with Transfer Learning

Jiang, Siyu; Wu, Guobin*guangdong univ foreign studies
INTERNATIONAL JOURNAL OF PATTERN RECOGNITION AND ARTIFICIAL INTELLIGENCE2021-6-15DOI: 10.1142/S0218001421520091
1被引8阅读
引用收藏
分享

摘要

In this paper, we tackle the task of natural language video localization (NLVL): given an untrimmed video and a description language query, the goal is to localize the temporal segment within the video that best describes the natural language description. NLVL is challenging at the intersection of language and video understanding because a video may contain multiple segments of interests and the language may describe complicated temporal dependencies. Though existing approaches have achieved good performance, most of them did not fully consider the inherent differences between language and video modalities. Here, we propose Moment Relation Network (MRN) to reduce the divergence of the probability distribution of these two modalities. Specifically, MRN trains video and language subnets, and then uses transfer learning techniques to map the extracted features into an embedding-shared space where we calculate the similarity of two modalities using Mahalanobis distance metric, which is used to localize moments. Extensive experiments on benchmark datasets show that the proposed MRN significantly outperforms the state-of-the-art under the widely used metrics by a large margin.

学科与关键词

关键词
Metric transfer learningnatural language video localizationvideo retrieval
相关
加载中…
  • A Survey on Video Moment Localization
    Liu, Meng; Nie, Liqiang*; Wang, Yunxiao; Wang, Meng; Rui, YongACM Computing Surveys, 2023, 55(9): 188.
  • Hierarchical Representation Network With Auxiliary Tasks for Video Captioning and Video Question Answering
    Gao, Lianli; Lei, Yu; Zeng, Pengpeng; Song, Jingkuan; Wang, Meng; Shen, Heng Tao*IEEE Transactions on Image Processing, 2022, 31: 202-215.被引 31
  • Video Moment Retrieval With Cross-Modal Neural Architecture Search
    Yang, Xun; Wang, Shanshan; Dong, Jian; Dong, Jianfeng; Wang, Meng; Chua, Tat-SengIEEE Transactions on Image Processing, 2022, 31: 1204-1216.被引 23
查看全部 →

发表信息

论文状态
公开发表
期刊
INTERNATIONAL JOURNAL OF PATTERN RECOGNITION AND ARTIFICIAL INTELLIGENCE
发表日期
2021-06-15
卷期
第 35 卷, 第 07 期
页码
2152009
DOI
10.1142/S0218001421520091
语言
ENGLISH
期刊收录
NSL
期刊等级
4区
成果收录
SCIE

作者

  • J

    Jiang, Siyu (第一作者)

    guangdong univ foreign studies

  • W

    Wu, Guobin (通讯)

登录解锁更多功能

  • 收藏这篇,随时在“我的 · 收藏”找回
  • 认领你的成果,生成推广素材
  • 关注作者获取最新成果
  • 加入项目组、协同写作
注册 / 登录

产品服务

  • 科研之友
  • 创新城
  • 科创云

服务支持

  • 帮助中心
  • 隐私政策
  • 服务条款

联系方式

在线客服:【】
客服热线:400-1616-289
电子邮箱:support@scholarmate.com

关注或下载科研之友

微信二维码
微信公众号
客户端下载二维码
下载客户端
科研成果科研人员科研机构科研动态爱瑞思软件

©2026 深圳市科研之友网络服务有限公司

公安备案图标粤公网安备 44030502000213
粤ICP备 16046710 号粤B2-20110417