ScholarMate
English简体中文繁體中文
客服热线:400-1616-289
登录注册
会议论文

Multimodal Representation with Embedded Visual Guiding Objects for Named Entity Recognition in Social Media Posts

Wu, Zhiwei*; Zheng, Changmeng*; Cai, Yi; Chen, Junying; Leung, Ho-fung; Li, Qingchinese univ hong kong; hong kong polytech univ
28th ACM International Conference on Multimedia (MM)2020DOI: 10.1145/3394171.3413650
19被引4阅读
引用收藏
分享

摘要

Visual contexts often help to recognize named entities more precisely in short texts such as tweets or snapchat. For example, one can identify "Charlie" as a name of a dog according to the user posts. Previous works on multimodal named entity recognition ignore the corresponding relations of visual objects and entities. Visual objects are considered as fine-grained image representations. For a sentence with multiple entity types, objects of the relevant image can be utilized to capture different entity information. In this paper, we propose a neural network which combines object-level image information and character-level text information to predict entities. Vision and language are bridged by leveraging object labels as embeddings, and a dense co-attention mechanism is introduced for fine-grained interactions. Experimental results in Twitter dataset demonstrate that our method outperforms the state-of-the-art methods.

学科与关键词

关键词
multimodal named entity recognitionmodality gapfine-grained image representations
相关
加载中…
  • Object-Aware Multimodal Named Entity Recognition in Social Media Posts With Adversarial Learning
    Zheng, Changmeng; Wu, Zhiwei; Wang, Tao; Cai, Yi*; Li, QingIEEE TRANSACTIONS ON MULTIMEDIA, 2021, 23: 2520-2532.被引 321
  • PromptMNER: Prompt-Based Entity-Related Visual Clue Extraction and Integration for Multimodal Named Entity Recognition
    Wang, Xuwu; Tian, Junfeng; Gui, Min; Li, Zhixu*; Ye, Jiabo; Yan, Ming; Xiao, Yanghua*27th International Conference on Database Systems for Advanced Applications (DASFAA), IIIT Hyderabad, ELECTR NETWORK, 2022-04-11 To 2022-04-14.被引 6
  • Towards Bridged Vision and Language: Learning Cross-Modal Knowledge Representation for Relation Extraction
    Feng, Junhao; Wang, Guohua; Zheng, Changmeng*; Cai, Yi; Fu, Ze; Wang, Yaowei; Wei, Xiao-Yong; Li, QingIEEE Transactions on Circuits and Systems for Video Technology, 2024, 34(1): 561-575.
查看全部 →

会议信息

会议名称
28th ACM International Conference on Multimedia (MM)
会议日期
2020-10-12 至 2020-10-16
论文集名
MM '20: PROCEEDINGS OF THE 28TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA
发表日期
2020
起止页码
1038-1046
DOI
10.1145/3394171.3413650
资助标注
Fundamental Research Funds for the Central Universities, SCUT [D2182480]; Science and Technology Planning Project of Guangdong Province [2017B050506004]; Science and Technology Programs of Guangzhou [201704030076, 201707010223, 201802010027, 201902010046]; National Natural Science Foundation of China [61802130]; Guangdong Natural Science Foundation [2018A030310355]; Guangzhou Science and Technology Program [201707010223]; CUHK Direct Grant for Research [134920340]; Research Grants Council of the Hong Kong Special Administrative Region, China (Collaborative Research Fund) [C1031-18G]
收录
ISTP

作者

  • W

    Wu, Zhiwei (通讯) (第一作者)

  • Z

    Zheng, Changmeng (通讯)

查看全部 6 位作者收起作者
  • C

    Cai, Yi

  • C

    Chen, Junying

  • L

    Leung, Ho-fung

    chinese univ hong kong

  • L

    Li, Qing

    hong kong polytech univ

登录解锁更多功能

  • 收藏这篇,随时在“我的 · 收藏”找回
  • 认领你的成果,生成推广素材
  • 关注作者获取最新成果
  • 加入项目组、协同写作
注册 / 登录

产品服务

  • 科研之友
  • 创新城
  • 科创云

服务支持

  • 帮助中心
  • 隐私政策
  • 服务条款

联系方式

在线客服:【】
客服热线:400-1616-289
电子邮箱:support@scholarmate.com

关注或下载科研之友

微信二维码
微信公众号
客户端下载二维码
下载客户端
科研成果科研人员科研机构科研动态爱瑞思软件

©2026 深圳市科研之友网络服务有限公司

公安备案图标粤公网安备 44030502000213
粤ICP备 16046710 号粤B2-20110417