ScholarMate
English简体中文繁體中文
客服热线:400-1616-289
登录注册
期刊论文

A robotic model of hippocampal reverse replay for reinforcement learning

Whelan, Matthew T.; Jimenez-Rodriguez, Alejandro; Prescott, Tony J.; Vasilaki, Eleni*
Bioinspiration & Biomimetics2023-1-1DOI: 10.1088/1748-3190/ac9ffc
1被引13阅读
引用收藏
分享

摘要

Hippocampal reverse replay, a phenomenon in which recently active hippocampal cells reactivate in the reverse order, is thought to contribute to learning, particularly reinforcement learning (RL), in animals. Here, we present a novel computational model which exploits reverse replay to improve stability and performance on a homing task. The model takes inspiration from the hippocampal-striatal network, and learning occurs via a three-factor RL rule. To augment this model with hippocampal reverse replay, we derived a policy gradient learning rule that associates place-cell activity with responses in cells representing actions and a supervised learning rule of the same form, interpreting the replay activity as a 'target' frequency. We evaluated the model using a simulated robot spatial navigation task inspired by the Morris water maze. Results suggest that reverse replay can improve performance stability over multiple trials. Our model exploits reverse reply as an additional source for propagating information about desirable synaptic changes, reducing the requirements for long-time scales in eligibility traces combined with low learning rates. We conclude that reverse replay can positively contribute to RL, although less stable learning is possible in its absence. Analogously, we postulate that reverse replay may enhance RL in the mammalian hippocampal-striatal system rather than provide its core mechanism.

学科与关键词

关键词
hippocampal replyreinforcement learningroboticscomputational neuroscience
相关
加载中…
  • Learning predictive cognitive maps with spiking neurons during behavior and replays
    Bono, Jacopo; Zannone, Sara; Pedrosa, Victor; Clopath, Claudia*eLife, 2023, 12: e80671.被引 11
  • A Hippocampal Model for Behavioral Time Acquisition and Fast Bidirectional Replay of Spatio-Temporal Memory Sequences
    Gauy, Marcelo Matheus*; Lengler, Johannes; Einarsson, Hafsteinn; Meier, Florian; Weissenberger, Felix; Yanik, Mehmet Fatih; Steger, AngelikaFrontiers in Neuroscience, 2018, 12: 961.被引 8
  • Replays of spatial memories suppress topological fluctuations in cognitive map
    Babichev, Andrey; Morozov, Dmitriy; Dabaghian, Yuri*Network Neuroscience, 2019, 3(3): 707-724.被引 9
查看全部 →

发表信息

论文状态
公开发表
期刊
Bioinspiration & Biomimetics
发表日期
2023-01-01
卷期
第 18 卷, 第 1 期
页码
015007
DOI
10.1088/1748-3190/ac9ffc
语言
ENGLISH
成果收录
SCIE
资助标注
EPSRC [EP/S030964/1] Funding Source: UKRI

作者

  • W

    Whelan, Matthew T. (第一作者)

  • J

    Jimenez-Rodriguez, Alejandro

查看全部 4 位作者收起作者
  • P

    Prescott, Tony J.

  • V

    Vasilaki, Eleni (通讯)

登录解锁更多功能

  • 收藏这篇,随时在“我的 · 收藏”找回
  • 认领你的成果,生成推广素材
  • 关注作者获取最新成果
  • 加入项目组、协同写作
注册 / 登录

产品服务

  • 科研之友
  • 创新城
  • 科创云

服务支持

  • 帮助中心
  • 隐私政策
  • 服务条款

联系方式

在线客服:【】
客服热线:400-1616-289
电子邮箱:support@scholarmate.com

关注或下载科研之友

微信二维码
微信公众号
客户端下载二维码
下载客户端
科研成果科研人员科研机构科研动态爱瑞思软件

©2026 深圳市科研之友网络服务有限公司

公安备案图标粤公网安备 44030502000213
粤ICP备 16046710 号粤B2-20110417