ScholarMate
English简体中文繁體中文
客服热线:400-1616-289
登录注册
期刊论文

A Developmental Approach for Training Deep Belief Networks

Zambra, Matteo; Testolin, Alberto*; Zorzi, Marco*
Cognitive Computation2023-1DOI: 10.1007/s12559-022-10085-5
10被引6阅读
引用收藏
分享

摘要

Deep belief networks (DBNs) are stochastic neural networks that can extract rich internal representations of the environment from the sensory data. DBNs had a catalytic effect in triggering the deep learning revolution, demonstrating for the very first time the feasibility of unsupervised learning in networks with many layers of hidden neurons. These hierarchical architectures incorporate plausible biological and cognitive properties, making them particularly appealing as computational models of human perception and cognition. However, learning in DBNs is usually carried out in a greedy, layer-wise fashion, which does not allow to simulate the holistic maturation of cortical circuits and prevents from modeling cognitive development. Here we present iDBN, an iterative learning algorithm for DBNs that allows to jointly update the connection weights across all layers of the model. We evaluate the proposed iterative algorithm on two different sets of visual stimuli, measuring the generative capabilities of the learned model and its potential to support supervised downstream tasks. We also track network development in terms of graph theoretical properties and investigate the potential extension of iDBN to continual learning scenarios. DBNs trained using our iterative approach achieve a final performance comparable to that of the greedy counterparts, at the same time allowing to accurately analyze the gradual development of internal representations in the deep network and the progressive improvement in task performance. Our work paves the way to the use of iDBN for modeling neurocognitive development.

学科与关键词

关键词
Unsupervised deep learningComputational modelingCognitive developmentHierarchical generative modelsIterative learning
相关
加载中…
  • Deep learning systems as complex networks
    Testolin, Alberto; Piccolini, Michele; Suweis, Samir*Journal of Complex Networks, 2020, 8(1): cnz018.被引 14
  • Towards Deep Developmental Learning
    Sigaud, Olivier*; Droniou, AlainIEEE Transactions on Cognitive and Developmental Systems, 2016, 8(2): 99-114.被引 24
  • A random energy approach to deep learning
    Xie, Rongrong; Marsili, Matteo*JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT, 2022, 2022(7): 073404.被引 1
查看全部 →

发表信息

论文状态
公开发表
期刊
Cognitive Computation
发表日期
2023-01
卷期
第 15 卷, 第 1 期
页码
103-120
DOI
10.1007/s12559-022-10085-5
语言
ENGLISH
成果收录
SCIE
资助标注
Cariparo Excellence Grant (project "Numsense")

作者

  • Z

    Zambra, Matteo (第一作者)

  • T

    Testolin, Alberto (通讯)

查看全部 3 位作者收起作者
  • Z

    Zorzi, Marco (通讯)

登录解锁更多功能

  • 收藏这篇,随时在“我的 · 收藏”找回
  • 认领你的成果,生成推广素材
  • 关注作者获取最新成果
  • 加入项目组、协同写作
注册 / 登录

产品服务

  • 科研之友
  • 创新城
  • 科创云

服务支持

  • 帮助中心
  • 隐私政策
  • 服务条款

联系方式

在线客服:【】
客服热线:400-1616-289
电子邮箱:support@scholarmate.com

关注或下载科研之友

微信二维码
微信公众号
客户端下载二维码
下载客户端
科研成果科研人员科研机构科研动态爱瑞思软件

©2026 深圳市科研之友网络服务有限公司

公安备案图标粤公网安备 44030502000213
粤ICP备 16046710 号粤B2-20110417