ScholarMate
English简体中文繁體中文
客服热线:400-1616-289
登录注册
期刊论文

Depth Selection for Deep ReLU Nets in Feature Extraction and Generalization

Han, Zhi; Yu, Siquan; Lin, Shao-Bo*; Zhou, Ding-Xuan东北大学; 西安交通大学; 中国科学院
IEEE Transactions on Pattern Analysis and Machine Intelligence2022-4-1DOI: 10.1109/TPAMI.2020.3032422
16被引7阅读
引用收藏
分享

摘要

Deep learning is recognized to be capable of discovering deep features for representation learning and pattern recognition without requiring elegant feature engineering techniques by taking advantages of human ingenuity and prior knowledge. Thus it has triggered enormous research activities in machine learning and pattern recognition. One of the most important challenges of deep learning is to figure out relations between a feature and the depth of deep neural networks (deep nets for short) to reflect the necessity of depth. Our purpose is to quantify this feature-depth correspondence in feature extraction and generalization. We present the adaptivity of features to depths and vice-verse via showing a depth-parameter trade-off in extracting both single feature and composite features. Based on these results, we prove that implementing the classical empirical risk minimization on deep nets can achieve the optimal generalization performance for numerous learning tasks. Our theoretical results are verified by a series of numerical experiments including toy simulations and a real application of earthquake seismic intensity prediction.

学科与关键词

关键词
Feature extractionData miningDeep learningTask analysisOptimizationMachine learning algorithmsDeep netsfeature extractionsgeneralizationlearning theory
相关
加载中…
  • Realization of Spatial Sparseness by Deep ReLU Nets With Massive Data
    Chui, Charles K.; Lin, Shao-Bo*; Zhang, Bo; Zhou, Ding-XuanIEEE Transactions on Neural Networks and Learning Systems, 2022, 33(1): 229-243.被引 14
  • Approximation of Nonlinear Functionals Using Deep ReLU Networks (vol 29, 50, 2023)
    Song, Linhao; Fan, Jun*; Chen, Di-Rong; Zhou, Ding-XuanJOURNAL OF FOURIER ANALYSIS AND APPLICATIONS, 2023, 29(5): 57.
  • A random energy approach to deep learning
    Xie, Rongrong; Marsili, Matteo*JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT, 2022, 2022(7): 073404.被引 1
查看全部 →

发表信息

论文状态
公开发表
期刊
IEEE Transactions on Pattern Analysis and Machine Intelligence
发表日期
2022-04-01
卷期
第 44 卷, 第 4 期
页码
1853-1868
DOI
10.1109/TPAMI.2020.3032422
语言
ENGLISH
期刊等级
1区
成果收录
SCIE
资助标注
National Natural Science Foundation of China [61876133, 61977038, 61773367, 61821005]; Youth Innovation Promotion Association of the Chinese Academy of Sciences [2016183]; Research Grant Council of Hong Kong [CityU 11306617]; Hong Kong Institute for Data Science

作者

  • H

    Han, Zhi (第一作者)

    chinese acad sci · shenyang inst automat

  • Y

    Yu, Siquan

    chinese acad sci · shenyang inst automat · northeastern univ

查看全部 4 位作者收起作者
  • L

    Lin, Shao-Bo (通讯)

    xi an jiao tong univ

  • Z

    Zhou, Ding-Xuan

登录解锁更多功能

  • 收藏这篇,随时在“我的 · 收藏”找回
  • 认领你的成果,生成推广素材
  • 关注作者获取最新成果
  • 加入项目组、协同写作
注册 / 登录

产品服务

  • 科研之友
  • 创新城
  • 科创云

服务支持

  • 帮助中心
  • 隐私政策
  • 服务条款

联系方式

在线客服:【】
客服热线:400-1616-289
电子邮箱:support@scholarmate.com

关注或下载科研之友

微信二维码
微信公众号
客户端下载二维码
下载客户端
科研成果科研人员科研机构科研动态爱瑞思软件

©2026 深圳市科研之友网络服务有限公司

公安备案图标粤公网安备 44030502000213
粤ICP备 16046710 号粤B2-20110417