摘要
We study a generic ensemble of deep belief networks (DBN) which is parametrized by the distribution of energy levels of the hidden states of each layer. We show that, within a random energy approach, statistical dependence can propagate from the visible to deep layers only if each layer is tuned close to the critical point during learning. As a consequence, efficiently trained learning machines are characterised by a broad distribution of energy levels. The analysis of DBNs and restricted Boltzmann machines on different datasets confirms these conclusions.
学科与关键词
关键词
deep learningenergy landscapeslearning theorymachine learning
- Deep learning systems as complex networksTestolin, Alberto; Piccolini, Michele; Suweis, Samir*Journal of Complex Networks, 2020, 8(1): cnz018.被引 14
- A Developmental Approach for Training Deep Belief NetworksZambra, Matteo; Testolin, Alberto*; Zorzi, Marco*Cognitive Computation, 2023, 15(1): 103-120.被引 10
- Maximal relevance and optimal learning machinesDuranthon, O.; Marsili, M.*; Xie, R.JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT, 2021, 2021(3): 033409.被引 6
