Slow Dynamics Due to Singularities of Hierarchical Learning Machines
スポンサーリンク
概要
- 論文の詳細を見る
Recently, slow dynamics in learning of neural networks has been known to be closely related to singularities, which exist in parameter spaces of hierarchical learning models. To show the influence of singular structure on learning dynamics, we take statistical mechanical approaches and investigate online-learning dynamics under various learning scenario with different relationship between optimum and singularities. From the investigation, we found a quasi-plateau phenomenon which differs from the well known plateau. The quasi-plateau and plateau become extremely serious when an optimal point is in a neighborhood of a singularity. The quasi-plateau and plateau disappear in the natural gradient learning, which takes singular structures into account and uses Riemannian measure for the parameter space.
- 理論物理学刊行会の論文
- 2005-04-30
著者
-
Park Hyeyoung
Riken Brain Science Institute
-
PARK Hyeyoung
Computer Science Dept., Kyungpook National Univ.
-
Inoue Masato
Department Of Behavioral And Brain Sciences Primate Research Institute Kyoto University
-
Okada Masato
Department Of Bioscience Tokyo University Of Agriculture
-
Okada Masato
"Intelligent Coorperation and Control", PRESTO, JST
-
INOUE Masato
Department of Computational Intelligence and Systems Science, Interdisciplinary Graduate School of Science and Engineering, Tokyo Institute of Technology:RIKEN BSI
-
OKADA Masato
RIKEN BSI:Japan Scientific Technology Corp.:Graduate School of Frontier Science, The University of Tokyo
-
OKADA Masato
Graduate School of Frontier Sciences, The University of Tokyo:RIKEN Brain Science Institute:Intelligent Cooperation and Control
-
INOUE Masato
Deparatment of Occupational Health, Graduate School of Medicine, Gifu University
関連論文
- Neurons in the macaque orbitofrontal cortex code relative preference of both rewarding and aversive outcomes
- Analysis Method Combining Monte Carlo Simulation and Principal Component Analysis : Application to Sourlas Code(General)
- A PCA Approach to Sourlas Code Analysis
- Grouping preprocess to accurately extend application of EM algorithm to haplotype inference
- New Device for Endoscopic Image Display During Microsurgical Clipping of Cerebral Aneurysms : Technical Note
- Nucleotide Sequence of a Principal Sigma Factor Gene (hrdB) of Streptomyces griseus
- Characterization of mRNA Expression of IκBα and NF-κB Subfamilies in Primary Adult T-cell Leukemia Cells
- Sparse and Dense Encoding in Layered Associative Network of Spiking Neurons(Cross-disciplinary physics and related areas of science and technology)
- Theory of Interaction of Memory Patterns in Layered Associative Networks(Cross-disciplinary physics and related areas of science and technology)
- Tight-Binding Approach to Initial Stage of the Graphitization Process on a Vicinal SiC Surface
- Slow Dynamics Due to Singularities of Hierarchical Learning Machines
- On-Line Learning Dynamics of Multilayer Perceptrons with Unidentifiable Parameters
- Reliability and validity of Kasahara's scale of melancholic type of personality (Typus melancholicus) in a German sample population
- Mental health problems after stroke
- Constitutive activation of neuronal Src causes aberrant dendritic morphogenesis in mouse cerebellar Purkinje cells
- Perception of shape-from-motion in macaque monkeys and humans
- Detection of Herpes Simplex DNA in Semen and Menstrual Blood of Individuals Attending an Infertility Clinic
- Stochastic Transitions of Attractors in Associative Memory Models with Correlated Noise(Condensed matter: structure and mechanical and thermal properties)
- Many Body Effect in Inner Shell Photoemission and Photoabsorption Spectra of La Compounds
- Linear CDMA Detection Algorithm Based on Statistical Neurodynamics and Belief Propagation and the Stability Conditions
- Bayesian-Optimal Image Reconstruction for Translational-Symmetric Filters(Cross-disciplinary physics and related areas of science and technology)
- Ambiguity in the Effective Potential of Composite Fields
- Novel antibacterial compounds specifically targeting the essential WalR response regulator
- Statistical Mechanics for Neural Spike Data Analysis Using Log-Linear Model
- Subjective Symptoms among Female Workers and Winter Working Conditions in a Consumer Cooperative
- Defective Carbon Nanotube for Use as a Thermal Rectifier
- Slow Dynamics Due to Singularities of Hierarchical Learning Machines
- A PCA Approach to Sourlas Code Analysis
- Analysis of Ensemble Learning Using Simple Perceptrons Based on Online Learning Theory