Slow Dynamics Due to Singularities of Hierarchical Learning Machines

スポンサーリンク

概要

• 論文の詳細を見る

• Recently, slow dynamics in learning of neural networks has been known to be closely related to singularities, which exist in parameter spaces of hierarchical learning models. To show the influence of singular structure on learning dynamics, we take statistical mechanical approaches and investigate online-learning dynamics under various learning scenario with different relationship between optimum and singularities. From the investigation, we found a quasi-plateau phenomenon which differs from the well known plateau. The quasi-plateau and plateau become extremely serious when an optimal point is in a neighborhood of a singularity. The quasi-plateau and plateau disappear in the natural gradient learning, which takes singular structures into account and uses Riemannian measure for the parameter space.

• 理論物理学刊行会の論文
• 2005-04-30

著者

• INOUE Masato

  Department of Behavioral and Brain Sciences, Primate Research Institute, Kyoto University

• OKADA Masato

  Department of Complexity Science and Engineering, Graduate School of Frontier Sciences, The Universi

• Park Hyeyoung

  Riken Brain Science Institute

• PARK Hyeyoung

  Computer Science Dept., Kyungpook National Univ.

• Inoue Masato

  Department Of Electrical Engineering And Bioscience School Of Science And Engineering Waseda Univers

• Inoue Masato

  Department Of Behavioral And Brain Sciences Primate Research Institute Kyoto University

• Okada Masato

  Graduate School Of Frontier Sciences The University Of Tokyo

• Okada Masato

  Department Of Complexity Science And Engineering Graduate School Of Frontier Sciences The University

• Okada Masato

  Department Of Bioscience Tokyo University Of Agriculture

関連論文

• Neurons in the macaque orbitofrontal cortex code relative preference of both rewarding and aversive outcomes
• Statistical Mechanics for Neural Spike Data Analysis Using Log-Linear Model
• Analysis Method Combining Monte Carlo Simulation and Principal Component Analysis : Application to Sourlas Code(General)
• A PCA Approach to Sourlas Code Analysis
• Grouping preprocess to accurately extend application of EM algorithm to haplotype inference
• New Device for Endoscopic Image Display During Microsurgical Clipping of Cerebral Aneurysms : Technical Note
• Statistical Mechanical Study of Code-Division Multiple-Access Multiuser Detectors : Analysis of Replica Symmetric and One-Step Replica Symmetry Breaking Solutions(General)
• Statistical Mechanical Analysis of CDMA Multiuser Detectors : AT Stability and Entropy of the RS Solution, and 1RSB Solution
• Nucleotide Sequence of a Principal Sigma Factor Gene (hrdB) of Streptomyces griseus
• Retrieval Property of Attractor Network with Synaptic Depression(General)
• Analysis of Ensemble Learning Using Simple Perceptrons Based on Online Learning Theory
• Characterization of mRNA Expression of IκBα and NF-κB Subfamilies in Primary Adult T-cell Leukemia Cells
• Sparse and Dense Encoding in Layered Associative Network of Spiking Neurons(Cross-disciplinary physics and related areas of science and technology)
• Theory of Interaction of Memory Patterns in Layered Associative Networks(Cross-disciplinary physics and related areas of science and technology)
• Residual Energies after Slow Quantum Annealing(General)
• Slow Dynamics Due to Singularities of Hierarchical Learning Machines
• On-Line Learning Dynamics of Multilayer Perceptrons with Unidentifiable Parameters
• Multiple Stability of a Sparsely Encoded Attractor Neural Network Model for the Inferior Temporal Cortex(General)
• Globally Coupled Resonate-and-Fire Models(Oscillation, Chaos and Network Dynamics in Nonlinear Science)
• Reliability and validity of Kasahara's scale of melancholic type of personality (Typus melancholicus) in a German sample population
• Mental health problems after stroke
• Constitutive activation of neuronal Src causes aberrant dendritic morphogenesis in mouse cerebellar Purkinje cells
• Perception of shape-from-motion in macaque monkeys and humans
• Detection of Herpes Simplex DNA in Semen and Menstrual Blood of Individuals Attending an Infertility Clinic
• Naive Mean Field Approximation for Image Restoration
• Statistical Mechanics of Mexican-Hat-Type Horizontal Connection
• Stochastic Transitions of Attractors in Associative Memory Models with Correlated Noise(Condensed matter: structure and mechanical and thermal properties)
• Many Body Effect in Inner Shell Photoemission and Photoabsorption Spectra of La Compounds
• Theory of Recurrent Neural Network with Common Synaptic Inputs(General)
• Linear CDMA Detection Algorithm Based on Statistical Neurodynamics and Belief Propagation and the Stability Conditions
• Bayesian-Optimal Image Reconstruction for Translational-Symmetric Filters(Cross-disciplinary physics and related areas of science and technology)
• Ensemble Learning of Linear Perceptrons : On-Line Learning Theory(General)
• Ambiguity in the Effective Potential of Composite Fields
• Neural Network Model of Spatial Memory: Associative Recall of Maps
• Statistical Mechanics of Mutual Learning with a Latent Teacher(General)
• Novel antibacterial compounds specifically targeting the essential WalR response regulator
• Subjective Symptoms among Female Workers and Winter Working Conditions in a Consumer Cooperative
• Defective Carbon Nanotube for Use as a Thermal Rectifier
• Slow Dynamics Due to Singularities of Hierarchical Learning Machines

もっと見る 閉じる

スポンサーリンク