On-Line Learning Theory of Soft Committee Machines with Correlated Hidden Units : Steepest Gradient Descent and Natural Gradient Descent

スポンサーリンク

概要

• 論文の詳細を見る

• The permutation symmetry of the hidden units in multilayer perceptrons causes the saddle structure and plateaus of the learning dynamics in gradient learning methods. The correlation of the weight vectors of hidden units in a teacher network is thought to affect this saddle structure, resulting in a prolonged learning time, but this mechanism is still unclear. In this paper, we discuss it with regard to soft committee machines and on-line learning using statistical mechanics. Conventional gradient descent needs more time to break the symmetry as the correlation of the teacher weight vectors rises. On the other hand, no plateaus occur with natural gradient descent regardless of the correlation for the limit of a low learning rate. Analytical results support these dynamics around the saddle point.

• 社団法人日本物理学会の論文
• 2003-04-15

著者

• OKADA Masato

  Laboratory for Mathematical Neuroscience, RIKEN Brain Science Institute

• Okada M

  Riken Saitama

• Park Hyeyoung

  Laboratory For Mathematical Neuroscience Riken Brain Science Institute

• Inoue Masato

  Laboratory for Mathematical Neuroscience, Brain Science Institute, Institute of Physical and Chemica

• INOUEU Masato

  Laboratory for Mathematical Neuroscience, RIKEN Brain Science institute

• Inoueu Masato

  Laboratory For Mathematical Neuroscience Riken Brain Science Institute : Department Of Otolaryngolog

• Okada Masato

  Laboratory For Mathematical Neuroscience Brain Science Institute Riken

• INOUE Masasi

  Department of Material Science, Faculty of Science, Hiroshima University

関連論文

• A PCA Approach to Sourlas Code Analysis
• High Field Magnetooptics of Diluted Magnetic Semiconductors(Magnetooptics)
• High Field Magnetooptics of a Diluted Magnetic Semiconductor Cd_Co_xSe
• Angle-Resolved Inverse Photoemission Spectra of Layered 1T-VSe_2,1T-TiS_2,IT-TaS_2,2H-NbSe_2 and 2H-TaSe_2
• Dynamic Studies of Transport Properties of γ-Mo_4O_ Crystals Using Photoinduced Transient Thermoelectric Effect under Static Electric Field
• Photoemission Studies on Intercalation Compounds of M_xTiS_2 : (M=3d Transition Metals)
• Optical Properties of One-Dimensional Inflation Lattice. : I. Fractal Structure of Optical Spectrum
• Anisotropic Spin-Glass and Cluster-Glass of Layered Fe_xTiS_2 Crystals
• Anisotropies in the Magnetotransport Properties of Quai-Two-Dimensional η-Mo_4O_ Crystals
• Simulation of Atomic Distribution of Guest Atoms in Layered 1T-TiS_2 Crystal using Monte Calro Method and Its Effect on the Magnetic Properties and Local Structures : Condensed Matter: Structure, etc.
• Charge-Density Wave Instabilities in Orthorhombic γ-Mo_4O_ : II. LOW TEMPERATURE PROPERTIES OF SOLIDS : Charge Density Waves
• Point-Contact Spectroscopy of 3D Transition-Metal Intercalate Fe_x TiS_2 : II. LOW TEMPERATURE PROPERTIES OF SOLIDS : Metals and Semiconductors
• A Self-Consistent Electronic Structure Calculation of InAs-GaSb Interface
• Stimulus-Induced Behavior in F1 Hybrids of Seizure-Sensitive and Seizure-Resistant Gerbils(Neurobiology)
• Magnetotransport Measurements of η-Mo_4O_ Crystals Using a Hybrid Magnet(Transport and Fermiology)
• High Magnetic Field Transport Properties of η-Mo_4O_ Crystals
• Activities of INSAM in Hiroshima University
• On-Line Learning Theory of Soft Committee Machines with Correlated Hidden Units : Steepest Gradient Descent and Natural Gradient Descent
• Electronic Structure of Generalized Fibonacci Lattices. : II. The Energy Spectrum and the Stability Analysis
• Electronic Structure of Generalized Fibonacci Lattices.I.Invariant,Quasi-Invariant and Cyclic Orbits
• 2-D Oxygen Order in the 55 K-Superconducting Phase of Y-Ba-Cu-O
• Theory of Superconductivity of Inter-Layer Cooper Pairing
• On-line learning through simple perceptron learning with a margin
• Self-organization of globally continuous and locally distributed information representation
• Impact of deviation from precise balance of spike-timing-dependent plasticity
• Dynamically Coupled Oscillators : Cooperative Behavior via Dynamical Interaction
• Retrieval Properties of Hopfield and Correlated Attractors in an Associative Memory Model (General)
• A PCA Approach to Sourlas Code Analysis

もっと見る 閉じる

スポンサーリンク