Learning from Stochastic Rules by Spherical Perceptrons under Finite Temperature ---Optimal Temperature and Asymptotic Learning Curve---
スポンサーリンク
概要
- 論文の詳細を見る
In the problem of learning under external disturbance, there is a possibility that the existence of some tolerance or flexibility in the system weakens the effect of noise and helps the system to perform more efficiently. In a previous letter, we gave one example of such phenomena in learning from stochastic rules by spherical perceptrons adopting the Gibbs algorithm using statistical mechanical methods. By the replica method, we showed that, in the output noise model, there exists an optimal temperature at which the generalization error takes its minimum for the stable replica symmetric (RS) solution. On the other hand, for other types of noise including input noise, it was shown that no such temperature exists up to the one-step replica symmetry breaking (1RSB) solution. That is, it was shown that for the asymptotic region of a large number of training sets, the RS solution becomes unstable, and the asymptotic behavior is determined by the 1RSB solution, The asymptotic expressions for learning curves were derived, and it turned out that, within the 1RSB solution, the learning curve does not depend on temperature. In this study, we give a detailed derivation of these results and also the results obtained by simulated annealing and exchange Monte Carlo simulation. The numerical results support the theoretical predictions.
- Physical Society of Japanの論文
- 2011-04-15
著者
-
Uezu Tatsuya
Graduate School Of Human Culture Nara Women's University
-
Kiyokawa Shuji
Faculty of Science, Department of Physics, Nara Women's University, Nara 630, Japan
関連論文
- Statistical Mechanical Study of Code-Division Multiple-Access Multiuser Detectors : Analysis of Replica Symmetric and One-Step Replica Symmetry Breaking Solutions(General)
- Statistical Mechanical Analysis of CDMA Multiuser Detectors : AT Stability and Entropy of the RS Solution, and 1RSB Solution
- On-line Learning of Perceptron from Noisy Data by One and Two Teachers(General)
- Multiple Stability of a Sparsely Encoded Attractor Neural Network Model for the Inferior Temporal Cortex(General)
- Distinction of Coexistent Attractors in an Attractor Neural Network Model Using a Relaxation Process of Fluctuations in Firing Rates : Analysis with Statistical Mechanics(General)
- Dynamical Behavior of Phase Oscillator Networks on the Bethe Lattice(General)
- Theory of Time Domain Ensemble On-Line Learning of Perceptron under the Existence of External Noise(General)
- Statistical Mechanics of Time-Domain Ensemble Learning(General)
- Response to Invasion by Antigens and Effects of Threshold in an Immune Network Dynamical System Model with a Small Number of Degrees of Freedom
- Analysis of an Immune Network Dynamical System Model with a Small Number of Degrees of Freedom
- A Large Scale Dynamical System Immune Network Model with Finite Connectivity(Oscillation, Chaos and Network Dynamics in Nonlinear Science)
- Analysis of XY Model with Mexican-Hat Interaction on a Circle
- On the Conditions for the Existence of Perfect Learning and Power Law Behaviour in Learning from Stochastic Examples by Ising Perceptrons
- Learning of Non-Monotonic Target Functions by Ising Perceptrons : Learning Curve, Perfect Learning and Perfect Anti-Learning (General)
- Online Learning of Perceptron from Noisy Data: A Case in which Both Student and Teacher Suffer from External Noise
- Retrieval Properties of Hopfield and Correlated Attractors in an Associative Memory Model (General)
- Analysis of XY Model with Mexican-Hat Interaction on a Circle ---Derivation of Saddle Point Equations and Study of Bifurcation Structure---
- Solvable Model of a Phase Oscillator Network on a Circle with Infinite-Range Mexican-Hat-Type Interaction
- Multiple Stability of a Sparsely Encoded Attractor Neural Network Model for the Inferior Temporal Cortex
- Statistical Mechanical Analysis of CDMA Multiuser Detectors : AT Stability and Entropy of the RS Solution, and 1RSB Solution
- Stability Analysis of Attractor Neural Network Model of Inferior Temporal Cortex —Relationship between Attractor Stability and Learning Order—
- Learning from Stochastic Rules by Spherical Perceptrons under Finite Temperature ---Optimal Temperature and Asymptotic Learning Curve---
- Continuous Attractor that Appears in Autoassociative Memory Model Extended to XY Spin System
- Superconvergence of Period-Doubling Cascade in Trapezoid Maps : Its Rigorous Proof and Superconvergence of the Period-Doubling Cascade Starting from a Period p Solution