Statistical Mechanics of Online Learning for Ensemble Teachers(General)
スポンサーリンク
概要
- 論文の詳細を見る
We analyze the generalization performance of a student in a model composed of linear perceptrons: a true teacher, ensemble teachers, and the student. Calculating the generalization error of the student analytically using statistical mechanics in the framework of on-line learning, we prove that when the learning rate satisfies η<1, the larger the number K is and the more variety the ensemble teachers have, the smaller the generalization error is. On the other hand, when η>1, the properties are completely reversed. If the variety of the ensemble teachers is rich enough, the direction cosine between the true teacher and the student becomes unity in the limit of η→0 and K→∞. Intuitive interpretations of these results are given.
- 社団法人日本物理学会の論文
- 2006-04-15
著者
-
OKADA Masato
Division of Protein Metabolism, Institute for Protein Research, Osaka University
-
Miyoshi Seiji
Department Of Electronic Engineering Kobe City College Of Technology
-
Miyoshi Seiji
Department Of Electrical And Electronic Engineering Faculty Of Engineering Science Kansai University
-
Okada Masato
Division Of Protein Metabolism Institute For Protein Research Osaka University
-
OKADA Masato
Division of Transdisciplinary Sciences, Graduate School of Frontier Sciences, The University of Tokyo:RIKEN Brain Science Institute:JST PRESTO
-
Miyoshi Seiji
Department of Electronic Engineering, Kobe City College of Technology
-
Okada Masato
Division of Allergy and Rheumatology, St. Luke's International Hospital, Japan
関連論文
- Comparison of Survival-Promoting Effects of Brain-Derived Neurotrophic Factor and Neurotrophin-3 on PC12h Cells Stably Expressing TrkB Receptor^1
- Effect of Slow Switching of Ensemble Teachers in On-line Learning
- Statistical Mechanics of Online Learning for Ensemble Teachers(General)
- A Theoretical Analysis of On-Line Learning Using Correlated Examples
- Distinction of Coexistent Attractors in an Attractor Neural Network Model Using a Relaxation Process of Fluctuations in Firing Rates : Analysis with Statistical Mechanics(General)
- Theory of Time Domain Ensemble On-Line Learning of Perceptron under the Existence of External Noise(General)
- Statistical Mechanics of Time-Domain Ensemble Learning(General)
- Depolarization-Induced Tyrosine Phosphorylation of p130^
- Estimation of Distribution Algorithm Incorporating Switching
- Associative memory by recurrent neural networks with delay elements
- Statistical Mechanical Analysis of Simultaneous Perturbation Learning
- A Novel Clock-Related Protein, SCOP, in the Suprachiasmatic Nucleus
- Statistical Mechanics of Nonlinear On-line Learning for Ensemble Teachers(General)
- Analysis of On-Line Learning when a Moving Teacher Goes around a True Teacher(General)
- Statistical Mechanics of On-line Learning When a Moving Teacher Goes around an Unlearnable True Teacher(General)
- Statistical Mechanics of Linear and Nonlinear Time-Domain Ensemble Learning(General)
- Effect of Slow Switching of Ensemble Teachers in On-line Learning
- Belief Propagation for Error Correcting Codes and Lossy Compression Using Multilayer Perceptrons
- Image Restoration and Segmentation using Region-Based Latent Variables: Bayesian Inference Based on Variational Method