Statistical Mechanics of Nonlinear On-line Learning for Ensemble Teachers(General)

スポンサーリンク

概要

• 論文の詳細を見る

• We analyze the generalization performance of a student in a model composed of nonlinear perceptrons : a true teacher, ensemble teachers, and the student. We calculate the generalization error of the student analytically or numerically using statistical mechanics in the framework of on-line learning. We treat two well-known learning rules : Hebbian learning and perceptron learning. As a result, it is proven that the nonlinear model shows qualitatively different behaviors from the linear model. Moreover, it is clarified that Hebbian learning and perceptron learning show qualitatively different behaviors from each other. In Hebbian learning, we can analytically obtain the solutions. In this case, the generalization error monotonically decreases. The steady value of the generalization error is independent of the learning rate. The larger the number of teachers is and the more variety the ensemble teachers have, the smaller the generalization error is. In perceptron learning, we have to numerically obtain the solutions. In this case, the dynamical behaviors of the generalization error are nonmonotonic. The smaller the learning rate is, the larger the number of teachers is ; and the more variety the ensemble teachers have, the smaller the minimum value of the generalization error is.

• 社団法人日本物理学会の論文
• 2007-11-15

著者

• OKADA Masato

  Division of Protein Metabolism, Institute for Protein Research, Osaka University

• Miyoshi Seiji

  Department Of Electronic Engineering Kobe City College Of Technology

• Miyoshi Seiji

  Department Of Electrical And Electronic Engineering Faculty Of Engineering Science Kansai University

• Okada Masato

  Division Of Transdisciplinary Sciences Graduate School Of Frontier Sciences The University Of Tokyo:

• UTSUMI Hideto

  Department of Electronic Engineering, Kobe City College of Technology

• Utsumi Hideto

  Department Of Electronic Engineering Kobe City College Of Technology:department Of Electrical And El

• Okada Masato

  Division Of Protein Metabolism Institute For Protein Research Osaka University

• Okada Masato

  Division of Allergy and Rheumatology, St. Luke's International Hospital, Japan

関連論文

• Comparison of Survival-Promoting Effects of Brain-Derived Neurotrophic Factor and Neurotrophin-3 on PC12h Cells Stably Expressing TrkB Receptor^1
• Effect of Slow Switching of Ensemble Teachers in On-line Learning
• Statistical Mechanics of Online Learning for Ensemble Teachers(General)
• A Theoretical Analysis of On-Line Learning Using Correlated Examples
• Distinction of Coexistent Attractors in an Attractor Neural Network Model Using a Relaxation Process of Fluctuations in Firing Rates : Analysis with Statistical Mechanics(General)
• Theory of Time Domain Ensemble On-Line Learning of Perceptron under the Existence of External Noise(General)
• Statistical Mechanics of Time-Domain Ensemble Learning(General)
• Depolarization-Induced Tyrosine Phosphorylation of p130^
• Estimation of Distribution Algorithm Incorporating Switching
• Associative memory by recurrent neural networks with delay elements
• Statistical Mechanical Analysis of Simultaneous Perturbation Learning
• A Novel Clock-Related Protein, SCOP, in the Suprachiasmatic Nucleus
• Statistical Mechanics of Nonlinear On-line Learning for Ensemble Teachers(General)
• Analysis of On-Line Learning when a Moving Teacher Goes around a True Teacher(General)
• Statistical Mechanics of On-line Learning When a Moving Teacher Goes around an Unlearnable True Teacher(General)
• Statistical Mechanics of Linear and Nonlinear Time-Domain Ensemble Learning(General)
• Effect of Slow Switching of Ensemble Teachers in On-line Learning
• Belief Propagation for Error Correcting Codes and Lossy Compression Using Multilayer Perceptrons
• Image Restoration and Segmentation using Region-Based Latent Variables: Bayesian Inference Based on Variational Method

もっと見る 閉じる

スポンサーリンク