A Theoretical Analysis of On-Line Learning Using Correlated Examples
スポンサーリンク
概要
- 論文の詳細を見る
In this paper we analytically investigate the generalization performance of learning using correlated inputs in the framework of online learning with a statistical mechanical method. We consider a model composed of linear perceptrons with Gaussian noise. First, we analyze the case of the gradient method. We analytically clarify that the larger the correlation among inputs is or the larger the number of inputs is, the stricter the condition the learning rate should satisfy is, and the slower the learning speed is. Second, we treat the block orthogonal projection learning as an alternative learning rule and derive the theory. In a noiseless case, the learning speed does not depend on the correlation and is proportional to the number of inputs used in an update. The learning speed is identical to that of the gradient method with uncorrelated inputs. On the other hand, when there is noise, the larger the correlation among inputs is, the slower the learning speed is and the larger the residual generalization error is.
- （社）電子情報通信学会の論文
- 2008-09-01
著者
-
Miyoshi Seiji
Department Of Electronic Engineering Kobe City College Of Technology
-
Miyoshi Seiji
Department Of Electrical And Electronic Engineering Faculty Of Engineering Science Kansai University
-
Miyoshi Seiji
Kansai Univ. Suita‐shi Jpn
-
Sakurai Shingo
Advanced Course Of Electrical And Electronic Engineering Kobe City College Of Technology
-
SEKI Chihiro
Advanced course of Electrical and Electronic Engineering, Kobe City College of Technology
-
MATSUNO Masafumi
Fujitsu FSAS Inc.
-
Seki Chihiro
Advanced Course Of Electrical And Electronic Engineering Kobe City College Of Technology
関連論文
- Effect of Slow Switching of Ensemble Teachers in On-line Learning
- Statistical Mechanics of Online Learning for Ensemble Teachers(General)
- A Theoretical Analysis of On-Line Learning Using Correlated Examples
- Theory of Time Domain Ensemble On-Line Learning of Perceptron under the Existence of External Noise(General)
- Statistical Mechanics of Time-Domain Ensemble Learning(General)
- Estimation of Distribution Algorithm Incorporating Switching
- Associative memory by recurrent neural networks with delay elements
- Statistical Mechanical Analysis of Simultaneous Perturbation Learning
- Statistical Mechanics of Nonlinear On-line Learning for Ensemble Teachers(General)
- Analysis of On-Line Learning when a Moving Teacher Goes around a True Teacher(General)
- Statistical Mechanics of On-line Learning When a Moving Teacher Goes around an Unlearnable True Teacher(General)
- Statistical Mechanics of Linear and Nonlinear Time-Domain Ensemble Learning(General)
- Effect of Slow Switching of Ensemble Teachers in On-line Learning