Theoretical analysis of learning speed in gradient descent algorithm replacing derivative with constant

スポンサーリンク

概要

• 論文の詳細を見る

• In on-line gradient descent learning, the local property of the derivative term of the output can slow convergence. Improving the derivative term, such as by using the natural gradient, has been proposed for speeding up the convergence. Beside this sophisticated method, we propose an algorithm that replace the derivative term with a constant in this paper and showed that this greatly increases convergence speed under some conditions. The proposed algorithm inspired by linear perceptron learning, and it can avoid locality of the derivative term. We derived the closed deterministic differential equations by using a statistical mechanics method and show validity of analytical solutions by comparing that of computer simulations.

• 2012-11-29

著者

• Kazuyuki Hara

  College of Industrial Technology, Nihon University

• Kazuyuki Hara

  College Of Industrial Technology Nihon University

• Kentaro Katahira

  Japan Science Technology Agency Erato Okanoya Emotional Information Project|brain Science Institute

• Kentaro Katahira

  Center For Evolutionary Cognitive Sciences The University Of Tokyo|brain Science Institute Riken

関連論文

• Statistical Mechanics of On-line Node-perturbation Learning
• Statistical Mechanics of On-Line Node-Perturbation Learning
• Statistical Mechanics of On-line Node-perturbation Learning
• Theoretical analysis of learning speed in gradient descent algorithm replacing derivative with constant
• Theoretical analysis of learning speed in gradient descent algorithm replacing derivative with constant
• Theoretical Analysis of Learning Speed in Gradient Descent Algorithm Replacing Derivative with Constant

スポンサーリンク