An Approach to Improve Back-propagation algorithm by Using Adaptive Gain(<Special Issue>SOFT COMPUTING METHODOLOGIES AND ITS APPLICATIONS)

概要

論文の詳細を見る
Most of the gradient based optimization algorithms employed during training process of back propagation networks use negative gradient of error as a gradient based search direction. This paper presents a novel approach to improve the training efficiency of back propagation neural network algorithms by adaptively modifying the gradient based search direction. The proposed algorithm uses the value of gain parameter in the activation function to modify the gradient based search direction. It has been shown that this modification can significantly enhance the computational efficiency of training process. The proposed algorithm is generic and can be implemented in almost all gradient based optimization processes. The robustness of the proposed algorithm is shown by comparing convergence rates for gradient descent, conjugate gradient and quasi-Newton methods on many benchmark examples.
バイオメディカル・ファジィ・システム学会の論文