Comparison of Convergence Behavior and Generalization Ability in Backpropagation Learning with Linear and Sigmoid Output Units

スポンサーリンク

概要

• 論文の詳細を見る

• The most commonly used activation function in Backpropagation learning is sigmoidal while linear function is also sometimes used at the output layer with the view that choice between these activation functions does not make considerable differences in network's performance. In this letter, we show distinct performance between a network with linear output units and a similar network with sigmoid output units in terms of convergence behavior and generalization ability.^<(10)> We experimented with two types of cost functions, namely, sum-squared error used in standard Backpropagation and log-likelihood recently reported.^<(3),(4)> We find that, with sum-squared error cost function and hidden units with nonsteep sigmoid function, use of linear units at the output layer instead of sigmoidal ones accelerates the convergence speed considerably while generalization ability is slightly degraded. Network with sigmoid output units trained by log-likelihood cost function yields even faster convergence and better generalization but does not converge at all with linear output units. It is also shown that a network with linear output units needs more hidden units for convergence.

• 社団法人電子情報通信学会の論文
• 1993-06-25

著者

• Kamruzzaman Joarder

  the Department of Electrical & Electronic Engineering, Bangladesh University of Engineering and Tech

• Kumagai Yukio

  the Department of Computer Science and Systems Engineering, Muroran Institute of Technology

• Kumagai Yukio

  Department Of Computer Science And Systems Engineering Muroran Institute Of Technology

• Kamruzzaman J

  Monash Univ. Aus

• Hikita Hiromitsu

  Department of Mechanical Systems Engineering, Muroran Institute of Technology

• Hikita Hiromitsu

  the Department of Mechanical Systems Engineering, Muroran Institute of Technology

• Hikita Hiromitsu

  Department Of Mechanical Systems Engineering Muroran Institute Of Technology

関連論文

• Invariant Object Recognition by Artificial Neural Network Using Fahlman and Lebiere's Learning Algorithm
• Performance Formulation and Evaluation of Associative Memory Extended to Higher Order
• Generalization Ability of Extended Cascaded Artificial Neural Network Architecture
• Comparison of Convergence Behavior and Generalization Ability in Backpropagation Learning with Linear and Sigmoid Output Units
• Robust Performance Using Cascaded Artificial Neural Network Architecture
• Incremental Learning and Generalization Ability of Artificial Neural Network Trained by Fahlman and Lebiere's Learning Algorithm

スポンサーリンク