Generalization Performance of Subspace Bayes Approach in Linear Neural Networks(Algorithm Theory)

スポンサーリンク

概要

• 論文の詳細を見る

• In unidentifiable models, the Bayes estimation has the advantage of generalization performance over the maximum likelihood estimation. However, accurate approximation of the posterior distribution requires huge computational costs. In this paper, we consider an alternative approximation method, which we call a subspace Bayes approach. A subspace Bayes approach is an empirical Bayes approach where a part of the parameters are regarded as hyperparameters. Consequently, in some three-layer models, this approach requires much less computational costs than Markov chain Monte Carlo methods. We show that, in three-layer linear neural networks, a subspace Bayes approach is asymptotically equivalent to a positive-part James-Stein type shrinkage estimation, and theoretically clarify its generalization error and training error. We also discuss the domination over the maximum likelihood estimation and the relation to the variational Bayes approach.

• 社団法人電子情報通信学会の論文
• 2006-03-01

著者

• Nakajima Shinichi

  Tokyo Institute Of Technology:nikon Corporation

• WATANABE Sumio

  Tokyo Institute of Technology

• Watanabe Sumio

  Tokyo Inst. Of Technol. Yokohama‐shi Jpn

関連論文

• Generalization Performance of Subspace Bayes Approach in Linear Neural Networks(Algorithm Theory)
• Equations of States in Statistical Learning for on Unrealizable and Regular Case
• Algebraic geometrical methods for hierarchical learning machines

スポンサーリンク