A Probabilistic Algorithm to Calculate the Learning Curves of Hierarchical Learning Machines with Singularities

スポンサーリンク

概要

• 論文の詳細を見る

• Recently we have proven that the learning curve of Bayesian estimation is determined by the algebraic geometrical property of singularities in the parameter space. In order to obtain the concrete coefficients of the learning curve, we need Hironaka' s resolution of singularities or Sato-Bernstein's b-function of the Kullback distance. However, computational complexities of them are unknown, and it is still difficult to calculate the learning curves in complex learning machines. In this paper, we prove a theorem that the coefficient of the learning curve is equal to the order of the volume V (t) of parameters whose Kullback distance is smaller than t, and propose a new probabilistic algorithm to calculate the learning curve based on the theorem. Effectiveness and efficiency of the proposed method are shown by applications of the method to a three-layer perceptron.

• 一般社団法人電子情報通信学会の論文
• 2002-03-01

著者

• Yamazaki Keisuke

  Interdisciplinary Graduate School Of Science And Engineering Tokyo Institute Of Technology

• WATANABE Sumio

  P&I Lab., Tokyo Institute of Technology

関連論文

• Stochastic complexities of general mixture models in variational Bayesian learning
• Singularities in Learning Theory (Recent Topics on Real and Complex Singularities RIMS研究集会報告集)
• A Probabilistic Algorithm to Calculate the Learning Curves of Hierarchical Learning Machines with Singularities

スポンサーリンク