On Variational Message Passing and Its Relation to Other Message Passing Inference Algorithms
スポンサーリンク
概要
- 論文の詳細を見る
Variational methods are frequently used for performing inference in graphical models. The sum-product algorithm is often intractable for systems with continuous variables, and variational methods are then an interesting alternative; moreover, variational methods are guaranteed to convergence. On the other hand, it is well known that for discrete variables, the sum-product algorithm usually leads to better results than variational methods (since the Bethe approximation to the Gibbs free energy is more accurate than the mean field approximation); however, the sum-product algorithm is not guaranteed to convergence on cyclic graphs. Those considerations naturally lead to the following questions: Can one perhaps mix both approaches in one algorithm in order to combine their strengths? Which of those combinations lead to variational methods, and hence, are guaranteed to converge? Can such combinations be derived directly from a factor graph of the system at hand by mechanically applying message computation rules, in this way. bypassing error-prone variational calculus? In this note, an answer to those question is outlined. The key idea is to formulate variational methods as message passing algorithms operating on factor graphs, elaborating on earlier work by Beal et al., Winn et al. and Xing et al. In this note, it is also shown that the variational message passing algorithm is strongly related to the message passing formulation of expectation maximization (EM). It is demonstrated how variational methods can be combined with various other message passing algorithms, e.g., Kalman filters and smoothers, iterated conditional modes, EM, gradient methods, and particle filters. Some of those combinations have been explored in the literature, others seem to be new. Generic message computation rules for such combinations are formulated.
- 一般社団法人電子情報通信学会の論文
- 2006-11-28
著者
関連論文
- Quantifying the Synchrony Between Two Event Strings by Performing Inference in a Probabilistic Model
- On Variational Message Passing and Its Relation to Other Message Passing Inference Algorithms