Statistical Mechanics of On-Line Mutual Learning with Many Linear Perceptrons

スポンサーリンク

概要

• 論文の詳細を見る

• We propose a new mutual learning using many weak learners (or students) which converges into the identical state of Bagging that is kind of ensemble learning, within the framework of on-line learning, and have analyzed its asymptotic property through the statistical mechanics method. Mutual learning involving three or more students fundamentally differs from the two-student case with regard to the variety of selecting a student to act as teacher. The proposed model consists of two learning steps: many students independently learn from a teacher, and then the students learn from others through the mutual learning. In mutual learning, students learn from other students and the generalization error is improved even if the teacher has not taken part in the mutual learning. We demonstrate that the learning style of selecting a student to act as teacher randomly is superior to that of cyclic order by using principle component analysis.

• 2009-11-15

著者

• Hara Kazuyuki

  Tokyo Metropolitan Coll. Of Industrial Technol. Tokyo

• Okada Masato

  Graduate School Of Engineering Science Osaka University

• Okada Masato

  Graduate School of Frontier Sciences, The University of Tokyo, 5-1-5 Kashiwanoha, Kashiwa, Chiba 277-8561

• Miyoshi Seiji

  Faculty of Engineering Science, Kansai University, 3-3-35 Yamate-cho, Suita, Osaka 564-8680

• Nakayama Yoichi

  Tokyo Metropolitan College of Technology, 1-10-40 Higashi-oi, Shinagawa, Tokyo 140-0011

関連論文

• Statistical Mechanical Study of Code-Division Multiple-Access Multiuser Detectors : Analysis of Replica Symmetric and One-Step Replica Symmetry Breaking Solutions(General)
• Statistical Mechanical Analysis of CDMA Multiuser Detectors : AT Stability and Entropy of the RS Solution, and 1RSB Solution
• Retrieval Property of Attractor Network with Synaptic Depression(General)
• Analysis of Ensemble Learning Using Simple Perceptrons Based on Online Learning Theory
• Retrieval of Branching Sequences in an Associative Memory Model with Common External Input and Bias Input(Cross-disciplinary physics and related areas of science and technology)
• Residual Energies after Slow Quantum Annealing(General)
• Multiple Stability of a Sparsely Encoded Attractor Neural Network Model for the Inferior Temporal Cortex(General)
• Image Restoration with a Truncated Gaussian Model(General)
• Statistical Mechanics of On-line Node-perturbation Learning
• Ensemble Learning of Linear Perceptrons : On-Line Learning Theory(General)
• Analysis of XY Model with Mexican-Hat Interaction on a Circle
• Exact Inference in Discontinuous Firing Rate Estimation Using Belief Propagation
• Influence of Synaptic Depression on Memory Storage Capacity
• Neural Network Model of Spatial Memory: Associative Recall of Maps
• Statistical Mechanics of On-Line Mutual Learning with Many Linear Perceptrons
• Statistical Mechanics of Mutual Learning with a Latent Teacher(General)
• Mean Field Analysis of Stochastic Neural Network Models with Synaptic Depression
• Optimization of the Asymptotic Property of Mutual Learning Involving an Integration Mechanism of Ensemble Learning(General)
• Rate Reduction for Associative Memory Model n Hodgkin-Huxley-Type Network(Cross-disciplinary physics and related areas of science and technology)
• Higher Order Effects on Rate Reduction for Networks of Hodgkin-Huxley Neurons(Cross-disciplinary physics and related areas of science and technology)
• Estimation of Intracellular Calcium Ion Concentration by Nonlinear State Space Modeling and Expectation-Maximization Algorithm for Parameter Estimation
• Neural Network Model with Discrete and Continuous Information Representation
• Estimating Membrane Resistance over Dendrite Using Markov Random Field
• Stability Analysis of Stochastic Neural Network with Depression and Facilitation Synapses
• Analysis of XY Model with Mexican-Hat Interaction on a Circle ---Derivation of Saddle Point Equations and Study of Bifurcation Structure---
• Statistical Mechanics of On-line Ensemble Teacher Learning through a Novel Perceptron Learning Rule
• Solvable Model of a Phase Oscillator Network on a Circle with Infinite-Range Mexican-Hat-Type Interaction
• Multiple Stability of a Sparsely Encoded Attractor Neural Network Model for the Inferior Temporal Cortex
• Image Segmentation and Restoration Using Switching State-Space Model and Variational Bayesian Method
• Statistical Mechanical Analysis of CDMA Multiuser Detectors : AT Stability and Entropy of the RS Solution, and 1RSB Solution
• Stability Analysis of Attractor Neural Network Model of Inferior Temporal Cortex —Relationship between Attractor Stability and Learning Order—
• Bayesian Image Restoration for Medical Images Using Radon Transform
• Image Restoration and Segmentation using Region-Based Latent Variables: Bayesian Inference Based on Variational Method
• Statistical Mechanics of Mexican-Hat-Type Horizontal Connection
• Statistical Mechanics of Node-Perturbation Learning for Nonlinear Perceptron
• Inter-Layer Correlation in a Feed-Forward Network with Intra-Layer Common Noise
• Continuous Attractor that Appears in Autoassociative Memory Model Extended to XY Spin System
• Statistical–Mechanical Analysis of Attractor Dynamics in a Hysteretic Neuron Network
• A Numerical Analysis of Learning Coefficient in Radial Basis Function Network
• Inter-Layer Correlation in a Feed-Forward Network with Intra-Layer Common Noise
• Analysis of Ensemble Learning Using Simple Perceptrons Based on Online Learning Theory

もっと見る 閉じる

スポンサーリンク