Relative Density-Ratio Estimation for Robust Distribution Comparison (情報論的学習理論と機械学習)

スポンサーリンク

概要

• 論文の詳細を見る

• Divergence estimators based on direct approximation of density-ratios without going through separate approximation of numerator and denominator densities have been successfully applied to machine learning tasks that involve distribution comparison such as outlier detection, transfer learning, and two-sample homogeneity test. However, since density-ratio functions often possess high fluctuation, divergence estimation is still a challenging task in practice. In this paper, we propose to use relative divergences for distribution comparison, which involves approximation of relative density-ratios. Since relative density-ratios are always smoother than corresponding ordinary density-ratios, our proposed method is favorable in terms of the non-parametric convergence speed. Furthermore, we show that the proposed divergence estimator has asymptotic variance independent of the model complexity under a parametric setup, implying that the proposed estimator hardly overfits even with complex models. Through experiments, we demonstrate the usefulness of the proposed approach.

• 一般社団法人電子情報通信学会の論文
• 2011-11-02

著者

• Hachiya Hirotaka

  Department Of Computer Science Tokyo Institute Of Technology

• Suzuki Taiji

  Department Of Mathematical Informatics Graduate School Of Information Science And Technology Univers

• YAMADA Makoto

  Tokyo Institute of Technology

• Hachiya Hirotaka

  Tokyo Inst. Of Technol.

• Suzuki Taiji

  University of Tokyo

• Kanamori Takafumi

  Department Of Computer Science And Mathematical Informatics Nagoya University

• Yamada Makoto

  Tokyo Inst. Of Technol.

• Yamada Makoto

  Department Of Chemistry And Biomolecular Science Toho University

• Sugiyama Masashi

  Department Of Chemistry Faculty Of Science Tokyo University Of Science

• Sugiyama Masashi

  Department of Applied Chemistry, Yamanashi University

関連論文

• Statistical active learning for efficient value function approximation in reinforcement learning (ニューロコンピューティング)
• Improving the Accuracy of Least-Squares Probabilistic Classifiers
• Improving the Accuracy of Least-Squares Probabilistic Classifiers
• Least Absolute Policy Iteration — A Robust Approach to Value Function Approximation
• A New Meta-Criterion for Regularized Subspace Information Criterion
• Approximating the Best Linear Unbiased Estimator of Non-Gaussian Signals with Gaussian Noise
• Spontaneous Rupture of the Iliac Vein : Report of a Case
• Methods of cross-domain object matching (情報論的学習理論と機械学習)
• GAME-THEORETIC DERIVATION OF DISCRETE DISTRIBUTIONS AND DISCRETE PRICING FORMULAS
• Regularization Strategies and Empirical Bayesian Learning for MKL
• A Unified Framework of Density Ratio Estimation under Bregman Divergence
• γ-Functional Prolines Based on Naturally Occurred Hydroxyproline. III
• γ-Functional Prolines Based on Naturally Occurring Hydroxyproline. II
• Multi-task learning with least-squares probabilistic classifiers (パターン認識・メディア理解)
• Multi-task learning with least-squares probabilistic classifiers (情報論的学習理論と機械学習)
• Adaptive importance sampling with automatic model selection in value function approximation (ニューロコンピューティング)
• Analytic Optimization of Adaptive Ridge Parameters Based on Regularized Subspace Information Criterion(Neural Networks and Bioengineering)
• Adaptive Ridge Learning in Kernel Eigenspace and Its Model Selection
• Myonase is Localized in Skeletal Muscle Myofibrils
• Recent Advances and Trends in Large-Scale Kernel Methods
• Covered Stent Treatment for Traumatic Cervical Carotid Artery Aneurysms : Two Case Reports
• Syntheses of New Artificial Zinc Finger Proteins Containing Trisbipyridine-ruthenium Amino Acid at The N-or C-terminus as Fluorescent Probes
• Practical Synthesis of 4-cis Hydroxy-L-Proline
• Improving Model-based Reinforcement Learning with Multitask Learning
• Improving Model-based Reinforcement Learning with Multitask Learning
• Ovarian Histological Findings in an Adult Patient with the Steroidogenic Acute Regulatory Protein (StAR) Deficiency Reveal the Impairment of Steroidogenesis by Lipoid Deposition
• Analytic Optimization of Shrinkage Parameters Based on Regularized Subspace Information Criterion(Neural Networks and Bioengineering)
• Least-Squares Conditional Density Estimation
• Direct Importance Estimation with a Mixture of Probabilistic Principal Component Analyzers
• カーネル密度比推定の統計的解析(学習問題の解析,テキスト・Webマイニング,一般)
• Constructing Kernel Functions for Binary Regression(Pattern Recognition)
• Optimal design of regularization term and regularization parameter by subspace information criterion
• Solid-papillary carcinoma of the breast : Clinicopathological study of 20 cases
• A fast sequence kernel for sequential data classification (Speech) -- (国際ワークショップ"Asian workshop on speech science and technology")
• Information-maximization clustering: analytic solution and model selection (情報論的学習理論と機械学習)
• Conditional Density Estimation Based on Density Ratio Estimation
• Conditional Density Estimation Based on Density Ratio Estimation
• New feature selection method for reinforcement learning: conditional mutual information reveals implicit state-reward dependency (情報論的学習理論と機械学習)
• Least Absolute Policy Iteration-A Robust Approach to Value Function Approximation
• A density ratio approach to two-sample test (パターン認識・メディア理解)
• A density ratio approach to two-sample test (情報論的学習理論と機械学習)
• Theoretical Analysis of Density Ratio Estimation
• Independent component analysis by direct density-ratio estimation (ニューロコンピューティング)
• A New Meta-Criterion for Regularized Subspace Information Criterion(Pattern Recognition)
• Spectral Methods for Thesaurus Construction
• Adaptive importance sampling with automatic model selection in reward weighted regression (ニューロコンピューティング)
• SERAPH: semi-supervised metric learning paradigm with hyper sparsity (情報論的学習理論と機械学習)
• Analysis and improvement of policy gradient estimation (情報論的学習理論と機械学習)
• Direct density-ratio estimation with dimensionality reduction via hetero-distributional subspace analysis (情報論的学習理論と機械学習)
• Output divergence criterion for active learning in collaborative settings (数理モデル化と問題解決・バイオ情報学)
• Estimation of squared-loss mutual information from paired and unpaired samples (情報論的学習理論と機械学習)
• Direct Importance Estimation with Gaussian Mixture Models
• Dependence minimizing regression with model selection for non-linear causal inference under non-Gaussian noise (情報論的学習理論と機械学習)
• Canonical dependency analysis based on squared-loss mutual information (情報論的学習理論と機械学習)
• Improving the Accuracy of Least-Squares Probabilistic Classifiers
• Artist agent A[2]: stroke painterly rendering based on reinforcement learning (パターン認識・メディア理解)
• Artist agent A[2]: stroke painterly rendering based on reinforcement learning (情報論的学習理論と機械学習)
• Efficient Sample Reuse in Policy Gradients with Parameter-based Exploration (情報論的学習理論と機械学習)
• Density Difference Estimation (情報論的学習理論と機械学習)
• Generalization Error Estimation for Non-linear Learning Methods(Neural Networks and Bioengineering)
• Improving Precision of the Subspace Information Criterion(Neural Networks and Bioengineering)
• Canonical dependency analysis based on squared-loss mutual information (パターン認識・メディア理解)
• Density-ratio matching under the Bregman divergence : a unified framework of density-ratio estimation
• Change-Point Detection in Time-Series Data by Relative Density-Ratio Estimation (情報論的学習理論と機械学習)
• Modified Newton Approach to Policy Search (情報論的学習理論と機械学習)
• Computationally Efficient Multi-Label Classification by Least-Squares Probabilistic Classifier (情報論的学習理論と機械学習)
• Relative Density-Ratio Estimation for Robust Distribution Comparison (情報論的学習理論と機械学習)
• Change-Point Detection in Time-Series Data by Relative Density-Ratio Estimation
• Modified Newton Approach to Policy Search
• Squared-loss Mutual Information Regularization
• Computationally Efficient Multi-Label Classification by Least-Squares Probabilistic Classifier
• Feature Selection via l_1-Penalized Squared-Loss Mutual Information
• Semi-Supervised Learning of Class Balance under Class-Prior Change by Distribution Matching (情報論的学習理論と機械学習)
• Density Difference Estimation
• Relative Density-Ratio Estimation for Robust Distribution Comparison
• Artist Agent: A Reinforcement Learning Approach to Automatic Stroke Generation in Oriental Ink Painting
• Prospective Assessment of Pain and Functional Status After Percutaneous Vertebral Body-Perforation Procedure for Treatment of Vertebral Compression Fractures
• Winning the Kaggle Algorithmic Trading Challenge with the Composition of Many Models and Feature Engineering (情報論的学習理論と機械学習)
• Computationally Efficient Multi-Label Classification by Least-Squares Probabilistic Classifiers
• Direct Density Ratio Estimation for Large-scale Covariate Shift Adaptation
• Early Stopping Heuristics in Pool-Based Incremental Active Learning for Least-Squares Probabilistic Classifier (情報論的学習理論と機械学習)
• Multi-Task Approach to Reinforcement Learning for Factored-State Markov Decision Problems
• Efficient Sample Reuse in Policy Gradients with Parameter-based Exploration (情報論的学習理論と機械学習)
• Output Divergence Criterion for Active Learning in Collaborative Settings
• Output Divergence Criterion for Active Learning in Collaborative Settings
• Photochromism of benzylviologens containing methyl groups on pyridinium rings and embedded in solid poly(N-vinyl-2-pyrrolidone) matrix.
• Clustering Unclustered Data : Unsupervised Binary Labeling of Two Datasets Having Different Class Balances
• Prospective Assessment of Pain and Functional Status After Percutaneous Vertebral Body-Perforation Procedure for Treatment of Vertebral Compression Fractures
• Direct Approximation of Quadratic Mutual Information and Its Application to Dependence-Maximization Clustering
• Direct Learning of Sparse Changes in Markov Networks by Density Ratio Estimation
• On Kernel Parameter Selection in Hilbert-Schmidt Independence Criterion
• Squared-loss Mutual Information Regularization
• Early Stopping Heuristics in Pool-Based Incremental Active Learning for Least-Squares Probabilistic Classifier
• Winning the Kaggle Algorithmic Trading Challenge with the Composition of Many Models and Feature Engineering
• Improving Importance Estimation in Pool-based Batch Active Learning for Approximate Linear Regression

もっと見る 閉じる

スポンサーリンク