不完全知覚判定法を導入した Profit Sharing
スポンサーリンク
概要
- 論文の詳細を見る
To apply reinforcement learning to difficult classes such as real-environment learning, we need to use a method robust to perceptual aliasing problem. The exploitation-oriented methods such as Profit Sharing can deal with the perceptual aliasing problem to a certain extent. However, when the agent needs to select different actions at the same sensory input, the learning efficiency worsens. To overcome the problem, several state partition methods using history information of state-action pairs are proposed. These methods try to convert a POMDP environment into an MDP environment, and thus they are sometimes very useful. However, their computation cost is very high especially in large state spaces. In contrast, memory-less approaches try to escape from the aliased states by outputting actions stochastically. However, these methods output actions stochastically even in unaliased states, and thus the learning efficiency is bad. If we desire to guarantee the rationality in POMDPs, it is efficient to output actions stochastically only in the aliased states and to output one action deterministically in the other unaliased states. Hence, to discriminate between aliased states and unaliased states, the utilization of χ⊃2; -goodness-of-fit test is proposed by Miyazaki et al. They point out that, in aliased states, the distributions of the state transitions by random search and a particular policy are different. This difference doesnt occur owing to non-deterministic actions. Hence, if the agent can collect enough samples to implement the test, the agent can distinguish between aliased states and unaliased states well. However, such a test needs a large amount of data, and its a problem how the agent collects samples without worsening learning efficiency. If the agent uses random search in the course of learning, the learning efficiency worsens especially in unaliased states. Therefore, in this research, we propose a new method called Extended On-line Profit Sharing with Judgement (EOPSwJ) to detect important incomplete perception, which doesnt need large computation cost and numerous samples. We use two criterions for detecting important incomplete perceptions to attain a task. One is the rate of transitions to each state, and the other is the deterministic rate of actions. We confirm the availability of EOPSwJ using two simulations.
- 社団法人 人工知能学会の論文
- 2004-11-01
著者
-
増田 士朗
東京都立科学技術大学
-
増田 士朗
首都大学東京 大学院システムデザイン研究科
-
増田 士朗
東京都立科学技術大学 工学部
-
増田 士朗
東京都立科学技術大学生産情報システム工学科
-
斎藤 健
東京都立科学技術大学大学院工学研究科
-
斎藤 健
東京都立科学技術大学
関連論文
- 可調整パラメータを持つmas-plus線形システムに対するモデル予測制御
- 線形パラメータ表現されたmax-plus線形システムの最適逆システム
- フレッシュマンのための適応制御 : モデリングしながら制御する
- 熱延仕上ミル張力・ルーパ系のハイブリッドシステムモデルとモデル予測制御
- 熱延仕上ミル張力・ルーパ系のハイブリッドシステムモデルとモデル予測制御(鉄鋼業における最新の計測,制御,システム技術)
- 閉ループデータに基づく直接的PID調整とその不安定プロセスへの適用
- 予見・予測制御
- モデル予測制御-III : 一般化予測制御(GPC)とその周辺
- MIMO-FIFO型構造を有する繰返し処理システムのバックワード型オンラインMPLスケジューリング(システムと制御)
- min-Plus線形システムに基づくモデル予測制御