Reinforcement Learning with dual tables for a partial and a whole space

概要

論文の詳細を見る
The reduction on the trial frequency isimportant for reinforcement learning under an actualenvironment.We propose the Q-learning method that selects properactions of robot in unknown environment by using the Self-Instruction based on the experience in known environment.Concretely, it has two Q-tables, one is smaller, based on apartial space of the environment, the other is larger, based onthe whole space of the environment. At each learning step, Qvaluesof these Q-tables are updated at the same time, but anaction is selected by using Q-table that has smaller entropy ofQ-values at the situation. We think that the smaller Q-table isused for the knowledge storing as self-instructing. The larger isused for the experiment storing.We experimented the proposed method with using an actualmobile robot. In the experimental environment, exist a mobilerobot, two goals and one of a red, a green, a yellow and a blueobject. The robot has a task to carry a colored object into thecorresponding goal. In this experiment, the Q-table for thewhole has a state for the view of the object and the goals withthe colors, the Q-table for the partial has the state withoutcolor information. We verified that the proposed method ismore effective than the ordinaries in an actual environment.