Please login first

  • Open Access
  • 2 Reads
  • 0 Citations
  • 0 Recommendations

Efficient RL Algorithm by Combing AC with Dual Piecewise Model Learning
Shan Zhong 1 , Quan Liu 2 , Qiming Fu 3

1  School of Computer Science and Technology, Soochow University, Suzhou, Jiangsu, 215006,School of Computer Science and Engineering, Changshu Institute of Technology, Changshu, 215500,Jiangsu Province Key Laboratory of Intelligent Building Energy Efficiency, Suzhou University of Science and Technology, Suzhou, Jiangsu, 215006
2  School of Computer Science and Technology, Soochow University, Suzhou, Jiangsu, 215006,Collaborative Innovation Center of Novel Software Technology and Industrialization, Nanjing, 210000,Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun, 130012
3  Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun, 130012,Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun, 130012,College of Electronic & Information Engineering, Suzhou University of Science and Technology, Jiangsu, Suzhou, 215006

Published: 24 January 2017 by MDPI AG in MOL2NET 2016, International Conference on Multidisciplinary Sciences, 2nd edition in MOL2NET 2016, International Conference on Multidisciplinary Sciences, 2nd edition
MDPI AG, 10.3390/mol2net-02-03895
Abstract:

As classic methods for handling continuous action space problem for continuous action space problem in RL, the actor-critic (AC) algorithm and its variants still fail to be sample efficiency. Therefore, we propose a method based on learning two linear models for planning. The two linear models refers to state-based piecewise model and action-based piecewise model, which are determined by the divisions for the state and action space, respectively. Through division, the models are learned more accurately. To accelerate the convergence, the sample near the goal is saved and used to learn the model, the value and the policy to balance the distribution of the samples. On two classic RL benchmarks with continuous MDPs, the proposed method shows the ability of learning an optimal policy by combing both models, and it also outperforms the representative methods in terms of convergence rate and sample efficiency.


Comments on this paper Get comment updates
Currently there are no comments available.